Dual camera optical zoom technology explained

Dual camera smartphone technology certainly isn’t new anymore, but we are seeing a new trend emerge whereby manufacturers are offering “optical zoom” capabilities inside their smartphones. Two notable models touting this feature are Apple’s iPhone 7 Plus and the new OnePlus 5. If you’ve been wondering exactly how this technology works and if these phones really offer optical zoom capabilities, then you’ve come to the right article.

The smartphone zoom problem

Before delving into the specifics of how this works, it’s probably best to recap why manufacturers have felt the need to offer optical zoom capabilities inside a smartphone. Unlike DSLR lenses with variable focal length lenses or point-and-shoot cameras with zoom lenses, smartphone cameras are stuck with small, fixed lenses. This means that a smartphone camera’s focal length is fixed, leaving no option but to rely on digital zoom to close in on details in an image.

However, digital zoom isn’t much good for anything more than minor zoom-ins, as we are stuck with a limited resolution and set field of view for near and far details, meaning that a pixel can only capture so much detail at a distance. This limitation is why images appear pixelated as you zoom in, and it’s exactly the same as simply magnifying the image in a gallery app post shutter.

Variable lenses, such as those found in DSLR setups, allow for an adjustable focal length and therefore field of view too. A longer focal length narrows the camera’s field of view, but this means that the same limited resolution image sensor captures its detail over a smaller area, so each pixel represents a finer point in space. This is why optical zoom doesn’t suffer from the same pixelation problems as digital zoom.

When it comes to measuring a camera’s or lens’ zoom, we need a reference or starting point for the camera or lens we’re talking about. To find any zoom level we divide the current focal length by the minimum available. So moving from 25 mm to 50 mm is equal to a 2x zoom, as is moving from 18 mm to 36 mm. Zoom is a relative term, but it has a direct relationship between two focal lengths.

Dual lenses offer a hybrid solution

When it comes to dual camera smartphones with a “telephoto” lens, we’re actually looking at two sensors, each with a lens pairing that offers up fixed, but different, focal lengths. For example, one may offer a 24 mm focal length and the other 36 mm, giving us 1.5x worth of “optical zoom” potential. Of course there is no real zoom going on as no parts are moving, but the end result is the same, so we’ll continue using that term. The sensor sizes, pixel sizes, and lens aperture may also vary between sensors, which of course will have their own influences on image quality from each camera.

Looking at the iPhone 7 Plus, Apple has opted for one 28 mm and one 56 mm lens, offering up a 2x optical zoom potential. This means that we can capture close up shots with a 28 mm 1x zoom, and then simply switch to the 56 mm 2x zoom lens and capture pictures at greater distances without any loss in detail.

Not only that, but this second camera also improves the image quality of 4x and higher zoom levels too. Because we’re starting at full resolution at 2x (rather than already halving it as we would be if we digitally zoomed in from 1x), we only lose half the information where we’d normally be losing three quarters of it at 4x. In other words, we shouldn’t reach that horribly pixelated look so quickly as we have more resolution for distant details.

It’s a similar state of affairs with the OnePlus 5, which offers up 1.6x optical zoom capabilities on the second lens before cropping down a 20 MP image to achieve a 2x zoom, and then relying on digital thereafter. OnePlus says that this is part of its Smart Capture software that takes the best information from multiple frames and stitches them together, which clearly isn’t optimal but it’s an intriguing hybrid approach. Even though the camera is throwing away some data to achieve a 2x zoom, the quality should still be improved over the old digital zoom method.

As well as offering up more detail at a distance, these dual camera setups also allow for advanced HDR processing that we’ve seen from other multi camera configurations. Even if the exact image details aren’t being shared with the final image, additional light and dark data can be used to bolster dynamic range processing, improving the look of the final image regardless of zoom level.

The results aren’t perfect

You may have realized that this still isn’t a perfect solution though. We need digital zoom if we want to venture beyond the maximum focal length, but what happens when we want to use a midway zoom level, such as 1.2, 1.3, or 1.5x?

The obvious solution is to simply rely on a digital zoom for these minor levels, as the degradation in quality won’t be too notable in most circumstances. Eventually, it will be possible to switch over to the second camera’s fixed optical zoom level, but this has the downsides of meaning that the zoom levels just below the switch will offer the worst quality.

So for example, 1.1x through 1.6x could be a digital zoom, then switching over to the second camera at 1.7x. This may explain why we’ve seen companies only opt for modest zoom distances, as the system is still reliant on digital zoom and too large a gap between lenses would produce a notable dip in quality.

To make this method a little smarter, the exact point of switchover doesn’t have to be set in stone. An image signal processor (ISP) could monitor the noise level of each camera and decide to make the switch to the secondary camera only if it will produce better results. We believe that both the iPhone 7 Plus and OnePlus 5 are using a system similar to this.

OnePlus’ Smart Capture method of combining data from multiple higher resolution images takes this one step further. By using multiple frames worth of higher resolution data, noise can be averaged out when zooming to those in-between values like 1.2 or 1.8x. Although technically this can be done with just a single lens camera.

Simple camera switching isn’t really ideal though, and is clearly nowhere near matching the zoom capabilities insinuated by “optical zoom” advertising taglines. A possible solution is to turn to some more powerful image analysis algorithms in order to stitch up a picture from both cameras. Unfortunately this still has drawbacks compared with real optical zoom lenses though, as averaging and stitching is seldom as good as the real thing.

Today’s’ high-end ISPs are capable of merging together data from cameras with different focal lengths. This is already done for advanced HDR techniques and is taken a step further by ideas like the multi-lens Light L16 camera. This type of technique can take high frequency detail from the higher zoom lens and crop and stitch this information into the center of the picture that’s more zoomed out. So at 1.5x, the center on the image will contain extra detail, while the edges will be digitally zoomed in from the 1x standard lens. It wouldn’t be perfect, but your subject matter will look sharper and retain some extra detail when compared with single sensor smartphone cameras.

While this sounds simple enough in theory, such an algorithm would be highly processor intensive and would mean much longer times between clicking the shutter and viewing the final image, which is clearly no good for fast action shots or burst modes.

Furthermore, this type of processing would be further complicated by the different sensor pixel sizes and apertures between the dual cameras, which will alter the light capture characteristics. If the zoom camera isn’t capturing enough light, as it typically has smaller pixels, then it will suffer from more noise and would make image stitching a pointless task.

Based on the results we’ve seen so far, it’s unlikely that any smartphone OEM has achieved this level of complexity yet. The closest disclosed technology to this would probably be HUAWEI’s “Hybrid Zoom”, which takes fine detail data from the 20 MP camera and colors it using RGB data from the 12 MP sensor, allowing for superior 2x digital zoom.

In summary, making the most of dual camera potential is much more complicated than it first seems, and it requires a variety of sophisticated processing techniques… not to mention that the results are going to be heavily dependent on the shooting environment.

Wrapping up

Although dual camera technology represents a notable step forward for smartphone photography, we should be aware that there are some major caveats right now. The first being that even dual cameras are still limited to fixed focal lengths and are therefore incapable of offering up the true range of optical zoom capabilities that the marketing might lead us to believe.

That being said, there are notable benefits to this technology over a traditional digital zoom, including superior quality zoom at long distances. You also get the option to enable superior post processing effects, including digital bokeh and advanced HDR. And let’s not forget the creative freedom that these configurations offer photography enthusiasts.

This technology is still in its infancy, and we’ll no doubt see notable improvements to camera quality and zoom functionality after second-gen products hit the market. Do you highly rate dual camera technologies, or are you happy with your existing single sensor setup?