Dual cameras are coming: what to expect

Dual camera technology has hopped in and out of fashion in the world of Android for the past couple of years, but it has seen a bit of a resurgence in 2016, with a number of Android flagships opting for dual cameras. Most notably, the LG G5 boasts one regular and one wide angle camera, while the HUAWEI P9 has opted for an interesting color and monochrome combination. Even Apple is now rumored to be hopping onto the trend, with the lastest report suggesting that Sony and LG Innotek are competing over image sensor orders for the iPhone 7.

If you’re interested in the details, Nomura Securities and Citi Research believe that the 5.5-inch iPhone 7 will come with a dual-camera setup, but Sony may not be able to fulfill demand, so the hardware contract may instead go to LG. Apple also acquired Israeli camera tech company LinX back in April 2015 for $20 million. The company had been working on dual image sensor hardware and software that promised to boost image quality.

If dual cameras are to be the next trend for mobile, here’s a breakdown of the type of features that you might see in the coming year or so.

All shapes and sizes

The interesting thing about dual image sensor setups is that they can come in a variety of different specifications. Some offer up two high quality image sensors paired with different lenses to capture wide angled shots or allow for a greater apertures. Switching between what is essentially two different cameras can provide users with extra shooting options, or lenses and sensors more suited to capturing images in low light conditions. Combining data from slightly different camera setups can help reduce noise and improve contrast in a not too dissimilar way to many existing smartphone HDR implementations, which takes multiple pictures at slightly different brightness levels to capture deep blacks while preserving whites.

The other popular design is to use a secondary, and usually lower resolution, sensor to provide different kinds of data that can be fed into image processing algorithms. Dedicated monochrome or color sensors can be used to improve the contrast, dynamic range or color palette of a picture by providing additional information about a scene to an image processor. Alternatively, lower resolution secondary sensors can focus on different points and provide basic depth data, which opens up the door to more complex 3D imaging.

It’s very unlikely that we will see a wide range of OEMs opt for identical camera setups in the future, as powerful solutions don’t necessarily require two identical image sensors that act like a pair of eyes. Really, dual image sensors are opening the door to new and effective software solutions, and there’s lots of experimenting still to do before an optimum solution is settled upon.

Image refocusing

One of the first features spotted in dual sensor smartphones was the option to refocus an image once it had been saved, by picking out sub pixel disparities between two images captured from ever so slightly different viewpoints. This works in a very similar way to the way that our eyes detect distance, and this information is then used to apply a synthetic blurring algorithm to emulate a depth of field effect.

This type of technology isn’t limited to dual-camera setups though, as software implementations found in camera apps such as the Google Camera can imitate the results. However, cameras with different apertures could bring background softening and focusing results that are much closer to DSLR cameras than today’s solely software based ideas. Not to mention the boosts to noise reduction and color reproduction.

3D capture and depth mapping

The next step beyond refocusing is to use this data to calculate and store depth information. This can then be used to recreate 3D models of an object, room, or even a landscape. By collecting this data on a per-pixel basis, RGB data from single or multiple frames can be used to almost completely reconstruct a picture in 3D.

Multi- camera configurations inside Google’s Project Tango and Intel’s RealSense have already demonstrated quite promising results here, and they should offer up cost effective products for the professional 3D modeling, architecture, and other markets. This type of technology and data could even allow for highly advanced photo editing, where objects can be added or removed from a scene as the camera has data about where foreground and background objects sit.

Furthermore, with virtual reality and mobile products seemingly moving ahead together, highly accurate depth sensing technology could be useful above and beyond just taking pictures. Being able to tell the distance to the nearest wall or detect the position of your hands in the real world would be very useful when immersed in a virtual space. Of course, 3D models and additional depth data requires more storage, so pictures of this type would start taking up a lot more space than your regular JPEG shots.

Superior digital zoom

One of the more interesting features touted by some of these multi-sensor developers, including Apple’s LinX, is the ability to improve zoom image quality and to produce more detailed looking pictures from lower resolution sensor combos. The exact details about how this works are a closely guarded secret, but my suspicion is that clever noise detection algorithms are able to cancel out common noise that appears in two or more images. Combined with larger individual pixel sizes to keep noise and crosstalk to a minimum, and this could produce quite sharp looking pictures that should look much cleaner when zoomed in to 100 percent.

CorePhotonics has shown off a similar idea that uses one wide angle lens and one telephoto lens to fold light to produce a five times “optical” zoom. From here, some proprietary software fuses the two images together, and this extra resolution allows viewers to pick out finer details when zooming into a digital image. The company has two sensors in development, one dual 13 megapixel setup with a 5x zoom, and one 13 and 8 megapixel combination that allows for a 3x zoom. Along with better looking zoomed in pictures, this technique can also be used to reduce noise in low light shots.

The one thing that we haven’t seen from any of these technologies is how long it actually takes to capture and process these images. Current refocusing technologies can take a little while to process an entire image and re-save. That said, the HUAWEI P9 doesn’t take a notably long time to capture its pictures, but perhaps this will depend on the type of sensors being used.

[related_videos align=”center” type=”custom” videos=”689768,679964,675613,674935″]

The key to better smartphone photography?

With space inside smartphones at a premium, handset manufacturers are going to have some very interesting choices to make when picking out their next camera components. One group of 2016 flagships have taken the plunge, while another group has opted for a single high resolution image sensor that boast larger pixel sizes and better quality lenses to improve low light performance. Dual image alternatives of equal quality are not only more complex on the hardware and software side, but also require more space and sometimes higher cost components to implement. That being said, the noise reduction available to even lower resolution image sensors might help to keep the costs reasonable.

Our own hands on time with a number of the latest flagship smartphones and voting by our readers found that the LG G5 is the best handset around right now for pictures, and was closely followed by the Galaxy S7 Edge, the iPhone 6S Plus, and the Nexus 6P, while the HUAWEI P9 came second to last. So perhaps dual camera systems aren’t quite up to scratch just yet. That being said, the P9 was still rated higher than the single sensor HTC10.

It’s very important to get the basics right first and offer up a solid camera experience that even novice users can get their heads around. If dual-image sensor setups are to be successful outside of niche markets, they have to produce superior results while still being affordable and user friendly. It’s a tall order, but there are many in the industry that firmly believe dual cameras are the future for mobile devices. Where do you stand?