Beyond dual cameras: how Light wants to take on the DSLR

Smartphone photography enthusiasts may remember when we talked about the Light L16 “multi-aperture computational camera” last year. To celebrate the camera’s commercial launch, the company has spilt a few more details about how the setup works. At $1,699 a pop, now up from its $1,299 pre-order price, the Light L16 certainly doesn’t come cheap, but it does offer a glimpse of what should be possible in future flagship smartphones, once costs come down.

In case you missed our last look at the gadget, the L16 boasts 16 different inexpensive plastic-lens image sensors packed into a slims phone sized form factor. Combined with some proprietary algorithms, this allows the camera to offer up 52-megapixel images with a variable focal point between 28 and 150mm, which works out to roughly a 5.4x zoom. For comparison, the dual-camera technology inside the iPhone 7 Plus only offers up a fixed 2x zoom / focal point adjustment, while the Kirin 960 processor heading to the Mate 9 goes a step further by allowing for a more adjustable 4x zoom.

However, there’s much more to this technology than simulating an optical zoom. The Light project is all about bringing the features and quality usually associated with DSLR cameras to the convenience of the smartphone form factor.

I had to plan ahead if I wanted to take pictures with one of my good cameras, which I did less and less. As I talked with other avid photographers, I discovered I wasn’t the only one who had expensive camera gear gathering dust. It’s not that any of us were happy with the quality of the pictures we were taking with our phones—indeed, we were all frustrated by it. But at the end of the day, convenience always won out. – Dr Rajiv Laroia, Light L16 creator

The problem with phones

There’s a very good reason that smartphone cameras can’t match up with the quality of DSLRs and that’s largely due to size. Handset image sensors are substantially smaller, typically less than 30mm² compared with DSLR sensor sizes of at least 370mm² all the way up to Full Frame 864mm² sensors. Smaller sensors capture much less light in each pixel and also suffer from greatly increased noise as the megapixel count increases, because individual pixels become smaller. As a result, image quality in smartphone cameras is diminished compared to larger DSLR sensors.

How it works

The Light L16 is a rather strange gadget to behold, as it features 16 individual sensors, five of which have a fairly standard 28mm field of view, five others feature telephoto lenses that offer the equivalent of a 70mm focal length, and then there are six 150mm equivalent cameras included, as well. Each lens has a fixed aperture of f/2.4, an actuator for moving the lens to focus the image, and is accompanied by a 13-megapixel AR1335 CMOS image sensor.

The real magic comes in with the last 11 of these sensors, which are accompanied a mirror in front of the lens. This mirror can be moved slightly to adjust the center of the module’s field of view, which is important for aligning the different sensors when taking pictures at different focal points. So, rather than using mechanical lenses, the L16 uses mirrors to adjust points of focus. The image below shows an example of 28, 70, and 150mm focal lengths.

When it comes to actually taking a picture, the L16 uses up to 10 of the image sensors which are in the best location to capture detail for the desired focal length. When capturing a 70mm picture, the mirrors point the 70mm sensors directly out of the camera, while four 150mm modules see their mirrors adjust to match the 70mm field of view. Data from the 70mm image is then used to align and stitch together data from the “more zoomed in” 150mm sensors to produce a crisp 52-megapixel resolution result.

When it comes to capturing a picture of a focal length in between the camera hardware values of 28, 70, and 150mm, the same overlapping technique is used, along with some cropping. So when capturing a 50mm image, for example, the 28mm cameras create the base image which is cropped to a 50mm frame. The 70mm camera mirrors then align these sensors to overlap the new 50mm frame, and pixels from these pictures are then stitched together to give a 40-megapixel pixel result.

While the resolution boost is nice, we should also remember that this detail is being captured with many more camera sensors. This is actually registering much more light than a single sensor, and gives a wider dynamic range because each of these modules also has a slightly different exposure when taking a picture.

With some additional software tricks, overexposing some pictures can help remove noise from dark areas, and under exposure on a different sensor can help capture highlights in more detail at the same time. Modern smartphones can do HDR by taking a succession of differently exposed pictures over time, but this can produce smearing as the pixels don’t always line up correctly. In theory, Light’s option should provide a better looking HDR image, as all the pictures are captured simultaneously.

In addition to light data, the distances between these sensors also allows the Light L16 to capture depth data in the scene. This can be used to implement a software based bokeh effect, allowing photographers to change the level and type of blur after the image has been taken, either with the camera software or in a separate editing program on a computer. Photographers can also pick between different blur types, such as disk-shaped bokeh, star-shaped bokeh, or a Gaussian blur. You would have to spend a fortune on lenses to achieve the same flexibility with a DSLR.

Beyond the L16

The Light L16 is not without some trade-offs though. The processing power of current smartphones isn’t enough to handle the huge amount of data in a timely fashion and within a reasonable power budget. The current Snapdragon 820-based implementation allows for real time processing of these 16 sensors, but only resulting in a resolution of 3 megapixels. That’s enough for social media, but not really making the most of the hardware available here.

In the future, Light intends to integrate a dedicated image processor in its next generation design, which will enable the system to process everything in hardware, thereby speeding up full resolution image processing and adjusting depth of field on the fly. In the meantime, users will have to use a PC for full resolution image adjustments. Speaking of next generation hardware, Light is aiming to produce a new model that will offer a 600mm lens equivalent option in the future. This would offer a zoom of 21.4x between 28mm and 600mm, and would bring an experience currently limited to $12,000 top of the line DSLR lenses to a compact form factor.

Further improvements are on the way to Light with tweaks to the sensors themselves. Rather than capturing a filtered RGB spectrum, Light intends to mix in sensors that don’t feature filters to collect even more light. We have already seen similar ideas in smartphones with the monochrome image sensors using in the HUAWEI P9 and HONOR 8, which helps to capture more light and can use this data and software algorithms to improve HDR.

Along with a next generation L16 camera, Dr Laroia says that he’s looking forward to seeing Light technology migrate into cellphones. Light is currently working with On Semiconductor to produce its new filter-less sensor array, and manufacturer Hon Hai Technology Group (Foxconn) has already licensed the technology from Light. Unfortunately, we don’t have any details about any smartphone products that will make use of Light’s technology, nor can we be sure exactly sure what specifications will make their way to a smartphone design.

Of course, we have also to wait until the technology actually arrives inside smartphones before we can compare results with today’s and tomorrow’s flagships. Even so, the technology looks to offer some substantial benefits and the move to dual cameras has paved the way for more interesting ideas to begin appearing in consumer products.