Blueprint for the future of photography — computational photography

We’re into our second week of the #ShotOnSnapdragon photo competition and with it another closer look at how platforms like the Qualcomm(R) Snapdragon(™) 855 Mobile Platform improve the capabilities of your smartphone’s photography. If you haven’t already, be sure to check out our tips to help improve your photography skills as well.

Building on last week’s look at Image Signal Processors (ISP), today we’re exploring how Qualcomm Spectra(™) ISP technology combined with Computational Photography capabilities takes your smartphone pictures to a whole new level. By computational photography, we mean advanced imaging techniques that enhance and extend the regular capabilities of digital photography. Some of these enable photographers to manually tweak their images, while others work subtly in the background ensuring that your pictures always look the best. Let’s dive in.

Mobile Platforms built for crunching image data

While an ISP is perfect at handling regular photography and video capture tasks, more advanced computational photography algorithms require more powerful hardware. Use cases include software bokeh blur for a professional grade depth of field effect, High Dynamic Range color processing, multi-camera shooting, Electronic Image Stabilization (EIS), and superior digital zoom capabilities.

Snapdragon Mobile Platforms are built from many different processing parts, each with their own specialties and capabilities – many of which are suitable for advanced image processing. In addition to the general purpose CPU and image capturing Qualcomm Spectra ISP, Snapdragon Mobile Platforms contain a Qualcomm(R) Hexagon(™) Digital Signal Processor (DSP) and Qualcomm(R) Adreno(™) Graphics Processing Unit (GPU) that are both well suited to crunching advanced imaging algorithms. Utilizing these components across the platform is known in the mobile industry as Heterogeneous Processing.

Each of these processing units is useful at different types of algorithms. CPUs, for example, offer the best performance where the algorithm isn’t highly parallel and requires

branching operations. Meanwhile highly parallel algorithms, which often crop up in image post-processing techniques like HDR and object removable, run much faster on a GPU. Finally, a DSP is the perfect processing unit for real-time sensor-based applications and machine learning tasks such as object detection.

As an example, smartphones powered by Snapdragon offer Electronic Image Stabilization (EIS) without the need for any extra mechanical parts. By tracking movement from the phone’s accelerometer hardware and running tracking algorithms on the Adreno GPU, the Spectra ISP can take a very stable looking image at just the right time. This stops your pictures from coming out blurry, even if you don’t have steady hands.

The latest Qualcomm Spectra 380 ISP inside the Snapdragon 855 and Qualcomm Spectra 350 inside the Snapdragon 730 include dedicated Computer Vision (CV-ISP) processing components for even more efficient acceleration. This allows the CV-ISP itself to run object detection, background replacement, and real-time 4K HDR bokeh blur, freeing up processing resources on the CPU, GPU, and DSP to run other processes.

Making photography smarter

Software bokeh depth-of-field

Portrait modes and software bokeh have become a staple effect of smartphone cameras. Offering DSLR-like blur, various lighting effects, and even background replacement effects to drastically change the look of your pictures.

To create a software blur effect, the camera requires depth information about the scene you’re shooting in. One way to do this is to include a dedicated depth camera, such as a time of flight sensor, to detect actual distance. Alternatively, a phone with two cameras can take a picture at two different focal lengths simultaneously. Snapdragon Mobile Platforms support multiple cameras should a smartphone manufacturers want to implement this technology. An alternative method used by a lot of single cameras is to take pictures at multiple focus points in very quick succession.

Once the phone has data at multiple focal lengths, stereo algorithms, which work in a similar way to our eyes, are applied to discern which objects are in the foreground and which are in the background. A second blurring algorithm is then applied to the image to soften objects that aren’t in focus. The benefit of using this method is that the focal subject, amount of blur, and other effects can be changed after the photo has been taken. This data can also be saved in the HEIF image format.

With the CV-ISP inside the latest Snapdragon Mobile Platforms, handsets are able to apply bokeh blur to not just photos but to live video as well. The CV-ISP produces bokeh blur at up to 4K in real-time video, while also supporting high dynamic range, 10-bit quality as well.

Superior zooming

Zooming in on subjects in a very useful photography technique to frame the perfect picture. However, some phone cameras have historically been stuck with digital zoom, which significantly reduces quality once you begin zooming in quite far. Optical zoom or telephoto cameras have appeared to help in this regard, but extra cameras and lenses can be expensive. It’s possible to take very good zoomed in pictures with just a single camera, thanks to some clever algorithms and hardware to run them efficiently.

Qualcomm(R) OptiZoom(™) Camera Feature is engineered to capture 12 images in a quick burst. Algorithms combine data from these 12 shots together by observing differences and similarities between pixels. These sub-pixel differences between shots are used to glean extra details, creating a higher resolution image packed with extra detail. This is also known as super-resolution zoom, and it creates vastly superior results to digitally cropping in on your picture after pressing the shutter.

High Dynamic Range (HDR)

HDR processing is required in scenes with a large variation between light and dark in order to avoid over or underexposure. Examples include trying to photography an object against a brightly lit background such as a cloudy sky or shooting in low light with a single light source.

Smartphones powered by Snapdragon create high dynamic range pictures by combining multiple pictures together, each with a different exposure setting. Darker pictures expose highlights correctly, such as the sun and clouds, while brighter exposures bring out highlights in the dark. Algorithms combine details from all of these pictures together to create an image that offers better highlights and shadows than a standard picture. Qualcomm Technologies’ “ghost-free” HDR technology also works to avoid blurry edges by detecting and removing any motion from the shot.

Helping you take better photos

Modern smartphones have moved past just providing great camera hardware, they now also offer cutting edge processing techniques and tricks to close the ever smaller gap with professional grade DSLR cameras. Snapdragon Mobile Platforms are at the heart of many of the devices offering consumers the very best in mobile photography.

Of course, advanced processing is only part of the picture. If you want to take excellent snaps, be sure to check out our expert tips on how to take the perfect picture. And don’t forget to submit your entries to the #ShotOnSnapdragon photo competition for your chance to win a Snapdragon 855-powered smartphone.

Content sponsored by Qualcomm Technologies, Inc. 

Qualcomm Snapdragon, Qualcomm Spectra, Qualcomm Adreno, Qualcomm Kryo, Qualcomm OptiZoom and Qualcomm Hexagon are products of Qualcomm Technologies, Inc. and/or its subsidiaries.