Why Qualcomm’s Snapdragon 835 is a big deal for mobile VR

Virtual reality entered the mainstream in 2016 and 2017 looks set to push the associated technologies into their next generation. Mobile is a promising avenue for virtual reality which is ripe for development, and Qualcomm’s latest Snapdragon 835 mobile application processor could end up being an important catalyst.

Qualcomm’s newly unveiled Snapdragon 835 is promising plenty of improvements for smartphones this year, but the company has also integrated lots of features into the chip that will help power the next generation of mobile virtual reality applications, and future augmented reality hardware too. While smartphone based projects like Daydream, which the Snapdragon 835 supports, are the primary focus for many manufacturers, Qualcomm’s Snapdragon is also designed to power standalone virtual reality headsets too. Here’s a look at exactly what the company has done to empower the next generation of portable virtual reality headsets.

Extra processing grunt and new display features

Graphics processing power is essential for virtual reality applications, and Qualcomm has boosted the 3D performance of its Adreno 540 GPU by up to 25 percent over the Adreno 530 inside the Snapdragon 820. A needed boost for sure, and the Adreno 540 also supports a range of lower level graphics API, which will give developers better access to resources and help them boost performance.

Vulkan, OpenGL ES 3.2, full OpenCL 2.0, and Microsoft’s DirectX 12 are all supported this time around. Vulkan and DX12 are very important as they can greaty increase multi-core CPU utilization over OpenGL ES, which will be a boon for the Snapdragon 835. Qualcomm has moved back to an octa-core arrangement with its Kryo 280 CPUs, from a quad-core arrangement with Snapdragon 820, which could provide a lot more CPU power above any beyond core architecture improvements.

On top of additional performance, improvements to the Snapdragon 835’s display (DPU) and video (VPU) processing units will offer up benefits for virtual reality applications. The introduction of Q-Sync into the VPU will lock compatible display refresh rates to the GPU frame rate, much like NVIDIA’s G-SYNC technology and AMD’s support of the FreeSync standard. Virtual reality framerates still need to remain high, but Q-Sync will be helpful at reducing motion sickness caused by stutter from dropped frames.

The DPU now also supports display resolution at 4K with a 60fps output. While the refresh rate is perhaps not quite as fast as we’d like for VR, we should see lower resolution displays supported with the required frame rate. The DPU also supports 10-bit HDR content, allowing for better looking virtual reality content with a higher contrast ratio. Immersion is the key after all.

Enhanced audio and sensors

It’s not just graphical horsepower that’s important for bringing immersive VR to the mobile space, accurate sensors and binaural audio technologies are just as important.

With the Snapdragon 835, Qualcomm has introduced support for six unique measurements axes. This augments existing X, Y, and Z rotational tracking with height and directional movement tracking as well, which will allow users to move through virtual spaces without the need for external tracking equipment. Qualcomm has accomplished this by supporting improved 800 and 1000Hz sensor sampling speeds for accelerometer and gyroscope data respectively. This can be combined with imaging data from a monocular camera on a headset to support position and orientation data. Qualcomm also boasts that this computation can be carried out entirely on the Snapdragon 835’s Hexagon DSP with just a 15ms motion to photon latency, leaving the CPU and GPU free to render a scene for the wearer.

On the audio side, there’s new support for object and scene based placement in a 3D space. Part of Qualcomm’s SDK can assist designers to build 3D audio for virtual reality environments. The 835 also includes support for HRTE binaural audio processing, which is used to emulate the characteristics of the human ear for realistic sound placement. Again, this can be calculated on the DSP with minimal input from the CPU in order to speed up processing and save on battery life.

Machine learning and smart processing

As you can see, Qualcomm’s efforts to improve mobile virtual reality are quite heavily reliant on the smart use of the various different processors tucked into the Snapdragon 835. Heterogeneous compute is part of the solution, but the company is also looking to machine learning algorithms to improve performance and bring new features to the platform too.

One such example is the use of eye tracking technologies to assist with foveated rendering. Foveated rendering is a technique used to reduce GPU load in virtual reality rendering by decreasing the rendering resolution at the edges of the screen, there the wearer doesn’t tend to observe. However, this can break immersion if the user’s looks off to the side of the screen. Integrating eye tracking cameras in the headset and using machine learning algorithms on the 835’s DSP can track the wearer’s eye movement with minimal latency and processing overhead. This can then be used in conjunction with GPU foveated rendering techniques to reduce the image quality and therefore GPU load on parts of the screen that the user currently isn’t looking at.

Alternatively, iris screening technologies and machine learning algorithms can be used to assist with setting up a virtual reality headset for a user to wear. Each person has a unique inter-pupillary distance and this affects the focus of the VR image as it comes through the lenses. Usually, some setup time is required and adjustments made to the headset to accommodate each wearer. However, machine learning tools and iris tracking could be used to automatically calibrate rendered objects, such as an augmented or virtual reality HUD, so that they are in focus.

As a final example, the Snapdragon 835 supports gesture recognition from a camera input, which can be used to interact with objects and games in virtual reality, rather than having to rely on physical controllers. Once more, images can be analysed using machine learning tools on the Hexagon DSP, rather the CPU or GPU, in order to lighten the load on these components and produce faster, more accurate results.

We mustn’t forget that the Snapdragon 835 is designed to be Qualcomm’s most power efficient flagship mobile SoC to date. The new high efficiency Kryo 280 CPU cores and move to 10nm FinFET process node, combined with intelligent use other processing cores, can see power users gain 2.5 hours of battery life over the 820. This means that phones and standalone headsets should be able to run VR apps and games for longer and will also presumable produce less heat too, which are notable gains for mobile VR.

Assisting developers

Eking out every drop of performance is going to be vital to obtaining suitable virtual reality performance in mobile products, and Qualcomm is now giving the developers the tools to get closer to the metal. The Symphony System Manager that debuted with the Snapdragon 820 VR platform extends to the Snapdragon 835 and allows software developers to assign tasks to specific CPU cores, the GPU, and even the DSP, meaning the possibility of a higher level of optimization for VR apps. Qualcomm has also revealed that the low level Vulkan API can be run on just a single little core of its Snapdragon 835, leaving plenty of spare resources for developers to work with.

On top of better utilization of its core components, Qualcomm is assisting virtual reality software developers through its Snapdragon VR SDK. The SDK can assist developers with tasks ranging from utilizing the Snapdragon 820 and 835’s sensors and DSP, through to stereoscopic rendering.

For hardware developers, a Snapdragon VR 835 reference platform provides a starting point for engineers and manufacturers to design their own standalone VR headset powered by Qualcomm’s latest flagship. The Snapdragon 835 also supports Google’s Daydream platform, meaning that Snapdragon 835 products will work with Google’s virtual reality hardware as well.

Wrap Up

Qualcomm’s Snapdragon 835 builds on the heterogeneous compute, machine learning, and virtual reality features that made their debut with the Snapdragon 820 last year. The end result is a SoC that caters well to the growing demands of mobile virtual and augmented reality. While very high performance hardware will remain limited to the desktop PC space, Qualcomm’s efforts with the 835 looks capable of enabling VR developers to offer compelling experiences in a much more constrained power and thermal budget.

While the Snapdragon 835 is still designed very much with smartphones in mind, Qualcomm is also making a bold push into the mobile virtual and augmented reality markets with its new flagship SoC. I’m sure we will see plenty of VR hardware and content powered by the platform over the coming months and years.

This article originally appeared on VRSource.com