How does virtual reality work?

The definition of virtual reality

Virtual reality allows the user to be immersed into a virtual world, unlike regular screens in front of the user which do not allow for such an experience. VR can include 4 of the 5 senses, including vision, hearing, touch and possibly even smell. With this power, VR can can take people for a virtual world fairly easily. The only current problems are the availability of such hardware and the price at which it can be purchased. Google is combating this with Google Cardboard and the Daydream ecosystem. But as it currently stands, high quality VR is not possible without spending quite a bit of cash to get a powerful computer and a headset go with it. As prices fall on graphics cards that can run desktop VR at the required settings and with Google making the Daydream ecosystem, it will not be long before high quality content is readily available.

Seeing is believing

As you can see, the angle of the weapon is slightly different on each side, as is the crosshair, but when you actually put on the headset and play the game, everything lines up perfectly. The way the stereoscopic screen looks varies platform to platform as each headset differs quite a bit in the way it display content, the above image is from a game made for Google Cardboard using Unreal Engine.

Different VR platforms also have different specifications on the headsets themselves. The HTCVive and Oculus Rift both have 90hz displays, while the Playstation VR has a 60hz display. It is a rule of thumb that you want your frames per second to match your monitor’s refresh rate, so it is recommended that the Vive and Rift both maintain 90 FPS while the PSVR maintains 60 FPS. Mobile is a different story, as different phones have different resolutions, but maintaining at least 60 FPS is the goal. We will get into exactly what this means next.

Expanding more on how FPS and refresh rate work, FPS and the refresh rate of a monitor are two separate things independent from each other. Frames per second is how fast your GPU can display images, per second. 60 FPS means that the GPU is outputting 60 images every second. The refresh rate of a monitor is how fast the monitor can display images per second, measured in hertz (Hz). This means that if you are playing a game and the FPS is 120 but your monitor refresh rate is 60 Hz, you will only be able to display 60 FPS. You are essentially losing half of your frames, which is not a good thing as “tearing” can occur.

Tearing is the phenomenon of objects in a game breaking up into a few pieces and being displayed in two different locations along the X axis giving a tearing effect. This is where Vertical Sync (VSync) comes in. This limits the framerate to your monitor’s refresh rate. This way no frames are lost and in turn no tearing is experienced. This is why for the best VR experience, the same number for the frame rate and refresh rate need to be achieved, or sickness can occur.

• HTC Vive – everything you need to know
• Oculus Rift – everything you need to know
• Google Daydream – everything you need to know

There are also other components that go into the whole VR experience including Field of View (FOV) and latency. These play a major role in how we perceive VR and if not done correctly, can cause motion sickness as well. Let’s take a look.

Field of view is the extent of the visible world that can be seen at any given time. For example, humans have about a 180 degree FOV while looking straight ahead, and 270 degrees with eye movement. This is an important feature in VR, since you will be wearing the headset to transport you into a virtual world.

The human eye is very good at noticing vision imperfections, with tunnel vision being an example of such a phenomena. Even if a VR headset had a 180 degree FOV, you may still be able to tell a difference. The Vive and Rift both have 110 degree FOVs, Cardboard has 90, the GearVR has 96 and it is rumored that Daydream may have as much as 120. This should, generally speaking, greatly affect the VR experience and could make or break a certain headset for people, not to mention any health issues that we will get into later.

Latency is also a factor that can make or break VR, with anything over 20 milliseconds not being fast enough to trick your brain into thinking you are in a different world. There are a bunch of variables that go into latency, including CPU, GPU, the screen, cables and so on. The screen will have an average latency, around 4-5 ms depending on the screen for example. The time it takes a full pixel to switch is another 3 ms and the engine could also take a few. With just three variables, you are looking at latency in the double digits in some cases. The key to decreasing latency is the refresh rate of the monitor. The formula is as follows: 1000 (ms) / refresh rate (hz). So while the latency issue could be solved with a 90hz monitor instead of a 60hz monitor, it isn’t that easy as we have discussed. Later we will talk about the PC hardware requirements for virtual reality.

Not meeting an acceptable frame rate, FOV or latency can cause motion sickness. This happens enough to actually coin its own name, known as “cybersickness”. All three of these concepts need to be met to cut down on the changes of cybersickness. Without the correct frames per second with the refresh rate of the display, frame skipping, micro stutters and lag are possible. Latency may even be a bigger problem, with movement and interaction lag caused by slow response times of the hardware, it is possible to completely lose your sense of direction and become disoriented. Field of view, while important, shouldn’t cause as many issues as the others mentioned, but will definitely take away from the experience and may cause some disorientation.

Movement and interaction

Using these sensors, your phone can accurately predict where you are looking at any given time while using VR. With the announcement Google Daydream, Android VR users will be able to use a separate phone as a controller to move and interact within the environment. Desktop VR like the HTC Vive or Oculus Rift either use a controller or controllers reminiscent to the Wiimote for different purposes. Using computer vision (explained here), VR accuracy can be greatly improved by having cameras and other sensors set up in the room you are using the VR headset.

VR headsets can have special controllers, as mentioned earlier, but how exactly do they work? Looking at the HTCVive, there are two infrared sensors and two controllers in the box, totaling 70 different sensors with the headset. All of this tracks you and your controllers allowing you to freely move around the room while playing games. Notice how the Vive controllers have a circle cutout? That is more than likely there for tracking purposes. The Oculus Rift offers a different experience using close to the same technology.

Out of the box, the Rift actually uses an Xbox One controller. But there is an optional set of controllers that offer similar functionality to the Vive, known as “Touch by Oculus.” These two controllers rearrange the One controller’s buttons onto what can only be described as foregrips with large rings covering your fingers. Oculus is keeping the way these work under tight wraps, but the package does include two sensors similar to the Vive so presumably they work in a similar fashion, they may also have accelerometers and gyroscopes as well.

The power of sound

The power required for virtual reality

The main issue facing hardware is that for the Vive and Rift, your PC doesn’t just have to run a 1080p game at 60 FPS, it has to run at a higher resolution at 90 FPS. Most hardware can not do that.

It turns out that there is a very limited number of computers with these specs or better, so this will more than likely slow down the adoption of VR on the desktop. For mobile however, any Android phone with KitKat (4.4) or higher should not have any issues with basic VR functionality. Daydream features do require at least a Nexus 6P at the time of writing however.

The future of virtual reality and Android

Google has been at the forefront when it comes to VR on mobile. Available now, the Google VR SDK and NDK allow for some very powerful VR development, and with Google Daydream being released later this year, mobile VR will see another leap in what is possible. Samsung has also had success with the Gear VR. Third party engines are also integrating Google VR into their engines. Unreal Engine now supports Google VR in 4.12 and Unity also is Google VR ready and Daydream ready.

Wrap Up

Virtual reality has a lot of potential, and with lower prices and a bigger push from companies, VR can be a great success. The way VR works is a very smart combination of different technologies all working together to create a great experience. From stereoscopic viewpoints to 3D audio, VR is the future now, and it can only get better. Let us know in the comments if you think VR is the next big thing! Be sure to stay tuned to Android Authority and VR Source for everything VR!/