Apple Vision Pro features: Everything that Apple's AR-VR headset can do

At WWDC 2023, Apple laid the foundation for the future when it revealed the Apple Vision Pro. This headset is more than an AR—VR headset, as it does mixed reality while being a complete computer by itself. That puts the Vision Pro in a class of its own, distinct from other mixed reality products you may have seen from competitors. What can the Vision Pro do, though? We cover all the Apple Vision Pro features you can expect if you plan to spend $3,500 on Apple’s AR-VR headset when it comes out.

Apple Vision Pro features: An infinite canvas in front of you

With traditional smartphones, tablets, laptops, and other electronics, you are limited to the physical confines of the display in front of you. So if your phone has a 6-inch display, your laptop has a 13-inch display, or your TV has a 65-inch display, you are still limited to that physical dimension. Apps and media you consume cannot exceed this physicality, even if you apply display trickery as foldables and rollables do.

Apple Vision Pro and other AR-VR headsets provide an infinite canvas. Through visionOS, the Vision Pro frees apps and media from the boundaries of a “display,” letting them appear side by side at any scale. The screen real estate is infinite, in the sense that everything in front of you and around you is a display surface.

Want to have five different windows open in front of you? Or want to watch a movie on a giant 100-feet-like screen? The Vision Pro lets you do that.

With a custom micro-OLED display system that features 23 million pixels, putting the Vision Pro on is like having a portable 4K display in front of each eye. You can create an enormous private workspace in front of you, with no physical dimensions to worry about.

Vision Pro gets its visionOS

The Vision Pro is Apple’s foray into a completely new product category. To power the Vision Pro hardware, there is new software in the form of visionOS.

Apple says that visionOS was designed from the ground up to support the low-latency requirements of spatial computing. There’s also the backing of decades of experience and innovation from developing macOS, iOS, and iPadOS. The result of all this coming together is a dedicated OS for the headset built to take advantage of the space around the user.

visionOS features a brand-new three-dimensional interface. It enables digital content to look and feel present in a user’s physical world. The UX elements of visionOS respond dynamically to natural light and cast shadows, providing the user a sense of scale and distance.

The Vision Pro’s highlight is that the headset does not need dedicated controllers, unlike other AR-VR headsets in the market. Consequently, visionOS had to introduce an entirely new input system controlled by a person’s eyes, hands, and voice.

Users can browse apps by looking at them as the headset tracks their eyes. You tap your fingers to select a UX element like a button or icon, and you do it while resting your arms and shoulders in their natural state. You can scroll with a flick of the wrist. And, of course, you can use your voice to dictate. Combining these simple inputs allows you to navigate the entire visionOS ecosystem without feeling confused or out of place.

Apple promises that eye-tracking information is not shared with Apple, third-party apps, or websites.

Vision Pro gets its visionOS App Store

App Stores have been the reason for the iPhone‘s popularity and success, and it was obvious that Apple would recreate some of the same magic for the Vision Pro too. The headset runs its own visionOS, and the operating system has its own app store.

The visionOS App Store will house apps from third-party developers who will be designing new app experiences that take advantage of the uniqueness of the Vision Pro headset. They will also be reimagining existing apps and use cases for spatial computing. In addition, the visionOS App Store will showcase hundreds of thousands of iPhone and iPad apps that run great and automatically work with the new input system for Vision Pro.

Environments lets you fade your reality away

Having your reality re-projected in front of you is incredible and helps you stay grounded and remain aware of what is happening around you. But with a twist of the Digital Crown present on the top right surface of the Vision Pro headset, wearers can invoke the Environments feature.

Environments are dynamic landscapes that remove you from your physical surroundings and transport you to a digital world. Your surroundings fade out, and you are now present in the middle of a beautiful landscape. This increases immersion and improves focus by reducing distractions and clutter around you.

FaceTime becomes spatial

Since now you have an infinite canvas in front of you, you can have significantly more immersive FaceTime video calls with others. Everyone on the call can now appear in life-sized tiles. Coupled with Spatial Audio, the Vision Pro will emulate the feeling of having video call participants live in front of you, speaking from where they are positioned.

You can also do more with FaceTime. You can watch a movie, browse photos, or collaborate on a presentation while on a call on the headset.

What happens to your video feed when you are wearing the Apple Vision Pro? You use Persona!

When using FaceTime with an Apple Vision Pro, your video feed uses a Persona instead of a regular front-camera video feed as you would on an iPhone.

Personas are a digital representation of the person wearing the headset. It is created using Apple’s machine-learning techniques. As a plus over plain digital avatars, Personas reflect face and hand movements in real-time.

Spatial Audio adds to the immersion

The Vision Pro boasts several technologies around Spatial Audio. Dual-driver audio pods are positioned right above each ear, delivering personalized sound without blocking your ambient sounds. This way, Ambient Spatial Audio makes it feel like the sound is coming from all around you.

The Apple Vision Pro also adopts audio raytracing, analyzing your room’s acoustic properties (including the physical materials) to adapt and match the sound to your surroundings.

The Apple Vision Pro also does Spatial Photo and Video

The Vision Pro is the first Apple product that can shoot 3D photos and videos. A button on the headset’s top makes it quick and easy to capture spatial photos and videos.

Spatial photos and videos have depth beyond what you experience on 2D photos and videos, making you feel closer to the moment. You can expand the photos and videos to get even closer to them.

For panoramas, the Apple Vision Pro has its own neat trick. Panorama photos will expand in front of you, making it feel like you are at the center of it.

EyeSight informs others when you are around

Wearing a mixed reality headset isolates you from the rest of the world around you, so you need mechanisms to bring reality back into the picture. Most headsets rely on cameras on the exterior visor to capture your reality and project it back through the internal displays near your eyes. But that still does not solve your being somewhat absent from others around you.

Apple Vision Pro solves this through EyeSight. The headset also uses cameras on the inside of the headset to capture your eyes and their movement. There is an external screen on the headset that then projects this feed out. This feature is called EyeSight, and it’s Apple’s spin on bringing you back into the present without needing to remove the headset.

When someone approaches someone wearing a Vision Pro and draws their attention, the exterior display will show the user’s eyes. Simultaneously, the wearer will also see the person on the outside. This will give both parties the feeling of presence and present a visual cue of when someone is available to interact when wearing the Vision Pro headset.

EyeSight also provides a visual indicator to others when a user is capturing a spatial photo or video.

When a wearer is immersed in an environment or using an app (i.e., not in a state of acute awareness), EyeSight will provide a different, more opaque visual cue that indicates that the user is focused within the headset and not paying active attention to their surroundings.

Optic ID is a brand-new authentication system for the Vision Pro

The Apple Vision Pro is largely envisioned as a personal device like an iPhone, not a community device like a home tablet. So Apple needed a new form of secure authentication that they could use on a headset since both Face ID and Touch ID would be challenging to implement on this form factor. Enter Optic ID.

Optic ID is an authentication system that analyses a user’s iris under various invisible LED light exposures. It then compares it to the enrolled Optic ID data to unlock the Apple Vision Pro.

Optic ID data is fully encrypted and not accessible to apps. It also never leaves the device.

Apple Vision Pro features: Designed for premium performance

The Apple Vision Pro is as much of a computer as a sunglass — that is, it must look good while delivering high-end performance. And objectively, while it may look dystopian when worn, there’s no denying it looks premium.

The design of the headset is surprisingly compact. A singular piece of three-dimensional formed and laminated glass is on the exterior. This houses the display for the Eyesight feature and also acts as a lens for the vast array of cameras and sensors needed to map your surroundings.

The glass flows into an aluminum alloy frame that curves around the user’s face. There’s a modular system for the face cushion, allowing for a better “tailored” fit that can accommodate a wide range of people and facial features. Since this is an Apple product, the face cushion is called Light Seal, and it is made of a soft textile that comes in various shapes and sizes.

A flexible strap connects the main headset to the headband while housing the speakers needed for Spatial Audio.

The Head Band is the next prominent piece of the Apple Vision Pro. Apple says it is ” three-dimensionally knitted as a single piece” that can provide cushioning, breathability, and stretch. The band is available in multiple sizes and is secured with a simple mechanism for easy swapping. Apple did not disclose other band styles but mentioned that you could swap to different Head Band styles. A dial on the right side lets you tighten or loosen the Head Band for a precise fit.

Most of Apple’s marketing materials do not feature this, but an additional strap goes over your head. This will help distribute the headset’s weight better across your head, making it feel less forward-heavy.

The Apple Vision Pro is powered by an external battery pack that connects to the headset through a proprietary connector, promising up to two hours of battery life. The battery pack charges through USB-C and can be used in pass-through mode.

Apple Vision Pro features: Innovative hardware

No talks about the Apple Vision Pro’s hardware would be complete without talking about the impressive hardware that Apple has managed to fit inside the headset.

The primary display on the Vision Pro (the ones in front of your eyes) uses micro-OLED technology to pack 23 million pixels into two displays, each size of a postage stamp. The displays are overlaid with custom catadioptric lenses that deliver sharp, clear imagery. Users with vision correction needs will need to use special ZEISS Optical Inserts to correctly see the display and ensure eye tracking accuracy for system navigation.

On the user’s side of the headset, there are high-speed cameras and a ring of LEDs that project invisible light patterns onto the user’s eyes. This is how Apple has achieved a high-performance eye-tracking system that powers the UX, system navigation, and EyeSight on the Vision Pro.

A flexible strap on the side of the headband houses the speakers for the Spatial Audio system. Two individually amplified drivers inside each audio pod deliver Personalized Spatial Audio based on the user’s head and ear geometry.

The Apple Vision Pro is powered by not one but two Apple chips in a unique dual-chip design. There is an Apple M2 SoC on board for handling the usual performance bits, while a brand-new Apple R1 chip is responsible for processing input from the 12 onboard cameras, five sensors, and six microphones. The R1 chip streams new images to the displays within 12 milliseconds, eight times faster than the blink of an eye!

Apple promises that data from the cameras and other sensors are processed at a system level. Individual apps do not need to see a user’s surroundings to enable spatial experiences.

Continuity with Mac

Since the Vision Pro is still very much an Apple product, it can tap right into the Apple ecosystem for an unparalleled ecosystem experience. Apple only showcased one use case for the same for now, wherein users on Mac simply have to wear the Vision Pro and look at the Mac to expand the screen beyond the Mac.

The Vision Pro also works well with Bluetooth accessories like the Magic Keyboard and the Magic Trackpad.

The Apple Vision Pro is surprisingly full of features for a first-generation product. Everything that Apple has highlighted so far has been straight from the company. Once the headset reaches the hands of developers, we hope to see even more innovative use cases and features coming for the headset.

Apple Vision Pro’s starting price of $3,499 renders it out of reach for many people. But it is no denying that this sets the foundation for Apple’s spatial computing dreams. What is built today will be enjoyed by future generations of the product. Who knows, maybe one of those may become the next iPhone.

FAQs