from ars thecnica
Apple Vision Pro: Augmented and Virtual Reality Headset
The Apple Vision Pro is a groundbreaking augmented and virtual reality headset developed by Apple over the course of more than a decade. Unveiled at WWDC 2023 in June, the Vision Pro represents Apple’s entry into a new product category, marking a significant milestone since the launch of the Apple Watch in 2015.
While it is commonly referred to as a headset, Apple prefers to call it a spatial computer due to its unique ability to seamlessly merge digital content with the real world. This distinguishing feature sets the Vision Pro apart from conventional headsets.
The Apple Vision Pro offers a mixed reality experience by overlaying augmented reality content onto the surrounding environment and immersing users in entirely virtual experiences. It is important to note that the headset does not provide a see-through view; instead, everything seen is digital. To integrate augmented reality elements without obstructing the surroundings, Apple utilizes advanced cameras to map the physical world and enhance it with virtual elements.
For a fully immersive virtual reality experience, the Vision Pro disables the cameras, creating a secluded environment that enables users to focus solely on the content displayed on the headset’s screens. Users can effortlessly transition between the augmented and virtual reality modes using the on-device Digital Crown.
In terms of design, the Apple Vision Pro resembles a pair of ski goggles, featuring a single piece of laminated glass at the front seamlessly integrated into an aluminum alloy frame. A soft and snug Light Seal, which attaches magnetically to the frame, effectively blocks out external light, ensuring an optimal viewing experience.
The headset incorporates two Audio Straps with built-in speakers positioned on the sides, providing Spatial Audio that blends the audio from the headset with the ambient sounds of the real world. These Audio Straps are connected to a 3D knitted headband that holds the Vision Pro securely in place. Apple has taken special care to ensure the headband is breathable, cushiony, and stretchy, prioritizing user comfort. Additionally, a Fit Dial allows users to adjust the headset for a perfect fit. Apple plans to offer Light Seals and headbands in various sizes, and these components are interchangeable.
Inside the frame, the Vision Pro boasts two micro-OLED displays, delivering over 4K pixels to each eye, resulting in a combined total of 23 million pixels. Additionally, there is an external display called EyeSight, which projects an image of the user’s eyes, indicating whether they are in immersive mode or aware of their surroundings. For individuals who wear glasses, the headset accommodates custom prescription Zeiss Optical Inserts that can be magnetically attached to the lenses.
The Apple Vision Pro stands out with its innovative control mechanisms. Instead of traditional controllers, the headset relies on eye tracking, hand gestures, and voice commands. Users can navigate and select apps simply by looking at them and tapping their fingers. Scrolling can be accomplished with a flick of the fingers.
Equipped with a multitude of cameras and sensors, the Vision Pro creates a comprehensive map of the user’s environment while simultaneously tracking hand and eye movements. Optic ID, a feature that utilizes infrared lights and cameras to scan the user’s iris, provides secure authentication. Similar to Face ID and Touch ID, Optic ID leverages the unique iris pattern for device unlocking, making purchases, and serving as a password replacement.
The Vision Pro incorporates two Apple silicon chips: the M2 chip, also found in Mac devices, and a new R1 chip. The M2 chip powers the visionOS, executes computer vision algorithms, and handles graphics, while the R1 chip processes input from the cameras, sensors, and microphones.
An integrated camera allows users to capture 3D photos and videos by simply tapping the top button of the device. Apple emphasizes the ability to relive these memories in a more immersive manner. Moreover, the Vision Pro can display existing photos and videos on a large scale, intensifying the immersive experience. When recording video, the headset features a prominent animation on the external display to indicate that recording is in progress.
Due to weight considerations, Apple opted not to include a built-in battery in the Vision Pro. Instead, the headset can be powered using a braided cable connected to a battery pack worn at the hip or plugged into an adapter. The battery pack provides approximately two hours of battery life on a single charge.
With the Apple Vision Pro, content is displayed in the space surrounding the user, allowing apps and windows to be positioned in mid-air and rearranged as desired. Multiple windows are supported, and Apple boasts an “infinite canvas” for users to work with. Apps can be displayed within the user’s physical environment, keeping them aware of their surroundings, or in a more immersive view, where the content takes precedence on a virtual background known as an Environment.
The Vision Pro operates on the visionOS, a dedicated operating system created specifically for the headset. It features its own App Store with apps tailored for the device, while also supporting iPhone and iPad apps. Users can connect the Vision Pro to a Mac, effectively utilizing the headset as an extended display. Bluetooth accessories enable text input and control, and users can also utilize virtual typing or dictation for text-based interactions.
The main Home View of the visionOS provides access to popular Apple apps such as Mail, Messages, Music, Safari, and Photos, with data seamlessly synced through iCloud. The interface closely resembles that of an iPhone, but users have the flexibility to open and arrange apps virtually. Apple is actively updating its main apps for visionOS and has developed APIs to facilitate app development for the platform. With the Vision Pro, entertainment experiences like watching TV shows and movies become more immersive, with content expanding to create a personalized theater experience complete with spatial audio.
FaceTime has been specifically redesigned for the Vision Pro, featuring large tiles to display participants, while the wearer’s digital persona is accurately represented to others. This allows for collaborative document editing, app sharing, and clear identification of the speaker through spatial audio.
The Cinema Environment within the Vision Pro offers an enhanced viewing experience, allowing users to enjoy shows and movies at the frame rate and aspect ratio chosen by the content creator. Alternatively, a nature-themed Environment creates a screen that feels 100 feet wide. Apple has also created Apple Immersive Videos, 180-degree 3D 8K recordings that place users directly inside the action. Popular streaming services like Apple TV+ and Disney+ are available on the Vision Pro. Additionally, the headset supports Apple Arcade, providing access to a curated collection of 100 iPad games at launch. Bluetooth game controllers can be used for gaming purposes, connecting seamlessly to the headset.
The Apple Vision Pro is priced starting at $3,499, and it is scheduled to launch in early 2024. Initially, it will be available for purchase on Apple.com and exclusively in stores within the United States.
A More Affordable Second-Generation AR/VR Headset
Apple is already working on a more affordable version of the AR/VR headset to bring the technology to a wider audience. While the initial AR/VR headset is expected to be priced around $3,000, Apple aims to develop a model that aligns with the pricing of the iPhone.
To achieve this lower price point, Apple plans to utilize more cost-effective components, including chips on par with those found in iPhones instead of Mac-level chips. Additionally, the second-generation headset may feature lower resolution internal displays and more economical materials.
The development of the more affordable headset is still in its early stages, and Apple has not yet produced a working prototype. The projected launch for this headset is 2025, with Foxconn already involved in the manufacturing process.
According to Apple analyst Ming-Chi Kuo, the second-generation AR/VR headset will be available in both high-end and low-end models, similar to the iPhone lineup. Apple is expected to offer a more affordable model in addition to a follow-up to the higher-end, more expensive first-generation model.
Apple’s Long-Term AR/VR Plans
Apple analyst Ming-Chi Kuo speculates that Apple may be planning to develop augmented reality “contact lenses” that could become available sometime in the 2030s. These lenses would bring the era of “visible computing” to the next level, transitioning to a concept known as “invisible computing.” However, it is important to note that the development of these contact lenses is currently uncertain and not confirmed as a guaranteed product.
In addition to contact lenses, Apple had been working on augmented reality “Apple Glasses,” initially planned for release approximately one year after the AR/VR headset. However, this project has been put on hold to prioritize the development of a more affordable version of the AR/VR headset. The Apple Glasses are now expected to launch in 2027 at the earliest.
A future iteration of the AR/VR headset could include accessibility settings designed to assist individuals with eye diseases and visual impairment