Apple Vision Pro

In May 2015, Apple acquired the German augmented reality (AR) company Metaio, originally spun off from Volkswagen.[6] rumored to be originally motivated by Project Titan.[7] That year, Apple hired Mike Rockwell from Dolby Laboratories. Rockwell formed a team that included Metaio co-founder Peter Meier and Apple Watch manager Fletcher Rothkopf. The team, called the Technology Development Group, developed an AR demo in 2016 but faced opposition from then-chief design officer Jony Ive and his team. Augmented reality and virtual reality (VR) expert and former NASA specialist Jeff Norris was hired in April 2017.[8][9] Rockwell's team helped deliver ARKit in 2017 with iOS 11. Rockwell's team sought to create a headset and worked with Ive's team; the decision to reveal the wearer's eyes through a front-facing eye display went over well with the industrial design team.[10]

The headset's development experienced a period of uncertainty with the departure of Ive in 2019. His successor, Evans Hankey, left the company in 2023.[11] Senior engineering manager Geoff Stahl, who reports to Rockwell, led the development of its visionOS operating system,[9][12] after previously working on games and graphics technology at Apple.[13] Apple's extended reality headset is meant as a bridge to future lightweight AR glasses, which are not yet technically feasible.[14][15] In November 2017, Apple acquired Canadian XR company Vrvana for $30 million. The Vrvana Totem was able to overlay fully opaque, true-color animations on top of the real world rather than the ghost-like projections of other AR headsets, which cannot display the color black. It was able to do this while avoiding the often noticeable lag between the cameras capturing the outside world while simultaneously maintaining a 120-degree field of view at 90 Hz. The Totem also used IR illuminators and infrared cameras to perform spatial and hand tracking. [16][17]

According to leaker Wayne Ma, Apple was originally going to allow macOS software to be dragged from the display to the user's environment, but was scrapped early on due to the limitations of being based on iOS, and noted that the hand-tracking system was not precise enough to game. Workers also discussed collaborations with brands such as Nike for working out with the headset, while others investigated face cushions that were better suited for sweaty, high-intensity workouts, but was scrapped due to the battery pack and the fragile screen. A feature called "co-presence", a projection of a FaceTime user's full body was also scrapped due to unknown reasons.[18]

Information about a headset, then rumored to be called the Reality Pro,[19] began surfacing in 2022. In May 2022, Apple executives previewed the device, including CEO Tim Cook.[20] The company began recruiting directors to develop content for the headset in June. One such director, Jon Favreau, was enlisted to bring the dinosaurs on his Apple TV+ show Prehistoric Planet to life.[21] By April, Apple was attempting to attract developers to make software and services.[22] Apple filed over 5,000 patents for technologies which contributed to the development of Vision Pro.[23] The Apple Vision Pro was announced at the Worldwide Developers Conference on June 5, 2023, with availability beginning in early 2024 in the United States at a starting price of US$3,499 and later in 2024 in additional countries, including the United Kingdom, Canada, Japan, India, Brazil, Australia, France, Germany, Spain, Italy and South Korea, at varying starting prices.[24] An estimated million units will be shipped initially.[25]

On June 6, Apple acquired the AR headset startup Mira, whose technology is used at Super Nintendo World's Mario Kart ride. The company has a contract with the United States Air Force and Navy. Eleven of the company's employees were onboarded.[26]

Apple Vision Pro has a laminated glass display front, an aluminum frame covered by a flexible cushion, and an adjustable headband. The frame contains five sensors, six microphones and 12 cameras. Two micro-OLED displays with a total of 23 megapixels usually running at 90 FPS from a maximum of 96, each with a size of a postage stamp, will be seen by the user through the lens. The eyes are tracked by a system of LEDs and infrared cameras, which form the basis of the device's iris scanner named Optic ID (used for authentication, like the iPhone's Face ID). Custom optical inserts are supported for users with prescription glasses; these lenses will attach magnetically to the main lens and are developed in partnership with Carl Zeiss AG. The device's speaker is inside the headband and is placed directly over the user's ears. It can also virtualize surround sound.[2][27]

The Vision Pro uses the Apple M2 system on a chip. It is accompanied by a co-processor known as Apple R1, which is used for real-time sensor input processing. The device can be powered by an external power supply, a USB-C port on a Mac, or a battery pack rated for two hours of use.[28][29][30]

The user's face is scanned by the headset during setup to generate a persona—a realistic avatar used by OS features.[31] During a public demo at Worldwide Developers Conference (WWDC), hands-on participants faces were scanned by an iPhone with TrueDepth camera for fitting purposes, and ears were scanned to optimize the speakers, with Apple stating this process can be done with the user's own iPhone or at an Apple Store as part of the purchase process.[32]

A "digital crown" dial on the headset is used to control the amount of virtual background occupying the user's field of view, ranging from a mixed-reality view where apps and media appear to float in the user's real-world surroundings, to completely hiding the user's surroundings.[28][29] The Vision Pro has an outward-facing display called EyeSight, which displays the eyes of the user's CGI persona. Its eyes appear dimmed when in AR and obscured when in full immersion to indicate the user's environmental awareness. When someone else approaches or speaks, even if the user is fully immersed, EyeSight shows their persona's virtual eyes normally and makes the other person visible.[29][27]

visionOS

Developer	Apple Inc.
OS family	Unix-like, based on Darwin (BSD), iOS, iPadOS
Working state	Current
Initial release	Early 2024
Marketing target	Mixed reality headsets, Apple Vision Pro
Platforms	ARMv8-A
Kernel type	Hybrid (XNU)
License	Proprietary software with open-source components
Official website	www.apple.com/apple-vision-pro/
Support status
Beta

Apple Vision Pro runs visionOS (internally called xrOS during its development[33]), which is derived primarily from iOS core frameworks (including UIKit, SwiftUI, and ARKit), and XR-specific frameworks for foveated rendering and real-time interaction.[1][34]

The operating system uses a 3D user interface navigated via finger tracking, eye tracking, and speech recognition. For example, the user can click an element by looking at it and pinching two fingers together, move the element by moving their pinched fingers, and scroll by flicking their wrist. Apps are displayed in floating windows that can be arranged in 3D space. visionOS supports a virtual keyboard for text input, the Siri virtual assistant, and external Bluetooth peripherals including Magic Keyboard, Magic Trackpad, and gamepads.[29][35] Calls can be presented via windows of participants's personas.[32]

The Walt Disney Company announced plans to develop apps for visionOS, including a Disney+ app incorporating mixed-reality experiences.[36] Over 100 Apple Arcade games will be compatible with visionOS at launch.[35][37][38] The company is working with Unity Technologies to support the Unity engine on visionOS.[39][40] WebXR will be supported in Safari for visionOS.[41] visionOS will be backwards compatible with existing iOS/iPadOS apps, which will be rendered in windows within the user environment.[42]

Ars Technica reported that the Vision Pro was "truly something I had never seen before," noting the intuitiveness of its user interface in a choreographed demo given by Apple, and praising a dinosaur tech demo for its immersiveness. Feeling that its displays were dim but "much better than other headsets I've used on this front, even if it still wasn't perfect," and that the personas looked "surreal" but conveyed body language better than a more stylized avatar (such as Animoji or Horizon Worlds).[32] Arguing that the Vision Pro was not a virtual reality (VR) platform, nor a competitor to Meta Platforms' Quest (formerly Oculus) product line, due to its positioning as "primarily an AR device that just happens to have a few VR features," and not as a mass market consumer product.[32]

The Verge similarly noted that Apple had deliberately avoided presenting the Vision Pro as a VR platform, such as not referring to the device as a "headset," describing it as an AR device and "spatial computer," and not showcasing fully-immersive experiences such as games and metaverse platforms in its presentation (only demonstrating non-VR games, displayed in windows and controlled using an external gamepad). He suggested that this positioning "leaves wiggle room for the likely future of this technology that looks nothing like a bulky VR headset: AR glasses."[43]

Bloomberg noted that loud gasps were heard among the crowd in response to the announcement of the Vision Pro price tag, followed by criticism of the cost by both consumers and online commentators. The three priciest components in the Vision Pro are its camera and sensor array, its dual Apple silicon chips, and the twin 4K micro-OLED virtual reality displays. Apple is reportedly working on a cheaper model that is scheduled for release in late 2025 or early 2026 as well as a second-generation model with a faster processor.[44]

The Vision Pro has been criticized due to its high cost, seen by some as too high to go mainstream,[45][46][47] its short battery lifespan,[48] appearing distracting to others,[48] and its lack of HDMI input[49][50] and haptic feedback.[48]