Apple today unveiled Apple Vision Pro, a revolutionary spatial computer that seamlessly blends digital content with the physical world, while allowing users to stay present and connected to others.

While there were rumors of some Apple executives being disappointed by the Apple Vision Pro presentation, we can now see that either that rumor was completely unfounded or those executives should be given their walking papers because the Apple Vision Pro's presentation delivered today was 10 times more dramatic for me than that for the iPhone presentation in 2007. It was jaw-dropping.

(New: Click on image to Enlarge)

And with Mike Rockwell, Apple’s vice president of the Technology Development Group, revealing that their new headset was backed by 5,000 patents, a smile broke out on my face. With Patently Apple posting 400+ patent reports, there's obviously a lot more in the pipeline that I can't wait to get my hands on!  Then again, we have hundreds of other patents covering other categories such as In-Air Gestures that are part of this new device.

One patent report that we posted just last Thursday titled "A Future Apple Watch and XR Headset will allow a user to Set-up a 3D Avatar of themselves using on device Cameras," came true today and is seen in Apple's image below. That patent report couldn't have been more timely. Here's another patent on Eye and Hand-Tracking – and there are many, many others on those features.  

(New: Click on image below to Enlarge)

Apple's keynote also touched on those that wear glasses beginning at around the :2:05 minute mark. Apple teamed up with Zeiss to create custom optical inserts that magnetically attach to the lenses. They accommodate a huge range of vision correction. Patently Apple covered a range of patents on this very feature (01, 02 and 03).

Apple also introduced built-in Audio Pods for their headset that provides spatial audio. We covered an Apple patent covering Spatial Audio for Apple's headset here and here.

Lastly, Patently Apple posted a supply chain rumor report on May 17, 2023 that claimed that Apple's headset would run on two processors. Well, technically that was right considering that Apple's headset runs both an M2 chip + a new R1 processor.

(Click on image to Enlarge)

Further, Apple noted today in their keynote that Vision Pro creates an infinite canvas for apps that scales beyond the boundaries of a traditional display and introduces a fully three-dimensional user interface controlled by the most natural and intuitive inputs possible — a user’s eyes, hands, and voice.

(Click on image to Enlarge)

VisionOS: The World's First Spatial Operating System

The rumors about the operating system being branded xrOS and so forth were completely wrong. Apple is calling their revolutionary operating system visionOS, the world’s first spatial operating system, Vision Pro lets users interact with digital content in a way that feels like it is physically present in their space.

The breakthrough design of Vision Pro features an ultra-high-resolution display system that packs 23 million pixels across two displays, and custom Apple silicon in a unique dual-chip design to ensure every experience feels like it’s taking place in front of the user’s eyes in real time.

Tim Cook, Apple’s CEO: "Today marks the beginning of a new era for computing. Just as the Mac introduced us to personal computing, and iPhone introduced us to mobile computing, Apple Vision Pro introduces us to spatial computing. Built upon decades of Apple innovation, Vision Pro is years ahead and unlike anything created before — with a revolutionary new input system and thousands of groundbreaking innovations. It unlocks incredible experiences for our users and exciting new opportunities for our developers."

Mike Rockwell, Apple’s vice president of the Technology Development Group: "Creating our first spatial computer required invention across nearly every facet of the system. Through a tight integration of hardware and software, we designed a standalone spatial computer in a compact wearable form factor that is the most advanced personal electronics device ever."

Extraordinary New Experiences

Apple Vision Pro brings a new dimension to powerful, personal computing by changing the way users interact with their favorite apps, capture and relive memories, enjoy stunning TV shows and movies, and connect with others in FaceTime.

An Infinite Canvas for Apps at Work & at Home

(Click on image to Greatly Enlarge)

Apple's visionOS features a three-dimensional interface that frees apps from the boundaries of a display so they can appear side by side at any scale. Apple Vision Pro enables users to be even more productive, with infinite screen real estate, access to their favorite apps, and all-new ways to multitask. And with support for Magic Keyboard and Magic Trackpad, users can set up the perfect workspace or bring the powerful capabilities of their Mac into Vision Pro wirelessly, creating an enormous, private, and portable 4K display with incredibly crisp text.

Patently Apple pointed to this kind of development in a few reports, with the latest having been posted on May 18, 2023. We noted in the report, that "When you think about it, why would you buy an iMac or PC with a 32" display when you could be working with up to a 100" virtual canvas using AR devices. Being portable makes it priceless."

That report also pointed to a 2018 patent that we covered first wherein a future headset would allow users to edit documents in a 3D environment as Apple's patent figure below illustrates.

Another notable patent was posted by use on May 17, 2023 titled: "Apple wins an advanced camera system patent for their future HMD that includes an Indicator Light that alerts outsiders that the camera is filming." This was one of the features presented today. Apple also touched on lenticular display technology implemented that we covered back on May 6, 2023.

Engaging Entertainment Experiences

With two ultra-high-resolution displays, Apple Vision Pro can transform any space into a personal movie theater with a screen that feels 100 feet wide and an advanced Spatial Audio system. Users can watch movies and TV shows, or enjoy stunning three-dimensional movies. Apple Immersive Video offers 180-degree high-resolution recordings with Spatial Audio, and users can access an exciting lineup of immersive videos that transport them to entirely new places.

Spatial computing makes new types of games possible with titles that can span a spectrum of immersion and bring gamers into all-new worlds. Users can also play over 100 Apple Arcade games on a screen as large as they want, with incredible immersive audio and support for popular game controllers.

Immersive Environments

With Environments, a user’s world can grow beyond the dimensions of a physical room with dynamic, beautiful landscapes that can help them focus or reduce clutter in busy spaces. A twist of the Digital Crown lets a user control how present or immersed they are in an environment.

FaceTime becomes Spatial

(Click on image to Greatly Enlarge)

With Apple Vision Pro, FaceTime calls take advantage of the room around the user, with everyone on the call reflected in life-size tiles, as well as Spatial Audio, so it sounds as if participants are speaking right from where they are positioned. Users wearing Vision Pro during a FaceTime call are reflected as a Persona — a digital representation of themselves created using Apple’s most advanced machine learning techniques — which reflects face and hand movements in real time. Users can do things together like watch a movie, browse photos, or collaborate on a presentation.

Even more App Experiences

Apple Vision Pro has an all-new App Store where users can discover apps and content from developers, and access hundreds of thousands of familiar iPhone and iPad apps that run great and automatically work with the new input system for Vision Pro. Apple’s developer community can go even further and take advantage of the powerful and unique capabilities of Vision Pro and visionOS to design brand-new app experiences, and reimagine existing ones for spatial computing.

A Revolutionary Operating System and User Interface

(Click on image to Greatly Enlarge)

Built on the foundation of decades of engineering innovation in macOS, iOS, and iPadOS, visionOS was designed from the ground up to support the low-latency requirements of spatial computing. The result is a revolutionary operating system that delivers powerful spatial experiences that can take advantage of the space around the user, unlocking new opportunities at work and at home.

visionOS features a brand-new three-dimensional interface that makes digital content look and feel present in a user’s physical world. By responding dynamically to natural light and casting shadows, it helps the user understand scale and distance.

To enable user navigation and interaction with spatial content, Apple Vision Pro introduces an entirely new input system controlled by a person’s eyes, hands, and voice. Users can browse through apps by simply looking at them, tapping their fingers to select, flicking their wrist to scroll, or using voice to dictate. 

There are dozens of patents on Eye/Gaze Tracking systems in our 400+ patent archive and here are just three of them: 01, 02 and 03.

Apple Vision Pro also features EyeSight, an extraordinary innovation that helps users stay connected with those around them. When a person approaches someone wearing Vision Pro, the device feels transparent — letting the user see them while also displaying the user’s eyes. When a user is immersed in an environment or using an app, EyeSight gives visual cues to others about what the user is focused on.

Apple Vision Pro: Breakthrough Design

Apple Vision Pro builds on Apple innovation and experience designing high-performance products like Mac, iPhone, and wearables like Apple Watch, culminating in the most advanced personal electronics device ever. To achieve ambitious goals for performance, mobility, and wearability, Apple utilized the most advanced materials possible.

Apple Vision Pro has an astonishing amount of technology in a compact design. A singular piece of three-dimensionally formed and laminated glass is polished to create an optical surface that acts as a lens for the wide array of cameras and sensors needed to blend the physical world with digital content.

The glass flows into the custom aluminum alloy frame that gently curves around the user’s face, while the modular system allows for a tailored fit to accommodate a wide range of people.

The Light Seal is made of a soft textile, and comes in a range of shapes and sizes, flexing to conform to a user’s face for a precise fit. Flexible straps ensure audio remains close to the user’s ears, while a Head Band — available in multiple sizes — is three-dimensionally knitted as a single piece to provide cushioning, breathability, and stretch.1 The band is secured with a simple mechanism, making it easy to change to another size or style of band.

(Click on image to Greatly Enlarge)

Unrivaled Innovation in Hardware

Apple Vision Pro is designed to deliver phenomenal compute performance in a compact wearable form factor. Featuring a breakthrough ultra-high-resolution display system built on top of an Apple silicon chip, Vision Pro uses micro-OLED technology to pack 23 million pixels into two displays, each the size of a postage stamp, with wide color and high dynamic range. This technological breakthrough, combined with custom catadioptric lenses that enable incredible sharpness and clarity, delivers jaw-dropping experiences. Users with vision correction needs will use ZEISS Optical Inserts to ensure visual fidelity and eye tracking accuracy.

(Click on image to Enlarge)

Apple Vision Pro features an ultra-high-resolution display system that packs 23 million pixels across two displays — more than a 4K TV for each eye — and the brand-new R1 chip, for a virtually lag-free, real-time view of the world.

An advanced Spatial Audio system is core to the Apple Vision Pro experience, creating the feeling that sounds are coming from the environment around the user and matching the sound to the space. Two individually amplified drivers inside each audio pod deliver Personalized Spatial Audio based on the user’s own head and ear geometry.

In addition to creating a breakthrough display and advanced audio experiences, the high-performance eye tracking system in Apple Vision Pro uses high-speed cameras and a ring of LEDs that project invisible light patterns onto the user’s eyes for responsive, intuitive input.

These groundbreaking innovations are powered by Apple silicon in a unique dual-chip design. M2 delivers unparalleled standalone performance, while the brand-new R1 chip processes input from 12 cameras, five sensors, and six microphones to ensure that content feels like it is appearing right in front of the user’s eyes, in real time. R1 streams new images to the displays within 12 milliseconds — 8x faster than the blink of an eye. Apple Vision Pro is designed for all-day use when plugged in, and up to two hours of use with its external, high-performance battery.

A person working with Apple Vision Pro and their computer.

Industry-Leading Privacy and Security

Apple Vision Pro is built on a strong foundation of privacy and security, and keeps users in control of their data.

Optic ID is a new secure authentication system that analyzes a user’s iris under various invisible LED light exposures, and then compares it to the enrolled Optic ID data that is protected by the Secure Enclave to instantly unlock Apple Vision Pro. A user’s Optic ID data is fully encrypted, is not accessible to apps, and never leaves their device, meaning it is not stored on Apple servers. 

Where a user looks stays private while navigating Apple Vision Pro, and eye tracking information is not shared with Apple, third-party apps, or websites. Additionally, data from the camera and other sensors is processed at the system level, so individual apps do not need to see a user’s surroundings to enable spatial experiences. EyeSight also includes a visual indicator that makes it clear to others when a user is capturing a spatial photo or video.

The new Apple Vision Pro Headset is nothing shy of revolutionary and provides with a new vision for the future. I couldn't even keep up with the patents that came to life tody, though I've presented a wide array of them in this report covering an amazing range of next-gen technologies and features. The last two patents that I'll point today, cover Apple discussing on Iris ID and presenting 3D presentations in the classroom.

Congratulations to the entire Apple engineering team that worked on this project for years. What an amazing feat that you've accomplished.  

Considering that their new headset runs an M2 processor, it could technically be an iOS based desktop, if that helps you swallow the initial $3500 U.S., $4,700 Canadian price tag. It won't be mainstream for some time, but watch out when Apple finally lowers the price.

Pricing and Availability

Apple Vision Pro starts at $3,499 (U.S.), and will be available early next year on apple.com and at Apple Store locations in the U.S., with more countries coming later next year. Customers will be able to learn about, experience, and personalize their fit for Vision Pro at Apple Store locations. For more information about Vision Pro, visit apple.com/apple-vision-pro.