When Apple launched Vision Pro last week, Mike Rockwell, VP, Technology Development Group, stated that Apple had filed over 5,000 patents supporting the technology behind their headset. Yet how many patents are for future features or features that may never make it to market? While Apple presented an grand overview of Vision Pro, I'm sure that there could be features that they may not have even been made public so as to keep their competitors in the dark.

This week an Apple was published by the US Patent & Trademark Office relating to a XR Headset "Scene Camera." No such camera by that description was introduced last week and yet it's one of the 5,000 patents filed.

I was recently asked by one of our fans, "Where is the iPhone patent." It was an honest question and yet, there is no singular "iPhone patent." That's not how a product comes to be. There is now Apple Vision Pro patent, there are 5,000 patents that Mike Rockwell and his team decide get to pick and choose the technologies that will go into a particular version of a headset.

In one of Apple's XR Headset published this week we see a well thought out camera system. It's a clear project being that two different teams were assigned to explore it. The first patent that we discovered on a scene camera was published back in 2020. We posted a patent report on this titled " Apple Invents a 'Scene Camera' System for a future HMD device that brings Vivid Imagery directly to the user's eyes."

The second patent, filed by a secondary engineering team, is simply titled "Camera System" though in context with it being a scene camera. Apple clearly notes in their filing that "various embodiments of scene cameras for video see-through head-mounted displays (HMDs) that may be used in mixed reality (MR) or virtual reality (VR) systems are described.

In conventional HMDs, one or more scene cameras may be mounted at the front of the HMD. However, typically the entrance pupil and thus the point of view (POV) of the scene cameras is substantially offset from and thus different than the POV of the user’s eyes.

Embodiments of scene camera configurations are described that at least partially correct the point of view (POV) of the cameras to match the POV of the user by shifting the entrance pupils of the cameras towards the user’s eyes.

In some embodiments, an HMD includes two-dimensional arrays of small form factor cameras (e.g., one array for the left eye, and a second array for the right eye) that capture images of respective portions of a real-world scene in front of the user. The cameras are positioned along a spherical curve or surface so that the cameras have non-overlapping, adjacent fields of view (FOVs). To achieve a more accurate representation of the perspective of the user, the cameras’ optics are configured so that the entrance pupils of the cameras in the array are positioned behind the cameras’ image planes formed at the image sensors, and at or near the user’s eye while the cameras also form optimized images at the sensor. Thus, each array of cameras captures views of the scene from substantially the same perspective as the user’s respective eye.

Each camera in an array is configured to jointly optimize the entrance pupil plane at the user’s eye and the image plane at the camera’s sensor. In some embodiments, each camera includes, in order from the object side to the image side, a first lens group including one or more lens elements, an aperture stop (e.g., a “pinhole” aperture stop), a second lens group including one or more lens elements, and a sensor. The lens element(s) in the first lens group may be formed (e.g., cut or “diced”) so that the cameras fit together in the array with minimal or no gaps between the lenses in the first lens group.

Apple's patent FIG. 1A below illustrates a head-mounted display (HMD) that includes an array of cameras with entrance pupils at or near the user's eye; FIG. 2 illustrates a portion of a camera array.

Apple's patent FIG. 3 below illustrates an example camera with entrance pupil at or near the user's eye that may be used in an array as illustrated in FIGS. 1A and 2.

Apple's FIGS. 9A and 9B below illustrate reducing the number of cameras by reducing resolution and shifting the entrance pupil of the cameras in peripheral regions.

Apple's patent FIG. 16 above graphically illustrates adding a negative meniscus lens in front of a camera array to increase the field of view (FOV) of the cameras towards the periphery; FIG. 17 graphically illustrates adding a negative meniscus lens in front of a camera array to progressively offset the point of view (POV) of the cameras towards the periphery.

Apple's patent FIGS. 20A through 20C below graphically illustrate an example scene camera that includes a negative meniscus lens in front of the cameras in the camera array.

For more details, review Apple's patent application number 2023018509.

A Few of Apple's Inventors

• Emanuele Mandelli: Manager, Camera Custom Silicon
• Branko Petljanski: Senior Engineer – Camera Electronics
• Dan Hennigan: Optical Mechanical Engineering Manager
• Saito Kenichi: Optical Engineer (previously worked at Cannon & Fuji Photo Film)
• Brett Miller: Engineering Manager, Camera Incubation
• Noah Bedard: Prototyping Engineer 
• Kathrin Berkner: Senior Engineering Manager - Camera Incubation