Today the U.S. Patent and Trademark Office officially granted Apple two Head Mounted Device (HMD) patents. The first covers a mixed reality headset that includes a 'scene camera' that's part of a holographic direct retinal projector system that is designed to mesh views of the real-world and augmented reality imagery merge seamlessly. The second patent covers an adjustable lens system to ensure that real-world and AR content stay aligned at all times.

HMD Scene Camera

Apple notes that simulated environments of virtual reality systems and/or the mixed environments of mixed reality systems may be utilized to provide an interactive user experience for multiple applications, such as applications that add virtual content to a real-time view of the viewer's environment, applications that generate 3D virtual worlds, interacting with virtual training environments, gaming, remotely controlling drones or other mechanical systems, viewing digital media content, interacting with the Internet, exploring virtual landscapes or environments, or the like.

One of the ways to make Augmented Reality or Mixed Reality systems seem so real to the user is discussed in Apple first HMD granted patent of the day covering a specialty scene camera for mixed reality (MR) direct retinal projector systems.

Embodiments of an MR system are described that include a scene camera that captures images of the real-world scene in front of the user. The images may, for example, be analyzed to locate edges and objects in the scene.

In some embodiments, the images may also be analyzed to determine depth information for the scene. The information obtained from the analysis may, for example, be used to place virtual content in appropriate locations in the mixed view of reality provided by the direct retinal projector system.

To achieve a more accurate representation of the perspective of the user, the scene camera is located on the side of the MR headset and facing the inside surface of the lens.

The lens includes a holographic medium recorded with one or more transmission holograms that diffract a portion of the light from the scene that is directed to the user's eye to the scene camera. In this way, the scene camera captures images of the environment from substantially the same perspective as the user's eye.

To stop unwanted direct light from reaching the scene camera, a band-pass filter, tuned to the transmission hologram wavelength, may be used to block all direct view wavelengths other than the transmission hologram operating wavelength.

In addition, a holographic medium may be applied to an outer surface of the lens and recorded with reflection holograms tuned to the same wavelength as the transmission holograms.

The combination of the band-pass filter and reflection holograms will block the unwanted direct light while still allowing the wavelength of light diffracted by the transmission holograms to reach the photosensor of the scene camera unhindered.

The reflection holograms may also prevent the portion of the direct light to the scene camera corresponding to the wavelength of the transmission holograms from being diffracted to the user's eye by the transmission holograms.

Apple's patent FIG. 1 below illustrates a mixed reality (MR) system that includes a lens with projection holograms to redirect light beams from a light engine into a user's eye while also passing direct light from the environment to the user's eye.

Apple's patent FIG. 8 above illustrates an example MR system in which the light engine and scene camera are contained in an on-frame unit.

For more details, review Apple's granted patent 10,955,677

Adjustable Lens System

Apple's second HMD granted patent relates to optical systems, and, more particularly, to devices with tunable lenses. The head-mounted device may be a pair of virtual reality glasses or may be an augmented reality headset that allows a viewer to view both computer-generated images and real-world objects in the viewer's surrounding environment.

The adjustable lenses may be liquid crystal lenses. A camera and other sensors in the head-mounted device may monitor the eyes of the user and gather other information. Control circuitry in the head-mounted device may control the adjustable lenses based on measured characteristics of the eyes of the user such as interpupillary distance and direction-of-view. The control circuitry may match the distance between the centers of the adjustable lenses to the measured interpupillary distance and may align the lens centers with the measured direction-of-view. The adjustable lenses may have transparent electrodes that are supplied with time-varying control signals by the control circuitry.

Apple's patent FIG. 1 below illustrates a future HMD with an adjustable lens system; FIGS. 4 and 5 are diagrams of an illustrative tunable lens showing how the position of the lens center may be dynamically adjusted.

Apple's patent FIGS. 7 and 8 above are diagrams showing how the locations of the centers of a pair of lenses may be adjusted to accommodate different directions of view through the lenses in accordance with an embodiment.

For finer details, review Apple's granted patent 10,955,724