Last Thursday the US Patent & Trademark Office published a series of patent applications from Apple that related to their future Head Mounted Display (HMD) device (01, 02, 03, 04 & 05). Today's report covers the last HMD invention from last week that specifically relates to scene cameras that use a series of mirrors that is able to bring the scene in front of the HMD closer to the user's eyes to achieve a more accurate representation the scene.

Virtual reality (VR) allows users to experience and/or interact with an immersive artificial environment, such that the user may feel as if they were physically in that environment. virtual reality systems may display stereoscopic scenes to users in order to create an illusion of depth, and a computer may adjust the scene content in real-time to provide the illusion of moving within the scene. When a user views images through a virtual reality system, the user may feel as though they're moving within the scenes from a first-person point of view.

Mixed reality (MR) covers a spectrum from augmented reality (AR) systems that combine computer generated information (referred to as virtual content) with views of the real world to augment, or add virtual content to, a user's view of their real environment (referred to as), to augmented virtuality (AV) systems that combine representations of real world objects with views of a computer generated three-dimensional (3D) virtual world.

The simulated environments of virtual reality systems and/or the mixed environments of mixed reality systems may thus be utilized to provide an interactive user experience for multiple applications, such as applications that add virtual content to a real-time view of the viewer's environment, applications that generate 3D virtual worlds, interacting with virtual training environments, gaming, remotely controlling drones or other mechanical systems, viewing digital media content, interacting with the Internet, exploring virtual landscapes or environments, or the like.

To make simulated or augmented environments as realistic as possible, Apple's invention focuses on what is known in the art as scene cameras.

According to Apple, the scene cameras may, for example, be used in video see-through devices in mixed reality (MR) or virtual reality (VR) systems.

In conventional video see-through devices, one or more scene cameras may be mounted at the front of the device. However, these scene cameras typically don't align with a user's actual Point Of View (POV). Apple's scene cameras will at the very least partially correct the POV of the cameras to more closely match the POV of a user by imaging the entrance pupils of the cameras at a location closer to the user's eyes.

Apple's scene camera system may include mirrors and cameras that capture light from the scene reflected by the mirrors. By using the mirrors to reflect the light, the cameras' entrance pupils are imaged at a location closer to a user's eyes to achieve a more accurate representation of the perspective of the user.

Apple's patent FIG. 1 below illustrates an array of cameras with entrance pupils at or near a user's eye; FIG. 4 illustrates using curved mirrors in a scene camera system; FIG. 5 illustrates using segmented mirrors in a scene camera system, according to some embodiments.

Apple's patent FIG. 6 above is a high-level flowchart of a method of operation for a scene camera system.

Apple's patent application 20200103669 that was published last Thursday by the U.S. Patent Office was filed back in Q3 2019. Considering that this is a patent application, the timing of such a product to market is unknown at this time.

Some of Apple Inventors

Noah Bedard: Prototyping Engineer; Came to Apple via Ricoh Innovations as Senior Research Scientist

Branko Petljanski: Engineering Manager, Incubation; Came to Apple via Panavision

Eddie Huo: Product Design Engineer; designed mechanical systems in multiple iPhone models from architectural concepts to mass production.

Brett Miller: Engineering Manager, Camera Incubation; Came to Apple via Intel, Perceptual Computing.