On Thursday the US Patent & Trademark Office published a patent application from Apple that relates to techniques for alerting a user, who is immersed in a virtual reality environment, to physical obstacles in their physical environment that are in their way and could cause an injury.

The patent filing states that a virtual reality device immerses a user in a virtual environment. A user may become so immersed in a virtual environment that, while moving about the virtual environment, the user collides with adjacent physical objects in the physical environment. For example, while wearing a virtual reality head mounted device (HMD), a user may move about a large (virtual) room in virtual reality, but in the process, collide with a physical wall of the physical room in which the user is physically located.

In accordance with some embodiments described herein, while a virtual reality environment is displayed to the user, a virtual reality device determines a distance between the device and a physical object of a physical environment.

The virtual reality device further determines whether the device has come within a (first) threshold distance to a physical object of a physical environment. If the device determines it is too close to the physical object, the device displays a visual effect in the virtual reality environment alerting the user to the presence of a nearby physical obstacle.

The visual effect corresponds to the distance associated with the physical object and includes, for example, one or more virtual objects. In some examples, the visual effect has the appearance of a glass pane, to represent a nearby physical wall with which the user might collide if the user continues to move forward.

The bottom line is that the virtual reality device alerts the user to nearby physical objects. If the user continues to move towards the physical object, such that the virtual reality device comes even closer to the physical object (within a second threshold distance) despite the visual effects, the device displays a live image or video of the physical environment. In this way, the user is provided with a pass-through visual to the physical environment such that the user can see the exact physical obstacle without removing the head-mounted virtual reality device.

In some embodiments, the HMD could be configured to determine whether it is moving toward a physical object in physical environment. The HMD could be configured to make such determination using one or more cameras and/or depth sensors built into the HMD.

For example, using a plurality of cameras (e.g., digital cameras, infrared cameras), the HMD could determine a distance between the device and one or more physical objects in physical environment like your living room.  

As one example, the distance is determined based on the discrepancies of the 3D perception captured by two or more cameras of the HMD. As another example, a depth sensor (e.g., a 3D time-of-flight sensor) is used to estimate the distance between each of the physical objects in physical environment and the HMD. A depth sensor may be, for example, be a LiDAR sensor.

Apple's patent application 20200258278 That was published today by the U.S. Patent Office was filed back in  March 2020. Considering that this is a patent application, the timing of such a product to market is unknown at this time.

For the record, this is an odd patent in that it is recorded as having been allowed to expire in 2017 and then again in 2018 according the U.S. Patent "Public Patent Application Information Retrieval" tool. This certainly doesn't sound like something Apple would do two years in a row. 

Further, one of the inventors, Duncan McRoberts, works in VR gaming, but no association with Apple. Another of the inventors (Mirhosseini) has published several papers on VR, with one titled "Exploration of Large Omnidirectional Images in Immersive Environment," but there's absolutely no connection to Apple. A third inventor, Avi Bar-Zeev, was briefly at Apple though left the company in January 2019. The latest patent filing was made on March 27, 2020 or 14 months after Bar-Zeev left Apple.

Lastly, the patent uses different lingo not commonly used within an Apple patent, such as "Simulated Reality."

Of course, whether Apple acquired it or not, doesn't take away from the fact that it would be a good to have such a feature available in Apple's future VR HMD. Some users/gamers could get quite carried away swinging their arms to and fro as they get into the game's action. A safety feature to protect those types of gamers wouldn't hurt.