Apple invents a Mixed Reality Headset Display that presents users with Sophisticated high-end multi-tiered Visuals for AR Gaming
In August 2020 Patently Apple posted an IP report titled "Apple invents an HMD alarm system to alert a gamer they're too close to furniture that could Cause Harm." Apple's patent related to techniques for alerting a user, who happens to be immersed in a virtual reality environment, to physical obstacles in their physical environment that are in their way that could cause an injury.
The patent specifically stated that a virtual reality device immerses a user in a virtual environment. A user may become so immersed in a virtual environment that, while moving about the virtual environment, the user collides with adjacent physical objects in the physical environment. For example, while wearing a virtual reality head mounted device (HMD), a user may move about a large (virtual) room in virtual reality, but in the process, collide with a physical wall of the physical room in which the user is physically located.
In Apple's example below in FIG. 3C, the Head Mounted Device (HMD #206) can be configured to determine a distance between the HMD and each of the physical objects in the user's physical environment (home or office). It determines distance by using a plurality of depth cameras integrated in the HMD. The depth cameras could include LiDAR sensors.
Apple's patent FIG. 3C clearly outlines the issue and possible solution for users to avoid colliding with physical furniture while being immersed in a VR game.
On Thursday, the US Patent & Trademark Office published a second patent application from Apple on this project titled "Techniques for switching between Immersive Levels." The second patent in this project is more focused on advanced AR gaming.
Apple notes in their patent background that while the user is fully or partially immersed in a VR setting of virtual objects, physical objects in the surrounding physical setting continue to exist. For example, the user may be fully immersed in VR corresponding to a pre-historic world populated with dinosaurs. While that virtual pre-historic world may be deficient in living room furniture, the living room in which the user is located continues to include a coffee table. Moreover, even though the user's dog may be absent from that virtual pre-historic world, the dog may continue to roam about the living room.
An existing technique to avoid any undesirable interactions with physical objects in the surrounding physical setting that are unencumbered by the virtual reality setting involves a user abruptly removing the electronic device providing the experience upon sensing such undesirable interactions. However, as experiences become increasingly immersive, the user may be unable to sense such undesirable interactions fast enough to avoid them.
Moreover, abruptly removing the electronic device during an experience detracts from that experience. As such, it is desirable to address the concerns related to these undesirable interactions while also minimizing any negative impacts on the experience.
Today's patent application covers an invention that generally relates to systems, methods, and devices for selectively transitioning between levels of simulated reality (SR) immersion presented by an electronic device, and in particular, to selectively transitioning between levels of SR immersion using an input device of the electronic device.
In one implementation, an HMD includes a display, an image sensor, and an input device that are each communicatively coupled to a processor of the electronic device. The display is configured to first present content representing a virtual reality setting. Secondly, the display is to present a second content representing a view corresponding to a physical setting in which the electronic device is located, or a combination thereof.
The system may display virtual objects on the transparent or semi-transparent display, so that an individual, using the system, observes the virtual objects superimposed over the physical setting.
In another example, a system may comprise a projection system that projects virtual objects into the physical setting. The virtual objects may be projected, for example, on a physical surface or as a holograph, so that an individual, using the system, observes the virtual objects superimposed over the physical setting.
In another implementation scenario, as part of the SR experience, the user may interact with both virtual objects in the pre-historic world and physical objects in physical setting using physical objects from the physical setting that are unassociated with HMD. Using this example, if the user picks up a ball from the couch and throws that ball, the family dog and the dinosaur may both chase that ball. That ball may both inadvertently knock over a vase resting on the physical coffee table and disturb leaves of a tree from the pre-historic world during the MR experience.
Apple's patent FIG. 6B below depicts the view through display #600 as comprising three layers of visual content (i.e., layers 610, 620, and 630), one skilled in the art will recognize that display may include any number of layers. The patent figure illustrates how a user will be able to view things in the physical and virtual worlds at the very same time that could prevent a user from banging into furniture and hurting themselves which was first discussed in last Summer's patent as well.
This is a very deep patent and to review its details review Apple's patent application 20210081034.
Considering that this is a patent application, the timing of such a product to market is unknown at this time.
By the way, in case you read it somewhere, the patent has nothing to do with Apple planning to stop 'Apple Glass' wearers from walking into traffic (ha!).