Apple wins a VR Headset patent relating to Redirecting users Walking in VR Environments such as Games
Today the U.S. Patent and Trademark Office officially granted Apple a patent that relates to a future HMD System that could be used with a gaming console or computer. The patent covers redirected walking methods that may be implemented in virtual reality (VR) systems for guiding or redirecting users' movements within constrained physical environments or in unconstrained physical environments, while the users perceive that they are freely moving about within virtual worlds provided by the VR systems.
Apple's granted patent, not made public before, covers various embodiments of methods and apparatus for providing redirected walking in three-dimensional (3D) virtual reality (VR) environments provided by VR applications executing on VR systems are described.
Embodiments of redirected walking methods are described that may be implemented by VR applications executing on VR systems for guiding or redirecting users' movements within constrained physical environments such as rooms, gyms, yards, or fields, or in unconstrained physical environments, while the users perceive that they are freely moving about within the virtual worlds provided by the VR applications and systems.
The redirected walking methods may generally involve rendering views of a VR world based on a user's movements in the real environment, adjusting the views (e.g., by slightly shifting each view to subtly rotate the VR world) according to the user's position and movements with respect to objects in and boundaries of the real environment, and displaying the adjusted views to the user. In response to visual cues introduced to the views by the redirected walking methods, the user may adjust their path in the real-world environment. For example, a rotation of the VR environment introduced by a shift of one or more VR views may cause the user to turn in a direction opposite to the rotation.
The redirected walking methods may introduce a deliberate mismatch in the mapping of virtual to real space geometry; the mismatch may cause the user to adjust their real-world path or orientation to achieve a desired virtual path or orientation. This mismatch may be algorithmically determined to optimally use the provided real space in the exploration of the virtual space.
The redirected walking methods may include a method in which a subtle visual yaw rotation of the VR world is introduced while the user is walking straight in the VR world, for example along a straight path. This rotation of the VR world may cause the user to subconsciously correct their physical motion path in the direction opposite to the VR world rotation in order to keep walking straight (e.g., stay on the straight path) in the VR world.
This rotation technique may, for example, allow the VR application and system to steer the user around a circle, for example a circle with a six meter or greater radius, without the user realizing it. To the user, it may appear that they are walking straight on a straight path in the virtual world, while in the physical world they are actually walking in a circle.
The redirected walking methods may also include a method in which the rate of turns that the user makes in the VR environment are decreased or increased. By decreasing the apparent rate of a turn visible in the VR world, the user is manipulated to compensate with a steeper turn in the real world. Decreases of up to 20% of the rotational rate and increases of up to 70% of the rotational rate may remain unnoticed by users.
The redirected walking methods may also include a method in which the VR world is rotated around the user during blinks. The HMD may include eye tracking sensors that the VR system may use to detect blinks. On average, blinks happen about 10 times per minute. Assuming that a 2-degree yaw rotation of the VR world goes undetected during a blink, the user may be rotated at a rate of 20 degrees per minute when the user is not moving by subtly rotating the VR world by up to 2 degrees when each blink is detected.
The redirected walking methods may also include a method in which the rate of rotation of the VR world is changed during head rotations without the user necessarily physically moving anywhere. For example, if the VR system detects via motion sensors of the HMD that the user is turning their head to the left or right, the rate at which the VR view turns may be slightly increased while the head is turning away (e.g., up to 2 degrees/second) and slightly decreased while the head is turning back (e.g., up to 1 degree/second).
Note that, in some embodiments, in some cases the head motion may be slightly decreased while the head is turning away (e.g., up to 1 degrees/second) and slightly increased while the head is turning back (e.g., up to 2 degree/second). Changing the rate of rotation of the VR world as the head is rotated in the user's view of the VR world being somewhat rotated (e.g., up to 5 degrees) during the head turn, which may cause the user to compensate by reorienting themselves in the physical world to face in the correct direction in the virtual world.
Apple's patent FIG. 1 below illustrates a virtual reality system. In some embodiments, a virtual reality system #10 may include an HMD (#100) such as a headset, helmet, goggles, or glasses that may be worn by a user (#190), and a base station (#160) configured to render VR frames for display by the HMD. In some embodiments, the HMD and base station may each include wireless communications technology that allows the HMD and base station to communicate and exchange data via a connection (#180). However, in some embodiments, a wired connection between the HMD and base station may be used.
While not shown in FIG. 1, in some embodiments the virtual reality system may include one or more other components. For example, the system may include a controller or game controller that may be held by the user that allows the user to interact with virtual content in the 3D virtual view (#102), or to otherwise control the VR experience.
(Click on image to Enlarge)
The base station #160 above may be an external device (e.g., a computing system, game console, etc.) that is communicatively coupled to HMD via the connection.
Apple's patent FIG. 2 below illustrates world-facing and user-facing sensors of a head-mounted display (HMD) in a virtual reality system as illustrated in FIG. 1.
(Click on image to Enlarge)
Apple's patent FIGS. 6A and 6B below graphically illustrate an example real world path that a user may traverse in a constrained physical environment when traversing a virtual path in a VR environment using a VR system that implements one or more of the redirected walking methods, according to some embodiments. FIG. 6A shows a constrained real world environment #600, such as a room or yard, that in this example is approximately 9.times.12 meters in dimension. The environment may include one or more stationary obstacles such as chairs, tables, trees, or bushes, and while not shown may also include moving objects such as other users.
Apple's patent FIG. 6B illustrates an example virtual path (#615) in a virtual world that the user (#690) may travel. The virtual path may be a predefined path (with possible forks) or may be a path that the user freely chooses to travel in an unconstrained virtual world.
(Click on image to Enlarge)
There's a lot of detail to explore and you could do so by reviewing Apple's granted patent 11,158,126. One of the inventors noted on the patent is Yury Petrov: Research Scientist (worked 2.5 years at Oculus before coming to Apple).