Apple won a Patent relating to Range Finding and Accessory Tracking for XR Headsets and Future Smartglasses
Today the U.S. Patent and Trademark Office officially granted Apple a patent that relates to Range finding methods and apparatus that may be implemented by VR/MR systems that include a head-mounted display (HMD) and an accessory that the user holds or wears. Range finding sensors (e.g., ultrasonic transducers) may be included on the HMD and on the accessory and used to track distances to and relative position of walls, objects, and other obstacles within constrained physical environments such as rooms, gyms, yards, or fields, or in unconstrained physical environments.
Apple's patent covers various embodiments of methods and apparatus for range finding in virtual reality (VR) and mixed reality (MR) systems. Embodiments of range finding methods and apparatus are described that may, for example, be implemented by VR/MR systems that include a head-mounted display (HMD) such as a form of Vision wear including smartglasses, and an accessory that the user is holding or wearing (e.g., a hand-held device such as a controller for the HMD or a mobile multipurpose device like iPhone).
In embodiments, range finding sensors (e.g., transmitters and receivers (or transceivers), for example ultrasonic transducers), may be included on the HMD and on the accessory and used to track distances to and relative position of walls, objects, and other obstacles within constrained physical environments such as rooms, gyms, yards, or fields, or in unconstrained physical environments.
Range finding information from the sensors on the HMD and accessory can be used to generate a 3D map of the user's environment that can be used for various purposes in the VR/MR system. In addition to mapping the user's environment, embodiments of the range finding methods and apparatus may also be used to track the relative position of the accessory with respect to the HMD.
To track distances to and relative position of walls, objects, and other obstacles within physical environments, the ultrasonic transducers on the HMD and/or the ultrasonic transducers on the accessory send ultrasonic signals or “pings” that reflect or echo off surfaces in the environment; the echoes are received by some number of ultrasonic transducers on the HMD and/or on the accessory. Time delays for the echoes can be measured and analyzed to estimate the distances and relative positions of the surfaces. Including range finding sensors such as ultrasonic transducers on the accessory allows more of the environment to be scanned than can be scanned by the range finding sensors on the HMD.
The position of the accessory with respect to the HMD can be tracked using the ultrasonic transducers, or a combination of the ultrasonic transducers with motion and orientation information obtained from an inertial-measurement unit (IMU) in the accessory and/or in the HMD.
In some embodiments, to track the accessory, the ultrasonic transducers on the HMD (e.g., three transducers) send ultrasonic signals or “pings” that are received by some number of ultrasonic transducers on the accessory. Alternatively, the ultrasonic transducers on the accessory (e.g., three transducers) send ultrasonic signals that are received by some number of ultrasonic transducers on the HMD. Time of flight for the signals from the transmitters to the receivers may be analyzed to determine distance, relative position, and orientation of the accessory with respect to the HMD.
Apple's patent FIG. 1 below illustrates an example virtual reality (VR) or mixed reality (MR) system that includes a head-mounted display (HMD) and accessory that include ultrasonic sensors for mapping the user's environment; FIG. 2 illustrates an example VR/MR system that includes ultrasonic sensors on the HMD and accessory that are used to track relative position of the accessory to the HMD.
Apple's patent FIG. 3 above is a block diagram illustrating components of an example VR/MR system as illustrated in FIGS. 1 and 2.
As shown in FIG. 5 below, the user holds the accessory #510 in their hand or wears it on their arm, and thus the ultrasonic transducers on the accessory may scan portions of the environment that are not within the range of the ultrasonic transducers on the HMD #500, for example to the side or rear of the user. Also, the user may move their arm about while holding the accessory to map portions of the room that are not within the range of the ultrasonic transducers on the HMD. In addition, the range finding sensors on the accessory may detect potential collisions of the user's hand or arm with surfaces and objects in the environment when the user is moving their arm; the user may be alerted to prevent a collision, for example using audio, visual, and/or haptic feedback.
For full details, review Apple's granted Patent 12124622.
Today's Other HMD Granted Patents issued Today
- 12124042: Systems With Adjustable Cushions
- 12124043: Electronic Devices With Drop Protection
- 12125158: Coherent Occlusion Of Objects