Apple will Officially Open its new store in Kuala Lumpur on Saturday June 22
Apple Invents Smartglasses with Adjustable Arm Mechanisms that provide users with a Secure Fit & more

Apple invents a 3D Navigational System for devices like Vision Pro to assist users in Urban Settings

1-cover-Future-Apple-Vision-with-Urban-Navigation

Today the U.S. Patent and Trademark Office officially published a patent application from Apple that relates to displaying content in devices like an HMD (Vision Pro) that determine and present a directional awareness indicator based on context of a real-world physical environment.

Apple notes in their patent background that electronic devices are often used to present users with virtual objects, such as application content, that complement surrounding physical environments that are perceivable in views provided by such electronic devices. Some existing techniques provide views of three-dimensional (3D) environments that may be difficult to navigate. For example, a user may view the physical environment with additional virtual content while walking around the physical environment and lose their sense of direction. In such situations, it may be desirable to provide a means of efficiently providing direction or an indication for orienting a user in a 3D environment in an HMD or other device.

Providing Directional Awareness Indicators based on Context

Apple's invention covers various implementations that include devices, systems, and methods that provide directional awareness indicators (e.g., subtle visual/audio, non-intrusive cues) in certain detected contexts to supplement a user's natural sense of direction. In one example, a ping is played from the north every five minutes using spatial audio.

Example contexts that may trigger such indicators include a user (a) being in a new city or unfamiliar location, (b) performing an activity such as hiking for which orientation is important, (c) acting disoriented or lost, and/or (d) being proximate to a particular location, object, or person.

A directional awareness indicator may identify a cardinal direction (e.g., north) or a direction towards an anchored location or device (e.g., a camp site or another user's device (shared with permission) at a rock concert). The directional awareness indicators may change visually or audibly, for example, based on the factors such as proximity to a particular location, object, or person.

In some implementations, directional awareness indicators may be provided in one or more different sets of views to improve a user experience (e.g., while wearing a head mounted display (HMD) for pass through video).

Some implementations allow interactions with the directional awareness indicators (e.g., an application widget). In some implementations, a device (e.g., an HMD, a handheld or laptop) provides views of a three-dimensional (3D) environment (e.g., a visual and/or auditory experience) to the user and obtains, with a sensor, physiological data (e.g., gaze characteristics) and motion data (e.g., controller moving the avatar, head movements, etc.) associated with a response of the user.

Based on the obtained physiological data, the techniques described herein can determine a user's vestibular cues during the viewing of a 3D environment (e.g., an extended reality (XR) environment) by tracking the user's gaze characteristic(s) and other interactions (e.g., user movements in the physical environment).

Based on the vestibular cues, the techniques can detect interactions with the directional awareness indicators and provide a different set of views to improve a user's experience while viewing the 3D environment.

In some aspects, presenting the directional awareness indicator is further based on a three-dimensional (3D) position relative to the electronic device. In some aspects, the directional awareness indicator comprises an audio cue played to be heard from the 3D position using spatial audio, wherein the 3D position is determined based on the identified direction.

In some aspects, the directional awareness indicator is a visual cue positioned to appear at the 3D position in a view of the physical environment provided via the electronic device, wherein the 3D position is determined based on the identified direction. In some aspects, the directional awareness indicator is not presented based on a criterion with respect to the detected context associated with the use of the electronic device in the physical environment.

Apple's patent FIG. 6 below illustrates an example head-mounted device (HMD), the focal device of this invention; FIG. 5 illustrates device components of an exemplary device such as a "Directional Awareness Illustration Set";  FIG. 3A illustrates an example location map based on the movement of a user with an electronic device in an urban setting.

2.Navigation-System-for-users-of-HMDs

Apple's patent FIG. 3B above illustrates an exemplary view of the electronic device for the user of FIG. 3A that includes a directional awareness indicator in accordance with some implementations.

To review the full details of this invention, check out patent application 20240200962.

10.51FX - Patent Application Bar