Apple's latest Binaural Audio Patent describes a Turn-by-Turn-like System for Pedestrians in a Mall, Park Path & Beyond
Today the US Patent & Trademark Office published a patent application from Apple that relates to Binaural Audio. It's their third such patent in the last 30 days. Our first report on this technology was posted on July 23 titled "Apple Takes Binaural Audio to the next-level by adding Head-Tracking for Next-Gen AirPods Pro and Future HMD." That was followed by our second report posted on August 11 titled "Apple Wins a Patent for new AirPods Pro Safety Features when Biking, Jogging and working out on a new Exercise Mat with Sensors."
Today's published patent application uses similar patent figures that were presented in our July report, such as the image below. Apple's patent FIG. 1A below illustrates a user with an iPhone and AirPods Pro using Binaural Audio and 2A illustrates a future head mounted device (HMD) using Binaural Audio VR gaming.
Apple's patent FIG. 6B above illustrates how different Binaural Audio is, in that it tracks the users head so as to move the audio with the user as if it's always in front of them or projecting sound relative to the user's head so noises behind them sound behind them or to the right of them or above them. Great for RPG type gaming in VR mode.
Apple notes in today's published patent application that navigation applications for devices include but not limited to mobile multipurpose devices like an iPhone and vehicle A/V systems like CarPlay that provide visual cues and/or vocal directions such as "turn left" or "turn right" to direct users to destinations.
In the case of mobile multipurpose devices such as smartphones and tablet/pad devices, navigation applications may be used when walking or riding a bicycle to guide a user on a path through a city, park, etc. using visual cues and/or vocal directions.
In the case of Virtual reality (VR), it allows users to experience and/or interact with an immersive artificial environment, such that the user feels as if they were physically in that environment. For example, virtual reality systems may display stereoscopic scenes to users in order to create an illusion of depth, and a computer may adjust the scene content in real-time to provide the illusion of the user moving within the scene.
What's New in Apple's Third Binaural Audio Patent?
Apple describes a spatial audio navigation system that begins with the user's current location, their target location, and then having that information mapped out using a pathfinding algorithm to determine a real-world path between the user's current location and the target location as presented below in patent FIG. 6C.
(Click on image to Enlarge)
Apple's notes that FIG. 6C above illustrates a discrete path tracing method in which a sound is moved in discrete intervals along the path. As the user gets closer to their destination, the sound increases in the user's AirPods to provide the user with feedback that they're on course.
More specifically, Apple notes that "the spatial audio navigation system may then use directional audio played through the headset to guide the user on the path from the current location to the target location. The user's current location and direction of motion as determined by the mobile multipurpose device and head orientation and movement as determined by the headset are used by the spatial audio navigation system to adjust the perceived or virtual directionality and distance of the audio, for example by adjusting the volume of the left and right audio channels as the user moves in the real world to guide the user along the path to the target location.
Instead of or in addition to adjusting the volume, other aspects of the audio may be attenuated to affect the virtual directionality and distance of the audio, including but not limited to frequency and reverberation."
According to Apple, the position sensors found in the HMD or AirPods may include one or more of, but are not limited to, GPS (global positioning system) technology sensors, dGPS (differential GPS) technology sensors, cameras, indoor positioning technology sensors, SLAM (simultaneous localization and mapping) technology sensors, etc.
The headset worn by the user includes motion sensors used to detect and track motion and orientation of the user's head with respect to the real-world. The motion sensors may include one or more of, but are not limited to, IMUs (inertial measurement units), gyroscopes, attitude sensors, compasses, etc. The real-world map information may include, but is not limited to, information about routes between locations, appropriate passable areas (sidewalks, streets, doors, etc.), and obstacles (walls, buildings, fences, etc.).
Whether Apple will allow users to choose between verbal navigational cues or directional audio cues like beeps to when a turn is needed is unknown at this time.
Apple's patent application 20200264006 that was published today by the U.S. Patent Office was filed back in Q1 2020. For those wanting to dive in deeper could do so here. Considering that this is a patent application, the timing of such a product to market is unknown at this time.