Today the U.S. Patent and Trademark Office officially granted Apple a patent that relates to head tracking for spatial audio applications.

Apple notes in their patent background that spatial audio creates a three-dimensional (3D) virtual auditory space that allows a user wearing an auxiliary device with inertial sensors to pinpoint where a sound source is located in the 3D virtual auditory space, while watching a movie, playing a video game or interacting with augmented reality (AR) or virtual reality (VR) content on a source device (e.g., a tablet computer). Some existing spatial audio platforms include a head tracker that uses a video camera to track the head of the user and a face detector that provides a face reference frame derived from camera images. A head tracking filter uses the face reference frame to correct for drift error in the inertial sensor measurements. In some applications, however, a face reference frame is not available to the head tracking filter for various reasons, such as communication errors (e.g., dropped packets), low-lighting conditions, etc.

Apple's granted patent covers adaptive audio centering for head tracking in spatial audio applications. In an embodiment, a method comprises: obtaining first motion data from an auxiliary device communicatively coupled to a source device, the source device configured to provide spatial audio content and the auxiliary device configured to playback the spatial audio content; obtaining second motion data from one or more motion sensors of the source device; determining whether the source device and auxiliary device are in a period of mutual quiescence based on the first and second motion data; in accordance with determining that the source device and the auxiliary device are in a period of mutual quiescence, re-centering the spatial audio in a three-dimensional virtual auditory space; and rendering the 3D virtual auditory space for playback on the auxiliary device.

Apple's patent FIG. 1A below illustrates a system for implementing a spatial audio application; FIG. 1B illustrates a centered 3D virtual auditory space; FIG. 1C illustrates boresight correction using bleed-to-zero (BTZ).

Apple's patent FIG. 2 above is a flow diagram of a process for audio re-centering based on mutual quiescence.

Apple's patent FIG. 7C above is a multi-panel timeline illustrating an example scenario where the user is walking while viewing content from their source device in hand. The first panel shows boresight yaw, the second panel shows boresight pitch. The lines #710 and #711 are truth data and the lines #712 and #713 are estimated boresight yaw and pitch angles. The third panel shows tracking status: tracking1IMU is single IMU tracking mode, where just headset IMU #1111 is used in tracking.

Also shown is complex transition states and BTZ events. The fourth panel shows the magnitude of rotation rates on source device #102 and auxiliary device #101. The rotation rates provides a sense of how much motion there is on both devices while user #100 is walking. Note that the BTZ process was triggered multiple times during walking, despite the noisy bounces in the sensor data due to the walking. Also when the user turns their head, the BTZ process exits immediately to capture the head turn so tracking is perceived as responsive by the user.

To review this detailed invention check out granted patent 12219344.  Apple's inventors include Alexander Singh Alvarado (Engineering Manager) and Xiaoyuan Tu (Lead Scientist & Senior Software Engineer, Motion and Location Technology), referred to as a "Distinguished Technologist." 

Other Spatial Audio Patents on Record

01:  Apple wins a Patent relating to Spatial Audio in context with Augmented Reality and a Future HMD

02: Seven Major Spatial Audio Patents from Apple published today Deeply Dive into the science of motion, head tracking & more