Today the U.S. Patent and Trademark Office officially granted Apple a patent that relates to aspects of advanced audio systems used to render spatial audio primarily in headphones and Apple’s future Mixed Reality Headset based on Head-to-Torso Orientation. Though over time, Apple’s advanced Spatial Audio may be available to working with AirPods with Macs, an iPhone and more.

Headphones can reproduce a spatial audio signal communicated by a device to simulate a soundscape around the user. An effective spatial sound reproduction can recreate sounds such that the user perceives the sound as coming from a location within the soundscape external to the user's head, just as the user would experience the sound if encountered in the real world.

When a sound travels to a listener from a surrounding environment in the real world, the sound propagates along a direct path, e.g., through air to the listener's ear canal entrance, and along one or more indirect paths, e.g., by reflecting and diffracting around the listener's head or shoulders. As the sound travels along the indirect paths, artifacts can be introduced into the acoustic signal that the ear canal entrance receives. User-specific artifacts can be incorporated into binaural audio by signal processing algorithms that use spatial audio filters. For example, a head-related transfer function (HRTF) describes how a sound located somewhere in space, relative to a listener's body, is filtered, e.g., reflected or diffracted around the listener's head, torso, and outer ear, before entering their auditory system. The HRTF-described cues allow the auditory system to determine where in space a sound is coming from.

To implement accurate spatial audio reproduction, a virtual audio system can use the HRTF to create the illusion that sound is coming from somewhere in space. More particularly, an HRTF-related audio filter can be applied to an audio input signal to shape the signal in such a way that reproduction of the shaped signal realistically simulates a sound traveling to the user from the relative location at which the HRTF was measured. Accordingly, a listener can use simple stereo headphones to create the illusion of a sound source somewhere in a listening environment by applying the binaural audio filter to the audio input signal.

Existing virtual audio rendering systems are required to know the user's head orientation relative to the virtual sound source in order to select an appropriate head-related transfer function (HRTF). Typically, the HRTF is defined and measured as having a dependence on an azimuth angle, elevation angle, and sometimes a distance between the virtual sound source and the user's head.

Definitions of the HRTF dataset up until now do not encapsulate the dimension related to the orientation of the rest of the body relative to the user's head. More particularly, changes away from a nominal forward-facing head-to-torso orientation are not accounted for when using a HRTF dataset; the torso is assumed to rotate and move with the head. In this way, a user that turns their head to the right while keeping his torso stationary, e.g., facing forward, will have the unsettling experience of hearing sound as though they turned their torso to the right concurrently with his head.

In other words, the virtual audio rendering systems do not differentiate between cases when the head and torso are moved separately and cases when the head and torso are moved together. This disregard for head-to-torso orientations by existing virtual audio rendering systems results in spatial audio renderings that do not accurately reproduce the effects that the torso orientation has on sound sources in real life.

Apple’s granted patent covers a media system and a method of using the media system to accurately reproduce virtual audio taking into account a user's head orientation relative to the user's torso.

The media system includes one or more processors configured to determine a head-to-source orientation and a head-to-torso orientation. The head-to-source orientation can be a relative position and/or orientation between a head of a user and a sound source. The relative orientation can be determined from head tracking data generated by a head tracking device, such as a head mounted device having inertial measurement units.

The head-to-torso orientation can be a relative position and/or orientation between the head of the user and a torso of the user. The relative orientation can be directly measured, e.g., by one or more sensors of the head tracking device or a companion device. Alternatively, the relative orientation can be inferred based only on the head tracking data generated by the head tracking device.

Estimation of the head-to-torso orientation based on the head tracking data can include determining that the torso moves toward alignment with the head when the head orientation data meets a head movement condition.

For example, the torso may move when the head moves. Alternatively, the torso may move when the head has moved and then stopped moving at a new orientation.

In an aspect, the torso may move when the head moves in a particular pattern. In any case, the movement of the torso can be related to the head movement, e.g., numerically through an average or median of head tracking data, or in some other manner, e.g., by moving the torso according to a particular pattern that corresponds to the pattern detected for the head movement.

Apple’s patent FIG. 1 below is a pictorial view of a user positioned relative to a sound source in a soundscape; FIG. 2 is a flowchart of a method of reproducing spatial audio based on a head-to-torso orientation.

Apple’s patent FIG. 4 below is a pictorial view of a head of a user moving relative to a sound source in a soundscape; FIG. 9 is a pictorial view of a head above torso mesh generation used to simulate a head-related transfer function for various combinations of head-to-source and head-to-torso orientations.

For more details, review Apple's granted patent 11,375,333.

Apple was also granted patent 11,375,306 today titled “Headphones with increased back volume” that generally relates to various headphone features. More particularly, the various features help improve the overall user experience by incorporating an array of sensors and new mechanical features into the headphones. For more details, review Apple’s granted patent here.