On Thursday the US Patent & Trademark Office published a patent application from Apple that relates to user posture transition detection and classification using a linked biomechanical model. The invention more specifically relates to spatial audio in AirPods. This new system will ensure that spatial audio is enabled when the user is sitting down watching online content and automatically disable spatial audio when the user stands and/or walks away from the viewable content.

Apple notes that in a spatial audio embodiments, motion data from the headset (AirPods/AirPods Max) and source device motion are fused in the source device, such that relative position and altitude can be calculated by the source device for use in head pose tracking for certain tracking scenarios to ensure that the spatial audio sound bed is properly centered.

In particular, head pose tracking using relative motion is useful for determining whether the user's head is turning while their torso is stationary or whether the user's torso has moved. Both types of motion would be sensed by the headset and would be indistinguishable. Generally, the use of motion data from both the headset and companion device allows more complex user posture transitions to be detected.

In spatial audio applications, it is desirable to provide a natural and immediate re-enablement of head pose tracking after it is disabled due to, for example, distracted user viewing. This requires that head pose tracking be automatically enabled and re-enabled based on the user's posture (e.g., sitting, standing). 

For example, while the user is sitting, head pose tracking is enabled (FIG. 1, #101), allowing the user to enjoy an immersive audio experience while watching the content displayed on the source device. In an embodiment, system 100 determines that the user is sitting based on a linked biomechanical model and motion data (e.g., acceleration data), as described in reference to FIGS. 3 and 4.

(Click on image to Enlarge)

If the user transitions from a sitting posture to a standing posture and walks away from the source device, the transition is detected by the system (#100 using the linked biomechanical model and motion data), causing head pose tracking to be disabled #102. In an embodiment, head pose tracking is disabled #102 if the system detects a “sit-to-stand” posture transition and also detects that the user is walking based on the motion data. In an embodiment, the spatial audio processing is disabled if walking is detected.

Apple's patent FIG. 3 below illustrates a linked biomechanical model for the biomechanics; FIG. 4A is a block diagram of a system for detecting and classifying posture transitions using the linked biomechanical model described in FIG. 3.  

Apple's patent FIG. 6 above is a flow diagram of a process of posture transition detection and classification using a linked biomechanical model.

It's pretty clear by this in-depth patent that Apple takes Spatial Audio seriously. To the review the finer points, review Apple's patent application number US 20230100254.

Apple Inventors

• Aditya Sarathy: Senior Scientist
• Xiaoyuan Tu: Distinguished Technologist, Lead Scientist & Sensor Engineer, Motion and Location Technology
• Suresh Malakar: Software Engineer
• Linhui (Lynn) Yu: Health Sensing Hardware Engineer