Today the U.S. Patent and Trademark Office officially published a patent application from Apple that generally relates to posture awareness and, in particular, to systems, devices, and methods for tiered posture awareness. Posture affects how 3D content is presented to the user as well as changes audio volume of spatial audio.   

Apple notes in their patent background that many people may spend a significant number of hours at their computers or other devices during both work and non-work hours. This time spent using a computer or other devices may negatively impact the posture of said person.

Method And Device For Tiered Posture Awareness

Various implementations disclosed in Apple's patent filing include devices, systems, and methods for tiered posture awareness. According to some implementations, the method is performed at a computing system including non-transitory memory and one or more processors, wherein the computing system is communicatively coupled to a display device, such as an HMD or Smartglasses, and one or more input devices.

The method includes: while presenting a three-dimensional (3D) environment, via the display device, obtaining head pose information for a user associated with the computing system; determining an accumulated strain value for the user based on the head pose information; and in accordance with a determination that the accumulated strain value for the user exceeds a first posture awareness threshold: determining a location for virtual content based on a height value associated with the user and a depth value associated with the 3D environment; and presenting, via the display device, the virtual content at the determined location while continuing to present the 3D environment via the display device.

In some implementations, a display corresponds to a transparent lens, and the electronic device corresponds to a pair of glasses worn by the user. As such, in some implementations, the glasses present a user interface by projecting the XR content the display, which is, in turn, is overlaid on the user's physical environment. In some implementations, the user wears the electronic device such as a near-eye system. In such implementations, the glasses present the XR environment by displaying data corresponding to the XR environment on the one or more displays (on the glasses) or by projecting data corresponding to the XR environment onto the user's retinas.

In some implementations, the controller (#110 FIG. 2 below) and/or the glasses cause an XR representation of the user to move within the XR environment based on movement information (e.g., body pose data, eye tracking data, hand/limb/finger/extremity tracking data, etc.) from glasses (or HMD) and/or optional remote input devices within the physical environment.

Apple's patent FIG. 2 below is a block diagram of an example controller.

As shown in patent FIG. 4D below, the current strain information #480 includes: muscle information #482A associated with a first muscle or muscle group/region; muscle information #482B associated with a second muscle or muscle group/region; muscle information #482C associated with a third muscle or muscle group/region; muscle information #482D associated with a fourth muscle or muscle group/region; muscle information #482E associated with a fifth muscle or muscle group/region; and current accumulated strain information #486.

To that end, in various implementations, the muscle strain engine #463 includes a head/body/neck mechanics engine #462 and a strain analyzer #464 with strain increase logic #465A and strain decrease logic #465B.

The muscle strain engine #463 determines current muscle strain values (#'s 485A, 485B, 485C, 485D, and 485E) for muscles or muscle groups/regions #484A-E respectively, of the user.  Furthermore, the muscle strain engine updates (increases or decreases) the muscle strain values 485A-E over time based on rotational and/or translational movement of the user #150 that triggers the strain increase logic #465A and/or the strain decrease logic #465B.

In some implementations, a posture awareness engine #468A is configured to determine/generate tiered posture awareness feedback #469.

The tiered posture awareness system communicates with the smartglasses or HMD so that when a user is titling their head, the content will be reflected on their device screen accordingly. Secondly, the smartglasses or HMD system may communicate with the user if their posture could affect their health.

On the latter point, Apple notes that "in accordance with a determination that a current accumulated strain value exceeds the third posture awareness threshold the electronic device presents, via the display, an alert notification indicating that the user has been overstraining their muscles while presenting a 3D environment.

In some implementations, the electronic device 120 presents the alert notification at a default location or in a default manner within the 3D environment (e.g., a pop-up notification centered within the display, a banner notification adjacent to the top edge of the display or the like."  

Another point Apple makes states: "According to some implementations, one or more attributes of the spatial audio effect (e.g., the volume, the distance relative to the user, and/or the like) are based on the magnitude of the current accumulated strain value. As such, in one example, a volume of the spatial audio effect increases as the accumulated neck strain increases, and the volume of the spatial audio effect decreases as the accumulated neck strain decreases. As another example, a distance of the spatial audio effect decreases relative to the user as the accumulated neck strain increases, and the distance of the spatial audio effect increases relative to the user as the accumulated neck strain decreases."

Apple's patent FIG. 4A below is a block diagram of a first portion of a data processing architecture; FIG. 4B illustrates example data structures.

Apple's patent FIG. 4C below is a block diagram of a second portion of a data processing architecture.

If anything, the depth of this patent is mind boggling. Unless you're an engineer, high-end developer or a code reader like the one in the movie the Matrix, you'll certainly fall into a rabbit hole and get lost. For those wanting to review this in-depth patent in detail, review Apple's patent application 20240023830.