Apple's Distinguished Engineer Tomlinson Holman who's been with Apple for close to a decade, previously worked at LucasFilm for close to 15 years, was the Chief Scientist at Audyssey Laboratories and was founder of TMH Corporation for 25 years. Holman has provided Apple with technical direction in Audio.

So, with Holman listed as one of the inventors of a new audio patent application about spatial audio, you know that Apple took this very seriously from day one. In fact, it's likely the inspiration behind a host of recent patents on this subject matter and what drove Apple to bring spatial audio to AirPods Pro, next gen over-ear headphones and beyond.

Spatial audio was introduced to Apple fans around the globe at this year's WWDC-20. We specifically covered it our report titled "It was a Historic WWDC Keynote as Apple introduced Macs with 'Apple Silicon' coming to market this Fall & Spatial Audio for AirPods Pro."

Further, Apple's WWDC-20 video session titled "What's new in USD" Apple describes the session in-part this way:

"Discover proposed schema and structure updates to the Universal Scene Description (USD) standard. Learn how you can use Reality Composer to build AR content with interactive properties like anchoring, physics, behaviors, 3D text, and spatial audio that exports to USDZ. And, discover streamlined workflows that help you bring newly-created objects into your app." For more on this, view the full session here.

The video below is set to start when Mary-Ann Ionascu, Apple's Senior Firmware Engineer, revealed Spatial Audio at this year's WWDC that's coming to AirPods Pro.

Spatial Audio didn't come out of thin air, it was a project that began in 2017 with Holman and Apple's other engineers listed on today's patent which also includes: Chris Eubank Engineering Manager, Audio, part of Technology Development Group; Matt Connolly, software Engineer and Stephen Pinto (no LinkedIn profile).

To date, Patently Apple has covered at least four spatial audio related patents as follows:

01: Apple files new 'Spatial Audio' patent for its future HMD and is likely to apply to AirPods Pro and Next-Gen Apple TV

02: (2018) Apple Wins Patent for Advanced Spatial Audio for Beamforming Speakers like HomePod, Home Theaters and beyond

03: (2020) Apple Takes Binaural Audio to the next-level by adding Head-Tracking for Next-Gen AirPods Pro and Future HMD

04: (2019) Apple invents Over-the-Ear Headphones that include Head-Tracking with Automatic Dynamic Re-Centering Operations

Today the US Patent & Trademark Office published a patent application from Apple that relates to the technical field of spatial audio processing in a simulated reality environment.

Apple notes that producing three-dimensional (3D) sound effects in augmented reality (AR), virtual reality (VR), and mixed reality (MR) applications, all of which in this disclosure are encompassed by the term simulated reality (SR), is challenging because existing audio formats were originally designed for producing 3D sound in a physical environment with fixed speaker locations and stationary listeners, such as in a movie theater. Examples of spatial audio formats designed to produce 3D sound include MPEG-H (Moving Picture Experts Group) 3D Audio standards, HOA (Higher-order Ambisonics) spatial audio techniques, and DOLBY ATMOS surround sound technology.

It appears that Apple's patent was/is part of the foundation for Apple's developer tools related to audio. Considering that we learned about spatial audio at Apple's WWDC-20, it's likely part of the developer tools for Apple developers.

Apple's patent reveals that "One alternative for producing 3D sound effects in SR environments is to manipulate individual discrete sounds contained in audio objects that can be virtually located anywhere in the 3D environment. Composing audio for SR applications using existing spatial audio formats and objects is difficult since there is no uniform way to access a variety of sound sources and incorporate them into a dynamic SR environment.

Embodiments of a file format for spatial audio as herein described enable augmented SR application developers to compose sounds for use in SR applications in which the audio data that is encoding a sound capable of being composed into a SR application is stored as an audio asset including asset metadata describing not only how the sound was encoded but also how a listener in a SR environment that has spatial audio rendering capability experiences the sound."

Further, Apple notes that the SR developer can retrieve the audio asset during development, preview how the listener would experience the sound during playback, attach the audio asset to a SR object associated with experiencing the sound in the SR environment, and edit the asset metadata until the desired 3D sound effects that affect how the listener experiences the sound are achieved.

In one embodiment, audio assets are created and stored in an audio asset library from which the audio asset can be retrieved for composing into the SR application. The composition of the application may encompass attaching the audio asset to the SR object associated with experiencing the sound in the SR environment.

In one embodiment, how the listener experiences the sound in the SR environment depends upon how 3D sound, also referred to as spatial sound, is simulated in a sound playback system, including sounds systems that use binaural rendering (e.g., through headphones), rendering for a loudspeaker-only based playback system, or rendering for a combination headphone-loudspeaker system. Embodiments of the file format for spatial audio described here support a variety of sound sources and audio encodings for reproducing sound to simulate spatial audio in sound playback systems, including single listener playback systems using binaural rendering.

Apple's patent FIG. 1 below is a block diagram illustrating an overview of an audio asset library for use in composing sound for SR environments; FIG. 4 is a block diagram illustrating further details of the audio asset library for use in composing sound for SR environments.

Apple's patent FIG. 5 above is an illustration of an exemplary ingestion, composition and experience of sound processing of audio assets. 

Developers and super geeks could dive deeper into Apple's patent application 20200288258 here. Considering that this is about a spatial audio file format that was likely revealed during this year's WWDC-20 training session on spatial audio, this is likely a patent fulfilled.