Today the U.S. Patent and Trademark Office officially granted Apple a patent that relates to spatial audio / 3D audio.

Apple's Spatial Audio or 3D Audio granted patent is one that is geared towards developers using Apple's new "Audio Engine" designed to deliver excitingly realistic audio in future games, especially VR games where gamers are immersed in simulated environments where rushing-water or waterfalls are able to pop out in a whole new audio dimension.

Today's granted patent on spatial audio patent is a little different from most on this topic, as it's aimed mainly at tools for developers. More specifically, an "Audio Engine" for gaming developers who are designing Simulated Reality games that could be AR/MR/VR based.

More specifically, Apple notes in patent point #75: "In one or more embodiments, a graphic rendering module is a software program (application) that allows a developer of higher layer applications (e.g., games) to define a spatial representation of the objects, called out in the higher layer application, in a graphical scene (and is responsible for rendering or drawing the visual aspects of 3D or 2D graphic objects in the virtual environment that is being displayed (projected).

In one or more embodiments, such a framework may include geometry objects that represent a piece of geometry in the scene, camera objects that represent points of view, and light objects that represent light sources. The graphic rendering module may include a rendering API like Direct3D, OpenGL or others that have a software abstraction layer for the GPU."

Apple further notes that "an audio engine" is provided for acoustically rendering a three-dimensional virtual environment. The audio engine uses geometric volumes to represent sound sources and any sound occluders. A volumetric response is generated based on sound projected from a volumetric sound source to a listener, taking into consideration any volumetric occluders in-between. The audio engine also provides for modification of a level of detail of sound over time based on distance between a listener and a sound source.

Apple's patent FIG. 2 below illustrates a representational view for explaining an example 3D virtual audio environment; FIG. 3A illustrates representational views for explaining example objects having geometric volumes; FIG. 3B illustrates a representational view for explaining an example object having simplified bounding volumes.

In the example of Apple's patent FIG. 2 above, the virtual scene in a game includes two listeners (e.g., listener #212A, listener #212B) and two objects that may be displayed, namely a river (#210) and a house (#215).

The river is an object that is a sound source, which is represented volumetrically in the 3D virtual environment. Consequently, instead of being configured to produce sound from one or more discrete points that constitute the object (as in FIG. 1), the river is said to produce sound from the surface of the entire volume of the shape that represents it. The house is a sound occluding object that occludes sounds from the river to the listener (#212A). The house is also represented volumetrically in the 3D virtual environment.

However, a sound producing object such as a river may also have a more complicated shape. FIG. 3B illustrates an example of a river have a winding path. In one aspect, several simplified bounding volumes (e.g., 36, 37, 38, 39) are determined that as a whole or together represent the geometric volume of a river. Although the volumes 36-39 are not shown as overlapping in FIG. 2, these volumes may in some instances overlap.

With AirPods Pro building in Spatial Audio, future gaming will sound more realistic than ever imagined and only get better when Apple delivers their Video Headset in the not-too-distant future. 

Apple's patent FIG. 7 below illustrates an example system for performing 3D audio rendering.

For more details, review Apple's granted patent 11,146,905.