Today the U.S. Patent and Trademark Office officially published a series of 61 newly granted utility patents for Apple Inc. In this particular report we briefly cover spatial audio for Vision Pro, flexible & foldable displays, AirTags and more. And, as always, we wrap up this week's granted patent report with our traditional listing of the remaining granted patents that were issued to Apple this week.

Generating Spatial Procedural Audio   

Apple's granted patent relates to procedural audio that's important for generating realistic sound components for many applications, such as for video games, extended reality (XR) environments (e.g., virtual reality (VR), mixed reality (MR), etc.), as well as comfort noises and soundscapes.

The main benefit of procedural audio compared to using libraries of pre-recorded materials, is that the procedural audio may be generated on-the-fly, therefore ensuring that the audio is unique for a given situation. Traditional procedural audio techniques have many disadvantages. Computer applications that implement these techniques rely on both physical characteristics of an object (e.g., contained within image data that is captured by one or more cameras) for which procedural audio is to be produced and semi-empirical models of objects that emit the desired sound. For example, to produce sounds of birds chirping, applications require input video that includes birds, as well as the empirical models that represent sounds (e.g., chirps) produced by those particular birds. In addition, such applications may not produce the audio with sufficient realism.

Apple's granted patent solves these deficiencies by providing a method performed by an audio system for generating realistic audio effects and ambiences using machine learning (ML) models. For instance, a ML model (e.g., a Deep Neural Network (DNN)), may be a generative model (such as a Generative Adversarial Network (GAN)) that is configured to generate spatial procedural audio output in response to input. In particular, a GAN may be used that has an input based on an (first) input noise signal (e.g., a signal that contains noise such as white noise or colored noise) to generate a mono audio signal that includes a sound produced by an object (e.g., a door slam) and a spatial parameter for the mono audio signal. The spatial parameter may include a Direction of Arrival (DOA) of the sound and/or a diffuseness of the sound. In the case of the door slam, the DOA may indicate a location from which the sound is to originate (e.g., within an XR environment). The mono audio signal may be spatially encoded according to the spatial parameter to generate spatial audio data for the sound.

For instance, the spatial audio data may include multichannel audio in a surround sound multi-channel format (e.g., 5.1, 7.1, etc.), a Vector-Based Amplitude Panning (VBAP) representation of the sound, a Higher Order Ambisonics (HOA) representation of a sound space that includes the sound, etc. As a result, the spatial audio data that is generated from the use of the DNN may be realistic and have unique context-dependent spatial characteristics of the audio (e.g., the door slam coming from one point in space) according to the generated spatial parameter. In this way, the granted patent invention generates the spatial audio data by performing audio signal processing operations only, rather than needing to rely on image (or video) data and semi-empirical models as in traditional techniques. Accordingly, the present disclosure uses the DNN to “imagine” (e.g., in real-time) how certain audio objects or soundscapes sound like (signal-wise), as well as imagine the overall resulting sound field including most (or all) spatial information.

Apple's patent FIG. 1 below shows a block diagram of an audio system for generating spatial audio data; FIG. 4 shows a block diagram of an audio system for training a ML model for generating spatial audio data.

For more, review Apple's granted patent 11997463.

Flexible and Foldable Displays

One of Apple's original patents relating to flexible and foldable displays first surfaced in 2011 and has now been updated under their eighth granted patent 11994906. We originally covered this invention in 2013. Apple has added 19 new patent claims covering a wide range of additions including the addition of a flexible battery having cylindrical battery cells.

Other Patents of Interest

11992116: (Original AirTags Master Patent) Wirelessly Coupled Accessory System For An Electronic Device. 20 new patent claims.

11997836: Wireless Charging Performance Improvements (22 new patent claims.)

11995253: Systems With Deployable Controllers And User Identification (Related to Project titan))

11993118: Motion Control System With Electropermanent Magnets

This Week's Remaining Granted Patents