Today, the last granted patent day in 2022, the U.S. Patent and Trademark Office officially published a series of 62 newly granted patents for Apple Inc. In this particular report we cover a key Spatial Audio patent and links to four camera patents. And as always, we wrap up this week's granted patent report with our traditional listing of the remaining granted patents that were issued to Apple today.

 

Spatial Audio Downmixing

Apple was granted a patent today that relates to computerized data processing systems and methods for audio processing, and in particular to spatial audio processing.

More specifically, the patent covers 'Spatial Audio Downmixing' and the hardware it will be available on. Spatial Audio will be supported on future HDTVs and Streaming Services like Apple TV+ via MPEG-H and/or 'Higher-Order Ambisonics.'

Apple's patent background notes that producing three-dimensional (3D) sound effects in augmented reality (AR), virtual reality (VR), and mixed reality (MR) encompassed by the term "simulated reality" or SR, applications is commonly used to enhance media content.

For example, sound designers add 3D sound effects by manipulating sounds contained in spatial audio objects to enhance a scene in an SR application, where the sounds are ambient sounds and/or discrete sounds that can be virtually located for playback by the spatial audio system anywhere in the virtual 3D space created by the SR application.

A head mounted system may be designed to receive an external display (e.g., a smartphone). The head mounted system may have imaging sensor(s) and/or microphones for taking images/video and/or capturing audio of the physical setting, respectively.

Other examples of SR systems include heads up displays, automotive windshields with the ability to display graphics, windows with the ability to display graphics, lenses with the ability to display graphics, headphones or earphones, speaker arrangements, input mechanisms (e.g., controllers having or not having haptic feedback), tablets, smartphones, and desktop or laptop computers.

In one embodiment the audio data in which the sound is recorded or encoded is stored as a spatial audio object that preserves spatial characteristics of one or more recorded sounds.

In another embodiment, the spatial audio object contains several channels of audio data representing the one or more recorded sounds, each channel being associated with any one or more of a direction and a location (distance), e.g. of a source of the recorded sound.

Apple's patent FIG. 4 illustrates the spatial sound preview process in further detail. In one embodiment, a composed spatial audio object #404, such as the combined forest/waterfall ambient sounds that is presented in FIG. 1 further below.

(Click on image to Enlarge)

Further to FIG. 4, Apple notes that the spatial sound preview user interface #406 generates (operation #408) a visualized spatial sound object #206 (see FIG. 2 below), such as a virtual globe (e.g., sphere, bubble, cube, polyhedron, etc.) in response to a request (e.g., from the user) to preview the sound represented in the composed spatial audio object #404.

In the example of FIG. 4, the object is a virtual sphere having a central origin from which all of the spatial sounds represented by the different triangles will emanate.

In other words, each triangle may represent a loudspeaker (acoustic output transducer) that is pointed outward and placed at the same location (the central origin of the virtual sphere).

Apple's patent FIG. 1 below is a block diagram illustrating an overview of spatial sound use in SR environments; FIG. 2 is a block diagram illustrating a spatial sound preview.

For more details, review Apple's granted patent US 11540081 B2.

Apple Inventors

Chris Eubank: Engineering Manager, Audio

Matt Connolly: Software Engineer (background in audio)

Stephen Pinto: No LinkedIn Profile was found

Camera Related Patents

Apple was also granted a series of four camera related patents today. Links to these patents are presented below:  

One: Folded Optics Camera And Actuator Assembly

Two: Folded Camera

Apple's patent FIG. 1 below representing the '864 patent illustrates a simplified view of a folded optics arrangement camera and shows how light is bent within the folded optics; Apple's patent FIG. 6 below representing patent '936 illustrates a cross-sectional illustration of a folded camera with five refractive lens elements in the lens system.

Three: Scene Camera

Four: Camera With Low-profile Actuator Arrangement

Today’s Remaining Granted Patents