On December 30, Patently Apple posted a report titled "Apple's Acoustics VP hints that future AirPods Pro may use a Proprietary Technology so as to deliver a superior Spatial Audio Experience." Apple's Acoustics VP, Gary Geaves said during an interview that "Obviously the wireless technology is critical for the content delivery that you talk about, but also things like the amount of latency you get when you move your head, and if that’s too long, between you moving your head and the sound changing or remaining static, it will make you feel quite ill, so we have to concentrate very hard on squeezing the most that we can out of the Bluetooth technology, and there’s a number of tricks we can play to maximize or get around some of the limits of Bluetooth. But it’s fair to say that we would like more bandwidth and… I’ll stop right there."

Today the U.S. Patent and Trademark Office officially granted Apple a patent that relates to an audio source device that transitions between transmitting audio data as an over-the-air optical signal to an earphone, to transmitting the audio data as an over-the-air radio frequency (RF) signal. The verbiage in this granted patent seems to match up with what Gary Geaves was describing that could be coming to a next-gen AirPods Pro (or other wireless earphones).

The patent also illustrates that the technology could be used with Apple's future Mixed Reality Headset and/or smartglasses.

Apple states in their granted patent that wireless communication capabilities are available in a broad array of accessory devices that can be configured to communicate with audio source devices. For example, wireless audio output devices, such as a wireless headset or a pair of wireless earphones, can connect via a wireless personal area network (WPAN) connection to an audio source device, such as a smart phone, in order to receive an audio stream.

In one aspect, the WPAN connection may be via an Advanced Audio Distribution Profile (A2DP) connection or another audio profile connection of a BLUETOOTH communication protocol.

To stream high-quality audio data, the audio source device packetizes the audio data (e.g., partitions the audio data into units for transmission) according to the A2DP profile and stores the packets of audio data in transmit buffers. Packets are then transmitted over an over-the-air (or wireless) radio frequency (RF) signal to the wireless audio output device. The received packets are stored in long buffers in the output device in order to provide continuous audio playback in situations when future packets are dropped (e.g., during transmission interference). Audio data in the buffers are de-packetized and processed for audio output through at least one speaker. This process is repeated while audio output is desired at the audio output device.

Although convenient, wireless communication protocols have latency drawbacks. For instance, streaming audio data through BLUETOOTH may require tens of milliseconds for audio processing to generate an encoded audio packet and up to a few hundred milliseconds of buffering, resulting in over 250 milliseconds of end-to-end latency.

In addition, any error-correction schemes (e.g., forward error correction (FEC) code) that are used to detect errors in packets may add additional latency. This end-to-end latency may not be noticeable by a user for some audio (e.g., high-latency audio), such as a music composition because long buffers in the audio output device may maintain audio playback continuity. For example, upon initiating playback of the musical composition, the audio output device may delay playback to first fill its buffer. Once filed, audio data is retrieved from the buffer for output.

Some audio (e.g., low-latency audio), however, may require a shorter latency. For example, audio associated with a physical action upon the audio source device (e.g., a selection of a user interface (UI) item on a touch screen that provides audible feedback when selected) may require a short audible response time (e.g., less than ten milliseconds). In this way, audio that is time sensitive that is transmitted via BLUETOOTH may be delayed due to the high latency, causing discontinuity between selection of the UI item and the audible feedback. This discontinuity may degrade user experience.

Apple's invention provides a method for low-latency optical audio transmission. For example, audio data may be transmitted over an over-the-air optical signal that is produced (or projected) from an audio source device to an audio output (receiver) device. Unlike wireless protocols that may require audio data be encoded and packetized (e.g., with a header, etc.), audio data transmission via an optical signal may be transmitted as raw digital data (e.g., uncompressed).

This type of transmission may reduce the end-to-end latency considerably, since there is no additional latency due to audio processing (e.g., compression/encoding of the audio source device) at the source device and latency due to audio processing (e.g., decompression) at the audio output device is required.

With the reduction of latency, the output device may output audio feedback nearly instantaneously in response to user interactions with the source device.

In other words, the present granted patent may reduce latency, such that output of audio feedback is perceived by a user to be (practically) instantaneous to when a UI item is selected. More about the differences between wireless protocols and the low-latency optical-audio transmission is described herein.

Apple's patent FIGS. 3A and 3B below illustrate examples of the audio source device; FIG. 7 shows an example of the system that optically transmits audio data through human tissue.

Apple's patent FIG. 5 below illustrates a progression of stages of an audio source device transitioning from transmitting audio data as an optical signal to transmitting the audio data as a wireless RF signal.

For more details, explore Apple's granted patent 11,234,078.

A Few of the Inventors

Jahan Minoo: Director, AirPods HW

Rob Silfvast: Engineering, Technology, Product Development Leadership

Daniel Javaheri-Zadeh: System Technologies Engineer