The U.S. Patent and Trademark Office officially published a series of 61 newly granted patents for Apple Inc. today. In this particular report we cover Apple's patent titled "System to move a virtual sound away from a listener using a crosstalk canceller." The patent covers an audio processing system for a future MacBook Pro with one or more processors that process an audio signal that is split into at least three paths. Audio signal processing for virtual acoustics can greatly enhance a movie, a sports event, a video game or other screen viewing experience, adding to the feeling of "being there".

In Apple's patent background they note that a virtual acoustic system is one that gives the user the illusion that sound is emanating from elsewhere in an indoor or outdoor space than directly from a loudspeaker (e.g., one that is placed in a room, one that is built into a laptop computer, etc. Audio signal processing for virtual acoustics can greatly enhance a movie, a sports even, a video game or other screen viewing experience, adding to the feeling of "being there."

Various known audio processing algorithms, executed by digital processors, modify one or more recorded, synthesized, mixed or otherwise produced digital audio signals in such a way as to position a virtual source according to modeling that is based on human perception of sound, including the role of ear acoustics, other reflecting and absorbing surfaces, distance and angle of source, and other factors.

In the case of headphones, specially processed audio signals (binaural rendering) are sent to left and right ears of a listener without the crosstalk that is inevitably received by the ears when listening to stereo loudspeakers.

For viewers and listeners that prefer loudspeakers, for example those that may be built into a laptop computer, a crosstalk canceller is employed in some virtual acoustic systems to produce sounds from multiple loudspeakers in such a way that for example a "left" audio signal is predominantly heard only at the left ear of the listener, and a "right" audio signal is predominantly heard only at the right ear of the listener (by virtue of sound wave cancellation in the air surrounding the listener.) This allows the left and right audio signals to contain spatial cues that enable a virtual sound to be "positioned" at a desired location between the loudspeakers.

Apple's invention covers an audio processing system with one or more processors that process an audio signal that is split into at least three paths (versus simple left and right speakers).  The first path has a direct gain and a direct virtual source algorithm operating on the audio signal.

Some versions of the audio processing system have no crosstalk canceling on the first path, while other versions have a first crosstalk canceller on the first path.

The second path has a number of early reflection gains that are applied to the audio signal, which produces multiple early reflections, respectively. In addition, each of these early reflections undergoes a delay, and the early reflections are processed by an early reflections virtual source algorithm. Some versions of the audio processing system have no crosstalk canceling on the second path, while other versions have a second crosstalk canceller on the second path.

The third path has a reverberation gain and binaural reverberation filters operating on the audio signal. The third path has a crosstalk canceller, which may be termed a third crosstalk canceller.

A mixer combines left and right channel outputs of each of the first path, second path and third path. The mixer thus produces a left loudspeaker signal and a right loudspeaker signal.

Another aspect of the invention is a digital processor-based method for processing an audio signal, for example in preparation for playback through a left loudspeaker and a right loudspeaker. The audio signal represents a virtual sound source.

The audio signal is split to a first processing path, a second processing path and a third processing path. On the first processing path, the audio signal is operated on, with a direct gain and a direct virtual source algorithm. On the second processing path, the audio signal is operated on with a plurality of early reflection gains, which produces a plurality of early reflections, respectively. Each of the early reflections is subjected to a delay; the early reflections are also processed by an early reflections virtual source algorithm.

On the third processing path, the audio signal is operated on, with a reverb gain and binaural reverb filters, and crosstalk canceling. The left and right channel outputs of each of the first, second and third processing paths are combined, to produce a left loudspeaker signal and a right loudspeaker signal.

Apple's patent FIG. 1 below depicts a crosstalk canceller, in a MacBook (#102), processing an audio signal to move a virtual source of sound to one side of the loudspeaker of the MacBook, as perceived by a listener; FIG. 2 depicts a different use of a crosstalk canceller, in an aspect of the present disclosure that is processing an audio signal to move a virtual source farther away from, and behind, the loudspeaker of a laptop computer, as perceived by a listener.

Apple's patent FIG. 3 above depicts three components of sound reaching a listener from a sound source, in this example a loudspeaker, the three components being direct sound, early reflections and reverberation (late reverb.); and FIG. 4 depicts a virtual direct source moved backwards, relative to the loudspeakers of a MacBook and the listener, and virtual early reflections of sound.

Apple's granted patent 10,524,080 was originally filed in Q2 2018 and published today by the US Patent and Trademark Office.