On May 27, 2010, the US Patent & Trademark Office published two distinct Apple patent applications. The first covers a new wireless earphone-microphone device for use with an iPhone and/or the 3G based iPad. The second patent covers advancing iPhone's audio quality by using audio beamforming with input from a microphone array. This will greatly enhance voice clarity and voice recording of events, for example.
Earphone – Microphone Assembly Patent
Apple's patent is about an earphone that is combined or integrated with a microphone to form a headset that could be used for two-way communications with an iPhone or even a computer if used with voice over IP (Internet Protocol) software. The headset could communicate with the host device through either a wired connection or a wireless link.
Apple's patent presents both a method and system for detecting the repositioning of an earphone assembly. Apple's proposed system comprises an earphone assembly (earphone) that is to be coupled to a host. Within the earphone is a pressure transducer that detects a pressure change caused by the repositioning of the earphone against the user's ear. Upon detection of the pressure change, the pressure transducer transmits a signal to a signaling mechanism that is also outside the host. The signaling mechanism sends a repositioning detection signal to the host in response to the signal from the pressure transducer. The repositioning detection signal may be superimposed on a microphone output audio signal when it is transmitted to the host.
The iPod, iPad Scenario: Upon receipt of the repositioning detecting signal, the host changes its operation mode. In one scenario, the host may include an iPod or iPad that is connected to the earphone and is playing music through the earphone. The player pauses music playing when the repositioning detecting signal indicates removal of the earphone from the user's ear. The media player may automatically resume music playing when the repositioning detecting signal indicates that the earphone has been re-inserted into the user's ear or is otherwise "at the ear."
The iPhone Scenario: In another scenario, the host may include a telephone module which automatically switches to speakerphone mode when the repositioning detecting signal indicates the removal of the earphone from the user's ear. The telephone module may switch back to receiver mode (handset mode) when the repositioning detecting signal indicates that the earphone has resumed its at-the-ear position.
Apple credits Jae Lee and Wendell Sander as the inventors of patent application 20100128887, originally filed in Q4 2008.
Second Patent: Stabilizing Directional Audio Input from a Moving Microphone Array
Under typical imperfect conditions, a single microphone that is embedded in a mobile device does a poor job of capturing sound because of background sounds that are captured along with the sound of interest. An array of microphones can do a better job of isolating a sound source and rejecting ambient noise and reverberation.
Beamforming is a way of combining sounds from two or more microphones that allows preferential capture of sounds coming from certain directions. In a delay-and-sum beamformer sounds from each microphone are delayed relative to sounds from the other microphones, and the delayed signals are added. The amount of delay determines the beam angle--the angle in which the array preferentially "listens." When a sound arrives from this angle, the sound signals from the multiple phones are added constructively. The resulting sum is stronger, and the sound is received relatively well. When a sound arrives from another angle, the delayed signals from the various microphones add destructively--with positive and negative parts of the sound waves canceling out to some degree--and the sum is not as loud as an equivalent sound arriving from the beam angle.
For example, if the sound comes into the microphone on the right before it enters the microphone on the left, then you know the sound source is to the right of the microphone array. During sound capturing, the microphone array processor can aim a capturing beam in the direction of the sound source. Beamforming allows a microphone array to simulate a highly directional microphone pointing toward the sound source. The directivity of the microphone array reduces the amount of captured ambient noises and reverberated sound as compared to a single microphone. This may provide a clearer representation of a speaker's voice.
A beamforming microphone array may be made up of distributed omnidirectional microphones linked to a processor that combines the several inputs into an output with a coherent form. Arrays may be formed using numbers of closely spaced microphones. Given a fixed physical relationship in space between the different individual microphone transducer array elements, simultaneous digital signal processor (DSP) processing of the signals from each of the individual microphones in the array can create one or more "virtual" microphones. Different algorithms permit the creation of virtual microphones with extremely complex virtual polar patterns and even the possibility to steer the individual lobes of the virtual microphones patterns so as to home-in-on, or to reject, particular sources of sound. Beamforming techniques, however, rely on knowledge of the location of the sound source. Therefore it is necessary to aim the beamforming at the intended sound source to benefit from the use of a microphone array. It would be desirable to maintain the aim of the beamforming when the microphone array is part of a mobile device.
Apple's Summary
A device includes a microphone array fixed to the device. A signal processor produces an audio output using audio beamforming with input from the microphone array. The signal processor aims the beamforming in a selected direction. An orientation sensor--such as a compass, an accelerometer, or an inertial sensor--is coupled to the signal processor. The orientation sensor detects a change in the orientation of the microphone array and provides an orientation signal to the signal processor for adjusting the aim of the beamforming to maintain the selected direction. The device may include a camera that captures an image. An image processor may identify an audio source in the image and provide a signal adjusting the selected direction to follow the audio source. The image processor may receive the orientation signal and adjust the image for changes in the orientation of the camera before tracking movement of the audio source.
Apple's patent FIG. 1A is a block diagram of a mobile device 10 to provide an audio output. The device may be an audio recorder, a cellular telephone, or other device for capturing an audio source and providing an audio output. A microphone array 12, 14 is fixed to the device. The microphone array includes at least two individual microphones. A signal processor 24 is coupled to the microphone array to produce the audio output using audio beamforming with input from the microphone array. The signal processor aims the audio beamforming in a selected direction, as suggested by the dashed line directed toward a speaker 30. The audio beamforming provides a directional audio sensitivity so that the sound from the speaker at whom the audio beamforming is aimed is emphasized. Other sound, such as a conversation between people 32, 34 adjacent the speaker, may be attenuated by the beamforming.
Apple's patent FIG. 1B shows the mobile device 10 having moved relative to the speaker 30. The device includes an orientation sensor 22 coupled to the signal processor. The orientation sensor detects a change in the orientation of the device and hence the microphone array since the orientation sensor and the microphone array are both fixed to the device so that they all move in unison. The orientation sensor provides an orientation signal to the signal processor for adjusting the aim of the audio beamforming to maintain the selected direction of the beamforming despite the movement of the device.
Apple credits Shaohai Chen, Phillip Tamchina and Jae Han Lee as the inventors of patent application 20100128892, originally filed in Q4 2008.
Notice: Patently Apple presents only a brief summary of patents with associated graphic(s) for journalistic news purposes as each such patent application is revealed by the U.S. Patent & Trade Office. Readers are cautioned that the full text of any patent application should be read in its entirety for further details. For additional information on any patent reviewed here today, simply feed the individual patent number(s) noted in this report into this search engine. About Comments: Patently Apple reserves the right to post, dismiss or edit comments.
Social Networking Notes
To our Twitter Fans: Patently Apple is now on Twitter. 1) You could access the subscription link to our Twitter site by clicking on the corresponding sidebar button. Once on our Twitter site, look for our RSS link in the right side bar. 2) We've added Twitter's "Tweet This" button to our post footer for your convenience.
To our Facebook Fans: Patently has added Facebook's "Like" button to our sidebar. The "Like" button works very much like a Patently Apple RSS feed for Facebook Fans. By clicking on our "Like" button, you'll automatically establish a link that will send all of our new reports to your newsfeed.
To our Digg Fans: Patently Apple has just added the "Digg This" button to our post footer for your convenience.
This ties in elegantly with the 2nd (assumed) microphone beside the headphone socket on the iPhone 4G/HD. We assumed it was only for noise cancellation, but would double nicely for this.
iPhone OS 4 also has support for stereophonic panning which could implicate a stereo recording function when doing video capture.
nice.
Posted by: Michael | May 27, 2010 at 01:19 PM