Apple's Patent Background

Apple's patent background states that many modern electronic devices implement a wide variety of input and/or output (I/O) devices within a single housing to provide an enhanced user experience. The user experience may include functionality provided by the I/O devices, as well as an appearance of the device. Apertures formed in the housing of the device to allow sound waves to impact microphone diaphragms may detract from the visual appeal of the housing.

The positioning of microphones within a housing can in part determine the effectiveness of the microphones. For example, if microphones are positioned near noisy components such as near a keyboard or a central processing unit (CPU) or fan, the noise may make it difficult to discern other sounds, such as a user's speech. Additionally, microphones are generally more effective when they are near and/or aligned with the origin of the sounds that they are intended to detect.

Apple's Future Audio Detection System

In one embodiment, Apple's invention may take on the form of an audio detection system having a display assembly. The display assembly may include a screen and at least one electromagnetic energy emitter configured to direct energy at an inside surface of the screen.

The described invention states that at least one sensor could be configured to sense the emitted energy after it is reflected from the inside surface of the screen and generate electrical signals corresponding the sensed reflected energy. A processor coupled to the at least one sensor generates an audio signal representative of sound waves that impact an outer surface of the screen.

In a second embodiment, Apple's invention may take the form of a computer system having a display that includes a screen having an interior surface and an exterior surface. The exterior surface is visible to a user. One or more sensors are coupled to the display and configured to detect vibrations of the screen generated by sound waves impacting the exterior surface of the screen. A processor in communication with the one or more sensors is configured to generate an output representative of sound waves.

In a third embodiment, Apple's invention may include a method of operating a computing device. The method includes obtaining an electrical signal corresponding to vibration of a screen of the computing device resulting from sound waves impacting the screen and filtering the signal to remove noise components. The method also includes generating an output signal representative of the sound waves that impacted the screen.

Apple's Invention could apply to Displays Far beyond Traditional Devices

Although Apple's patent FIG. 1 noted below illustrates a notebook, the present techniques may be applied to all suitable computing devices including desktop computers, tablets, mobile phones, smart phones, and so forth. Moreover, the present techniques may find application in other areas as well. For example, applications may be found in vehicles, helmets with heads-up displays, and/or televisions. As such, the specific embodiments discussed herein are not intended to be limiting.

Possible Application for Vehicles

Apple notes that in one embodiment, the display may act as the diaphragm and the sensors may be configured to sense and interpret voice commands while driving. Further, using the spatial determination capabilities may allow for commands from a passenger to be directed to the passenger side of the vehicle, while commands from a driver may be applied to the entire vehicle or the driver's side only, as the case may be. This may be applied, for example, to voice control of a climate control system of the vehicle.

Possible Application for Heads-Up Displays

Another example implementation may be in a helmet with a heads-up display. Here, a visor may be used as a diaphragm in conjunction with the direct or indirect sensors for sensing and interpreting the user's discussion and/or voice commands.

Possible Application for a Future Television

In yet another example implementation, Apple states that television screen may be used as a diaphragm for sound sensing. In one embodiment, the sound sensing may be used in a feedback loop to adjust the volume of the television set when someone is talking or when there is high volume of ambient noise. For example, the television may turn down its volume when someone is talking and may increase its volume when there is a high level of ambient noise.

About the Emitters

The computing device will generally include one or more emitters 114, such as lasers, LEDs, radio frequency (RF) emitters and so forth that may be implemented, along with appropriate sensors 116, to detect sounds. A digital signal processor (DSP) 118 may be coupled to the sensors 116 to process the sensor output and recreate sounds from the modulated light sensed by the sensors. The emitters may be controlled by the processor.

Apple notes that the emitters may emit a carrier signal of any suitable electromagnetic frequency. In some embodiments, they may be light emitters. That is, in some embodiments, the emitters may operate in a visible or non-visible range of the electromagnetic spectrum, such as in the near infrared (IR) band, or the IR band.

In still other embodiments, the emitters may operate in other regions of the electromagnetic spectrum, such as the radio frequency (RF) band of the spectrum, for example. In some embodiments, a coating or film may be provided on an inside surface of the display screen 102 that allows for passage of visible light and reflects other frequency ranges (e.g., reflects RF, near IR, and so forth).

Apple's patent FIG. 3 is a partial cross-sectional view of the display taken along line III-III of FIG. 1 showing the Z-stack for the display. The Z-stack may include the screen and a light source 120 that projects light towards the screen to illuminate the one or more pixels of the display. Generally, the emitters and sensors are obscured by the bezel (# 104 of FIG. 1) or are located behind the Z-stack of the display. As such, the emitters and sensors are generally not visible to a user of the computing device

Audio Sensors & Emitters Built into Various Kinds of Displays

In Apple's patent FIG. 6 noted below we see an array (#140) of sensors 116 receiving light reflected from the screen. In particular, the emitters (#114) are configured to direct light through gaps in between the pixels (not shown) which is reflected back to the sensors. Generally, light reflected from the center of the screen or near the center of the screen will provide a better signal to noise ratio when compared to those from the edges of the screen. In some embodiments, a dead pixel may be provided to allow electromagnetic energy emitted from the emitters to reach the screen unobstructed.

Apple's patent FIG. 7 noted above illustrates an embodiment having arrays of sensors 142, 144, 146, 148 about the perimeter of the screen. The output of each of the sensors in the array may be provided to the DSP (#118 of FIG. 1) for processing of the received light signals. The processing may include computing a reverse phase array to retrieve lower frequency components of the reflected signal, as well as beamforming to provide directional selectivity.

Patent Credits

Apple credits Aleksandar Pance, Brett Bilbrey, Eric Smith, and Jahan Minoo as the inventors of this patent which was originally filed under serial number 153990 in Q2, 2011. 

NOTICE: Patently Apple presents a detailed summary of patent applications with associated graphics for journalistic news purposes as each such patent application is revealed by the U.S. Patent & Trade Office. Readers are cautioned that the full text of any patent application should be read in its entirety for full and accurate details. Revelations found in patent applications shouldn't be interpreted as rumor or fast-tracked according to rumor timetables. About Comments: Patently Apple reserves the right to post, dismiss or edit comments.