Apple Talks More about Reconfigurable MacBooks & Air Controls
Apple in the Sky with Diamonds: A Cloud Based Safe Deposit Box

Future Apple Notebooks & Tablets may use Acoustic Commands

1 - cover - future apple notebooks and tablets may use acoustic commands - feb 2011 
Yesterday we presented you with two of Apple's most detailed patent applications covering Apple's extraordinary research into the notion of live and reconfigurable notebooks. Those patents detailed various scenarios pertaining to a "keyboard-less" notebook in addition to detailing a smart bezel, using hand-wave controls and much more. In today's patent report we cover Apple's third and final installment on this subject matter which pushes the envelope even further this time around by introducing us to acoustic commands. Acoustic commands use unique noise fingerprints of tapping combinations, scratches and/or swirling-motions to control device functionality.  


Basic Patent Overview


Apple's patent covers notebook and tablet housings being integrated with acoustic transducers that could operate as an input. For example, in one embodiment, the electronic device includes a housing and one or more acoustic transducers positioned within the housing each operative to generate an electrical signal in response to detecting a sound resulting from an impact with the housing.


Additionally, the electronic device includes a microprocessor coupled to the one or more acoustic transducers and configured to receive the electrical signals generated by the one or more acoustic transducers. The microprocessor is operative to interpret the generated electrical signals as input to the electronic device.


Another embodiment takes the form of a method of manufacturing an electronic device implementing acoustic transducers within a housing of the electronic device to sense input via the housing. The method may include configuring one or more acoustic transducers located within a housing to sense sound originating on the housing and coupling the one or more acoustic transducers to a microprocessor. The microprocessor may be programmed to interpret electrical signals generated by the one or more acoustic transducers as input, wherein the interpretation includes determining the origin of acoustic interactions based on a timing differential between the generated electrical signals.


Acoustic Commands


Apple's patent discusses how acoustic transducers could be configured to accept acoustic commands that consist of tapping, scratching and other interactions with a surface of the housing. The transducer generates electrical signals in response to the interactions. The electrical signals may be sent to a processor that interprets them as input to an iPad or MacBook Pro-like device. The interpretation maybe based on the type of input, nature of the input, the location of the contact on the housing, the amplitude of input, as well as other various other factors. For example, a scratch may be interpreted differently from a tap, and so forth. Additionally, a tap on the housing near an output or input device may actuate the device whereas a tap on another surface of the housing may be interpreted as a keystroke.


Moreover, a vocabulary of acoustic commands may be provided for the purpose of interpreting the electronic signals. As used herein, vocabulary may refer to recognizing input patterns as indicating a particular function. For example, in one embodiment, the vocabulary may allow for the device to interpret a pattern or series of taps and or scratches, etc., and/or other acoustic inputs as indicating particular functions, such as opening or closing an application, adjusting volume, etc., for example. The electronic device may provide an output or feedback upon receiving the input. In some contexts, however, interactions with the surface of the housing may not be interpreted as input to the electronic device. For example, if the device determines that the contact is incidental, accidental or not recognized as input (i.e., not part of the vocabulary), no feedback or output may be provided.


Carnegie Mellon Research: An Acoustic Related Video


The following video was created by Chris Harrison - a fourth year Ph.D. student at Carnegie Mellon University. I came across one of his great "Masters Projects" presentations on "Scratch Input" after conversing with Chris. Chris makes the benefits of using acoustic based technology come to life on his MacBook at the 3:18 mark of the video below. Many of the idea presented in Apple's patent are covered here in one way or another to make it all come to life. Of course this is covering raw research, so put that into context.




The Keyboard-less Notebook


Apple's patent FIG. 2A illustrates a possible future notebook made of aluminum, titanium, steel or any suitable metal or metal alloy and using an LED or OLED display. Alternative housings could be made of plastic, glass, ceramic, composites or any other suitable material.


2 - Apple patent - keyboard-less macbook Pro with Multiple Acoustic Transducers - feb 2011 

As shown, the lower portion of the notebook includes a relatively flat surface rather than containing a keyboard of a conventional notebook. The notebooks inner surface (20) is part of the housing and is configured to act as an input for the notebook. In some embodiments, the surface may have micro perforations that are not generally visible, but through which light may pass. In other embodiments, the surface is solid. Additionally, in some embodiments, the surface may be opaque and in other embodiments may be translucent or partially translucent to allow light to shine through the surface.


Notebooks with Multiple Acoustic Transducers


As noted in patent FIG. 2A, multiple acoustic transducers (22) may be coupled to the housing or otherwise located within the housing underneath the surface. The acoustic transducers may include microphones and/or speakers. Specifically, in the illustrated embodiment, microphones may be used to sense when a user taps, scratches, or otherwise touches surfaces of the notebook. One or more acoustic transducers may be used to detect sound and employ it as an input. In particular, four acoustic transducers are shown. In other embodiments, there may be more or fewer transducers to perform a particular input function and/or to provide a higher level of precision.


Tablets with Multiple Acoustic Transducers


3 - apple patent - multiple acoustic transducers for the iPad and other iOS Devices -  feb 2011 

Apple's patent FIG. 4 clearly illustrates that acoustic transducers will also apply to future tablet computers including the iPhone and iPod touch as examples.


Interesting Examples for Acoustic Commands


The following are a few interesting examples relating to acoustic commands to help you visualize its value:


Simply dragging a finger in an upwards direction across the back of the housing may increase the volume of output by a speaker, or of headphones coupled to a notebook or iOS device.


In another embodiment, dragging a finger across a device's surface may drag and drop content, such as icons, displayed on the device.


In yet another embodiment, movement in a particular pattern across the housing of the device may lock and/or unlock it; Once again hinting at dropping the Home Button on iOS devices. For example, dragging a finger to "draw" a circle in a clockwise pattern may unlock the device. Additionally, taps and or other interactions with the housing or surface may be interpreted to actuate or turn off devices located near where the user interacted with the surface. For example, if a tap is detected on the housing near Apple's iSight camera, the camera may be turned on or off.


A variety of other input may be defined in the vocabulary to describe a particular input and provide a specific output or response. For example, tapping the speakers may mute or unmute any audio being played. In another example, tapping the speakers once may toggle play and pause, tapping the speakers twice may skip to the next track and tapping the speakers three times may skip to the previous track. In still yet another example, making a circular gesture on the surface may adjust the audio volume (i.e., clockwise gesture for increasing volume and counterclockwise gesture to decrease volume.) Further, in yet another example, tapping the display housing may wake up the display (or the entire device) from a sleep state. It should be understood that many other acoustic inputs may be defined by the vocabulary in addition to or instead of the foregoing examples.


Apple credits Aleksandar Pance, Nicholas Vincent, Duncan Kerr and Brett Bilbrey as the inventors of patent application 20110037734, originally filed in Q3 2009.


Notice: Patently Apple presents only a brief summary of patents with associated graphic(s) for journalistic news purposes as each such patent application is revealed by the U.S. Patent & Trade Office. Readers are cautioned that the full text of any patent application should be read in its entirety for further details. Patents shouldn't be digested as rumors or fast-tracked according to rumor time tables. Apple patents represent true research that could lead to future products and should be understood in that light. About Comments: Patently Apple reserves the right to post, dismiss or edit comments.  




Others are working on this as well as noted here:

Many other Universities are working on this including The University of Cambridge UK. Jens Christensen is one such Ph. D student working on such research.


Here's a PDF on some earlier related type of work...

Ishma Siddiqi

Why wasn't this thought of before? :)

The comments to this entry are closed.