On January 21, 2010, the US Patent & Trademark Office published a patent application from Apple that reveals one of the next possible chapters for Apple's Magic Mouse. Apple introduced the world's first Multi-Touch Mouse this past October. The Magic Mouse features a seamless touch-sensitive enclosure that allows it to be a single or multi-button mouse with advanced gesture support. Using intuitive gestures, users can easily scroll through long documents, pan across large images or swipe to move forward or backward through a collection of web pages or photos. In today's patent, we see that Apple is already in the process of either developing a new input device (or devices) or could actually be planning an advanced version of the Magic Mouse to process kinematical inputs and gestures that the patent claims will directly benefit video games, interactive navigation and industrial applications amongst others. The next tricks that the Magic Mouse will apparently learn includes advanced gestures revealed as tilt, tap, slide, scoop and yes, brush. The latter gesture suggests that Apple could be gearing up to introduce a new paint or illustration program. That was indirectly presented in another recent patent relating to yet another input device in the form of a 3D remote wand of sorts. In fact, some of the motions described here today, would actually suit the other patent pending input device better.
Gesture Storage, Force and Motion Sensors
Apple's patent FIG.2 is a system diagram of a modular arrangement of input device, first described in terms of being a mouse. In some embodiments, the memory 212 is adapted to store gesture profiles comprising force and velocity thresholds and/or applicable ranges for each type of gesture.
An interface module 216 enables data to be transmitted and/or received between two or more devices. In one embodiment, data transmitted to a receiving device is first packetized and processed according to one or more standardized network protocols. The interface module 216 may accommodate any wired or wireless protocol including, without limitation, USB, FireWire, Ethernet, Gigabit Ethernet, MoCA, radio frequency tuners, modems, WiFi, Bluetooth, WiMax, and/or Infrared Data Association.
One or more motion sensors 210 enable the input device to determine velocity values during a given instant, or alternatively, over a given period of time. Various types of motion sensors, such as gyroscopes, accelerometers, optical sensors, etc. may be used with the present invention. In one embodiment, the motion sensors comprise one or more accelerometers adapted to detect the current acceleration of the input device. The accelerometers can measure acceleration of the input device by any number of means, including, for example, inclination sensing, vibration, and/or shock values. In one embodiment, the input device comprises at least one sensor which can track movement of the input device irrespective of any interaction or contact with the surface 104.
One or more force sensors 214 determine the forces applied to the input device 102. Forces such as a normal force, shear force, frictional force, angular force, or any combination thereof may be measured and utilized in accordance with one or more embodiments of the present invention. Any type of force sensor or force sensor combination may be used for accomplishing force detection, including, for example, switch sensors, contact sensors, and/or accelerometers. Additionally, the force sensors 214 can comprise lightsource/slit/photosensor combinations, or one or more capacitive sensors on a compliant support. In one embodiment, the force sensors 214 detect force independently from the motion sensors 210. Also, according to one embodiment, forces are sensed as surface reactions to one or more input forces (as described in more detail below; see FIG. 5a-5c and accompanying text).
Applied Velocity & Plurality of Forces
Apple's patent FIG. 4 noted above is a block diagram indicating an input device 102 with an applied velocity 404 according to one embodiment of the present invention. Notice that the velocity comprises both a magnitude (length of arrow) and a direction, thus forming a velocity vector. In some embodiments, one or more motion sensors 210 (FIG. 2. such as accelerometers) comprised within the input device are responsible for determining the velocity of the input device.
Above you'll find several patent figures describing various forces that could be applied to this unique input device. Specifically, patent FIG. 5a is a block diagram illustrating a plurality of forces exerting upon an input device; FIG. 5b illustrates a surface reacting to a downward force applied to an input device and FIG. 5c illustrates a surface reacting to a lateral force applied to an input device.
Gesture Profiles: Brush and Scoop Examples
Apple's patent FIG. 6a noted below is a set of graphs depicting a first gesture profile according to one embodiment of the present invention. The set of graphs comprises a force graph and a velocity graph each plotted against a common time axis 600. The graphs together depict a "brush" gesture that is similar to a brushing motion. The brush gesture is comprised of an input sequence where the input device 102 is first moved laterally across a surface 104 and then is subsequently lifted 620 above it (as depicted in FIG. 6b).
The inadvertent detection of a brush gesture might occur as a result of infinitesimal lateral forces exerted upon the input device 102 as an expected part of the user lifting the input device 102 off of the surface 104. Alternatively, the V.sub.min 604 may be implemented to ensure that the lateral forces are of a certain velocity before being interpreted as a brush. For example, in one embodiment, multiple brush gestures are defined such that each gesture comprises a separate velocity threshold. An application may be set to handle "fast brushes" differently than "slow brushes." Alternatively, the application may be set to only accept "fast brushes." Thus, V.sub.min 604 may be set according to the application designer's specific preferences.
Apple's patent FIG. 8a shown above is a set of graphs depicting a second gesture profile. As in FIG. 6a, the set of graphs comprises a force graph and a velocity graph plotted against a common time axis 800. The graphs together depict a "scoop" gesture that is similar to a scooping motion. Further into the patent they present the "nudge" gesture.
Gesture Profiles: Tilt, Slide and Tap
Apple's patent FIGS. 15a and 15b are block diagrams depicting a sixth gesture profile according to one embodiment of the present invention. These figures depict a "tilt and tap" gesture comprising of an input sequence where the input device 102 is first tilted and then tapped against a surface 104.
In one embodiment, the gesture is enabled if there is a detected decrease in z-direction acceleration in a first region of the input device 102 along with a detected increase in z-direction acceleration in a second region of the device (i.e., in the direction of tilt). In another embodiment, the input device 102 comprises a tilt sensor adapted to sense the inclination of the input device 102. The gesture is enabled when the detected inclination falls within a certain range when measured from a reference axis.
As shown in FIG. 15b, once the direction of tilt and/or tilt axis has been ascertained, the gesture will become triggered if a tap is subsequently detected. Tap detection may be implemented in any number of ways. For example, according to one embodiment, an impulse in acceleration corresponding to the first region of the device and/or one or more force signals indicate motion of the input device 102 followed by contact with a surface 104 (i.e., a single tap). Note also that in some embodiments, additional taps trigger to separate gestures or map to separate gesture profiles (e.g., double-taps and triple-taps).
Apple credits Omar Leung as the sole inventor of patent application 20100013768, originally filed in Q3 2008. For more information, view this temporary link.
Notice: Patently Apple presents only a brief summary of patents with associated graphic(s) for journalistic news purposes as each such patent application and/or grant is revealed by the U.S. Patent & Trade Office. Readers are cautioned that the full text of any patent application and/or grant should be read in its entirety for further details. For additional information on any patent reviewed here today, simply feed the individual patent number noted above into this search engine.
Please note that any of the temporary links presented in today's report are in fact temporary and may redirect you to unrelated patents in the future. In such cases refer back to the search engine instructions above.