Patent Summary

 

In this first patent pertaining to motion, Apple's invention describes a motion sensitive input control that could prevent unintended input caused by inadvertent movement of a computing device. For example, if a user operates a computing device on public transportation and the user's finger hovers over an input area of the computing device during a bump in the ride, the bump could cause a finger to inadvertently tap the input area. If the computing device is configured to implement a click action when a tap input event is detected on the input area, the inadvertent tap could result in an unintentional click in the user interface of the computing device. This could lead to undesirable user interface behavior, such as the deletion or premature sending of an e-mail depending on where the cursor is located at the time of the inadvertent tap input event.

 

In one embodiment, motion sensitive input control could prevent unintended input by disregarding an input event if a change in motion of the computing device is detected simultaneously with or immediately prior to the detected input event. In another embodiment, motion sensitive input control could prevent unintended input by reducing the sensitivity of an input device during a motion-based state associated with the computing device. In this manner, the likelihood of inadvertent motion of a computing device causing an unintended input event can be reduced.

 

The sensitivity of the input device could be reduced in a variety of ways. For example, in one embodiment, an input event threshold associated with the magnitude of a detected measure of input (e.g., capacitance for a capacitive input device) could be adjusted during a motion-based state. In another embodiment, a threshold associated with how long an input must be recognized in order for the input to be recognized as an input event can be adjusted during a motion-based state.

 

Overview Basics

 

While Apple's current patent focuses on motion input controls pertaining to a MacBook, the patent does indeed apply to iOS devices as is likewise confirmed by their 2009 patent application.

 

 
 

Apple's patent FIG. 2 illustrates an exemplary process in which a computing device could determine whether to disregard or process an input event

 

Any suitable input event could be detected by input device 210 for consideration by processor 200 in block 250, such as a touch event or gesture for example. A "touch event" could refer to an event other than a mere touching of the input device. Rather, a touch event could be one of several types of touch events, including, for example, a "touch begin" event (e.g., initial touch is detected), a "touch move" event (e.g., after initial touch is detected, the coordinates of the touch change), and a "touch end" event (e.g., after initial touch is detected, the touch is no longer detected).

 

One, Two or Three Dimensional Motion Sensors

 

Further, any suitable type of motion sensor could be utilized in accordance with the teachings of the present invention. For example, a dedicated one-dimensional motion sensor could be utilized in an embodiment in which inadvertent motion of the computing device in only one direction is likely to cause an unintended input to be prevented (e.g., preventing a laptop trackpad from bouncing up into a finger due to inadvertent motion, causing a false tap input event).

 

A dedicated two-dimensional motion sensor could be utilized in an embodiment in which inadvertent motion of the computing device in either or both of two directions is likely to cause an unintended input to be prevented (e.g., preventing a laptop trackpad from lateral movement due to inadvertent motion when a finger is already touching the trackpad, causing a false gesture or tracking input event).

 

And a dedicated three-dimensional motion sensor could be utilized in an embodiment in which inadvertent motion of the computing device in any direction is likely to cause an unintended input to be prevented (e.g., preventing an input device such as a trackball from inadvertent contact in any direction with a finger, causing a false trackball movement input event). In another embodiment, the computing device could utilize a three-dimensional motion sensor in accordance with the teachings of the present invention and for other purposes, such as detecting when a computing device is falling in order to park its hard drive and take any other preventive action.

 

Apple's patent FIG. 3 illustrates an exemplary process in which a computing device could determine whether to reduce the sensitivity of an input device. Patent FIG. 7 illustrates an exemplary computing device including a multi-touch sensor panel and new "sense channels."

 

Sensors in Search of Applications

 

Apple's research and development teams are working with countless types of next generation sensors in search of applications. Beyond motion sensors as described above, we know that Apple is working on an Ambient Light Sensor system in connection with a future 3D version of OS X. We also know that Apple is working with a next generation iSight camera for notebooks that will recognize user gestures. That particular patent is in concert with a wider trackpad, but it doesn't have to be. That's the beauty of research. Cross-pollinating ideas are common. The ambient light sensor patent above describes building these sensors right into the bezel of a proposed future MacBook – which could likewise mean integrating other kinds of motion sensors that would recognize gestures in the bezel.

 

Virtual and Motion Keyboards

 

Desktop: Apple is also working on the next big thing, a leap towards a full pledged virtual keyboard system for a future Apple desktop. And with Apple experimenting with their Magic Trackpad today, their virtual keyboard team will be able to learn and advance their future virtual keyboard project based on what they'll learn from the trackpad. It's a building process. But until we get there, and that could be many years out, Apple has another team simultaneously working on a kind of hybrid keyboard system that will include an advanced camera and motion system.

 

MacBook: It should be noted at this point that although the patent focuses their communications on a desktop keyboard application using motion tracking sensors, the patent clarifies in patent point # 53 that the "Computing system can be any of a variety of types employing image processing techniques for a motion sensitive mechanical keyboard." Apple's patent FIGS. 13A is in fact a notebook. So that means that the following information applies to both a possible future desktop and MacBook. 

 

A Hybrid Keyboard with Motion Tracking

 

 

As illustrated in FIG. 4, we're able to see two sets of slot cameras, each consisting of two cameras covering top and side quadrants of each half of the keyboard. The cameras are so positioned to capture the XYZ motion of the respective hands/fingers.

 

The Camera May be Hidden in the Keyboard

 

Apple states that the advantage of this slotted camera system is that the cameras are always in the correct position, and could be of low profile and even hidden in the keyboard enclosure.

 

Mouse Mode

 

Additionally, Apple states that Z data could be provided which could be used for cursor input operations that discriminate between hands/fingers resting on keys and the hands/fingers being lifted off keys. In one embodiment, all of the cameras together cover the entire keyboard and could be enabled to sense motion in mouse mode.

 

Separating the function of the keyboard into two or more distinct modes that the user deliberately invokes has the advantage of eliminating the chance that random or postural changes in hand/finger position could be misinterpreted as a cursor input (e.g., point, scroll, drag, zoom). In this manner, the keyboard doesn't need to determine when the user intends to issue commands to control screen activities (e.g., scrolling) because the user informs the keyboard of the user's intent by switching modes.

 

Mode Switching

 

Mode switching could be implemented in various ways. In some embodiments, mode switching could be implemented in ways that don't require the user to look down at the keyboard thereby improving the user experience.

 

In one embodiment, a dedicated "mouse" key could be provided such that mouse mode is entered for the duration that the mouse key is held down. In another embodiment, the dedicated mouse key could include a "sticky" key, such that a tap of the key switches between modes.

 

In a further embodiment, the modes could be switched when the user concurrently taps an arbitrary combination of the keys. For example, in one embodiment, the arbitrary combination of the keys could include any four of keys. In another embodiment, the arbitrary combination of the keys could be restricted to adjacent keys in order to affect the mode switch.

 

 

In Apple's patent FIG. 2 we see an illustration of a process for switching between typing and mouse operations using the keyboard. In mouse mode in the illustrated embodiment, the hand that is not being used for pointing or gesturing could hold down a number of adjacent keys (e.g., 2, 3, or 4) while the other hand/fingers move about the keyboard surface and are tracked by the motion sensing circuitry.

 

For example, while a dedicated mouse key is held down or if a 4-key tap occurs (block 200), the keyboard could enter mouse mode such that motion sensing circuitry tracks hand/finger motion (block 205). If not, the keyboard could remain in typing mode and hand/finger motion could be ignored (block 210). While two keys are held down (block 215), motion sensing circuitry could track hand/finger motion to effect a pan (for detected horizontal motion) and scroll (for detected vertical motion) (block 220).

 

Air-Drag Operation

 

The keyboard could also interpret a two-key tap (block 225) as a primary click (similar to a left click on a conventional mouse) (block 230). While three keys are held down (block 235), the motion sensing circuitry could track hand/finger motion to effect a drag operation (similar to a click-hold and drag operation by a conventional mouse) (block 240). The keyboard could also interpret a three-key tap (block 245) as a secondary click (similar to a right click on a conventional mouse) (block 250).

 

And finally in Apple's patent FIG. 3B, key press hand 300 could hold down two adjacent keys while the hand/finger movement of motion hand 310 could be tracked by the motion sensing circuitry. Up and down movement could control scroll while left and right movement could control pan.

 

The second patent, which was first published on January 13, 2011, was invented by John Elias and originally filed in Q3 2009. 

 

Update: New Patent Details Emerge

 

In a patent predating this month's motion tracking keyboard patent, Patently Apple has found a few new details to add to the bigger picture of the motion tracking keyboard as follows: 

 

More on Mode Switching

 

Apple's patent FIG. 4 as noted below illustates a process for seamless mode switching between typing and mouse operations by monitoring the number of fingers touching down on its surface - very much like patent FIG. 2 noted in the earlier illustration above.   

  

What's new here is the listing of a four-finger touch option that would be able to trigger mouse mode actions of such things as a scroll and/or pan action.

 

Specific Action Gestures

 

Apple's patent states that hand rotation and hand expansion/contraction gestures could be used for zooming and/or opening files; hand swipes and slides could be used to accellerate operations like text cursor positioning. 

 

High Resolution Near-Field Sensor Array

 

Touch sensing associated with the keyboard could be implemented via capacitive sensing using a high-resolution near-field sensor array capable of tracking individual contacts made by each finger tip and hand part touching the keys.

 

Touch Sensitive Keyboard Membrane

 

Apple's patent FIG. 6 illustrates and exemplary touch sensitive keyboard membrane that could be embedded into the keyboard. This would allow for sensor nodes to be closer to the edge of the keys and could be configured to have a greater sensing resolution. And lastly, Apple's patent FIG. 7  illustrates the incorporation of "drive and sense lines" into the new motion tracking keyboard.

 

Notice: Patently Apple presents only a brief summary of patents with associated graphic(s) for journalistic news purposes as each such patent application is revealed by the U.S. Patent & Trade Office. Readers are cautioned that the full text of any patent application should be read in its entirety for further details. Patents shouldn't be digested as rumors or fast-tracked according to rumor time tables. Apple patents represent true research that could lead to future products and should be understood in that light. About Comments: Patently Apple reserves the right to post, dismiss or edit comments.