Apple Wins a Patent for Swipe Gestures for Virtual Keyboards

Apple's latest invention victory relates to a method of interpreting swipe gesture input to a device having a touch-sensitive input. The touch-sensitive input could include a virtual keyboard area, in which taps of a touch object generate text input. The method could include detecting a swipe gesture across the virtual keyboard, determining a direction of the swipe gesture, and performing a predetermined function determined by the direction of the swipe gesture. A swipe gesture could include a touchdown of a touch object followed by a sliding motion of the touch object across the virtual keyboard.

Detecting a swipe gesture could include acquiring touch image data from the touch-sensitive device, processing the image to generate one or more finger path events, determining a displacement of the one or more finger path events, and detecting a swipe gesture if the displacement exceeds a predetermined threshold. If the displacement does not exceed the threshold, the input could be interpreted as a conventional tap. The time of the motion associated with the input could also be compared to a maximum swipe gesture timeout threshold. If the timeout threshold is exceeded, the input could be interpreted as a conventional tap.

Determining the direction of a swipe gesture could include comparing a magnitude of a vertical displacement of the swipe gesture to a magnitude of a horizontal displacement of the swipe gesture. A swipe gesture could be interpreted as a vertical swipe gesture if the vertical displacement is greater than the horizontal displacement. Alternatively, a swipe gesture could be interpreted as a horizontal swipe gesture if the vertical displacement is less than the horizontal displacement.

Alternatively, determining the direction of a swipe gesture could include comparing a ratio of the vertical displacement of the swipe gesture to the horizontal displacement of the swipe gesture to a predetermined threshold. A swipe gesture could be interpreted as a vertical swipe gesture if the ratio is greater than the threshold, while a swipe gesture could be interpreted as a horizontal swipe gesture if the ratio is less than the threshold.

A vertical swipe gesture could be interpreted as an upward or downward swipe gesture depending on the sign (i.e., positive or negative) of the vertical displacement. A horizontal swipe gesturecould be interpreted as a rightward or leftward swipe gesture depending on the sign (i.e., positive or negative) of the horizontal displacement.

A Variety of Functions could be Invoked using the Swipe Gestures

A variety of functions could be invoked using the swipe gestures. For example, the swipe gestures could be used for spacing, erasing, or punctuation insertion. Feedback could be presented to the user indicating the function performed. The functions performed by particular swipe gestures may also be customized by the user.

Particular swipe gestures that could be linked to particular functions could include using a rightward swipe gesture to invoke a space, using a leftward swipe gesture to invoke a backspace, using a downward swipe gesture to invoke a carriage return and/or a new line, and using an upward swipe gesture to invoke a shift. Additionally, a second upward swipe gesture could be used to invoke a caps lock. The second upward swipe could either be performed sequentially to the first upward swipe or concurrently with the first upward swipe, i.e., as a multi-fingered swipe gesture. Other alternative mappings of swipe gestures to functions include using an upward swipe gesture to insert a punctuation character, such as a period or apostrophe.

Multi-Fingered Swipe Gestures used to Invoke Additional Functionality

Multi-fingered swipe gestures could be used to invoke additional functionality. For example, a multi-fingered leftward swipe could be used to invoke deletion of a word in a manner analogous to using a leftward swipe gesture as a backspace. Similarly, a multi-fingered rightward swipe could be used to insert a punctuation character and a space, such as a period and a space at the end of a sentence. Swipe gestures could also be used to invoke alternative keyboards containing punctuation, numbers, or special symbols of various types.

Touch Image Processor, Tap and Slide Recognizers

In some embodiments, the computer system could include a touch image processor that receives touch image data from the multi-touch interface and generates finger path tracking events. These finger path tracking events could be processed by a keyboard tap recognizer and a keyboard slide recognizer, which generate key tap and swipe events, respectively, in response to the touch input received. A keyboard input manager could receive the key tap events and the keyboard swipe events and generates text events for an application running on the computer system and feedback popup graphics to be displayed on a display of the computer system.

Apple's patent FIGS. 1A-1B depict a front plan view of a user typing using an exemplary electronic device with touch screen display; FIGS. 2A-2C depict a user invoking a backspace using a leftward swipe gesture.

Apple's patent 5B and 5C depict a user invoking a carriage return/new line using a downward swipe gesture; FIG. 9 depicts a block diagram of an exemplary swipe recognition system.

Apple's patent FIG. 10 depicts a flow chart of an exemplary swipe gesture detection technique; FIG. 12 depicts a simplified block diagram of a computer system implementing one or more embodiments of the present invention.

Apple's First Claim: A computer system having a processor operatively coupled to a memory and a touch interface, the touch interface comprising a virtual keyboard area in which taps of a touch object generate text input, the computer system being adapted to: detect a swipe gesture across a plurality of keys on the virtual keyboard while the virtual keyboard is displayed, the swipe gesture including an initial touchdown point and a direction; determine the direction of the swipe gesture; and perform a predetermined function determined by the direction of the swipe gesture without regard to the initial touchdown point of the swipe gesture on the displayed virtual keyboard, wherein: a rightward swipe gesture invokes at least one of a space, an auto-completion, or display of an auto-completion menu without regard to an initial touchdown point of the rightward swipe gesture on the displayed virtual keyboard; a leftward swipe gesture invokes at least one of a character deletion, a word deletion, or a line deletion without regard to an initial touchdown point of the leftward swipe gesture on the displayed virtual keyboard; a downward swipe gesture invokes at least one of a carriage return, a new line, inserting punctuation, or an alternate keyboard without regard to an initial touchdown point of the downward swipe gesture on the displayed virtual keyboard; and an upward swipe gesture invokes at least one of a shift, a caps lock, or an alternate keyboard without regard to an initial touchdown point of the upward swipe gesture on the displayed virtual keyboard.

Apple's Granted Patent Incorporates a Great Number of Important Patents

Apple's granted patent is related to the following U.S. patents and patent applications, each of which is hereby incorporated by reference in its entirety: U.S. Pat. No. 6,323,846, titled "Method and Apparatus for Integrating Manual Input," issued Nov. 27, 2001; U.S. patent application Ser. No. 10/840,862, titled "Multipoint Touchscreen," issued May 6, 2004; U.S. Provisional Patent Application No. 60/804,361, titled "Touch Screen Liquid Crystal Display," filed Jun. 9, 2006; U.S. Provisional Patent Application No. 60/883,979, titled "Touch Screen Liquid Crystal Display," filed Jan. 8, 2007; U.S. patent application Ser. No. 11/367,749, titled "Multi-functional Hand-held Device," filed Mar. 3, 2006; U.S. patent application Ser. No. 11/228,737, titled "Activating Virtual Keys of a Touch-Screen Virtual Keyboard," filed Sep. 16, 2005; U.S. patent application Ser. No. 11/228,700, titled "Operation of a Computer with a Touch Screen Interface," filed Sep. 16, 2005; U.S. Pat. No. 6,677,932, titled "System and Method for Recognizing Touch Typing Under Limited Tactile Feedback Conditions," issued Jan. 13, 2004; and U.S. Pat. No. 6,570,557, titled "Multi-touch System and Method for Emulating Modifier Keys Via Fingertip Chords," issued May 27, 2003.

To review Apple's other 21 patent claims and invention detailing, see granted patent 8,059,101. Apple credits Wayne Carl Westerman (see below), Henri Lamiraux and Matthew Dreisbach as the inventors of this patent which was originally filed in June 2007, the month that the original iPhone officially debuted.

The Importance of FingerWorks Founders for All Things Multi-Touch at Apple

As you could see, Wayne Carl Westerman is listed as the lead engineer on today's Virtual Keyboard related patent. On page 469 of Walter Isaacson's Biography, Steve Jobs, by Simon and Schuster 2011we read:

"A small company in Delaware called FingerWorks was already making a line of multi-touch trackpads. Founded by two academics as the University of Delaware, John Elias and Wayne Westerman, FingerWorks had developed some tablets with multi-touch sensing capabilities and taken out patents on ways to translate various finger gestures such as pinches and swipes, into useful functions. In early 2005, Apple quietly acquired the company, all of its patents, and the services of its founders. FingerWorks quit selling its products to others, and it began filing new patents in Apple's name."

The majority of Apple's multi-touch related patents are credited to Wayne Westerman and John Elias.

Notice: Patently Apple presents only a brief summary of granted patents with associated graphics for journalistic news purposes as each Granted Patent is revealed by the U.S. Patent & Trademark Office. Readers are cautioned that the full text of any Granted Patent should be read in its entirety for full details. About Comments: Patently Apple reserves the right to post, dismiss or edit comments.

Other Apple virtual keyboard patents: one, two, three, four, five and six.