A Basic Overview of Apple's Invention Regarding a Touchless Gesture System for iDevices

Apple's invention generally relates to electronic devices that are capable of recognizing touchless gestures and performing one or more actions in response to the touchless gestures.

Apple's invention describes enabling a user to interact with an electronic device using spatial gestures without touching the electronic device. In one embodiment, the electronic device provides a contactless mode of operation during which a user can interact with the electronic device without touching the device. For purposes of this disclosure, a gesture made by a user with respect to an electronic device without touching or contacting the electronic device may be referred to as a contactless gesture or a touchless gesture.

A touchless gesture may include the motion of an object relative to the electronic device, the motion of one or more portions of an object relative to the electronic device or simply the detection of a stationary object. In certain embodiments, a touchless gesture may be used to indicate an action to be performed by an electronic device and also to set an action-related parameter value that is then used when the action is performed.

A user may make a touchless gesture, for example, using the user's hand or finger or other part of the user's body. However, this is not intended to be restrictive. In some other embodiments, a touchless gesture may be made by the user using an object such as a pen, a stylus, and the like. In general, a user may make a touchless gesture using an object such as the user's hand or finger, or some other object. The user may use both the position of the object and/or the motion of the object relative to the device to provide input to the device.

In certain embodiments, an electronic device is configured to detect one or more touchless gestures made by a user relative to the device and perform one or more actions in response to and based upon the one or more touchless gestures. For example, in one instance, the electronic device may be configured to detect the distance between the user's hand and the electronic device.

The touchless gesture made by the user, or some other gesture, may be used to determine an action to be performed by the electronic device. The value of a parameter associated with the action may then be set based upon the detected distance between the user's hand and the electronic device. The electronic device may then perform the action using the parameter value. For example, the action that is performed may be a zoom or pan operation. The distance of the user's hand from the electronic device may be used to set the value of a zoom or pan rate parameter.

In one instance, the value of the zoom or pan rate may be proportional to the distance of the user's hand from the electronic device (e.g., a smaller distance may correspond to a smaller zoom or pan rate value and a greater distance may correspond to a higher zoom or pan rate value). The zoom or pan operation may then be performed using the set zoom or pan rate value.

In certain other embodiments, a user of an electronic device may make repetitive touchless gestures with respect to the electronic device. These repetitive touchless gestures may be detected by the electronic device and used to set a value of a parameter on the electronic device. The same gestures or a different user gesture may be used to determine an action to be performed by the electronic device. The value of a parameter associated with the action may then be set based upon the number of repetitive gestures. The electronic device may then perform the determined action, using the parameter value. Examples of repetitive touchless gestures include, but are not limited to, repetitive swipes of the user's hand in front of the device, and other touchless gestures. In one instance, the value of the zoom or pan rate may be proportional to the number of times the user's hand passes in front of the electronic device (e.g., a lesser number may correspond to a smaller zoom or pan rate value and a greater number may correspond to a higher zoom or pan rate value). The zoom or pan operation may then be performed using the set zoom or pan rate value.

In certain embodiments, the velocity associated with a touchless user gesture relative to the electronic device may be used to set a parameter associated with the action to be performed. The same gesture or a different touchless user gesture may be used to determine the action to be performed, and to set the parameter value. The electronic device may then perform the action, where the performance of the action is based on or influenced by the parameter value.

The touchless gestures described above may be performed by the user in a spatial zone around the electronic device. For example, in one instance, a touchless gesture may be performed in a zone in front of the electronic device, such as in front of a touch screen of the electronic device. In another instance, the touchless gesture may be performed in a zone behind the electronic device away from the touch screen. The touchless gesture may also be performed in other zones around the electronic device.

A Prime Example of a Touchless Gesture

Apple's patent FIG. 1 presented below depicts a simplified diagram of an example electronic device #100 that may incorporate their new invention. In certain embodiments, the electronic device may be placed into a contactless mode where it is configured to recognize touchless gestures and use them as input. Myriad methods may be used to enable the contactless mode such as by selecting a button or switch on the electronic device, performing a touch gesture on electronic device and so forth.

Apple's patent FIG. 1 depicts a user's hand #140 positioned on the front side of an iDevice. The hand is depicted in a first position at a distance D1 from the iDevice performing a touchless gesture.

The user's hand is also shown as able to move along path #135 closer to or further away from the iDevice. When operating in contactless mode, the iDevice sensor #130 may be used to detect the presence of a user's hand and the gestures being made. Apple's patent FIG. 2 that's not shown is identical to FIG. 1 except the user's hands are gesturing in an up and down motion instead of side-to-side as depicted in FIG. 1 above.

Visible or Audible Touchless Gesture Recognition

In certain embodiments, the user's iDevice may provide feedback to them when a gesture has been recognized and the value of an attribute of the gesture measured. In one example, the user's iDevice may output an audible tone or display an indicator on the touch screen to indicate that the gesture has been recognized and the value of an attribute of the gesture has been successfully determined.

The System's Advanced Sensors

Ultrasonic Sensors: According to Apple, the touchless gesture system may employ one or more ultrasonic sensors. Such sensors work on a principle similar to radar or sonar. Ultrasonic sensors generate high frequency sound waves and evaluate the echo which is received back by the sensor.

This type of sensor calculates the time interval between sending the signal and receiving the echo to determine the distance to an object. Some sensors have transmitters that are separate from the receivers while others may be a substantially unitary device comprising both a transmitter and a receiver.

In some embodiments a plurality of ultrasonic sensors are used which can form a reasonably detailed "sound-based" image of the object. Ultrasonic sensors can be used in conjunction with a central processing unit to determine the velocity, acceleration and rotation of an object, a gesture of an object and the distance of an object from the electronic device.

Optical Sensors: The new touchless gesture system may also or alternatively employ one or more optical sensors. Such sensors work similar to the ultrasonic sensors discussed above, however instead of generating high frequency sound waves these sensors generate light waves which are reflected back to the sensor by the object.

The light source may be, for example, infra-red, white light, a laser or other type of light. The sensors calculate the time interval and sometimes the frequency and/or phase shift between sending the signal and receiving the echo to determine the distance to an object. Some sensors may also be able to determine the direction of the reflected light and use that to detect the position or distance of the object.

Some non-imaging optical sensors have transmitters that are separate from the receivers while others may be a substantially unitary device including both a transmitter and a receiver. In some embodiments a plurality of non-imaging optical sensors are used which can form a reasonably detailed "light-based" image of the object.

Non-imaging optical sensors can be used in conjunction with a central processing unit to determine the velocity, acceleration and rotation of an object, a gesture of an object and the distance of an object from the electronic device.

Other sensors that Apple may employ include proximity sensors, hall-effect sensors, radar sensors, thermal sensors and others.

Touchless Gesture Sensing Zones

Apple's patent FIGS. 3, 4, 5 and 6 noted above illustrate and emphasize the touchless gesture system's wide scope of recognizable sensing zones covering an iDevice's front side, backside, actual sides and height.

In Apple's patent FIG. 13 noted below we're able to see an overview of the touchless gesturing system that could be incorporated into future iDevices, Macs and found in use at future Apple Stores and Kiosks.

Apple's patent FIG. 7 below depicts a simplified flowchart illustrating the method of touchless interaction with an iDevice.

Apple iDevice or Mac users will be able to perform many forms of air-gesturing commands with new capabilities and forms of zooming in or out, assigning wave movements to certain activities be it opening various apps, scrolling through a document, dimming your display and so forth. As Apple phrases it, "Gesture detection may be preconfigured from the manufacturer or it may be user configurable."

Apple further notes that the system will be able to recognize gestures using two fingers of a hand moving close together or further apart to form a gesture. Another example would be to rotate one's hand or complex gestures including one's hand transitioning from a fingers extended position to a fist.

Apple's patent application that published by the U.S. Patent Office today was originally filed on Dec 4 2013.

A Note for Tech Sites covering our Report: We ask tech sites covering our report to kindly limit the use of our graphics to one image. Thanking you in advance for your cooperation. 

Patently Apple presents a detailed summary of patent applications with associated graphics for journalistic news purposes as each such patent application is revealed by the U.S. Patent & Trade Office. Readers are cautioned that the full text of any patent application should be read in its entirety for full and accurate details. About Making Comments on our Site: Patently Apple reserves the right to post, dismiss or edit any comments. Comments are reviewed daily from 4am to 8pm MST and sporadically over the weekend.