Way back in late 2011 Apple was granted a patent for a display system that would recognize and/or detect hovering gestures. Today, the U.S. Patent & Trademark Office published a patent application from Apple that reveals their ongoing work on this feature.

Apple Advances Touch and Hover Sensing Technologies

Apple notes that some touch sensitive devices are able to recognize both touch and hover-events, i.e., an object near but not touching the touch sensor panel, and the position of the hover event at the panel. The touch sensitive device can then process the hover event in a manner similar to that for a touch event.

While touch and hover capabilities in a touch sensitive device are desirable, together they can present a challenge to cooperative performance for accurate, reliable detection of touch and hover events.

Apple's invention relates to improving touch and hover sensing. Various aspects of touch and hover sensing can be addressed to improve detection of touch and hover events. In some embodiments, a touch and hover sensing device can ensure that a desired hover event is not masked by an incidental touch event, e.g., a hand holding the device, by compensating for the touch event in the sensing signal that represents both events.

Conversely, in some embodiments, when both the hover and touch events are desired, the device can make adjustments to its sensors and/or the sensing signal to ensure that both events are detected. In some embodiments, the device can improve the accuracy of its determination of the device user interface location to which a hovering object is pointing by profiling the object shape. In some embodiments, the device can differentiate between object distance and area (or size) so as to properly process the corresponding sensing signal and subsequently perform the intended actions. In some embodiments, the device can improve detection of concurrent hover events.

In some embodiments, the device can compensate for signal drift in the sensing signal by adjusting the baseline capacitance of the device. In some embodiments, the device can compensate for resistance from the touch and hover sensors by making adjustments to the sensors and/or the voltage patterns driving the device. In some embodiments, the device can compensate the sensing signal for sensitivity variations of the sensors (generally at issue during a hover event), by applying a gain factor as a function of the location of the hover event to the sensing signal. In some embodiments, the device can improve sensor switching between a touch mode and a hover mode by compensating for parasitic capacitance introduced by the switching components in the sensing signal. In some embodiments, the device can improve integration of a display with the sensors by reducing interference from the display at the sensors.

Under a segment of the patent filing called "Object Shape Profiling," Apple notes that an object hovering over a sensing device (finger or stylus) can point to an area on the device's UI display to cause an action. In some instances, there can be difficulty in determining specifically where the object is pointing so as to cause the intended action. Apple's patent FIG. 9 illustrates such an example. Here, finger #914-b of hand #914 can hover over touch and hover sensing device #900, where the finger is pointing somewhere in area #928 of the device UI display. To help identify where within the region #928 the finger is pointing, various methods associated with hand shape profiling can be used.

Under a segment of the patent filing called "Distance and Area Differentiation," Apple notes that their FIG. 14 as noted above, illustrates an exemplary touch and hover sensing device that can differentiate between a small close object and a large distant object according to various embodiments. In the example of FIG. 14, a camera (visual capture device #1430) can be disposed at a position proximate to the sensor lines of touch and hover sensing device #1400 to capture image(s) and/or video of a hovering object. The captured image(s) and/or video can then be used to determine the distance and area of the object.

The distance of the object from the touch and hover sensing device can be determined according to the object and device positions in the captured image(s) and/or video using any suitable image/video object recognition algorithm. The area of the object can be determined according to the size of the object in the captured image(s) and/or video using any suitable image/video object recognition algorithm.

Apple's patent FIG. 15 illustrates another exemplary touch and hover sensing device that can differentiate between a small close object and a large distant object by using detectors #1530 that can be disposed at positions proximate to the sensor lines of touch and hover sensing device #1500 to detect a hovering object.

The distance and area of the object can be determined according to various characteristics of the detectors' signal using any suitable signal processing algorithm. Examples of the detectors can include sonar, infrared, optical, radio, and the like.

In our last example, we see patent FIG. 29 illustrating another exemplary touch and hover sensing device that can compensate for sensor resistance. In the example of FIG. 29, touch and hover sensing device #2900 can dynamically gang sensor lines according to the distance of the hand (or hovering object #2914). Accordingly, the sensor lines 2901-a and 2901-b can be ganged together as a single sensor line 2910-a and the sensor lines 2901-c and 2901-d can be ganged together as sensor line 2910-b to realize lower sensor resistance. Ganging can be performed for horizontal sensor lines, vertical sensor line, or both.

Patent Credits

Apple credits Senior Technologist Brian King who used to work in NASA's jet propulsion lab. King has extensive experience in analog and digital image and signal processing (optics, acoustic, inertial sensing, radar, SAR) which play an important role in today's invention.

The other inventors associated with this invention includes Omar Leung, Paul Puskarich, Jeffrey Bernstein, Andrea Mucignat, Avi Cieplinski, Muhammad Umar, Praveen Subramani, Marc Piche, David Amm and Duncan Kerr as the inventors of patent application 20150169114 which was originally filed in February 2015.

Considering that this is a patent application, the timing of such a product to market is unknown at this time.

Patently Apple presents a detailed summary of patent applications with associated graphics for journalistic news purposes as each such patent application is revealed by the U.S. Patent & Trade Office. Readers are cautioned that the full text of any patent application should be read in its entirety for full and accurate details. About Making Comments on our Site: Patently Apple reserves the right to post, dismiss or edit any comments. Comments are reviewed daily from 5am to 6pm MST and sporadically over the weekend.