Apple has been working to allow users to control their devices without ever touching the display with 'Hover' capabilities going back to 2008. Apple's patent filing turned into a granted patent in 2011 and we covered it in a report titled "Apple Wins a Multi-Touch Patent Relating to Hover & Gesture Detection." In 2015 we posted another report on this technology titled "Apple Advances Touch and Hover Sensing Technologies for Future iDevices."

Today the U.S. Patent Office granted Apple a major hover technology patent. The patent shows that Apple has advanced and extended hover controls to Apple Pencil for iPad Pro as our cover graphic reveals in-part. In addition, gestures that Apple fans use every day like pinch to zoom and swipe gestures on their displays are just a few that could be converted and used as hover gestures in the future.

Apple's patent FIG. 5B below illustrates stylus (Apple Pencil #203) relative to a touch-sensitive surface from a side viewpoint of the touch-sensitive surface;

Apple's patent FIG. 7K above illustrates an exemplary user interface for interacting with a user interface object through proximity-based (hover) and contact-based inputs.  

Apple's patent FIG. 10G below illustrates an exemplary user interface for displaying different visual feedback when an input object is detected to meet different levels of hover proximity criteria.

Apple later states that in some embodiments, the device, such as an iPad, generates an audio output to indicate that Apple Pencil has made initial contact with the touch-sensitive surface. The audio feedback enhances the visual feedback to alert the user that object has exited the hover state and entered the contact state, and the user interface behavior will be different in the contact state, as compared to the hover state.

Apple states that proximity-based inputs also referred to as hover gestures or hover controls will primarily work with future versions of the iPhone, iPad and iPod Touch.

Apple of course doesn't limit any invention and so Apple notes that hover controls could also one day work with displays on an iMac, MacBook, a MacBook Trackpad, and work with accessories like a Magic Keyboard and/or Magic Trackpad.

Hover controls will work with future work related apps like word processing, spreadsheets, image editing, FaceTime, Apple Music and yes even games on iDevices. This way your finger won't interfere with your game play.

In-Air gestures count as a hover gesture in the patent. Apple states that "The same concepts apply in an analogous manner to other types of gestures. For example, a swipe gesture, a pinch gesture, a depinch gesture, and/or a long press gesture are optionally detected based on the satisfaction of criteria that are either independent of intensities of contacts included in the gesture, or do not require that contact(s) that perform the gesture reach intensity thresholds in order to be recognized.

For example, a swipe gesture is detected based on an amount of movement of one or more contacts; a pinch gesture is detected based on movement of two or more contacts towards each other; a depinch gesture is detected based on movement of two or more contacts away from each other; and a long press gesture is detected based on a duration of the contact on the touch-sensitive surface with less than a threshold amount of movement.

Although Apple began working on the hover control project eleven years ago, today's granted patent would have to be considered, what I call, a master patent wherein the inventors go through dozens of scenarios of how Hover controls will work with finger or Apple Pencil. You could spend a day or two just by drilling down into their hundred or more patent figures looking for every little nuance.

With Apple reviving their attention on this project and extending it to Apple Pencil, you have to wonder if this project is closer to a point of bringing it to market.  

Hover controls will also include in-air gestures and this is likely something that could be coming to a future iPhone using Apple's depth camera / TrueDepth camera.

In fact, Patently Apple covered this in a March report titled "Apple Patent Covers In-Air Gesturing for Macs & iDevices as Google, Samsung and Microsoft race to bring this to Market. Sony has a new 3D camera that will be coming to Android phones later this year and so it may be time for Apple's many patents on this subject to come to life.

Android phones had hover controls going back to at least 2013 and many of the most recent reviews of this feature on YouTube actually date back 5 years ago. It's probably a dead feature and why Apple has kept pushing this project forward until such a feature could make sense and add value to Apple products.

Apple's granted patent 10,318,034 was originally filed in Q3 2017 and published today by the US Patent and Trademark Office. To review this monster of a patent click here.

Inventors

Jeffrey Traer Bernstein: Design Director at Apple. Jeffrey notes in his bio that he leads the design + prototyping team in Human Interface Devices.

Julian Missig: Design Manager. Julian notes in his bio that within the Prototyping group, he manages a small team to design and prototype experimental interface and interaction concepts across all of Apple's products including: Multi-Touch trackpad gestures, iPhone X, True Depth, Touch Bar, True Tone Display, iPad Pro, Apple Pencil, Smart Keyboard, tvOS, Siri Remote, 3D Touch, Force Touch, Taptic Engine, Continuity, Magic Trackpad and more.

Mark Hauenstein: Interface Designer / Prototyper

About Making Comments on our Site: Patently Apple reserves the right to post, dismiss or edit any comments. Those using abusive language or negative behavior will result in being blacklisted on Disqus.