Apple has been granted a patent for a Hand Gesturing User Interface that could extend to Finger Gesturing & Beyond
Back in September 2019 Patently Apple posted a report titled "Google's Hardware Event to be held in New York on October 15." One of the major features that was to be introduced was the Pixel 4's new In-Air Gesturing System called "Aware" which is illustrated in our cover graphic above. Apple has been working on in-air gesturing for years and our archives on this area of technology cover most of their patents to date.
Today the U.S. Patent and Trademark Office officially granted Apple a patent this invention titled "Gesture based user interface."
Apple's granted patent covers an invention of a gesture based user interface, comprising a movement monitor configured to monitor a user's hand and to provide a signal based on movements of the hand; a display; and a processor configured to move a cursor responsive to the signal from the movement monitor, within a predetermined region on the display, formed of a plurality of sub-regions associated with respective control commands, and to provide the commands to a controlled application responsively to the movements of the hand.
Optionally, the processor is configured to ignore downward components of movements of the hand. Optionally, the quick-access sub-regions are located within the predetermined region such that downward movements do not lead to a quick-access sub-region.
Apple's patent FIG. 1 below is a block diagram of a control system based on identifying hand gestures; FIG. 2 is a schematic view of a horizontal bar user interface; and FIG. 4 is a schematic illustration of a joystick user interface.
In some example embodiments, the quick-access sub-region #208 of FIG. 2 could be assigned to a play/pause command and to a volume control for Apple TV, a future television and other devices.
For example, one sub-region #206 may be assigned to entrance into a sub-interface for controlling display speed (e.g., fast-forward and rewind) and another may be assigned to entrance into a movie selection interface.
In other embodiments, quick access regions are assigned to a command for opening a frequently used menu and/or to a command for returning to a parent or main menu.
Lastly, the patent states that beyond hand gesturing, the invention could be in context with other future movements such as leg movements, body movements and finger movements.
For finer details about Apple's invention, review Apple's granted patent 10,928,921.
Before Apple could introduce hand gesturing/In-Air gesturing, it will be required that their TrueDepth camera be advanced so that it could recognize a face or hand gestures at a much greater distance than their current Face ID camera can. Apple is currently working on this.
Last week Patently Apple posted an IP report titled "Apple Reveals a next-gen Face ID Camera that uses a new iTOF system that is powerful enough to use on Future Macs." The second patent was posted in December 2020 titled "Apple Wins Patent for Next-Gen HomePod with Camera System that includes Gaze Controls & Long-Range Face ID."
So while hand gesturing is likely to surface in the coming years, Apple is currently working on several approaches that could enable this feature to work on several types of devices. All of this will take time to work itself to market.