Today the US Patent & Trademark Office published a patent application from Apple that relates to an enhanced virtual trackpad area system. The patent was inherited by Apple when they acquired the Israeli firm PrimeSense for their deep technologies related to what became Face ID and others such as gaze and in-air gesture systems. The original Xbox Kinect camera system used PrimeSense's technology for in-air gesturing. That was in 2010. Imagine how this technology has advanced over the last decade – and this patent reveals some of how its possible advancements.   

Since inheriting this patent from PrimeSense, Apple has worked on making it their own by cancelling the previous all-important patent claims and replaced them with 20 new claims reflecting their possible implementation of this technology in future Mac computers. Our archives for in-air gesturing patents now has 60, not including today's patent application to demonstrate that in-air gesturing has been an ongoing project at Apple for both Apple's future mixed reality headsets and Macs.

Apple's patent covers gestures that can be performed by a user in order to engage interactive items presented on a display coupled to a computer executing a user interface that includes three-dimensional (3D) sensing.

A user can interactively select given items simply by gazing at them and manipulating the given interactive item by performing two-dimensional (2D) gestures on a tactile input device, such as a touchscreen or a touchpad.

In some embodiments a future Mac could be able to define virtual surface that emulates a touchpad or a touchscreen. The virtual surface can be implemented on a physical surface such as a book or a desktop, and the user can interact with the user interface by performing 2D gestures on the physical surface.

In alternative embodiments, the virtual surface can be implemented in space in proximity to the user, and the user can interact with the computer by performing 3D gestures.

In further embodiments, when configuring the physical surface as a virtual surface, the physical surface can be configured as a single input device, such as a touchpad. Alternatively, the physical surface can be divided into physical regions, and a respective functionality can be assigned to each of the physical regions.

For example, a first physical region can be configured as a keyboard, a second physical region can be configured as a mouse, and a third physical region can be configured as a touchpad.

Apple's patent FIG. 1 below is a schematic, pictorial illustration of a computer system implementing a non-tactile three-dimensional (3D) user interface; FIG. 3 is a flow diagram that schematically illustrates a method of detecting gazes and gestures.

Apple's patent FIGS. 8A-8D below are schematic pictorial illustrations of physical regions on a physical surface;

Apple's patent FIG. 11 above is a schematic pictorial illustration showing how a "pie menu" may be incorporated into the drawing application; FIGS. 12A and 12B are schematic pictorial illustrations of the physical surface illuminated by a projector.

Apple has added specific gestures and/or in-air gestures to their patent claims that they may decide to implement in the future including tap, press and hold, slide to drag, swipe, pinch, grab, rotate, swipe from edge gestures.

To review Apple's latest version of this invention, review Apple's patent application number 20220164032.