The U.S. Patent and Trademark Office officially published a series of 44 newly granted patents for Apple Inc. today. In this particular report we cover another PrimeSense invention that illustrates how a computer system could implement a non-tactile three-dimensional (3D) user interface. The invention illustrates how the 3D interface could be used to advance the iMac with a gaze-enhanced virtual touchscreen.

Granted Patent: Gaze-Enhanced Virtual Touchscreen

Apple's newly granted patent generally relates to user interfaces for computerized systems, and specifically to user interfaces that are based on three-dimensional sensing. Apple has many such patents on record since acquiring PrimeSense (one, two, three, four and five). Today's granted patent introduces many new facets to this technology as it relates to what it could do with a future desktop.

Apple's granted patent FIG. 1 noted below is a schematic, pictorial illustration of a computer system implementing a non-tactile three-dimensional (3D) user interface; FIG. 3 FIG. 3 is a flow diagram that schematically illustrates a method of detecting gazes and gestures.

Apple's granted patent covers a method, including presenting, by a computer, multiple interactive items on a display coupled to the computer, receiving an input indicating a direction of a gaze of a user of the computer, selecting, in response to the gaze direction, one of the multiple interactive items, subsequent to selecting the one of the interactive items, receiving a sequence of three-dimensional (3D) maps containing at least a hand of the user, analyzing the 3D maps to detect a gesture performed by the user, and performing an operation on the selected interactive item in response to the gesture.

There is also provided, in accordance with an embodiment of the present invention an apparatus, including a sensing device configured to receive three dimensional (3D) maps containing at least a head and a hand of a user, and to receive a two dimensional (2D) image containing at least an eye of the user, a display, and a computer coupled to the sensing device and the display, and configured to present, on the display, multiple interactive items, to receive an input indicating a direction of a gaze performed by a user of the computer, to select, in response to the gaze direction, one of the multiple interactive items, to receive, subsequent to selecting the one of the interactive items, a sequence of the 3D maps containing at least the hand of the user, to analyze the 3D maps to detect a gesture performed by the user, and to perform an operation on the selected one of the interactive items in response to the gesture.

Apple's granted patent FIGS. 8A-8D noted above are schematic pictorial illustrations of a physical regions #142 and #144 on a physical surface. In the example shown in FIG. 8, the computer configures region #142 as a virtual touchscreen, and configures region #144 as a virtual mouse.

In FIG. 8A, computer uses information provided by the 3D sensing device to detect the location of the user's hand and any fingers that are touching the virtual touchscreen by mapping a corresponding point on the display. Although the example in FIG. 8A shows a single finger in contact with region virtual touchscreen, the 3D sensing device and the computer can be configured to detect any number of fingers thereby enabling the user to perform complex, multi-finger control gestures, including scrolling, zoom, pan, and so forth. In some embodiments, the computer can configure the virtual touchscreen to be a virtual keyboard able to accept "input" from all fingers as noted in the figures below. .

In the patent figures below we see that that users will be able to use their fingers as a paint brush and the 3D camera will be able to present the information into an application.

While a user's hand is hovering over the virtual keyboard, they could use various gestures that could be understood by the OS via the system's 3D camera.

In Apple's patent FIGS. 13A and 13B we're able to see schematic pictorial illustrations showing one or more physical objects #190 positioned on physical surface while user performs a gesture, in accordance with an embodiment of the present invention. In some embodiments, the 3D projector can project, onto physical surface a respective contour image #192 encompassing each of the one or more physical objects, thereby indicating a location of each of the one or more physical objects.

In a way, it appears that Apple's proposed computer system could resemble aspects of HP's Sprout Computer that is designed to eventually work hand-in-hand with a 3D printer. 

Prior to Apple acquiring PrimeSense, the patent notes that this system was destined for Windows 8. Will Apple take advantage of this technology for the iMac? Only time will tell.

Apple's granted patent 9,377,863 was originally filed in Q1 2013 and published today by the US Patent and Trademark Office.

Patently Apple presents only a brief summary of granted patents with associated graphics for journalistic news purposes as each Granted Patent is revealed by the U.S. Patent & Trademark Office. Readers are cautioned that the full text of any Granted Patent should be read in its entirety for full details. About Making Comments on our Site: Patently Apple reserves the right to post, dismiss or edit any comments.