The U.S. Patent and Trademark Office officially published a series of 48 newly granted patents for Apple Inc. today. In this particular report we cover Apple's work on 3D mapping in relation to controlling a TV or computer display radial menu to access games, mail, Safari, office and general applications. With Apple launching the iPhone 7 with their first dual lens camera capable of 3D depth mapping, they've moved one step closer to the day when an iMac display or TV could provide added controls by using simple in-air gestures and gaze controls to operate onscreen menus, apps and content as today's invention confirms.

Granted Patent: Gaze-Based Display Control

Apple's newly granted patent covers their invention relating to human-machine interfaces, and specifically to interfaces that combine multiple user interaction modalities.

More specifically, Apple's granted patent covers a method including receiving a three-dimensional (3D) map of at least a part of a body of a user of a computerized system, receiving a two dimensional (2D) image of the user, the image including an eye of the user, extracting, from the 3D map and the 2D image, 3D coordinates of a head of the user, and identifying, based on the 3D coordinates of the head and the image of the eye, a direction of a gaze performed by the user.

A second method covers receiving an image including an eye of a user of a computerized system, identifying, based the image of the eye, a direction of a gaze performed by the user, identifying, based on the direction of the gaze, a region on a display coupled to the computerized system, and performing an operation on content presented in the region.

Apple was granted a similar patent back in May. Even though the two granted patents share common patent figures, the emphasis of the first granted patent was on 'pointed-based display interaction'' that supported patent FIGS. 8, 9 and 10 noted further below.

Todays granted patent 9,454,225 focuses on 'Gaze-based display control.' The big differentiator between the patents can be found in each of their respective patent claims. The patent in May had 23 patent claims focused on 3D maps of a user's hand and more specifically from the knuckle to the tip of a finger that would be used to point to items on a TV or computer display as if it were a human cursor. Today's granted patent carries 9 patent claims and is focused on the start button present in a radial menu located at the bottom right corner of a display as noted in patent FIGS. 6A and 6B below. This is where the user would begin to control a display based on gaze controls.

Apple's patent FIG. 1 noted above is a schematic, pictorial illustration of a computer system implementing a mixed-modality user interface; FIG. 3 is a flow diagram that schematically illustrates a method of detecting a direction of a gaze of a user.

Apple's patent FIG. 8 is a schematic pictorial illustration of the user performing a Point-Select gesture to select a first given interactive item, in accordance with an embodiment of the present invention; FIG. 9 is a schematic pictorial illustration of the user performing a Point-Touch gesture to manipulate a second given interactive item; FIG. 10 is a schematic pictorial illustration showing an alternative Point-Select gesture, also referred to herein as a Trigger gesture.

Apple's granted patent 9,454,225 was originally filed in Q3 2013 and published today by the US Patent and Trademark Office. The engineers listed on this granted patent were formerly a part an Israeli company called PrimeSense that Apple acquired. The engineers continue to work at Apple's Israeli facilities.

Patently Apple presents only a brief summary of granted patents with associated graphics for journalistic news purposes as each Granted Patent is revealed by the U.S. Patent & Trademark Office. Readers are cautioned that the full text of any Granted Patent should be read in its entirety for full details. About Making Comments on our Site: Patently Apple reserves the right to post, dismiss or edit any comments. Those using abusive language or behavior will result in being blacklisted on Disqus.