The U.S. Patent and Trademark Office officially published a series of 47 newly granted patents for Apple Inc. today. In this particular report we cover Apple's patent titled "Pointing-Based Display Interaction." This granted patent is one that Apple gained when acquiring Israeli firm PrimeSense. The granted invention covers interacting with a computer or television in a multimodal manner using gaze and pointing interactive controls to manipulate and interact with onscreen menus, interactive games and more. Yesterday CNBC interviewed Cyrus Mewawalla, head of TMT thematic research at CM Research who surprisingly still believes that Apple is secretly working on their own television that would disrupt the status quo. Many of PrimeSense's inventions that Apple acquired cover this area of technology extensively. Is the Apple HDTV still on Apple's unofficial road map? Only time will tell.

Granted Patent: Pointing-Based Display Interaction

Apple's newly granted patent covers their invention relating to human-machine interfaces, and specifically to interfaces that combine multiple user interaction modalities.

More specifically, Apple's granted patent covers a method including receiving a three-dimensional (3D) map of at least a part of a body of a user of a computerized system, receiving a two dimensional (2D) image of the user, the image including an eye of the user, extracting, from the 3D map and the 2D image, 3D coordinates of a head of the user, and identifying, based on the 3D coordinates of the head and the image of the eye, a direction of a gaze performed by the user.

A second method covers receiving an image including an eye of a user of a computerized system, identifying, based the image of the eye, a direction of a gaze performed by the user, identifying, based on the direction of the gaze, a region on a display coupled to the computerized system, and performing an operation on content presented in the region.

A third method covers presenting, by a computerized system, multiple interactive items on a display coupled to the computer, receiving, from a sensing device coupled to the computer, an input representing a gaze direction of a user, identifying a target point on the display based on the gaze direction, associating the target point with a first interactive item appearing on the display, and responsively to the target point, opening one or more second interactive items on the display.

A fourth method covers receiving and segmenting a first sequence of three-dimensional (3D) maps over time of at least a part of a body of a user of a computerized system in order to extract 3D coordinates of a first point and a second point of the user, the 3D maps indicating a motion of the second point with respect to a display coupled to the computerized system, calculating a line segment that intersects the first point and the second point, identifying a target point where the line segment intersects the display, and engaging an interactive item presented on the display in proximity to the target point.

Apple's patent FIG. 8 below is a schematic pictorial illustration of the user performing a Point-Select gesture to select a first given interactive item; FIG. 9 is a schematic pictorial illustration of the user performing a Point-Touch gesture to manipulate a second given interactive item; and FIG. 10 is a schematic pictorial illustration showing an alternative Point-Select gesture, also referred to herein as a Trigger gesture.

The trigger gesture is later explained as being used in interactive video games and other virtual worlds.

Apple's patent FIG. 1 noted below is a schematic, pictorial illustration of a computer system implementing a mixed-modality user interface; FIG. 3 is a flow diagram that schematically illustrates a method of detecting a direction of a gaze of a user.

Apple's patent FIGS. 6A-6C noted below are schematic representations illustrating a sequence of operations performed using the gaze operated user interface on a TV or computer display.

Apple's granted patent 9,342,146 was originally filed in Q3 2013 and published today by the US Patent and Trademark Office.

Patently Apple originally pointed to this Primesense patent being acquired by Apple back in a 2013 report titled "Take a Peek at a Few Key PrimeSense Patents that Apple Gained." At the time the patent was noted as being number 20130321271. Parts of this invention were also covered in a secondary PrimeSense patent that was just granted to Apple for "Gaze Detection in a 3D Mapping Environment" back in March.

Patently Apple presents only a brief summary of granted patents with associated graphics for journalistic news purposes as each Granted Patent is revealed by the U.S. Patent & Trademark Office. Readers are cautioned that the full text of any Granted Patent should be read in its entirety for full details. About Making Comments on our Site: Patently Apple reserves the right to post, dismiss or edit any comments.