Yesterday the U.S. Patent and Trademark Office officially published a series of 71 newly granted patents for Apple Inc. In this particular report we briefly cover Apple's granted patent titled "Glint-Assisted Gaze Tracker" that relates to Apple Vision Pro. And as always, we wrap up this week's granted patent report with our traditional listing of the remaining granted patents that were issued to Apple this week.

Glint-Assisted Gaze Tracker

Apple's granted patent covers methods and apparatus for glint-assisted gaze tracking in VR/AR head-mounted displays (HMDs). Images captured by gaze tracking cameras may be input to a glint detection process and a pupil detection process, for example implemented by one or more processors of a controller of the HMD.

The glint detection process may detect glints in the images and pass the glint information to the pupil detection process, where the detected glints may be used in detecting the pupil location and contour. The glint information may also be passed by the glint detection process to a glint-LED matching process that matches the detected glints to particular ones of the light-emitting elements of the gaze tracking system. Results of the glint-LED matching process (detected glints and LED correspondences) and pupil detection process (detected pupil ellipse) are passed to a gaze estimation process, for example implemented by one or more processors of the controller, to estimate the user's point of gaze.

In the gaze estimation process, a 3D cornea center estimation process estimates the center of the user's cornea in 3D space based on the detected glints and LED correspondences and user calibration data representing the specific user's eye parameters. A 3D pupil center estimation process estimates the center of the user's pupil in 3D space based on the detected pupil ellipse, the user calibration data, and output of the cornea center estimation process. An optical axis reconstruction process reconstructs the optical axis of the user's eye (the axis connecting the cornea center and the pupil center) in 3D space based on output of the cornea center estimation process and the pupil center estimation process. A visual axis reconstruction process reconstructs the visual axis of the user's eye (the axis connecting the fovea and the cornea center) in 3D space based on output of the optical axis reconstruction process and the user calibration data. A distorted display point estimation process estimates a point on the HMD display (the point of gaze) based on the output of the visual axis reconstruction process and the device-specific HMD calibration data.

Apple's patent FIGS. 3A, 3B, 3C and 3D below show side and front views of example HMDs that implement a gaze tracking system.

Apple's patent FIGS. 9A through 9C graphically illustrate glint-LED matching in image space; FIGS. 11A and 11B show example results of a glint matching in image space method compared to results when a glint geometric matching method is applied in 3D space to detect and correct potential mismatches using the glint matching in image space method.

Apple's patent FIG. 12A below illustrates a model of a human eye in relation to a display of an HMD; FIG. 12B illustrates a mathematical model for pupil center estimation; and FIG. 12C illustrates a mathematical model for cornea center estimation.

Review granted patent 11755106 for more details including patent claims. Other HMD related patents include Noise Mitigation that was granted yesterday and Optical Systems with Scanning Mirror Input Couplers that was published back on August 29^th. 

This Week's Remaining Granted Patents