One of Apple Vision Pro’s amazing capabilities is its eye-tracking system. And with all major Apple devices, there are dozens of patents, if not more, per feature. Today the US Patent & Trademark Office published a patent application from Apple that zeros in on a next-gen eye-tracking system that combines cameras with Self-Mixing Interferometry (SMI) sensors.

In Apple’s patent background they note that eye monitoring technologies can be used to improve near-eye displays (e.g., head-mounted displays (HMDs)), augmented reality (AR) systems, virtual reality (VR) systems, and so on. For example, gaze vector tracking, also known as gaze position tracking, can be used as an input for display fovea rendering or human-computer interaction. Traditional eye monitoring technologies are camera-based or video-based and rely on active illumination of an eye, eye image acquisition, and extraction of eye features such as a pupil center and cornea glints. The power consumption, form factor, computational cost, and latency of such eye monitoring technologies can be a significant burden for more user-friendly next generation HMD, AR, and VR systems (e.g., lighter weight, battery-operated, and more fully featured systems).

This is what Apple’s latest patent addresses by shifting away from camera-based eye-tracking to eye-tracking using Self-Mixing Interferometry (SMI) or combination camera/SMI.  

Eye Tracking Using Self-Mixing Interferometry

Apple’s patent covers systems, devices, methods, and apparatus described in the present disclosure utilize one or more self-mixing interferometry (SMI) sensors to track eye movement. In some embodiments, SMI sensors can be used alone, or in combination with a camera, to also determine a gaze vector or eye position.

In a first aspect, the present disclosure describes an eye tracking device. The eye tracking device may include a head-mountable frame, an optical sensor subsystem mounted to the head-mountable frame, and a processor. The optical sensor subsystem may include a set of one or more SMI sensors. The processor may be configured to operate the optical sensor subsystem to cause the set of one or more SMI sensors to emit a set of one or more beams of light toward an eye of a user; to receive a set of one or more SMI signals from the set of one or more SMI sensors; and to track a movement of the eye using the set of one or more SMI signals.

In a second aspect, the present disclosure describes another eye tracking device. The eye tracking device may include a set of one or more SMI sensors, a camera, and a processor. The processor may be configured to cause the camera to acquire, at a first frequency, a set of images of an eye of a user; cause a set of one or more SMI sensors to emit a set of one or more beams of light toward an eye of a user; sample, at a second frequency greater than the first frequency, a set of one or more SMI signals generated by the set of one or more SMI sensors; determine a gaze vector of the eye using at least one image in the set of images; and track a movement of the eye using the set of one or more SMI signals.

In a third aspect, the present disclosure describes a method of tracking movement of an eye. The method may include operating an optical sensor subsystem to cause a set of one or more SMI sensors in the optical sensor subsystem to emit a set of one or more beams of light toward an eye of a user; receiving a set of one or more SMI signals from the set of one or more SMI sensors; and tracking a movement of the eye using the set of one or more SMI signals.

Apple’s patent FIG. 1 below shows an example block diagram of an eye tracking device; FIG. 3A shows a first example eye tracking device in which an optical sensing subsystem and processor are mounted to a pair of glasses; FIG. 3B shows a second example eye tracking device in which an optical sensing subsystem and processor are mounted to a VR headset; FIG. 4A shows a side elevation view of an example set of SMI sensors that may be mounted to a head-mountable frame and configured to emit light toward an eye; FIG. 4B shows a front elevation view of the eye and set of SMI sensors shown in FIG. 4A.

Apple’s patent FIGS. 12A and 12B below illustrate how one or more surfaces or structures of an eye may be mapped using a set of one or more SMI sensors.

Apple’s patent FIG. 11B above illustrates an example method for tracking eye movement, using a set of one or more SMI sensors in combination with a camera or other sensors.

To review its full details, review Apple’s patent application 20230333371. A few other recent eye-tracking system including: 01, 02, 03.  For even more eye-tracking patents, review our HMD Archive,