Today the US Patent & Trademark Office published a patent application from Apple that relates to a future HMD configured to detect eye movements within an eye-tracking system. Gaze estimation can also be used for eye gesture recognition and to provide inputs to various user interfaces, such as to select or switch between settings, and to provide input shortcuts.

HMD Eye Detection Methods

Some devices provide eye tracking systems to enable human-computer interaction. In some uses, a device can capture the eye movement of a user, using the movement as an intuitive input for operating and invoking features of the device, allowing faster and more efficient interaction. Devices that decrease friction of interactions between a user and device are desirable.

Apple's invention is directed to a head-mounted device having a plurality of electrodes configured to detect optical events such as the movement of one or more eyes or coarse eye gestures.

In some examples, the one or more electrodes can be coupled to dielectric elastomer materials whose shape can be changed to vary contact between a user of the head-mounted device and the one or more electrodes to ensure sufficient contact and electrode signal quality.

In some examples, the one or more electrodes can be coupled to pressure sensors and control circuitry to monitor and adjust the applied pressure.

In some examples, the optical events can be used as triggers for operating the device, including transitioning between operational power modes.

In some examples, the triggers can invoke higher resolution sensing capabilities of the head-mounted device.

In some examples, the electrodes can be used as an on-head detector to wake-up and/or unlock the device.

As computer input technology continues to develop, various advances in input devices have allowed users to interact with computational systems more efficiently. Head-mounted devices including elements configured to detect movement of eyes are one example of an innovation that can further improve the speed, efficiency, and ease with which users can interact with one or more computational systems.

In other embodiments, the gaze of a user can be detected to seamlessly invoke functions of the device. For example, while a user's hands are otherwise occupied, one or more ocular events (e.g., eye blinks, eye focus, and/or the movement of (eye) gaze) can be monitored to wake-up and/or unlock the device. Gaze estimation can also be used for eye gesture recognition and to provide inputs to various user interfaces, such as to select or switch between settings, and to provide input shortcuts.

An ocular event can comprise one or more blinks, fixations and/or saccades of the eyes. It is further understood that ocular events can comprise a sequence of one or more ocular events (e.g., a sequence of one or more blinks, fixations, and/or saccades in some combination). Other use cases for the detection of ocular events includes on head device recognition (e.g., sense that the device is being worn on a user's head and turn on the device), accessibility for the disabled, and eye health.

In some embodiments, a head-mounted device can comprise electrodes configured to monitor electrical impulses corresponding to an ocular event. For example, electrodes coupled to dielectric elastomer (DE) materials can be arranged around features of the body including, but not limited to, in contact with and/or above the nose, behind and/or around the ears, the temples, and/or any other location suitable for detecting and capturing the electrical impulses associated with one or more ocular events or other brain related events and neural activities.

Apple's patent 1 below illustrates a symbolic hardware diagram of a system for detecting ocular events;  FIG. 2 illustrates a functional block diagram of a system for detecting ocular events.

Apple's patent FIG. 3 illustrates a plurality of electrodes at locations around the face and head of a user for detecting ocular events; FIG. 6A illustrates an example electrode configuration including one or more elements in an inactive state, but which can be activated to improve coupling between the electrodes and tissue, and additionally a pressure sensor co-located with the electrodes; FIG. 6B illustrates the example electrode and pressure sensor configuration of FIG. 6A in an activated state to improve coupling between the electrodes and tissue.

Apple's patent FIG. 7 above illustrates exemplary signal processing of electrode signals; and FIG. 10 illustrates a method of reduced power consumption eye tracking.

For more details, review Apple's patent application number US 20230102507.

Apple Inventors

• Erdrin Azemi: Senior Engineering Manager, AI/ML Research
• Ali Moin: AI Research
• Christoph Horst Krah: Electrical Engineer, Optics
• Kaan Dogrusoz: ML & Sensing R&D
• Moe (Mohammad) Yeke Yazdandoost: Tech Lead Sensing System Architect