Today the US Patent & Trademark Office published a patent application from Apple that relates to a feature designed primarily for a future mixed reality headset or AR glasses that will avoid display burn-in and more.  

Display Operation Based on Eye Activity

Apple's patent application relates to an eye monitoring system built into the frame support system of a headset designed to detect eye saccades and eye blinks and then make needed adjustments to the eye displays in realtime without the user even knowing this is occurring in the background.

Saccades are fast, jerky and mostly ballistic eye rotations. Humans make several saccadic eye movements per second to utilize this highest-resolution part of the retina to look at the object of interest.

During saccades and blinks, a user's visual sensitivity is temporarily suppressed. Control circuitry in the headset may take advantage of momentary suppression of the user's visual sensitivity to make adjustments to the operation of the display.

For example, the control circuitry can coordinate operation of the display with periods of suppressed visual sensitivity that are associated with the saccades and blinks to reduce power consumption and/or to make potentially obtrusive image changes. By making adjustments during periods of suppressed visual sensitivity, these adjustments can be hidden from the user of the headset.

Control circuitry in the electronic device may coordinate operation of the display with periods of suppressed visual sensitivity that are associated with the saccades and blinks. By making adjustments to display circuitry and image content during periods of suppressed visual sensitivity, potentially visually obtrusive changes to displayed images can be hidden from a user of the electronic device. Adjustments to display operation may help to avoid or reduce burn-in effects, may help reduce power consumption, and may otherwise improve device performance.

Apple's patent FIG. 2 below is a top view of an illustrative eye monitoring system; FIG. 5 is a flow chart of illustrative operations involved in using an electronic device with a display and eye monitoring system.

Further to Apple's patent FIG. 2 above we see a diagram showing how the eye monitoring system (#16) may gather information on a user's eye (#30). The System may, if desired, include gaze tracking system components such as light-emitting diodes, lasers, or other light sources, image sensors, photodetectors, and/or other light-sensing devices, and/or other equipment for monitoring eye movements and blinks.

The Eye tracking in system may be performed by estimating the reflection of an infrared beam from the surface of the cornea ("corneal reflection"). The system may be a video-based eye tracker that uses the pupil center (without any reflection from the eye) to estimate gaze.

Like with most patents, Apple notes that the invention isn't limited to just glasses but could also be used in future systems such as a vehicle based heads-up display (Project Titan), Macs, a television and more.   

To explore more of the details of this invention, check out patent application number 20200019238 officially titled "Electronic Devices with Display Operation Based on Eye Activity," that was published today and originally filed in June 2019.

Apple's Inventors

Andrew Watson: Chief Vision Scientist

Mehmet N. Ağaoğlu: Vision Scientist/Display Engineer