During Apple's introduction of Vision Pro in June, Mike Rockwell, Executive VP, Advanced Technology Group introduced visionOS and noted that the new operating system added a host of new capabilities to support a dynamically foveated rendering pipeline that delivers maximum image quality.

While foveated displays are usually aimed for VR/XR headsets, a newly granted Apple patent covers a foveated display for possible future Macs, iPads and more.  

Overall, Apple's granted patent covers an electronic device that includes a display and an eye tracker configured to collect eye tracking data regarding a gaze of one or more of a user's eyes across the display.  The electronic device includes processing circuitry that is operatively coupled to the display and configured to foveate one or more areas of the display according to the eye tracking data. If the eye tracking data input is lost, the processing circuitry is configured to recover from the loss of eye tracking data by changing one or more aspects of the foveated areas (e.g., size, resolution, etc.) until a threshold is satisfied. As time elapses since loss of eye tracking, the foveated areas move toward a center or a salient region of the display.

As noted in Apple's patent FIG. 1 illustrates a block diagram of an electronic device that may provide recovery techniques when eye tracking is lost for a foveated display; FIGS. 2, 4 and 5 illustrate a sample of devices the patent if aimed at, with the MacBook being one of the devices most focused on.  

Apple's patent FIG. 7A above is a diagram representative of the electronic display utilizing static foveation. In static foveation, a size and a location of the various resolution areas of the electronic display 18 are fixed; FIG. 7B is a diagram that illustrates the electronic display utilizing dynamic foveation. The diagram includes a first frame #72, a second frame #74, and a third frame #76, each having a high resolution area #78, a medium resolution area #80, and a low resolution area #82;  FIG. 11 is a flow chart depicting operations to recover from loss of eye tracking.

Further, Apple notes that for dynamic foveation, an eye tracking system is used to determine a focal point of the eyes of the user on the electronic display. That is, a continuous input from the eye tracking system is provided to a foveation system and used to determine the size and location of the high resolution area on the electronic display. If the eye tracking system is not able to determine a focal point of the eyes of the user or if a connection to the eye tracking system is interrupted, the areas of varying resolution may no longer correspond to the focal point of the eyes of the user. Without the input, the foveation system may no longer function and may cause issues with a quality of an experience of a user or viewing comfort because the high resolution area of the display maintains the same location regardless of the focal point of the eyes of the user. Thus, a failure of the eye tracking system may cause a reduction in image quality on the display as perceived by the user.

pple's granted patent provides techniques to reduce an occurrence of the reduction in image quality of the image on the display. Specifically, embodiments presented herein provide techniques for foveation of a display when eye tracking is not available or when an error in eye tracking occurs.

For more details, review Apple's granted patent 11789529 titled "Recovery From Eye-tracking Loss In Foveated Displays."

Foveated Imaging Applications for computer displays include video gaming, simulation and more. Apple notes in this patent that " electronic displays may also use dynamic foveation. In dynamic foveation, the areas an electronic display which the various resolutions are used may change between two or more images based on the focal point of the eyes of the user. As an example, content that uses multiple images, such as videos and video games, may be presented to viewers by displaying the images in rapid succession.