Today the US Patent & Trademark Office published a patent application from Apple that relates to modifying displays in future AR Glasses or MR Headset (HMD) so that they function accurately when viewing AR content in both bright and low light situations.

Apple's patent background provides us with an overview of the issues that their invention is to overcome. Apple notes that in augmented reality (AR), computer-generated content is composited with a user's physical environment in order to comingle computer generated visual content with real-world objects.

A user may experience AR content by wearing a head-mountable device (HMD) that includes a translucent or transparent display which allows the passing-through of light from the user's physical environment to the user's eyes. The HMD operates as an additive display by adding computer-generated AR content to the light from the user's physical environment.

In some circumstances, however, light from the physical environment has a color composition and/or brightness that interferes with computer-generated content in a manner that degrades the AR experience.

For example, light from the physical environment limits a level of contrast between the physical environment and displayed computer-generated content.

As another example, color composition of the physical environment, such as the presence of predominantly one color, may interfere with the color composition of displayed computer-generated content by providing dominant hues that are difficult to mask using additive display methods and hardware.

Some previously available systems include a physical fixed dimming layer that is integrated with a translucent display and the physical environment. Similar to sunglasses, the physical fixed dimming layer functions to block a certain amount light from passing through to the display. However, the display displays a constantly darker version of the user's physical environment, thereby degrading the user's experience and preventing use of such systems in low light situations.

Apple's patent covers a method which includes sensing a plurality of light superposition characteristic values associated with ambient light from a physical environment.

The ambient light emanates from the physical environment towards one side of the translucent display. The plurality of light superposition characteristic values quantifies interactions with the ambient light.

The method includes determining a plurality of display correction values associated with the electronic device based on a function of the plurality of light superposition characteristic values and predetermined display characteristics of a computer-generated reality (CGR) object.

The method includes changing one or more display operating parameters associated with the electronic device in accordance with the plurality of display correction values in order to satisfy the predetermined display characteristics of the CGR object within a performance threshold.

In Apple's patent Figures below (FIG. 2A, 2C and 2F) they're conveying that light from the physical environment (#200) is interfering with the display of a Computer Generated Reality (CGR, or AR) object, in this case a dog. According to Apple, "The red fire hydrant (#220) and the blue wall (#222) include different patterns (e.g., the red fire hydrant includes a horizontal-line pattern whereas the blue wall includes a cross-hatch pattern) in order to indicate that they are different colors.

As illustrated in FIG. 2C, the HMD is adding a CGR dog (#230) to the translucent display (#214), as indicated by the plus sign, which is illustrated for purely illustrative purposes. The CGR dog is white in color. In other words, the HMD renders the CGR dog as white and thus the preferred display color of the CGR (or AR) dog once added to the translucent display is likewise white.

As illustrated in patent FIG. 2F above, the sunlight reflecting off of the blue wall interferes with display of the CGR (AR) dog. Namely, the blue wall overlaps with a second portion (#242) of the CGR (AR dog) as indicated by a lighter cross-hatch pattern within the second portion (as compared with the cross-hatch pattern of the blue wall). Accordingly, the blue color of the blue wall mixes with the white color of the second portion of the CGR (AR) dog, causing the second portion of the CGR (AR) dog to be light blue rather than the preferred white color of the remainder of the AR dog.

Jumping to Apple's patent FIG. 4D above we're able to see artifact #420 that may appear resulting from the modified rendering of the AR dog. As another example, in some implementations, instead of or in addition to modifying color compositions, the HMD tints the area #420 (e.g., performs local tinting), such as reducing the luminance of the portion of the blue wall corresponding to area #420. Accordingly, as illustrated in FIG. 4D, the AR dog added to the translucent display appears to be desired white color because the magnitude of the blue light reflecting off of the blue wall is reduced.

Apple's patent FIG. 5 below is an example of a flow diagram of a method (#500) of changing display operating parameters based on determined display correction values in accordance with some implementations.

(Click on image below to Enlarge)

For more details, review Apple's patent application number 20210157143 titled "Modifying Display Operating Parameters based on Light Superposition from a Physical Environment," here.

Considering that this is a patent application, the timing of such a product to market is unknown at this time.