During the introduction of the Apple Vision Pro at this year's WWDC23, Apple introduced a specific feature called Eyesight. Apple's marketing material notes that "Apple Vision Pro helps you remain connected to those around you. EyeSight reveals your eyes and lets those nearby know when you’re using apps or fully immersed in an experience. When someone approaches, Apple Vision Pro simultaneously lets you see the person and reveals your eyes to them."

Technically speaking, Apple's Mike Rockwell described Eyesight this way: ""We also thought hard about how others can interact with you while wearing Vision Pro. Eyesight utilizes a unique curved OLED panel with a lenticular lens to project the correct perspective of your eyes to each person looking at you. The result is a 3D display that makes the device look transparent." Rockwell went on to describe how an enrollment process with Vision Pro would create an avatar of the user that strikingly resembles the Vision Pro user.

A new Microsoft patent discovered in Europe this weekend was published on November 16, 2023 titled "Controlling Computer-Generated Facial Experiences."

Microsoft notes that their invention relates to displaying computer-generated facial expressions. One example provides a method for displaying computer-generated facial expressions. The method comprises receiving expression data, and generating one or more facial expressions for an eye region of a user based at least on the expression data. The method further comprises displaying the one or more facial expressions for the eye region on an outward-facing display of a head mounted device. 

Microsoft's patent FIGS. 11A and 11B below illustrate an example scenario in which an outward-facing display on a head mounted device is activated and turns on computer-generated facial expressions on the display, especially in the eye region.

Microsoft's patent FIG. 12 above shows a block diagram of an example head-mounted device configured to display computer-generated facial expressions; FIG. 13 illustrates a flow diagram of an example method of displaying computer-generated facial expressions.

Examples are disclosed in the patent that relate to controlling the display of computer-generated facial expressions utilizing a facial tracking sensor system comprising one or more facial tracking sensors.

Further, values are received from each facial tracking sensor over time, and a value range is determined for each facial tracking sensor based upon the data received, the value range comprising minimum and maximum values received during a period of time (e.g. during a rolling time window). The value range and incoming sensor values are processed to translate the incoming sensor values to blendshape mappings, wherein the blendshape mappings correspond to locations of the sensor values within the value range.

Then, expression data is determined based at least upon the blendshape mapping, and is provided to one or more devices for presentation. In such a manner, a set of facial tracking sensors can be used together to sense overall approximation of an expression of a user and thereby control the expressions presented in the form of an avatar. The avatar may be presented to other people than the user of the sensor device. This may help with interpersonal communication in an AR (augmented reality) and/or VR (virtual reality) environment.

For more on this, review the full European patent #WO/2023/219685. Microsoft's patent strongly suggests that they may be working on an XR Headset beyond HoloLens. Today's HoloLens allows outsiders to see a HoloLens user's eyes. So this invention is clearly not for HoloLens.

In August 2022 Patently Apple posted an IP report showing that Facebook had invented a method to view the user's eyes in a headset as presented in the patent figure below.