Monkey See, Monkey Do: Meta files patent designed to mimic Apple Vision Pro's EyeSight Feature
Is it surprising that Meta has filed a patent designed to mimic Apple Vision Pro's EyeSight Feature? No. In fact, I wouldn't be surprised if the Samsung/Google XR headset coming this fall provides a similar feature. Meta Platforms patent is formally titled "Embedded Sensors in Immersive Reality Headset to enable Social Presence." It also confirms that a redesign of their Quest headset or an all-new headset design is being considered to support this feature. To distinguish it from Apple's Eyesight, Meta notes that some of the sensors in face interface my include health sensors.
Meta's Patent Abstract: A method for updating a subject avatar with a facial expression collected in real time. The method includes receiving, from a sensor on a facial interface of a headset, a signal indicative of a movement in a facial muscle from a user of the headset, determining a facial expression of the user with the signal from the sensor in the facial interface of the headset, based on a machine learning algorithm trained to associate the facial expression to the movement of the facial muscle, adjusting a subject avatar for the user of the headset based on the facial expression, and providing the subject avatar to an immersive reality application hosted by a remote server. A headset, a computer-readable medium storing instructions which, when executed by a processor causes the headset to perform the above method, and the processor, are also provided.
Meta's patent FIG. 2 below illustrates a facial interface for a headset, including multiple sensors. Some of the sensor may include ECG, EEG and PPG sensors to assess cardiovascular conditions of a user; FIG. 3 illustrates a chart of facial expressions associated to facial muscles that activate them.
Meta's patent FIG. 4 above is a flow chart illustrating steps in a method for updating subject avatars in real time for immersive reality applications.
For full details review Europe's patent #WO2024102264.