Today the US Patent & Trademark Office published a continuation patent from Apple that relates to systems and methods for monitoring and detecting head gestures performed by a user wearing headphones/AirPods, smartglasses or a head mounted display device that is equipped with multiple light-sensing assemblies.

In-part, this is a patent fulfilled as Apple’s AirPods Pro 2 introduced a new hands-free Siri experience. Apple notes in their marketing that “With Siri Interactions, you can simply nod your head yes or gently shake your head no when Siri asks if you’d like to answer an incoming call, hear a text message, or manage a notification.

Further, in the future, Apple smartglasses will be able to provide biomedical data that is simply mind boggling in scope. 

Apple won a patent today that covers a method of detecting head gestures performed by a user's head wearing an electronic device. As noted above, this was in-part delivered yesterday with AirPods 4 Pro.

Apple’s granted patent notes that the head-wearable electronic device may include any suitable number of photodiodes or any other suitable type(s) of optical sensors, each of which may be provided at a respective different location along the structure of the head-wearable device in order to sense light at a respective different position along the head of the user when the device is worn by the user's head.

Due to such positioning, the sensor data from the light sensors can capture movement of anatomical features in the tissue of the head of the user and can be used to determine any suitable gestures of the user, where such user gestures or head gestures may refer to any suitable gestures (e.g., voluntary and/or involuntary gestures of any suitable portion(s) of a user's head) and/or motions and/or actions and/or vocalizations and/or emotions and/or thoughts and/or brain functions and/or heart rate characteristics and/or other biometric characteristics of the user.

Further, different light emitters of the device can emit light at different wavelengths (e.g., infrared (“IR”) light, green light, etc.), which may penetrate to different depths in the tissue of the user's head before reflecting back to or otherwise being detected by the photodiodes of the device.

Accordingly, sensor data from the photodiodes can capture expansion, contraction, and/or any other suitable movement in the tissue of the user during a head gesture. Various different head gestures, including, but not limited to, chewing, blinking, winking, smiling, eyebrow raising, jaw motioning (e.g., jaw protrusion, jaw retrusion, lateral jaw excursion, jaw depression, jaw elevation, etc.), mouth opening, and/or the like, may be detected by recognizing patterns in sensor data that may be characteristic of each head gesture, as head tissue may expand and contract and anatomical features in the tissue may move during such user gestures.

Apple’s patent FIG. 1 below is a schematic view of an illustrative system 1 that includes a head-wearable electronic device #100 for monitoring a user. Head-wearable electronic device may be any suitable electronic device that may include at least one light-sensing assembly and that may be at least partially worn on any suitable portion of a user's head. For example, head-wearable electronic device may include, but is not limited to, a helmet, eyeglasses (e.g., sunglasses, reading glasses, novelty glasses, augmented reality eyeglasses, etc.), a headset, headphones, earphones, over-ear speakers, in-ear speakers, a head-mounted display device (e.g., any monocular or binocular or optical head-mounted display device for virtual reality applications or any other suitable use), goggles, a hat, a headband, a mask, a hood, a head chain, earrings, earmuffs, or any combination thereof that may be at least partially worn on a user's head and operative to position at least one light-sensing assembly against or facing or otherwise proximate any suitable portion of the user's head for sensing light that may be reflected thereby and/or transmitted therethrough.

Apple’s patent FIG. 2A above illustrates a future pair of smartglasses that supports head-gesturing and more.

Apple’s patent FIGS. 4A-4E below illustrate exemplary head gestures in accordance with examples of the disclosure.

Further to the 4-Series patent figures above, Apple notes that sensor data from the photodiodes can capture expansion, contraction, and/or any other suitable movement in the tissue of the user during a head gesture. Various different head gestures, including, but not limited to, chewing, blinking, winking, smiling, eyebrow raising, jaw motioning (e.g., jaw protrusion, jaw retrusion, lateral jaw excursion, jaw depression, jaw elevation, etc.), mouth opening, and/or the like, may be detected by recognizing patterns in sensor data that may be characteristic of each head gesture, as head tissue may expand and contract and anatomical features in the tissue may move during such user gestures."

As another example, a method for monitoring a user wearing a head-wearable electronic device (HWD) on the user's head using a head gesture model custodian system is provided that may include initially configuring, at the head gesture model custodian system, a learning engine, receiving, at the head gesture model custodian system from the HWD, HWD sensor category data for at least one HWD sensor category for a head gesture and a score for the head gesture, training, at the head gesture model custodian system, the learning engine using the received HWD sensor category data and the received score, accessing, at the head gesture model custodian system, HWD sensor category data for the at least one HWD sensor category for another head gesture, scoring the other head gesture, using the learning engine for the HWD at the head gesture model custodian system, with the accessed HWD sensor category data for the other head gesture, and, when the score for the other head gesture satisfies a condition, generating, with the head gesture model custodian system, control data associated with the satisfied condition.

The Sensor Assembly

The sensor assembly is a key part of this invention. Apple notes that the sensor assembly #114 may include any suitable sensor components or subassemblies for detecting any suitable biometric data and/or health data and/or sleep data and/or mindfulness data and/or the like of a user of the Head Wearable Device (HWD 100).

These sensors can generate data providing health-related information associated with the user. For example, PPG sensors can provide information regarding a user's respiratory rate, blood pressure, heart rate ("HR"), heart rate variability ("HRV"), and/or oxygen saturation. ECG sensors can provide information regarding a user's heartbeats. GSR sensors can provide information regarding a user's skin moisture, which may be indicative of sweating and can prioritize a thermostat application to determine a user's body temperature. In some examples, each sensor can be a separate device, while, in other examples, any combination of two or more of the sensors can be included within a single device.

For example, a gyroscope, accelerometer, photoplethysmogram, galvanic skin response sensor, and temperature sensor can be included within a wearable electronic device, such as an HWD, while a scale, blood pressure cuff, blood glucose monitor, SpO2 sensor, respiration sensor, posture sensor, stress sensor, and asthma inhaler can each be separate devices.

To review the full details of this invention, check out granted patent 12086304.