On October 25, 2023 the Apple Watch double tap gesture was introduced. Apple noted at the time that "The Neural Engine processes data from the accelerometer, gyroscope, and optical heart sensor with a new machine learning algorithm. The algorithm detects the unique signature of tiny wrist movements and changes in blood flow when the index finger and thumb perform a double tap."

When CNET reviewed the new Apple Watch feature they noted that "Companies like Meta have already outlined a future where wrist trackers and headsets intertwine. In Meta's vision, neural input technologies like electromyography (EMG) will accurately detect hand motions with subtle accuracy." 

On that note, it should be noted that Apple is already working on using EMG technology for gestures on Apple Watch as revealed in a new patent application titled "Machine Learning based Gesture Detection," published yesterday by the U.S. Patent and Trademark Office (USPTO).

Apple's patent point #14 covering patent FIG. 1 presented below states: "Sensor device #102 may include one or more electrodes configured as an electromyography (EMG) sensor. An EMG sensor may provide, for example, measurements of electrical signals in nerves that control skeletal muscles in the hand (#104), including skeletal muscles for finger #106 and thumb #108. An EMG sensor may provide an indication of degree of force applied by a skeletal muscle to a bone attached to it, and hence the EMG sensor data may suggest a direction, force, and/or speed of movement of a body part, such movement of the hand relative to wrist #110, or movement of the finger or thumb.

In an aspect, a collection of a plurality of sensors, such as an IMU, EMG and PPG, may all be housed in the single sensor device (Apple Watch #102). Integration of such multiple non-invasive sensors in a single device mounted or attached to a user's body part may provide a variety of data that may be integrated for improved gesture detection in a single device that is convenient to wear and use. However, embodiments of the current disclosure are not so limited. In other aspects, multiple sensors for a user's hand or other body part may be positioned in separate devices that each measure aspects of the same body part, and/or aspects that may be indicative of motion of the body part. For example, an IMU or EMG sensor may be included in car in headphones or earbuds worn by the user, and a camera worn on a head or another body part of the user may sense images of the hand of the user.

Apple's patent FIG. 2 above illustrates an example system for gesture recognition; FIG. 3 illustrates an example process for detecting a gesture.  

Like with most Apple inventions, they don't want to be limited by the main device, an Apple Watch. Apple notes that the technology could also apply to a laptop computer, a smartphone, a tablet device, a wearable device such as a goggles or smartglasses, an earbud or other audio device, a case for an audio device, and the like.

For full details, review Apple's patent application 20250004566. 

Some of Apple's Inventors

• Behrooz Shahsavari: Senior Manager - Generative Models & Spatial Computing (Vision Pro)
• Hojjat Seyed Mousavi: Manager - AI/ML R&D in multimodal sensing & HCI
• Software Engineering Manager
• Charlie Maalouf: Applied AI/ML Research
• Giovanni Agnoli: Engineering Manager
• Christopher Sandino: Sensing + ML Research