With next-gen Mixed Reality headsets and AR smartglasses being developed by Apple, Google, Facebook and many other players, one of the focuses these companies are working on includes interfaces that could be used in sync with in-air hand or finger gestures to control UIs, move virtual objects and more. To date, Patently Apple has covered 59 Apple patents covering in-air gestures.  Apple's latest patent on this that was posted last month was titled "A new Apple Patent deeply describes the use of In-Air Gesturing to control a new kind of Mixed Reality Headset Interface."

In a new Google patent published last month in Europe titled "Gesture-Triggered Augmented Reality," Google engineering teams are hard at work to create AR Glasses that will work with accessory devices that may include a smartwatch, smart band, smart armband, smart bracelet, smart earbuds and a smart ring.

In at least one aspect, Google's patent application generally describes an augmented reality (AR) system that includes at least one head-mounted computing-device (e.g., AR glasses, AR earbud). The head-mounted computing-device includes a camera, which can be configured into an ON-mode to capture images or an OFF-mode to not capture images.

The head-mounted computing-device further includes a processor (i.e., a first processor) that is configured by software instructions (e.g., stored in a non-transitory computer readable medium) to receive a trigger signal from at least one limb-mounted computing device such as an upper-limb-mounted (mountable) computing-device, which indicates that the upper-limb-mounted computing-device has detected an AR-initiation gesture. Then, based on the trigger signal, the processor can configure the camera of the head-mounted computing-device from the OFF-mode to the ON-mode. The head-mounted computing-device may receive the trigger signal directly or indirectly (e.g., via a smartphone) from the limb-mounted computing device.

Further, upper-limb-mounted computing-devices includes a gesture sensor that is configured to measure gesture data, which corresponds to movements of the upper-limb-mounted computing-device.

The upper-limb-mounted computing-device further includes a processor that is configured by software instructions (e.g., stored on a non-transitory computer-readable memory) to receive the gesture data from the gesture sensor; detect an AR-initiation gesture; generate a trigger signal indicating that the AR-initiation gesture has been detected; and transmit the trigger signal to a head-mounted computing-device. The trigger signal can configure the head-mounted computing-device in an ON-mode to capture images.

Google's patent FIG. 1 below is a block diagram of an augmented-reality (AR) system; FIG. 3 illustrates a perspective view of an AR system operated by a user.

(Click on image to Enlarge)

Google's patent FIG. 4A below illustrates a first possible implementation of AR-glasses as a head mounted computing-device (HMD).

(Click on image to Enlarge)

The head-mounted computing-device of the AR system may not be required to function as a heads-up display. Google's patent FIG. 5 below illustrates a second possible implementation of a head-mounted computing-device.

As shown, the head-mounted computing-device may be an AR ear-mounted sound device (i.e., earbud). The AR earbud #500 may include an earpiece that is configured for mounting inside an ear of a user. The AR earbud 500 may include a speaker that is configured to present audio to a user. The AR earbud may also include a camera. The camera may be aligned so that when the AR earbud is worn in the ear of the user #510, the camera images the field of view #520 of the user. Accordingly, the AR earbud may analyze images captured by the camera and transmit AR audio to a user based on this analysis.

Google's patent FIG. 6 above illustrates a first possible implementation of an upper-limb-mounted computing-device. As shown, the upper-limb-mounted computing-device may be a smart watch.

The smart watch includes a gesture sensor, such as described above. In one possible implementation, the smart watch includes a processor configured to analyze the measured gesture data to detect an AR-initiation gesture.

For example, the AR-initiation gesture may be a raising of the wrist portion. In another possible implementation, the smart watch transmits the measured gesture data to a smart phone for analysis. In this implementation, a processor of the smart phone may be configured to analyze the gesture data to detect an AR-initiation gesture. This implementation may advantageously utilize larger processing and/or battery resources of the smartphone. The present disclosure recognizes that other variations to optimize resources may exist.

Google's patent FIG. 7 above illustrates a second possible implementation of an upper-limb-mounted computing-device. The upper-limb-mounted computing-device may be a smart armband or bracelet.

The plurality of gesture sensor may be used to determine hand gestures by measuring muscle movement of the forearm. Accordingly, the AR-initiation gesture may include movements of the forearm associated with raising the hand or may include forearm movements associated with a hand gesture, such as waving the hand.

Google's patent FIG. 8 above illustrates a third possible implementation of an upper-limb-mounted computing-device. As shown, the upper-limb-mounted computing-device may be a smart ring.

The smart ring 800 may be worn on the finger of a user and may include an interface switch 801 (e.g., a button). The smart ring may include a gesture sensor, as described above, and may process gesture data to detect an AR-initiation gesture or may transmit gesture data to another device (e.g., smartphone). The AR-initiation gesture may include movements of the finger associated with raising the hand or may include finger movements associated with a hand gesture, such as waving the hand.

Though not illustrated, Google notes that AR glasses may include an interface device (e.g., button, touchpad, switch, etc.) that a user can manually control to trigger the camera to transition from the OFF-mode to the ON-mode (i.e., woke form sleeping). The camera can be triggered to transition from the ON-mode to the OFF-mode (i.e., put to sleep) by manual control as well.

For more details about this patent, see the World Intellectual Property Organization (WIPO) patent number WO2022087566.