While Apple Vision Pro introduced ground breaking in-air gesture accuracy for controlling visionOS, Microsoft thinks that they have a better idea in the form of using tongue-gestures to control an operating system and/or applications. They even envision using tongue gestures to type on a virtual keyboard.

Microsoft's patent notes that while speech recognition and gaze tracking can be employed as alternative hands-free techniques for providing input to a computer, these techniques also have certain limitations. The following invention provides a next-gen form of input with the user's tongue.

Microsoft's patent application generally relates to techniques for detecting tongue gestures. One example relates to a method or technique that can include receiving one or more motion signals from an inertial sensor. The method or technique can also include detecting a tongue gesture based at least on one or more motion signals and outputting the tongue gesture.

Another example relates to a method or technique that can include instructing a user to perform a particular tongue gesture and measuring one or more motion signals from an inertial sensor while the user performs the particular tongue gesture. The method or technique can also include training a machine learning model to detect the particular tongue gesture using the one or more motion signals and outputting the trained machine learning model.

Another example includes a system that can include an inertial measurement unit, a processor, and a storage medium. The inertial measurement unit can be configured to provide motion signals. The storage medium can store instructions which, when executed by the processor, cause the system to detect a tongue gesture based at least on the motion signals and control an application based at least on the tongue gesture.

Microsoft's patent FIGS. 2A-2D below illustrate side views of example tongue gestures, consistent with some implementations of the present concepts.

Microsoft's FIG. 4 above illustrates an example application scenario #400 that conveys how various sensing modalities can be combined to allow users to control an application. Here, the user is shown a virtual keyboard #401 to enter text into a text entry box #402. The user can scan with their eyes over individual letters of the virtual keyboard. As the user scans, gaze tracking using one or more sensors can be employed to determine which letter the user's gaze is directed, and that letter can be visually distinguished (e.g., bolded, enlarged, etc.) to inform the user which letter they have currently targeted. When the user reaches the letter they wish to enter, they can perform a specific tongue gesture (e.g., a single upper teeth tap) to enter that letter. Another gesture (e.g., a left cheek tap) could be employed to backspace over the letter that has been entered.

For more on this, review Microsoft's patent application 20240085985 titled "Inertial Sensing of Tongue Gestures," published in the U.S. on March 14, 2024.

Perhaps if I tried this tongue gesture concept it would make sense but for now, on paper, it sounds like an off-the-wall concept that's destined for File-13.