Facebook Invents a VR Glove System that uses Millimeter Wave Transponders for Hand Tracking and more
Earlier in Q2 2020 the US Patent & Trademark Office published a patent application filed by Facebook for future VR Gloves that will use millimeter wave transponders instead of the current methodology of relying on inertial motion tracking or optical motion capture.
Facebook's patent states that in artificial reality systems, hand tracking and gesture recognition can be used for user input. For example, the artificial reality system can recognize known gestures and respond accordingly.
If a haptic glove is used, the artificial reality system can also provide haptic feedback to the user. For example, if a user makes a gesture that corresponds to picking up an object, a haptic tracking glove can track the user's motion, such as reaching and grasping, and provide tactile feedback so that the user senses an object is in their hand.
Facebook's invention covers an artificial reality system that can use signals re-radiated by transponders on a locatable glove, such as a haptic glove, to determine the location of the glove.
The re-radiated signals may differ in at least one characteristic, so that the artificial reality system can identify the signals from each individual positioning transponder and determine each individual positioning transponder's position. This allows more accurate determination of a hand position compared to prior techniques.
The transponders may re-radiate signals in a millimeter-wave band. The extremely high frequency (EHF) band, as designated by the International Telecommunication Union (ITU) encompasses frequencies from 30 GHz through 300 GHz, and wavelengths between 1 mm and 1 cm. Radio waves in or near this band are commonly referred to as millimeter-waves, or as being in the millimeter-wave band.
In some embodiments, the antennas may emit and receive waves having wavelengths outside the EHF band, such as any wavelength or band in the ultra-high frequency (UHF) band (1 meter to 1 decimeter), super high frequency (SHF) band (1 dm to 1 cm), or terahertz band (100 .mu.m to 1 mm).
In some embodiments, the signals may be received even if the positioning transponders are not in the direct line of sight of an interrogating antenna, e.g., through a garment or through the fabric of a glove. At some wavelengths, the signals may enable travel though objects or though the user's body, e.g., through the user's hand.
Facebook's patent FIG. 1 below is a block diagram of a system environment 100 in which a locatable garment 140 operates. The system environment 100 may be, for example, an artificial reality system; FIG. 3 illustrates a plan view of a locatable glove with positioning transponders controlled by switches.
Facebook's patent FIG. 4 below illustrates a locatable glove with positioning transponders and a pose determination device receiving signals from the positioning transponders; FIG. 5 is a flow chart showing a process for determining the pose of a user's hand.
Facebook's granted patent that was published by the U.S. Patent and Trademark Office (USPTO) in Q2 2020 was filed in Q3 2019.
Gesture ID System
Later in Q2 2020 USPTO published a secondary VR Gesture ID System the covered an arm and/or wrist device band. More specifically, Facebook's patent FIG. 1 below is a perspective view of an exemplary gesture identification system comprising a wearable gesture identification device that performs automated gesture identification with improved robustness against variations in use parameters and with limited computational resources in accordance with the present systems, devices, and methods.
A typical training procedure, carried out before the system is operable to identify gestures performed by the user (i.e., pre-runtime), requires the user to perform a series of training trials for multiple training gestures (i.e., multiple training trials for each one of multiple training gestures). The system calibrates use parameters based on the signals (e.g., the EMG signals) detected during the training gestures. The quality of the calibration typically increases with the number of training trials performed for each training gesture, so the training procedure may involve many training trials.
While the main purpose for the device is for a hand gesture recognition and virtual game control system based on 3D accelerometer and EMG sensors," the device could be used in the medical field and beyond.