Apple wins a patent for modeling 3D images of a user's fingers so as to enable an XR Headset to recognize In-Air Gestures
Today the U.S. Patent and Trademark Office officially granted Apple a patent that relates to 3D modeling of fingers to be used in a Mixed Reality Headset for both recognizing finger gestures and visual touch detection.
3D Modeling of Fingers to be used in a Mixed Reality Headset for Object and Gesture Recognition
Apple's patent in-part describes generating a 3D model of fingers that could be used for touching virtual objects and for in-air gesturing for the headset to understand commands.
Apple notes in patent point #31 that "a user may spin a finger in front of a camera in order interact with a physical environment or to sign into an account" and so forth.
Apple's patent FIG. 4 presents a flow chart of an example technique for utilizing a 3D model of a fingertip for gesture recognition; FIG. 5 shows an example system diagram illustrating a system setup for generating a 3D model of a fingertip.
Further to FIG. 4, The hand model may include 3D models of a hand corresponding to various poses of the hand. The flowchart continues at #410, and the electronic device 100 (iPhone) updates the 3D model of the hand based on the first and second image. For example, a depth image may be obtained from a perspective such that a touch point between the touching object and the surface is occluded by the touching object (e.g., the fingertip touching a surface may be occluded by the hand from the point of view of the user).
For finer details, review Apple's granted patent US 11620790 B2.