Today the U.S. Patent and Trademark Office officially granted Apple a patent that relates primarily to HMDs (Vision Pro, Smartglasses), systems and methods for creating custom hand gestures to control specific functions that the user wants, like opening a specific application.

In Apple's patent background they note that in various implementations, an electronic device, such as Vision Pro, detects pre-defined hand gestures performed by a user and, in response, performs a corresponding function. However, in various implementations, a user may be physically unable to perform the pre-defined hand gesture or may desire that a hand gesture perform a different and specific function.

Method And Device For Defining Custom Hand Gestures

Apple's granted patent covers devices, systems, and methods for defining a custom hand gesture. In various implementations, the method is performed by a device including an image sensor, one or more processors, and non-transitory memory. The method includes receiving a request to define a first hand gesture.

The method includes capturing, via the image sensor, a first performance of the first hand gesture. The method includes extracting, from the first performance of the first hand gesture, a first set of features. The method includes defining, based on the first set of features, one or more gesture matching criteria for the first hand gesture.

Apple's patent FIGS. 4A-4F below illustrate an XR environment, based on a physical environment, including a graphical user interface for selecting a function and selecting a hand gesture to associate with the function.

While the patent is mainly focused on HMDs, the patent states that the invention could be applied to many devices including the following: head-mountable systems, projection-based systems, heads-up displays (HUDs), vehicle windshields having integrated display capability, windows having integrated display capability, displays formed as lenses designed to be placed on a person's eyes (e.g., similar to contact lenses), headphones/earphones, speaker arrays, input systems (e.g., wearable or handheld controllers with or without haptic feedback), smartphones, tablets, and desktop/laptop computers.

For full details, review granted patent 12045392.