Although visionOS for Apple Vision Pro can be controlled with simple in-air gestures, Apple continues to work on finger devices that could provide Vision Pro and future smartglasses with more sophisticated accuracy working with or manipulated 3D objects in addition to providing users with heightened haptic feedback that will provide users with the ability to feel objects in mixed reality applications, like a game and more. The advanced haptics could also provide users with a greater sensation of typing on a virtual keyboard.  

Finger-Mounted Device With Sensors And Haptics

Apple invention covers a finger-mounted device that may include finger-mounted units coupled to control circuitry. The control circuitry may wirelessly transmit information gathered with the finger mounted units to an external device to control the external device.

The control circuitry may also use the finger-mounted units to provide a user's fingers with feedback such as haptic feedback.

For example, the control circuitry may supply haptic output to a user's fingers based on wirelessly received information from the external device. The haptic output may correspond to virtual reality or augmented reality haptic output.

The body of each finger-mounted unit may have a U-shaped cross-sectional profile that leaves the finger pad of each finger exposed when the body is coupled to a fingertip of a user's finger. The control circuitry may gather finger press input, lateral finger movement input, and finger tap input using the sensors and may provide haptic output using the haptic output device.

In addition, a wearable device such as a finger-mounted device may be used to gather input from a user's fingers as the user interacts with surfaces in the user's environment and may be used to provide clicks and other haptic output during these interactions. The input that is gathered in this way may include information on how firmly a user is pressing against objects (finger press input), finger tap input associated with light taps of a user's finger against a surface, lateral finger movement information such as shear force information indicating how firmly a user is pressing their finger sideways on a surface, and other user input. Haptic output may be provided to the user to confirm to the user that a light tap input has been recognized or to otherwise provide feedback to the user.

The haptic feedback may provide the user with a sensation of tapping on a physical keyboard or other input device with a movable button member even when the user is tapping on a hard flat surface such as a tabletop. The haptic output provided with the wearable electronic device to the user may be virtual reality haptic output or augmented reality haptic output that is provided while a user is wearing a head-mounted display or other device that creates a virtual reality or augmented reality environment for a user.

Apple's patent FIG. 1 below is a schematic diagram of an illustrative device such as a finger-mounted device; FIG. 2 is top view of a user's hand and illustrative finger-mounted device components on finger tips of the user's hand. Some or all of control circuitry #14 may be contained in units #22 or may be mounted in separate housing structures (e.g., part of a wrist band, a glove, a partial glove such as a fingerless glove or glove in which portions have been removed under the pads of a user's finger tips, etc.).

Apple's patent FIG. 4 above is a perspective view of an illustrative finger-mounted device; FIGS. 14 and 15 are views showing illustrative mounting arrangements for finger-mounted devices.

Apple's patent FIG. 26 below is a side view of an illustrative finger-mounted device being worn on a finger at a location other than the tip of the finger; FIG. 27 is a side view of an illustrative finger-mounted device with optical sensors for gathering touch input from the upper surface of a user's finger; and FIG. 28 is a diagram showing how markers may be used in calibrating a system in which a finger-mounted device is used.

Further to FIG. 28, Apple notes that the user may interact with the displayed visual content in the head mounted display by supplying force input, motion input (e.g., air gestures, three-dimensional air gestures), taps, shearing force input, and other input.

Apple's patent FIG. 3 is an alternative ring style device design. Apple notes that FIG. 3 may, as an example, be formed from a soft elastomeric material, fabric, or other flexible material that allows the user to feel surfaces through the ring (unit #22). If desired, sensors, haptic devices, and or other components may be mounted under the pad of finger #32 in locations such as location #36.

Apple updated their invention just four days prior to unveiling Vision Pro, proving it's an active project. Apple added 20 new patent claims to strengthen their invention as follows:

1. A finger-mounted electronic device configured to manipulate display content on a display in a head-mounted device, the finger-mounted electronic device comprising: a body that surrounds a central opening configured to receive a finger; a sensor mounted in the body and configured to detect a finger tap with which a visual item on the display is selected, wherein the sensor is selected from the group consisting of: an accelerometer, a force sensor, and an optical sensor; and a visual marker on the body with which the head-mounted device tracks a location of the finger-mounted device, wherein the visual item on the display is moved based on the location of the finger-mounted device.

2. The finger-mounted electronic device defined in claim 1 wherein the force sensor comprises multiple force sensing components each of which produces a separate respective force sensor measurement based on different amounts of finger compression.

3. The finger-mounted electronic device defined in claim 1 wherein at least one of the accelerometer and the force sensor are configured to detect finger press input on a surface and lateral finger motion input on the surface.

4. The finger-mounted electronic device defined in claim 1 wherein the visual marker comprises an infrared light-emitting diode.

5. The finger-mounted electronic device defined in claim 1 wherein the body completely surrounds the central opening and has a curved outer surface.

6. The finger-mounted electronic device defined in claim 1 wherein the sensor is configured to detect a drag-and-drop gesture.

7. The finger-mounted electronic device defined in claim 6 wherein the visual item on the display is placed in a virtual workspace in response to the drag-and-drop gesture.

8. The finger-mounted electronic device defined in claim 1 further comprising a haptic output device configured to provide haptic feedback in response to the finger tap.

9. The finger-mounted electronic device defined in claim 1 further comprising a touch sensor on the body.

10. The finger-mounted electronic device defined in claim 1 wherein the optical sensor is configured to detect touch input on a surface of the finger.

11. A finger-mounted electronic device, comprising: a body configured to wrap around an entire circumference of a finger; an accelerometer in the body and configured to detect a three-dimensional air gesture; a force sensor in the body and configured to detect a finger tap; and a visual marker on the body with which a head-mounted device tracks a location of the finger-mounted electronic device, wherein the head-mounted device is configured to display content that is manipulated based on the three-dimensional air gesture, the finger tap, and the location of the finger-mounted electronic device.

12. The finger-mounted electronic device defined in claim 11 wherein the display content comprises a visual item that is selected in response to the finger tap.
13. The finger-mounted electronic device defined in claim 11 wherein the display content is scrolled in response to the three-dimensional air gesture.

14. The finger-mounted electronic device defined in claim 11 further comprising a haptic output device configured to provide haptic output in response to the three-dimensional air gesture and the finger tap.

15. The finger-mounted electronic device defined in claim 11 wherein the force sensor and the accelerometer are configured to detect a drag-and-drop gesture with which the display content is manipulated.

16. A system, comprising: a finger-mounted electronic device having a first sensor configured to detect first finger input associated with a first finger; a hand-worn electronic device having a second sensor configured to detect second finger input associated with a second finger; and a head-mounted device having a display configured to produce display content, wherein the display content is manipulated based on the first finger input and the second finger input.

17. The system defined in claim 16 wherein the first sensor and the second sensor are each selected from the group consisting of: a force sensor, an accelerometer, and an optical sensor.

18. The system defined in claim 16 wherein the first finger input and the second finger input are each selected from the group consisting of: a three-dimensional hand gesture, a finger tap, and contact with an external surface.
19. The system defined in claim 16 wherein the finger-mounted electronic device and the hand-worn electronic device each have visual markers with which the head-mounted device tracks locations of the finger-mounted electronic device and the hand-worn electronic device.

20. The system defined in claim 19 wherein the visual markers comprise infrared light-emitting diodes.

To review the full details of this invention, check out patent application 20240168558.