It's been a very busy year for patents relating to finger devices that are designed to work with a future Apple mixed reality head mounted display. Patently Apple has covered four of Apple's patents on these finger devices specifically for a headset since January (01, 02, 03 and 04). Today the US Patent & Trademark Office published Apple's fifth patent application relating to finger devices that could communicate with a headset to assist a user manipulate content on the headset's display instead of using specialized headset gloves that are clumsy to wear and use. The finger devices detect hand motions such as 3D in-air hand gestures and provide haptic responses to make game play realistic. The finger devices will use fabric materials in the design.

Apple's patent covers a system that may include one or more finger-mounted electronic devices such as finger devices with U-shaped housings configured to be mounted on a user's fingers while gathering sensor input and supplying haptic output.

The finger-mounted devices may each have a body. The body serves as a support structure for components such as force sensors, accelerometers, and other sensors and for haptic output devices. During operation, a user may wear the finger mounted units on the tips of the user's fingers while interacting with external objects.

The body of a finger-mounted electronic device may have first and second side body members that leave the finger pad exposed and an upper body member extending between the first and second side body members. Some or all of the body may be covered in fabric or leather. Fabric may wrap around the first and second side body members and may extend across the upper body member. The fabric may cover electronic components. A touch sensor may have electrodes that are formed from conductive material on the fabric or conductive strands in the fabric. Infrared-reflective ink may form visual markers on the fabric for an infrared tracking system. The fabric may have light-transmissive portions that overlap optical components. The fabric may have breathability features such as pits, bumps, or through-holes.

The fabric may be attached to the body using wrapping techniques, intertwining techniques, adhesive attachment methods, molding attachment methods, or other suitable techniques.

Apple's patent also covers cameras for the Headset and finger devices.

Apple's patent FIG. 1 is a schematic diagram of an illustrative system with a finger device to work in sync with a Mixed Reality Headset; FIG. 2 is a top view of an illustrative finger of a user on which a finger device has been placed; FIG. 3 is a cross-sectional side view of an illustrative finger device on the finger of a user.  

Apple's patent FIG. 19 above is a perspective view of an illustrative finger device having one or more touch sensors; FIG. 20 is a schematic diagram of an illustrative touch sensor that may be used in a finger device.

Apple's patent FIG. 4 below is a cross-sectional side view of an illustrative finger device having a fabric or leather layer in addition to illustrating hinges on the finger device.  

The hinges (#34) may be located in upper portion (#104), side portions (#102), and/or between the upper portion and side portions. The hinges may include friction hinges, spring-loaded hinges, freely-rotating hinge joints, other hinge structures, or a combination of these hinge structures. The hinges may allow upper portion and left and right side portions of the housing (#26) to be moved towards or away from each other to accommodate fingers of different sizes.

Apple's patent FIG. 5 below is a cross-sectional side view of an illustrative fabric that may be used in a finger device.

Apple's patent FIG. 32 above is a schematic diagram of a system having a finger device with visual markers on fabric and an electronic device (the headset) having a camera that tracks the visual markers; FIG. 33 is a cross-sectional side view of an illustrative finger device having optical components that are aligned with light-transmissive portions of a fabric; and FIG. 34 is a cross-sectional side view of an illustrative finger device having fabric with a light-transmissive region formed from openings in the fabric.

Visual Markers on the Finger Devices

More specifically regarding patent FIG. 32, the external equipment such as electronic device #20 (the headset) in the system may contain sensors such as one or more cameras (#136) (e.g., visual light cameras, infrared cameras, etc.).

The headset may, for example, form part of an augmented reality (mixed reality) or virtual reality system (e.g., a system that includes a head-mounted device, glasses, a helmet, or other head-mountable support structures).

Visual markers #82 may be placed on the finger devices (#10) and, if desired, on other locations in the user's environment. The markers may be, for example, passive visual markers such as bar codes, cross symbols, or other visually identifiable patterns and may be applied to the finger device(s) and/or other objects such as a tabletop or other work surface.

The markers may, if desired, include light-emitting components (e.g., visual light-emitting diodes and/or infrared light-emitting diodes modulated using identifiable modulation codes) that are detected using cameras.

The markers may help inform the system of the location of the user's virtual work surface and one or more of the user's fingers as a user is interacting with a computer or other equipment in the system.

Visual markers on the finger device(s) and/or inertial measurement units in device 10 (e.g., accelerometers, compasses, and/or gyroscopes) may be used in tracking the user's finger locations (e.g., the locations of finger-mounted devices 10) relative to other markers in the user's work area. At the same time, the system may display associated visual content for the user.

The user may interact with the displayed visual content by supplying force input, motion input (e.g., air gestures), taps, shearing force input, and other input gathered from the finger device by inertial measurement units in the finger device and/or force sensors and other sensors in the finger device.

During operation of the system while monitoring the finger device for force input, gesture input (e.g., taps, three-dimensional air gestures, etc.) that indicate that a user has selected (e.g., highlighted), moved, or otherwise manipulated a displayed visual element and/or provided commands to the system.

The system may use the markers on the finger device and/or inertial measurement units in the device to detect the left hand wave gesture and can move visual elements being presented to the user with a display in the headset in response to the left hand wave gesture.

Lastly Apple hints that finger devices could one day also work with devices like Apple TV, a gaming device, television, Macs and more as well as being used to control home automation.

Apple's patent application that was published today by the U.S. Patent Office was filed back in Q3 2018. Considering that this is a patent application, the timing of such a product to market is unknown at this time.

Apple's inventors

Paul Wang: Architect, Product Design

James Stryker: Manager, Product Design

Yoonhoo Jo: No LinkedIn Profile could be found.