Today the US Patent & Trademark Office published a patent application from Apple that relates to magnetic sensor based proximity sensing systems, methods and devices such as fingertip nodes. More particularly, Apple's invention relates to a magnetic sensor based proximity sensing that is capable of measuring the movement of individual finger and thumb bones. Apple has filed patents for similar work that relates to possible future VR Gloves for gamers and military training and finger devices for a mixed reality headset that could replace sensor gloves.

Apple's invention relates to a magnetic sensor based proximity sensing architecture that enables precise location/positioning of electronic devices (e.g. smartphone, tablet, other handset or wearable devices) in proximity to a modulated magnetic source.

Multiple magnetic sensors aligned to detect magnetic field changes in different field directions axes can also be used to determine the three-dimensional position of the magnetic field and can provide more information in locating the fingers and the hands.

One exemplary application of the magnetic sensor based proximity sensing architecture is a device with fingertip nodes that can include a plurality of magnetic sensors to track the movement of one or more finger and/or hand sections.

By placing a magnetic sensor on each fingertip, for example, inverse kinematics can be applied to compute the orientation, position, and angle of objects (e.g., finger bones) using proximity signals detected by the magnetic sensors.

The control device with fingertip nodes can also include one or more other electronic components, such as a plurality of electrodes for sensing the heading, enabling capacitive touch, and/or contact sensing between finger tips.

The control device with fingertip nodes can also include force sensors, actuators for haptic feedback, temperature sensors, and heaters.

The control device with fingertip nodes can further include logic such as an on-board controller, a connector, a transceiver, a battery, and the like. The control device with fingertip nodes can also include a host controller that renders the profile of the hand on the screen. Signals from the fingertip nodes can be carried to the host controllers using wireless technology.

Overview of a Control Device with Fingertip Nodes

Apple's patent FIG. 3 below illustrates a front view of an exemplary control device with fingertip nodes.

More specifically, FIG. 3 illustrates a front view of an exemplary control device #400 with fingertip nodes #410 according to examples of the disclosure. In some examples, one fingertip node can be slipped on to each of a user's fingertips.

In some examples, a magnetic field generator #430 can be located proximate to the user's palm in a location that is stationary relative to the movement of bones in the user's fingers and hands.

Although FIG. 3 illustrates the magnetic field generator located proximate to the palm of the user's hand, in some examples, the magnetic field generator can be located proximate to the user's wrist, the back of the user's hand, or other locations.

In some examples, the magnetic field generator can be included in a wearable accessory (e.g., a watch or bracelet). In some examples, the magnetic field generator #430 can be included as a component in a hand controller (not shown) that can include other electronic components such as a wireless transceiver, a magnetic sensor, a controller, buses, one or more LEDs, and a battery.

In some examples, each fingertip node can include a plurality of electronic components, where some or all of the electronic components can be knitted, woven or embedded into the material of the fingertip node.

The electronic components can include one or more of magnetic sensors, demodulators, filter and ADC, a controller, buses, one or more LEDs, a battery and a wireless transceiver.

The fingertip node can be configured to capture the motion of the user's fingers. The plurality of magnetic sensors can be configured to track the movement of one or more of the user's fingertips.

The controller can include logic configured to communicate with the electronic components via the plurality of buses. The LED(s) can be configured to provide optical feedback to the user.

A magnetic sensor arrangement can be configured to detect magnetic field components for multiple directions (e.g., x, y, and z components), and the multiple field components can be used to determine a three-dimensional position of the fingertip node.

In some examples, the multiple components can be obtained by including multiple magnetic sensors oriented to be sensitive to magnetic field variation in orthogonal directions. In some examples, a three-axis sensor can be used to determine the direction of the magnetic field as well as the distance.

Apple's patent FIG. 1 above illustrates an exemplary architecture based on magnetic sensor based proximity sensing; FIG. 4A illustrates a block diagram of an exemplary control device with fingertip nodes or magnetic sensors on the hands

Operation of the Device with Fingertip Nodes and/or Magnetic Sensors

Apple's patent FIG. 5 illustrates an exemplary process #700 for determining the locations, angles and motions of the hand and fingers and their respective bones using a device with fingertip nodes and/or magnetic sensors on the hands according to examples of the disclosure.

In some examples, at step #710, the electromagnetic coil (spiral, solenoid, or circular) can generate a modulated magnetic field B(t). In some examples, a single modulated magnetic field can be transmitted from a location on the user's body such as a neck-worn transmitter, a headset, a waistband, or the like.

In some examples, a modulated magnetic field can be transmitted from one or more coils located on or near the palms/wrists of a user's hands.

In some examples, at step #720, the fingertip nodes and/or magnetic sensors (e.g., located on the palms/wrists) can receive the modulated magnetic field and perform demodulation at the modulation frequency of the desired reference transmitter.

For example, the finger nodes of the right hand may perform demodulation at a first frequency corresponding to a transmitter located on the right palm/wrist. Similarly, the finger nodes of a left hand may perform demodulation at a second frequency corresponding to a transmitter located on the left palm/wrist.

In addition, magnetic sensors for locating the positions of each of the left and right hand (e.g., located on each palm/wrist of the user) may perform demodulation at a third modulation frequency corresponding transmitter on the user's body. In some examples, a single modulation source, e.g., on the user's body, can be used to determine both hand and finger positions using a single modulation frequency. In addition, at step 720, the distance "d" between the fingertip nodes/magnetic sensors and a corresponding transmitting source (e.g., electromagnetic coil) can be determined based on magnetic field amplitude.

In some examples, at step #730, the fingertip nodes/magnetic sensors can then transmit data corresponding to the measured distance. In some examples, the data can be transmitted over a low power wireless communication link (e.g., BLE).

At step #740, the position information of the fingers and/or hands can be used to computer inverse kinematics to determine the orientation, position, and angle of finger and hand bones. In some examples, at step 740, the inverse kinematics calculation can be performed in a hand controller or other intermediate device, before transmitting to a host controller that generates the environment.

In some examples, at step #740, the distance information for the fingers and hands can be transmitted directly to the host, and inverse kinematics can be performed on the host.

In some examples, at step #750, the host device can use the orientation, position, and angle of objects (e.g., finger and hand bones) determined by the inverse kinematics at step $740 to then render the hands and the fingers in space with the accurate joint positions in the environment.

Apple's patent FIG. 6 above illustrates an exemplary architecture based on magnetic sensor based proximity sensing in a stylus-tablet system according to examples of the disclosure. In some examples, the magnetic field generator can be located in the tablet #820 and a magnetic sensor can be located in the stylus #810 (Apple Pencil). In some examples, a magnetic field generator can be provided in each corner of the tablet and distances from each of the four corners can be used to determine the position of the Apple Pencil relative to the tablet.

In some examples, Apple Pencil can include a magnetic sensor based proximity sensor on each end of the Apple Pencil, allowing both distance and orientation of the stylus to be detected. In some examples, magnetic sensor based proximity sensing can be used to perform gesture recognition between the tablet and Apple Pencil without requiring Apple Pencil to contact the sensing surface.

Apples patent FIG. 8 above illustrates an exemplary architecture based on magnetic sensor sensing of a modulated magnetic field in a Near Field Communication system according to examples of the disclosure. Magnetic sensor based communication sensing can be used to enhance the performance of a Near Field Communication system. Near Field Communication system generally needs precise alignment between the transmitter and the receiver but with the magnetic sensor based sensing, a greater amount of misalignment can be tolerated.

Apple's patent application that was published today by the U.S. Patent Office was originally filed back in Q1 2019. Considering that this is a patent application, the timing of such a product to market is unknown at this time.

Two of the inventors listed include: 1) Savas Gider, Senior Hardware Engineer developing next-gen motion sensors for mobile and other products. (2) Jian Guo, Engineering Manager, Sensing System Design.

About Making Comments on our Site: Patently Apple reserves the right to post, dismiss or edit any comments. Those using abusive language or negative behavior will result in being blacklisted on Disqus.