Apple has filed for a series of patents for Project Titan that over the years that utilizes various radar systems (01, 02, 03 & 04). In February we posted an IP report titled "Apple has Won a Patent for Devices using a 'Circular Radar-Antenna Array' that Introduces a new UI that could recognize Hand Gestures +."

Today the US Patent & Trademark Office published a patent application from Apple that relates to object sensing systems, and more particularly, to detecting and tracking one or more objects such as styluses and a smart glove using radar. More specifically, such systems can allow multiple users in the same meeting room, each having a stylus, to collectively edit, draw or otherwise contribute ideas to a shared document being created or displayed on a device such as a smart board or smart display.

In Apple's patent background they note that the object detection range of proximity sensing systems is often limited, and the position and motion sensing resolution of these systems is often limited by the number and size of sensing elements in the array. Because of these limitations, alternative systems such as ultrasonic sensing systems and radar sensing systems have been developed that use triangulation to detect objects.

However, when the object (e.g., a stylus) requires very accurate position detection with sub-millimeter accuracy, for example, even these systems can suffer from inadequate position detection accuracy.

Apple's solution relates to detecting and tracking one or more objects of interest (e.g., an Apple Pencil or fingertips of a smart glove) with improved accuracy using radar-based tracking systems.

In some examples, multiple radars implemented in a device (e.g., a MacBook or iPad, etc.) can be used to transmit signals to, and receive signals from, the one or more objects of interest.

A device, as that term is used herein, includes but is not limited to portable and handheld electronic devices, small standalone units in communication with other electronics whose main function is to provide radar functionality at remote locations, stationary electronic devices, and larger environment devices such as a smart room or a smart whiteboard, for example.

To disambiguate an object of interest from undesired objects such as the hand of a user holding the object of interest, the object of interest can include a transponder that applies a delay element to, or otherwise processes, a signal received from a radar, and thereafter transmits a return signal back to the radar.

In examples that process the signal received from the radar, the object of interest match filters the received signal, and if a match is confirmed, transmits a unique return signal back to the radar that can be matched and correlated to filter out undesired reflections. Clear identification of the desired return signal can lead to more accurate object distance determinations, more accurate triangulation, and ultimately improved position detection and tracking accuracy.

Triangulation techniques can then be employed using distance data from multiple radars to determine object location with high accuracy. In some examples, accuracy of 100 micrometers or better can be achieved.

Radar-based object tracking systems can enable two-dimensional or three-dimensional object position and gesture determinations of an object of interest moving on a surface or in free space.

In addition, if multiple objects of interest employ delay elements with unique delays, multiple objects of interest can be simultaneously tracked by the plurality of radars in the object tracking system, allowing for improved collaborative experiences. For example, such systems can allow multiple users in the same meeting room, each having a stylus, to collectively edit, draw or otherwise contribute ideas to a shared document being created or displayed on a device such as a smart board or smart display.

In another example, wearable devices such as gloves outfitted with a delay element on one or more fingertips can provide one or more objects of interest (each fingertip with a delay element being an object of interest) for performing gestures.

The Apple devices presented below will be able to take advantage of Apple's future radar-based object tracking system. Apple adds a few possible new devices in-the-works such as a "Smart Board," a "Smart Screen" (to compete with Google Nest Hub or Amazon's Alexa device), a "Smart Glove" and next-gen Apple Pencil.

Apple's patent FIG. 2 below illustrates an exemplary block diagram of a radar-based object tracking system that includes a next-gen Apple Pencil; FIG. 4 illustrates a future M-Chip MacBook Pro that uses a radar-based object tracking system that could work with the next-gen Apple Pencil #404.

Further, to track the position of Apple Pencil #404, each radar #402 can transmit a signal at a certain frequency and measure its distance to the tip of the stylus (e.g., R1, R2 and R3, respectively) using reflections from the stylus tip. Triangulation algorithms (e.g., systems of equations) can then be applied to the measured distances to determine the location of the stylus tip.

For more details, review Apple's patent application 20210405174. Considering that this is a patent application, the timing of such a product to market is unknown at this time.

Apple Inventors

Michael (Misha) Kerner: Engineering Manager (Apple, Israel); came to Apple after spending 11 years at Intel.

Michael Delishansky: PHY System Engineer

Zohar Agon: Senior Algorithms Engineer

Shachar Shayovitz: PhD Candidate at TAU and Algorithm Engineer

Michael Ney: Algorithm Engineer

Shay Freundlich: Wireless Systems

John H.: Full Stack Software Engineer