Apple Introduces a new kind of Smart Ring System to be used Primarily in Context with AR, VR and MR Applications
Apple began their smart ring project back in 2014 with their first patent application surfacing in 2015. We covered it in a report titled "Apple introduces us to the Apple Ring in all its Glory." Last March we posted another sophisticated ring patent application report titled "Apple Wins Patent for a Ring Accessory to Control Content presented on Future Virtual Reality Glasses & Headset." We also followed up with a series of continuation patents for their smart ring (01, 02 & 03) over time.
Today the US Patent & Trademark Office published yet another patent application from Apple relating to their smart ring project, this time using a different method than ones used in the past by introducing a new self-mixing interferometry (SMI) sensor-based gesture input system. The majority of the invention relates to a smart ring being used alone or in pairs, with or without an Apple Pencil for AR, VR and MR applications.
Sensor systems may variously sense the presence of objects, distances to objects or proximities of objects, movements of objects (e.g., whether objects are moving, or the speed, acceleration, or direction of movement of objects), and so on.
Given the wide range of sensor system applications, any new development in the configuration or operation of a sensor system can be useful especially if they improve the sensitivity or speed of sensor system operations.
Gesture Input System using a Smart Ring
Apple's latest ring invention covers systems, devices, methods, apparatus and operation of an SMI-based gesture input system that includes one or more self-mixing interferometry (SMI) sensors.
In one instance, the SMI sensor(s) may be used to determine a relationship between a ring and a handheld device, such as an Apple Pencil.
More specifically, the patent relates to the configuration and operation of SMI-based gesture input systems--i.e., systems that can identify gestures made by a user using signals received from one or more SMI sensors.
An SMI sensor can be used to optically measure the relative motion (displacement) between the SMI sensor and a target (e.g., a surface or object), with sub-wavelength resolution.
When displacement measurements are associated with measurement times, the velocity of the target may also be measured. Furthermore, by modulating the SMI sensor with a known wavelength modulation (e.g., a triangular modulation), the absolute distance from the SMI sensor to the target may be measured.
In augmented reality (AR), virtual reality (VR), and mixed reality (MR) applications, as well as other applications, it can be useful to track a user's finger movement(s) and/or identify a user's gestures (e.g., gestures made with one or more fingers, a hand, an arm, etc.).
In some applications, it is useful for a user to be able to provide input to a system by interacting with a surface (e.g., making a gesture on any random surface, such as a tabletop, wall, or piece of paper), or by making a gesture in free space. Making a gesture in free space or an in-air gesture using an Apple Pencil was first revealed in a patent report that we posted back in 2018. One of the patent figures from that report is presented below.
In today's patent, Apple further notes that an SMI-based gesture input system may be used to track a user's finger movements with reference to any surface, including, in some cases, the surface of another finger, the user's palm, and so on.
Apple's SMI-based gesture input systems and devices in the form of smart rings could be worn singularly or in groups of two or more.
The systems may be provided with more or fewer SMI sensors, which generally enable finer or lower resolution tracking, or more or less complex gesture detection/identification. For example, with one SMI sensor, scrolling along a single axis may be detected. With two SMI sensors, user motion in a plane may be tracked.
With three or more SMI sensors, movements in x, y, and z directions may be tracked. Motion tracking with six degrees of freedom may also be tracked with three or more SMI sensors.
Apple's patent FIG. 1 below shows an example SMI-based gesture input system that includes a wearable device such as a smart ring (seen in-part); Apple's patent FIG. 7 shows smart rings placed on two of the user's fingers; FIG. 4 shows a wearable device having a set of SMI sensors, from which a processor of the device may select a subset of SMI sensors to determine a relationship between the wearable device and a surface.
Apple further notes that FIG. 7 illustrates dual rings used in context with Mixed Reality applications. Finger rings worn on a user's thumb and index finger, for example, may be used to identify gestures such as a pinch, zoom, rotate, and so on.
Apple's patent FIG. 6 shows an example of the system described with reference to FIG. 5 (further below), in which the wearable device is a finger ring and the object is an Apple Pencil (or digital marker or paintbrush). These figures are also in context with AR, VR and MR applications.
Apple's patent FIGS. 9C and 9D illustrate different beam-shaping or beam-steering optics that may be used with any of the SMI sensors; FIG. 13 shows an example method of identifying a type of gesture.
Listed Apple Inventors
Mark Winkler: Senior Engineering Manager, Laser & Optical Sensing
Mehmet Mutlu: Display Optics Design and Architecture Manager
Fatih Cihan: Senior Optical HW Engineer
Tong Chen: Optical Sensing Engineer
Considering that this is a patent application, the timing of such a product to market is unknown at this time.