Apple has won a series of Design Patents in Hong Kong & Europe while filing for new Trademarks in the U.S. for 'Apple M4' and more
Apple has released the trailer for the Second Season of the Science Fiction Dystopian Drama 'Silo'

Meta Files an Orion-Centric Smartglasses Patent covering Single Pixel 3D Retinal Imaging

1 Meta

This morning Bloomberg's Mark Gurman stated that Apple is preparing a response to Meta's camera glasses, though clearly behind the curve. While Gurman surmises that Apple will have their basic glasses device launching in and around 2027, Meta keeps on filing patents in the hopes of staying well ahead of Apple.

This week a European patent application was discovered titled Single Pixel 3D Retinal Imaging" that relates to Meta's  future Orion smartglasses.

Meta's patent covers systems, methods, devices, and computer program products for eye tracking using compressive sensing for eye trackers comprising a plurality of single pixel detectors and a digital micro-mirror device (DMD) as a spatial light modulator (SLM).

For example, two or more single pixel detectors and a DMD device may be used to perform a three-dimensional (3D) reconstruction of the retina.

Various aspects and examples may be usable on a range of artificial reality devices and applications, including but not limited to wearable devices, head-mounted systems, headsets, glasses, helmets, visors, gaming devices, and smart devices.

A DMD device may render content (e.g., binary images or grayscale images) to a user's pupil or retina. Each of a plurality of single pixel detectors may capture a signal (e.g., light) of the rendered content on the user's retina. For example, by inverse Fourier transform, each pixel detector may recover a grayscale image of a user's retina. A 3D shape of the retina may be built by using stereo 3D reconstruction techniques among the recovered pixels. Moreover, a movement or characteristic of the eye may be determined based on the recovered image or 3D shape.

According to a first aspect, there is provided an apparatus comprising: one or more processors; at least one memory storing instructions: at least one digital micro-mirror device configured to: emit light, comprising one or more binary images or one or more grayscale images, onto at least one eye of a user; a plurality of single pixel detectors configured to: determine a reflection of the light from the at least one eye; determine, by a first single pixel detector of the plurality of single pixel detectors, a first signal associated with the reflection of the light from a retina of the at least one eye; determine, by a second single pixel detector of the plurality of single pixel detectors, a second signal associated with the reflection of the light from the retina of the at least one eye; and wherein when the one or more processors execute the instructions, the apparatus is configured to: perform a transformation on the first signal and the second signal to determine at least one grayscale image of the retina; and determine at least one three-dimensional (3D) shape of the retina based on performing at least one 3D reconstruction associated with one or more pixels of the grayscale image of the retina.

When the one or more processors further execute the instructions, the apparatus may be configured to: track one or more movements of the at least one eye based in part on the determined at least one 3D shape of the retina.

When the one or more processors execute the instructions, the apparatus may be configured to: perform the track the one or more movements of the at least one eye based in part on determining one or more eye tracking parameters associated with the 3D shape of the retina.

The one or more eye tracking parameters may comprise at least one of an angle of eye-gaze of the at least one eye, a torsion of the at least one eye, a shape of the at least one eye, a current focal distance of the at least one eye, a dilation of a pupil of the at least one eye, an inter-pupillary distance of the at least one eye and a second eye of the user, a translation of the at least one eye, an accommodation of the at least one eye, other features of a state of the at least one eye, or a combination thereof.

Prior to performing the transformation, the at least one digital micro-mirror device may be further configured to: receive the first signal and the second signal from the plurality of single pixel detectors.

Meta's patent FIG. 1 below illustrates an artificial reality system comprising a headset. The HMD #110 can include an eye tracking system #118 to track the vergence movement of the user wearing the HMD. In some aspects, the eye tracking system may include one or more single pixel detectors #120 (e.g., PD, APD, SPAD). Moreover, the eye tracking system may include one or more digital micro-mirror device (DMDs) #122 as a spatial light modulator (SLM) SLM.

The eye tracking system may determine a 3D reconstruction of a retina of a wearer of the HMD 110. For example, the DMD(s) may project binary images or grayscale images onto the retina and the single pixel detectors #120 may capture the signal. By inverse Fourier transform, each single pixel detectors may recover a grayscale image of the retina. By using stereo 3D reconstruction technologies among grayscale images, a 3D shape of the retina may be built by the eye tracking system.

2 Meta

Meta's patent FIG. 2 above illustrates a block diagram of a retinal imaging system;   FIG. 4 illustrates an example flowchart illustrating operations to facilitate eye tracking.

For full details, review Meta's European patent 4410180.

10.0x35 Patently Mobile Patent Reports