Apple reinvents Eye-Tracking for future Smartglasses that could potentially decrease the thickness of the Frame around the Lenses+
AI glasses have become one of the hottest new device categories that are exciting techies worldwide with Meta being given credit with their Ray-Ban Meta AI glasses and revelations about their project Orion, a high-end smartglasses project revealed back in September 2024. Apple has been working on smartglasses since at least 2019, so it's not anything new for Apple and has published a long series of smartglasses patents that you could review in our archives. Some of the most recent patent include the following: 01, 02, 03, 04, 05, and 06.
Today the U.S. Patent Office published a patent application of Apple's that relates to their smartglasses project titled "Eye Reflections using IR Light Sources on a Transparent Substrate." Our report also links to an additional seven new HMD related patents from Apple.
In Apple's patent background for this patent, they note that existing eye-tracking techniques analyze glints that are reflected off of a user's eye and captured via an image sensor. Some head mounted systems may include eye-tracking techniques to analyze glint's using light projected from light sources located at an edge of a device (e.g., the frame of a pair of glasses). However, the current eye-tracking systems may lack accuracy, require more than one camera to capture a sufficient number of glints, and require eye camera placement that is suboptimal for capturing a sufficient number of glints.
Due to this, it may be desirable to provide a means of efficiently positioning of light sources to produce glints for assessing an eye characteristic (e.g., gaze direction, eye orientation, identifying an iris of the eye, etc.) for head mountable systems.
Apple's invention covers various implementations that include devices, systems, and methods that assess an eye characteristic (e.g., gaze direction, eye orientation, identifying an iris of the eye, etc.) of a user wearing a head mounted device (HMD).
The eye characteristic assessment is based on a determination of a location of a glint produced using a light source, such as an infrared (IR) light emitting diode (LED), a micro-IR LED, a mini-IR LED, or the like. Light sources are placed on a transparent substrate (e.g., a lens), which can be placed between a display (or view of a physical environment) and a human eye, to illuminate the eye to produce glints (e.g., a reflection of an IR LED on the eye). The light sources (e.g., IR LEDs) are positioned on or within the transparent substrate rather than (or in addition to) on a surrounding rim of the HMD.
In some implementations, the light sources may be connected to a power source with a transparent conductive material and may be driven individually. The light sources, such as IR LEDs, may be sufficiently small (e.g., less than 100 μm) such that a user in unlikely to notice them given their size and close proximity to the eye during use of the HMD (e.g., micro-IR LEDs, mini-IR LEDs, and the like). In some implementations, different wavelengths produced by the IR LEDs may be used for different applications.
The advantages of including the light sources in the lens/transparent substrate as opposed to on an edge of a frame of the eye piece/lens would allow a wider selection of areas to place the light sources, and potentially decrease the thickness of the frame around the lens.
Additionally, because of a multi-stack architecture (e.g., a lens that includes a bias (−), air gaps, a waveguide, and a bias (+) layer), the light sources (e.g., IR LEDs) could be imbedded in the middle of the stack. This multi-stack architecture would not be perceptible to the user when in use and would have a better accuracy then on a surrounding rim of an HMD because the light sources are closer to an optical axis of an eye of the user.
The positioning of each light source with respect to an adjacent light source also becomes important. For example, the light sources should not be too close to each other since then reflected light (e.g., a glint) can be too close and decrease the accuracy of the gaze estimate or assessing other eye characteristics. Additionally, the light sources should not be too far from an optical axis (e.g., around a rim of the frame) since the light sources will not reflect for some eye characteristics properly (e.g., the reflections may end up on the sclera and not the pupil).
In general, one innovative aspect of the invention in this specification can be embodied in an electronic device including a frame, an image sensor, a transparent substrate coupled to the frame, the transparent substrate including a plurality of infrared (IR) light sources, where the plurality of IR light sources are configured in a spatial arrangement within the transparent substrate or on a surface of the transparent substrate, a waveguide coupled to the transparent substrate, wherein the waveguide is configured to display a projected image, and a processor coupled to the plurality of IR light sources. The processor is configured to receive sensor data from the image sensor, the sensor data corresponding to a plurality of reflections of light produced by the plurality of IR light sources and reflected from an eye.
Apple's patent FIG. 2A below illustrates an example of a user wearing an head mounted display (HMD); FIG. 2B illustrates an example view of a transparent substrate (lens) of the HMD of FIG. 2A; FIG. 3 illustrates an example eye-tracking system.
Apple's patent FIG. 4 illustrates an example of a plurality of light sources within a transparent substrate; and FIG. 6 illustrates different spatial arrangements of clusters of light sources for the plurality of light sources.
For full details, review Apple's patent application 20250004283. Two of the listed inventors on this patent includes Scott Foster (Engineering Manager) and Branko Petljanski (Senior Engineer, Camera Electronics).
A Few Other HMD Patent Applications Published Today
20250008057: Managing Display of Participants in a Real-Time Communication Session
20250004501: Sealed Components for Heat Management and Grounding
20250005864: Methods for Optimization of Virtual UI's in a 3D Environment
20250005855: Locations of Media Controls for Media Content and Captions for Media Content in 3D Environments
20250004342: Waveguide Display With Sealed Tint Layer
20250004545: Head-Mounted Device Input
20250004273: Electronic Devices With Nose Sensing
Apple's patent covers a head-mounted device may include optical assemblies for presenting images to a user. Optical assembly positioning systems may be used to adjust the spacing between the optical assemblies to accommodate different interpupillary distances. The optical assembly positioning systems may have motors. Nose pressure sensors may be provided to detect nose pressure as optical assemblies are moved towards each other and towards nose surfaces by the motors. Motion of the optical assemblies can be halted whenever the detected nose pressure rises. The nose pressure sensors may each include an air pressure sensor and a pressure-sensing compressible air-filled chamber coupled to the air pressure sensor.
Apple's patent FIG. 5 above is a perspective view of a rear portion of device 10 showing how chambers 88 may be located on or in nose bridge portion NB of housing #12 at locations that face inwardly towards the locations of nose surfaces #61 when device #10 is worn on the head of a user. Chambers #88 of FIG. 5 may be, for example, compressible pillow-shaped elastomeric members with air-filled cavities.