Last Thursday the U.S. Patent and Trademark Office officially published a patent application from Apple that relates to future smartglasses and, more particularly, to a dual gaze tracking system for smartglasses that is supported by the frame of the glasses so that the gaze of a user may be monitored. For AR glasses, gaze tracking is a major feature to ensure that projected holographic images align properly in relation to a real-world environment.

Smartglasses with Dual Gaze Tracking Systems

According to Apple's patent application, gaze tracking systems may be used to gather information on the direction of the user's gaze. The head-mounted support structure may form a pair of glasses or other head-mounted housing. The glasses may include, for example, a frame and a pair of temples or other elongated side support members that are coupled to the frame by hinges.

Glasses lenses (e.g., transparent lens members with or without an associated lens power) may be supported by the frame. Display systems on the left and right sides of the device may provide computer-generated images to the user while the user is viewing real-world objects through the lenses. In some configurations, selectable menus and other selectable items may be displayed over real-world images.

The gaze tracking systems may include image-sensor-based systems such as eye-glint-based systems. A glint-based gaze tracking system may include light-emitting devices that emit light beams that create eye glints on the surface of a user's eyes and may include an image sensor that measures the eye glints. The pattern of eye glints may be analyzed to produce information on the user's gaze.

The gaze tracking systems may also include a low-power gaze tracking system. The low-power gaze tracking system may use light detectors to measure light reflections from the user's eyes. The light reflections may correspond to reflections of the light beams emitted by the light-emitting devices used by the glint-based gaze tracking system. The user's gaze may be measured by analyzing the magnitudes of the detected light reflections.

The head-mounted device may use different gaze tracking systems in different operating modes. For example, a glint-based gaze tracking system or other system based on an image sensor may in a given mode of operation, be used to gather gaze information to supplement and/or replace gaze tracking information from a low-power gaze tracking system. In another mode of operation, the low power gaze tracking system may be used to track the user's gaze without using the glint-based gaze tracking system.

Sensors that may be included in Future Smartglasses

Apple notes that the input-output circuitry may include the following sensors for smartglasses: gaze tracking sensors (e.g., a gaze tracking system based on an image sensor and, if desired, a light source that emits one or more beams of light that are tracked using the image sensor after reflecting from a user's eyes, a low-power gaze tracking system formed using photodetectors, etc.). Sensors may also include, for example, three-dimensional sensors (e.g., three-dimensional image sensors such as structured light sensors that emit beams of light and that use two-dimensional digital image sensors to gather image data for three-dimensional images from light spots that are produced when a target is illuminated by the beams of light, binocular three-dimensional image sensors that gather three-dimensional images using two or more cameras in a binocular imaging arrangement, three-dimensional lidar (light detection and ranging) sensors, three-dimensional radio-frequency sensors, or other sensors that gather three-dimensional image data), cameras (e.g., infrared and/or visible digital image sensors), touch sensors, capacitive proximity sensors, light-based (optical) proximity sensors, other proximity sensors, force sensors, sensors such as contact sensors based on switches, gas sensors, pressure sensors, moisture sensors, magnetic sensors, audio sensors (microphones), ambient light sensors, microphones for gathering voice commands and other audio input, sensors that are configured to gather information on motion, position, and/or orientation (e.g., accelerometers, gyroscopes, compasses, and/or inertial measurement units that include all of these sensors or a subset of one or two of these sensors), and/or other sensors.

To review the full details of this invention, check out patent application 20240219715. The lead inventor is noted as Brian S. Lau, Product Design Engineer; Vision Products.

Eye Tracking for smartglasses is extremely important and involves many aspects covered a series of Apple patents: here a few other related patents on file: 01, 02, 03, 04, 05 and more.