On January 4, 2024, an international patent application from Apple surfaced in the database of the World IP Organization in Geneva that relates to smart glasses with an eye-tracking system, dual cameras, interchangeable components and more.  

Apple's patent background notes that  eye-tracking is a process for measuring the eye movement or the eye-gaze direction of an individual. Various eye-tracking technologies have been developed for use in a variety of applications, such as vision research, human computer interfaces, telesurgery, advertising research, visual communication, and various military applications.

Wearable devices and systems benefit from eye-tracking information and function as an advantageous platform for eye-tracking because eye measurements collected from a platform close to one or both eyes reduce measurement errors generated by head movements and other sources. Prior wearable systems that incorporated eye tracking systems were typically bulky and expensive due to inconvenient camera locations or the use of camera platforms that are obtrusive to the vision of the user. Apple's invention addresses these shortcomings.

Eye-Tracking System for Smartglasses

Apple's invention relates to an eye-tracking system for smartglasses. Eye-tracking can include capturing and/or measuring either the point of gaze (where one is looking) or the motion of an eye relative to the head. The retina of the eye includes an area of dense nerves and high- visual acuity called the fovea. The lens of the eye focuses light on the fovea, and muscles moves the eyes to direct the lens and fovea where they want to look.

An eye-tracking device for measuring eye positions and eye movement can be used in research on the visual system, in psychology, as an input device for human computer interaction, virtual and mixed or augmented reality (VR/AR) applications, and in product design.

The eye-tracking system can detect position and movements of the user's eyes or detect other information about the eyes such as pupil dilation. Other applications can include, but are not limited to, creation of eye image animations used for avatars in a VR/AR environment.

In one example, the invention includes an optical device that includes a first securement arm, a second securement arm, and a lens frame defining a lens aperture and including a nose bridge. The lens frame can be connected to the first and second securement arms. The optical device can further include an eye-tracking camera mounted to a platform extending from the lens frame, the platform disposed proximate the nose bridge.

In some examples, an electronic component can be disposed in the lens frame, the electronic component electronically connected to the camera.

In some examples, the electronic component can include a battery, a sensor, a communication module, or a processor. The electronic component can be disposed in the nose bridge of the optical device.

In some examples, a lens can be disposed in the lens aperture, the lens include an inner surface. A distance between the eye-tracking camera and a surface of an eye during use is less than a distance between the inner surface of the lens and the surface of the eye.

In some examples, the optical device can include a nose pad connected to the platform. The nose pad can include an interchangeable nose pad. In some examples, the interchangeable nose pad can include a clamp, a snap, or a magnet. In some examples, the platform can be removably connected to the lens frame.

According to some aspects, an eye-tracking system can include an optical device having a frame defining a nasal region and a lens. The eye-tracking system can also include a camera disposed in the nasal region and a processor connected to the camera. The processor can be configured to receive an image from the camera and identify a property of an eye based on the received image.

In some examples, the camera can include an infrared camera. In some examples, the camera can be oriented at a horizontal angle between about 30° and about 50° relative to the lens and a center of an eye during use.

In some examples, the camera can be oriented at a vertical angle between about 0° and about 30° relative to the lens and a center of an eye during use. The camera can include a resolution between about 320 and about 640 pixels.

In some examples, the camera can include a lens having a diameter between about 2mm and about 4mm. In some examples, the camera can be a first camera and the eye-tracking system further includes a second camera disposed in the nasal region. In some examples, the processor can be disposed in the frame of the optical device.

A head mounted display (HMD) can include a frame defining a platform in a nasal region of the frame, the platform extending from the nasal region and a lens disposed in a frame. The smart eyeglass system can also include a nose pad secured to the frame, a camera mounted to the platform adjacent the nose pad and a sensor responsive to an eye movement of the wearer. In some examples, the platform and the nose pad can be calibrated to a face of the wearer. The lens can include a prescription lens.

In some examples, the HMD can further include a processor connected to the camera. The processor can receive an image from the camera and determine the gaze point of an eye.

In Apple's patent FIG. 1 below, the smartglasses system is generally configured similar to traditional spectacles or eyeglasses. Optionally, one or more components of the optical device #100 may be interchangeable (e.g., to allow different size, shape, and/or other configurations of components to be exchanged). For example, the assembly #104 can include components from a modular kit, as desired based on a particular individual user. For example, the lens #108 can be replaced with a different lens to correspond to a different user's prescription or to correspond to the activity of the user. 

Apple's patent FIG 2 below shows a back view of the optical device #100 with an eye-tracking camera #118 disposed in the lens frame #106. The eye-tracking camera can be mounted to a platform #120 that extends from the lens frame towards the face of the wearer. The lens frame can define a nasal region and the lens. In some examples, the platform can be connected to the lens frame by any suitable connector or can be integrated into the lens frame The platform can be disposed in the nasal region proximate the nose bridge #114 The eye-tracking camera can be mounted in the nasal region near the nose bridge and on the platform  so that the eye-tracking camera has an unobstructed view of the eye of the wearer. The positioning in the nasal region near the nose bridge can provide a good angle to the parts of the eye required for tracking without obstruction from the eye lashes or other impediments. The platform extends the camera closer to the eye of the wearer so that the lens or the frame doesn't obstruct the camera. The Communications Module #123 can send notifications or alerts to other electronic devices.

Apple's patent FIG. 3 above is an enlargement of the lens area. The lens of the camera can be optimized to provide the required range and field to capture the eye motions of the wearer for the required eye-tracking. The camera #118 includes a size to reduce or minimize any obstruction of the lens #108 by the anatomy of the user during use. 

In some examples, the camera 118 can include an infrared camera. In some examples, a near-infrared light can be directed towards the center of the eyes (pupil), causing detectable reflections in both the pupil and the cornea. The reflections that are the vector between the cornea and the pupil can then be tracked by the infrared camera

In Apple's patent FIG. 7 below we can see an enlarged view of the lens frame #106 of the optical device with the platform #120 being removably connected to the lens frame. In some examples, to ensure that the camera is positioned as required, the platform can be interchangeable. In some examples, the platform can include a clamp, a snap, or a magnet to secure the platform to the lens frame. The platform can include various types of cameras. In some examples, a first platform can include an infrared camera and a second platform can include a standard camera. In other examples, the platform can be interchanged with another platform that extends further towards the face of the wearer.

Apple's patent FIG. 8 below shows a back view of the optical device with an eye-tracking system including a first camera 118a and a second camera 118b disposed in the frame of the optical device The first camera 118a and the second camera 118b can be disposed in the nasal region defined by the frame. The processor #130 is also shown disposed in the frame.

Apple's patent FIG. l0A above shows a top view of a wearer of the eye-tracking system and FIG. 10B shows a side view of a wearer of the eye-tracking system. In some examples, the camera can be positioned to capture the gaze of the wearer.

For eye-tracking, an angle orientation between the camera and the lens  of the optical device can include a compound angle that orients the camera from the nasal region towards the eye #140. The camera optical paths shown in FIGS. 10A-10B have advantages over other eye-tracking systems. The views of FIG. 10A- l0B allow a more centered view of the eye and because the camera does not pass through the eyepiece, there is no distortion caused by the lens.

For full details, review Apple's patent application at WIPO under #WO2024006966. WIPO is Headquartered n Geneva Switzerland.