When the Apple Vision Pro was introduced last week, Apple fans were blown away. Apple's eye-tracking / gaze-tacking technology came to life providing their new headset with leap frog technology for Spatial Computing. Even touch reviewers like Marques Brownlee described Apple's eye-tracking as magical. Although I've covered eye/gaze tracking patents for years, I really thought this was 5-10 years away. So seeing it working to perfection on Apple Vision Pro made a lot of interesting patents come to mind that could be closer to reality than first thought.

For instance, today the US Patent & Trademark Office published a patent application from Apple that demonstrates their ongoing work to provide a next-gen HomePod with an integrated camera, gaze controls and upgraded version of Siri that could handle commands given by the user and allow Siri to respond in a personal way via an advanced version of Face ID that could recognize users at a greater distance.

The added cameras are able to capture a user's eyes and gaze directions when requesting Siri to perform a task such as turning on a very specific lamp in a room that has multiple lamps, for instance.

Apple's patent Figures 8A to 8C presented below illustrate one scenario where a user looks at HomePod while giving the command "turn ... on that light." The user first looks at HomePod and then shifts his head to gaze at the specific lamp that he wants turned on. HomePod executes the command while Siri responds "Table lamp turned on."

In the series of patent figures above you'll also see new Apple camera accessories (#816A) laid out in the room with one on the top shelf of a wall unit while the other is sitting on a window sill shown behind the user as pointed out in patent figure 8b.

In patent figure 8J below, the user is giving HomePod/Siri a command while not looking at HomePod directly. In this case HomePod will access the connected camera accessories in the room to determine what lamp the user is requesting to turn on, this time being the room's stand-up lamp. 

In the second round of patent figures presented below you're able to see a woman making eye contact with HomePod and requesting that the lamp be turned on.

The response is personal with Siri acknowledging that it's "Jane" who made the request.  This indicates that Apple's next-gen TrueDepth camera used on this future HomePod will be able to use Face ID at a distance instead of the current version on iPhones that has a limited range. The ID first has to have been set-up on an iDevice previously for Face ID to work on this future version of HomePod.

Also provided is patent FIG. 6o (letter o) that outlines the camera system as well as a series of light formations located around the top of HomePod.

One set of lights in a certain color group, like purple, could be used to communicate that the command was not understood. Another group of lights in a second color could be used to communicate that the person making the command could not be confirmed or isn't authorized to use the device by using the Face ID aspect of the camera.

As an alternative, there could be an image of the lamp on a user's Apple Watch and by applying force touch on the lamp's image on the display, the brightness of the lamp could be adjusted as shown in the patent figures 8D and 8E.

A Little More on the Depth Camera

Apples patent filing notes that the device could include one or more depth camera sensors that receives data from the environment to create a three-dimensional model of an object (e.g., a face) within a scene from a viewpoint (e.g., a depth camera sensor) - which translates to a longer-range version of the depth camera for Face ID.

At another point in the patent Apple's engineers state that "In some embodiments, electronic device #1000 (HomePod) obtains information about the user speaking using external cameras (accessory cameras), an iPhone, or the integrated camera in HomePod. 

In some embodiments, the HomePod uses the obtained information to perform facial recognition, voice recognition, or to calculate a distance to the user speaking to determine a user identity.

For finer details and to review Apple's addition of 18 new patent claims, review Apple's patent application number 20230185373. Apple has been working on this invention for over five years.

With Apple now having an advantage with their advanced eye/gaze tracking technology, they should bring this technology to market in HomePod as soon as possible so as to give it an advantage over all smart speakers on the market today.