Today the U.S. Patent and Trademark Office officially granted Apple a patent that relates to a next-gen HomePod that will provide a built-in camera system that supports gaze controls, longer-range Face ID and voice recognition so as to execute commands given by those in a home (or office) environment.

Apple's granted patent also describes a camera accessory system that users could place throughout a room (and their home) that work as added eyes for this possible future version of HomePod.

The added cameras are able to capture a user's eyes and gaze directions when requesting Siri to perform a task such as turning on a very specific lamp in a room that has multiple lamps, for instance.

Apple's patent Figures 8A to 8C presented below illustrate one scenario where a user looks at HomePod while giving the command "turn ... on that light." The user first looks at HomePod and then shifts his head to gaze at the specific lamp that he wants turned on. HomePod executes the command while Siri responds "Table lamp turned on."

In the series of patent figures above you'll also see new Apple camera accessories (#816A) laid out in the room with one on the top shelf of a wall unit while the other is sitting on a window sill shown behind the user as pointed out in patent figure 8b.

In patent figure 8J below, the user is giving HomePod/Siri a command while not looking at HomePod directly. In this case HomePod will access the connected camera accessories in the room to determine what lamp the user is requesting to turn on, this time being the room's stand-up lamp.  

In the second round of patent figures presented below you're able to see a woman making eye contact with HomePod's camera and requesting that a particular lamp be turned on.

The response is personal with Siri acknowledging that it's "Jane" who made the request.  This indicates that Apple's next-gen TrueDepth camera used on this future HomePod will be able to use Face ID at a distance instead of the current version on iPhones that has a limited range. The ID first has to have been set-up on an iDevice previously for Face ID to work on this future version of HomePod.

Also provided is patent FIG. 6o (letter o) that outlines the camera system as well as a series of light formations located around the top of HomePod.

One set of lights in a certain color group, like purple, could be used to communicate that the command was not understood. Another group of lights in a second color could be used to communicate that the person making the command could not be confirmed or isn't authorized to use the device by using the Face ID aspect of the camera.

As an alternative, there could be an image of the lamp on a user's Apple Watch and by applying force touch on the lamp's image on the display, the brightness of the lamp could be adjusted as shown in the patent figures 8D and 8E below.  

Apple's patent FIG. 1A above is a flow chart schematic in-part that illustrates the HomePod's I/O Subsystem designed with an Optical Sensor Controller; a Depth Camera Controller and sensors; an Intensity Sensor Controller and more.

A Little More on the Depth Camera

Apples patent filing notes that the device could include one or more depth camera sensors that receives data from the environment to create a three-dimensional model of an object (e.g., a face) within a scene from a viewpoint (e.g., a depth camera sensor) - which translates to a longer-range version of the depth camera for Face ID.

At another point in the patent Apple's engineers state that "In some embodiments, electronic device #1000 (HomePod) obtains information about the user speaking using external cameras (accessory cameras), an iPhone, or the integrated camera in HomePod.

In some embodiments, the HomePod uses the obtained information to perform facial recognition, voice recognition, or to calculate a distance to the user speaking to determine a user identity.

For more details, review Apple's granted patent US 11619991 B2.

Apple Inventors

• Karlin Bark: Experience Design and Prototyping / Interaction Architecture
• Sean Kelly: Software Engineer
• Felipe Bacim: UX Prototyping Engineer