Early this morning, Patently Apple discovered a patent application that Apple filed in Europe titled "Identity Recognition Utilizing Face-Associated Body Characteristics." The patent covers a home surveillance camera system with indoor and outdoor cameras communicating with a person in the home that someone is at the front or back door. The camera will stream live video of the visitor or criminal on an iPad, iPhone or HomePod (as shown in our cover graphic). Whether Apple intends to work with third party developers on this system or is planning to sell such a system directly in a store or part of a future Home Installation Service is unknown at this time.

In Apple's patent background they note that techniques exist for performing identity recognition of a person using an image of the person's face. For example, a mobile phone camera may capture an image showing a portion of a person's face. An application may analyze the image to determine characteristics of the person's face and then attempt to match the person's face with other known faces. However, identity recognition is a growing field and various challenges exist related to performing recognition. For example, sometimes a video camera may not be able to perform facial recognition of a person, given a particular video feed (e.g., if the person is facing away from the camera). In these scenarios, it may be difficult to accurately perform identity recognition of the person.

Apple's invention describes techniques that enable performing recognition of an identity of a particular person based on physical characteristics of the particular person. The physical characteristics of the particular person may be associated with their identity based on previously performing facial recognition of the particular person.

In one example, a device may receive a first video feed, for example, showing a particular person (e.g., including their face and torso portion of their body) walking towards a camera. The device may perform facial recognition to identify the identity of the particular person. The device may further identify physical characteristics of the particular person from the first video feed, for example, based on analyzing other body features of the particular person (e.g., including their torso, clothing worn, etc.). The dev1ce may then store physical characteristic information of the particular person in association with the identity of the particular person based on having recognized the face of the particular person from the first video feed.

A gallery of images may be stored, where each image of the gallery includes a torso, and the images of the torso can be associated with the particular person who's face had been identified. In some instances, the identified face may correspond to a person in a contacts list, such that the face/person are known by an owner/user of the device. Subsequently, the device can receive a second video feed showing a second person whose face is determined to not be recognized by the device (e.g., an obstructed view or poor image quality) or is not visible to the device (e.g., walking away from the camera).

The device can compare the stored physical characteristic information of the first person (e.g., the gallery of images) with additional physical characteristic information of the second person shown in the second video feed (note: the second person may be the first person; however, this is yet to be determined). Based on the comparison, the device can provide a notification indicating whether the identity of the second person corresponds to the identity of the first person.

The notification may be a message (e.g., a text, email, pop-up, etc.) or it may be a name or visual identifier on a screen displaying the frames or video. In this way, techniques may enable a device to identify a person without having a view or quality image of the face. Additionally, the device may provide notifications of the presence of a particular person in a wider range of scenarios and/or, with higher precision/recall, for example, including when the person's face may not be shown in a video feed and/or recognizable by the device. In some examples, the gallery of images corresponding to physical characteristics (e.g., torso, etc.) may only be stored for a particular amount of time (e.g., one day, one week, or the like). In this way, the gallery of images may be repopulated each day (or longer) for each detected person.

Apple's patent FIG. 1 below is a simplified block diagram #100 that illustrates a system notification service operating in an example home environment #101. The home env1ronment may be associated with one or more people (e.g., contact persons) who have some common affiliation (e.g., family members, roommates, a housekeeper, a babysitter, etc.). ln this example, user #112 may represent an affiliated user (e.g., "Person A," who may be the housekeeper).

Also, within the home environment there may be a device #102 (e.g., an iPad,  a smart home controller [HomePod], an iPhone, a home automation device that is part of a home automation system.  The device (e.g., a resident device of the home) may be communicatively connected to one or more observation cameras associated with the home environment.

For example, the resident device may receive video feeds from observation camera #108 and observation camera #110, both cameras being associated with the home environment. Each video feed may include a plurality of video frames. In this example, both cameras may correspond to mounted observation cameras that are positioned to observe different locations associated with the home environment For example, observation camera #108 may observe user #112 (e.g., including the face and other body characteristics of user 112) approaching the 20 entrance to the home (e.g., nearing the front door of the home) at time T1 (during a first time period).

Meanwhile, observation camera #110 may be positioned to observe an interior area (e.g., a corridor, room, etc.) of the home environment #101. In this example, the observation camera #110 may be positioned so that, during a subsequent time period (at time 'TN) from when the user #112 entered the home, the observation camera #110 may capture user #114.

The device #102 may include a notification service that executes a detection model to detect a person's identity (e.g., a trained machine learning model). As described further herein, the notification service may be enabled to provide a notification of the identity of a person shown in a video feed, even when the face of the person may not be recognized by the notification service.

Apple's patent FIG. 2 below is another simplified block diagram illustrating at least some example techniques for providing a notification based on determining the presence of a particular person at a location.  

(Click on image to Enlarge)

Apple's patent FIG. 3 below is another simplified block diagram illustrating at least some example techniques for providing a notification based on determining the presence of a particular person at a location.

(Click on image to Enlarge)

Apple's patent FIG. 4 below is another simplified block diagram illustrating at least some example techniques involving a user interface (UI) for providing a notification based on determining the presence of a particular person at a location.

(Click on image to Greatly Enlarge)

This is a very long patent application packed with details. To review patent application WO2022241294, click here. The International filing was originally made in May 2022 and published publicly on Thursday November 17, 2022.

Patently Apple posted a patent report back in April 2020 that first described a future HomePod having an integrated camera. Then in December 2021, we covered Apple's first patent application relating to a future home surveillance system. In that patent, the home user could view who was at the door via a camera system stream shown on the user's Apple TV device.

(Click on image to Enlarge)

Today's patent revelations expand on Apple's project that started in 2020 by adding a HomePod to the mix of devices that video could be streamed to in a home surveillance system.

Today's Patent Inventors

Vinay Sharma: AI/ML, Computer Vision, Deep Learning. Human and Object Understanding

Hendrik Dahlkamp: Machine Learning Manager, HomeKit Secure Video

Nitin Gupta: Machine Learning Engineer

Jingwen Zhu: Research Engineer

Jonghoon J.: Deep learning for computer vision

Floris Chabert. Applied Research

Andrew Edwards: No LinkedIn profile found