Apple wins a patent for a Home Security Camera System that recognizes visitors or intruders using Face ID & new 'Bodyprint' ID
Today the U.S. Patent and Trademark Office officially granted Apple a patent that relates to a possible future home security system. More specifically, the patent covers techniques for providing a notification indicating an identity of a first person based on face-associated body characteristics. For example, a device performs facial recognition to identify the identity of the first person shown in a first video feed. The device also identifies and stores physical characteristic information of the first person from the first video feed, the stored information associated with the identity of the first person based on the recognized face.
Apple notes that a device may receive a first video feed, for example, showing a particular person (e.g., including their face and torso portion of their body) walking towards a camera. The device may perform facial recognition to identify the identity of the particular person. The device may further identify physical characteristics of the particular person from the first video feed, for example, based on analyzing other body features of the particular person (e.g., including their torso, clothing worn, etc.). The device may then store physical characteristic information of the particular person in association with the identity of the particular person based on having recognized the face of the particular person from the first video feed. A gallery of images may be stored, where each image of the gallery includes a torso, and the images of the torso can be associated with the particular person who's face had been identified.
In some instances, the identified face may correspond to a person in a contacts list, such that the face/person are known by an owner/user of the device. Subsequently, the device can receive a second video feed showing a second person whose face is determined to not be recognized by the device (e.g., an obstructed view or poor image quality) or is not visible to the device (e.g., walking away from the camera). '
The device can compare the stored physical characteristic information of the first person (e.g., the gallery of images) with additional physical characteristic information of the second person shown in the second video feed.
Additionally, the device may provide notifications of the presence of a particular person in a wider range of scenarios and/or with higher precision/recall, for example, including when the person's face may not be shown in a video feed and/or recognizable by the device. In some examples, the gallery of images corresponding to physical characteristics (e.g., torso, etc.) may only be stored for a particular amount of time (e.g., one day, one week, or the like). In this way, the gallery of images may be repopulated each day (or longer) for each detected person.
Apple's patent FIG. 1 below is a simplified block diagram 100 that illustrates a system notification service operating in an example environment; FIG. 2 is another simplified block diagram illustrating at least some example techniques for providing a notification based on determining the presence of a particular person at a location.
The resident device may generate a bodyprint of the first person. The bodyprint may include a multidimensional vector that is associated with (e.g., represents) the one or more physical characteristics of the first person, whereby a dimension of the vector may be associated with at least one characteristic of the body of the contact.
The home resident will be able to view the person at the door using a home TV with a picture-in-picture feature, Tablet, HomePod (with a tablet), iPhone and more.
Apple's camera system will use a deep learning model to train itself in identifying bodyprints. Over time, the system will be able to identify false bodyprints.
Apple's patent FIG. 4 below illustrates at least some example techniques for providing a notification based on determining the presence of a particular person at a location. In diagram #400 of FIG. 4, several elements are depicted, including observation camera #402, a video feed #404, body croppings #408, a user device #406, a pop-up notification #410, and a video presentation #412.
Apple's patent FIG. 8 above is a simplified flow diagram illustrating a process for determining whether to add a bodyprint image to a cluster of bodyprint images corresponding to a recognized person.
To review the full details of this invention, check out granted patent 12154386.
Apple Inventors
- Hendrik Dahlkamp: HomeKit Secure Video / Machine Learning Manager
- Vinay Sharma: AI/ML, Computer Vision, Deep Learning. Human and Object Understanding
- Floris Chabert: Applied research
- Jingwen Zhu: Research Engineer (previous employers: Konica Minolta and Samsung)
- Nitin Gupta: Machine Learning Engineer (came to Apple in 'Lighthouse' acquisition)
- Jonghoon J.: Deep learning for computer vision (came to Apple via Lighthouse acquisition)
- Andrew Edwards: Machine Learning Engineer (No longer at Apple)