On February 20, 2014, the US Patent & Trademark Office published a patent application from Apple that reveals newly advanced facial and feature recognition methods of combining multiple image detectors to obtain robust feature detection. Apple's previous work in this area covered advancing the accuracy of facial recognition and how facial recognition could one day apply to locking and unlocking devices. Theoretically speaking, Apple could one day add facial recognition to the authentication process as a secondary security measure to Apple's fingerprint based Touch ID feature. At present, a combination feature such as this has been explored by Google and others.
Apple's Patent Background
Digital images can be analyzed to identify certain features of interest in an image. For example, feature detectors may analyze an image to identify faces, people, pets, or other objects of interest. The feature detectors identify certain regions of an image that exhibit properties of the feature that the detector is configured to identify. For example, a face detector may identify portions of an image having characteristic shapes, textures, or colors that are similar to the properties of known faces used to train the detector. However, the properties of a feature of interest may vary widely from image to image.
For example, a face in an image captured in bright light may have different properties than the same face in an image captured indoors with lower lighting. Similarly, a forward-looking face may have different properties than a side view of the same face. Moreover, in certain images, features that are important to the detection of a particular feature may be occluded. For example, a face detector that relies on the location of a subject's eyes may not recognize a face in an image where one of a subject's eyes is occluded (e.g., by hair in front of the eye in the image).
In light of these limitations, it would be desirable to combine information from multiple individual feature detectors to obtain a robust detector to identify a combined feature in an image.
Facial Recognition Invention: Method of Combining Multiple Image Detectors
Apple's invention generally relates to identifying features in an image. More particularly, but not by way of limitation, the invention relates to techniques to combine individual feature detectors configured to identify different image properties to obtain robust feature detection.
Apple notes tht multiple feature detectors may identify individual parts of a combined feature. For example, individual feature detectors associated with a face detector may identify different facial feature regions (e.g., eyes, upper face, nose, ears, lower face, etc.).
Similarly, individual feature detectors associated with a person detector may identify different person feature regions (e.g., face, torso, clothing patters, arms, legs, etc.).
Although the multiple feature detectors associated with a clustering rule may identify individual parts of the combined feature, Apple states that the multiple feature detectors may also identify the same feature based on the detection of different properties. For example, a face detector clustering rule may include a first individual feature detector that identifies an entire face based on color and a second individual feature detector that identifies an entire face based on texture. Accordingly, the individual feature detectors that make up a clustering rule are not necessarily limited to the detection of parts of the combined feature but are instead directed to the detection of features associated with the combined feature.
In one embodiment, the invention provides a method to analyze a digital image with multiple feature detectors that are associated with a detection rule to collectively identify a combined feature. The analysis of the digital image by the multiple feature detectors may result in the identification of multiple candidate regions. The detection rule may operate directly on the multiple candidate regions to organize the candidate regions into candidate region groups and to detect the combined feature based on a candidate region group that satisfies the detection rule.An indication of the detected combined feature may be stored in a memory. The method may be embodied in program code and stored on a non-transitory medium.
In another embodiment, the invention provides a method to select multiple individual feature detectors that are each configured to detect a component in a digital image. Each of the individual feature detectors may have corresponding transformation parameters that relate the component to a combined feature. Application of the multiple individual feature detectors to an image may result in the identification of multiple candidate regions. The spatial properties of the candidate regions may be adjusted based on the transformation properties of the individual feature detector that identified the candidate region. It may then be determined that the adjusted candidate regions are indicative of one or more of the combined features in the digital image and an indication of the one or more combined features may be stored in a memory. The method may be embodied in program code and stored on a non-transitory medium.
In yet another embodiment, the invention provides a method to identify multiple candidate regions in an image based on an analysis of the image by multiple feature detectors that are part of a combined feature detection rule. The geometric properties of the candidate regions may be adjusted based on the feature detector that identified the candidate region and the adjusted candidate regions may be clustered into related candidate region groups. One or more combined feature regions may be identified based on the related candidate region groups and an indication of the combined feature regions may be stored in a memory. The method may be embodied in program code and stored on a non-transitory medium.
In still another embodiment, the invention provides a method to receive a selection of a combined feature of interest to be detected in an image. A combined feature detection rule that specifies multiple individual feature detectors may be selected based on the indicated combined feature of interest. An analysis of the image with the multiple individual feature detectors specified in the rule may result in the identification of multiple candidate regions. The candidate regions may be spatially adjusted based on the feature detector that identified the candidate region and grouped into candidate region groups. One or more regions of the image that contain the combined feature of interest may be identified based on the candidate region groups and data that describes the regions may be stored in a memory. The method may be embodied in program code and stored on a non-transitory medium.
Apple's patent FIG. 4 noted below illustrates the spatial adjustment of a candidate region identified by an individual feature detector having associated translational and scale transformation parameters
In Apple's patent FIG. 5 noted below we see the spatial adjustment of a candidate region identified by an individual feature detector having associated rotational, translational, and scale transformation parameters
And Lastly in Apple's patent FIG. 6 we see the grouping of multiple spatially adjusted candidate regions identified by multiple individual feature detectors.
Apple credits Jan Solem, Oualid Merzouga and Michael Ruosson as the inventors of patent application 20140050404 which was originally filed in Q3 2012. For More details, see Apple's invention. To review more of Apple's patent pending inventions on facial recognition, including presence detection and pattern detection, see our Archives.
Patently Apple presents a detailed summary of patent applications with associated graphics for journalistic news purposes as each such patent application is revealed by the U.S. Patent & Trade Office. Readers are cautioned that the full text of any patent application should be read in its entirety for full and accurate details. Revelations found in patent applications shouldn't be interpreted as rumor or fast-tracked according to rumor timetables. About Making Comments on our Site: Patently Apple reserves the right to post, dismiss or edit any comments.
New on Patent Bolt this Week
Samsung Invents a Foldable Display that Integrates Ultrasonic and Infrared Sensors to Better Guide a Possible Next Generation Pen