Apple's Patent Background

One feature of some versions of digital image management software is the ability to analyze digital images and determine whether a person that appears in one digital image also appears in another digital image. This analysis may be performed in response to a user manually locating a person in a digital image or that person being automatically detected by the software. The digital image management software then automatically searches through other digital images (managed by the digital image management software) to identify digital images of the same person. In this way, a group of digital images may be associated based on a particular person that is identified in each. For example, all photos with "Grandma Jane" may be automatically identified without the user having to manually identify Grandma Jane in each digital image.

The automatic detection of a person's face in a digital image is referred to as facial detection. The automatic recognition of a person's face in a digital image is referred to as facial recognition. In order to automatically recognize a person's face that is detected in a digital image, facial detection/recognition software generates a set of features or a feature vector (referred to as a "faceprint") that indicate characteristics of the person's face. (A "faceprint" is a subset of feature vectors that may be used for object recognition. Thus, feature vectors for non-facial objects can be generated and used for object recognition.) The generated faceprint is then compared to other faceprints to determine whether the generated faceprint matches (or is similar enough to) one or more of the other faceprints. If so, then the facial detection/recognition software determines that the person corresponding to the generated faceprint is likely to be the same person that corresponds to the "matched" faceprint(s).

Once the digital images with Grandma Jane in them are identified, the digital image management software may store data that associates all digital images, taken by the user, that have Grandma Jane identified in them. However, such associations are only made of people identified in pictures managed by the digital image management program.

Faceprint Management of Friends, Famous People or Noteworthy Objects

"Faceprints" of famous people and/or noteworthy objects are provided to a digital image management application without the digital image management application having to generate the faceprints or manage the digital images from which the faceprints were generated. The "famous" or "noteworthy" faceprints are maintained separately from faceprints generated by the digital image management application, which are generally of friends, family, and other associates of the user of the management application.

According to one technique, when the digital image management application analyzes a digital image, detects a face, and generates a faceprint, that faceprint is first compared to the "friends" faceprints. If no match is found, then the generated faceprint is compared to "famous" faceprints. The threshold for determining a match between faceprints may be different depending on whether the comparison is with "friends" faceprints or "famous" faceprints.

Automatically Identifying Movie Stars

Apple's patent provides techniques for identifying iconic images or famous faces in digital images. Specifically, rather than merely finding other photos that have "Grandma Jane" in them after the user has manually identified a photo with Grandma Jane, digital image management software that employs the techniques described in this invention may, for example, identify those photos, within a user's collection, that have famous movie stars in the form of a typical or iconic image. Further, the famous movie stars may be identified in the photos automatically without the user having to manually identify the movie star in any photo. Movie stars may even be identified automatically even in situations where the user does not know or would not recognize the movie star himself.

Iconic Faceprints

In one embodiment, digital image management software accomplishes the automatic recognition of famous or noteworthy faces by using a database of "iconic faceprints." A "faceprint" is data that indicates different features of a person's face. These "iconic" faceprints are not of faces identified in digital pictures taken by the end-user of the digital image management software. Instead, iconic faceprints are faceprints of a famous person's face, such as Tom Hanks, or of an iconic image, such as the Mona Lisa, Michelangelo's David, or Santa Claus, that are generated separately from and remotely relative to the digital image management software used by an end-user. In one embodiment, iconic faceprints are "shipped" with the digital image management software, unlike the facial recognition data that is created based on user photos that are accessible to the management software.

Iconic Objects with Multiple Faceprints

In another aspect of Apple's invention, one or more objects identified in remotely-generated faceprint database may be associated with multiple faceprints. For example, multiple faceprints of Paul McCartney may be stored in remotely-generated faceprint database. In such an embodiment, such faceprints are selected based on how Paul McCartney has changed (e.g., aged) over time. For example, the remotely-generated faceprint database may include a faceprint for each decade of his life beginning with, e.g., the 1960's decade. Thus, the remotely-generated faceprint database may include at least 5 faceprints, each of which is associated with Paul McCartney.

In a related embodiment, a faceprint is generated for multiples views of an object, such as a person's face. For example, in addition to a direct view of Paul McCartney, one or more faceprints may be generated for profile views (e.g., left and right views) of Paul McCartney's face. Thus, the remotely-generated faceprint database might include, for each decade or other time period, multiple faceprints for multiple left profile views of Paul McCartney's face.

Apple's patent FIG. 1 is a block diagram that depicts an example digital image management system; patent FIG. 2 is a block diagram that depicts a logical structure of a remotely-generated faceprint database; and patent FIG. 3 is a flow diagram that depicts a process for using the remotely-generated faceprint database.

Patent Credits

Apple credits Jeremy Holland, Jan Solem and William Hensler as the inventors of this patent which was originally filed under serial number 158210 in Q2 2011. It should be noted that Jan Eric Solem owned Polar Rose, a company that Apple acquired back in September 2010. Their technology has since been incorporated into Apple's new facial recognition or detection feature that was introduced in iPhone 4S's camera. He was also listed as one of the inventors of a patent granted to Apple a year ago relating to 3D object recognition. Other Facial Recognition patents from Apple could be found in our archives.

Continuation Patents Published Today

In addition to the noted patent applications presented in today's report, the US Patent and Trademark Office did publish a series of older continuation patents dating back to between 2007 and 2010. The continuation patents that we list below are specifically referenced as such under the section titled "Cross-Reference to Related Applications."

Generally speaking, continuation patents represent tweaks made to patent claims in an effort to get the patents granted by the US Patent Office and don't necessarily represent any noteworthy new development from the original patent filing. With that said, here are today's continuation patents should you wish to review them:

Old Gaming Patent Revised

20120315994: This is a Continuation of an old 2009 Patent Application Titled "Interactive Gaming with Co-Located, Networked Direction and Location Aware Devices." We covered this patent in a November 2010 report titled "Apple Working on an Interactive Laser Tagging Game or Framework."

20120315864: This is a Continuation of an old 2007 Patent Application Titled "WIRELESS ADAPTER FOR MEDIA PLAYER SYSTEM."

20120315880: This is a Continuation of an old 2009 Patent Application Titled "DYNAMIC CONTEXT-BASED AUTO-RESPONSE GENERATION."

20120317441: This is a Continuation of an old 2008 Patent Application Titled "NON-FAULTING AND FIRST FAULTING INSTRUCTIONS FOR PROCESSING VECTORS."

20120313911: This is a Continuation of an old 2010 Patent Application Titled "AMBIENT LIGHT CALIBRATION FOR ENERGY EFFICIENCY IN DISPLAY SYSTEMS."

NOTICE: Patently Apple presents a detailed summary of patent applications with associated graphics for journalistic news purposes as each such patent application is revealed by the U.S. Patent & Trade Office. Readers are cautioned that the full text of any patent application should be read in its entirety for full and accurate details. Revelations found in patent applications shouldn't be interpreted as rumor or fast-tracked according to rumor timetables. About Comments: Patently Apple reserves the right to post, dismiss or edit comments.

Notice to Patently Apple Facebook Fans

Due to constructive feedback from our many Facebook fans, we've now added the handy "Subscribe" button to our Facebook page as of December 10, 2012.



Top Industry Players are mapping out the Future of Dual Display Devices

Sony Working on WALL-E-Like Robots for Care Giving Market