Apple's Patent Background

Digital photography, a popular pastime, utilizes digital technology to produce digital images. Digital images (or "images") also can be created, for example, by graphical editing software (e.g., photo editing or drawing software), digital video equipment, and scanners.

After production, an original image (e.g., a photograph) can be processed in many ways. Some instances of image processing can produce a new image from an original image. For example, cropping (i.e., extracting a portion of an image) and transforming (e.g., convolving an image with a convolution kernel) can produce a new image from an original image. Other instances of image processing can ascertain information about an original image without producing a new image. For example, facial-detection processing can determine whether an image depicts one or more faces.

Images (e.g., photographs) can depict one or more subjects, such as people or objects, with varying levels of detail. A sharp image, having a high level of detail, can include visual features (e.g., fine details of a flower) that would not be discernible in an image with a comparatively low level of detail. Further, image detail can correspond to image components of varying frequencies (e.g., high frequency or low frequency).

Advancing the Accuracy of Facial Recognition

Apple's engineers recognized a need to assess the accuracy of a detection made by a facial detection system. Further, the need to determine whether an image includes sufficient detail to perform an accurate facial recognition process also was recognized. Additionally, the need to eliminate blurry images having fewer high frequency image components relative to low frequency image components in an overall facial correlation system was recognized. In addition, the need to eliminate facial detections in images not corresponding to facial representations in an overall facial correlation system was recognized.

In general, one aspect of the subject matter described in Apple's invention can be implemented in a method that includes receiving an original image; transforming the original image to generate one or more blurred images; deriving image differences corresponding to ranges of detail frequency of the original image based, at least in part, on the one or more blurred images and the original image; determining, based on the image differences, a detail level value corresponding to the original image; and providing the detail level value to an image management application.

These and other implementations can optionally include one or more of the following features. Generating the one or more blurred images can include generating blurred images by successively and cumulatively transforming the original image. The transforming the original image can include convolving the original image using a kernel configured to graphically blur an image. The deriving the image differences can include averaging absolute corresponding pixel differences between one of the one or more blurred images and the original image. The deriving the image differences can include comparing single color channels of the one or more blurred images and the original image. The deriving image differences further can include deriving a high frequency difference, a medium-high frequency difference, a medium-low frequency difference, and a low frequency difference. The determining the detail level value further can include dividing a lowest high frequency value selected from the high frequency difference and the medium-high frequency difference by a highest low frequency value selected from the medium-low frequency difference, the low frequency difference, and a constant value representing substantially zero frequency difference. The original image can be received from the image management application; and the provided detail level value can indicate a likelihood of the original image depicting a face recognizable by the image management application.

Advantages to Apple's New Techniques

Apple states that the accuracy of facial detection and recognition can be improved by eliminating images having insufficient detail (e.g., non-facial image data or unclear facial image data). Aspects of image detail level detection (e.g., image transformation and image comparison) can permit efficient implementations (e.g., detecting perceptible detail at an 8-bit image depth rather than a higher depth). Efficient implementations can increase overall processing-resource efficiency and can be used in real-time systems.

One or more of the following additional advantages can also be realized. Using one color channel for image detail level detection rather than more than one color channel included in the original image can provide several advantages. For example, using one channel can decrease the image data (e.g., an 8-bit grayscale image rather than a 24-bit color image) to be processed thus decreasing the computing resources required. In another example, a given channel (e.g., the green or red channel) can include less noise than the original image improving the quality of the detail level detection.

In Apple's patent FIG. 1 above we see an exemplary overview of detecting image detail levels. Apple illustrates a washed-out image (102), a blurry image (104), a clear image (106) and a noisy image (108) of faces that could be interpreted and analyzed with greater detail with their new system.

New Image Management Application for Detecting Image Detail Levels

In Apple's FIG. 2 shown above we see an exemplary image management application (#200) for detecting image detail levels. The image management application can be a software application executing on a computer system.

The image management application can include a user-interface through which the image management application receives input from a user and displays output to the user. Although shown as a single application, in some implementations the functionality of the image management application can be accomplished using multiple applications.

An image correlator 202 can be an application, process, or system component configured to correlate images. For example, the image correlator can correlate images of a same individual using facial detection and facial recognition functionality.

About Apple's Patent Figures: Apple's patent FIG. 4 illustrated above is a dataflow diagram showing an exemplary detail detection dataflow; FIG. 5A is a bar chart representing band pass information corresponding to a sharp image; FIG. 5B is a bar chart representing band pass information corresponding to a blurry image distorted by random noise artifacts; FIG. 5C is a bar chart representing band pass information corresponding to a substantially blank image; FIG. 5D is a bar chart representing band pass information corresponding to a blurry image.

Patent Credits

Apple credits Ben Weiss as the sole inventor of this patent application which was originally filed under serial number 661680 in Q4 2012. I couldn't find a clear link to Ben Weiss holding a current position at Apple. This patent could have been acquired by Apple from this Ben Weiss, though we can't confirm this. Weiss invented the photo-editing and retouching application called "Kai's Photo Soap," which was a part of a MetaCreation's application.

Today's patent application goes towards strengthening Apple's established body of work regarding all aspects of facial recognition and detection.

A Few of Apple's other Patent Applications that were Published Today

Review Managing Access to Digital Content Items: Apple's invention relates to managing access to digital content items among end-users and preventing a transferor from accessing a digital content item after authorized access to the digital content item has been transferred to another. Other patent applications on this subject could be reviewed here and here.

Review Charge Recycling System and Method: Apple's invention generally relates to a technique for implementing charge recycling in a display device.

Review Method for Estimating Temperature at a Critical Point: Apple's invention generally relates to data processing systems and more particularly but not exclusively to the estimation of temperatures in data processing systems.

Patently Apple presents a detailed summary of patent applications with associated graphics for journalistic news purposes as each such patent application is revealed by the U.S. Patent & Trade Office. Readers are cautioned that the full text of any patent application should be read in its entirety for full and accurate details. Revelations found in patent applications shouldn't be interpreted as rumor or fast-tracked according to rumor timetables. About Comments: Patently Apple reserves the right to post, dismiss or edit comments.