By now most know that advanced 3D Facial Recognition is coming to Apple's high-end iPhone launching in September. Once Apple has successfully delivered this next-gen feature, Apple is likely to bring it over to the Mac and today's patent is all about just that. Apple has been working on both facial and presence recognition for some time now. In a new patent application published by the U.S. Patent Office today, we learn that Apple is working on a sophisticated system that combines both features and much more for future Macs. In fact, the proposed system introduces a security method for Macs that combines presence, face detection routines and motion detection to determine authentication of the user to first login to their system and to wake it up from sleep. Apple further notes that "The fusion and detection logic may include the use of neural networks, support vector machines, and/or some other form of probabilistic machine learning based algorithm to arrive at a determination of whether a user is present."

Patent Background

Many computing devices are equipped with power saving features/modes intended to reduce power consumption when a user is not using the devices. Often, these power saving features are implemented though timers that count down a set amount of time from when the user last provides an input to the device. For example, a particular device may be configured to enter a sleep mode, or other mode that consumes less power than a fully operational mode, when a user has not provided input for five minutes.

Occasionally, however, a device may enter the power saving features/modes while a user is still using the device. For example, the power saving features may be entered because the user failed to provide input within the time period set for the timer while reading content on the device, viewing a movie, or listening to music. Additionally, recovery from the power saving feature/mode may take time, may even require the user to enter credentials, and generally may be a nuisance to the user.

Presence Detection

One embodiment of Apple's invention may take the form of a method of operating a computing device to provide presence based functionality. The method may include operating the computing device in a reduced power state and collecting a first set of data from a first sensor. Based on the first set of data, the computing device determines if an object is within a threshold distance of the computing device and, if the object is within the threshold distance, the device activates a secondary sensor to collect a second set of data. Based on the second set of data, the device determines if the object is a person. If the object is a person, the device determines a position of the person relative to the computing device and executes a change of state in the computing device based on the position of the person relative to the computing device. If the object is not a person, the computing device remains in a reduced power state.

Additionally, a computing system is provided having a main processor and an image based presence sensor coupled to the main processor. The image based presence sensor includes an image sensor, and a processor coupled to the image sensor and processor configured to process the image to determine if a user is present in the image. If the processor determines that a user is present in the image, an indication that a user has been determined to be present is sent from the processor to the main processor and the main processor changes a state of the computing system based on the indication.

Advanced Facial Recognition

Another embodiment of Apple's invention may take the form of a method for determining if a user is in proximity of a computing device. The method includes capturing an image using an image sensor and computing at least one of the following from the captured image: a skin tone detection parameter, a face detection parameter and a movement detection parameter. The method also includes utilizing at least one of the skin tone detection parameter, face detection parameter and the movement detection parameter to make a determination as to whether a user is present and, if it is determined that a user is present, changing a state of the computing device.

In Apple's patent FIG. 6 noted below we're able to see a flowchart #200 illustrating presence sensing. Initially, a camera is used to obtain an image (Block 202). A light level determination may be made (Block 204) and provided to a skin tone detection routine (Block 206). Optionally, in some embodiments, the light level determination may be provided to other routines as well, as indicated by arrow #203. Additionally, the captured image may be pre-processed (Block 208). In some cases, the preprocessing may include down scaling of the image, changing the color space of the image and/or enhancing the image, for example. Other detector specific preprocessing may also be performed (Blocks #214, 215 and 217). For example, the image may optionally be blurred by preprocessing in Block 214 before being provided to the skin tone detection routine (Block 206). Additionally, preprocessing in Block 215 may include changing color into grayscale before providing the image to the face detection routine (Block 210) and/or performing edge detection in the preprocessing of Block 217 before providing the image to the movement detection routine (Block 212). The skin tone detection routine, face detection routine and movement detection routine are discussed in greater detail below with reference to FIGS. 7-11.

The results of the skin tone detection routine, face detection routine and movement detection routine may be weighted and combined using fusion and detection logic (Block 216) and a user presence classification is determined (Block 218).

The fusion and detection logic may include the use of neural networks, support vector machines, and/or some other form of probabilistic machine learning based algorithm to arrive at a determination of whether a user is present.

In Apple's patent FIG. 10 below we're able to see possible sets of windows into which the images may be divided for single frame motion detection. Specifically, FIG. 10 shows single frames divided into both non-overlapping windows and concentric windows for statistical purposes. In each case, the luminace of the images (frames at the top of page) and intensity gradient magnitude of the images (frames at the bottom of the page) are considered. For example, an image #300 may be divided into multiple non-overlapping exposure statistics windows #302. Alternatively, the image #300 may be divided into multiple concentric overlapping exposure statistics windows #304. The statistics for each window may be determined based on a luminance image (as in image 300) or based on an intensity gradient magnitude image # 306. The use of windows provides more robust capture of motion when computing sums of the gradient magnitude.

Apple's patent application 20170193282 was filed back in February 2017. For more details about this invention, click here. Considering that this is a patent application, the timing of such a product to market is unknown at this time.

Patently Apple presents a detailed summary of patent applications with associated graphics for journalistic news purposes as each such patent application is revealed by the U.S. Patent & Trade Office. Readers are cautioned that the full text of any patent application should be read in its entirety for full and accurate details. About Making Comments on our Site: Patently Apple reserves the right to post, dismiss or edit any comments. Those using abusive language or negative behavior will result in being blacklisted on Disqus.