On Monday Patently Apple posted a report titled "Samsung's Galaxy S10 Face Unlock Feature has been proven to be absolutely useless." Samsung is learning quickly that copying Apple's Face ID isn't as easy as they thought. That's not to say that Face ID is perfect, because it's had a few hiccups when it comes to differentiating between identical twins. Apple's SVP Worldwide Marketing pointed out this minor flaw in his iPhone X keynote segment introducing Face ID by using a slide titled 'Evil Twin' as you can see below.

Since that time Apple has been hard at work inventing ways to secure future iPhones for twins with advanced Face ID features like heat mapping and subepidermal imaging.

Apple notes in a patent application published today by the U.S. Patent Office that when it comes to authentication using facial recognition, there are potential cases where a user attempting to be authenticated (authorized) by a device cannot be distinguished from another user with closely related facial features. For example, it may be difficult for a facial recognition authentication process to distinguish between siblings, twins, and other closely related faces. When the authentication process cannot distinguish between closely related faces, additional authentication processes may be used to reduce or prevent increases in a false acceptance rate.

Face ID with Subepidermal Imaging & Machine Learning

Apple's invention states that "Subepidermal imaging of a face of a user attempting to unlock a device may be used to enhance a facial recognition authentication process (e.g., enhance the security of the facial recognition authentication process).

Subepidermal images of the user may be used to assess subepidermal features such as blood vessels (e.g., veins) when the device is attempting to authenticate the user. The subepidermal features may be compared to templates of subepidermal features for an authorized (e.g., enrolled) user of the device.

Assessment of subepidermal features during the facial recognition authentication process may be useful in distinguishing between users that have closely related facial features (e.g., siblings or twins).

In addition, assessment of subepidermal features may be used to prevent unlocking of the device by an unauthorized user wearing a mask or using another face replication method."

Apple's patent FIG. 8 below depicts a flowchart of an embodiment of additional authentication process #300. Additional the authentication process may be, for example, a vein matching authentication process used to distinguish between an unauthorized user and an authorized user that have closely related (e.g., very similar) facial features to prevent the unauthorized user from accessing device 100 without authorization.

Users with closely related (or very similar) facial features include users that have facial features that are close enough in distance (e.g., as defined by feature vectors in the feature space) that the similar facial features may be clustered together and/or there is at least some overlap between the similar facial features when assessing a matching score based on the facial features.

In certain embodiments, the process is operated by a different neural network modules and/or machine learning models than process. In some embodiments, different processes are operated by the same neural network module and/or machine learning model.

In certain embodiments, the process (#300) begins with capturing one or more subepidermal images of a user's face in (#302). A subepidermal image may be captured using an image sensor. In certain embodiments, the subepidermal image is captured when illuminating the face of the user with near-IR or another long wavelength illumination.

The subepidermal image may, however, be captured at another wavelength that provides more light penetration beneath the skin of the user (e.g., more penetration into the subepidermal layers). For example, the subepidermal image may be captured using illumination shifted about 10nm or more from the illumination used to capture the unlock attempt images. Shifting of the wavelength to lower wavelengths may, however, produce some light that is visible to a human eye (e.g., some light in or near the visible spectrum).

In some embodiments, the illumination for capturing subepidermal images is provided by one or more portions of illuminator 105A and/or illuminator 105B, shown in FIG. 2. For example, illuminator 105A and/or illuminator 105B may include an array of light sources such as, but not limited to, VCSELs (vertical-cavity surface-emitting lasers). A first set of light sources in the array may provide illumination at a wavelength for unlock attempt images while a second set of light sources may provide illumination at a wavelength for subepidermal images. The different sets of light sources may be turned on/off separately to allow a specific illumination (e.g., a specific wavelength of illumination) to be provided.

In certain embodiments, the captured subepidermal image includes an image of one or more blood vessels (e.g., veins) in the user's face. Apple's patent FIG. 9 above depicts a representation of an embodiment of an image of subepidermal features in a user's face.

As shown in FIG. 9 above, the face includes a plurality of blood vessels (#402 e.g., veins). Unlike some other facial features on the surface of the skin of a user's face, veins in the subepidermal layers of the face are typically unique to an individual and vein patterns are different between different individuals, even siblings or twins. Thus, assessment of the veins (and vein patterns) in the subepidermal layers of the face may be used to distinguish between siblings, twins, or other users with similar facial features on the surface of the face.

In some embodiments, absorption properties of blood at a selected wavelength may be used to detect vein or other blood vessels in the subepidermal layers. Absorption properties of blood vessels may be useful in differentiating vein features in the eyes of the user.

Face ID with Landmark Heat Maps

In a second patent application published today by USPTO Apple notes that in some cases, an image captured of a user during a facial recognition process (e.g., either an enrollment process or an authentication process) may include at least some occlusion of the user in the image. Occlusion of the user includes the blocking or obscuring of the user (e.g., the face of the user or some portion of the user's face) by some object (e.g., a finger, a hand, hair, masks, scarfs, etc.) in the image. Occlusion of the user in captured images may reduce the effectiveness of processing the image in the facial recognition process.

Landmark and occlusion heat maps may be generated and used to assess occlusion of landmarks on a user's face in a captured image. Landmark heat maps may be grid representations of the user's face that are used to estimate the location of landmarks on the user's face in the captured image. The occlusion heat map may be a grid representation of the user's face that includes scaled values representing the amount of occlusion in the regions of the grid. The estimated locations of the landmarks may be used in combination with the occlusion heat map to determine if and how much occlusion of the landmarks there may be in the captured image (e.g., an occlusion score for each of the landmarks). Determined values of occlusion for the landmarks may be used to control one or more operations of the device.

Apple's first patent application covered in this report was originally filed in February 2018 while the second patent application was originally filed in March 2018. Considering that these are  patent applications, the timing of such features for Face ID coming to market is unknown at this time.

About Making Comments on our Site: Patently Apple reserves the right to post, dismiss or edit any comments. Those using abusive language or negative behavior will result in being blacklisted on Disqus.