Apple Wins a Patent for a 3D Camera System for Controlling Games, Apps and Possibly Facial Recognition
The U.S. Patent and Trademark Office officially published a series of 62 newly granted patents for Apple Inc. today. In this particular report we cover an Apple patent acquired from PrimeSense regarding technology relating to a 3D depth camera that could be used to control apps and gameplay with in-air control/gestures and possible cover instant on facial recognition in Apple devices like an iMac or in theory beyond.
Granted Patent: Depth Mapping based on Pattern Matching and Stereoscopic Information
Apple's newly granted patent 9,582,889 covers their invention relating to computer vision, and specifically to three-dimensional (3D) mapping and imaging. The granted patent notes that patent FIG. 1 as noted below, is a schematic pictorial illustration of a system #20 for 3D mapping and imaging wherein an imaging assembly #24 is configured to capture and process 3D maps and images of a user interacting with a video game and/or other applications on their computer.
While Apple spells out what the camera could be used for, it's been covered before. What's new in this granted patent is that they focus on the camera as noted below in our patent figure collage.
Where we may be able to extract another clue as to what this invention could be used for I noticed an entry in the references section (usually used by a patent examiner or a patent applicant like Apple) referencing work related to "Video-rate capture of Dynamic Face Shape and Appearance", Sixth IEEE International Conference on Automatic Face and Gesture Recognition (FGR 2004), Seoul, Korea, May 17-19, 2004, cited by applicant." The gesture recognition angle has been referenced many times in Apple/PrimeSense patents that we've covered, but automatic face recognition hasn't. So that's a possibly interesting twist.
Facial recognition was back in the news recently when Apple acquired Israeli firm RealFace. Apple has already killed the video that promoted the RealFace application. While KGI Securities analyst Ming Chi Kuo added facial recognition to one of his recent iPhone 8 rumors, his source was Apple patents that of course we have covered extensively (one, two, three + Archives) without creating rumors to promote them.
At the end of the day, a 3D camera as described in this granted patent could be used for facial recognition that could be extended to the iMac which would be able to counter Wintel's 'Windows Hello' instant facial recognition login feature that debuted with Windows 10.
Apple's patent FIG. 1 noted below is a schematic, pictorial illustration of a system for 3D mapping; FIG. 2 is a schematic top view of an imaging assembly/camera system; and FIG. 3 is a flow chart that schematically illustrates a method for 3D mapping.
Technically speaking, stereoscopic depth mapping or 3D mapping cameras generally use two or more cameras, spaced apart by a known distance, capture images of the same scene. The same image features appear in both images, but at locations in the respective images that are relatively offset by parallax due to the spacing of the cameras and the distance of the features from the camera plane. The measured offset of a given feature, together with the known spacing between the cameras, is thus used to compute the depth coordinate of the feature.
Usually, the multiple cameras that are used in a stereoscopic system are of the same type and characteristics. In the embodiment shown in FIG. 2 noted above, the two different types of cameras used include an IR camera embodied in the subassembly #36 and the RGB color camera embodied in subassembly #42.
The cameras, of course, have different spectral responses and may also have different-sized pixels. Both of these cameras, however, are present in system 20 anyway, in order to enable the computer to reconstruct a 3D color image of the object. Therefore, the use of these two heterogeneous cameras for stereoscopic depth mapping comes at little or no additional hardware cost and requires only that additional processing be applied to the images that are output by the cameras.
Apple's Israeli PrimeSense team of engineers have used the same patent FIG. 1 to cover a wide range of 3D mapping inventions covering head and eye tracking, in-air gesturing (also here) and gaze control. Apple's granted patent was originally filed in Q3 2010 and published today by the US Patent and Trademark Office.
Patently Apple presents only a brief summary of granted patents with associated graphics for journalistic news purposes as each Granted Patent is revealed by the U.S. Patent & Trademark Office. Readers are cautioned that the full text of any Granted Patent should be read in its entirety for full details. About Making Comments on our Site: Patently Apple reserves the right to post, dismiss or edit any comments.