Apple Wins Patent for a next-gen depth-based Touch Detection System for future iPhones & Head Mounted Device
The U.S. Patent and Trademark Office officially published a series of 67 newly granted patents for Apple Inc. today. In this particular report we cover Apple's patent titled Depth-based touch detection that could be used with future iPhones and Head Mounted Device. Computer system functionality can be further improved by eliminating the need for conventional input devices like a keyboard.
In Apple's patent background they note that detecting when and where a user's finger touches a real environmental surface can enable intuitive interactions between the user, the environment, and a hardware system (e.g., a computer or gaming system).
Using cameras for touch detection has many advantages over methods that rely on sensors embedded in a surface (e.g., capacitive sensor). Further, some modern digital devices like head-mounted devices (HMD) and smart phones are equipped with vision sensors--including depth cameras.
Current depth-based touch detection approaches use depth cameras to provide distance measurements between the camera and the finger and between the camera and the environmental surface. One approach requires a fixed depth camera setup and cannot be applied to dynamic scenes. Another approach first identifies the finger, segments the finger, and then flood fills neighboring pixels from the center of the fingertip so that when sufficient pixels are so filled, a touch is detected. However, because this approach does not account even consider normalizing pixel depth-data, it can be quite error prone.
In still another approach, finger touches are determined based on a pre-computed reference frame, an analysis of the hand's contour and the fitting of depth curves. Each of these approaches requires predefined thresholds to distinguish touch and no-touch conditions. They also suffer from large hover distances (i.e., a touch may be indicated when the finger hovers 10 millimeters or less above the surface; thereby introducing a large number of false-positive touch detections).
Apple's granted patent covers touch detection operations that can assist with improving the functionality of computing devices or systems that accept non-keyboard input.
Computer functionality can be improved by enabling such computing devices or systems to use arbitrary surfaces (e.g., a tabletop or other surface) from which to get input instead of conventional keyboards and/or pointer devices (e.g., a mouse or stylus).
Computer system functionality can be further improved by eliminating the need for conventional input devices; giving a user the freedom to use the computer system in arbitrary environments.
Apple's patent FIG. 2A illustrates various aspects of a depth-based touch detection operation.
Apple's granted patent 10,572,072 was originally filed in Q3 2017 and published today by the US Patent and Trademark Office. The two inventors, Lejing Wang and Daniel Kurz came to Apple via Metaio when the company was acquired in 2015.
Our cover graphic is patent FIG. 2B in-part that covers a fingertip patch technique.