Problems with Today's Slide-to-Unlock System

Everyone mobile user by now is familiar with Apple's famous Slide-to-Unlock feature for their iDevices. But if you're not it works like this. The lock mode is used to prevent inadvertent operation of a touch screen display, e.g., while the device is in a user's pocket or purse or when another object is placed against the device. The lock mode may also be used to prevent an unauthorized person from using the device. Generally, the device is programmed to enter the lock mode when a user presses a specific button or a series of buttons or when it has been idle for a certain period of time. When a user desires to use a device that is locked, the user will typically be required to drag a slide bar, press a specific button or a series of buttons (e.g., to enter a password) to unlock the device. However, a user may find these steps inconvenient and time consuming.

For example, a user may be reading a document using the device when the device detects that it has been idle for a certain period of time. In this case, the device will automatically enter the lock mode where it turns off or dims its display screen, and the user will be required to unlock the device before being able to resume reading the document.

In another example, a user may be prone to forgetting the password needed to unlock the device. As a result, the user may decide to configure the device so that it does not automatically lock. If she then forgets or chooses not to manually lock her device, then that leaves the device susceptible to inadvertent operation or unauthorized use.

Locking and Unlocking Your iDevices with Face Recognition

Apple's invention relates to a mobile device that's configured to automatically lock based on determining that a user's face is no longer present in images captured by the device's built-in camera. For instance, consider that the device is initially unlocked. In that state, a built-in camera captures one or more images, and the images are then analyzed to determine whether a user's face is present therein. If a user's face is not present in the images captured over a predetermined amount of time, the device automatically locks. Thus, the device is automatically locked when it determines that no user is currently using the device without having to wait for an idle timer to expire or a manual switch off by the user. The camera capturing and face recognition processing may be triggered by the device having detecting that it has been motionless for a threshold period of time.

In another embodiment, a mobile device is configured to automatically unlock. Consider that the device is initially locked. In that state, the camera captures an initial image. When movement of the device is detected, the camera captures a new image. The device then determines whether it has moved to a use position (i.e., a position that indicates that a user is likely to want to use the device) by comparing the new image with the initial image. If the device has moved to a use position, the camera captures a subsequent image, and the subsequent image is analyzed to detect a user's face. If a user's face is detected in the subsequent image, the device is automatically unlocked. This unlocks the locked device without requiring the user to press a sequence of buttons (e.g., to enter a password) each time the user wants to use the device.

Apple's patent FIG. 1 shows a mobile device 1 being held by a user in a manner such that the device is unlocked or remains unlocked by detecting a user's face. In particular, the device may be any camera-equipped mobile device, such as the iPhone, iPod touch and iPad that have a built-in camera. Camera 3 will be used to capture an image of the user's face that may later be used to authenticate the user as an authorized user of the device.

Advanced Sensors

Apple's patent FIG. 2 is an example block diagram of the device with a face recognition system as noted in FIG.1. The device may include many kinds of sensors such as a motion sensor 11 (also referred to as an inertial sensor), that may be used to detect movement of the device. Additionally, the motion sensor may include a position, orientation, or movement (POM) sensor, such as an accelerometer, a gyroscope, a light sensor, an infrared (IR) sensor, a proximity sensor, a capacitive proximity sensor, an acoustic sensor, a sonic or sonar sensor, a radar sensor, an image sensor, a video sensor, a global positioning (GPS) detector, an RP detector, an RF or acoustic doppler detector, a compass, a magnetometer, or other like sensor.

For example, the motion sensor may be a light sensor that detects movement or absence of movement of the device by detecting the intensity of ambient light or a sudden change in the intensity of ambient light. The motion sensor generates a signal based on at least one of a position, orientation, and movement of the device. The signal may include the character of the motion, such as acceleration, velocity, direction, directional change, duration, amplitude, frequency, or any other characterization of movement. The processor 18 receives the sensor signal and controls one or more operations, e.g., the unlocking function described below, of the device based in part on the sensor signal.

Utilizes Movement Patterns

In one embodiment, the device may include movement patterns (16) that are stored in storage (13). A movement pattern may be associated with a function, e.g., the unlocking function, of the device. For example, the processor may compare the sensor signal with the stored movement patterns to determine whether the user moved the device to a use position, i.e., a position that indicates that the user is likely to want to use the device. The comparing function may include employing a pattern recognition algorithm or technique or a statistical model.

In another embodiment, the processor may use the sensor signal to infer or calculate tilt or inclination of the device relative to a static, linear acceleration (i.e., gravity) by correlating tilt angles with detected linear acceleration. In this way, the processor may calculate or infer when the device is subjected to dynamic accelerations by, for example, the hand of the user.

A Sensor that Understand Unintended Movement

In yet another embodiment, the motion sensor may include other signal processing circuits such as a low pass filter, a threshold detector, an amplitude detector, or a frequency detector to remove signals representing unintended movement of the device. For example, a low pass filter may be used to remove or block noise or spurious signals representing brief, unintended movement of the device or other movement of the device that may occur during, for example, walking, jogging, or bouncing in a moving vehicle.

The Grip Detector

In another embodiment, the motion sensor may be used in combination with a grip detector or a proximity sensor (not shown) in the unlocking function of the device. The grip detector may be used to recognize when the user is holding the device, touching the display screen, or pressing a button. The grip detector may include one or more switches and/or buttons to detect the presence of at least one finger. The proximity sensor may sense when the device is in the user's hand so that the device can ignore changes in position, orientation, or movement that are not based on the user's hand movements and/or positioning. By requiring the user to explicitly interact with a portion of the device while moving the device, the possibility of inadvertently initiating the unlocking function is minimized or eliminated.

Camera Circuitry & Authorization

The device also includes camera circuitry (12) that implements the digital camera functionality of the device. One or more solid state image sensors are built into the device, and each may be located at a focal plane of an optical system that includes a respective lens. An optical image of a scene within the camera's field of view is formed on the image sensor, and the sensor responds by capturing the scene in the form of a digital image or picture consisting of pixels that may then be stored in storage.

The camera circuitry may be used to capture the images that are analyzed by the processor in the locking or unlocking functionality of the device. For example, the camera circuitry may capture an image of a facial profile of an authorized user of the device and this image may be stored with the images of authorized users (17) in storage. When the camera circuitry subsequently captures an image of a user who wants to unlock the device, the processor may compare the facial profile in the subsequently captured image with the facial profiles in each image of the images of authorized users to determine whether the user is an authorized user. This may be accomplished using suitable facial recognition software that matches faces that look like the same person.

Operations of a Lock Function

The operations of a lock function of a device is shown in FIG. 3, and the operations of an unlock function is shown in FIG. 4.

Unique Child Lock Feature

The lock and unlock functions may be used to lock part or all of a device from use and to unlock part or all of a device for use. For example, the device may be turned on or off, the key pad (e.g., the keys used for dialing) may be enabled or disabled, the touch screen user input interface of the device may be enabled or disabled, and/or dialing or sending a telephone number to initiate a call may be enabled or disabled while still permitting the device to receive and to answer an incoming call. It will be appreciated that other functions may be turned on or off. Furthermore, specific functions may be enabled or disabled depending on the user of the device. For example, if the unlocking function detects that the user wanting to unlock the device is a child, the device may enable the applications or functions that are appropriate for a child and disable other functions such as dialing or sending a telephone number to initiate a call. The applications and functions of the device that are enabled and disabled by the unlocking and locking functions may be configurable by a user of the device.

Learns the user's Intended Movements

The device may be configured to detect intended movement of the device. Intended movement means that the movement of the device was intended as compared to motion that occurs as a result of being dropped, placed on a table or other surface, carried by the user while walking or jogging, or bouncing in a moving car or some other type of motion that may be considered random motion or noise. The device may detect intended movement by, for example, performing signal processing on the signal from the motion sensor as described above, or, alternatively or additionally, comparing the movement pattern to stored movement patterns that represent intended movement of the device. The device may also determine that the movement is intended by sensing that the device is in the user's hand using a proximity sensor or grip detector. Other ways of determining whether movement of the device was intended by the user are possible.

When the device successfully matches the facial profile in the subsequent image to a facial profile in a stored image of an authorized user, the device automatically unlocks (block 47). If the authorized user authentication operation does not successfully find a match, the device remains locked (block 44). Authorized user authentication is akin to a secret password or other personal security information and may prevent unauthorized use of the device when, for example, the device is misplaced or stolen.

Patent Credits and Notes

Apple's patent application was originally filed in Q1 2011 by inventors Lihua Zhao and Richard Tsai and published today by the US Patent and Trademark Office.

For those wishing to compare features of Apple's patent with Google's patent, see Google's patent application 8,261,090. Google's patent application has yet to be granted by USPTO.

Notice

Patently Apple presents a detailed summary of patent applications with associated graphics for journalistic news purposes as each such patent application is revealed by the U.S. Patent & Trade Office. Readers are cautioned that the full text of any patent application should be read in its entirety for full and accurate details. Revelations found in patent applications shouldn't be interpreted as rumor or fast-tracked according to rumor timetables. About Comments: Patently Apple reserves the right to post, dismiss or edit comments.