On May 30, 2013, the US Patent & Trademark Office published a patent application from Apple that reveals Apple's gaze detection technology. It should be noted that Apple's current patent is a divisional patent which means that Apple is breaking this aspect of their 2008 parent patent out as an individual patent. Technically, Apple had this technology in a patent prior to any other competitor including Samsung which actually implemented it first in their Galaxy S4 smartphone earlier this year. Apple also added gaze technologies to a January patent in context to an HDTV related patent and the likes of Sony and BlackBerry are working on similar features. Today, when you put your iPhone to your face the display is turned off. This is one of the technologies found in Apple's gaze detection patent. In the future, gaze technology will be able to stop a video when you look away or stop your tunes from playing based on this gaze detection feature. At the end of the day, Apple's gaze detection technology is really more about conserving power than it is about providing a flashy gimmick.
Wasting Computer Power
Electronic devices with small batteries have limited battery capacity. Unless care is taken to consume power wisely, an electronic device with a small battery may exhibit unacceptably short battery life. Techniques for reducing power consumption may be particularly important in wireless devices that support cellular telephone communications, because users of cellular telephone devices often demand long "talk" times.
Conventional portable electronic devices use various techniques for reducing their power consumption. Because display screens in electronic devices can consume relatively large amounts of power, power conservation techniques in portable electronic devices with display screens typically involve turning off the display screens at particular times.
Unfortunately, conventional power conservation techniques may turn off display screens at inappropriate times, thereby interfering with a user's ability to interact with a device. Conventional techniques may also leave display screens on at inappropriate times, wasting valuable battery power. It would therefore be desirable to be able to provide improved ways in which to conserve power in electronic devices.
Apple Considers New Power Management Utilizing Gaze Detection Technology
Apple's invention generally relates to computer devices that will have built-in gaze detection capabilities. The invention could apply to MacBooks, iDevices, Ultraportables and wearable computers pendants, a wrist and/or other miniature devices.
Apple states that one or more gaze detection sensors such as a camera may be used by the electronic device to determine whether a user's gaze is directed towards the electronic device (e.g., whether the user of the electronic device is looking at the electronic device). In particular, the electronic device may use gaze detection sensors to determine whether or not the user is looking at a display portion of the electronic device.
In an illustrative embodiment, the electronic device may have power management capabilities that are used to help conserve power. The electronic device may operate in two or more operating modes. One operation mode may be used to optimize performance. Another operating mode may help to extend battery life. The electronic device may use results from gaze detection operations to determine an appropriate mode in which to operate the electronic device.
Active and Standby Modes
In one example, the electronic device may operate in an active mode when the electronic device determines, using gaze detection sensors, that the user's gaze is directed towards the electronic device and may operate in one or more standby modes when the device determines that the user's gaze is not directed towards the electronic device.
When the electronic device is operating in one of the standby modes, circuitry and components such as a display screen, touch screen components, gaze detection components, and a central processing unit or CPU in the electronic device may be powered down or operated in a low-power mode to minimize power consumption in the electronic device.
With one suitable arrangement, when the electronic device is in the active mode and detects that the user has looked away from the device, the electronic device may dim or turn off a display screen. If desired, the electronic device can dim the display screen to a standby brightness level after the device has determined that the user has looked away from the device. After a given period of time has elapsed in which no user input has been received by the electronic device, the electronic device can turn off the display screen to conserve power. When the electronic device detects that the user's gaze is directed towards the electronic device, the electronic device may enter the active mode and return the display screen to an active brightness level (e.g., turn on the display screen or brighten the display screen to the active brightness level).
Applies to Playing Tunes
If desired, the electronic device may be performing an operation, while in the active mode, that is uninterrupted when the electronic device switches to operating in one of the standby modes. For example, the electronic device may be performing a music playback operation while in the active mode and, when the electronic device detects the user's gaze is not directed towards the electronic device, the electronic device may enter one of the standby modes without interrupting the music playback operation.
Applies to Playing Videos
With one suitable arrangement, the electronic device may interrupt an operation when the electronic device begins operating in one of the standby mode. For example, the electronic device may be performing a video playback operation while in the active mode. In this example, when the electronic device detects that the user's gaze is no longer directed towards the electronic device, the electronic device may enter one of the standby modes, dim the display screen that was being used for the video playback operation, and pause the video playback operation. If desired, the electronic device may be set to resume the video playback operation when it detects that the user has redirected their gaze towards the electronic device.
In an illustrative embodiment, the electronic device may use readings from sensors such as proximity sensors, ambient light sensors, and motion sensors such as accelerometers to determine whether or not to perform gaze detection operations. For example, the electronic device may suspend gaze detection operations whenever a proximity sensor, ambient light sensor, or accelerometer indicates that gaze detection operations are inappropriate (e.g., because of an object in close proximity with the electronic device, insufficient ambient light for gaze detection sensors to detect the user's gaze, excessive vibration which may degrade the performance of gaze detection sensors, etc.).
Suspending Gaze Detection when Inappropriate
As illustrated by FIG. 9, a device may also use information from environmental sensors such as proximity sensors and ambient light sensors to determine whether or not to perform gaze detection operations. Environmental sensors such as these may, if desired, be used in conjunction with an environmental sensor such as an accelerometer that detects device motion.
When a device is performing gaze detection operations the device may suspend gaze detection operations whenever a sensor in a device indicates that gaze detection operations are inappropriate or not needed. With one suitable arrangement, a device may be able to detect when gaze detection sensors such as camera would be incapable of detecting a user's gaze due to excessive vibration detected by an accelerometer. For example, a device may suspend gaze detection operations in response to signals from the accelerometer in the device that indicate the device is shaking or otherwise moving rapidly.
In that example, a device may switch to another mode when the accelerometer detects that the acceleration of the device exceeds a given threshold level. In another example, the device may be able to detect, using a proximity sensor, that gaze detection operations are inappropriate because an object is in close proximity to a device and is blocking the device's gaze detection sensors (e.g., such as when a user places a device against their ear and thereby blocks the camera. If desired, a device may suspend gaze detection operations when an ambient light sensor detects that there is insufficient light in the environment around a device for a camera to capture images in which a user's gaze could be detected.
The device may also deactivate a camera associated with gaze detection operations and suspend a gaze detection application running on circuitry when data from one or more sensors in a device indicate that gaze detection operations are inappropriate or wasteful of power.
Apple's patent FIG. 10 shows steps involved in processing a command to reduce the power consumption of the display. Power reduction commands may be processed by a device based on gaze detection data or any other suitable data.
Apple credits Andrew Hodge and Michael Rosenblatt as the inventors of patent application 20130135198 which was originally filed in Q1 2013. It should be noted that our patent report is based on Apple's "Detailed Description" segment of their patent application. To review Apple's patent claims, see Apple's patent filing. Considering that this is a patent application, the timing to market of such an Apple product is unknown at this time.
A Note for Tech Sites Covering our Report: We ask tech sites covering our report to kindly limit the use of our graphics to one image. Thanking you in advance for your cooperation.
Patently Apple presents a detailed summary of patent applications with associated graphics for journalistic news purposes as each such patent application is revealed by the U.S. Patent & Trade Office. Readers are cautioned that the full text of any patent application should be read in its entirety for full and accurate details. Revelations found in patent applications shouldn't be interpreted as rumor or fast-tracked according to rumor timetables. About Comments: Patently Apple reserves the right to post, dismiss or edit comments.
New on Patent Bolt this Week