On August 19, 2010, the US Patent & Trademark Office published a patent application from Apple that appears to be describing a next generation autofocus feature for future iOS devices that will add speed enhanced autofocus operations. Technically speaking, any of the camera features described in Apple's patent could equally apply to iSight.
Patent Background
In order to take sharper pictures of a scene, a digital camera may be equipped with a moveable lens system that is controlled by an automatic lens focusing system or autofocus system. When the user has pressed the camera's shutter button part way, the autofocus system responds by quickly calculating the correct lens position that results in a subject in the scene being in focus, before the user has pressed the shutter button all the way (at which point a final picture of the scene is taken or accepted and then stored in the camera). Obtaining the correct focus quickly is important in digital cameras running on battery power because time and power usage should be minimized to the extent possible. The autofocus system should also work in a continuous image capture mode where the camera takes a rapid sequence of still pictures (e.g., video). For instance, in a video mode the focus is automatically adjusted in real-time as the scene changes.
The hill-dimb algorithm is often used in autofocus systems. Unfortunately, such autofocus systems could be relatively slow in determining a focus setting due to the large number of autofocus images that must be captured and compared. For example, an autofocus system using the hill-dimb algorithm could take up to nine (9) frames to calculate the lens focus position taking as long as 0.5 seconds to 2.0 seconds to determine focus conditions. Autofocus algorithms such as the hill-climb algorithm in general also suffer in a low-light environment since the autofocus has difficulty capturing enough edge information to properly focus.
Patent Summary
Apple's patent describes an autofocus digital camera which includes a moveable lens system, an image sensor coupled with the moveable lens system, a processor coupled with the image sensor, and data storage coupled with the processor. The data storage stores a table with multiple entries, each entry including a face size and a corresponding lens focus position. The processor performs an autofocus operation when taking a picture, in which the processor detects a face in an image captured by the image sensor. The size of the detected face is then calculated. A stored lens focus position is selected from the table in storage, based on the calculated face size. The moveable lens system is then signaled to move to a final lens position, which is the selected lens focus position. This enables the moveable lens system to jump directly to a lens position that is expected to result in a sharp picture of the scene, without having to go through a full sweep of lens positions.
Apple's patent FIG. 2 illustrates an example method for building an autofocus look up table; FIG. 3 is a diagram of a detected face image in a picture, relative to the field of view when the picture was taken.
Autofocus Speed Enhancement
Apple's patent FIG. 4 illustrates the process 300. The process 300 is advantageous because it may add a noticeable speed enhancement to the entire picture taking operation.
The process begins with operation 302 where the camera captures an initial image of the scene. In still capture mode, the operation may be preceded by operation 301 in which the camera detects that its user has just started to press the shutter button. The captured initial image is then analyzed in operation 304, to detect faces therein (note there may be multiple faces that are detected in the image, but just one of them may be sufficient for the subsequent face size calculation). Any suitable face detection algorithm may be used. The size of a single, selected face is calculated (operation 306). This may be done by, for example, counting the number of pixels that form the face, or estimating the face size more roughly by measuring the x and y pixel dimensions of the smallest rectangle or square in which the face can fit, and then relating those quantities to actual distance dimensions or the FOV (Field Of View) for that image. A ratio of the area taken up by the single face to the total area of the FOV may be calculated. The ratio may alternatively be given as a percentage of the particular FOV (used for taking the initial image) that is covered by the single face.
Once the face size has been calculated, a face size entry is selected, from the stored autofocus table containing a number of such entries – that is closest to the calculated face size (operation 308). The corresponding or associated lens focus position is retrieved from the selected table entry (operation 310). The camera's optical or lens system is then signaled to move directly to the retrieved lens focus position (operation 312). Once the optical system has responded to this signal by automatically moving its lens accordingly, the camera is now ready to take a picture of the scene.
Auto focus used in iSight and/or iOS Portable Devices
Apple's patent FIG. 5 is a block diagram of a digital camera in which an embodiment of the invention is implemented. In the illustrated embodiment, the camera 400 includes a lens system 441 having a lens actuator that could be signaled automatically to move a lens so as to focus incident light from a scene, an image sensor 439 coupled with the lens system to receive the focused incident light and in response produce a digital image thereof, a processor 437 coupled to control the image sensor, data storage 435 coupled with the processor to store a digital image captured by the sensor, and an external input/output interface 443 through which the processor 437 can communicate with devices external to the camera.
The processor may comprise any suitable embedded microprocessor or state machine that could manage overall operations of the camera. For example, the camera may be a "thin" device with minimal functionality (such as a personal computer peripheral "Web" camera), where the processor is a hardwired state machine that responds to commands received from the external I/O interface, the latter being a personal computer serial communications interface such as a Universal Serial Bus port. Alternatively, the camera may be embedded in a smart phone handset (or an iOS device with a camera) in which case the processor could be an embedded applications processor in the handset.
Apple credits E-Cheng Chang as the sole inventor of patent application 20100208091, originally filed in Q1 2009. It's unknown at this time if Apple has added the new speed enhancements to their new iPhone 4 but it appears that the technology described in today's patent is far more advanced than the current "tap-to-focus" feature.
Notice: Patently Apple presents only a brief summary of patents with associated graphic(s) for journalistic news purposes as each such patent application is revealed by the U.S. Patent & Trade Office. Readers are cautioned that the full text of any patent application should be read in its entirety for further details. About Comments: Patently Apple reserves the right to post, dismiss or edit comments.
