Apple's Camera Patent #1: Systems, Methods and Devices for Flash Exposure Control using Preflash Statistics

The Problem of Improperly Exposed Images with Current Strobe Flash

Electronic devices commonly include cameras or other image capture circuitry. Such image capture circuitry may capture photos based on light illuminating a scene. In certain low-light conditions, ambient light alone may not sufficiently illuminate a scene and resulting images of such scenes may be dark or underexposed.

To improve image capture under such low-light conditions, many electronic devices also may include a strobe flash illumination device, such as a xenon or light emitting diode (LED) flash. These strobe flashes may supplement the ambient light illuminating the scene. However, the actual effect of the supplemental illumination on the scene may not be observed by image capture circuitry of an electronic device until it occurs. Thus, if the amount of illumination provided by the flash is too high or too low relative to the image capture control statistics used to capture the image, the image may not be properly exposed.

Apple's Proposed Solution

Apple's invention relates to systems, methods, and devices for obtaining a properly exposed strobe-illuminated image. One method for doing so may include gathering image capture statistics during a first period when a strobe is not emitting light and during a second period when the strobe emits a preflash.

These image capture statistics may include distinct image capture control statistics and luma values associated with the periods. Final image capture control statistics then may be determined based at least in part on the first luma value normalized to the first image capture control statistics and the second luma value normalized to the second image capture control statistics. Thereafter, the final image capture control statistics may be used to capture a properly exposed strobe-illuminated image when the strobe emits a main flash.

Apple's patent FIG. 8A shown below illustrates a flowchart describing a method for determining image capture control statistics for the proper exposure of a strobe-illuminated image in greater detail; patent FIG. 3 illustrates the backside of an iPhone where we see the backside camera and the strobe (patent point 22).

In Apple's patent FIG. 4 below we see an illustrative schematic diagram representing various image capture statistics associated with an image capture sequence.

Apple's patent application was originally filed in Q2 2010 by inventors Haitao Guo and David Kuo.

Apple's Camera Patent # 2: Compensation for Black Level Changes

Apple's camera patent number two covers their invention which relates to a technique for applying black level compensation to image data. An image processing system includes a first image processing pipeline configured to receive frames of image data generated by an image sensor and to alter the frames of image data to compensate for black level shift. The image processing system may also include a feed-forward loop having a second image processing pipeline configured to receive at least one of the frames of image data, to process the at least one frame, and to adjust a black level compensation parameter of the first image processing pipeline.

Apple's patent FIG. 9 depicts a block diagram of an image processing system for providing feed-forward black level compensation; FIG. 17 is a block diagram illustrating an image signal processing (ISP) system that may be configured to apply lens shading correction.

Apple's patent application 20110298945 was originally filed in Q2 2010 by inventors Ting Chen, Jay Zipnick, Andrew Yanowitz and Haitao Guo. Apple's patent application is an in-depth technical patent pertaining to cameras found in iOS and OS X hardware. Super camera buffs, technicians and developers could check this patent out here (Note: this is 24-48 hour temporary link only).

Apple's Camera Patent # 3: Adaptive Lens Shading Correction

Apple's third camera invention relates to systems, methods, and devices for dual processing of raw image data by main image processing and alternative image processing capabilities of an electronic device.

Apple's Patent FIG. 16 is a three-dimensional profile depicting lens shading gain values that may be applied to an image that exhibits light intensity characteristics when performing lens shading correction; Patent FIG. 17 is a block diagram illustrating an image signal processing (ISP) system that may be configured to apply lens shading correction.

Apple's patent application 20110298944 was originally filed in Q2 2010 by inventors David Kuo, Haitao Guo, Jay Zipnick, Ting Chen and Sumit Chawla.

A fourth Apple patent application (20110298933) covers "Dual Processing of Raw Image Data." And finally, a fifth camera related patent application that surfaced last week relates to video cameras. Apple's invention titled "Scene-Aware Auto-Exposure Control generally relates to stabilizing auto-exposure operations in an iOS digital camera (see patent FIG. 6 below). The technology discussed in this particular patent application closely relates to the technology found in another patent application that we covered in a report this past weekend titled "Apple Patent Discusses FaceTime Advancements in the Works."

Apple's patent FIG. 6 illustrates a scene-aware auto-exposure control process (100) comprises three primary operational phases: unstable (FIG. 1A), stable (FIG. 1B) and metastable (FIG. 1C). During unstable operations no consistent lighting environment has been detected and, accordingly, auto-exposure operations take place as normal. During stable operations a consistent lighting environment has been detected and, therefore, auto-exposure operations may be suppressed. During metastable operations, a lack of information makes continued stable operations suspect. Accordingly, either a stable lighting environment is detected in a relatively short time (in which case stable operations resume) or auto-exposure control is returned to normal (unstable operations resume).

At times it's hard to believe the amount of technology that will be going into future iOS cameras and I'm sure that the traditional standalone camera makers wince at the mere thought of it. That's the power of these next generation iOS devices. They're challenging the cell phone industry, the camera industry, the MP3 industry, and in the not-too-distant future even challenge GPS mapping companies. Apple's homepage is currently promoting the iPhone 4S and calls it the most amazing phone yet. Well, in 2012, it's only going to get better.

The Use of Our Graphics: We'd ask those covering our report to limit the use of our graphics to a single graphic. Thank you.

Sidenote: An Imagination Technologies Slide

Apple has more than a twenty percent stake in a company called Imagination Technologies that are behind Apple's graphic processor chip. In a recent presentation put on by Imagination Technologies, we're now able to view one of their slides that clearly and openly reveal that their next generation processors will be bringing more imaging related capabilities to the table; capabilities that will likely be coming to the next generation of iOS devices in 2012 – and just maybe, allowing some of the inventions that were presented in today's report to come to life fairly quickly. Time will tell.

Notice: Patently Apple presents a detailed summary of patent applications with associated graphics for journalistic news purposes as each such patent application is revealed by the U.S. Patent & Trade Office. Readers are cautioned that the full text of any patent application should be read in its entirety for full and accurate details. Revelations found in patent applications shouldn't be interpreted as rumor or fast-tracked according to rumor timetables. Apple's patent applications have provided the Mac community with a clear heads-up on some of Apple's greatest product trends including the iPod, iPhone, iPad, iOS cameras, LED displays, iCloud services for iTunes and more. About Comments: Patently Apple reserves the right to post, dismiss or edit comments.