In 2013 Apple updated iMovie with a new Rolling Shutter feature. Apple noted on their support page that "Many camcorders record video in a way that can cause image distortion if the camera moves a lot during recording (especially quick panning movements), or if the camera is recording fast motion. A video clip may appear wobbly or skewed in these instances. iMovie has a Fix Rolling Shutter feature to reduce this kind of motion distortion." Today, Apple's patent application that's behind this feature has been published by the U.S. Patent & Trademark Office. In a second patent application published today, we learn a little more about the technology behind Apple's autofocus feature using an artificial muscle which is referred to in the industry as an Electro-Active Polymer Device.

Apple's Patent Background

Modern digital cameras, such as complementary metal-oxide-semiconductor (CMOS) cameras, frequently employ a method of image acquisition in which each frame is recorded by scanning across the frame row by row from top to bottom (or column by column across the frame) rather than by taking a single snapshot of the entire scene at a single point in time. Thus, not all parts of the image may be recorded at the same time. This method of image acquisition is sometimes called rolling shutter, because the shutter is moving (e.g., rolling) across the image area of the scene.

Due to the rolling shutter, visual distortions may be introduced and may be visually apparent when the entire image is displayed. In general, any movement of the camera as the image is captured may result in a warping or distortion of the image. Images captured using rolling shutter acquisition may exhibit different types of visual effects or distortions. One type of distortion may be caused by the camera not being held completely still throughout the entire process of scanning a frame. If the camera is moving horizontally, the image gets sheared. If the camera moves vertically, image is stretched or compressed.

For example, in images captured while a camera is vibrating, such when used from a moving vehicle, a wobble distortion may be created. Video exhibiting this distortion may appear to wobble unnaturally in a jelly-like fashion. Similarly, if a camera moves from side to side, a skew distortion may result in which the image may bend diagonally in one direction or another. If a CMOS camera sensor is used to take an image of a moving object, different areas of the object may appear smeared or to change proportion as compared to other areas of the object.

Apple's Invention Covers In-Stream Rolling Shutter Compensation

In-stream rolling shutter compensation may be utilized to modify image data as it is received from a camera sensor to compensate for detected camera motion. When performing in-stream rolling shutter compensation, an image processor may perform motion matching on image data coming through a camera sensor to determine whether the camera is moving and how the camera is moving (e.g., the direction of motion). As the image data is received, the current image may be analyzed in strips to find matching locations between the current image (e.g., frame) and a previous image. Matches between the two frames may be found by comparing graphical profiles containing summed row and column pixel data for each image strip. For example, luminance data for each pixel in a column (of the current strip) may be combined together (e.g., summed, averaged, etc.) to compute a column graphical profile while luminance data for pixels in a row may be combined to compute a row graphical profile.

The horizontal and vertical graphical profiles may be compared to corresponding data from the previous image to determine matching locations between the two frames. A motion vector for the strip may be determined based on the spatial distance between pairs of matching locations (e.g., between the current and previous frames) and may represent camera motion between the two frames. As the image data is written out (e.g., to memory) the current image data may be adjusted (e.g., warped) to compensate for the perceived camera motion based on the computed motion vector.

Considering that Apple introduced this feature in 2013, the year that the patent was filed, the likelihood of this patent application supporting that feature is very high. For camera buffs who want to dig deeper into Apple's invention, click here.

Apple's Second Camera Related Invention: Driver Circuit for Electro-Active Polymer Devices

Apple's second camera related invention relates to electronic circuits for driving the electrodes of an electro-active polymer (EAP) device – which relates to Apple's iSight Camera's autofocus.

One of the key aspects of the invention relates to a camera application. In Apple's patent FIG. 4 noted below we're able to see a combined cross-section view and circuit schematic of a multiple electro-active polymer (EAP) device actuator for a camera function. The camera functionality may be integrated into a battery powered portable consumer electronics device such as smartphone, a tablet computer, or a laptop computer, in which physical space comes at a premium.

The figure depicts camera optics including a lens barrel #8 to which an artificial muscle structure 7 is attached. The artificial muscle #7 has a first EAP device made of an EAP1 electrode and a portion of a common electrode that sandwich a portion of a dielectric elastomer layer.

In addition, the muscle structure 7 has a second EAP device made of an EAP2 electrode and another portion of the common electrode that sandwich a further dielectric elastomer layer. The EAP1 device may be coupled to control a first aspect of the optics such as focusing imaging lens position (position of lens barrel #8 along the camera optical axis), while the EAP2 device may be coupled to control another aspect such as aperture size that controls how much light from the scene enters the imaging lens. Other camera lens and optics arrangements and pre-tension schemes for use with such a multi-EAP device actuator are of course possible.

Apple's first EAP related patent surfaced back in June 2014. We covered in our report titled "Apple Patents Reveal a New Camera Electro-Active Polymer Actuator." We presented a follow-up report days later titled "'Future Smartphone Cameras may use a Micro Electrochromic Iris made from Smart Glass Eliminating the use of Actuators," that discussed the future of the smart camera Iris that would eliminate actuators altogether.

For more details on Apple's second camera related patent, click here.

Patently Apple presents a detailed summary of patent applications with associated graphics for journalistic news purposes as each such patent application is revealed by the U.S. Patent & Trade Office. Readers are cautioned that the full text of any patent application should be read in its entirety for full and accurate details. About Making Comments on our Site: Patently Apple reserves the right to post, dismiss or edit any comments. Comments are reviewed daily from 5am to 7pm MST and sporadically over the weekend.