Apple Granted Patent for a 3D camera using Anaglyphic Display during Capture

Apple has been granted a patent today for an invention relating to the field of digital cameras, and more particularly to a digital camera which captures stereoscopic or 3D images. While Apple is noted as being the assignee on this granted patent, it appears that Apple has acquired this patent form inventor Ken Parulski who is widely recognized as a digital photography pioneer and inventor of more than 200 issued patents. Parulski's patents are broadly licensed with earnings for his company aKAP Innovation in the 2 billion per annum range. Kodak and other camera companies have also acquired some of Parulski's patents.

Patent Background and Problem to Solve

Parulski notes that "Digital cameras are primarily designed for use in creating two-dimensional images. In a two-dimensional image, only one perspective is needed. Human vision, however, views scenes from two perspectives, with one perspective provided by each eye of an observer. The parallax that exists between the perspectives is interpreted by the mind of the observer to provide depth information about the scene being viewed.

Various electronic and photochemical imaging techniques have been developed that capture images of a scene taken from different perspectives in order to provide three-dimensional (3-D) images. These images are processed and displayed to a user so that one eye of an observer views an image of the scene from one perspective while the other eye of the observer views another image of the scene taken from another perspective. This creates the parallax difference necessary to produce the appearance of depth in the mind of the observer. This is known as "stereoscopic photography" or "3-D photography."

3-D images can be captured using a digital camera having multiple image sensors, such as the FinePix REAL 3D W1 digital camera available from Fujifilm Corporation of Tokyo, Japan. Such cameras are relatively expensive, however, since they typically require at least two camera lenses, image sensors, and image processors.

To reduce cost, it is known to use a camera that provides a single optical path that is moved along a fixed track, as described in U.S. Pat. No. 5,883,695 entitled "Method and Apparatus for Producing Stereoscopic Images with Single Sensor," and also U.S. Pat. No. 5,325,193 to Pritchard et al., entitled "Single Camera Autostereoscopic Imaging System." In such systems, different perspectives are captured as the camera is moved to fixed locations along the path.

In all 3-D imaging systems, the apparent depth in the scene is proportional to the extent of the parallax-induced differences between the positions of corresponding features in the presented images. The extent of such parallax-induced position differences is determined in part by the degree of separation between the captured images and in part by the distance from the captured images to the scene. Typically, 3-D imaging systems combine images that are captured at generally the same distance from the scene. This simulates the way that the eyes of a human observer will see a scene. Accordingly, the apparent extent of depth in the resultant output is typically modified by varying the degree of separation between the camera positions for the captured images. This creates an important issue for a photographer in preparing a multi-perspective image: the challenge of selecting the proper combination of camera positions necessary to provide a desired depth effect.

It is desirable to provide the photographer greater control in selecting the extent of separation between camera positions, and therefore the extent of the apparent depth in an image. This control can be provided by allowing the photographer to selectively position the camera to take individual images of the same scene from selected perspectives. These images are later reassembled to form a multi-perspective image. One difficulty in using systems and methods of this type is that it is often difficult for the photographer to know at the time of capture what effect the combination of the images captured at the different camera positions will achieve when they are eventually rendered.

However, in giving the photographer greater control, it is important to provide the photographer with the ability to predict how the resultant multi-perspective image will appear when rendered. Cameras that provide a verification system of individual images captured by a camera do not solve this problem because they are typically adapted to show only one captured image at a time.

It is known to provide a downloadable software "APP" for a smart phone, such as an Apple iPhone that displays previously captured 3-D images using an anaglyphic display method. For example, the "Anaglyph Camera" software application by Nigel Crawley Software enables a user to display two different images of a subject, captured using their iPhone's camera, as an anaglyph image.

Anaglyph images are designed to be viewed by a user wearing special 3-D anaglyph glasses having different colored filters for the two eyes (e/g/. a red filter for the left eye and a cyan filter for the right eye). After capturing the images, the user puts on a pair of 3-D anaglyph glasses to view an automatically generated anaglyph, which provides a 3-D effect. The user can then save this 3-D anaglyph image to the photo library on their iPhone, for later viewing. However, this does not enable the user to view a 3-D image during capture, since the glasses are not used until after both images are captured. As a result, it is difficult for the user to capture an image with the desired 3-D effect.

It is known to provide a digital camera using a single image sensor and two separate electronic viewfinders, one for each eye, as described in commonly-assigned U.S. Pat. No. 7,466,336 to Regan et al., entitled "Camera and method for composing multi-perspective images," which is incorporated herein by reference.

A first image is captured, and displayed on one of the electronic viewfinders as a "frozen" image. Next, the user moves the digital camera slightly to compose a second image, which is displayed as a preview image using the other electronic viewfinder. This enables the user to view the 3-D effect as the second image is composed. This approach requires two separate electronic viewfinders, which increases the size and cost of the camera.

There remains a need to provide a compact, low cost, 3-D digital camera that permits a photographer to see a preview or verification representation of a 3-D image during the image composition process.

The Invention's Solution

Apple's granted patent represents a digital camera for capturing stereoscopic images, comprising: an image sensor for capturing a digital image; an optical system for forming an image of a scene onto the image sensor; a user interface having user interface elements; a color image display; a data processing system; a buffer memory; a storage memory for storing captured images; and a program memory communicatively connected to the data processing system and storing instructions configured to cause the data processing system to implement a method for capturing stereoscopic images, wherein the method includes: selecting a stereoscopic image capture mode using the user interface elements; capturing a first digital image of a scene using the image sensor in response to user activation of a user interface element; storing the first digital image in the buffer memory; displaying a stream of stereoscopic preview images on the color image display, wherein the stereoscopic preview images are anaglyph stereoscopic images formed by combining the stored first digital image with a stream of evaluation digital images of the scene captured using the image sensor; capturing a second digital image of the scene in response to user activation of a user interface element; and storing a stereoscopic image based on the first digital image and the second digital image in the storage memory.

The present invention has the advantage that stereoscopic images having a preferred 3-D effect can be captured using a digital camera with a single image sensor and optical system. A stream of stereoscopic preview images are provided to aid in the image composition process in order to capture the stereoscopic image providing a preferred 3-D effect.

It has the additional advantage that the stereoscopic preview images are provided on a conventional color image display, without the need of any special 3-D display technology – though the stereoscopic preview images are viewable with inexpensive, readily available anaglyph glasses if so desired.

It has the further advantage that the captured stereoscopic images can be reviewed by automatically forming anaglyph stereoscopic images for display on the image display.

Apple's patent FIG. 1 noted below is a high-level diagram showing the components of a digital camera system.

Apple's patent FIGS. 4A-4C noted below depict preview images displayed on the image display of the digital camera during the process of capturing a stereoscopic image.

In Apple's patent FIG. 5 noted below we're able to see a flowchart showing steps for reviewing captured stereoscopic images according to one embodiment of the present invention.

Random Patent Points

In a preferred embodiment, the camera lens is a zoom lens and is controlled by zoom and focus motor drives.

The processed images are then stored using the image memory. It is understood that the image memory can be any form of memory known to those skilled in the art including, but not limited to, a removable Flash memory card, internal Flash memory chips, magnetic memory, or optical memory. In some embodiments, the image memory can include both internal Flash memory chips and a standard interface to a removable Flash memory card, such as a Secure Digital (SD) card. Alternatively, a different memory card format can be used, such as a micro SD card, Compact Flash (CF) card, MultiMedia Card (MMC), xD card or Memory Stick.

To review the finer details of this invention, see Apple's Granted Patent 8,780,180

Apple Granted an Advanced Haptics Patent

Apple has been granted a patent today for their invention relating to "Methods and systems for providing haptic control." Patently Apple covered this invention extensively in our May 2012 patent report titled "Apple Reveals Wildly Intelligent Multi-Tiered Haptics System"

Apple credits Russell Maschmeyer and Gordon Cameron as the inventors of granted patent 8,780,060 which was originally filed in Q4 2010 and published today by the US Patent and Trademark Office. To review a more detailed account of this invention along with more graphics, see our 2012 report.

Apple Granted Five Design Patents Today

Apple was granted five design patents today covering their Thunderbolt to Gigabit Ethernet Adapter, their iconic battery life image and a series of three additional icons (D709,086, D709,087 and D709,095 (not shown))

The Remaining Patents granted to Apple Today

A Note for Tech Sites covering our Report: We ask tech sites covering our report to kindly limit the use of our graphics to one image. We thank you in advance for your cooperation. 

Patently Apple presents only a brief summary of granted patents with associated graphics for journalistic news purposes as each Granted Patent is revealed by the U.S. Patent & Trademark Office. Readers are cautioned that the full text of any Granted Patent should be read in its entirety for full details. About Making Comments on our Site: Patently Apple reserves the right to post, dismiss or edit any comments. Comments are reviewed daily from 4am to 8pm MST and sporadically on the weekend.