An Overview of Apple's Invention

Apple's invention is focused on handheld wireless communication devices that have at least two cameras facing in opposite directions. The device receives a first video stream and a second video stream simultaneously from the two cameras. The device detects a speech activity of a person who is captured in the video streams by detecting the direction of sound or lip movement of the person. Based on the detection, the device automatically switches between the first video stream and the second video stream to generate a multiplexed video stream. The multiplexed video stream contains interleaving segments of the first video stream and the second video stream.

In one embodiment, the device detects the speech of a person by detecting the direction of sound. The device may include more than one microphone for detecting the direction of sound. A first microphone may point to the same direction to which one camera points, and a second microphone may point to the same direction to which the other camera points. Based on the direction of sound, the device automatically switches between the first video stream and the second video stream to generate a multiplexed video stream in which the video is, in a sense, "synchronized" to the speaker.

In another embodiment, the device detects the speech of a person by detecting the lip movement of the person. The device may include an image analyzer for analyzing the images in the video streams. Based on the lip movement, the device automatically switches between the first video stream and the second video stream to generate a multiplexed video stream.

In one embodiment, the multiplexed video stream may be transmitted to a far-end party by an uplink channel of a live video call. In another embodiment, the multiplexed video stream may be stored in the memory of the handheld communication device for viewing at a later time.

The handheld communication device may be configured or programmed by its user to support one or more of the above-described features.

Apple's Unique iReporter Mode

One of the key aspects of Apple's new patent application is squarely aimed at the modern day iReporter – Those in the public who capture a live event or incident anywhere, anytime, and then shares it with rest of the world via a website for greater distribution. It could be a citizen capturing a great photo of President Obama speaking at a rally or recording a coming tornado, live. Apple thinks that their latest invention will be able to assist iReporters that use an iPhone in many situations including live interviews.

What will it do and how will it Work?

Apple's proposed iPhone feature would be initiated by the user by either pressing a physical button positioned near the volume buttons or presented as a virtual button presented on the iPhone's UI as shown in our cover graphic. This particular feature that is generically referred to as the "auto-select" button allows the user to select the new "report mode." In Apple's patent FIG. 2 shown below we see a diagram illustrating a person using this proposed iPhone feature.

According to Apple, the user may hold this future iPhone in an upright position, such that the front-facing camera faces the user and the rear-facing camera faces a scene that the user wishes to capture in a video. The user may start a video capturing session that turns on both the front-facing camera and the rear-facing camera. The user may then activate the auto-select feature of the iPhone which would enable it to automatically switch between the two simultaneously captured video streams for a report.

Think of an iReporter filming a documentary on Elks in the Great White North. While he's filming the elk in the wild, the camera remains fixed on the elk and its natural surroundings. Then when the iReporter adds a narrative, for example, the iPhone would automatically switch camera views and focus on the iReporter speaking. When the iReporter falls silent, the camera would return to filming the elk. The two video streams may be interleaved throughout the multiplex video stream.

Apple states that the video could be transmitted live over various networks or recorded for later editing by the reporter at work or at home.

The iPhone's "Interview Mode"

In Apple's patent FIG. 3 below we see a diagram illustrating a scenario in which a first person may activate the auto-select feature of the iPhone to select "interview mode." The interview mode could be used when a person is recording a conversation with another person or when there are several people participating in a video conference.

In interview mode, a future iPhone may use both of the two microphones 113 and 114 to detect the direction of sound and to switch the video streams to face the direction of the detected sound. In another embodiment, the iPhone may use image processing techniques to detect lip movement of the first and second person.

Considering that the camera switches back and forth between subject and interviewer/reporter, I think that it's a very cool feature that will appeal to both the amateur and professional reporter alike.

Apple's patent FIG. 5 noted below is a block diagram illustrating an embodiment of the video processing module 480 which is the brains behind these new iPhone features involving multiplexed video streaming.

For those wishing to delve into the deeper side of the video processing unit which appears to control these new features, review patent points 032 through to 038. Apple's patent application 20110164105 was originally filed in Q1 2010 by inventors Jae Han Lee and E-Cheng Chang. It should be noted that although iReporter is a recognized CNN term, the general use of it in this report is referring to any Internet-Centric-Reporter. The only reference to iReporter in context with CNN is in our opening summery where it's properly identified as such.     

Other Noteworthy Patent Applications Published Last Week

We may list other patent applications here later today.

Notice: Patently Apple presents a brief yet detailed summary of patent applications with associated graphic(s) for journalistic news purposes as each such patent application is revealed by the U.S. Patent & Trade Office. Readers are cautioned that the full text of any patent application should be read in its entirety for full and accurate details. Revelations found in patent applications shouldn't be interpreted as rumor or fast-tracked according to rumor timetables. Apple's patent applications have provided the Mac community with a clear heads-up on some of Apple's greatest products including the iPod, iPhone, the iPad, iOS cameras, LED displays, iCloud services for iTunes and more.

About Comments: Patently Apple reserves the right to post, dismiss or edit comments.

Here are a Few Great Community Sites Covering our Original Report

MacSurfer, NYTimes (Headlines Around the Web),Twitter, Facebook, Apple Investor News, Google Reader, UpgradeOSX, TechWatching, Macnews, iPhone World Canada, CBS MarketWatch, Melamorsicata Italy, 9to5 Mac, iFun | iPhone Germany, iPhoneclub Netherlands, Digital Italy, iGeneration France, MacMagazine Brazil, iPhoneItalia Italy, The Mobile Indian New Delhi India (English), Fast Company, AppAdvice, Heise online Germany, ZDNet Asia, GMA News TV, OMNI (Citizen Journalist site), CNET, Periodismo Ciudadano Spanish, and more