The One-Way Limitation of the Current "AirPlay" Feature

In recent years, systems have been developed for mirroring the display output of a computing device such that the output is viewable on both a display of the computing device and a secondary display that may be remote from the computing device. For example, the "Airplay mirroring" feature implemented on certain Apple computing devices (e.g., the iPhone.TM. and iPad.TM.) allows information that is presented by an application on a screen of the computing device to be wirelessly streamed to a television via an intermediate device (e.g., Apple TV.TM.). Thus, when Airplay mirroring is enabled, users can simultaneously view the same media or application content on the computing device display and the television.

One current limitation with this feature is that the communication between the computing device and the intermediate device/television is generally one way (i.e., from the computing device to the intermediate device/television). Accordingly, there is no way for a user viewing the television to provide, though an input interface of the television or the intermediate device, commands back to the computing device for interacting with the application executing on the computing device.

Making "AirPlay" a Two-Way Feature with Television

Apple's invention relates to providing techniques for concurrently presenting multiple, distinct user interfaces for a single software application on multiple display devices. Each of the user interfaces can be interactive, such that user input received with respect to any of the user interfaces can change the state of the application and/or modify data associated with the application. Further, this state or data change can be reflected in all (or a subset) of the user interfaces.

The application, noted earlier as "Airplay," can further generate a second UI configured to be presented on a second display device (e.g., a television) while the first UI is being presented on the first display device. In certain embodiments, the second display device can be physically remote from the first display device and the computing device, and can be indirectly coupled with the computing device via an intermediate device (e.g., a digital media receiver, a router, an Internet-enabled cable/set-top box, etc.). The second UI can have a second layout and expose a second set of functions (distinct from the first UI) that are tailored for a user viewing the second display device.

With the foregoing techniques, users can concurrently interact with a single application via multiple UIs, where each UI is presented on a different display device and is controlled via a different input interface. As noted in the Background section, prior art mirroring mechanisms allow information that is presented by an application on a screen of a computing device to be wirelessly streamed to a remote television via an intermediate device. However, the communication between the application and the intermediate device/television is one way--a user viewing the television cannot provide, though an input interface of the television or the intermediate device, commands back to the computing device for interacting with the application. Rather, the application must be controlled via an input interface of the computing device. Certain embodiments of the present invention overcome this limitation and can allow users of both the computing device and the intermediate device/television to simultaneously control/interact with the application via respective input interfaces.

Further, the multiple UIs generated according to embodiments of the present invention can be distinct from each other and thus can be designed for different usage scenarios. For instance, the second UI described above can have a simplified layout and expose simplified control functions that are particularly suited for presenting and interacting with the application via, e.g., a television, since a user sitting in front of the television will likely be positioned relatively far from the screen and only have access to a simple input device (e.g., a remote control). In contrast, the first UI can have a more complex layout and expose more complex control functions that are particular suited for presenting and interacting with the application via, e.g., a computer display, since a user sitting in front of the computer display will likely be positioned relatively close to the screen and have access to one or more sophisticated input devices (e.g., keyboard, mouse, etc).

Apple References a Possible Future TV

Apple distinctly states that "Although digital media receiver 208 (Apple's current Apple TV box) and television 210 are shown as separate devices, in certain embodiments they can the combined into a single device (e.g., an Internet-enabled television).

Further, remote control 212 can be a simple remote (e.g., a remote control with a fixed input interface, such a fixed number of buttons) or a complex remote (e.g., a remote control with a configurable and/or dynamically modifiable input interface). As an example of the latter case, remote control 212 can be implemented using a smartphone or tablet.

Assign a Rating to Photos

Apple states that in one set of embodiments, the television viewer can assign a particular rating to the photo, such as "like," dislike," or a star rating. These ratings can be mapped to particular buttons on remote control, such that the assignment process can be carried out by activating a single button. By way of example, a "like" rating can be mapped to a "menu up" remote control button, a "dislike" rating can be mapped to a "menu down" remote control button, and a star rating of 1-5 can be mapped to numeric "1-5" remote control buttons.

Upon receiving a command for a particular rating, a digital photo application (Aperture or iMovie) can save the rating with the photo and update the UI presented on a television to display the new rating. For example, FIGS. 8-10 noted above illustrate versions of a user interface that depict the photo as being assigned a rating of "like," "dislike," and "2 stars" respectively.

Apple states that other types of button mappings are also possible. In certain embodiments, the mappings shown above can change in different contexts. For example, when a map is visible in the UI (per FIGS. 12 and 13), the up/down buttons may be used to zoom in/out of the map rather than assign like/dislike ratings.

In my view, Apple has only provided us with the tiniest sliver of information regarding a future "internet enabled television." This is just the tip of the iceberg: interactivity with your future TV. For now, we see that Apple is working on some new features for the next generation of their Airplay feature.

Patent Credits

Apple credits Nikhil Bhatt as the sole inventor of this patent application which was originally filed under serial number     300408 in Q4 2011. Considering that this is a patent application, the timing to market of such an Apple product is unknown at this time.

A Note for Tech Sites Covering our Report: We ask tech sites covering our report to kindly limit the use of our graphics to one image. Thanking you in advance for your cooperation. 

Patently Apple presents a detailed summary of patent applications with associated graphics for journalistic news purposes as each such patent application is revealed by the U.S. Patent & Trade Office. Readers are cautioned that the full text of any patent application should be read in its entirety for full and accurate details. Revelations found in patent applications shouldn't be interpreted as rumor or fast-tracked according to rumor timetables. About Comments: Patently Apple reserves the right to post, dismiss or edit comments.

New on Patent Bolt this Week

Google's "Ingress" Game Logo is now a Registered Trademark

Google Prepares Glass to be a Tool for the Hearing Impaired

BlackBerry Reveals a Future Flex Screen Smartphone Design

Microsoft Invents Voice-Controlled Camera for Windows Phone