The Problem to Solve

When an individual performs an action in a real world, physical environment, the individual experiences various physical phenomenon that indicates that the task is being performed or has been completed. For example, if an individual pours objects from a first container into a second container, the individual could observe the objects reacting to the forces of friction and gravity. If the objects having different shapes and masses, then the individual would observe different reactions to the forces.

Conventional personal computers include operating systems that often provide a virtual "desktop" metaphor where users could manipulate and organize various objects. This metaphor is easily understood by users because it is intuitive, and like the "pouring" act described above, relates to their real world, physical environment. Modern computing devices, such as smart phones, often provide a large variety of applications. Some of these applications, however, provide interfaces that lack an equivalent of the "desktop" metaphor and as a result are more difficult to comprehend by the average user.

Apple's Solution

Apple's solution begins by stating that a user could make an intuitive, physical gesture with a first device, which could then be detected by one or more onboard motion sensors. The detected motion triggers an animation having a "physics metaphor," where the object appears to react to forces in a real world, physical environment. The first device detects the presence of a second device and a communication link is established allowing a transfer of data represented by the object to the second device. During the transfer, the first device could animate the object to simulate the object leaving the first device and the second device could animate the object to simulate the object entering the second device.

In some implementations, in response to an intuitive, gesture made on a touch sensitive surface of a first device or by physically moving the device, an object could be transferred or broadcast to other devices or a network resource based on a direction, velocity or speed of the gesture.

Particular embodiments of the subject matter described in this specification could be implemented to realize one or more of the following advantages. 1) Users could transfer files and other data between devices using intuitive gestures combined with animation based on physics metaphors. 2) Users could transfer files to a network using intuitive physical gestures, and 3) Users could broadcast files and other data to other devices using intuitive interface or physical gestures.

Transferring Files Using Animation Based on Physics Metaphor

Apple's patent FIGS. 1A-1C note below illustrate an exemplary intuitive, gesture-based communication between two devices using animation based on a physics metaphor.

The first example illustrates an iPhone transferring files to an iPad and of course the reverse could be true as well. The iPhone's Interface is shown displaying a collection of graphical objects (114a-114f) which are in this case file icons representing stored files. In some implementations, the user could select one or more files for transfer to one or more devices by placing the files into a transfer state.

In the example shown, the user has selected four files for transfer by touching their respective objects 114a-114d for a predetermined amount of time and/or using a predetermined amount of pressure during the touch. The user could also select a group of files for transfer by drawing a circle around the icons with a finger or stylus, then using a touch, gesture or other input to select the group of files for transfer. In some implementations, the user could drag and drop individual files onto a container object (e.g., a "suitcase" icon) displayed on interface 112, and then use a touch, gesture or other input to select the container of files for transfer. Other means for selecting individual files or groups of files for transfer are also possible, including but not limited to selecting files through menus or other conventional user interface elements.

Example: A Pouring Motion

Apple's patent FIG. 1B illustrates an iPhone (device 110) in motion relative to the iPad (device 120). Here, the user has rotated their iPhone above the iPad's interface (122) in a manner similar to tipping a glass of water. This angular motion can be detected by one or more onboard motion sensors.

As shown in FIG. 1B, detached objects 114a-114d could be animated to simulate the effect of gravity by "sliding" toward the lowermost portion of interface 112 as device 110 is rotated. The animation of the objects creates the appearance that the objects have mass and are reacting to forces of a real world, physical environment. In FIG. 1C, we see that the iPad could interpret the rotation of the iPhone as a pouring motion as an indication of the user's intent to transfer the files represented by selected objects 114a-114d.

Upon determining that the user of the iPhone intends to transfer the files, the iPhone (device 110) determines if the iPad (device 120) is present and available to receive the files. After a link is established between the two devices by Bluetooth or RFID and authenticated, the iPhone could request that the iPad accept a file transfer.

More Metaphors, Cool Sound Effects & Repelling Forces

Apple's patent spends a lot of time describing various animated physics metaphors that would have the iOS device's interface react differently depending on the metaphor. Apple describes that the appearance of the interface could react to gravity, friction or drag, momentum, torque, acceleration, centripetal forces or any other force found in the real-world physical environment. The rate at which each object moves on an interface could be based on the size or "mass" of the file represented by the object. Larger files that have more "mass" could have their object animated to move slower in the interface while smaller files that have less "mass" could have their object animated to move faster in the interface.

In some implementations, a file could be detached from the interface so that it appears to "float" on in the interface. In Patent FIG. 1C we see that the files are edging to a corner of the iPhone. The patent states that there could be an effect that the files are actually going down a drain or sand flowing through an hourglass or a Genie being pulled out of a bottle. It could even appear like the files are dissolving like a tablet in water which could be accompanied by sound effects like liquid pouring, a tablet fizzing, gas traveling through a valve and believe it or not, they even point to the sound of a teleporter playing – ha – Beam me up Scotty!

On the flipside, you don't want all of your files falling out of your system and so Apple has dreamed up resistance fields when two or more devices come together. For example, the user may have to flick graphical objects with a velocity sufficient to overcome a simulated repelling force at the edge of the devices interface. Some examples of repelling forces include but are not limited to gravity and friction provided by a speed bump or wall of a bubble, where an object either overcomes the repelling force by having sufficient speed to rollover the bump or sufficient speed to break through the bubble wall or has insufficient speed and rolls or bounces back. So in some cases you'll want to apply a pouring effect if you're transferring a lot of files and other times you'll want the resistance field/repelling force to be more guarded so that only what you want transferred leaves your interface.

Quick Data Transfers Between Colleagues at Work

Apple's patent FIG. 2 shown below illustrates initiation of a communications session with a device in response to an interface gesture. For example, the user could touch graphical object 240 that states "5PM" and then make a "swipe" or "flick" gesture on an iPad's interface (250) with one or more fingers in the direction of other iOS devices in a conference room or class. The iPad senses the interface gesture input interacting with graphical object 240 and interprets the gesture as a request to transmit data represented by graphical object 240 to another device.

Before receiving the data transfer request, the iPad could scan the entire conference room or class for the presence of other available iOS devices. In this example, devices 220 and 230 are detected. If communication has not been established, the iPad could establish communication with other devices. The system, if all else fails, could be set up manually for transferring files.

Data Broadcasting

Apple's patent FIG. 3 illustrates initiation of a data broadcast from a device to multiple devices in response to a physical gesture. Devices 310, 320, 325, 330 are located in proximity to each other. Devices 310, 320, 325, 330 could be a computer enabled display device, such as an electronic tablet, computer monitor, projection screen, electronic whiteboard, teleconferencing screen, television, or other type of device that could display information.

In the example shown above, the user has selected graphical object 340 (a file icon) to indicate an intention to perform a data transfer action. Device 310 is also shown in a rotational or sweeping motion due to a user performing a clockwise (or counterclockwise) rotational or sweeping gesture that emulates a toss of a Frisbee. Motion sensors onboard device 310 senses this physical gesture and interprets it to indicate the user's intent to broadcast data represented by graphical object 340 to devices 320, 320, 325, 330.

The Vacuum Metaphor: Suck up Those Files!

Apple's patent FIG. 4 illustrates a data transfer between two devices in response to intuitive, physical gestures. In the example shown, the user intends to request that data corresponding to icon 436 be transferred from an iPad (device 420) to an iPhone (device 410). The user indicates an intent to transfer data by placing the iPhone in position and orientation (440a) relative to the iPad (device 420) and then moving the iPhone across display of the iPad. In some implementations, the gesture just described could be performed with a metaphor for the user holding and using the iPhone as a "vacuum" sucking up the files. Okay, that was a funny image.

And here's another funny image that Apple presented: In some cases, the sending device could be held over a receiving device and a touch sensitive surface on the sending device could be tapped to cause items to be transferred to the receiving device over a wireless communication link. In this example implementation, each tap could cause one item to transfer. This gesture could be analogized to tapping a Ketchup bottle to get the Ketchup to flow.

For more details, see patent application 20110163944. Apple's patent application was originally filed in Q1 2010 by inventors Brett Bilbrey, Nicholas King and Todd Benjamin.

Notice: Patently Apple presents a brief yet detailed summary of patent applications with associated graphic(s) for journalistic news purposes as each such patent application is revealed by the U.S. Patent & Trade Office. Readers are cautioned that the full text of any patent application should be read in its entirety for full and accurate details. Revelations found in patent applications shouldn't be interpreted as rumor or fast-tracked according to rumor timetables. Apple's patent applications have provided the Mac community with a clear heads-up on some of Apple's greatest products including the iPod, iPhone, the iPad, iOS cameras, LED displays, iCloud services for iTunes and more.

About Comments: Patently Apple reserves the right to post, dismiss or edit comments.

Here are a Few Great Community Sites that are covering our Original Report

MacSurfer, New York Times (Tech Headlines from around the Web), Apple Investor News, Google Reader, UpgradeOSX, TechWatching, Macnews, Planet Geek Portugal, iPhone World Canada, CBS MarketWatch, TechSpy,Alberto ate de Madrugada Portugal, Financial Times, Techmeme, MacStories, CNET, MacTechNews Germany, Blogrunner, Yahoo News, Indecplus China, Gizmodo, Gizmodo Brazil, Digi Norway, phoneArena and Engadget Germany.

9to5 Mac, Nu Netherlands and MacRumors, IT Magazine Switzerland, iClarified, Mac Life Germany, Gadgeterija Bosnia, Cult of Mac, Business Insider, MacPlus France, Macnotes Germany, Ars Technica, BGR, B92 Serbia, iPhone Download Blog, MacDailyNews, iMaiCafe Thailand, iPhoneHellas Greece, The Mac Observer, iCloudBR Brazil and Applespot Estonia.

MacWorld Netherlands, Crave Asia, GeMUG Georgia, PCtipp Switzerland, App Advice, GottaBeMobile, Intomobile, Appolicious, NEO 2 Greece, The Unlockr, ECDL Blog Greece, DiarioTi Spain, Geeky Gadgets, Engadget US, MacDaily Georgia, 359 GSM Bulgaria, Crave Taiwan, Super Stereo 98.1 FM Venezuela, Bright Netherlands, Macworld-UK, iPhoneWelt Germany, PC Advisor UK, Dice, theAPPERA, and more. 

Note: The sites that are linked to above offer Apple community members with an avenue to make comments about this report in many original languages. Additionally, many of these sites provide our guests with different takes on any given patent or concept that is presented in our reports to make it more fun, interesting and/or personal. If you have the time, join in!