Patent Overview

 

Conventional personal computers include operating systems that often provide a virtual "desktop" metaphor where users could manipulate and organize various graphical objects. This metaphor is easily understood by users because it is intuitive and relates to their real world, physical environment. Modern computing devices, such as smart phones, often provide a large variety of applications. Some of these applications, however, provide interfaces that lack an equivalent of the "desktop" metaphor and as a result are more difficult to comprehend by the average user.

 

Apple's solution is to present a series of new metaphors into the iOS ecosystem that will better reflect the mobile working environment going forward.

 

Example Interactions Digging a Hole in a Device User Interface

 

Apple is always looking for new ways to distinguish iOS from the pack and today Apple introduces us to a number of new GUI gestures and metaphors that are fun and Kooky – if not creatively insane. The first one describes the notion of "digging a hole" in your interface that will allow you to drop a file into it quickly or act as a garbage bin or other uses.

 

In Patently Apple's graphic below you'll find a series of three illustrations representing the phases related to the digging a hole gesture. It begins with FIG. 1 touch inputs 110 corresponding to a digging gesture made by a user. The user provides touch inputs 110 at a location on an iOS device's interface. A digging gesture could correspond to a user positioning his or her finger (or stylus) at a particular height above device 100 on the z-axis, and moving his or her finger down along the z-axis and to the side in the x-y plane.

 

Alternatively or in addition, the digging gesture could correspond to a scraping motion, e.g., one or more short gestures substantially in the x-y plane that mimic scraping at the display with a fingernail. In response to one or more touch inputs (110), the user interface displays a graphical representation of an opening in the user interface.

 

 

Apple's patent FIG. 1B illustrates graphical representation 122 of an opening that is displayed in the user interface. The opening is presented in the user interface at the location where touch inputs 110 were received. In some implementations, the size of the opening could be determined by the number of touch inputs received at the location on the device. For example, the more times a user provided separate touch inputs at the same location, the larger the opening could be. In some implementations, the size of the opening could be determined by a pressure corresponding to the touch inputs. For example, the greater the pressure of the touch inputs, the larger the opening could be. The size of the opening could be the diameter of the opening or the depth of the opening.

 

Associated Animations

 

In some implementations, the user interface also presents a graphical representation of dirt or other material next to opening 122. This dirt or other material represents material that was just removed to create the opening. The user interface could present an animation of more material appearing next to opening in response to each touch input. For example, the user interface could present an animation of dirt flying out of opening and into the pile of material.

 

Different Materials

 

Other representations of opening 122 could also be used. For example, the opening could be represented as a hole in the ground or a hole in dirt or rock. A pile of dirt or rocks could be displayed on the user interface next to the opening. As another example, the opening could be represented by a tunnel. The bottom of the tunnel could be illustrated as connecting to another location, for example, another device in communication with the device. The sides of the tunnel could be illustrated as being made out of plastic, metal, or another material.

 

As yet another example, the opening could be represented as a hole in the device itself. The hole could be represented as having sides and a bottom that are made up of the inner components of an electronic device, for example, wiring and circuit boards. Other representations of the hole or opening could be used for projects or at school to represent such things as a black hole or worm hole in space.

 

Uses for Creating a Hole

 

Moving onto to FIG. 1B, we see that once the opening is displayed, a user could then choose a file and drag and drop it into the hole or opening. In response to this input, the device processes display object or file 106. The device could process display object 106 in various ways. For example, the device could secure the system object corresponding to the display object, move the system object corresponding to display object to a new directory or copy the system object corresponding to display object 106 to another device. In some implementations, display object 106 (a file) is removed from the user interface as part of the processing. It may also be used to dump garbage files or photos etc.

 

In Apple's patent FIG. 1C illustrates example the user interface after display object 106 has been removed from the user interface 101. A user could then close the opening or hole through touch input, for example, by making one or more sweeping gestures over the opening.

 

Example Interactions Opening a Window or Door in a Device User Interface

 

An overview of Patently Apple's second graphic: Apples patent FIG. 2A illustrates an example touch input corresponding to drawing a pattern on a user interface; FIG. 2B illustrates an example graphical representation of a trap door; FIG. 2C illustrates an example graphical representation of an opening displayed in the user interface; and FIG. 2D illustrates the user interface after a display object has been removed from the user interface.

 

 

Apple's patent FIG. 2A illustrates example touch input 202 corresponding to drawing a pattern on the user interface of an iOS device. The user interface includes display object 206, illustrated as a system window displaying the contents of a file from Pages for example. In response to input 202, the user interface presents a graphical representation corresponding to the pattern. The graphical representation could be, for example, a graphical representation of a portal. A portal is a way of moving between two points, for example, a door, a window, a trap door, a tunnel, a sliding panel, or a gate.

 

Apple's patent FIG. 2B illustrates an example graphical representation of trap door 222. The trap door has a size and shape corresponding to the pattern drawn by the user with touch input 202. The touch input corresponds to a rectangular pattern; therefore, the trap door is rectangular. However, other patterns, for example, circles or squares, could also be drawn, resulting in different shaped representations. Trap door 222 is initially illustrated as being closed.

 

A user could provide another touch input to open trap door 222. In FIG. 2B, this input is illustrated by swiping gesture 224 along the y-axis. However, a user could alternatively use a swiping gesture along the x-axis, a lifting gesture along the z-axis, or a gesture along multiple axes.

 

As illustrated in FIG. 2C, a user drags and drops the file (object 206) into the trap door. It works very much like the hole in the first example.

 

In some implementations, the device could send a copy of the system object corresponding to the display object to another device. In some implementations, the device could cause the state of the system object corresponding to the display object to be captured by a backup program executing on the device. For example, a graphical representation of a window could be associated with sending the system object corresponding to the display object to a backup program and a graphical representation of a trap door could be associated with sending a copy of the system object to another device or securing the system object.

 

Shredder: Example Interactions Securely Deleting a Display Object

 

Apple's patent FIGS. 10A and 10B illustrate example user interactions shredding, e.g., securely deleting, a display object displayed on user interface 1002 of device 100.

 

 

For illustrative purposes, touch input 1006 is illustrated in FIG. 10A as corresponding to three fingers providing a swiping input in the y-direction. However, a different number of fingers and/or a different swiping direction could also be used.

 

The user interface 1002 could also present an animation indicating that the display object is being deleted. For example, the user interface could present an animation showing the display object or file being torn into multiple strips like a shredder.

 

Apple's patent application 20110197153 was originally filed in Q1 2010 by inventors Nicholas King, Todd Benjamin and Brett Bilbrey who were also responsible for a second patent application on this subject which introduces a few more new gestures in the works, as shown below.

 

3D File Spheres & Pouring Files into another Device Revisited

 

Apple's second patent FIG. 1A shown below illustrates an example device displaying an example user interface where a first graphical object is transformed into a second graphical object that responds to touch or motion input.

 

 

In some implementations, the GUI 103 could include one or more two or three-dimensional graphical objects. In the example shown, graphical objects are file icons 104, 106, representing files "A" and "B" and folder icon 108 representing folder "C." A user could interact with icons 104, 106, 108 using various inputs. For example, touching file icon 104 could result in file "A" being opened in user interface 103. Graphical objects could represent any type of data or content, including but not limited to files, folders, digital photos or videos, audio files, ebooks, etc.

 

In step 1 (at a first instant of time), a user could use a finger to draw a circle 110 around icons 104, 106, 108 to indicate that the icons are to be grouped together. For example, a user could touch the display at touch point 110 and draw circle 112 around icons 104, 106, 108 without removing their finger from the display. In some implementations, a dashed line or other visual indicator could be displayed to show the circle visually so as to indicate to the user that icons 104, 106, 108 are selected for inclusion into a group.

 

In step 2 (at a second instant of time), when the user removes their finger from the display, circle 112 is automatically transformed into three-dimensional graphical object 114, which contains icons 104, 106 and 108. In the example shown, graphical object 114 is a ball or sphere that is detached or "floating" on the user interface. Note that circle 112 (step 1) and graphical object 114 (step 2) are shown in FIG. 1A as being in two different locations on user interface 103. This was for illustrative purposes only. In practice, circle 12 could be transformed into graphical object 114 at the same location on the user interface. In some implementations, the user interface could also be automatically transformed into a three-dimensional user interface (UI) environment.

 

In some implementations, the iOS device includes onboard motion sensors (e.g., accelerometer, gyros) which could detect motion of the device. Graphical object 114 could move freely about the display in response to motion detected by onboard sensors. Graphical object 114 could be animated so as to make graphical object 114 appear to have mass, which could appear to respond to virtual physical forces in the UI, such as gravity, friction or drag. The graphical object could bounce or reflect off boundaries of the UI other graphical objects. Although the graphical object is shown as a sphere or ball in this example, other graphical objects can be used, such as a cylinder, wheel, block or any other geometric shape.

 

In some implementations, the size of graphical object 114 is based on the size of data represented by graphical object. For example, if the graphical object is a ball, then the radius of the sphere or ball will determine its size (mass). When the graphical object is manipulated on the UI, the behavior of the graphical object in response to touch or motion input could be based on its mass. Larger files (more mass) could be animated to move more slowly than smaller files (less mass) in accordance with Newtonian physics, i.e., acceleration=force/mass.

 

Example Data Transfer Using Graphical Objects

 

In Apple's patent FIG. 1B we see the iOS device of FIG. 1A transferring the graphical object 114 to a second device, proximate to the first device. This was first detailed in our July patent report titled "Apple Wants to Beat HP's WebOS Sharing Feature with Something Cooler." The files could be transferred using Bluetooth, WiFi, RFID or the like.

 

Example Sorting of Graphical Objects

 

In our last graphic below, we present patent figures 3A, 3B and 4. In patent FIGS. 3A and 3B we see an example process for sorting graphical objects 300 based on size (mass). In the example shown, graphical objects 300 (e.g., file and folder icons) could be sorted in the UI based on their size (mass). The iOS device could be placed in a state so that graphical objects 300 are detached from the UI and "float" on the UI as if under the influence of real world, physical forces, such as gravity, friction and drag. A sort function could be invoked by a user. Invocation of a sort function could be a touch or motion gesture, or any other desired input.

 

For example, as shown in FIG. 3A, a user could shake the iOS device to start a sort function. In response, the graphical objects could bounce around the UI providing a "snow globe" effect. The motion of each graphical object could settle to a stationary state on the UI based on the size of the data represented by the graphical object as shown in FIG. 3B.

 

The Aging Metaphor: Example Process for Visually Indicating Aging Data

 

In the last example of the day, we see patent FIG. 4 illustrating an example process of visually indicating the age of data represented by graphical objects. In the real world material objects age over time. For example, a material object may lose its color, shine or elasticity, start to rust, etc. In some implementations, graphical objects 402 representing data could be adjusted visually to indicate to a user the age of the data using this "aging metaphor."

 

 

In the example shown, five states of a folder icon 402 representing aging data are displayed on the UI. At a first time T0, icon 402a is displayed in the UI with 0% transparency. At a second time T1 (where T1>T0), icon 402b is displayed with 25% transparency. At a third time T2 (where T2>T1), icon 402c is displayed with 50% transparency. At a fourth time T3 (where T3>T2), icon 402d is displayed with 75% transparency. And finally at a fifth time T4 (where T4>T3), icon 402e is displayed with 100% transparency. In this example, the transparency of icon 402 was reduced linearly over five time points, as illustrated by curve 404. Accordingly, a user could use simple visual inspection of file icons on the UI to determine the relative age of the files represented by the icons. Curve 404 is shown in FIG. 4 for illustrative purposes and may not be displayed in practice.

 

In some implementations, other visual indicators can be used to indicate age of data other than transparency. For example, icons representing data or files could change color based on age. Age of data could be indicated by adjusting color brightness, hue and saturation of icons representing the data. Icons representing aging data could be animated to appear more active for newer data or files (e.g., a fast jiggling icon) than with older data or files (e.g., a slow jiggling icon).

 

In some implementations, icons representing data could be modified over time to look "hot" or "cold." For example, recently created, edited or reviewed data or files can be represented by icons that include an animated flame and/or be colored with varying shades of red to indicate that the corresponding data or files were recently created, edited or reviewed. And icons representing older files could be animated to appear "cold," such as drawing frost on the icon and/or coloring the icon with varying shades of blue to indicate that the corresponding data or files were created, edited or reviewed in the past.

 

All in all, we see that Apple is working on refreshingly new metaphors for future iOS implementations. And remember that with OS X Lion utilizing some of iOS's methodologies, we may very well see some of these new metaphors landing on our desktops at some point in the future as well.

 

Notice: Patently Apple presents a detailed summary of patent applications with associated graphics for journalistic news purposes as each such patent application is revealed by the U.S. Patent & Trade Office. Readers are cautioned that the full text of any patent application should be read in its entirety for full and accurate details. Revelations found in patent applications shouldn't be interpreted as rumor or fast-tracked according to rumor timetables. Apple's patent applications have provided the Mac community with a clear heads-up on some of Apple's greatest product trends including the iPod, iPhone, iPad, iOS cameras, LED displays, iCloud services for iTunes and more.

 

About Comments: Patently Apple reserves the right to post, dismiss or edit comments.

 

Here are a Few Great Community Sites covering our Original Report

 

MacSurfer, Twitter, Facebook, Apple Investor News, Google Reader, UpgradeOSX, TechWatching, Macnews, iPhone World Canada, MacMagazine Brazil, CBS MarketWatch, 9to5 Mac, MacDailyNews, iPodNN, MacPlus France, Techmeme, TechCrunch, Macdac Russia, IT Times Germany, Deep Apple Russia, iPhoneros Costa Rica, Macnotes Germany, MacObserver, and more.