Apple finally launched their "Cut, Copy and Paste" feature for the iPhone with the arrival of iPhone 3.0. Apple's Senior VP of iPhone Software Scott Forstall reviewed a few of the key Cut, Copy and Paste features during a special event presentation. He began with the Cut, Copy and Paste Bubble feature and then went on to discuss such things as "grab points," the "magnifier" and the ability to copy and paste across applications like Safari and Mail or even third party apps. Forstall also covered how to undo your copy and paste selections by using a "shake" technique in addition to covering how to use copy and paste with your photos into Mail. It took Apple a long time to bring Cut, Copy and Paste to the iPhone and they took a lot of heat for it. Apple's philosophy is that they want to get major features right. Some simply consider that to being nothing more than an excuse. Perhaps in some cases that may be true, but once you see Apple's ten in-depth patents covering this feature, you'll come to realize that Apple was really being honest about the delay of this feature. Forstall began his presentation by stating "we've been working really hard to design an easy to use straight forward user interface for Cut, Copy and Paste for our large touch screen display – and we think that we've nailed it." And for those of us who aren't naysayers, I think we could agree that they did just that.
Apple's patent background segment first lays out the problems with the current round of solutions available today in the marketplace regarding cut and paste on small screen mobile devices. By understanding the problems as Apple see them, we're indirectly being provided with an overview of what Apple's technology is out to remedy.
The patent begins to lay out this foundation by stating that as electronic devices become more compact, and the number of functions performed by a given device increase, it has become a significant challenge to design a user interface that allows users to easily interact with a multifunction device. This challenge is particular significant for handheld portable devices, which have much smaller screens than desktop or laptop computers. This situation is unfortunate because the user interface is the gateway through which users receive not only content but also responses to user actions or behaviors, including user attempts to access a device's features, tools, and functions. Some portable communication devices (e.g., mobile telephones, sometimes called mobile phones, cell phones, cellular telephones, and the like) have resorted to adding more pushbuttons, increasing the density of push buttons, overloading the functions of pushbuttons, or using complex menu systems to allow a user to access, store and manipulate data. These conventional user interfaces often result in complicated key sequences and menu hierarchies that must be memorized by the user.
Many conventional user interfaces, such as those that include physical pushbuttons, are also inflexible. This is unfortunate because it may prevent a user interface from being configured and/or adapted by either an application running on the portable device or by users. When coupled with the time consuming requirement to memorize multiple key sequences and menu hierarchies, and the difficulty in activating a desired pushbutton, such inflexibility is frustrating to most users.
Conventional user interfaces on multifunction devices with touch screen displays do not provide a simple and intuitive way to edit text and graphics. Editing functions such as cut, copy, and paste (if they even exist) are often difficult to find and use, thereby creating a significant cognitive burden on a user. In addition, conventional editing methods take longer than necessary, thereby wasting energy. This latter consideration is particularly important in battery-operated devices.
Accordingly, there is a need for multifunction devices with touch screen displays with faster, more efficient methods and interfaces for editing content. Such methods and interfaces reduce the cognitive burden on a user and produce a more efficient human-machine interface. For battery-operated multifunction devices, such methods and interfaces conserve power and increase the time between battery charges.
A Few Notes
With 57 summary points alone in the first patent, I would rather think that you'll agree that this isn't reading for everyone. Yet if you insist on understanding Apple's every technological cut and paste process and twist, then you'll have to review all nine patents covering this one title "Methods and Graphical User Interfaces for Editing on a Multifunction Device with a Touch Screen Display."
Simply feed any of the individual patent number(s) listed below into this search engine to begin your journey: 20100235770, 20100235778, 20100235735, 20100235734, 20100235729, 20100235726, 20100235784, 20100235783 and 20100235785. I must warn you that you may go blind trying to find the finer differences between each patent (ha!).
Other than the initial patent, the only one that stood out from the pack was one titled "System and Method for Identifying Words Based on a Sequence of Keyboard Events." You'll find that one under patent 20100235780.
Lastly, it should be noted that there's been a wave of patents of late that have begun to openly include the desktop and notebook as being touch related products: A sign of things to come for sure. Here's an example in this group of patents: Under Patent Point # 008 we read: "In some embodiments, the device is a desktop computer with a touch screen display. In some embodiments, the device is portable (e.g., a notebook computer or handheld device)." There's no ambiguity here folks.
At the end of the day, who knows? Perhaps Apple wanted to ensure that this feature was able to run consistently on both iOS and a future iteration of OS X Touch (Lion), before releasing it. Time will tell.
Exemplary User Interfaces Relating to Cut, Copy and Paste
Update 7:30 PM: A new feature is presented in FIG. 6N below. You'll note that an Undo button (604) and a Redo buttton (605) have been added. Whether that was an idea that was dropped in favor of the shake action or one that is yet to come isn't known at this time.
In response to detecting the finger gesture on the editable content, the device expands a portion (FIG. 5B) of the editable content on the touch screen display to a second magnification that is greater than the first magnification while continuing to display at least some of the editable content at the first magnification and displays a content magnifier (910) (e.g., as shown in FIG. 5B). In Patent FIG. 7, the device displays another view of a content magnifier 702.
The Trie Data Structure
In a related patent titled "System and Method for Identifying Words Based on a Sequence of Keyboard Events," Apple provides us with some insight into the brain behind their virtual keyboard. Apple's patent FIG. 4 shown below is a block diagram 400 illustrating an exemplary Trie Data Structure 402. This portion of the patent which describes the Trie Data Structure is really the heart of the patent. If you're a super geek or just happen to be an engineer interested in the field of virtual keyboards, then reviewing this segment of the patent is a must.
For the rest of us, the patent is about simplifying portable device dictionaries related to keyboard functionality so as to save a substantial amount of device memory.
Apple credits Senior VP of iPhone software Scott Forstall along with team members Bas Ording Kenneth Kocienda, Bradford Moore, Marcel van Os, Richard Williamson and Stephan Lemay as the inventors of patent application 20100235785, originally filed in Q3 2009. All of the patents were made public in September 2010 and the noted inventors vary per patent.
Notice: Patently Apple presents only a brief summary of patents with associated graphic(s) for journalistic news purposes as each such patent application is revealed by the U.S. Patent & Trade Office. Readers are cautioned that the full text of any patent application should be read in its entirety for further details. About Comments: Patently Apple reserves the right to post, dismiss or edit comments.