Apple Wins a Foundational 3D Interface Patent that Might be Timely
On February 28, 2012, the US Patent & Trademark Office published a granted patent from Apple that could be considered a foundational 3D interface patent. The patent describes a new "transition engine" that will be added to OS X and iOS. This new engine will allow a user to choose a tiny object on their new 3D desktop and zoom it out to see what it is or to read a filed PDF or view a photo or video and then allow it to zoom back to the position on their desktop with a cross fading effect in play. It could also transition using an animated effect. So instead of clicking on the object to open it, the user will have the ability to simply zoom an image or app in and out with ease. When you think about it awhile, it's one of the coolest and most refreshing ideas that I've read about in some time.
As Processing Power Dramatically Increases, Apple Wants to Take the Desktop Metaphor to the Next Level
A graphical user interface allows a large number of graphical objects or items to be displayed on a display screen at the same time. Leading personal computer operating systems such as Apple's OS provide user interfaces in which a number of graphical representations of system objects, such as windows, visualization objects representative of files, widgets, hyperlinks, peripherals, applications, and other representations of system objects can be displayed according to the interactions of the user. Taskbars, menus, virtual buttons, a mouse, a keyboard, and other user interface elements provide mechanisms for accessing and/or activating the system objects corresponding to the displayed representations.
The graphical objects and access to the corresponding system objects and related functions, however, should be presented in a manner that facilitates an intuitive user experience. The use of metaphors that represent concrete, familiar ideas facilitates such an intuitive user experience. For example, the metaphor of file folders can be used for storing documents; the metaphor of a file cabinet can be used for storing information on a hard, disk; and the metaphor of the desktop can be used for an operating system interface.
As the capabilities of processing devices progress, however, so do the demands placed on the graphical user interface to convey information to the users in an intuitive manner.
A Future Apple OS will introduce a new Component Called the "Transition Engine"
Apple's granted patent is indeed about a next generation operating system that will include a display environment engine, an interface engine and an all new transition engine. The display environment engine will generate a display environment for a user, including one or more object items disposed within the display environment. The interface engine will receive input from the user including a selection of an object item from the display environment.
The new transition engine will provide a graphical transition of the selected object item to an instance of an executing object associated with the selected object item, the transition retaining the aspect ratio of the object item through one or more successive transition states between a first transition state and a final transition state.
A New 3D Interface
According to Apple, although the patent figures are geared toward the desktop via OS X, the fact is that Apple will also have this next generational 3D interface integrated into iOS as well. A new 3D Eye-Tracking Interface patent application relating to the iPhone was posted earlier this month.
Apple's patent FIG. 2 is a screen shot illustrating an example of a display environment 200 (e.g., a multi-dimensional desktop) for a device which could include visualization objects, a visualization object receptacle, and stack items. An example implementation of a visualization object receptacle 300 is the "Dock" user interface in OS X.
The three-dimensional user interface in some examples, could include two side wall representations 205, 210, a ceiling representation 215, a floor (or desk) representation 220, and a back-wall representation 222. Similarly to a real-life workspace which enables the user to store documents or files on his/her desk, the three-dimensional user interface could enable a user to place application stems (e.g., visualization objects, visualization object receptacle, and stacked items such as, e.g., document representations, file representations, etc.) on a desk representation.
Application items 225e-m could also be linked to the side wall representations and/or the back-wall representation 225 to provide a similar appearance to a paper being thumbtacked to a bulletin board. The application items 225e-m could include visualization objects which represent a particular application associated with a file (e.g., a text document, picture file, movie file, music file, drawing file, etc.).
The transition engine is basically taking something on the desktop that starts off as a representative of a photo or file. When the user which to take a closer look at such photo or file, the transition takes place like a form of zoom effect going from very tiny to full a full size file. Apple describe that such an effect could take on a cross fading transition effect. Examples of this are illustrated below.
Apple's patent FIG. 3A shown above is a screen shot illustrating an example of a display environment having an object transition; FIG. 3B is a screen shot illustrating an example of a display environment having an object transition from a stack representation;
Apple's patent FIG. 3C shown below is a screen shot illustrating another example of a display environment having an object transition from a stack representation; Apple's patent FIG. 3D is a screen shot illustrating an example of a display environment having an object transition accompanied by an animation. In the example of FIG. 3D, the application item 360 is encrypted and can include, for example, a lock representation overlaying the application item.
Patent Credits and Some Closing Thoughts
Apple's patent application was originally filed in June 2007 as the iPhone was being unveiled, by inventors John Louch, Imran Chaudhri, Christopher Hynes, Timothy Bumgarner and Eric Peyton.
Apple has a number of patent applications on the subject of 3D interfaces and you could check out our Archives to view some of them. As we pointed out earlier in our report, we covered a new patent on the matter only weeks ago. It's without a doubt a real trend. And with the advances made to next generation processors and more advanced batteries, Apple sees the day when users will be able to use a 3D interface at all times – while allowing a user to switch back to a standard 2D to conserve energy when needed.
Earlier this month we posted a report titled "Apple's Macroscalar Architecture: Will it Take iOS to the Next Level?" The report was based on and inspired by the fact that out of nowhere, Apple decided to file for the "Macroscalar" trademark in Europe, the US and China. The trademark was associated with a patent Apple has on record for a Macroscalar Processor. The key to that patent was the clue concerning parallelism. Where did we hear that before? Well, it's interconnected with Apple's OpenCL.
The clue is that it's a processor dealing with instruction-level parallelism (ILP). Apple developed an advanced language for instruction-level parallelism called OpenCL which is now an Open Standard via the Khronos Group. OpenCL debuted on OS X Snow Leopard in June 2009.
Wikipedia states that OpenCL is analogous to the open industry standards OpenGL and OpenAL, for 3D graphics and computer audio, respectively. Khronos Group states that OpenGL is the most widely adopted 2D and 3D graphics API in the industry, bringing thousands of applications to a wide variety of computer platforms. It's beneficial to markets such as CAD, content creation, energy, entertainment, game development, manufacturing, medical, and virtual reality.
Now imagine a processor designed to ignite all things OpenCL? Yes, that's where a 3D user interface is likely to come into play. Does that mean it will be a part of Apple's A6 or A7 processors? I don't know. But Apple didn't trademark Macroscalar for nothing and the timing is starting to look very good. With a little patience, I think a 3d interface just might be on the horizon. And the building evidence that we're seeing on the horizon would strongly suggest that it's not a mirage. Then again, time will tell.
Notice: Patently Apple presents a detailed summary of patent applications with associated graphics for journalistic news purposes as each such patent application is revealed by the U.S. Patent & Trade Office. Readers are cautioned that the full text of any patent application should be read in its entirety for full and accurate details. Revelations found in patent applications shouldn't be interpreted as rumor or fast-tracked according to rumor timetables. Apple's patent applications have provided the Mac community with a clear heads-up on some of Apple's greatest product trends including the iPod, iPhone, iPad, iOS cameras, LED displays, iCloud services for iTunes and more. About Comments: Patently Apple reserves the right to post, dismiss or edit comments.
List: Here are a Few Sites covering our Original Report: MacSurfer, Twitter, Facebook, Apple Investor News, Google Reader, Macnews, iPhone World Canada, MarketWatch, MacDailyNews, The Atlantic Wire, iPhoneil Israel Hebrew, The Architosh, and more.
"Your lack of faith disturbs me…."
Posted by: meurdrac | February 29, 2012 at 05:56 AM
I think that the patent graphics are, as always Rob, presenting a "concept" of what could be and not how Apple will actually lay it out in its final form. It could end up being a curved backdrop but one with depth. I'd be able to have 2 times the apps on my iPad's home page that way. I think Apple has something very cool in the works on this project. It may take a few years, but I think they could, as always, do something that none of us could have imagined.
Posted by: MonkeyMo | February 29, 2012 at 05:26 AM
I'm sorry you feel such admiration for this set of ideas as I very much don't. We should take the opportunity to create worlds of experience that focus and enlighten the tasks we use it for, with an acknowledgement that it is created, not imitation "real".
Posted by: reboylin | February 28, 2012 at 11:03 PM