A New Patent Reveals Scrollable Menus & Toolbars for OS X Lion
During a recent Special Event, Apple's CEO Steve Jobs provided us with a peek into four key attributes coming to the next major operating system upgrade due next summer called "Lion." Specifically, Jobs reviewed the fundamentals of Lion's Mac App Store, Launchpad, Full Screen Apps and Mission Control. What Jobs didn't reveal was something called "scrollable menus and toolbars." This possible new OS X Lion feature came to light late last week in a new European patent filing. The new feature is clearly identified as one being designed for both the desktop (OS X) and handheld devices (iOS) – which is what OS X Lion is all about: "The Power of OS X – The Magic of iPad." Apple's Full Screen App feature is about simplifying the desktop and their latest scrollable menus and toolbars concept extends upon that very thinking.
Patent Background
User interfaces of computer applications often provide numerous commands or user interface tools for a user to choose from. For instance, many applications include drop down menus. Such menus often include lists of commands. There are standard commands such as opening a file, printing a document, saving a document, etc., as well as more application-specific commands such as inserting a page break (for a word-processing application), formatting cells (for a spreadsheet application), etc.
Many applications also include toolbars or tool palettes. For instance, many computing applications have one or more rows of toolbars at the top of their user interface. The toolbars include various types of tools. Drawing applications have drawing-tools – video editing applications have video-editing tools, etc. Some applications also have floating tool palettes. For instance, some video-editing applications include a floating trim tool palette that allows a user to select between various different trim tools for video-editing.
Such toolbars, tool palettes and drop-down menus could take up valuable real estate in the graphical user interface. Furthermore, the number of items in such menus and toolbars is limited by screen space. For example, a user doesn't want a floating tool palette that takes up too much of the screen. As more options are added to a tool palette, the tool palette must get larger or the size of the options must get smaller. Just as the tool palettes could only increase in size to a certain point, the size of the options could only decrease to a certain point before they become indistinguishable. As such, a need exists for ways to display toolbars or menus that don't take up significant amounts of screen space while keeping the options at a visually recognizable size.
Novel Methods of Presenting a Menu
Apple's patent and illustrations introduce us to a new concept of scrollable menus and toolbars that may very well end up in both OS X Lion and a future iteration of iOS. The new menus and toolbars may also be integrated into future iterations of Apple applications such as Aperture and/or suite applications like Pages, Numbers and iMovie.
To begin with, Apple states that their invention provides a novel method for presenting a menu in a graphical user interface. The menu includes several selectable menu items that are assigned in a particular order for scrolling through a selection window based on user input. In some embodiments, each of the menu items is associated with a command. When a menu item is in the selection window, a user could select the menu item in order to perform the command associated with the menu item.
The menu items may be displayed as standard text items in some embodiments (e.g., "File", "Save", "Print", etc.) or may be displayed as icons (e.g., icons representing various tools, such as drawing tools, video editing tools, etc.).
The selection window is an indicator (e.g., a highlight, box, etc.) that indicates that a particular menu item could be presently selected. In some embodiments, the selection window is static or nearly static as the menu items scroll through. In the menu, the order of the menu items is the order in which they scroll through the selection window in response to user input. However, when the menu is initially presented, the menu item that is initially displayed in the selection window need not be the first menu item in the order (i.e., the leftmost, topmost, etc.). As the menu items could scroll in two directions (e.g., left and right, up and down), some embodiments initially display the menu item in the middle of the order in the selection window.
Different embodiments use different techniques to determine this order. For instance, in some embodiments, the order could be defined by a user, by a random process, and/or by a developer of the program or operating system that uses the menu. In some embodiments, the order could also change based on use statistics, favoring menu items that are more regularly selected.
Scrolling Methods Could Differ
The nature of the scrolling could be in two different forms in some embodiments, depending on how the items are ordered. In some cases, the menu items can't scroll past the first and last items in the order. In other cases, however, the items could be continuously scrolled in a loop as the ordering of the menu items is only defined relative to one another, and there is only a first item to the extent that there is an item that is initially displayed in the selection window.
In either of these cases, some embodiments scroll the menu items through the selection window in such a fashion that the number of menu items is not limited by space in the graphical user interface. For instance, some embodiments only display the menu item that is presently in the selection window and a portion of the menu items on either side of the selection window. Some embodiments display a small number of menu items on either side of the selection window, with the items that are not actually in the selection window displayed as 5 partially faded. As the menu items scroll through the selection window, new items appear in the display as previously displayed items disappear.
Menu Shapes Could Differ
The shape of the displayed menu is also different in different embodiments. Some embodiments display the menu in linear form, with the menu items scrolling in one dimension (e.g., horizontally or vertically). Other embodiments display the menu as a semi circle, with the menu items scrolling along the semicircle. Other non-linear configurations of menu items are also possible.
The menu also might be presented differently in different embodiments in that some embodiments display the menu constantly at a set location, whereas other embodiments require user input to invoke the display of the menu. For example, the menu might be a toolbar in a media-editing application that occupies a particular location in the user interface of the application, or a menu in an operating system that is invoked by a keystroke.
Differing Navigation Schemes & Submenus
Different embodiments provide different schemes for navigating through the menu items. In some embodiments, users could click over the displayed items on either side of the selection window in order to cause the next item in the menu order to move into the selection window. Alternatively or conjunctively, in some embodiments, users could scroll the menu items by using various forms of sliding or swiping input (e.g., dragging a mouse). In some cases, the menu items are displayed as moving in large, discernible steps; while in other cases the items are displayed as moving in small steps that make the movement appear smooth.
When a selectable menu item is aligned in the selection window, a user could select the menu item in order to cause instructions associated with the selected menu item to be performed. In some embodiments, the instructions could activate a particular tool (e.g., if the menu is a group of video-editing tools), could cause a command (e.g., save a file, print a document, etc.) to be performed, or could open a submenu (See Patent FIG. 16 above). Submenus are opened perpendicular to the previous menu in some embodiments. For instance, when the menu items in an initial menu scroll horizontally through the selection window, some embodiments open a submenu such that the menu items in the submenu scroll vertically through the selection window.
Video Editing App Example
Apple's patent Figure 20 shown below illustrates a video-editing application 2000 that is used to create composite video presentations and that utilizes the new scrollable menus. The video-editing toolbar 2020 is an example of the use of the menus described above. Specifically, toolbar 2020 provides the user with multiple trimming tool options.
Apple credits Egan Schulz and Tom Langmacher as the inventors of this International Patent Application, originally filed in Q2 2010 and published November 4, 2010. Patently Apple would like to thank David Borel of Switzerland for emailing us about Apple's new patent application.
Notice: Patently Apple presents only a brief summary of patents with associated graphic(s) for journalistic news purposes as each such patent application is revealed by the U.S. Patent & Trade Office. Readers are cautioned that the full text of any patent application should be read in its entirety for further details. About Comments: Patently Apple reserves the right to post, dismiss or edit comments.
Mac Community Sites Covering our Report
MacSurfer, Apple Investor News, Google Reader, Techmeme, MacDailyNews, MacRumors, MacPlus France, Computerworld, GigaOM, MacStories, MacTechNews Germany, Melablog Italy, TiPb, 9 to 5 Mac, Applesfera Spain, MacKozer Poland, Saggiamente Italy, iThinkDifferent, iSpazio Italy, iClarified, ModMyi, iPhoneAddict France, TUAW, x86OSx Korea, Erictric, iPhone Hacks, MacFreak Netherlands, Apple Linkage Japan, MacNotes Germany, OneApple Italy, Klamm Germany, iCreate Netherlands, iPadevice Italy, iMaiCafe Thailand, Cult of Mac, Mac Observer, Ferra Russia, iPhone4 France, AreaMobile Germany, MacTalk Australia, maclatino Spanish, ITProPortal, Tengo un Mac Spain, journal.mycom Japan, and more.
Why is this so complicated for some people to digest. Imagine being on your iPad and having a single tool icon on your nice full-screen app. You touch it and it automatically fans out in a circle. Your finger glides around the circle until you have the tool you want and when you lift your finger, the tool-circle springs back to being a simple single icon and you have the tool you want. That's so Apple and that's so cool.
Posted by: Joe | November 09, 2010 at 11:31 AM
@ Mike
Thanks for your comment Mike because others may hold this view as well.
First off, the report clearly states you'll be able to use text or icons depending on the applications: "The menu items may be displayed as standard text items in some embodiments (e.g., "File", "Save", "Print", etc.) or may be displayed as icons (e.g., icons representing various tools, such as drawing tools, video editing tools, etc.)."
Secondly, the patent is using "generic tool" icons to make the point that icons could be used. So I wouldn't try to read too much into this Mike as it's just for illustrative purposes. In fact, you should have a sense of humor about this - as the engineers are having fun using "application tools" in the form of "work tools" to get their point across.
Posted by: Jack Purcher | November 08, 2010 at 08:34 PM
Ooooooh, this is dreadful - it's the "pictures make things easy" monster.
Yes, if you're about four years old, a picture makes things much easier. But if you're a bit older, pictures are more confusing than helpful. There's only so much you can do with a tiny square of space, so most pictures end up vague, ambiguous, and confusing.
I know. I work with it. It's one of the things I hate about Windows 7 and the latest version of Office for Windows. And unlike the current Mac toolbar, Office doesn't have a text plus image or text-only option, only pictures. It confirms my belief that the Windows UI is developed by 4-year-olds in a Redmond preschool hidden on the corporate campus.
Is this 'pictures make things easy' nonsense finally coming to OS X? Ugh! Let's hope not, or at least let's hope the graphics above are wrong and there's a text-only option like the current toolbar.
Why? Because I can read far faster than I can guess what a childish picture of a hard hat, a bicycle or a saw means. Does anyone know what they mean in the above illustrations?
I rest my case.
Posted by: Mike Perry | November 08, 2010 at 07:06 PM