Apple's Patent Background

Some electronic devices include a graphical display system for generating and presenting graphical objects, such as free-form drawing strokes, strings of text, and drawing shapes, on a display. A user of such devices may interact with the graphical display system via a user interface to select certain properties of a graphical object to be generated as well as to select at least one position on a display at which the generated graphical object is to be presented. However, currently available electronic devices may limit the ways by which a user may alter certain properties of a graphical object via the interface.

Apple's Virtual Drawing Space Application

Apple's invention relates to systems, methods, and computer-readable media for changing graphical object input tools.

A graphical object input tool may be an indicator that may be generated and presented on a display by a virtual drawing space application to show the current insertion point for new graphical object data that may be created by the input tool on the display. New graphical object data that may be generated using an input tool may be any suitable type of graphical data, such as a drawing stroke, a string of text, or a drawing shape. In some embodiments, at least one visual characteristic or property of an input tool may be indicative of a particular property of new graphical object data that may be inserted or otherwise presented at the tool's position on the display.

Various user input gestures may be provided to directly interact with a displayed input tool for changing various properties of the input tool. For example, rather than having to select options on a displayed menu, a user may provide input gestures at or near a displayed input tool to directly manipulate one or more properties of that input tool, such as its size or color. By visually changing how an input tool is represented on a user workspace so as to indicate a change in an input tool property, a user may be provided with a more efficient and intuitive user interface for generating graphical objects.

Moreover, by allowing a user to change an input tool property using an input gesture that may be independent of any displayed menu, the user may be allowed to focus directly on the input tool being used to generate the graphical object, for example, without a user having to periodically move his or her attention away from the input tool and towards a menu for altering an input tool property. Such an input gesture may be a multi-touch input gesture or an input gesture with a position that is directly associated with a position of an input tool presented on a display.

For example, in some embodiments, there is provided a method for generating graphical object data. The method may include defining input tool content with multiple input tool properties and initially rendering on a display an input tool that is indicative of at least a first input tool property of the multiple of input tool properties. The method may also include receiving an input gesture, changing the first input tool property based on the input gesture, and re-rendering the input tool on the display after the changing. In some embodiments, the received input gesture may be a multi-touch input gesture, such as a multi-touch rotate input gesture that may change an orientation input tool property of the input tool content, or such as a multi-touch pinch or pull input gesture that may change a size input tool property of the input tool content. In some embodiments, the received input gesture may be independent of any menu provided on the display. As another example, the received input gesture may be indicative of at least one position on the display where the input tool is initially rendered. This may provide the user with a greater sense of control over the input tool and its various input tool properties.

The Canvas

The screens 300B, E and H noted below may present an interface for a virtual drawing space application found on a device with which a user may create and manipulate graphical objects for making original works of art (e.g., a virtual drawing space application that may be similar to that of Photoshop or Illustrator by Adobe Systems Incorporated or Microsoft Paint by Microsoft Corporation). It is to be understood, however, that the noted screens are merely exemplary, and that the display may present any images representing any type of graphical objects and/or graphical object animation that may be generated and processed by graphical display system.

For example, as shown in the patent figures below, a virtual drawing space application may provide a canvas area 301 on a portion of the screen in which various graphical objects may be presented. The canvas may be a virtual drawing workspace portion of the screen in which pixel data may be created and manipulated for creating user works of art.

The application may also provide on a portion of the screen at least one artist menu 310 that may include one or more graphical input options that a user may choose from to access various tools and functionalities of the application that may then be utilized by the user to create various types of graphical objects in the canvas area. The menu may provide one or more toolbars, toolboxes, palettes, or any other suitable user interface menus that may be one or more layers or windows distinct for the canvas.

Moreover, the artist menu may also include a background image input option 318, which a user may select for importing video-based or photographic images into the canvas area. It is to be understood, however, that options 312-318 of the artist menu are merely exemplary, and a virtual drawing space application may provide various other types of options that a user may work with for creating content in canvas area.

Additionally, the drawing tool input sub-menu 312a (which is noted incorrectly as 311a in patent FIG 3B) may allow the user to select a drawing stroke input tool from a group of various pre-defined drawing stroke input tools or stamps, such as with an input tool sub-option 311 for presenting a "circular pen" drawing stroke input tool, an input tool sub-option 313 for presenting a "polygonal marker" drawing stroke input tool, and an input tool sub-option 315 for presenting a "triangular bristle" or brush drawing stroke input tool.

It is to be understood that additional or alternative pre-defined drawing stroke input tools of various other pre-defined shapes, other pre-defined patterns, and other various pre-defined input tool properties may also be provided by drawing tool input sub-menu 312a of artist menu 310.

Works with a Mouse or Touchscreen

Apple states that a device may be configured to allow a user to provide input tool information to the system using any suitable gesture or gestures of any suitable input component or input components, such as a mouse or touch screen. A user may provide a pinch gesture on a touch screen, and the system may be configured to process such a gesture as input tool information to reduce the size property of the input tool.

In some embodiments, rather than changing a color tool property, a single tap or click input gesture may be configured to change a pattern property of an input tool (e.g., from a pen, to a marker, to a bristle pattern tool). Alternatively, such a single tap or click input gesture, or any other suitable input gesture may be configured to change a shape property of an input tool (e.g., from a circle, to a polygon, to a triangle shape). It is to be understood that the system may be configured to change any input tool property in response to any particular input gesture or combination of input gestures generated by any type of input component or any combination of input components.

To clarify, Apple states that a gesture may be characterized by, but is not limited to, a pinching, sliding, swiping, rotating, flexing, dragging, or tapping motion between or with any other finger or fingers. A single gesture may be performed with one or more hands, by one or more users, or any combination thereof.

Overview of Apple's Graphical Display System

Apple's patent FIG. 2 shown below illustrates a schematic view of a graphical display system 201 of an electronic device that may be provided to generate and manipulate graphical data for presentation on a display. For example, in some embodiments, the graphical display system may generate and manipulate graphical data representations of two-dimensional and/or three-dimensional objects. The graphical display system may be configured to generate and manipulate realistic animated images in real time (e.g., using about 30 or more screens or frames per second) for presentation to a user on a display.

While the systems that this application will realistically apply to include any Mac and/or iDevice (iPad, iPhone), Apple insists on throwing in everything but the kitchen sink such as: a video player, still image player, game player, other media player, music recorder, movie or video camera or recorder, still camera, other media recorder, radio, medical equipment, domestic appliance, transportation vehicle instrument, musical instrument, calculator, other wireless communication device, personal digital assistant, remote control, pager, computer (e.g., a desktop, laptop, tablet, server, etc.), monitor, television, stereo equipment, set up box, set-top box, boom box, modem, router, printer, and combinations thereof.

Overview of Apple's Graphical Display System Relating to Object Layer Management

Apple's patent FIG. 2 shown below is from a second patent that directly relates to Apple's next generation graphics app. This particular illustration shows us another schematic view of a graphical display system in respect to object layer management.

In Apple's second patent background they state that some electronic devices include a graphical display system for generating and presenting graphical objects, such as free-form drawing strokes, images, strings of text, and drawing shapes, on a display. A user of such devices may interact with the graphical display system via a user interface to create different graphical objects in different layers, which may overlap and be stacked in various orders when presented for display. However, the ways in which currently available electronic devices allow a user to manage various layers of graphical object data may be confusing or overwhelming.

Apple's solution to this problem is found in a secondary patent application published today that relates to the same next generation graphics app as earlier covered. This secondary invention focuses in on systems, methods, and computer-readable media for managing layers of graphical object data.

Apple states that rather than explicitly creating and managing multiple graphical object layers (e.g., via a layers list that may be presented to and manipulated by a user), a graphical display system of an electronic device may be configured to utilize an implicit layer scheme that may be less confusing and less overwhelming to a casual user.

Such an implicit layer management scheme may provide an interface that may not confuse a user with a layers list or that may not put a user in a situation where he or she may try to create a first type of graphical object content when a layer that is incompatible with the first type of graphical object content has been activated. Therefore, a graphical display system may be configured to follow one or more rules or principles when defining, selecting, and/or managing various graphical object layers, such that the simplicity of a basic drawing space application may be combined with the flexibility and non-destructive manipulation capabilities of a more advanced layering application.

For example, in some embodiments, a graphical display system may be configured to generate any new graphical object in the top-most layer of a layer stack presented for display to a user. Additionally or alternatively, the system may be configured to determine whether to incorporate a new graphical object into a new layer or into a pre-existing layer based on the particular type of the new graphical object and/or based on the particular type of graphical object that may be provided by the current top-most layer. That is, different types of graphical objects may be handled differently by the layer management processes of a graphical display system. For example, any new non-drawing stroke graphical object may be created in a new layer and that new layer may be made the top-most layer in the layer stack. Moreover, unless the current top-most layer is a drawing stroke layer, any new drawing stroke graphical object may be created in a new layer and that new layer may be made the top-most layer. Therefore, in some embodiments, only if the current top-most layer is a drawing stroke layer, may a graphical display system be configured to create any new drawing stroke graphical object in that pre-existing current top-most layer.

As another example of how a graphical display system may be configured such that the simplicity of a basic drawing space application may be combined with the flexibility and non-destructive manipulation capabilities of a more advanced layering application, certain types of graphical object layers may be automatically or optionally provided with certain tools, while other types of graphical object layers may not be provided with those tools. For example, each layered image graphical object may be provided with one or more control points that may be manipulated for resizing and/or moving the image graphical object layer in various ways along a workspace. As another example, each layered image graphical object may be provided with a toolbar that may allow a user to manipulate the graphical object layer in various other ways (e.g., by moving the layer up or down in the stack of layers, by adding another graphical object into the layer, and/or by modifying one or more properties of the graphical object of the layer).

Patent Credits

Apple's patent applications were originally filed in Q1 2011 by inventors Conrad Carlen and Matthew Sarnoff and published today by the US Patent and Trademark Office. Once again it has to be said that patent graphics are only presented for the sake of conveying a concept and don't represent what Apple will actually unveil. This isn't a design patent. Anyways, you know that Scott Forstall and Apple's team of great user interface designers will actually deliver something very, very cool. It's the concept that's jaw dropping in scope and more so because so many thought that Apple was phasing out the Mac Pro. 

Perhaps these new graphic apps will require one of Apple's advanced mice (one, two). Only time will tell. At the end of the day, this news may be a bombshell to some: Boom! 

Note to Referring Sites: We ask that referring sites limit the use of our graphics to a maximum of two per report. Thank you for your cooperation.

One more thing: Another site is reporting that Apple's illuminated touchpad patent application published today is something new. Well, it's not. It's a continuation patent that dates back to 2006. We covered it way back in 2008. Being that this was never granted to Apple, it's likely that their patent application update relates only to patent claim tweaks and nothing ground breaking. The same is true for a continuation patent titled "Sticky Functionality." While another site is presenting this as new, it's in fact a 2002 continuation patent.   

Notice

Patently Apple presents a detailed summary of patent applications with associated graphics for journalistic news purposes as each such patent application is revealed by the U.S. Patent & Trade Office. Readers are cautioned that the full text of any patent application should be read in its entirety for full and accurate details. Revelations found in patent applications shouldn't be interpreted as rumor or fast-tracked according to rumor timetables. About Comments: Patently Apple reserves the right to post, dismiss or edit comments.

Side Note

On Tuesday we posted a report titled "Apple Wins Major Apple TV Patent Relating to Cable TV." We updated our report last night to include a news leak by the Wall Street Journal that Apple is in talks with some of the biggest U.S. Cable operators about letting consumers use an Apple device as a set-top-box for live televisions and other content. In light of Apple's granted patent, this was interesting news to say the least.

Sites Covering our Original Report

MacSurfer, Twitter, Facebook, Apple Investor News, Google Reader, Macnews, iPhone World Canada, Electronista, Mac Hash, MacDailyNews, Model Mayhem, Cult of Mac, MyApple Poland, iTopnews Germany, International Business Times and Techmeme.

iDownloadBlog, Macgasm, One More Thing Netherlands, iPhoneItalia Italy, AppAdvice, iJailbreak, ePHOTOzine, phones review, Macworld UK, MakeMac Indonesia, iPhoneDeVAR Arabic, Apfelpage Germany, Macworld-Brazil, Aberto ate de Madrugada Portugal, and Creativebits, and more.

Join the Conversation!  

The sites that we link to above offer you an avenue to make your comments about this report in other languages. These great community sites also provide our guests with varying takes on Apple's latest invention. Whether they're pro or con, you may find them to be interesting, fun or feisty. If you have the time, join the conversation!