Apple Wins Patents for XGrid, iCal, Parental Controls & Keyboard
Damages in Mirror Worlds vs. Apple Case Could Top 1.8 Billion

Apple Revisits their Future Visual Expander Feature

1A - Cover Graphic for July 20 Granted Patents - Multitouch Visual Expander 
The US Patent and Trademark Office has published a handful of Apple's patent applications today that mainly consists of continuation patents. Interestingly, within this group, we see that Apple has resubmitted and/or expanded upon their "Visual Expander" patent. The reason that this is interesting is that while the patent was resubmitted back in late June, Apple had actually been granted a patent for it in July. This advanced multi-touch feature which has yet to surface on any portable device, may in fact be one of the advances coming to a future iPad and/or iMac Touch. This report revisits Apple's "Visual Expander" patent in respect to multi-touch features for anyone who missed out on it on the first go-around while touching on Apple's "Back to the Mac" event scheduled for next week.  


Future Multi-Touch Feature: The Visual Expander


Apple was granted a strategically important multitouch patent back in July 2010 regarding a new multi-touch feature identified as a "Visual Expander." The funny thing is that Apple resubmitted the patent to the USPTO for some unknown reason. While it may have been to simply make a last minute tweak, the point remains that the patent is considered to be very active. And in light of Apple's announcement about a "Back to the Mac" event – anything about a future OS X feature becomes of interest. Whether this feature is for a future iteration of iOS or OS X is unknown at this time. For now, we revisit the advantages of Apple's future "Visual Expander" feature:


As you know, touch screens offer ease and versatility of operation and allow a user to make selections and move a cursor by simply touching the display screen via a finger (or stylus). Apple's granted patent states that "While touchscreens generally work well, they are difficult to use when features such as buttons, web page links, or UI controls presented on the touch screen display are too small for finger activation, i.e., the finger is too large relative to the button, link or UI controls. Furthermore, the features typically don't provide the user with any feedback indicating that a finger is located on the feature. This may make it difficult to determine what feature is being selected. As a result, the user may incorrectly select a feature. For example, in web browsing, users have no control over the size of the link or button presented by a website."


Apple's solution is to introduce us to a major new multitouch feature called the Visual Expander. Isn't that just an onscreen magnifier, you might ask. No. Apple explains the difference between the two in this way: "The virtual magnifying glass magnifies the GUI in the area of the magnifying glass, i.e. similarly to moving a magnifying glass over a printed piece of paper. The magnifying glass allows the user to traverse through the GUI so that the user could read smaller text. While virtual magnifying glasses work well, they are limited. For example, they typically don't allow features to be manipulated or selected inside the magnified area. Furthermore, they may not allow text editing therein."


Apple's granted patent notes that Apple's OS X is designed to magnify the dock including the icons when the cursor is moved over the docks icons. Yet the patent notes the limitation of this feature by stating that "While this works well, the feature has no control over the content presented on the remainder of the screen, i.e., the remaining portions of the screen do not magnify when the cursor is positioned thereover. Furthermore, this particular feature only works on the main system page. It does not work in programs or applications or even web pages."


 The Expanded State


Apple's granted patent FIGS. 9 and 10 illustrated below are diagrams showing GUI 400 in an unexpanded and expanded state. FIG. 9 is a side view and FIG. 10 is a top view. As shown, the expanded portion 402 is enlarged, magnified and raised (e.g., pushed up) relative to the remaining portions 404 of the graphical information. Although the expanded portion appears to be out of plane relative to the remaining portions, it should be noted that the expanded portion is actually in plane with the remaining portions. Visual techniques are used to make it appear as if it is raised even though it is displayed in the same plane.


2 - Apple Granted Patent - Visual Expander - Expanded State - July 20, 2010 

In the illustrated embodiment, the expanded portion includes a plateau region 406 and a transition region 408. The plateau region is configured to display the target area 410 (noted as the yellow circle) in a larger and magnified state – which roughly translates to a magnification of 3 times that of the target area 410. This will allow most adults, especially those with larger fingers, the ability to use touch areas and controls such as hyperlinks with ease, unlike some controls on the iPhone today.


Yet until we see this feature, it's difficult to say whether or not Apple is referring to their patent pending high end 3D OS interface or not. There's a video presented in Patently Apple's report on this proposed 3D OS that clearly demonstrates how the eye could be fooled into perceiving depth that is limited to the display. This appears to be how Apple is describing their new Visual Expander feature.


Magnifying and Using Hyperlinks 


Note: Here's a quick tip to help you understand the following graphic. A portion of the patent figure 12M shown below is actually jetting out of the graphic as opposed to being dragged, as the graphic first appears at a glance.


Apple's patent describes FIG. 12M as follows: "the finger 514 is moved from the title to a link positioned within the window 520. Similar to the buttons and title, the portions of the link in the plateau are fully enlarged and magnified while the portions in the transition region are distorted and further the portions outside the expansion are of normal size and magnification. As shown in FIG. 12N, the user exerts a greater pressure on the touchscreen while the link is in its expanded form. This increased pressure is recognized as a touch event, and the element associated with link is launched or opened (although not shown)."


3 - Apple Granted Patent for Visual Expander - fig 12m, n magifying & using a hyperlink 

Magnifying Specific Zones


As shown in FIG. 13C, after the finger 514 dwells for some time over the heading, the heading is expanded. In this illustration, the heading is presented in an enlarged and magnified state while the remaining portions are not. As shown in FIG. 13D, when the finger 514 maintains contact with the touchscreen and is moved over a different GUI object as for example the field of the window, the field is expanded. In this illustration, the field is presented in an enlarged and magnified state while the remaining portions are not (including the heading).


4 - Apple Granted patent - visual expander - magnifying specific zones 

Note: In the magnifying zones example illustrated above, the horizontal bar noted as "window" is actually jetting out in FIG. 13C. The same goes for the field of the window jetting out in FIG. 13D. With a 2D interface, you can't really see this without knowledge of what is actually being conveyed in the patent.


Magnifying Page Elements or Text


As shown in FIG. 12I, as the finger moves away from the buttons, it moves over an inside edge of the window 520 thereby causing the inside edge to be expanded as well as text in FIG. 12J.


5 - Apple Granted Patent - Visual Expander - magnifying page elements or Text 

Apple credits Peter Kennedy as the sole inventor of Patent Application 20100259500 which was resubmitted in late June of this year. The original filing was actually made in Q3 2004 and published in July 2010.


Apple Event: Back to the Mac


6 - Back to the Mac event oct 20, 2010 

Yesterday Apple announced their "Back to the Mac" event that is scheduled for October 20, 2010. The logo that is shown on their invitation is rather interesting considering that it appears to be representing a likened 3D image that could very well be hinting at a future 3D version of OS X.


Apple has a number of 3D related patents that would support such a move. Interestingly, Gizmodo added a graphic to their report yesterday that is noted above along with an unknown quote supposedly from Steve Jobs that states: We've had tremendous, unbelievable success with multi-touch. And that's why starting next year, we're bringing it to the Mac." If true, it would validate Apple's recently published iMac Touch patent that Macites globally applauded back in August.


I for one will greatly appreciate clarification of their operating system roadmap. I'm close to pulling the trigger on a new iMac but have one eye looking to 2011 for two reasons: 1) Intel's Sandy Bridge or AMD's Fusion, and 2) an advanced next generation OS.


The two noted issues are actually interrelated. We discussed that scenario in two of our reports titled " The Next OS Revolution Countdown Begins" and "Intel's CES Keynote 2010, Apple and iLife 3D." In fact, a third report of ours touched on the subject regarding the advent of Heterogenous Computing which takes advantage of Apple's OpenCL at the chip level.


This is what I'm hoping that Apple will clarify to some degree next week. Is Apple bringing multi-touch to the iMac? Will 3D be coming to OS X? Will we need to have a Mac with one of one of the next generation architectures noted above to be able run these features? Will Apple be shifting to AMD? With Otellini taking another crack at Apple, anything is possible.


That's what will make next week's big event so much fun. We may finally get some meaty answers to some really important questions. And today's patent only reminds us all once again that Apple is continually making huge strides in the advancement of multi-touch features that may one day land on our Macs.


At the end of the day, what do you think will unfold next week? You've almost had a full day to think about it – so what's on your mind today? Send in your predictions in our comment area below. We'd love to hear what you're thinking.




Notice: Patently Apple presents only a brief summary of patents with associated graphic(s) for journalistic news purposes as each such patent application and/or Granted Patent is revealed by the U.S. Patent & Trade Office. Readers are cautioned that the full text of any patent application and/or Issued Patent should be read in its entirety for further details. About Comments: Patently Apple reserves the right to post, dismiss or edit comments.


it looks as though a pivoting iMac with 3D desktop may be on the cards along with X-Lion.

OS X 10.7: Will be named Lion. I'd like to see the kernel become 64 bit all over. Snow Leopard omitted the large Phoenix like groundwork on QuickTime. It got gutted, and we haven't seen it fleshed out yet - AirPlay is one step in that. We'll see Core OS increase likely - bringing iOS and mac OS closer. Just look at - they're making their site multitouch and finger friendly.

The Mac Book Air: something bigger could be at work here. Perhaps more involved SSD usage. It's not just an SSD drive but more like SSD integration - as Flash is for an iPod. So this points to TRIM support for starters. I'd imagine they'll push this. And i'll bet that Optibay conversions for current MBP holders will increase too!

Other Possilbe Things: How about 64bit iTunes? Possibly iLife 11 64bit. I'd love an iWork AirPlay update to Keynote. Apple could talk Light Peak USB3 if they wanted to show where things were going. Would make MBA one helll of a machine in comparison to current old version.

We haven't heard about the mapping companies bought for a while - Apple could be adding some more integration into the key OS apps (mail contacts safari etc). the Google integration of phone to Chrome surely will get countered.

Pseudo 3D perspective in 10.7. Will iSight resolution get a bump to HD? Apple bought a facial recognition (Polar Rose) company and could use their technology to recognise eye position and head movement or "Head Tracking."

Multitouch gets an added bonus when using 3D OS - a natural 2 finger gesture could be used to not only rotate around the z axis, but also in the x and y axes objects.

If they add this, then they could roll it into games. That would be one awesome addition if playing on an iMac, or via a screen that had a compatible cam attached that could use it. Castles in the sky?

Apple wants to "roar" into 2011 like a "Lion" (next OS X Cat).

So what am I hoping for? Well I'm hoping for a 23 or 24" LED Display upgrade for the bottom price tier iMacs. I'm hoping for the i5 Quads to dip down to price tiers 2 and 3. I'd like to see Apple drop the top end imac in favor of a mini tower pushing games, games and games. That won't happen but this is a wish list for Santa, right?

For MacBooks. I'd like to see a switch to AMD's new cool-running Fusion chips. This new chip is due to arrive any day now on the market.

The switch to Fusion would leapfrog Sandy Bridge's big day at CES and piss on Intel's parade.

On the software front, I'd like to see a new graphics app for iWork. Nothing too fancy, just a good round set of tools for the basics.

I think it's just a way to say, here's a peek at something new coming to the next version of Leopard without really saying anything whatsoever about 3D or touch. Apple underwhelms us when it comes to OSX. Will next week change things? Let's hope because I'm tired of everything being focused on iOS. Some of us still want our desktops to wow us once again!

Sometime between 2011 and 2012 we're going to see a huge leap forward on many fronts. USB 3, maybe Light Peak I/O, 8 core rockets on 22nm silicon, LTE at 10X 3G speeds and the OS as we know it is long overdue for an overhaul as one of your reports about the coming revolution points out. So I think that Apple is timing this discussion well. A way to pump us up. Could you imagine if Apple actually put out a beta for us to test out? No. But I could dream can't I?

I wouldn't read too much into Apple's event graphics. I thought the last one meant they were going to launch a new paint program because of the splash of paint event graphic. It didn't mean that at all. So, it's fun to play with it like trying to figure out the deeper meaning behind a Beatles album cover (yes it dates me)but it's just marketing baloney.

Ya, I agree. I think the graphic is conveying a shift to 3D. But it could also just be Apple saying that they're turning a new leaf on OSX. Whatever that means i don't know..

I'd love to see Apple dump Intel. They've been pissed ever since Apple snubbed Atom for the iPhone. So bring on AMD's fusion processors. With OpenCL Apple will have it buzzing just as good as any Intel cpu. Bring it on!

No brainer, multitouch imac coming in 2011. we just saw the patent on it.

My vote is for a 3D version of OS X.

The comments to this entry are closed.