An Overview of General Haptics

Apple states that traditional tactile feedback systems provide a user with the ability to interact with a subsystem through touch or contact. Haptic systems facilitate these tactile interactions by using actuators, sensors, or both. Haptic systems are commonly used in robotics, video games (e.g., "rumbling" as used in some video game controllers), and other interactive systems which allow interaction via touch. An array of haptic elements is commonly used to provide touchscreen technology to users. Well, Apple thinks that it's time to advance haptics to an all new level.

New Advanced Haptics: Layered Haptic Controls

Apple's latest invention relates to systems and methods for providing layered haptic controls. Haptic systems may be used for actuation such as vibration, shape change (e.g., contouring a flat surface), or other suitable actuations or combination of actuations which may provide tactile feedback to a user.

Haptic systems may also be used for sensing stimuli such as, for example, contact on a display screen, patterns of contact on a screen, shape changes, physical changes of a system or component, or other suitable stimuli or combinations of stimuli which may be received.

Haptic systems may sense particular stimuli, change one or more characteristics of a shape change element, or both. Haptic systems may perform sensing functions and actuating functions at the same time. In some embodiments, haptic systems may be coupled to a display screen, audio system, device software, device hardware or other system to provide for any combination of tactile, visual, and audio interactions.

Actuation may occur, in some embodiments, substantially normal to a substantially planar surface, which may allow for three dimensional contouring of the planar surface.

Tiered Haptic Responses & Elastic Screen Interfaces

Apple states that their new invention may adopt a tiered haptic response approach in which one or more arrays of shape change elements may provide tactile interaction via an elastic screen interface. A suitable display screen may be included in the elastic screen interface. For example, stacked, planar arrays of piezoelectric elements may be used to provide variable actuation, sensing, or both. In some arrangements, each array may include, for example, piezoelectric elements of a particular size, providing multi-scale control in actuation and sensing.

In some arrangements, a particular type of shape change element may be included in each array. In some embodiments, different types of shape change elements may be included within a particular array. Shape change elements may provide any type of actuation such as, for example, vibration, net displacement, bending, deforming, any other suitable actuation mode, or any suitable combinations thereof.

For example, a stacked haptic arrangement may include a particular array, which may include electromechanical elements (e.g., solenoids to create a magnetic field). Another array in the stacked haptic arrangement may include electroactive polymer elements. The shape change elements of the arrays of this illustrative stacked haptic arrangement may be controlled by any suitable control system, which may include circuitry for activating electromechanical actuators, electroactive polymers, or both.

Stacked arrays may be used to create a contoured screen surface such as, for example, contour maps, shaped buttons, moving contours or shapes, or other surfaces with multi-scale features.

Forming Raised Elements on a Touch Display

Apple's patent FIG. 9 below illustrates a smartphone which includes a touch display 902 with integrated haptic systems, soft commands 908 and contoured screen feature 950. The smartphone may use one or more piezoelectric elements to form a raised button on the display, as seen in patent point # 908.

Further to FIG. 9, we see that contoured surfaces may be formed on the display using one or more shape change elements. For example, the contoured screen feature 950 may be a geographical contour map, in which portions of the display are raised in accordance with corresponding elevation data. In some embodiments, any suitable contour, deformation, topology, or other suitable surface feature, or combination of features, may be formed on the display such as, for example, raised buttons, raised arrows, depressions, patterns, or moving features (e.g., cursors). In some embodiments, tactile stimuli to display 902 may be detected by user device 900. For example, user device 900 may detect that a user has touched a particular surface feature on display 902, and may execute a particular task associated with the surface feature.

Elastic Sheets & Shape Change Element Arrays

Apple's patent FIG. 10 shows an illustrative top plan view of tiered haptic system 1000. Shown in FIG. 11 is illustrative cross-sectional view 1100 of the elements of FIG. 10. The tiered haptic system may include display screen 1002, elastic sheets 1110, 1112 and 1114, substrate 1120, shape change element arrays 1130, 1140 and 1150, any other suitable components or any suitable combinations thereof. The elastic screen sheet may provide a visual, tactile, or both, interface with which a user may interact.

Apple further notes that the elastic sheets may include thin sheets of any suitable materials such as, for example, elastomers (e.g., rubber, thermoplastics), polyurethane, polypropylene, polyethylene, polystyrene, polyester, any other suitable elastic material, or any suitable combinations thereof.

Tiered Haptics Could be applied to Flexible OLED Displays Including Video Glasses

According to Apple, the display screen noted in FIG. 10 may be any suitable type of display screen which may allow haptic interaction. For example, in some embodiments, the display screen may be a flexible organic light emitting diode screen (OLED), a graphene sheet, or any other suitable flexible display screen or any suitable combinations thereof. In some embodiments, display screen 1002 may be substantially inflexible, and may not form contoured screen surfaces. In some embodiments, display screen 1002 may include a protective cover such as, for example, clear plastic (e.g., Lexan) or any other suitable substantially transparent material. Common usages of Lexan include space and sports helmets, clear high-performance windshields and aircraft canopies, and bullet-resistant windows.

The mention of Lexan is very interesting considering that Apple's invention notes that their tiered haptic system could be applied to a head-mounted Display or Video Glasses. Additonally, the new haptics system could also apply to traditional devices such as a MacBook, an iPad, an iPhone, an iPod touch, a computer monitor or other devices like a television and video projector.

Apple's FIG. 20 shows an illustrative partial cross-sectional view of tiered-embedded haptic system 2000 which may include flexible display screen 2002 (e.g., OLED display screen). In some embodiments, the display screen may include a protective cover of any suitable substantially transparent material. In some embodiments, the display screen may be segmented or partitioned such that the display screen includes one or more smaller screens. Theoretically, Apple could apply this to their video glasses where only a portion of display lens would present content in a smaller window.

Could Apply to Monochrome e-Ink Type of Displays and Other Kinds of Screen Contouring

Additionally, Apple states that the display screen may display any combination of graphics, images, video, tables (e.g., iTunes listings), text, contacts (e.g., phone list), any other type of visual information or graphics, or any suitable combination thereof. Interestingly Apple states that the display could also be monochrome (e.g., black and white), color or grayscale and could be segmented or partitioned such that the display screen includes one or more smaller screens, which may allow for further screen contouring. The inclusion of monochrome displays is interesting in that it could translate to an e-ink type of display and or hybrid display which Apple has is working on. But when Apple adds "further screen contouring," it very much reminds me of a recent Microsoft patent that presents a possible future smartphone that has a contoured e-ink back display.

Other Interesting Notes

Apple states that "a control system of a tiered haptic system may determine the amount of pressure, force, displacement, or other physical response associated with the user stimuli. For example, a tiered haptic system may distinguish between relatively light contact and a relatively heavy contact on the screen surface. In some embodiments, a tiered haptic system may perform particular tasks depending on the physical response of the stimuli." This could apply to paint and drawing apps.

In some embodiments, haptic systems may map graphics onto contoured screen features. For example, a particular graphic (e.g., video clip) may be displayed on the display screen of a particular user device. The display screen may then undergo deformation (e.g., contouring) and use image processing techniques to alter the displayed graphic in response to the screen contouring. Hmm, I wonder if this could apply to video games to give controls a textural feel and depth.

In another example, a contour elevation map with text annotations may be displayed on a display screen of a user device. A user device may contour the display screen, by activating one or more shape change elements, to correspond to the elevation at a particular region on the contour map. The user device may alter the display to compensate for the contoured surface by stretching, compressing, moving, rotating, warping, curving or otherwise altering the displayed graphic (e.g., contour map with text annotations).

In a further example the haptic system could provide image processing techniques such as, for example, Euclidian transformations (e.g., translation, rotation), image morphing, feature detection, stereoscopy (e.g., 3-D imaging) and rendering (e.g., shading, texture mapping).

In some embodiments, regenerative power management may prolong battery life by recovering energy supplied by a user or other tactile stimulus.

Apple's patent FIG. 16 below shows us an illustrative cross-sectional view of a tiered haptic system with a flat display receiving tactile stimuli. Patent FIG. 17 shows us an illustrative cross-sectional view of a tiered haptic system with contoured display receiving tactile stimuli.

Apple's patent application was originally filed in Q4 2010 and only made public today by the US Patent and Trademark Office. Update June 26, 2012: Check out the note below to Russell Maschmeyer, one of the inventors of this patent. The other inventor of this patent is noted as Gordon Cameron.

As a general note about patent figure 9 presenting hard keys. Keep in mind that Apple talks about the invention relating to a video projector, TV and other devices. So the inclusion of hard keys is simply acknowledging that the invention will relate to many devices of which some may contain hard key buttons. Patent Figures are "conceptual drawings" and aren't presented as a design patent which will be filed at a later date.

Update May 4, 2012: The video below covers the basics of Haptics that may be of interest to some. 

Notice

Patently Apple presents a detailed summary of patent applications with associated graphics for journalistic news purposes as each such patent application is revealed by the U.S. Patent & Trade Office. Readers are cautioned that the full text of any patent application should be read in its entirety for full and accurate details. Revelations found in patent applications shouldn't be interpreted as rumor or fast-tracked according to rumor timetables. Apple's patent applications have provided the Mac community with a clear heads-up on some of Apple's greatest product trends including the iPod, iPhone, iPad, iOS cameras, LED displays, iCloud services for iTunes and more. About Comments: Patently Apple reserves the right to post, dismiss or edit comments.

 Sites Covering our Original Report

MacSurfer, Twitter, Facebook, Reddit, Megite, Apple Investor News, Google Reader, Macnews, iPhone World Canada, MarketWatch, DRKOT Czech Republic, StockTwits, Apple URLs, Silicon Investor, Apple Enthusiast, Real Clear Technology, 9to5Mac, iDownloadBlog, BGR, Yahoo! News, App Advice, MacUser Greece, Digitoday Finland,  iPhone Stuffs4U, iPhone3GSystem France, MacWorld UK Mobile and BGR Germany.

iPhoneclub Netherlands, PCPOP China, It-Viikko Finland,  iGeneration France, Gadget Report Romania, iApp Taiwan, WeiPhone China, iGuides Russia, One More Thing Netherlands, Macmais Brazil, Hardware.Info Netherlands, Seeking Alpha, TechSpot, Macerkopf Germany, GigaOM, Wall St. Cheat Sheet, Gadget Greek Greece, Kurzweil, MacDailyNews, Techmeme, PCWorld, MacWorld UK, WebNews Italy, Computerworld Japan, and more.

Congratulations to Jessica & Russ! 

 Why? Because Russell Maschmeyer is one of the inventors of this patent and he's getting married to Jessica. Here's their story.

Join in the Conversation! 

The sites that we link to above offer you an avenue to make your comments about this report in other languages. These great community sites also provide our guests with varying takes on Apple's latest invention. Whether they're pro or con, you may find them to be interesting, fun or feisty. If you have the time, join in!