Multi-Touch Skins for Sporting Equipment

In today's report, we delve into a number of examples covering Apple's next generation Three Dimensional Multi-Touch Skin technology. While we look at a wide array of ideas, we acknowledge that this is only the tip of the iceberg. This may be Apple's initial patent on the topic but by no means the last. We begin with its application in sporting equipment.

In Apple's patent FIGS. 1A and 1B shown below we see an exemplary illustrated embodiment of a single multi-touch skin spanning three dimensions of an object which in this case is a tennis racket handle. Alternatives, for example, could be a golf club or hockey stick handle. The multi-touch skin is wrapped around the entire circumference of the handle and extends along the length of the handle and in this manner spans three dimensions of the object enabling multi-touch input as a player grasps the handle during play.

The dimensions of multi-touch skin will vary depending on the size of the object and how the player grasps the object. For example, the dimensions of multi-touch skin for a dart would be much smaller than a multi-touch skin for a tennis racket, golf club or hockey stick, given the dart's size and how it's held by only two fingers.

The shape of a multi-touch skin will also vary depending on how the player grasps the intended object. For example, the multi-touch skin could be positioned in one or more patches spanning three dimensions of an object if the object, such as a gun, is held with spread fingers.

A Multi-touch skin could be configured to recognize multiple touch events that occur at different locations on the surface of an object at the same time and monitor the operation of any given object. This could be achieved through a multi-touch sensing arrangement. In the exemplary arrangement illustrated in FIG. 1A, surface 15 of the multi-touch skin is divided into several independent and spatially distinct sensing nodes, such as nodes 16, 18, 20 and 22. The nodes are typically hidden from view and are dispersed about the surface of the object. Each node represents a different position on the surface and is capable of generating a signal at the same time as other nodes. In the simplest case, a signal is produced each time an object is placed over a node. When, for example, the tennis racket handle is grasped by several fingers and a palm, multiple signals are generated corresponding to the nodes touched by the surface contact.

Sports Training

Once the coordinates of the touch are found, they could be analyzed. For example, the coordinates could be analyzed to monitor how the object is being operated and provide feedback. In the case of a tennis racket or a golf club, for example, the coordinates of the player's grasp of the handle could be detected. The coordinates could then be analyzed to determine if the player has correctly grasped the handle for a forehand stroke or a drive. The analysis may include obtaining other information, such as the orientation of the racket face or club head in relation to the grasp, to determine whether the player has correctly positioned his hand or hands on the handle.

Based on the analysis, an action could be performed. For example, a display, such as display 60 in FIG. 2, could be used to notify the player that he or she has correctly grasped the tennis racket or golf club for the stroke to be performed. The display, for example, could display a green light for a proper grasp and a red light for an incorrect grasp. If the grasp is incorrectly positioned on the handle, the display could further direct the player to correct the grasp. For example, the display could direct the player to move his or her index finger or thumb in a given direction.

In block 84 of FIG. 3 noted above, we see the process flow to block 86 where the current image or signal could be compared to a past image or signal. This is done in order to determine if there's been a change in pressure, location, direction, speed and acceleration for each object on the multi-touch skin. This information could then be subsequently used to perform an action as indicated in block 88.

For example, if the grasp was incorrectly positioned on the handle, the current image could be compared to the past image to determine whether the player correctly followed the display's instruction to move an index finger or thumb in a given direction. As another example, the current image could be compared to the past image to determine whether the player is correctly positioning his or her grasp during a stroke or drive. The display could notify the player if his or her grasp is positioned correctly during the stroke or drive and provide direction. Beyond sporting equipment, we now move on to a more consumer oriented device: a television remote.

Multi-Touch Skin for a TV Remote

Now onto the fun stuff starting with a multi-touch skin for a TV remote – you know – for Apple's secret hobby.

It could be appreciated that two or more two-dimensional multi-touch skins spanning three dimensions of an object could be used in place of a single multi-touch skin spanning three dimensions of the object. In patent FIG. 5 we see a conceptual yet exemplary embodiment of a future TV remote control (200). The remote control could have a power button (210) and additional buttons (214), such as a numeric pad, for its operations.

The placement of one or more multi-touch skins spanning three dimensions of an object, such as the remote control, enables gestures for operating the object. For example, the movement of the user's thumb across multi-touch skin 240 could be used to control the selection of a channel. A downward movement of a thumb can numerically decrease the channel number, while an upward movement of the thumb could numerically increase the channel number. Similarly, the movement of one or more fingers along multi-touch skin 240 could be used to reduce or increase volume. Thus, a user could grasp the remote control and simultaneously select the channel and volume through multi-touch gestures on skins 240 and 245. Whether performed simultaneously or not, these gestures could be tracked as multi-touch inputs. Hmm, where's the button that could start my popcorn machine?

Touch-Skins for a Steering Wheel

In Apple's patent FIG. 10 below we see another set of concepts that illustrate the broad applicability of the present invention. Patent FIG. 10 illustrates a steering wheel that could operate either as an actual or virtual vehicle steering wheel. The steering wheel has two multi-touch skins 410 and 420. Multi-touch skin 410 is positioned in the right upper quadrant of steering wheel while multi-touch skin 420 is positioned in the left upper quadrant of the steering wheel. Each multi-touch skin is wrapped around the entire circumference of the steering wheel to span three dimensions of the steering wheel.

When a user operates the steering wheel, they place their right and left hands on zones 410 and 420. Each skin zone thus receives a multi-touch input, and each skin outputs corresponding data. The data from each skin could be combined and analyzed by a processor. For vehicle training purposes, zones 410 and 420 represent the classic "10 and 2 o'clock" positions. As described above with respect to FIGS. 2 and 3, a current image and a past image could be compared to determine if the user has properly followed the notification and provide feedback if necessary.

Radio Controls: In more complex actions, multi-touch skins 410 and 420 could also be used to operate the vehicle relating to steering wheel 400 via gestural inputs. For example, the tapping of an index finger on multi-touch skin 410 could also be used to perform channel selection of a radio in the vehicle. Each tap could correspond to a selection of the next available channel. The tapping of an index finger and pinky finger on multi-touch skin 420 could be used to perform volume selection of the radio. Each tap of an index finger could, for example, correspond to a step to increase in volume, while a tap of a pinky finger could correspond to a step decrease in volume. In this way, the user could simultaneously change channels with one hand and control volume with the other hand, without having to focus his attention on the vehicle's radio.

Wiper Controls: Multi-touch skins 410 and 420 could also be used together to configure the operation of the vehicle (e.g., its steering wheel) and another skin. For example, the tapping of two fingers on multi-touch skin 410 could be used to change the mode for multi-touch skin 420 from volume selection for the radio to wiper speed selection. The tapping of an index finger on the multi-touch skin would then correspond to a step to increase the wiper speed while the tapping of a pinky finger would then correspond to a step to decrease the wiper speed.

Apple's patent also states that "any form of velocity or steering control in a device, such as joystick or steering wheel, could be achieved with multi-touch skin."

Introducing, the Portable Multi-Touch Skin 

In Apple's patent FIG. 11 noted above, we see another dimension of this invention: one whereby the multi-touch skin is actually portable. The portable multi-touch skin 510 is a flex circuit to which electrodes, traces and sensing circuits are attached. The flex circuit could then be temporarily placed or stretched over an object, such as object 500 illustrated above. Multi-touch skin 510 could be operatively coupled to a device, such as device 51 in FIG. 2, in order for the user to provide setting information relating to the object to be wrapped. For example, the setting information could indicate that the object is a guitar neck or a globe. The setting information could also provide information relating to the placement of the multi-touch skin vis-a-vis object 500.

Once the setting information is established, the user could input multi-touch events to operate the device. For example, the user could perform chords on the guitar fret in conjunction with instructional software shown on a display. The player's chords on the multi-touch skin could be tracked and then analyzed by the instructional software to determine if the player is properly performing the chords. That would work nicely with Apple's GarageBand '11 iLife app for Guitar Lessons. Hmm, theoretically future multi-touch skins might be able to help control auxiliary equipment like specialized guitar peddles wirelessly with different taps to switch to different effects.

To Infinity and Beyond!

Apple's invention, once understood, could go in many directions. To show you the diversity of the portable multi-touch skin, Apple briefly describes a user having the ability to touch two countries on a geographical globe and obtain distance information between the two countries from a corresponding display.

Yet beyond that, Apple notes that their technology could be applied to, without limitation, video game input devices, toys, musical instruments, computing and peripheral devices, medical devices, clothes and consumer electronic devices.

Apple states that the claims of the patent are to be understood to include what is specifically illustrated and described above, what is conceptually equivalent, and what could be obviously substituted. The invention could apply to future MacBooks, desktops, iPods, iPhones or other digital devices. 

If Apple ever decides to license this technology to third party developers, we'd definitely see a mini three dimensional multi-touch skin related revolution. But that's getting a little ahead of the game. For now, we could enthusiastically state that where Apple could go with this technology is almost limitless. While the concepts noted above may sound a little crazy today, so did the idea of carrying your entire music collection in your pocket a decade ago. That's what the Crazy Ones in Cupertino do: Crazy things that change our digital world!

Apple credits Steve Hotelling and Wayne Westerman (behind many of Apple's crucial touch patents. Wayne came over from Apple's FingerWorks acquisition) as the inventors of this patent application which was originally filed about 18 months after the iPhone first came to market in the summer of 2007.

Notice: Patently Apple presents only a brief summary of patents with associated graphic(s) for journalistic news purposes as each such patent application is revealed by the U.S. Patent & Trade Office. Readers are cautioned that the full text of any patent application should be read in its entirety for further details. Patents shouldn't be digested as rumors or fast-tracked according to rumor time tables. Apple patents represent true research that could lead to future products and should be understood in that light. About Comments: Patently Apple reserves the right to post, dismiss or edit comments.

Community Sites Covering our Original Report

MacSurfer, MacDailyNews, Google Reader, Apple Investor, Zaone Romania, Scoop.it, iPhoneros Spanish, Melamorsicata Italy, Macity Italy, macnews Germany, The Mac Observer, NUzakelijk Netherlands and Fast Company.