Overview of Apple's Patent

Configurations using optical in-LCD sensing panels could provide transmission and/or reception of data. In one transmit/receive configuration, an optical in-LCD sensing panel could be used for communication. Data is transmitted by displaying a communication image encoding the data on the sensing panel, and data is received by capturing, with the EM sensors embedded in the panel, a communication image displayed in proximity to the panel. In another configuration, data is received by scanning an object using an optical in-LCD sensing panel. The motion of a handheld device including the panel could be determined by scanning a surface with the EM sensors at different times and comparing the corresponding scan images. A control signal based on the motion of the handheld device could be transmitted to an external device, for example, to control a mouse. In another configuration, scan images could be combined based on the motion to generate a combined scan image of a surface.

HP Introduced NFC Sharing for New Mobile Devices

During yesterday's WebOS "Think Beyond Event," HP introduced NFC Sharing that will allow WebOS devices to share data and URL's by simply tapping the devices together. HP is branding this feature as Touchstone.

Throughout 2010 Patently Apple recognized this technology as one of Apple's leading trends. In general we created an NFC Archive listing all of Apple's known patents on the subject and within this group we've reported on how Apple wants to implement NFC within the home, within the context of an iWallet in the coming years and much more. In all fairness, it should be noted that HP's technology is only coming to market "this summer" – and so there's plenty of time for Apple to actually beat HP to market if their upcoming iPad 2 implements some form of NFC based features.

Optical In-LCD Sensing Panel: NFC-Like Technology with a Twist

In Apple's patent FIG. 5 it describes an iPhone, noted as device 301, communicating (504) with device 601 (another iPhone) by displaying and/or capturing one or more communication images on the iPhone's display – just as HP showed during their keynote.

Communication images are displayed as visible images using visible light sub-pixels. The visible communication images are captured with the photodiodes.

Communication by displaying and capturing images with an optical in-LCD sensing panel could take a variety of forms. Communication may be one-way, for example, a first device transferring data, such as an electronic business card, to a second device. Communication may be bidirectional, for example, a first communication image displayed by a first device may be captured by the sensors of a second device, and vice versa over the course of a data transmission.

Communication may be encoded in a variety of different ways. For example, a communication image may be an image of text that could be captured and read directly by the recipient with minimal processing. Communication of an image of text may provide an easy way to communicate information between electronic devices without the need for particular protocols or predetermined coding schemes to be coordinated beforehand.

OCR, QR and Barcodes

In addition, the captured image of text may be processed using, for example, optical character recognition (OCR). In other forms, a communication image may be a coded visual structure, such as a one-dimensional barcode, a QR code, which is essentially a two-dimensional barcode, etc. Using QR codes for communication images may allow more flexibility in the positioning of the communicating devices, because QR codes could allow reading at different angles/orientations.

Apple's Twist

Apple's patent FIG. 3 illustrates a device that could be first and foremost an iPhone, but could technically be any iOS device with Wi-Fi, Bluetooth and/or USB.

The iPhone in Mouse Mode

Apple's patent FIG. 7 noted above illustrates that an iOS device could operate as a user input device similar to that of a computer mouse. Before placing the iOS device face-down, a user selects a "mouse" mode via a GUI function on the iOS device (see flowchart below). This would almost be suggesting that the backside of an iOS device would be multi-touch sensitive. That's interesting.

It is noted that, unlike a conventional computer mouse, the position data generated by the foregoing example operation can include information about the rotational motion of device 301, in addition to information about the translational motion of the device. The rotational motion data may be useful in applications such as, for example, computer games; Something that was hinted at in another patent a year ago.

The iPhone in Scanner Mode

In contrast to conventional handheld scanners, the foregoing example scanner mode may not require the user to maintain a constant velocity when moving the device across the surface to be scanned because, for example, the stitching algorithm can determine the motion of device 301 and compensate for the motion when stitching together the images. In addition, conventional handheld scanners typically require the user to move the scanner in a straight line across a page, and to refrain from rotating the scanner. In contrast, in the foregoing example, the user would be free to move the device freely around on the page in practically any manner, e.g., the device could be rotated, move along an irregular path, moved at different speeds and directions, and the device would be able to compensate for the regular motion and to generate a scanned image as the user "paints" the page with the device.

Apple credits their design manager Jeffrey Bernstein and Brian Lynch as the inventors of this patent application.

Update 1:30 PM EST: Another source states that Bernstein now leads the Human Interface Device Software Prototype group at Apple. While we don't know what secret stuff he's working on today, we see in the video below that he was involved with advanced multitouch on large surfaces as far back as 2006. Now that's interesting. 

Other Noteworthy Patent Applications Published Today

Thinner Display Techniques & Multi-Touch iMac: In another patent application published today that is related to a divisional of U.S. patent application serial number 12/351, 767 we see that Apple is working on a way of manufacturing even thinner displays. They've produced a method of sandwiching two displays together in order to utilize existing manufacturing equipment and then separate the two panes after processing.  The thinner displays could be used on all iOS devices as well as MacBooks and the iMac. In fact, Apple adds this note to the desktop display: "The desktop computer may include a virtual keyboard which incorporates a SITO/DITO panel fabricated according to the embodiments of the invention." Proof that once again Apple is thinking of adding a multi-touch component to an iMac with a tiltable display.

Noise Cancellation: Four new Apple patent applications published today (20110033065, 20110033056, 20110022060 and 20110033064) relate to noise cancellation techniques that improve headset-based audio communications using devices like the iPhone or any of those presented below. The date of the original filing being 2009 would likely suggest that this has already been implemented in the iPhone 4. 

24 Bit Audio Coming to iTunes: While on the topic of audio, it should be pointed out that in the HP "Think Beyond" event link that we provided for you earlier in our report, there was an interesting admission by Universal. Although they praise HP lavishly for having the only notebook in the world that could finally deliver 24 bit quality audio, there was an admission in between the 1:10 to 1:12 mark of the event whereby we learn that Apple is working on upgrading their iTunes library from 16 to 24 bit. For audiophiles - that should be music to your ears.

iTunes for Micro Devices Explored: One last patent application worthy of a mention is that of 20110035222. Apple states that as devices get smaller there's increased difficulties in providing adequate user interfaces. Apple's shuffle tried to go without a user interface and based itself on audio commands or VoiceOver alone. That didn't work out too well and so Apple went back to adding a control button in addition to VoiceOver. Yet, Apple still feels that VoiceOver could be enhanced so that iTunes could be navigated by external factors. 

Apple is considering adding extra sensors to the Nike + iPod product, for example, so that as your workout increases in intensity so will the tempo of the music that it could choose for you in shuffle mode. Some future tiny devices will include a camera and based on the light and humidity, it'll choose  your tunes. That's a bit of a stretch. And then Apple thinks that metadata could go one step further so that artists like Bono or the Edge of U2 for example, could introduce their songs audibly if you're in shuffle mode on a button free device. This idea doesn't interest me much, but the question is, does it interest you?   

Notice: Patently Apple presents only a brief summary of patents with associated graphic(s) for journalistic news purposes as each such patent application is revealed by the U.S. Patent & Trade Office. Readers are cautioned that the full text of any patent application should be read in its entirety for further details. Patents shouldn't be digested as rumors or fast-tracked according to rumor time tables. Apple patents represent true research that could lead to future products and should be understood in that light. About Comments: Patently Apple reserves the right to post, dismiss or edit comments.