Granted Patent: Lifestream Annotation Method and System

Apple's newly granted patent covers their invention relating to online social media. More particularly, the present invention relates to collecting and enriching information. Apple's acquired patent has at least two surprises covering a possible new device in the form of lifestreaming smart glasses and the use of Xen Hypervisor.

The Patent's Background

Today, a tremendous number of online services and applications are available. The existing online services are related to virtually every aspect of life, including work, relationships, health, entertainment, news, etc. Access to these online services typically only requires an Internet connection. However, the abundance of services also causes information overload. As a result, disorganized information is floating around many users' lives and brains. Instead of positively influencing our lives, the services become cumbersome and are often neglected. In particular, the advent of social media has made it easier to share information with more people and make connections with people who we otherwise would not. However, the ubiquity of social media (such as through social networks) has created a further explosion in the information content of our lives and relationships. This information clutter reduces a user's productivity and shortens the user's attention span.

Typically, ideas begin with private thoughts that are later refined and shared to friends and, perhaps, the public at large. Who these ideas are to be shared with often evolve over time. Existing online services are not amenable to this process of developing ideas and deciding where to share them at a later time. For example, existing social media forces ideas immediately into the public domain. Information or media posted on social networks (e.g. Facebook.com, Twitter.com, and Flickr.com) or blogs (e.g. Wordpress.com and Blogger.com) are immediately accessible by other users. To post information in existing social media requires upfront cognitive decision-making related to the audience level of the information at the time the information was collected or generated. This process does not match the way human brains function.

Systems, such as emailing to oneself and note-taking applications (e.g. Evernote.com) exist for users to store private thoughts. However, these existing systems are not integrated with other services or applications. In other words, a user would have to access and sign on to multiple applications simultaneously to utilize information stored in the application storing the private thoughts.

Because of the lack of integration and inconvenience of these systems, they are seldom used, therefore, ideas are often forgotten. In addition, existing note-taking applications require users to provide all of the information content to the note. By forcing a user to provide all of the information content, the note-taking process is typically tedious, error-prone, and incomplete.

The present invention addresses at least the above-described difficult problems and advances the art with a semantic note taking system and method.

Invention: A Lifestream Annotation System

Apple's granted patent and invention is directed to a semantic note taking system and method for collecting information, enriching the information, and binding the information to services. A plurality of users are communicatively connected to an application server to create one or more notes that can be bound to one or more of a plurality of services and stored in a database. The application server operates a plurality of functions, including a note taking function for allowing each of the users to create one or more notes, a categorizing function for labeling each of the notes with one or more changeable categories, a context function for associating one or more context traits with each of the notes, a binding function to establish one or more changeable binding rules for each of the notes, wherein the binding rules determine one or more of the services where the note is to be bound, and wherein the binding rules are related to the content of the note, the categories of the note, the context traits of the note, a user binding selection, or any combination thereof, and a communication function for communicating each of the notes to one or more of the services where the note is to be bound.

A note includes text, audio media, visual media, audio-visual media, recorded data, a hyperlink, a pointer to an information source, or any combination thereof. In an embodiment, one or more of the context traits associated with the note includes a time, a location, physical data relating to the user device used to generate the note, or any combination thereof. In another embodiment, relevant data is derived based on one or more of the context traits and the relevant data is attached to the note. The relevant data can be derived by matching one or more of the context traits with data from an information module such as a calendar, an address book, a contact list, a user profile, a user history, or any combination thereof. In an embodiment, one or more of the context traits associated with the note is based on one or more categories of the note, the content of the note, a user history, or any combination thereof.

In a preferred embodiment, the binding rules include one or more publication properties, such as an audience level for the note and/or one or more locations to publish the note. The locations to publish the note can include a private domain of the user, one or more friends of the user, one or more computer-implemented social networks, a blog, an online discussion board, a website, or any combination thereof. In an embodiment, the binding rules are automatically determined based at least partially on one or more of the categories, one or more context traits, the note content, a user history, or any combination thereof.

Apple's Lifestreaming Smart Glasses

The surprising element to this invention was buried deep in the patent filing relating to lifestreaming smart glasses.

Apple's patent FIG. 17 noted below presents us with an example of a data capture device 1700 which is merely one example of the variety of possibilities. In this example 1710 pertains to sensing speaker and relative speaker position; 1720 pertains to measuring respiration patterns; 1730 pertains to measuring pupil dilation, blinking, tears, redness or the like;1740 pertains to audio/video stream recording; 1750 pertains to location, direction, rotation, tilt, vibration or the like; 1760 pertains to smartphone transmission; and 1770 pertains to electrical stress and arousal measurements.

Apple further notes that externally-sensed "annotation data elements" pertains for example to direction, location, identity of person or place the user is looking at. Facial recognition software, voice recognition software, GPS tracking software or devices, semantic analysis software, software for analysis of timing of events or speech, directional or motion analysis software, stress indicators or voice tone analysis software, intentional indicators or software (start, stop, etc.) or the like could be used for context sensing or changes in context. One or more externally sensed annotation data elements could be automatically derived by the computer device, the computer or application server or an online service based on context sensing or sensing changes in context in the externally sensed data.

Physiological sensed data of the user can be obtained using physiological sensors or biosensors. Examples of data derived are for example heart rate or heart rate variability, skin conductivity, brain activity, muscle action, respiration, voice tone, movement pattern, language use, or the like. Data capture devices or techniques are, for example, an electrocardiogram, electrodermogram, electroencephalogram, electromyogram, thermometer, photoplethysmogram, pneugraph, or the like (see also capture device of FIG. 17). One or more physiological sensed annotation data elements could be automatically derived by the computer device, the computer or application server or online service based on context sensing or sensing changes in the physiological sensed data.

Product Reviews

A Wine Review: Apple starts off their example section with something most working adults will appreciate, fine wines and wine review note system. Apple notes that a user enjoys a product, such as a glass of wine. During a dinner engagement the user jots down information related to the wine, such as the brand, vintage, variety, etc., by accessing a user interface to the semantic note taking system.

The note includes a label of "#wine" for future reference to the note. Context traits including the GPS coordinates where the note was created is associated with the note. Using the GPS coordinates and accessing a directory information module, the name of the restaurant is derived and automatically associated with the note. The note is saved in the database.

At a later date, the user chooses to discuss the wine with friends of the user who may be wine enthusiasts. By binding the wine review note to a social network or an email application, the note is communicated to one or more friends of the user. One of the friends writes a brief review of the wine and attaches that review to the note. After the note has been shared to friends of the user, the user decides to write a wine review to be posted on a wine website. The user refines the note and binds it to the wine review website to be published to the world.

Other Services

Health Related: The present invention can be directed to a dietary, health, or medical service. For example, with minimal friction, a user can submit his or her dietary behavior to be tracked by a diet service. In another example, medical information can be collected at a hospital visit or doctor's appointment and submitted to an electronic medical record (EMR) application.

Research Related: The semantic note taking system of the present invention can also be applied to a research study. In an embodiment of the present invention, research data is collected in a note. The research data can be collected from the field via a mobile device. The research data is enriched with time and location context traits. For repetitive data collection, a semantic skin would provide simple one click user entry of data. For example, the present invention can facilitate frictionless data gathering from door-to-door surveys. A door-to-door surveyor would need not enter the address of each participant of the survey as location context traits would be automatically associated with the note. Relevant data (e.g. resident names and demographics) can be automatically accessed based on the location context traits.

Other Key Patent Figures

Apple's patent FIGS 12A and 12B presented below show us examples of using semantic note taking with online retailers.

More specifically, Apple's patent FIGS. 12A-B show an example of a retail application with bidding from multiple online retailers 1230. In FIG. 12A, user U.sub.1 creates a note 1220 on a user device 1210. The note 1220 includes an identification of a product and binding rules that indicate it is to be bound to retailers R.sub.1, R.sub.2, and R.sub.N. The note 1220 is communicated to the retailers R.sub.1, R.sub.2, and R.sub.N based on the binding rules. After the note 1220 is received by the retailers R.sub.1, R.sub.2, and R.sub.N, the retailers send offers 1240 to user U.sub.1 for the product described in the note 1220. Alternatively or additionally, the retailers R.sub.1, R.sub.2, and R.sub.N can send advertisements, coupons, or any information relating to the product to the user U.sub.1.

Apple's patent FIG. 9 interestingly shows us the technology services underlying the core services that includes a Xen hypervisor which Wikipedia describes as follows: Xen is a hypervisor using a microkernel design, providing services that allow multiple computer operating systems to execute on the same computer hardware concurrently.

Screenshots of Catch App

Business Insider named Catch one of the top productivity apps of 2012. The Catch app may or may not be a part of the lifestreaming system that is found in today's granted patent.

Apple credits Andreas Schobel former co-founder of BroadMap Note-taking app, and Stephen Brown as the inventors of granted patent 8,930,490 which was originally filed in Q1 2012 and published today by the US Patent and Trademark Office.

A Note for Tech Sites Covering our Report: We ask tech sites covering our report to kindly limit the use of our graphics to one image. Thanking you in advance for your cooperation.

Patently Apple presents only a brief summary of granted patents with associated graphics for journalistic news purposes as each Granted Patent is revealed by the U.S. Patent & Trademark Office. Readers are cautioned that the full text of any Granted Patent should be read in its entirety for full details.

About Making Comments on our Site: Patently Apple reserves the right to post, dismiss or edit any comments. Comments are reviewed daily from 5am to 7pm MST and sporadically over the weekend.