Apple's first patent application regarding Siri working with Messages surfaced back in November 2016 in our report titled "Apple Patent Shows Siri being used in an iMessage Thread to Send Funds to a Friend using Apple Pay." Today, the U.S. Patent Office revealed more of Apple's work on Siri for Messages where users could interact with Siri via Messages if they're in a quite library or in a very noisy environment and still get Siri's assistance and more.

Apple's patent FIG. 3 published in November and noted above shows us a method of an iPhone utilizing Siri to assist with a communication session; In FIGS. 6A-6E noted above we're able to see a full example of Siri assisting participants of a communication session to complete a financial transaction.

In today's patent application regarding Siri on Messages we learn the following:

"It can be desirable to implement a digital assistant in a messaging environment. The messaging platform can enable multiple modes of input (e.g., text, audio, images, video, etc.) to be sent and received.

A user request can thus define a request to the digital assistant using a combination of inputs (e.g., image and text). As described herein, this can increase the functionality and capabilities of the digital assistant, thereby providing a richer interactive experience between a user and a digital assistant.

In addition, the messaging platform is a visual interface which permits interactions in a broader range of environments than voice/audio based platforms. A digital assistant in a message environment can thus enable greater accessibility to the digital assistant.

In particular, Siri (the digital assistant) can be accessible in noisy environments or in environments where audio output is not desired (e.g., the library).

Further, the messaging platform can be a conversational interface where short concise communications are exchanged between two or more parties and presented in a chronological format.

A digital assistant in a message environment can thus enable the digital assistant to be a participant in a multi-party conversation where the benefits associated with the digital assistant can be shared among the multiple participants.

Moreover, the chronological format enables a user to conveniently review previous interactions with the digital assistant and utilize the contextual history associated with the previous interactions to define a wider range of tasks.

In the example illustrated in FIG. 12G below, the media object displayed in message #1214 (e.g., at block 1106) can be an audio clip of a media item (e.g., a song, movie, soundtrack, etc.). In the present example, the audio clip can be a sung portion of the United States national anthem. The text of the second user input displayed in message #1216 (e.g., at block 1112) can be a request to perform a search based on the song corresponding to the audio clip (e.g., "What's this song?", "Who sang this?", "When was this song released?", etc.). Based on the audio clip and the text, it can be determined at block #1114 that the user intent includes performing audio recognition on the media item to identify the song in the audio clip. In this example, a determination can be obtained at block #1132 that the user intent requires performing audio processing (e.g., audio recognition) on the media object. In response to obtaining a determination that the user intent requires performing audio processing on the media object, block #1134 can be performed.

As shown in Apple's patent FIG. 12H above, the media object of block #1106 can be a picture of a bottle of wine (e.g., displayed as message 1220) and the text of block #1110 (e.g., displayed as message 1222) can define the user's opinion or preference for the wine (e.g., "I like this wine" or "This wine has vivid flavors of ground spice with hints of raspberry fruit and lemon zest"). In another example, the media object can be a picture of the parking space number at which the user parked his or her car and the text of block can indicate that the user parked at the location associated with the contents of the media object (e.g., "I parked here."). In the examples described above, it can be determined that the user intent is to store data that associated the defined attribute to the media object for future retrieval. Based on this user intent, it can be determined at block #1132 that the user intent does not require performing audio processing on the media object. In response to obtaining a determination that the user intent does not require performing audio processing on the media object, the electronic device can cause an action to be performed at block #1140 in accordance with the user intent of block #1114.

Apple's patent application 20170132019 was filed back in May 2016. For more details about this invention, click here. Considering that this is a patent application, the timing of such a product to market is unknown at this time.

Patently Apple presents a detailed summary of patent applications with associated graphics for journalistic news purposes as each such patent application is revealed by the U.S. Patent & Trade Office. Readers are cautioned that the full text of any patent application should be read in its entirety for full and accurate details. About Making Comments on our Site: Patently Apple reserves the right to post, dismiss or edit any comments. Those using abusive language or behavior will result in being blacklisted on Disqus.