Overview of Apple's Invention

Apple's patent generally relates to providing an audio stream to a listening device and, more particularly, to providing a personalized ambient audio stream using ambient audio from an audio-sharing network.

In a variety of situations, many people may desire to hear a conversation, lecture, concert or other audio event that is occurring nearby but is out of earshot. Such users may include hearing impaired individuals that wear hearing aids or other people that may desire to participate in such a larger conversation or event. Although microphones in hearing aids may amplify sounds occurring nearby, the microphones in the hearing aids may not necessarily detect more distant sounds that are still part of the larger conversation or event that a hearing aid wearer may desire to hear. Likewise, those who do not wear hearing aids may not be able to hear distant sounds that are part of the larger conversation or event.

Alone, a single individual may not be able to hear or detect all parts of a larger conversation such as in a conference or event like a concert or lecture. Collectively, however, those situated around the larger conversation or event may be able to hear all pertinent sounds.

Apple's Audio-Sharing Network

Apple's invention relates to systems, methods, and devices for sharing audio via an audio-sharing network of personal electronic devices and/or other networked electronic devices (e.g., networked microphones) in an area.

In Apple's first scenario, they describe the use of an iPhone within the context of an audio-sharing network. If a group of iPhone users are scattered throughout a lecture hall and they collectively turn on their iPhone's audio-sharing network feature by clicking on a homepage icon (see below in FIG. 2), they would experience an enhanced audio experience. By collecting and sharing ambient audio, some of which may be pertinent (e.g., the lecturer's comments and/or questions from those in attendance) and some of which may not be pertinent (e.g., murmurs, faint sounds, noise, and so forth) the audio experience can be enhanced.

The member devices of the audio-sharing network located throughout the lecture hall could provide audio to their respective users by combining or processing the various audio streams shared by the audio-sharing network to obtain personalized audio streams.

In some embodiments, the personalized audio streams may primarily include only the pertinent audio. These personalized audio streams may be provided to their respective users via the speakers in their iPhone, headset or hearing aid.

For example, as shown in patent FIG. 15 below, many sounds may be present in the university lecture hall 90 setting, only some of which may be desirable to students 94 sitting in the lecture. For example, a student in the back of the lecture hall may ask a question (at the back # 270) to which the lecturer (at the front # 92) may respond with an answer (# 272). Although the students may primarily desire to hear the question and the answer, other sounds may be present, such as random noise (#274), a murmur (#276), and/or other faint sounds (#278).

The Personalized Audio Stream

As shown in patent FIG. 16 below, the audio-sharing network 70 formed between the handheld devices 34A, 34B, 34C, 34D, and/or 34E may be near enough to obtain ambient audio that includes these various sounds 270, 272, 274, 276, and/or 278. From the audio streams provided by the various member devices of the audio-sharing network, the handheld device 34A may determine the personalized audio stream 76. In some embodiments, the personalized audio stream may primarily include the question and the answer, and may largely exclude the noise, the murmur and faint sounds. As shown in FIG. 16, the personalized audio stream 76 may be output to a personal listening device, such as the hearing aids 58.

In the example of FIG. 16, the handheld device 34A is shown to determine the personalized audio 76 to include primarily audio that is likely to be of interest to its listener. In some embodiments, the handheld device 34A may determine the personalized audio stream 76 by varying the volume levels of the audio streams received via the audio-sharing network, or by including or excluding certain of the audio streams received via the audio-sharing network.

The Audio-Sharing Network: Teleconferencing

As shown in patent FIG. 31 below, an audio-sharing network may (70) also be employed in the context of a teleconference (460). The teleconference may include several conferees (462) seated around a conference table (464). Some or all of the conferees may have iPhones as indicated by 34A, 34B, 34C, 34D, 34E and/or 34F. An audio-sharing network may be formed from among those devices 34A-F and a conference telephone (466) or any other suitable teleconferencing device.

As represented by a schematic diagram illustrated in patent FIG. 32 noted above, each of the handheld devices 34A, 34B, 34C, 34D, 34E and/or 34F may respectively obtain audio streams 74A, 74B, 74C, 74D, 74E, and/or 74F, which may be provided to the conference telephone. The conference telephone may obtain a personalized teleconference audio stream (476) in the manner described earlier in the lecture Hall example but this time it's the conference telephone that is benefiting from audio clarity which would shut out murmurs and background noise. As should be appreciated, the telephone network (478) may or may not be a traditional telephone network. Indeed, in some embodiments, the telephone network may be the Internet and the personalized audio stream may be provided as voice over Internet protocol (VOIP), for example.

Easy Muting & Recording Options

The next three patent graphics point to easy muting and recording options. In Patent FIG. 21 an iPhone is a member of the audio-sharing network that may provide audio to the audio-sharing network while the iPhone is facing upward, but not when it's rotated to face flat downward.

When the iPhone is lying flat, facing upward, the orientation-sensing circuitry may indicate to the iPhone in this orientation that it could provide an audio stream to the audio-sharing network. This same senor will know when the iPhone orientation is facing downward and will mute the audio stream going to the audio-sharing network.

Once the mute action in FIG. 21 has taken affect, the user can return the iPhone face up and see a "mute" screen. When the user wishes to add their phone to the audio-sharing network again, they'll simply shake it or move it in a way that will allow the iPhone to once again join the audio-sharing network. For instance, if the user is asking a lecturer or presenter a question, the user will want to have their phone reactivated quickly when they get to speak so that all in the audio-sharing network can clearly hear his question.

In the last scenario, a user may keep their iPhone in a pocket, away from the light, when it's not in use. Accordingly, in some embodiments, the iPhone is a member of the audio-sharing network and may remain muted (361) while in a user's pocket, as shown in patent FIG. 23. When the user removes their iPhone from the user's pocket 360 (e.g., to ask a question or otherwise participate in a conversation), the ambient light sensor 20 of the iPhone may detect light 362. When the quantity of light exceeds a threshold, indicating that the iPhone is no longer ensconced in a pocket, the iPhone may begin to obtain and/or provide audio to the audio-sharing network. The iPhone may also display the screen 340, indicating that the handheld device is now obtaining such audio.

Method of Joining an Audio-Sharing Network + Security Measures

Apple's patent filing goes through the joining of an audio-sharing network step by step and in great detail. Yet for our report we just wanted to point out that one of the process options of joining the audio-sharing network involves an NFC application as pointed out in patent FIG. 27 whereby a user could join the network quickly by simply tapping their iPhone with someone already on the audio-sharing network. Else the user would have to go through the entire process using a password and public encryption key. One of the reasons for the tight security is so to ensure that no one could electronically eavesdrop on your group's conversations.

In some embodiments, as shown in an authentication process 230 of patent FIG. 13 above, an audio security code 232 may be used to verify the location of the prospective joining iPhone 34A. In particular, as illustrated in FIG. 13, when the prospective joining iPhone 34A establishes a connection (72) to the iPhone 34B, the iPhone 34B may emit an audio security code. The audio security code 232 may be certain sounds that are audible to humans or ultrasonic and inaudible to humans. The iPhone 34A may be permitted to join the audio-sharing network when the iPhone 34A is close enough to the iPhone 34B to detect the audio security code.

For those with hearing impairments joining the audio-sharing network, Apple states that while the wireless hearing aid is outputting the personalized audio stream, the microphone of the wireless hearing aid may or may not be collecting additional ambient audio and outputting the additional ambient audio to the iPhone speaker. In some embodiments, the wireless hearing aid may represent a cochlear implant, which may use electrodes to stimulate the cochlear nerve in lieu of a speaker.

Other Patent Figures Related to Apple's Audio-Sharing Network Filing

Apple's patent FIGS. 17 and 18 are schematic diagrams of screens that may be displayed on the iPhone of to enable the iPhone to determine a personalized audio stream; patent FIG. 19 is a schematic diagram of a screen that may be displayed on an iPhone to allow a moderator of the listening network to easily implement network-wide audio settings.

Patent Credits

Apple's patent application was originally filed in Q1 2011 by inventor Gregory F. Hughes and published by the US Patent and Trademark Office last Thursday, July 26, 2012.

Note to Referring Sites: We ask that referring sites limit the use of our graphics to a maximum of two per report. Thank you for your cooperation.

Notice

Patently Apple presents a detailed summary of patent applications with associated graphics for journalistic news purposes as each such patent application is revealed by the U.S. Patent & Trade Office. Readers are cautioned that the full text of any patent application should be read in its entirety for full and accurate details. Revelations found in patent applications shouldn't be interpreted as rumor or fast-tracked according to rumor timetables. About Comments: Patently Apple reserves the right to post, dismiss or edit comments.

Sites Covering our Original Report

MacSurfer, Twitter, Facebook, Apple Investor News, Google Reader, Macnews, iPhone World Canada, MacDailyNews, Mac4Ever France, and more.