Apple's Media Players May Gain Cool Sense Line Controls
Apple Granted a Huge Patent Win for 3D Virtual Apple Store

Location Based Social Networking & Video Calling Coming to iPhone


There's an old adage concerning the finding of a location for your brick and mortar retail business: Location, location, location. While that may still hold water today, that saying is going to be hijacked by the next wave of social networking apps based on Location Awareness technology. In the Social Networking Era, it's all about having the ability to access the location of your friends, colleagues, associates and family – on demand.  This is going to be a very popular feature on future iPhones – if not the iPad in the coming months and years ahead. Apple's published patent momentum on Location Aware technology clearly indicates that Apple is ramping this technology up for release. On January 15, we learned of location aware technology in respect to pushing ads to users who would be interested in learning about near-by retail deals. This is perhaps Apple's vision of creating the virtual home-flyer concept. Then on January 25, we learned of the iPhone's location aware services in the works that will offer us features such as real-time public transit schedules, inner city parking availability and social networking so that you could coordinate your weekends a little better with friends on the fly. Today's patent revs-up the social networking angle and even points us to how this will work with video calling on your iPhone. To top it off, Apple's patent describes the iPhone's GPS system using maps with an optional step-by-step instruction feature, introduces us to new "Request and Release Info" virtual buttons to initiate location based services and finally – a future iPhone's high-end security features that will protect your current location from prying eyes and prowlers on the net.


Patent Overview

Several techniques that facilitate device-to-device location awareness during a telephone call are described in Apple's patent. One embodiment of the invention is a method for communications between a first mobile device and a second mobile device, described as follows. During an ongoing telephone call (communication session) between the first and second devices, an over the air message (e.g., a short message service, SMS, or text message) is sent from the first device. This may be in response to a user activating a designated virtual or physical button of the first device, or giving a verbal command, to send a location request message. The message requests the current location of the second device. It may also be viewed as requesting permission to reveal the current location to the user of the first device. An RF-based locating methodology that determines location information of the second device is then performed. This determined location information is then sent to the first device and can be automatically displayed to its user. Thus, this technique enables a person who is on a call with another person and who would like to meet the other person, to immediately find out the location of the other person.


To ensure privacy, the user of the second device may be prompted to give permission to release her location information (e.g., by actuating a virtual or physical button on the second device), during the ongoing telephone call. Alternatively, a stored profile of the user of the second device may be checked, for automatically obtaining permission to release location information to the requesting device.


The above-described process for location awareness may occur directly between the two mobile devices. For instance, the initial request for location information from the first device may be received by the second device as an SMS or text message sent from the first device. The second device can then send its location information back to the first device, via another SMS or text message. In that case, there is no requirement for modifying any cellular telephony network infrastructure to deploy such a service, so long as each of the devices has the needed device-to-device location awareness application running (that can accept a location request text message and reply by sending a location information and permission text message.


Video Conferencing for the iPhone

Apple's patent points to using iPhone video conferencing with this new location feature as follows: "Note that the reference to "voice call" here is not limited to a conventional, sound-only conversation. It may also include video of the two users, synchronized with their audio. The call may be a cellular network telephone call that has been initiated by either user.


Update: In order to video call, the iPhone will have to have a new kind of video camera. That technology was touched on in our January 5 report titled Apple Working on iChat & 3D UI for Portables.


Location Awareness System: Show Me Yours & I'll Show You Mine

Apple's patent FIG. 1 is a block diagram of a pair of mobile devices that have location awareness capability; FIG. 2 a flow diagram of the communications methodology between two users using their respective mobile devices for location awareness.



Once one user has given the requester permission to access their iPhone's coordinates via GPS, the display of the requester's iPhone may show the location of the first user's via a street map that contains a pointer or marker at the current location of the second device. The view of this map should be sufficiently wide so as to also show at the same time a marker at the location of the first device, thereby allowing the first user to obtain a better understanding of the distance between his current location and that of the second user (with whom he is having an ongoing telephone conversation).


The iPhone and in particular the location awareness module 141, may have the further ability of offering its user the option of displaying step-by-step street directions, to travel from the current location of that device to the current location of the remote device (block 216). Thus, the location awareness module running in the first device 100-1 could prompt the first user 99-1 as to whether or not he would like to see step-by-step directions to travel from their current location to that of the second device 100-2.


Apple's patent states that "This may be part of a balanced or two-way location information exchange protocol, i.e. "Show me yours and I'll show you mine", to which the second device may be subscribed."


The Future iPhone's Location Awareness Activation Buttons


Apple's patent FIG. 5 shown below, illustrates a virtual button labeled "Request Location Info" (image object 415) that in effect prompts the user during the phone call as to whether he would like to obtain the current location of another user who is also on the call. Note that if there are more than two users on the call (such as a conference call having three or more participants), actuating the Request Location Info button 415 may result in the location awareness module 141 further prompting the user to select one of the two or more other (remote) devices (for its location information). Once the location information has been received at the local device from one or more remote devices that are participating in the phone call, the user may be given the option of launching a map/travel application that will calculate step-by-step street directions for traveling from the current location of the local device to that of the selected, remote device.


As an alternative to placing the virtual button for Requesting Location Info (image object 415) on the front display of the touch screen 112 during the phone call, this virtual button may be exposed in a contact list or address book of the user. For example, the button may be added to the name address fields associated with each contact in the list or address book. In that case, the user may need to first actuate the Contacts button in the on-going voice call screen (see FIG. 5, tray 418), to bring to foreground her contacts list. She may then scroll through her virtual list of contacts (e.g., by name) until arriving at the record associated with the other user who is on the call. After selecting the contact list record of the other user, the user will be presented with the Request Location Info button associated with the selected record. She may now actuate the button, which triggers the automatic sending of the location request message without further input from her.  



Apple's patent FIG. 5 also shows that, on the on-going voice call screen, a further image object 413 may be displayed which is labeled "Release Location Info". This image object is associated with a virtual button that, when actuated by the user of the device during a voice call, causes a location release message to be sent to a selected remote device that is participating in the voice call. The location release message may contain permission to release the current location information of the local device (e.g., calculated using the local instance of the RF locator or GPS module 135). The location release message may also contain the current location information itself. The image object may be highlighted or flashed or otherwise made more prominent whenever a location request message is received from a remote device (that is participating in the phone call). In some embodiments, the display of image object may be suppressed until or in response to the local device receiving a location request message from a remote device.


Heavy Security


There may also be several security aspects to the location awareness methodologies described above, as follows. To ensure authenticity of the sender and to prevent spoofing of inter-device messages, the messages may be cryptographically signed and/or encrypted before being sent. A form of public key infrastructure, PKI, security process may be used in that case, to verify each user's identity. A User of a device may be identified by a cryptographic hash of the following combination: the user's name, the user's associated contact numbers, a shared or public secret key, and a private secret key. A stored profile of another user may also be treated in a similar manner, i.e. it may be cryptographically signed and/or encrypted and then stored in the device.


Portable Device Architecture with Location Awareness & Mapping Modules


Apple's patent FIG. 3 noted below provides us with a view of an iPhone-like architecture with the addition of a locatioin awareness module that tightly intergrated with a mapping module.


The GPS module 135 determines or computes the current geographic location of the device and provides this information for display or use by other applications, such as by a the telephone module 138 for user in location-based dialing and applications that provide location-based services, such as a weather widget, local Yellow Page widget, or map/navigation widgets (not shown). The widget modules 149 depicted here include a calculation widget which displays a soft keypad of a calculator and enables calculator functions, an alarm clock widget, and a dictionary widget that is associated or tied to the particular human language set in the device 100.


Other modules that may be provided in the device 100 include a map/travel module 144 that can display a street map of the current location of the device and obtain step-by-step street directions to a destination selected by the user, or as described above, obtained by the location awareness module 141 from another device with which there is an ongoing telephone call.



Apple credits Michael Lee, Justing Gregg and Chad Seguin as the inventors of patent application 20100029302, originally filed in Q3 2008.


Update: Another "Location Sharing" patent was revealed on December 31, 2009 under patent number 20090325603. The graphic below from that patent shows the mapping app using a pinning-system to mark the location of your buddy, colleague or family member.

Maps + Compass 
When we connect all of the recent location awareness patent dots together, we definitely begin to see a serious trend forming here.


NOTICE: Patently Apple presents only a brief summary of patents with associated graphic(s) for journalistic news purposes as each such patent application and/or grant is revealed by the U.S. Patent & Trade Office. Readers are cautioned that the full text of any patent application and/or grant should be read in its entirety for further details. For additional information on any patent reviewed here today, simply feed the individual patent number noted above into this search engine.



I'm concerned about privacy for the people who happen to be in abusive/controlling relationships and would be put in the uncomfortable position of having to say "no" to someone who won't accept that as an answer. Especially since this technology will most likely eventually be in all phones.

Sounds like the case for the iPad having widgets has become stronger.

I think that the location awareness aspects of the patent are likely to come to market sometime over the next 12-18 months (or sooner).

The video calling aspect of the patent, though in the works, is likely going to take 5G technology or equivalent to make it viable. That's easily 2-4 years away. I'd love it tomorrow too Mr. Abrook, as we all - but it's going to take a little more time than we'd like. Then again, you never know with Apple.


Thanks for the info Jack. Not having to wait until I'm back in the office or home to video conference will be great. Now if Verizon gets access to the iPhone this year I'm all over it.

Mr. R. Abrook, aka "inyaface" makes a point, but one that was answered in our patent report posted on January 5, 2010:

The introduction states" Equally important, in my view, is that this patent points to a new unique camera built into a future iPhone that will allow the user to rotate the position of the optical sensor within the iPhone's housing so that a single optical sensor could be used for both video conferencing and still and/or video image acquisition. Turn the camera out for pictures and turn the camera in for video conferencing. That may end up really being the magic to this patent."

I hope that answers your question Mr. Abrook. It's a great point that should have been addressed in the big picture. But, here it is for anyone else with that same question or thought.


How's the video conference going to work? Do you have to stand in front of a mirror with the camera facing you to be able to see the other person or is the phone going to have a second camera on the face like some Nokias?

Hmm, why would privacy complaints emerge?

You have to be talking to someone for this to even work and you have to "release" the info deliberately by pressing a specific button. You wouldn't give your private location info to just anyone, unless your nuts, and so - privacy doesn't seem to be an issue with this scheme that Apple is proposing.

Though in general, John, these technologies always start off innocently and then get twisted and hacked. And yes, some will always see Big Brother in everything. Ha!

Apple will spin it, and if they do a good enough job, it'll fly.

I can imagine the complaints about privacy emerging.

The comments to this entry are closed.