Patent Background

Modern mobile devices integrate a variety of communication technologies with geopositioning technology and sensors. The communication technologies (e.g., cellular, Wi-Fi, Bluetooth) allow users to communicate with other users in a variety of operating environments. Geopositioning technologies allow users to receive detailed digital maps and location-based services. Various sensors integrated in the mobile device receive input from the operating environment and provide output for a variety of interesting and useful applications running on the mobile device. For example, an accelerometer could detect when a device is held in portrait or landscape orientation, and then adjust a user interface based on the detected orientation. Sensors, such as the accelerometer in Apple Inc.'s iPhone have also been used in games. The user manipulates the iPhone in response to game action, such as trying to guide a virtual ball through a virtual maze or steer a virtual car or boat through a virtual racetrack.

Patent Summary

An interactive game environment includes two or more co-located, networked, direction and location aware interactive game devices. The game devices share a common reference coordinate frame (e.g., a three-dimensional Cartesian coordinate frame). Each game device maintains its own device state (e.g., position, orientation, time) in the reference coordinate frame. Each interactive game device shares its device state with the other interactive game devices using communication technology (e.g., Bluetooth, Wi-Fi, cellular). Each interactive game device can use the device states of the other interactive game devices to project the relative positions and orientations of the other interactive game devices into a local, fixed coordinate frame of the interactive game device. These projections allow each interactive gaming device to know the position and orientation of the other interactive game devices in an interactive game environment defined by the reference coordinate frame. An interactive game display could be generated and displayed on each game device. Various interactive games (e.g., Laser Tag) can utilize the relative positions and orientations shared between the interactive game devices to provide an enriched interactive gaming experience.

Calibration for Interactive Game Devices

Apple's patent FIG. 1 shown below illustrates a calibration protocol for interactive game devices 102, 104. The game devices 102, 104 could be any mobile device with the ability to determine its position and orientation with respect to a reference coordinate frame defining an interactive game environment, and includes communication technology (e.g., wireless technology) for sharing information with other game devices.

The game devices noted as iPhones 102 and 104 above could include a variety of integrated sensors or could be coupled to sensors through one or more ports of the game device (e.g., USB port). Some example sensors include but are not limited to: accelerometers, gyros, geopositioning technology (e.g., GPS, cell tower triangulation, Wi-Fi), a magnetic compass and any other technology that can be used to determine position and/orientation of a game device.

NED: A Geographical Coordinate System

Apple's patent FIG. 2A shown below illustrates a reference coordinate frame and a device coordinate frame. In this example, the well-known North-East-Down (NED) coordinate frame is shown. The NED coordinate frame is a Cartesian coordinate frame used in aircraft and spacecraft navigation. The North basis vector of the NED coordinates points North. The East basis vector points East and the cross product of the North and East basis vectors provides the Down basis vector to complete a right-handed coordinate frame.

Each interactive game device has a position vector in the NED coordinate frame that changes as the player moves their device in the real world, interactive game environment. Each device knows its position vector by being location aware. For example, each device can include a GPS receiver, which provides a current geographic location of the device.

During game play, each device communicates its position vector in NED coordinates to the other interactive game devices using a wired or wireless communication technology (e.g., Bluetooth, Wi-Fi). Each interactive game device could transform the NED position vectors received from other game devices into its respective device coordinate frame. The position vectors in the device coordinate frame could then be transformed by the game device to a two-dimensional display or screen coordinate frame which could be used to determine interactions with other devices and to, optionally, generate an interactive game display, as described in reference to FIG. 3 below.

Example Interactive Game Display

Apple's patent FIG. 3 shown below illustrates an interactive game display.

In some implementations, the positions of game devices in the reference coordinate frame defining the interactive game environment (e.g., NED coordinate frame) could be used to display player icons (302) in an interactive game display (300) on each game device. The game display could have its own display coordinate frame. In this example, the display coordinate frame for Device A (operated by Ron) has a fixed alignment with the device A coordinate frame, so no additional transformation from device A coordinate frame to display coordinate frame is needed.

FOV - Field of View

Player icons are displayed on the display if corresponding position vectors in reference coordinates are transformed into device (or screen) coordinates, and the transformed position vectors fall within a defined field of view (FOV). For example, to determine if a position vector for a given player is within the FOV, a dot product could be taken between the position vector in device coordinates and the x-axis of the device coordinate frame. If the angle between these vectors is less than the defined FOV (e.g., less than 85 degrees), then the player/device is deemed to be within the FOV and the corresponding player icons will be displayed on the display. The location of the player icons on the display could be determined by scaling the position vectors in device coordinates to units that are consistent with the units and dimensions of the display.

Laser Tag Gaming

For a Laser Tag game (the result of the methods described above) are that as players point the devices at each other in the real world and "tag" each other. The mathematics described in the patent will determine whether a player has been tagged without the device actually projecting a laser beam or other light source in the real world. If an interactive game display is used, player icons will appear and disappear from the respective displays of each device, as the game devices move about the real world environment. That is, the locations of the player icons on the respective displays will change position in the displays based on the relative locations of the game devices in the real world.

In the example shown, the interactive game display of Device A is shown (see FIG 3). There are three other players (three other game devices) in the example Laser Tag game. In Laser Tag, players try to "tag" other players to score points. The four players in this example game are Ron, Patrick, Rob and Isabel. Interactive game applications running on each device could allow each player to select a name or character for their player icon. The display could include an overlay of a cross-hair (patent point 306), which could be manipulated by the players to target and tag other players. For example, the player Patrick (player icon 302a) is currently being targeted by Ron, which is illustrated by the player icon 302a for Patrick being aligned with the cross-hair. In some implementations, the game display includes a portion 308 for firing a virtual laser and a portion 310 for displaying scores or other game information. Other user interfaces are also possible based on the needs of the game.

In some implementations, only two-dimensions of position are needed. For example, the down coordinate in NED may be ignored for some interactive games. For such games, game devices need only send North and East components of position vectors to other game devices to reduce the amount of information to be transferred to other game devices. In some implementations, a distance between game devices may be needed by the interactive game.

In some implementations, various objects could be fixed in the interactive game environment by the game application or by players. For example, as illustrated in FIG. 3, a virtual object 304 could be placed in the game environment to improve the gaming experience. In this example, Isabel is hiding behind virtual object 304, so that other players can't easily target her. The fixed location, physical dimensions of the virtual object, together with its distance from the shooter could be used to mathematically determine whether a LOS between two players in the interactive game environment is blocked by the virtual object 304.

In a shooting game, for example, this information could be taken into account for determining whether a player was hit or "protected" by the virtual object 304. Motion could be imparted to the virtual objects, or could be altered during game play (e.g., to simulate parts being destroyed by a laser, etc.) during game play.

Apple's patent FIG. 4 shown above is an example process performed by an interactive game device.

As in any good war game, you'd want and need to communicate with your team members. The system provides for an audio subsystem that could be coupled to a speaker and a microphone to facilitate voice-enabled functions, such as voice recognition, voice replication, digital recording, and telephony functions. I'm sure that specialty headsets will be made available from the gaming company or from third party developers. Some games today like the upcoming Call of Duty: Black Ops could work with headsets to coordinate game play in multiplayer mode. I doubt if they have to be as complicated as those I've linked to for Apple's laser tagging game, but you get the idea.

Apple credits Ronald Huang, Rob Mayor, Isabel Mahe and Patrick Piemonte as the inventors of patent application 20100279768, originally filed in Q2 2009.

Notice: Patently Apple presents only a brief summary of patents with associated graphic(s) for journalistic news purposes as each such patent application is revealed by the U.S. Patent & Trade Office. Readers are cautioned that the full text of any patent application should be read in its entirety for further details. About Comments: Patently Apple reserves the right to post, dismiss or edit comments.