Microsoft's Motion Capture System

Jumping right into Microsoft's patent, we leap to the part that's at the heart of a possible future gaming system – its core motion capture camera system. In Microsoft's Patent FIG. 5 we're able to see a depiction of an example embodiment of a motion capture system in which a gamer interacts with a video game in their home. 

The motion capture system noted above as patent point #10 includes a display (196), a depth camera system (20), and a computing environment or apparatus (12). The depth camera system may include an image camera component (22) having an illuminator (24), such as an infrared (IR) light emitter, an image sensor (26), such as an infrared camera, and a red-green-blue (RGB) camera 28. A human also referred to as a gamer or player, stands in a field of view (6) of the depth camera. Lines 2 and 4 denote a boundary of the field of view.

In this example, the depth camera system and computing environment provide an application in which an avatar (197) displayed on a television (196 or other display) track the movements of the gamer. For example, the avatar may raise an arm when the gamer raises an arm. The avatar is standing on a road (198) in a 3-D virtual world.

Generally, the motion capture system is used to recognize, analyze, and/or track one or more human targets. The computing environment can include a computer, a gaming system or console, or the like, as well as hardware components and/or software components to execute applications.

The depth camera system may include a camera which is used to visually monitor one or more people, such as the gamer in the example above, such that gestures and/or movements performed by the gamer may be captured, analyzed, and tracked to perform one or more controls or actions within an application, such as animating an avatar or on-screen character or selecting a menu item in a user interface (UI).

The motion capture system may be connected to an audiovisual device such as the display, a television, a monitor, a high-definition television (HDTV), or the like, or even a projection on a wall or other surface that provides a visual and audio output to the user.

The gamer may be tracked using the depth camera system such that the gestures and/or movements of the user are captured and used to animate an avatar or on-screen character and/or interpreted as input controls to the application being executed by computer environment.

Microsoft states that some movements of the gamer may be interpreted as controls that may correspond to actions other than controlling an avatar. For example, in one embodiment, the gamer may use movements to end, pause, or save a game, select a level, view high scores, communicate with a friend, and so forth. The player may use movements to select the game or other application from a main user interface, or to otherwise navigate a menu of options. Thus, a full range of motion of the gamer may be available, used, and analyzed in any suitable manner to interact with an application.

The motion capture system may further be used to interpret target movements as operating system and/or application controls that are outside the realm of games and other applications which are meant for entertainment and leisure. For example, virtually any controllable aspect of an operating system and/or application may be controlled by movements of the gamer.

According to Microsoft, the depth camera system may further include a microphone (30) which includes, e.g., a transducer or sensor that receives and converts sound waves into an electrical signal. Additionally, the microphone may be used to receive audio signals such as sounds that are provided by a person to control an application that is run by the computing environment. Microsoft states that "The audio signals can include vocal sounds of the person such as spoken words, whistling, shouts and other utterances as well as non-vocal sounds such as clapping hands or stomping feet."

Microsoft's patent FIG. 1 shown below illustrates one embodiment of an image sensor within an image camera component.

An Image Sensor within an Image Camera Component

Microsoft's patent FIG. 1 shows one embodiment of an image sensor (26) within an image camera component (22). The image sensor includes a photosensor (5) and a focusing lens (7). Note that the image camera component may also include an illuminator, which may illuminate the Field of View (FOV).

Microsoft's patent FIG. 2A illustrated above shows one embodiment of an illuminator (24) in an image camera component. In some embodiments, the image camera component is part of a depth camera system.

Basic Overview of the Motion Camera System

icrosoft's patent FIG. 6 noted above depicts an example block diagram of the motion capture system. The depth camera system may be configured to capture video with depth information including a depth image that may include depth values, via any suitable technique including, for example, time-of-flight, structured light, stereo image, or the like. The depth camera system may organize the depth information into "Z layers," or layers that may be perpendicular to a Z axis extending from the depth camera along its line of sight.

The depth camera system may include an image camera component that captures the depth image of a scene in a physical space. The depth image may include a two-dimensional (2-D) pixel area of the captured scene, where each pixel in the 2-D pixel area has an associated depth value which represents a linear distance from the image camera component 22, thereby providing a 3-D depth image.

The image camera component may include an illuminator, such an infrared (IR) light emitter, an image sensor, such as an infrared camera, and a red-green-blue (RGB) camera that may be used to capture the depth image of a scene or provide an additional camera for other applications. The illuminator may be configured to compensate for dependence in irradiance of objects captured by the image sensor. Thus, the illuminator may have a refractive optical element such as, but not limited to, any of the examples herein.

Using Skeletal Models

According to Microsoft, the depth camera system may provide the depth information and images captured by, for example, the image sensor and/or the RGB camera, and/or a skeletal model that may be generated by the depth camera system to the computing environment via the communication link. The computing environment may then use the model, depth information, and captured images to control an application such as a video game. The computing environment may include a gestures library, such as a collection of gesture filters, each having information concerning a gesture that may be performed by the skeletal model (as the gamer moves).

For example, a gesture filter can be provided for various hand gestures, such as swiping or flinging of the hands. By comparing a detected motion to each filter, a specified gesture or movement which is performed by the gamer can be identified.

The data captured by the depth camera system in the form of the skeletal model and movements associated with it may be compared to the gesture filters in the gesture library to identify when a gamer (as represented by the skeletal model) has performed one or more specific movements. Those movements may be associated with various controls of an application, such as video game.

The more I read about this, the cooler it gets. It would also appear that the motion camera in this scenario could very well work within Kinect and focused on both immersive and just regular gaming on your TV. Microsoft may very well begin at this level before immersing us into the game with projection. Either way, it all sounds great to me. And, as an avid gamer myself, I say to Microsoft: push the pedal to the metal on this project guys, we can't wait to try it out.

Microsoft's patent application was originally filed in Q1 2011 and published by the US Patent and Trademark Office in Q3 2012.

Note to Referring Sites: We ask that referring sites limit the use of our graphics to a single graphic. Thank you for your cooperation.

Notice

The Patent Bolt blog presents a detailed summary of patent applications with associated graphics for journalistic news purposes as each such patent application is revealed by the U.S. Patent & Trade Office. Readers are cautioned that the full text of any patent application should be read in its entirety for full and accurate details. Revelations found in patent applications shouldn't be interpreted as rumor or fast-tracked according to rumor timetables. About Comments: Patent Bolt reserves the right to post, dismiss or edit comments.