Back in September Apple's CEO Tim Cook was interviewed by ABC News and he focused on Augmented Reality being Apple's preference over Virtual Reality. Cook told ABC's reporter Roberts that AR "gives the capability for both of us to sit and be very present, talking to each other, but also have other things -- visually -- for both of us to see. Maybe it's something we're talking about, maybe it's someone else here who's not here present but who can be made to appear to be present. There are a lot of really cool things there." While Apple is talking up Augmented Reality (AR), Apple's competitor Microsoft wanted to get the leap on Apple in this emerging market and introduced HoloLens. In today's report we look at how Microsoft's increasing patent filings that at minimum shows us that they're planning to attack AR on multiple fronts with their one common Windows 10 platform and their offshoot for AR systems. 

Behind the scenes Microsoft patents are flying in monthly. In late January the U.S. Patent Office published three of Microsoft's patents relating Virtual Reality. The first is titled "Grasping virtual objects in Augmented Reality." The second patent application covers another type of head mounted system than their current HoloLens in a filing titled "Augmented reality system eye relief adjustment mechanism." The third patent application relates to Augmented Reality for a Surface tablet titled "Augmented reality light guide display."

#1: Grasping virtual objects in Augmented Reality

Microsoft's invention under patent application 9,552,673 relates to an augmented reality system which enables grasping of virtual objects and describes such as to stack virtual cubes or to manipulate virtual objects in other ways. In various embodiments a user's hand or another real object is tracked in an augmented reality environment. In examples, the shape of the tracked real object is approximated using at least two different types of particles and the virtual objects are updated according to simulated forces exerted between the augmented reality environment and at least some of the particles. In various embodiments 3D positions of a first one of the types of particles, kinematic particles, are updated according to the tracked real object; and passive particles move with linked kinematic particles without penetrating virtual objects. In some examples a real-time optic flow process is used to track motion of the real object.

Microsoft's patent FIG. 9 is a schematic diagram of an augmented reality system: FIG. 3 is a schematic diagram of a user stacking two virtual cubes using the interactive 3D computer graphics system; FIG. 10 illustrates an exemplary computing-based device for use with the augmented reality systems and techniques.

#2: Augmented Reality Head Mounted System

In Microsoft's second patent application 9,557,569 titled "Augmented reality system eye relief adjustment mechanism," we learn about an alternative head mounted display (HMD) system. Microsoft notes that conventional HMDs employ pivoting hinge and plate mechanisms to adjust eye relief for various user head sizes, which can be bulky, cumbersome to manipulate, and may not accommodate a broad range of user head sizes and eye relief distances.

Failure to accommodate a user's head size can result in the HMD being too tight and causing discomfort, or too loose and moving relative to the user's head during use, negatively impacting the user's experience. Failure to maintain proper eye relief distance can adversely impact the user's ability to properly perceive the augmented reality elements displayed on the display of the HMD.

Microsoft's solution is a head mounted display system that comprises of an adjustable crown band, a crown band circumference adjusting mechanism coupled to the adjustable crown band and configured to adjust a crown band circumference to releasably secure the crown band to the user's crown. The system includes an armature including left and right arms, each of the left and right arms mounted at a respective proximal end to corresponding left and right HMD attachment structures. Further, a display mounted at left and right sides to respective distal ends of the left and right arms, and an eye relief distance adjustment mechanism configured to move the left and right arms in a timed manner toward and away from the crown band along a substantially linear path, to thereby adjust an eye relief distance from the display to an eye of a the user when the crown band is fitted on the crown of the user.

Microsoft's patent FIG. 39 is an isolated perspective view of an armature; FIG. 38 is a perspective view of an armature of an HMD coupled to an HMD display.

#3: Augmented Reality Light Guide Display

Microsoft's third patent application 9,558,590 is titled "Augmented reality light guide display."

Microsoft notes that their invention relates to Augmented Reality light guide display techniques. In one or more implementations, an apparatus includes a housing configured in a hand-held form factor, one or more sensors configured to detect a position and orientation of the housing in three dimensions in a physical environment of the housing, a light guide that is at least partially transparent and supported by the housing, a light engine that is optically coupled to the light guide, and one or more modules disposed within the housing and implemented at least partially in hardware.

The one or more modules are configured to calculate a position and orientation of an augmentation and cause the light engine to output the augmentation for display using the light guide such that the augmentation is viewable concurrently with at least a portion of the physical environment through the light guide.

The input/output module #116 is also illustrated as including an augmented reality module #118 as noted in patent FIG. 1 above. The augmented reality module #118 is representative of functionality of the computing device / Surface tablet to augment a view of the physical surroundings #112 (e.g., the "real world") via a Surface tablet using the display device #110. In the illustrated example, for instance, the Surface tablet is illustrated as being physically positioned in surroundings that include three trees and fingers of the user's hand #106.

The augmented reality module is configured to output an augmentation (e.g., the car 114) to be viewed in conjunction with the physical surroundings. To generate this view and know "where" to place the augmentation, the augmented reality module may leverage a variety of techniques to determine an orientation and/or position of the Surface tablet in relation to the physical surroundings. For example, the augmented reality module may leverage a camera #120 to capture images of the physical surroundings. The augmented reality module may then process the images to locate one or more markers to determine how the Surface tablet is positioned, oriented, moved, and so on.

These markers may take a variety of forms. For instance, the augmented reality module may set one or more view points in the physical surroundings as markers and thus serve as a basis to determine orientation and/or positioning, such as where the trunks of the trees meet the ground. In another instance, the augmented reality module may leverage a view of one or more augmented reality (AR) tags that are physically positioned within the surrounding environment of the Surface tablet. Thus, the items in the physical surroundings may act as a basis to determine where Surface tablet is located as well as how it is oriented.

In the end we're able to see that Microsoft is attacking Augmented Reality on many fronts ranging from new desktop or tablet top computers, through a new AR Headset and bringing AR to Surface tablets. Microsoft knows, or at least believes that this is an area that they could beat Apple to market with.

Earlier last month Patently Apple posted a report titled "Lenovo Adopts Microsoft Platforms to Challenge Android based Headsets and Tablets in 2017 and beyond. The Lenovo HMD system will be based on the Microsoft Windows Holographic platform and it's likely to be the first of many to follow throughout 2017 and 2018 to get the ball rolling winning over developers to Microsoft's new platform over iOS which has yet to publicly address Augmented Reality.

Microsoft is running like the wind to establish a beachhead from which to launch a larger attack against Apple and their operating systems to win over developers and excite the next generation to the Windows platform and away from iOS and macOS. Whether their strategy will prevail is too earlier to call. It's really up to Apple. If they're too slow to market it's going to be harder for them to win over developers. The Augmented Reality market is going to be gathering steam this year and we're hoping that Apple won't be a no-show. Then again, Microsoft had a few year lead on Apple regarding tablets and never quite understood how to make it work. Apple came out with iPad and it became a consumer phenomenon.

So being first to market doesn't always guarantee a company much these days. Nevertheless, you can't rule out a hungry Microsoft going full tilt with the one goal in mind: beat Apple to market to get an earlier lead that they could build on and keep Apple on the periphery as a minor player. This is a focus that Microsoft missed years ago when the iPhone came out which basically brought down their entire kingdom. I think that Microsoft has learned their lesson and will fight Apple on every front going forward which starts with Augmented Reality and other fronts like Cortana based home automation systems. If anything, it'll make the coming market wars for AR and AI a lot of fun to cover over the next few years. Apple and Windows fans have never really liked each other and so the coming tech wars are going to have both camps verbally sparring with each other for some time to come. Enjoy.

About Posting Comments: Patently Apple reserves the right to post, dismiss or edit any comments. Those using abusive language or behavior will result in being blacklisted on Disqus.