Earlier today Patently Apple posted a report titled "The Digital Assistant War for Home Automation Device Supremacy is about to get Really Heated." The report was partly based on a new Reuters report pinning Apple versus Amazon for home automation supremacy. The Reuters report failed to put Microsoft in this battle even though it's widely known that Harman Kardon will be releasing an Echo-like device powered by the Microsoft digital assistant Cortana later this year. Better yet the U.S. Patent & Trademark Office published a patent application from Microsoft yesterday that revealed a very sophisticated, next generation home automation device that is driven by Cortana, Microsoft's digital assistant that in my opinion is miles ahead of the Echo – if they get it to market within a reasonable time frame.

Microsoft's home automation system could leap frog Amazon's Echo driven by Alexa and Google Home driven by Google Assistant. Way beyond voice activation, Microsoft's system will offer a highly sophisticated 3D depth camera in tune with a pico projector to allow video conferencing to occur in any room in your home or office as the imagery could be projected onto a wall instead of a computer display as noted in our cover graphic.

Microsoft's home automation system could be set up in a user's home or office and act as a form of video-intercom. The system could be set up in a home in various configurations such as a device like the Amazon Echo or on the wall or ceiling so that a conversation could take place while a user is moving from room to room.

Overview of Microsoft's Patent Background

Microsoft notes that screen-based desktop videoconferencing, screen-based video presentations and screen-based digital television are now mature technologies, but the requirement for a screen is taxing in terms of space requirements, mobility, and cost. Using smart phones or tablet computers for mobile video conferencing allows for mobility but introduces the difficulty of holding a screen while moving, small image size and difficulty multi-tasking.

More recently, augmented reality has become available as a way of presenting video or other image content. However, augmented reality headsets can be restrictive in some situations and can be difficult to use by novice users, or those with physical disabilities, the elderly and the very young. In the case of augmented reality video conferencing, augmented reality headsets impede the view of the face of both participants if both use headsets, or one participant if that person is not wearing a headset.

Microsoft's remedy is found in one of their latest published inventions that relate to a portable projection unit which has a rotating capture module and a rotating projection module.

The capture module has at least one color camera, at least one microphone and at least one depth camera and is configured to capture images of an environment of the projection unit.

Microsoft's patent FIG. 2 as noted above is a schematic diagram of a room with a projection unit #100 on a table. The projection unit is seen projecting an image #204 onto a wall in the room. In this case the image is of a person #206 who is a remote party of a video call ongoing between the projection unit and another communications device such as a smart phone, laptop computer, meeting room video conferencing equipment, tablet computer or other video conferencing device.

The projection unit is capturing images of the seated user as indicated by triangle #202 and transmitting the captured images to the remote party. In this way screen-less video conferencing is enabled in a simple manner without the need for augmented reality headsets.

The size and shape of the projection unit and the ability to dynamically capture and dynamically project, enable the user to operate the projection unit whilst they carry on a natural conversation eye to eye. The user looks over the projection unit to see the images of the remote participant.

In Microsoft's patent FIG. 3 we're able to see a perspective view of the projection device (unit 100). The projection device is seen as cylindrical and comprises a plurality of cylindrical modules #300, #302, #304, #306, #308 in a stacked arrangement. It's overall appearance is something like Amazon's Echo.

One or more of the cylindrical modules are rotatable about the axis of the cylinder. In some examples, at least two of the cylindrical modules are independently rotatable with respect to one another, about the axis of the cylinder. The projection unit has a flat base which can rest on a table top and, in some examples, is sized and shaped to be portable. The cylindrical design of FIG. 3 is an example only; other designs of projection unit can be used.

The projection unit 100 also comprises a rotating projection module #304 comprising at least one pico projector and optionally comprising an optical arrangement comprising mirrors and/or lenses configured to optically move light projected by the pico projector.

In some examples the rotating projection module is rotating by virtue of the fact that the whole module rotates about its center. In some examples the rotating projection module has the property of rotating by virtue of the fact that the projected light it emits is rotatable with respect to the pico projector by using the optical arrangement. In some examples the rotating projection module rotates about its center and also has an optical arrangement for rotating the projected light with respect to the pico projector.

The projection device comprises one or more loudspeakers. In the example of FIG. 3 cylindrical component #302 is a 360.degree speaker section comprising one or more loudspeakers. In some examples the projection device has one or more loudspeakers where the direction of sound output can be controlled such that the sound appears to be originating from a specified location in the environment.

The projection device comprises a rotating capture module which may be housed in cylindrical component #300. The rotating capture module comprises at least one depth camera, at least one color camera, and at least one microphone.

Microsoft filed their patent application back in July 2015. Considering that this is a patent application, the timing of such a product to market is unknown at this time.

For more on this invention, see our full Patently Mobile report here.