Microsoft wins Patent for Next-Gen HoloLens that may use a new Depth Camera System Similar to Apple's TrueDepth Camera
Today the U.S. Patent and Trademark Office granted Microsoft with a patent titled "Depth map with structured and flood light." The patent illustrates that a future version of the HoloLens will integrate a new depth camera system that is very much like Apple's TrueDepth camera.
Microsoft used to use depth camera technology for their Kinect Device from Israel's PrimeSense. When Apple acquired PrimeSense, Microsoft had to re-design their bulky depth camera system that mirrored what PrimeSense provided. That's why the depth camera system described in Microsoft's granted patent mirrors Apple's TrueDepth camera with a 'Flood illuminator and Dot projector as you'll see in the patent figures further below.
Microsoft's granted patent covers a method that includes receiving an image of a scene illuminated by both a predetermined structured light pattern and a flood fill illumination, generating an active brightness image of the scene based on the received image of the scene including detecting a plurality of dots of the predetermined structured light pattern, and removing the plurality of dots of the predetermined structured light pattern from the active brightness image, and generating a depth map of the scene based on the received image and the active brightness image.
While the Patent focuses on the new depth camera being for HoloLens, Microsoft doesn't limit the camera to that single device and lists other devices that could adopt this camera system such as a mobile phone (Surface Duo), a tablet (Surface Neo), a Surface notebook, a Surface desktop (Surface Studio), gaming consoled (Xbox Series X) or even a future smartwatch.
Apple's TrueDepth camera and ARKit 3 software foundation Apple is able to do some amazing things like their new People Occlusion which blurs the line between the real world and AR content. The imagery is sharp and amazing and their next-gen 3D TrueDepth camera could be introduced in Q4 of this year.
Instead of the imagery being applied to an iPhone or iPad display, Microsoft has designed their new depth camera system to work on HoloLens displays.
Microsoft's patent FIG. 1 below shows a side perspective view of a future computing device in the form of a head mounted display (HMD) device such as HoloLens. This next-gen HoloLens could include a new center depth camera very much like the TrueDepth Camera found on the iPhone since iPhone X as presented below.
Microsoft's patent FIG. 9 below shows another example depth sensor of the computing device of FIG. 1; FIG. 11 shows another example method for generating a depth map using the depth sensor of the computing device of FIG. 1.
Microsoft's patent FIG. 12A shows an example image captured by the camera 24 of the depth sensor of the computing device of FIG. 1; FIG. 12B shows an example active brightness image generated based on the image captured by the camera #24 of the depth sensor of the computing device of FIG. 1.
The U.S. Patent office granted Microsoft patent 10,535,151 today titled "Depth map with structured and flood light," that you can review in full here for more details.
While Microsoft has backed away from the consumer market and refocused HoloLens for industrial applications and the enterprise market, today Facebook's spokeswoman Ha Thai, said the company is "investing significantly in AR, VR and home hardware products" that appears to slant more towards the consumer side of the headset market.
Thai further stated that "At some point in the 2020s, we will get breakthrough augmented reality glasses that will redefine our relationship with technology."
Zuckerberg said in a post on Facebook this month. “Even though some of the early devices seem clunky, I think these will be the most human and social technology platforms anyone has built yet."
In late December Patently Apple posted a report titled "Facebook has been working on Smartglasses to Replace Smartphones that includes a secret new OS to make it happen." This is likely the "breakthrough" that Ha Thai was referring to.
Apple is likely to challenge Facebook in the future as their work to date is showing up in the form of patents. It's an ongoing project with Apple's various R&D teams and it's quite impressive to see the depth of the work that is being done month in and month out. You could review Apple's many patents in our archives on future headsets here.
In the end, Apple beat Microsoft to market with miniaturization of a true depth camera for a compact mobile device like the iPhone that was never done before successfully and in a mass market product.
While Apple's first headset is stuck in patents and Rumorville for now, we know that when it arrives it's likely to incorporate a next-gen TrueDepth Camera.