In March we learned that Facebook was able to make a deal with Plessey to build AR smartglasses because Apple's offer to acquire the company outright failed. Under the terms of the Facebook deal, Plessey will license its AR tech to Facebook and dedicate its entire production to the social media firm for the next "several" years.

Facebook said it plans to build a “glasses form factor that lets devices melt away so we can be more present with our friends, families and surroundings” and that the project will take 'years' to complete."

Plessey specializes in manufacturing microLED displays for AR headsets and heads-up-displays. The company was founded in 2010 and originally focused on building high-powered lighting, but in 2017 made a complete pivot to the microLED display market.

It’s that last detail that seems to be critical. MicroLED is an emerging new display technology that many believe is superior to existing LCD and OLED technologies. MicroLED is similar to OLED in that it’s self-emissive, but the main advantage is that it’s much brighter and more power-efficient. As such, MicroLED is believed to be ideal for AR devices, which should be usable even in direct sunshine, yet equipped with a small battery.

The company last year achieved a world first when it built a 1080-pixel, monolithic microdisplay with individually addressable MicroLEDs. At the time, it said the approach should enable it to build displays faster and more cheaply than the alternative method of bonding individual MicroLEDs to a substrate. It can also make 4K resolution displays of a similar size using the same technique, it said.

That may explain why Apple was also interested in getting a piece of Plessey. According to a report in The Information Monday, the iPhone maker apparently tried to acquire the company outright to further its own interest in AR, but failed to agree on terms.

With that said, Apple apparently moved on and invested over a billion dollars in a new AU Optics factory in May 2020 that will be dedicated to making next-gen micro-LED displays for Apple products including HMDs, Smartglasses, iPad Pro, Macs and more.  

Analyst Patrick Moorhead recently told SiliconANGLE that "Facebook missed the market for deep integration with smartphones, which is one reason it got an early start in AR by buying Oculus."

This is something that Patently Apple covered in a December 2019 report titled "Facebook has been working on Smartglasses to Replace Smartphones that includes a secret new OS to make it happen."

Our report back then noted that Andrew Bosworth, Facebook’s head of hardware told The Information news site that "We really want to make sure the next generation has space for us" referring to their future OS and added that "We don’t think we can trust the marketplace or competitors to ensure that’s the case. And so we’re going do it ourselves."

Facebook is on a double track with their headsets. On one track they're continuing to work on their Oculus Head Mounted Display products while pushing very hard to beat Apple and other potential rivals to market with a mode of AR Glasses that are cool and powerful enough to actually replace today's mobile device winner, the smartphone. If Facebook could win this race, they will be able to drive their future in directions that they want and not having to fight to get onto other platform devices.

Going forward, Patently Apple will attempt to keep Apple fans up to date on patents relating to both Headsets and Glasses from Apple and their competitors. I may not be able to stay on track of all of the patents simply because Facebook is pushing patents out the door with great speed and in great volume, with many covering very detailed features that are well over the heads of the general public. Most of the patent reports covering Facebook patents will mainly be covered on our Patently Mobile IP blog. 

The first patent in this series covers Smartglasses using structured light depth sensing.

1:  Smartglasses - Structured Light Depth Sensing

A variety of techniques for range and depth sensing have been developed. For example, a stereo triangulation technique includes imaging an object with two different cameras and determining a distance to the object using corresponding points in the images from the two cameras. Time of Flight (TOF) is another existing technique that may include transmitting laser pulses and measuring the time of flight between the transmitted and reflected laser pulse to calculate the depth of an object that reflected the laser pulse. However, these techniques are limited by the context they are deployed in. TOF techniques, for example, struggle in very near-range contexts because resolving the short time of flight of the pulses is difficult.

Facebook's patent FIG. 1A is a smartglasses example that includes a camera and waveguides for generating phase-shifted interference patterns for near-eye depth sensing; FIG. 6 illustrates an example flow chart for a method of near-range depth sensing.

For Patent FIG. 1A above, Facebook notes that lenses #121A&B may appear transparent to a user to facilitate augmented reality or mixed reality where a user can view a scene light from the environment around them while also receiving a image light directed to their eye(s) by waveguide(s) #150. Consequently, the lenses may be considered (or include) an optical combiner [of Reality & Augmented Reality].

In the embodiment above, image light is only directed into one eye of the wearer of HMD #100. In an embodiment, both displays #130A and #130B are included to direct image light into waveguides #150A and #150B, respectively.

For more on this, review Facebook's granted patent 10,607,353 that was published in March 2020.

The second report in this series is titled "Facebook wins a patent for an Audio System Designed for future Smartglasses."  Those interested in learning about this Facebook patent could do so here.

Facebook continues to both file patent applications and receive granted patents weekly in respect to HMDs and smartglasses technology. If this subject matter is of interest to you,  then be sure to bookmark our other IP blog called Patently Mobile.

We know that Mixed Reality HMDs and smartglasses will be one of the next battlefields in the industry in the coming years and staying on top of what Apple's leading competitors are up to will help us better understand what Apple will be up against. For those not familiar with Apple's intellectual property on future eyeware and related accessories, check out our archives here.