In-Air gesturing is being seen by many in Silicon Valley as a future feature that will be used in conjunction with smartphones, tablets, desktops, notebooks, smartwatches and smart TVs in the not-too-distant future. Google has been working on this future feature for their devices since at least October 2014 and we covered it extensively back in 2016 when the U.S. Patent made it public. In January 2019, Patently Mobile covered Google's granted patent on using Radar for in-air gesturing system. Today's report covers Google's second granted patent on radar sensing that was issued in the last month.

In this basic overview, Google's invention describes apparatuses and techniques for radar-based contextual sensing. In some aspects, a radar sensor of a device is activated to obtain radar data for a space of interest. Three-dimensional (3D) radar features are extracted from the radar data and positional data is received from sensors. Based on the positional data, spatial relation of the 3D radar features is determined to generate a set of 3D landmarks for the space. This set of 3D landmarks is then compared with known 3D context models to identify a 3D context model that matches the 3D landmarks.

To ensure that it is understood that the focus of the patent is about radar based in-air gesturing to control devices in one way or another, we start off by presenting you with patent figures 1, 3 and 4 below making this point crystal clear.

Google's patent FIG. 1 below illustrates an example environment that includes a wide range of computing devices having radar and other sensors; FIG. 3 illustrates two example implementations of the radar sensor radar fields and a user making in-air gestures; FIG. 4 illustrates another example implementation of the radar sensor shown in FIG. 1 using a penetrating radar field that could understand in-air gestures. Google's patent FIG. 3 in particular shows a user making in-air gestures on Chrome based notebooks and desktops. FIG. 1 shows you the extent of products that radar in-air gesturing could apply to over time.  

Google's patent FIG. 5 below illustrates an example of configurations and of components capable of implementing radar-based contextual sensing; FIG. 6 illustrates an example method for augmenting radar data with supplemental sensor data.

Google's patent FIG. 9 below illustrates an example of low-power sensor fusion implemented by a smart-television that includes a sensor fusion engine. Beyond in-air gesturing, radar can be used for biometrics such as face recognition or ID and instantly bring up the users preferred TV shows or continue a show not yet finished; FIG. 11 illustrates an example method for generating a context model for a space of interest; FIG. 12 illustrates an example of a room being contextually mapped.

In patent figure 1 Google shows us that in-air gesturing could apply to future smartwatches using radar sensors. Patent FIG. 19A shown below is from a recent Samsung granted patent wherein in-air gesturing is shown to be a possible future feature with a galaxy watch controlling a future Samsung TV. Hopefully I'll find some time later this month to cover Samsung's granted patent in more detail.

Other tech companies working on in-air gesturing include Apple (01, 02 & 03) and Microsoft. Microsoft used 3D in-air gesturing with their Kinect device that began with PrimeSense technology from Israel that Apple acquired in 2013 (01 & 02).

In late December Patently Apple posted a report titled "Apple could be onboard the 2019 3D Camera trend for Smartphones that will introduce In-Air Gesture Controls." The report pointed to Sony's new 3D camera module and showed a user making in-air hand-gestures on a smartphone.  

Earlier in December Huawei pointed to a folding phone and a 3D camera coming in 2019. Huawei introduced their folding phone late last month that they branded the "Mate X" and so their 3D camera could be ready later this year that will support in-air gesturing.

At the end of the day, the in-air gesturing feature will be coming to a multitude of devices and Google is definitely in the race. They're also the only tech company known today to be focusing on using radar sensors to achieve this feature for phones, chrome computers and smart TVs.

Google's recently granted patent on radar sensing was originally filed in Q4 2016.