Back in June 2015 Patently Apple posted a report titled "Apple Acquires Metaio Creator of Thermal Touch - A New Augmented Reality Interface for Wearables." One of Metaio's greatest inventions that Apple has acquired relates to "Thermal Touch" which is a vision of the near future for wearable computing user interfaces. By fusing information from an infrared and standard camera, nearly any surface can be transformed into a touch screen.

The technology is possibly intended for use with Apple's future head-mounted displays (MR headset or glasses) and makes use of the fact that touching an object heats it up slightly and provides a thermal signature. When combined with overlaid information in the display it allows everyday objects to become controls or for controls to be overlaid on surfaces.

To get a greater understanding of the possibilities of this technology, check out the original Metaio YouTube video on Thermal Touch below. The inventor Daniel Kurz, seen below, is now Apple's Senior Machine Learning Manager.

Apple's patent FIG. 2 below shows an example of a thermal image mapped to grayscale for visualization which may be used in connection with the present invention.

Apple's patent FIG. 4 FIG. 4 shows a thermal image similar as the one shown in FIG. 2, but which has been discretized and visualized with isolines; FIG. 5 shows a sequence of infrared thermal images illustrating a finger touching an object and removing the touch with heat registering different signatures.

Apple's patent FIG. 9 below illustrates two exemplary hardware setups that could be used in connection with the present invention. The method described in this patent filing could be used with an iPhone or iPad and some form of head mounted device in the form of a VR headset or future smartglasses that's equipped with a thermographic camera #902. The device may further comprise at least one visible light camera #903, an infrared camera #904, and/or an infrared or visible light projector #905.

Apple's patent FIGS. 10 and 11 above illustrate embodiments of a human-computer interface.

Today's patent is considered a "continuation-in-part" application ("CIP" or "CIP application") which is one in which the applicant adds subject matter not disclosed in the parent patent application, but repeats a substantial portion of the parent's specification, and shares at least one inventor with the parent application. The CIP application is a convenient way to claim enhancements developed after the parent application was filed.

In this case, the "one inventor" is Daniel Kurz, Apple's current Senior Machine Learning Manager, who came to Apple via the 2015 Metaio acquisition. In respect to "adding subject matter," Apple has cancelled the original patent's claims and replaced them with 20 new claims so as to conform to how Apple see's the invention playing out.

I believe that this Thermal Touch patent acquired from Metaio was one of the jewels that Apple wanted for their future headset. This is definitely a patent filing that software engineers and developers will love to dig into for rich details here. Continuation in-part patent application 20210117042 was filed in December 2020. Some of the original Thermal Touch patent filings date back to 2014.

Considering that this is a continuation patent, the timing of such a product to market is unknown at this time.