Yesterday Patently Apple posted a report titled "Ming-Chi Kuo's Releases his latest Predictions on Apple's 2nd-Gen MR Headset, iPhone SE, Apple Watch and AirPods." Today, Ming-Chi Kuo is back with round two of his predictions, this time diving into the plethora of sensors that Apple's future mixed reality headset will include.

Kuo notes in his latest report that "Gesture control and object detection are critical human-machine UI designs of Apple's AR/MR headset. Apple's AR/MR headset is equipped with more 3D sensing modules than iPhones, and the structured light specification will be significantly upgraded." Patently Apple has covered many in-air gesture patents in 2021 alone (01, 02, 03 and 04)

(Click on image to Enlarge)

Kuo further added that "WIN semi and Lumentum are critical winners of metaverse's innovative human machine UI, thanks to the Apple AR/MR headset's adoption of more and higher specification 3D sensing modules. The innovative human-machine UI for headset devices requires the integration of many technologies.

Gesture control and object detection are critical human-machine UI designs of Apple's AR/MR headset. Apple's AR/MR headset is equipped with more 3D sensing modules than iPhones, and the structured light specification will be significantly upgraded.

We predict that Apple's AR/MR headset will have four sets of 3D sensing (vs. one to two sets for iPhone/high-end smartphones). Among them, the specifications of the structured light for gesture control and object detection will be higher than the iPhone's Face ID.

We further predict that the structured light of the AR/MR headset can detect not only the position change of the user or other people’s hand and object in front of the user's eyes but also the dynamic detail change of the hand (just like the iPhone's Face ID/structured light/Animoji can detect user's dynamic expression change).  Capturing the details of hand movement can provide a more intuitive and vivid human-machine UI. 

Although both adopt structured light, the distance between the hand (user or other people’s) and the object detected by the headset device needs to be longer than that seen by iPhone's Face ID, so the structured light power consumption of the headset device is higher, which is beneficial to the increase of VCSEL ASP.

We predict that the detection distance of Apple's AR/MR headset with structured light is 100–200% farther than the detection distance of the iPhone Face ID.

To increase the field of view (FOV) for gesture detection, we predict that the Apple AR/MR headset will be equipped with three sets of ToFs to detect hand movement trajectories with low latency requirements.

Since the calibration of ToF is critical to accurate gesture detection, the tolerance requirement for the side casing of the headset manufactured by Jabil and Goertek is very demanding.

WIN semi and Lumentum are critical winners of metaverse's innovative human-machine UI, thanks to the Apple AR/MR headset's adoption of more and higher-specification 3D sensing modules.

The main suppliers of VCSELs for the iPhone are WIN semi (production) and Lumentum (design), so we believe that the VCSELs for 3D sensing in Apple's headset will be mainly provided by WIN semi and Lumentum.

Since Apple AR/MR headset devices are equipped with more 3D sensing modules and upgraded structured light, WIN semi and Lumentum are the key beneficiaries of Apple's metaverse hardware devices.

The role of 3D sensing in the metaverse era will become much more critical. The applications of 3D sensing in the smartphone are face recognition and providing depth-of-field information for photos, which can enhance user experience but is not essential. However, 3D sensing is the key to the human-machine UI of headset devices and is a must for a successful metaverse user experience.

The innovative human-machine UI for headset devices requires the integration of many technologies. It will be critical for headsets to replace the existing consumer electronic products with displays in the next ten years."

Lastly, Kuo notes that "Apple's future headset will include innovative human-machine UI technologies such as gesture control, object detection, eye tracking [01 & 02], iris recognition, voice control, skin detection, expression detection, and spatial detection."