This week the US Patent & Trademark Office published a patent application from Apple that relates to a possible future generation of Group FaceTime conferencing for iPhone, iPad, Macs and their Mixed Reality Headset. This future version will integrate gaze technology into their hardware camera systems in a way that will directly engage participants in a unique way.

Today, when a participant is speaking in a FaceTime call, their image is enlarged on-screen. But the person that they may be addressing stays static looking forward on the display as all other participants. In this future version of FaceTime, the speaker and/or presenter may be addressing a single person in conversation and the target participant, captured via gaze, will appear to turn their head to face the speaker. In some examples, like when wearing a mixed reality headset, this is done virtually.  

Textually, this is an extremely complex patent application to get your head around and so we'll skip right to patent FIG. 6A that gets us to the heart of the invention. Here we see 'User A' on his MacBook looking at and speaking to 'User B.' This version of FaceTime will support 'Gaze Controls." Once the MacBook's camera captures the gaze direction and which FaceTime participant 'User A' is targeting,  the image of the targeted 'User B' will be seen as turning their head to look at and address 'User A."

Apple's patent FIG. 6G below provides a look at how this could play out on other devices.  

Apple's patent FIG. 10a below presents an undescribed "computer system,' (which could be an iMac or a much larger system for lecturers and teachers in virtual classes) with 8 cameras surrounding the display. This is designed to better capture the accuracy of the user's gaze at a participant in the conference call.

Depending on which row the speaker is targeting, a different camera is activated to better capture gaze accuracy within that row. The software will then position the head of the specific user targeted, to appear as if they turned their head in the direction of the speaker addressing them specifically.

In Apple's patent FIG. 11 above, Apple describes how multiple cameras within their future Headset will be able to capture the user's face for a FaceTime conference call and present a virtual head that could turn in direction of the one speaking at any given time.  

In some embodiments the first participant is determined to be a presenter when the first participant is determined to be actively communicating or communicating above a threshold amount in the live video conference.

In some embodiments, the first set of criterion includes the criterion that is satisfied when the first participant is determined to be a presenter in the live video conference when the live video conference is in a presenter (e.g., screen sharing, teacher, lecturer, etc.) mode.

Without a doubt, Apple is planning to take FaceTime or Group FaceTime conferencing to the next level and when we see it in action we'll better understand the totality in what Apple is describing in the patent filing. For now, the patent is a bit of a mind-bender. To review this extremely complicated invention, review Apple's patent application number 20220286314.

Although there are four inventors listed on the patent, we were only able to find one in LinkedIn, James Foster. He's been on Apple's industrial design team for over 12 years.