Yesterday the U.S. Patent and Trademark Office officially published a series of 55 newly granted patents for Apple Inc. In this particular report we briefly cover an eye tracking system; two Apple Pencil inventions illustrating possible new future features; an invention regarding sound processing for VR environments and more. And as always, we wrap up this week's granted patent report with our traditional listing of the remaining granted patents that were issued to Apple today.

Eye Tracking System

In this first granted patent, Apple's eye tracking invention covers dual eye tracking cameras and an illumination source that emits infrared light towards the user's eyes, and diffraction gratings located at the eyepieces. Our report covers only the basics of the system. For greater details you'll have to review the patent filing that we link to at that bottom of the report.

So, what are Apple engineers focusing on in this invention that could be coming to Apple's future HMD? It starts with patent FIG. 9 below which illustrates a side view of an example Mixed Reality headset (HMD #200) that implements an eye tracking system. The eye tracking system may, for example, be used to track position and movement of the user's (#290) eyes (#292).

In some embodiments, the eye tracking system may instead or also be used to track dilation of the user's pupils, or other characteristics of the user's eyes.  

For more on this you could review our original 2021 patent application report on this or review Apple’s granted patent 11,360,557 issued yesterday.

Two Apple Pencil Patents

Apple was granted two Apple Pencil patents yesterday. The first is patent #11,360,616 titled “Stylus with touch input and compressive force sensors.” The patent covers Apple Pencil with new touch input and compressive force sensors.

Today’s Apple Pencil offers “double-tap” to change tools. Apple’s patent illustrates that they’re considering the addition of “Slide” and “Compressive Force” inputs in the future as presented below.

Apple’s second Apple Pencil granted patent 11,360,560 is titled “Touch-based input device with haptic feedback.” Patent FIG. 2 above relating to this patent illustrates a possible future button added to Apple Pencil.

Method & Device for Sound Processing for VR

Another granted patent of interest published yesterday by the U.S. Patent Office relates to sound processing for a synthesized reality/VR setting, and in particular, to systems, methods, and devices for transforming a real sound into a virtual sound based on an acoustic reverberation property of a synthesized reality setting.

In Apple’s patent FIG. 5 below represents a user viewing a character in a VR game that’s in a cave. In particular, it may be expected that a cave would introduce a large amount of reverb, particularly as compared to the amount of reverb in a small indoor room. Accordingly, in various implementations, in order to produce a more immersive experience, the HMD records the real sound produced in the physical setting, generates a virtual sound by transforming the real sound to change (e.g., increase) the amount of reverb associated with the real sound, and plays the virtual sound to the first user. Put another way, as the first user speaks, his HMD picks up his voice, introduces reverberation consistent with what might be expected of the SR setting, and plays back his voice with the additional reverb in a speaker of the HMD (as well as the HMDs of other users who are participating in the same SR setting), thereby increasing the level of realism of the simulated reality experience.

(Click on image to Enlarge)

The patent provides other scenarios relating to being on a beach and hearing waves crash, the sound of leaves crackling in the fall, and more.

Apple’s patent covers devices, systems, and methods for sound processing for an SR setting. In various implementations, the method is performed at a head-mounted device located in a physical setting and including one or more processors, non-transitory memory, a microphone, a speaker, and a display.

The method includes displaying, on the display, an image representation of the SR setting including a plurality of surfaces associated with an acoustic reverberation property of the SR setting. The method includes recording, via the microphone, a real sound produced in the physical setting. The method further comprises generating, using the one or more processors, a virtual sound by transforming the real sound based on the acoustic reverberation property of the SR setting. The method further comprises playing, via the speaker, the virtual sound.

For those wanting to explore the deep technical details of granted patent 11,363,378, click here.

The Remaining Patents Granted to Apple Today