The U.S. Patent and Trademark Office officially published a series of 62 newly granted patents for Apple Inc. today. In this particular report we cover an Apple invention that generally relates to augmented reality/virtual reality (AR/VR) content consumption, and in particular, to systems, methods, and devices for calibrating a head-mounted device (HMD) for improved AR/VR content delivery.

Virtual reality (VR), augmented reality (AR), and mixed reality (MR) are becoming more popular due to their remarkable ability to alter a user's perception of the world. For example, VR, AR, and MR are used for learning purposes, gaming purposes, content creation purposes, social media and interaction purposes, or the like.

A user may experience VR content by using a head-mounted device (HMD) that encloses the user's field-of-view and is tethered to a computer.

As another example, a user may experience AR content by wearing an HMD that still allows the user to see his/her surroundings (e.g., glasses with optical see-through).

In the above examples, the HMD is typically a one-size-fits-all device. However, users of the HMD have anatomical differences (e.g., head circumference, interpupillary distance (IPD), etc.) that impact the VR experience among users when wearing one of these devices. For example, content may be displayed out of focus or off center for one user as compared to another user due to differences in the IPD between the two users.

Apple's invention covers devices, systems, and methods for calibrating a head-mounted device (HMD) for improved augmented reality/virtual reality (AR/VR) content delivery.

According to some implementations, the method is performed at a device with one or more processors, non-transitory memory, an image sensor, and a depth sensor.

The method also includes: determining an interpupillary distance (IPD) measurement for a user based on a function of depth data obtained by the depth sensor and image data obtained by the image sensor.

Next, calibrating a head-mounted device (HMD) provided to deliver augmented reality/virtual reality (AR/VR) content by setting one or more presentation parameters of the HMD based on the IPD measurement in order to tailor one or more AR/VR displays of the HMD to a field-of-view of the user.

Apple's patent FIG. 1 below is a block diagram of an example operating environment 100. In some implementations, the controller #110 is configured to manage and coordinate an augmented reality/virtual reality (AR/VR) experience for the user. In some implementations, the controller includes a suitable combination of software, firmware, and/or hardware.

The HMD #120 presents an augmented reality/virtual reality (AR/VR) experience to the user while the user is virtually and/or physically present within the scene 105. In some implementations, while presenting an augmented reality (AR) experience, the HMD is configured to present AR content and to enable optical see-through of the scene #105. In some implementations, while presenting a virtual reality (VR) experience, the HMD is configured to present VR content.

In some implementations, the calibration device #130 is configured to optionally determine an interpupillary distance (IPD) measurement for the user. In some implementations, the calibration device is configured to optionally calibrate one or more presentation parameters of the HMD based on the IPD measurement for the user in order to tailor the one or more AR/VR displays of the HMD to a field-of-view of the user. In some implementations, the HMD is configured to auto-calibrate its one or more presentation parameters by determining the IPD measurement.

In patent FIG. 5 below, the calibration device captures a "selfie" of the user using one or more front facing image sensors of the calibration device. the device identifies a face #505 (or a region thereof) within the image data #500. For example, the device identifies the face based on known facial recognition or computer vision algorithms or techniques. In some implementations, the device identifies positions 510a and 510b of the eyes.

Apple's patent FIG. 6 above illustrates example depth data. In some implementations, the device identifies at least one reference point within the depth/mesh map #600 and determines a depth measurement based on a distance from the device to the at least reference point.

As one example, the device identifies the eyes #610a and/or #610b as reference points and determines a distance from one or both of the eyes to the device as the depth measurement.

In another example, the device identifies the tip of the nose #615 as a reference point and determines a distance from the tip of the nose to the device as the depth measurement. In yet other example, the device identifies the chin #620 as a reference point and determines a distance from the chin to the device as the depth measurement. In yet other example, the device identifies the bridge of the nose #625 as a reference point and determines a distance from the bridge of the nose to the device as the depth measurement.

Apple's patent FIG. 7 is a flowchart representation of a method #700 of determining an interpupillary distance (IPD) based on image and depth data in accordance with some implementations.

Apple's granted patent 10,820,795 was originally filed in Q2 2018.