Today the U.S. Patent and Trademark Office officially granted Apple a patent that relates to a relative inertial measurement system with visual correction. Such a system is used to compensate for oculovestibular mismatch within a headset which could cause nausea when what's occurring in the VR Headset is not matching what's occurring in the real-world environment in respect to motion.

For instance, if you're playing a game in a VR headset while you're sitting in a passenger seat of a car or bus, an oculovestibular mismatch could occur when a vehicle turns and what you're experiencing in the game doesn't match up directionally. For many people, this will cause nausea. Apple's invention is to compensate for this mismatch within the headset.

Patent Background: Oculovestibular Mismatch

Conventional virtual reality and augmented reality systems may not be able to separate motion of a user or a body part of a user from motion of a reference frame in which the user is travelling, such as a vehicle in which the user is travelling.

For example, a user wearing a conventional VR or AR device may be seated in a vehicle and the vehicle may accelerate from a stopped position to a high speed while the user wearing the VR or AR device sits in the vehicle without moving within the vehicle (e.g. no relative motion of the user relative to the reference frame of the vehicle). Because the conventional VR or AR device cannot separate the motion of the user's body from the motion of the vehicle, the conventional VR or AR device may attribute the motion of the vehicle to the user. In this way, images displayed to the user on the VR or the AR device may appear to the user as if the user is running through a scene at the same speed and in the same direction the vehicle is travelling.

A similar phenomenon occurs in regard to angular motion. For example, a user wearing a conventional VR or AR device may be riding in a vehicle that turns, however the user may not actually turn the user's head when the vehicle turns. A conventional AR or VR device may not be able to separate the motion of the user's head (e.g. not turning) from the motion of the vehicle (e.g. turning). Therefore, the turning motion of the vehicle may be attributed to the user and images displayed to the user on the VR or AR device may appear to be turning or spinning despite the user not turning the user's head. Such discrepancies between a user's relative motion within a vehicle and motion observed by the user via a scene displayed to the user may lead to nausea and sickness of the user.

For example, nausea may be caused by oculovestibular mismatch. Likewise, in the case of other types of devices that include IMUs, such as controllers, motion of a vehicle being attributed to the controller may lead to erratic control and unintended consequences.

Relative Inertial Measurement System with Visual Correction

Apple's invention covers a system, user device, and method that implement relative inertial measurement technology to determine relative motion of a user device relative to a non-fixed reference frame, such as a vehicle in which the user device is travelling.

In some embodiments, relative inertial measurement technology may be used to determine images to be displayed to a user via a user device comprising a head mounted display based on relative movements of the user's head while the user is wearing the head mounted display and while the user is travelling in a non-fixed reference frame, such as a vehicle. In some embodiments, relative inertial measurement technology may be used with a controller coupled to a user's body to determine motion of the user's body relative to motion of a non-fixed reference frame in which the user is travelling, such as a moving vehicle.

In some embodiments, a relative inertial measurement system includes at least two inertial measurement devices. A first inertial measurement device may be mechanically coupled with a user device. For example, a user device may be a head-mounted display of a virtual reality system or of an augmented reality system.

The first inertial measurement device may be mechanically coupled to a part of the head-mounted display. In this way, the first inertial measurement device may move with the part of the user's body to which the user device is coupled and may measure movement of the user device due to movement of the part of the user's body to which the user device is coupled. For example, a first inertial measurement device included in a head-mounted display may measure movement of a user's head.

In the case of a user device that is moving in a non-fixed reference frame, such as a vehicle, a first inertial measurement device included in the user device may measure both the movement of the non-fixed reference frame (e.g. the vehicle) and the movement of the user device within the non-fixed reference frame.

For example, if a user is wearing a head-mounted display that includes the first inertial measurement device and the user turns the user's head to the right 20 degrees at the same time a vehicle in which the user is riding makes a right hand turn of 90 degrees, the first inertial measurement device may measure inertial movement of 110 degrees (e.g. 90 degrees due to the vehicle turning and 20 degrees due to the user turning the user device coupled to the user's head within the vehicle).

Apple's patent FIG. 16 below illustrates an example of a head mounted user device configured to determine relative inertial motion; FIG. 1 illustrates a user riding in a moving reference frame (vehicle) and a system including multiple inertial measurement devices for determining relative motion of the user relative to the reference frame.

Apple's patent FIGS. 13A-13C and 14A-C below illustrate examples of relative motion of a user device relative to a reference frame.

Apple's patent FIG. 10 below illustrates multiple user devices in a common reference frame providing motion information for determining motion of the reference frame; FIG. 15 illustrates a block diagram of an example inertial measurement device.

In some embodiments, such as patent FIG. 10 above, a user device may be located in a public transportation vehicle (bus) that includes an inertial measurement device. The public transportation vehicle may broadcast inertial measurements of the public transportation vehicle to riders of the public transportation vehicle.

For example, if a subway car (not shown) follows a set path, the subway may broadcast predicative inertial data about an upcoming bend in the track before actually passing through the bend in the track.

This is a deep patent that will likely be more appreciated by engineers. To review the details, see Apple's granted patent 10,948,299. This granted patent was never made public with a patent application under Apple. Meaning that the invention was hidden from public view as an Apple patent by keeping the patent under one or all of the inventors. When such a patent is granted, Apple must take credit for it at that point.