On Thursday the U.S. Patent and Trademark Office officially published a patent application from Apple that relates to Vision Pro integrated motion sensors. More particularly, the present embodiments relate to aligned sensors having integrated inertial measurement units (IMUs).

In Apple's patent background they note that in modern use, electronic devices are highly mobile, especially wearable electronic devices. This mobility results in an increased need for electronic components capable of monitoring a status and positioning of the device and of the components on or in the device. These considerations can be particularly relevant to devices that rely on operatively connected components, such as sensor arrays, where components need to be aligned in a certain pose or orientation relative to one another.

With the desire for more complex monitoring and functionality, there is a need for a system for monitoring self-articulating systems without vastly increasing the complexity of the systems.

Motion Sensor Integration

Apple notes in their patent filing that with head mounted devices (HMD-like Vision Pro) knowledge of camera pose is critical for sufficient tracking precision and accuracy. As devices undergo environmental and handling stresses, they may experience small but significant deformations affecting camera pose. For example, the sensor may come out of alignment through drop events, impacts, thermal changes, user forces, band tension, housing flex, etc. Methods for determining camera pose relative to other cameras is critical to ensure a quality user experience.

Apple's solution relates to integrating an IMU (An Inertial Measurement Unit) )onto a sensor module in order to determine individual pose of the sensor module, separate from the pose of the HMD. This disclosure is particularly relevant for electronic devices having stereo camera systems whose alignment needs to be understood to properly interpret the received visual data.

In some examples, a determination that the system is misaligned can be made by comparing relative changes (deltas) in the IMUs. Based on the relative changes in IMU data, the pose of each camera module can be calculated. In some examples, an IMU integrated onto/into a camera module can advantageously save space within the electronic device by freeing up the volume that would be occupied by an IMU separate from the camera module. For example, by integrating the IMU onto the camera module stack tolerances can be minimized and the number of joints between the IMU and the camera module can be reduced.

Apple's patent FIG. 1 is an HMD like Vision Pro; FIG. 2  shows a schematic block diagram of an electronic device divided in two parts being the HMD and the Sensor Unit; FIG. 3 shows a top view of a head-mountable device having aligned sensors; FIG. 4 shows a top view of a head-mountable device having misaligned sensors.

Apple's patent FIG. 7A above shows a motion sensor disposed on a lens barrel of a camera module; FIG. 7B shows a motion sensor positioned in a lens barrel of a camera module; FIG. 8A shows a motion sensor positioned on a photon detector of a camera module; FIG. 8B shows a motion sensor positioned in a photon detector of a camera module.

As usual, Apple makes a point to not limit the invention to an HMD and notes that it could apply to other devices in the future such as an iPhone, iPad, MacBook, Apple Watch and more – and apply to systems with a single camera/sensor system.

To review the full details of this invention, check out patent application  20240276098 in Safari, Chrome or Vivaldi browsers. For Edge, the link to the patent will open in a PDF. 

Another HMD patent that was published yesterday and may be of interest to some is "Lens System." Check out patent application 20240272455 for full details in Safari.