Today the US Patent & Trademark Office published a patent application from Apple that relates to mapping and tracking of location information with a machine learning computing system. More particularly, but not by way of limitation, this invention relates to implementing machine learning operations that monitor the health of a Visual Inertial Odometry (VIO) system, such as a Simultaneous Localization and Mapping (SLAM) system.

Computer vision methods often involve having a computing system extract and analyze digital images from a real environment to understand real world contexts and form decisions based on these contexts. SLAM technology, in particular, involves automatically determining the position and/or orientation of a device relative to a real environment and creating a scaled geometrical model of the real environment without requiring pre-knowledge of the environment.

The creation of the scaled geometrical model of the real environment can also be referred to as reconstruction of the environment. The use of SLAM technology is pertinent to many applications, such as navigation of a robot system or a mobile system, 3D reconstruction, camera pose estimation and supporting Augmented Reality (AR) applications in an unknown real environment.

SLAM technology can utilize one or more cameras and one or more inertial measurement units (IMUs) to track and update a map for an unknown real environment. The cameras are able to capture images of the real environment and the IMUs are able to capture measurements relating to the SLAM system, such as force, acceleration, angular velocity, and/or magnetic field.

SLAM technology relies on information obtained from the images and IMU data to achieve, in real-time, localization and mapping at a relatively high position accuracy. Given a set of time-constructive images and IMU data, a SLAM system is typically able to determine its position and orientation at varying points in time. Unfortunately, the accuracy of the position and orientation estimations may vary depending on a variety of factors, such as image content and types of motions the SLAM system experiences. As a result, being able to determine and notify users and/or other system components that the position and orientation accuracy estimation is accurate or has become inaccurate is valuable for computer vision-based technologies.

Apple's invention covers a method to determine health values from one or more Visual Inertial Odometry (VIO) feature measurements.

Apple's patent FIG. 2 below is a block diagram of a computing system that estimates the health of a VIO system.

To even get a grasp of what this invention is associated with, the video below is on the topic of Real-time Visual-Inertial Odometry for Event Cameras. Apple's patent as noted above states that they system could apply to the navigation of a robot system or a mobile system as well as work with AR application. The video below starting at around the 2:19 mark covers an autonomous drone (robot) in relation to Real-Time Visual-Inertial Odometry.

This isn't an easy patent to read and so unless you're an engineer in this field, you're just not going to be able to appreciate the system that Apple has invented. Apple's patent application 20190164040 that was published today by the U.S. Patent Office was originally filed back in Q1 2018. Considering that this is a patent application, the timing of such a product to market is unknown at this time.

The U.S. Patent Office also published a patent application from Apple today titled "System and Method for Interactive Scene Projection" under number 20190164344. Patently Apple first discovered this invention in Europe back in February 2018. We posted a report about it titled "A New Project Titan Patent Surfaces in Europe Covering Interchangeable Heads-Up Display Interfaces & more." Review our report here for details.

About Making Comments on our Site: Patently Apple reserves the right to post, dismiss or edit any comments. Those using abusive language or negative behavior will result in being blacklisted on Disqus.