Last week Patently Apple covered a Project Titan project patent from Apple that we covered in a report titled "Apple Invents 'Adjustable Exterior Lighting' Systems for future Semi and fully Autonomous Vehicles" This morning we covered another two Project Titan patents regarding advanced adjustable tinting windows for semi and fully-autonomous vehicles.

The last Project Titan patent for this week covers a deep dive into processing graph representation of tactical maps using neural networks. In other words, taking a look at the sophisticated brain center of systems and algorithms for reasoning, decision making and motion planning with respect to controlling the motion of autonomous or partially autonomous vehicles. Apple also covers training a fleet of autonomous vehicles.

Motorized vehicles which are capable of sensing their environment and navigating to destinations with little or no ongoing input from occupants, and may therefore be referred to as “autonomous” or “self-driving” vehicles, are an increasing focus of research and development. Until relatively recently, due to the limitations of the available hardware and software, the maximum speed at which computations for analyzing relevant aspects of the vehicle's external environment could be performed was insufficient to enable non-trivial navigation decisions to be made without human guidance. Even with today's fast processors, large memories, and advanced algorithms, however, the task of making timely and reasonable decisions (which are based neither on excessively pessimistic assumptions, nor on excessively optimistic assumptions) regarding an autonomous vehicle's trajectory in the context of unpredictable behaviors of other entities (such as other drivers or other autonomous vehicles) and incomplete or noisy data about static and dynamic components of the vehicle's environment remains a significant challenge.

Processing Graph Representations of Tactical Maps using Neural Networks

Apple's patent covers various embodiments of methods and apparatus for analyzing graph representations of tactical maps for autonomous vehicles using neural network-based machine learning models.

According to some embodiments, a tactical map may comprise information about various static components of a vehicle's operating environment, such as road lane segments, intersections, and so on. The information included in a tactical map may indicate attributes or properties of individual static components as well as various types of relationships (e.g., geometric or topological relationships) that may exist among the static components.

From the raw tactical map, a homogenized graph representation may be generated in various embodiments, suitable for processing by a neural network model which has been trained to perform reasoning on graphs or graph-like data structures.

In the homogenized graph, nodes may represent instances of the static components, and edges may represent relationships (with respective edge types denoting relationships with different semantics). The graph may be considered to be homogenized in various embodiments in that individual nodes may be represented as having the same number of edges with the same edge types as other nodes, arranged in the same order, within the graph representation. In effect, in such embodiments, missing edges among nodes may be represented using connections from such nodes to a special “zero” node.

Such homogenization may help to simplify some of the computations performed at the neural network (e.g., by reducing the number of distinct parameters which have to be learned) in some embodiments. The results of the analysis of the graph corresponding to a tactical map may, for example, be combined with results of analyses of other environment components (such as moving vehicles in the vicinity, pedestrians and the like) to make decisions regarding possible or advisable future motions of the vehicle in some embodiments. Such decisions may be implemented by sending the appropriate motion control directives to various subcomponents of the vehicle (such as braking subsystems, turning subsystems, accelerating subsystems, and the like) in such embodiments.

Apple's patent FIG. 1 below illustrates an example system environment in which graph representations of tactical maps representing static components of an autonomous vehicle's environment may be analyzed using neural network-based models to help direct the movements of the vehicle.

(Click on image to Greatly Enlarge)

Apple's patent FIG. 2 below illustrates an example decision making scenario for an autonomous vehicle, as well as general problem characteristics associated with such decision making scenarios; FIG. 3 illustrates an example overview of the processing of tactical maps at an autonomous vehicle.

Apple's patent FIG. 5 below illustrates an example neural network architecture which may be used to process homogenized graph representations of tactical maps; FIG. 10 illustrates an overview of example stages of developing, deploying and using machine learning models for autonomous vehicles.

For those interested in Apple's Project Titan patents, check out the deep details behind Apple's latest patent application #US 11555706 B1.

Apple Inventors

• Juergen Wiest: Engineering Manager at Apple, SPG, Autonomous Systems, Machine Learning, AI. (Formerly worked at Daimler, Mercedes-Benz Group AG, Germany)
• Martin Levihn: Engineering Manager, Special Project Group (SPG)
• Tapani Raiko: Autonomous Systems, Principal Research Scientist
• Anayo Akametalu: No LinkedIn Profile was found.