DE112020006317T5 - COMPUTER SYSTEM AND METHOD FOR TRAINING A TRAFFIC AGENT IN A SIMULATION ENVIRONMENT - Google Patents

Abstract

The present invention relates to a computer-implemented training method for a traffic agent who navigates a road vehicle in a complex driving situation in a simulation environment, and a corresponding computer system, a computer system for simulating a road driving environment in complex driving situations for one or more vehicles, which includes one or more processors or consists of it consists, using a traffic agent trained according to the invention and the use of the computer-implemented training method according to the invention for training a traffic agent who navigates a road vehicle in a complex driving situation of a simulation environment.

Description

TECHNICAL PART:

The present invention relates to a computer-implemented training method for a traffic agent who navigates a road vehicle in a complex driving situation in a simulation environment, and a corresponding computer system, a computer system for simulating a road driving environment in complex driving situations for one or more vehicles, which includes one or more processors or from it consists, using a traffic agent trained according to the invention and the use of the computer-implemented training method according to the invention for training a traffic agent who navigates a road vehicle in a complex driving situation of a simulation environment.

STATE OF THE ART:

Before the driving characteristics of road vehicles are tested in reality, computer simulations of certain driving situations, e.g. B. when braking performed. Since the forecast period is usually only up to 2 seconds, complex driving situations, such as those B. are required when overtaking, are not predicted by these models.

The problem of developing a system that can safely control a car in a variety of traffic situations has been extensively studied and is of apparent interest for the development of autonomous vehicles. The main focus in this research area is on safe and efficient decisions under real-time conditions. However, the simulated safe and efficient decisions may not reflect human driving decisions in natural traffic.

It is therefore becoming increasingly important to simulate human driving decisions in natural traffic. Human driving decisions in natural traffic are also influenced by many factors and can be viewed at different levels. For example, human drivers may make different decisions in the same situation depending on their mental environment, such as B. overtaking, following a vehicle ahead or changing lanes.

Many existing models use a hierarchical structure such that more abstract decisions (e.g. which route to take) are first computed and then “propagated” to different layers which, based on that input, delve into increasing detail of the driving process . The driving stack is divided into several phases, which are intended to reflect the actually relevant components of the different approaches, e.g. B. in connection with simulation environments and not with the driving behavior of an autonomous vehicle.

• Wahrnehmung/Karte bezieht sich im Allgemeinen auf die Eingabe über die Umgebung, der anderen Komponenten zur Verfügung steht.
• Verkehrsregeln beziehen sich im Allgemeinen auf jede Komponente, die rechtliche Einschränkungen für Entscheidungen auf hoher Ebene vorsieht.
• Einsatzplanung bezieht sich im Allgemeinen auf eine Strategie, wann man sich langfristig wo aufhält (z. B. Routenplanung auf Fahrspurebene).
• Verkehrsfreie Referenzstrecke bezieht sich im Allgemeinen auf die Planung einer „optimalen“ Referenzstrecke, die andere Verkehrsteilnehmer ignoriert.
• Verhaltensplanung bezieht sich im Allgemeinen auf die Planung eines Verhaltensplans, d. h. wann genau Handlungen, wie z. B. Fahrspurwechsel, unter Einbeziehung anderer Teilnehmer durchgeführt werden sollen.
• Bei der Entscheidungsnachbereitung geht es im Allgemeinen darum, die Entscheidungen der vorangegangenen Komponenten zu korrigieren, damit sie gegebenenfalls mit den grundlegenden Sicherheitsregeln übereinstimmen.
• Bewegungs-/Bahnplanung bezieht sich im Allgemeinen auf die Planung der genauen zukünftigen Bahn für einen kurzen Zeithorizont (bis zu 2 Sekunden).
• Befehlsumsetzung bezieht sich im Allgemeinen auf die Berechnung der endgültigen Befehle, die an ein (reales oder simuliertes) Fahrzeug zu senden sind, wie z. B. Lenkanweisungen.
• Fahrzeugdynamik/Physik bezieht sich im Allgemeinen auf die Simulation des Fahrzeugverhaltens, das sich aus den generierten Befehlen ergibt.
• Positionsaktualisierung bezieht sich im Allgemeinen auf die Berechnung der resultierenden neuen Position des Fahrzeugs in der Simulation.

Such phases can be considered as follows:

• Perception/map generally refers to input about the environment that is available to other components.
• Traffic rules generally refer to any component that provides legal restrictions on high-level decisions.
• Mission planning generally refers to a strategy of when to be where over the long term (e.g., lane-level routing).
• Traffic-free reference route generally refers to planning an “optimal” reference route that ignores other road users.
• Behavioral planning generally refers to the planning of a behavioral plan, ie when exactly actions, such as B. lane change, are to be carried out with the involvement of other participants.
• In general, decision post-processing is about correcting the decisions of the preceding components so that they comply with the basic safety rules, if necessary.
• Motion/trajectory planning generally refers to planning the exact future trajectory for a short time horizon (up to 2 seconds).
• Command translation generally refers to the calculation of the final commands to be sent to a (real or simulated) vehicle, such as B. Steering instructions.
• Vehicle Dynamics/Physics generally refers to the simulation of vehicle behavior resulting from the generated commands.
• Position update generally refers to the calculation of the resulting new position of the vehicle in the simulation.

The use of these terms varies drastically in the literature.

It can be argued that these hierarchical models have certain limitations, such as B. the fact that they are not able to make high-level decisions that are made by "lower" components like a motion planner ([Motion Planner], component that decides e.g. timing of accelerations and lane changes ) must be changed or even rejected (see Junqing Wei, Jarrod M. Snider, Tianyu Gu, John Dolan and Bakhtiar Litkouhi. A behavioral planning framework for autonomous driving. Pages 458-464, 06 2014).

The authors argue that these models put a heavy load on motion and/or trajectory planning and can lead to the conflicting decisions mentioned above. At the same time, they find that trajectory sampling approaches do not meet real-time requirements when applied to long-term planning tasks. They therefore propose to start with a prediction and cost function (PCB)-based planning algorithm that performs a sampling approach in an abstract low-dimensionality motion planning state space. This process involves a forward simulation of the behavior of the other participants and can thus generate a behavior plan. This is then passed to latitude and longitude controllers for execution.

Although this plan is feasible in principle and thus executable for low layers, it formulates high-level decisions as an optimization problem which, as such, does not lead to the simulation of human behavior.

Nachiket and Trivedi have shown that it is possible to probabilistically predict maneuvers from naturalistic data using LSTMs if a vehicle's driving history can be included as input (see Nachiket Deo and Mohan M. Trivedi. Multimodal trajectory prediction of surrounding vehicles using maneuver-based LSTMs .CoRR, abs/1805.05499, 2018). Since the output of their model is not just a single predicted trajectory, but rather a probability distribution, such an approach would be well suited to simulating non-deterministic trajectory choice and controlling how “erratic” a simulated driver should behave. Similarly, Yoon and Kun proposed a method for probabilistic prediction of trajectory parameters of road users based on route histories (see S. Yoon and D. Kum. The multilayer perceptron approach to lateral motion prediction of surrounding vehicles for autonomous vehicles. In 2016 IEEE Intelligent Vehicles Symposium (IV), pages 1307-1312, June 2016).

However, these models depend on the use of the recent driving history of the vehicle in question (and/or the vehicles around it), which may not be available (initial conditions, computational limitations) or may be augmented in the case of simulation environments, since they are also derived from the simulation process originate. Furthermore, simulations in which historical driving patterns of a human driver are extrapolated into the future use naturalistic real-world driving trajectories, so that the validity of the computer-simulated predictions made based on the extrapolated driving patterns is generally reduced.

Another probabilistic approach was proposed by Hu, Zhan, and Tomizuka (Yeping Hu, Wei Zhan, and Masayoshi Tomizuka. Probabilistic prediction of vehicle semantic intention and motion. CoRR, abs/1804.03629, 2018). The authors propose a high-level model to determine a target insertion area, which is defined as the free space between two vehicles in the vicinity. The output of the model is a distribution over the resulting concrete insertion points, which may span multiple such ranges, together with a distribution over the maneuver time. Although the approach works at a high level of abstraction, it cannot predict longer-term maneuvers such as overtaking manoeuvres.

Similarly, there is an approach that predicts the trajectories of road users using Repulsive Potential Energy models parameterized with a driver class learned from a Support Vector Machine (SVM) ( see Hanwool Woo, Yonghoon Ji, Yusuke Tamura, Yasuhide Kuroda, Takashi Sugano, Yasunori Yamamoto, Atsushi Yamashita, and Hajime Asama Trajectory prediction of surrounding vehicles considering individual driving characteristics International Journal of Automotive Engineering, 9(4):282-288 , 2018). However, the forecast horizon was not examined more than two seconds ahead and the assessment was limited to a specific cut-in scenario.

In view of the shortcomings of the prior art, the aim of the present invention is to provide a computer system for simulating a road traffic environment in a driving situation including a complex driving situation (prediction period of more than 2 seconds) for one or more vehicles, so that the decision of a traffic agent is realistic human behavior (naturalistic behavior) reflects. At the same time, the traffic agent's decision should be feasible, i.e. capable of being executed by the underlying vehicle simulation without violating the laws of physics or basic vehicle safety.

BRIEF DESCRIPTION OF THE INVENTION:

The aforementioned object is at least partially achieved by the claimed subject matter of the invention. Advantages (preferred embodiments) are presented in the following detailed description and/or the accompanying figures as well as in the dependent claims.

Accordingly, a first aspect of the invention relates to a computer-implemented training method for a traffic agent that controls a road vehicle in a driving situation, including a complex driving situation, in a simulation environment. The training method provides driving data per time window t over a large number of time windows t _i for one or more road vehicles driven by a person in a realistic situation on a road and generates a corresponding long-term ground truth trajectory x [long -term ground truth trajectory x]. Furthermore, the training method evaluates at least part of the respective ground truth trajectories x for one or more of the specified time windows t _i with regard to the driving criteria of rule conformity and long-term suitability in complex driving situations and assigns a preference value ŷ accordingly. Furthermore, the training method generates one or more viable alternative long-term trajectories x _i at the given time windows t _i as a function of a corresponding ground-truth trajectory x and the correspondingly assigned ground-truth preference value ŷ, with a corresponding preference value y _i relative to the ground-truth preference value ŷ [ground-truth preference value ŷ] is assigned to the corresponding alternative long-term trajectories x _i . Finally, the training method trains a computer model for high-level decision-making in a driving situation that includes a complex driving situation, i) with the one or more ground truth trajectories x and the correspondingly assigned ground truth preference values ŷ and ii) with one or more corresponding feasible alternative trajectories x _i and the correspondingly associated preference values y _i , the computer model being trained to select, as an action to be taken by the traffic agent, a candidate trajectory x _c from the feasible alternative trajectories x _i , the correspondingly associated preference value y _c exceeding a predetermined threshold . The index i is independently selected from an integer of 1, 2, 3 or more. The index c in x _c and y _c represents the selected candidate trajectory and the preference value assigned to it from the group of alternative trajectories x _i .

A second aspect of the invention relates to a computer system for training a traffic agent navigating a road vehicle in a driving situation comprising a complex driving situation of a simulation environment. The computer system for training includes and consists of one or more processors, a memory device coupled to the one or more processors, and a traffic agent learning system that uses a neural network model for decision-making in a driving situation, including a complex driving situation, that is stored in the memory device and configured for execution by the one or more processors. The traffic agent learning system is configured to execute the computer-implemented training method of the present invention.

A third aspect of the invention relates to a computer system for simulating a road traffic environment in a driving situation, including a complex driving situation for one or more vehicles. The computer system includes or consists of one or more processors, a memory device coupled to the one or more processors, and a traffic agent that uses a neural network model for decision-making in a driving situation, including a complex driving situation, stored in the memory device and available for execution by the one or more processors. The traffic agent is trained according to the computer-implemented training of the invention to select as an action a candidate trajectory x _c to be executed by the traffic agent, wherein the preference value y _c associated according to the selected candidate trajectory x _c exceeds a predetermined threshold.

The inventive aspects of the present invention as disclosed herein may comprise any (sub)combination of the preferred inventive embodiments as set out in the dependent claims or as illustrated in the following detailed description and/or the accompanying figures are disclosed, provided the resulting combination of features makes sense to those skilled in the art.

character list

• 1a) und 1b) jeweils graphische Darstellungen der Architektur eines erfindungsgemäßen Trainingscomputersystems und eines erfindungsgemäßen Simulationscomputersystems zeigen.
• 2 eine grafische Darstellung der gemäß der vorliegenden Erfindung erzeugten Trajektorien zeigt.
• 3a) und 3b) Präzisions-/Recall- und Konfusionsmatrizen für erfindungsgemäße Modelle mit unterschiedlicher Anzahl von Neuronen (a) 2-Klassen-Modell mit 32/8 Neuronen; b) 2-Klassen-Modell mit 32/32 Neuronen) in der zweiten Schicht zeigen.
• 4 eine grafische Darstellung der Untersuchungen verschiedener Neuronenzahlen der ersten und zweiten Schicht für 2-Klassen- und 4-Klassen-Modelle zeigt.
• 5 eine grafische Darstellung des erfindungsgemäßen Modells für einige generierte Trajektorien zeigt.
• 6 eine grafische Darstellung eines auf Vorhersage- und Kostenfunktionen basierenden (PCB) Algorithmus für die Trajektoriengenerierung zeigt.

Other features and advantages of the present invention will become apparent from the accompanying drawings, wherein

• 1a) and 1b) each show graphical representations of the architecture of a training computer system according to the invention and a simulation computer system according to the invention.
• 2 Figure 12 shows a graphical representation of the trajectories generated according to the present invention.
• 3a) and 3b) Precision/recall and confusion matrices for models according to the invention with different numbers of neurons (a) 2-class model with 32/8 neurons; b) 2-class model with 32/32 neurons) in the second layer.
• 4 Figure 12 shows a graphical representation of the examinations of different first and second layer neuron counts for 2-class and 4-class models.
• 5 shows a graphical representation of the model according to the invention for some generated trajectories.
• 6 Figure 12 shows a graphical representation of a prediction and cost function based (PCB) algorithm for trajectory generation.

DETAILED DESCRIPTION OF THE INVENTION:

As will be explained in more detail below, the inventors of the various aspects of the present invention have found that the computer-implemented training method according to the present invention enables a traffic agent who controls a road vehicle in a driving situation including a complex driving situation of a simulation environment to create a human-like To choose a long-term (naturalistic) trajectory that is simultaneously feasible and can be executed by the underlying vehicle simulation without violating the laws of physics or basic vehicle safety.

The present invention changes the classic hierarchical driving system in such a way that the feasibility of alternative trajectory proposals can be checked before a naturalistic decision is made at a higher level, e.g. B. Overtaking, following another vehicle or changing lanes. This allows a high-level decision to be made without subsequently changing it in terms of executability.

In the context of the present invention, the expression "an additional or alternative preferred embodiment" or "an additional or alternative further preferred embodiment" or "an additional or alternative way of configuring this embodiment" means that the feature or combination of features, disclosed in this preferred embodiment may be combined in addition to or as an alternative to the features of the subject matter of the invention, including any preferred embodiment of any aspect of the invention, provided that the resulting combination of features makes sense to a person skilled in the art.

In the context of the present invention, the term "comprising" has a similarly broad meaning as the term "including" and means the inclusion of a specific integer or a specific step or a group of integers or steps, but not the exclusion of another whole number or some other step or group of integers or steps. This definition also applies to variations of the term "comprising" such as "include" and "includes".

In the context of the present invention, the term "configured" is to be understood in connection with systems and computer program components. When a system of one or more computers is configured to perform specific operations or actions, it means that the system has software, firmware, hardware, or a combination thereof installed that, when in use, causes the system to perform operations or actions . One or more computer programs configured to perform specific operations or actions means that the one or more programs include instructions that, when executed by a computing device, cause the device to perform the operations or actions.

In the context of the present invention, the expression "high level decision making" means that a traffic agent in driving situations of a simulation environment has to make a decision that involves a complex driving situation, such as e.g. B. Overtaking, following another vehicle or changing lanes. The complex driving situation generally lasts more than 2 seconds.

In the context of the present invention, the term "long-term trajectory" or "long-term candidate trajectory" means that the trajectory covers a period of more than 2 seconds into the future, preferably a specific time window t that is suitable for a complex driving situation, such as e.g. B. an overtaking maneuver or a lane change to perform.

In the context of the present invention, the expression "driving data per time window t over a plurality of time windows t _i for one or more road vehicles, each driven by a human being in a realistic situation on a road", means that status data of the one or more road vehicles per correspondingly given time window t _i , ie the respective time windows (time stamps). The index i in t _i stands for the number of the respective time window (time stamp), ie an integer of 1, 2, 3 or more. The plurality of predetermined time windows t _i can have different or the same, preferably equal, intervals, with the time interval between the plurality of predetermined time windows t _i generally being as short as possible, eg 0.04 seconds. The driving data usually represent vehicle condition data of one or more road vehicles that were driven by people in a realistic situation on a road with at least one lane in each direction, also called "naturalistic" driving situation. The driving data can therefore also be referred to as "naturalistic driving data". In the context of the present invention, no visualized information such. B. image data, used as such driving data.

To achieve the inventive objects, advantages, and objectives, the present invention as disclosed in this disclosure is directed to systems and methods that utilize computer hardware and software to train a virtual traffic agent that uses algorithms and Reinforced learning techniques navigated through a simulation environment. A virtual traffic agent (also referred to as “traffic agent” within the scope of the present invention) can be a car, a truck, a bus, a bicycle or a motorcycle, for example. After a virtual traffic agent has been trained according to the present invention, which simulates human driving behavior in particular in complex driving situations, one or more trained virtual traffic agents can be fed into a simulation environment with complex driving situations. Such an embodiment is preferred since the trained traffic agents can interact, cooperate and challenge with an autonomous vehicle system that controls an autonomous vehicle under test. Another advantage is that such an embodiment is suitable for testing the limits and weaknesses of the autonomous vehicle system, particularly in complex driving situations that can be attributed to assertive or aggressive driving behavior.

Thus, the systems and methods of the invention further have the technical effect and advantage of being an improvement in autonomous vehicle computing technology as the autonomous vehicle is trained in the inventive simulation environment reflecting human-like/naturalistic driving scenarios that are feasible at the same time.

• Gemäß Schritt a) des ersten erfindungsgemäßen Aspekts werden Fahrdaten pro Zeitfenster t über eine Vielzahl von Zeitfenstern t_i für ein oder mehrere jeweils von einem Menschen in einer realistischen Situation auf einer Straße gefahrene Straßenfahrzeuge bereitgestellt und für zumindest einen Teil der Straßenfahrzeuge als Ego-Fahrzeuge eine jeweilige Langzeitgrundwahrheitstrajektorie x erzeugt.

According to the first aspect of the present invention, a computer-implemented training method for a traffic agent navigating a road vehicle in a driving situation including a complex driving situation of a simulation environment is characterized in that the method comprises or consists of the following steps:

• According to step a) of the first aspect of the invention, driving data per time window t is provided over a large number of time windows t _i for one or more road vehicles driven by a person in a realistic situation on a road and for at least some of the road vehicles as ego vehicles respective long-term ground truth trajectory x generated.

The driving data for each of the road vehicles preferably includes or consists of the longitudinal velocity, the longitudinal acceleration and the position of the respective road vehicle in X, Y and Z coordinates at the given times t _i . The vehicle type can also be specified.

According to an additional or alternative preferred embodiment, the driving data are further processed in order to add the features of lateral acceleration, lateral speed, orientation of the vehicle in relation to the road direction (angular deviation) and inverse time to collision (TTC) per vehicle and given time window t _i capture. The inverse TTC represents the time up to the collision of the respective ego vehicle of the ground truth trajectory x with another vehicle, whereby its deceleration reaction is regarded as a collision.

According to step b) of the first aspect of the invention, at least a part, preferably the plurality of the respective ground truth trajectories x at the given points in time t _i is evaluated with regard to the driving criteria of rule conformity and long-term feasibility in a complex driving situation, and a respective ground truth preference value ŷ is assigned. In other words, the computer-implemented training method is configured to automatically assess the respective rate of compliance and long-term feasibility of the ground truth trajectories x.

According to step c) of the first aspect of the invention, one or more viable alternative long-term trajectories x _i are generated for the given time windows t _i as a function of a respective basic truth trajectory x and the correspondingly assigned basic truth preference value, with the long-term trajectories corresponding to a preference value y _i relative to the basic truth -Preference value ŷ is assigned according to the long-term trajectories x _i . The index i in x _i indicates the respective alternative long-term trajectory and is selected from the integers 1, 2, 3 or more. The index i in y _i indicates the correspondingly assigned preference value of the respective alternative long-term trajectory x _i and is selected from the integers 1, 2, 3 or more. In other words, the computer-implemented training method is configured to use a corresponding ground truth trajectory x and the corresponding associated ground truth preference value ŷ to generate the one or more feasible alternative long-term trajectories x _i with corresponding associated preference values y _i .

According to an additional or alternative preferred embodiment, the preference for the basic truth ŷ is set to ŷ=1 and the preference value y _i of the alternative long-term trajectories is calculated according to the function y _i =M(x _i )∈[0...1].

According to step d) of the first aspect of the invention, a computer model for high-level decision-making in a driving situation that includes a complex driving situation is trained i) with the one or more ground truth trajectories x and the correspondingly assigned ground truth preference values ŷ and ii) with a or a plurality of correspondingly feasible alternative trajectories x _i and the correspondingly associated preference values y _i , the computer model being trained to select as an action a candidate trajectory x _c from the feasible alternative trajectories x _i to be executed by the traffic agent, the corresponding associated preference value y _c exceeds a predetermined threshold.

In general, any suitable preference threshold for the selection of the candidate trajectory x _c can be used. In a preferred embodiment, the predetermined threshold value for the preference value y _c of the selected candidate trajectory x _c is set to 50% or more, alternatively 60% or more, alternatively 80% or more, alternatively 90% or more with respect to the preference value of the fundamental truth posited ŷ. In other words, the assigned preference value y _c of the selected candidate trajectory x _c has 50% or more, alternatively 60% or more, alternatively 80% or more, alternatively 90% or more of the respective ground truth preference value ŷ set to 100% becomes.

In the event that more than one alternative long-term trajectory x _i has an assigned preference value y _i that exceeds the predetermined threshold preference value, the training method according to the invention is preferably further configured such that the candidate trajectory x _c is the trajectory with the highest preference value y _c from the group of the alternative candidate trajectories x _i that exceeds the predetermined threshold preference value. The closer the preference value y _i of the selected candidate trajectory x _c in relation to the respective preference value of the basic truth ŷ, the better the fit of the selected candidate trajectory x _c is with regard to the simulation of naturalistic driving behavior with suitable rule conformity and long-term feasibility at the same time. In order to further increase the training result, the computer model for high-level decision-making in a driving situation that includes a complex driving situation is trained in step d) with one or more alternative long-term trajectories x _i that have associated preference values y _i that have the predetermined threshold preference value exceed, and trained with one or more alternative long-term trajectories x _i that have associated preference values y _i that fall below the predetermined threshold.

An additional or alternative preferred embodiment of the present invention relates to an embodiment in which further in step a) a perception value p _i is determined per given time window t _i for one or more of the ego vehicles, the perception value p _i being a function of the driving data of the ego vehicle and the driving data of the surrounding vehicles in a six-vehicle neighborhood. The index i in p _i indicates the perception value of the respective ego vehicle and is selected from the integers 1, 2, 3 or more.

• - Das Auto vor dem Ego-Fahrzeug (auf der gleichen Spur).
• - Das Auto, das dem Ego-Fahrzeug folgt (auf derselben Fahrspur).
• - Die beiden Autos vor dem Mittelpunkt des Ego-Fahrzeugs übertragen auf die beiden benachbarten Fahrspuren.
• - Die beiden Autos im hinteren Teil des Ego-Fahrzeugs übertragen auf die beiden Nachbarspuren.

With respect to the six-vehicle neighborhood, the vehicle roles according to the present invention are defined as follows:

• - The car in front of the ego vehicle (in the same lane).
• - The car following the ego vehicle (on the same lane).
• - The two cars in front of the center of the ego vehicle transferred to the two adjacent lanes.
• - The two cars in the back of the ego vehicle transfer to the two adjacent lanes.

Each of these points may or may not be present for a particular time/ego-vehicle combination and are accounted for in the model.

An additional or alternative preferred embodiment of the present invention relates to an embodiment in which the perception value p _i accordingly includes or consists of one or more status characteristics of the ego vehicle and the corresponding neighboring vehicles, the status characteristics of the ego vehicle preferably being longitudinal acceleration, lateral acceleration and bearing comprise or consist of and the state characteristics of each of the surrounding vehicles in the six-vehicle neighborhood include vehicle roll, longitudinal velocity, longitudinal acceleration, lateral acceleration, Euclidean distance between each ego vehicle and a respective surrounding vehicle (L ² Distance d) includes or consists of the inverse TTC and the relative bearing. In the context of the present invention, the term “bearing” of an ego vehicle stands for the alignment of the ego vehicle in relation to the global x/y axes. In the context of the present invention, the term “relative bearing” of a vehicle in the neighborhood of six represents the relative orientation of the respective vehicle in relation to the ego vehicle, in particular represented by the relative angle Θ, which is in the position of the ego vehicle is formed in relation to the straight line between the ego vehicle position and the position of the respective vehicle of the neighborhood of six in the global x/y axes and the respective x axis.

An additional or alternative preferred embodiment of the present invention relates to an embodiment, wherein in step b) the computational evaluation of the criteria for rule conformity is designed in such a way that it determines the conformity of the basic truth trajectory x and/or the alternative trajectories x _i for a given time window t _i is taken into account with the requirements of the legal driving regulations and the evaluation of the criteria for the long-term feasibility in complex driving situations is designed in such a way that it takes into account the feasibility of the alternative trajectories x _i through the underlying vehicle simulation, without violating the physical laws and basic vehicle safety conditions, such as eg driving with as few collisions as possible and without leaving the driving area.

An additional or alternative preferred embodiment of the present invention relates to an embodiment wherein in step c) the generation of one or more viable alternative long-term trajectories x _i for given time windows t _i comprises or consists of: i) transforming the ground truth trajectory x for the respective ego vehicles to express the position of the ego vehicles in terms of distance parameters θ̂ relative to the roadway, ii) generating one or more random alternative long-term candidate trajectories x _i , and iii) calculating the preference y _i for the distance parameters θ by generating a or several trajectory points and point-by-point comparison with the respective distance parameters θ̂ of the ground truth trajectory x.

An additional or alternative preferred embodiment of the present invention relates to an embodiment in which the ground truth trajectory x for the respective ego vehicle is transformed into a Frenet frame, i. H. to express a position s along its lane centerline and its (lateral) distance w to that line at any instant, and then simplifying the ground truth trajectory x to the distance parameters θ̂.

An additional or alternative preferred embodiment of the present invention relates to an embodiment in which the ground truth trajectory x is simplified to six distance parameters θ̂̂ (θ̂ ₁ to θ̂ ₆ ) describing θ̂ ₁ s-distance to the first lane change, θ̂ ₂ s- Distance length of first lane change θ̂ ₃ s-distance to second lane change, θ ₄ s-distance of second lane change, θ̂ ₅ w-distance of first lane change, and θ̂ ₆ w-distance w-distance of second lane change.

worin n eine ganze Zahl aus der Gruppe 1, 2, 3 oder mehr darstellt.In an example, the preference value y _i is determined according to the following function: $${\text{y}}_i:=1-\text{tanh}\left(\frac{1}{n}{\displaystyle \sum _{j=1}^n{\left[f\left(t_j,\theta \right)-f\left(t_j,\widehat{\theta }\right)\right]}^2}\right)$$

wherein n is an integer from the group 1, 2, 3 or more.

An additional or alternative preferred embodiment of the present invention relates to an embodiment in which the generation of alternative long-term trajectories x _i comprises generating a plurality of random trajectories x _i sampled uniformly in bounds for the distance parameter space θ and generating a plurality of random trajectories x _i derived from a normal distribution around the distance parameter space 0 with the same constraints.

An additional or alternative preferred embodiment of the present invention relates to an embodiment in which 5 or more, alternatively 10 or more, alternatively 20 or more, alternatively 30 or more, alternatively 40 or more, alternatively 50 or more random alternative trajectories x _i are generated, sampled uniformly in bounds for the distance parameter space θ, and generating 5 or more, alternatively 10 or more, alternatively 20 or more, alternatively 30 or more, alternatively 40 or more, alternatively 50 or more random alternative trajectories x _i can be generated from a normal distribution around the distance parameter space θ with the same restrictions.

All features and embodiments disclosed in relation to the first aspect of the present invention can be combined alone or in (sub)combination with the second aspect or the third aspect of the present invention, including any of the preferred embodiments thereof, provided the resulting combination of features is appropriate for a person skilled in the art.

According to the second aspect of the invention, a computer system for training a traffic agent navigating a road vehicle in a driving situation that includes a complex driving situation of a simulation environment, comprising or consisting of one or more processors, a memory device coupled to the one or more processors and a traffic agent learning system using a neural network model for decision making in a complex driving situation, stored in the storage device and configured to be executed by the one or more processors, characterized in that the traffic agent learning system is configured to carry out the computer-implemented training method of the first aspect.

An additional or alternative preferred embodiment of the present invention relates to an embodiment in which the neural network model for decision-making in a driving situation including a complex driving situation comprises a two-class neural network model, preferably with up to 32 neurons in the first layer and up to 32 neurons in the second layer, or it comprises a four-class neural network model, preferably with up to 96 neurons in the first layer and up to 16 neurons in the second layer. The number of neurons in the second layer is preferably correspondingly higher in order to achieve a good correct classification rate [accuracy] in the test. The number of neurons in the first layer seems to have less of an impact on the correct classification rate in the test.

According to an additional or alternative preferred embodiment, the computer system of the second aspect may be configured such that the traffic agent comprises separate modules. A module A can be configured to input and process naturalistic driving data and to generate one or more ground truth trajectories x. A module B can be configured to generate the perceptual values p _i of the one or more ground truth trajectories x. A module C, also referred to as “generative trajectory/movement planner” or “Gen. TP” may be configured to generate one or more viable alternative long-term trajectories x _i for given time windows t _i , correspondingly associated with a preference value y _i . Module D, which according to the present invention is also referred to as a high-level decision maker [“High Level Decision Maker”] or “HLDM”, represents a computer model for high-level decision-making in a driving situation that includes a complex driving situation, and is trained with one or more of the alternative feasible long-term trajectories x _i provided by module C and the respective basic perceptual trajectories x provided by module A and/or the perceptual values p _i of module B to produce as an action a candidate trajectory x _c that exceeds the predetermined threshold for the preference value y _i to be executed by the traffic agent.

An additional or alternative preferred embodiment of the present invention relates to an embodiment where modules A and B are combined into one module or where modules B and C are combined into one module.

One embodiment of the training computer system according to the invention is that module A and module B supply their correspondingly generated data to module C and/or module D, with the data preferably being used first in module C and then in module D. In other words: The driving data of the ego vehicles including the ground truth trajectories x from module A and their perception p _i from module B are first used in module C to generate the alternative long-term trajectories x _i . The driving data of the ego vehicles, including the ground truth trajectories x from module A and their perception p _i from module B, are then preferably also used in module D to select a long-term candidate trajectory x _c from the group of possible alternative long-term trajectories x _i provided by module C evaluate and select.

The advantage of the computer system according to the invention for training a traffic agent is that the decisions are made after the generation of viable alternative naturalistic long-term trajectories x _i and thus all alternative trajectories x _i selected by module D can be implemented naturally and without modification. In other words, the high-level decision-making neural network of the present invention produces human-like trajectories that are realizable simultaneously.

All features and embodiments disclosed in relation to the second aspect of the present invention can be combined alone or in (sub)combination with the first aspect or the third aspect of the present invention, including any of the preferred embodiments thereof, provided the resulting combination of features is appropriate for a person skilled in the art.

A third aspect of the invention relates to a computer system for simulating a road driving environment in a driving situation that includes a complex driving situation for one or more vehicles, comprising or consisting of one or more processors, a memory device coupled to the one or more processors, and a Traffic agent using a neural network model for decision making in a driving situation involving a complex driving situation stored in the storage device and configured to be executed by the one or more processors characterized in that the traffic agent according to the computer-implemented Training method of the first aspect of the invention is trained to as an action to select a candidate trajectory _xc that exceeds the predetermined threshold for the preference value _yc to be performed by the traffic agent in the simulation environment. In other words, while the computer system of the second aspect of the present invention relates to the training phase of a traffic agent learning system using naturalistic driving data, the computer system of the third aspect of the present invention relates to the deployment of the trained traffic agent in a simulation environment using simulated driving data .

An additional or alternative preferred embodiment of the present invention relates to an embodiment in which the neural network model for decision-making in a driving situation including a complex driving situation comprises a two-class neural network model, preferably with up to 32 neurons in the first layer and up to 32 neurons in the second layer, or it comprises a four-class neural network model, preferably with up to 96 neurons in the first layer and up to 16 neurons in the second layer. The number of neurons in the second layer is preferably correspondingly higher in order to achieve a good correct classification rate in the test. The number of neurons in the first layer seems to have less of an impact on the correct classification rate in the test.

With regard to the use of the traffic agent according to the invention, the simulation computer system of the third aspect can be configured in such a way that the traffic agent consists of separate modules. A module A can be designed to process simulated driving data from simulated vehicles in a simulation environment. A module B can be configured to generate the perception values p _i of the simulated vehicles per given time window t _i . A module C, also "generative trajectory/movement planner" or "Gen. TP”, which was trained according to the present inventive training method, is configured to generate one or more viable alternative long-term trajectories x _i per given time window t _i , to which a preference value y _i is assigned accordingly. Module D, which according to the present invention is also referred to as "High Level Decision Maker" or "HLDM" and has been trained according to the present invention, is configured to take as an action a long-term candidate trajectory x _c from the group provided by Module C viable alternative trajectories x _i exceeding the predetermined threshold for the preference value y _c to be performed (executed) by the traffic agent in the simulation environment.

An additional or alternative preferred embodiment of the present invention relates to an embodiment where modules A and B are combined into one module or where modules B and C are combined into one module.

Other modules required to execute the selected candidate trajectory, such as a module E, a so-called "Trajectory Plan Execution module" or "TP Exec." module and/or a module F, a so-called "vehicle module", for executing the selected trajectory in the simulation environment , are preferably also included. For details of the execution of a selected trajectory x _i , the module E can, if necessary, call up further processing steps, which are described in more detail below in the example section.

One embodiment of the simulation computer system according to the invention is that module A and module B first make their correspondingly generated simulation data available to module C and then to module D. In other words, the simulated driving data of the simulated vehicles provided by module A and/or B and their respective perceptions p _i are used to generate the alternative long-term trajectories x _i in module C and to select the candidate trajectory x _c in module D.

The advantage of the simulation computer system according to the invention is that the decisions are made after the generation of realizable naturalistic long-term trajectories x _i and thus all candidate trajectories x _c selected by module D can be realized naturally and without modification. In other words, the high-level decision maker of the present invention creates human-like trajectories that are realizable at the same time.

An additional or alternative preferred embodiment of the present invention relates to an embodiment where Module C and/or Module D is/are supplemented with a strategic exploration component such as a Prediction and Cost Functions Based (PCB) component or a similar component to calculate candidate trajectories longer than 2 seconds.

Such an extension could, for example, be suitable for influencing the trajectory proposal from module C and/or the trajectory selection from module D by making further demands on the trajectory generation/selection, e.g. B. the selection of a cost function to simulate a desired aggressiveness of the driving style.

All features and embodiments disclosed in relation to the third aspect of the present invention are combinable alone or in (sub)combination with the first aspect or the second aspect of the present invention, including any of their preferred embodiments, provided that the resulting Combination of features is reasonable for a person skilled in the art.

The present invention is described below with reference to exemplary embodiments, which only serve as examples and are not intended to limit the scope of the present protective right.

DETAILED DESCRIPTION OF FIGURES AND EXPERIMENTAL SETUP

Further features and advantages of the present invention result from the following description of exemplary embodiments of the aspects according to the invention with reference to the attached figures.

All features disclosed below in relation to the exemplary embodiments and/or the accompanying figures can be combined alone or in any sub-combination with features of the three aspects of the present invention, including features of preferred embodiments thereof, provided that the resulting combination of features is apparent to a person skilled in the art field of technology makes sense.

1a) Fig. 12 shows a graphical representation of an embodiment of the architecture 1, also called "drive stack", of a training computer system according to the present invention and 1b) Figure 12 shows a graphical representation of an embodiment of the architecture 1' of a simulation computer system according to the present invention. The driving stacks according to the invention are configured in such a way that they make realizable human-like (naturalistic) decisions in all simulated driving situations, including complex driving situations.

In the 1a) The training architecture 1 shown is configured in such a way that it combines the functions of module A and module B, as described above in relation to the training computer system according to the invention, in a module 11 . Accordingly, the combined module 11 is configured to perform the functions of module A, eg it is configured to input and process the naturalistic driving data and generate the ground truth trajectories x, is configured to use the perceptual values p _i of the one or more Ground truth trajectories x are generated per given time window t _i . Module 12 (Module C), which pursuant to 1a) also as "Gen. TP” is configured to generate one or more viable alternative long-term candidate trajectories x _i per given time window t _i , which are correspondingly assigned a preference value y _i . Module 13 (Module D), which pursuant to 1a) also referred to as "HLDM", represents a computer model for high-level decision-making in a driving situation that includes a complex driving situation, and is provided with one or more of the long-term trajectories x _i provided by module 12 and the respective ones provided by combined module 11 Trains ground truth trajectories x to select as an action a candidate trajectory x _c that exceeds the predetermined threshold for the preference value y _c to be performed by the traffic agent.

Out of 1a) does not indicate that modules A and B can alternatively be provided as separate modules.

According to 1a) the combined module 11 delivers the respective naturalistic driving data and the perception p _i to module 12 or module 13. In other words, the naturalistic driving data of the ego vehicles including the basic truth trajectories x and their perception p _i are first used in module 12 (dashed line) to generate the alternative long-term trajectories x _i . In addition, the naturalistic driving data of the ego vehicles including the ground truth trajectories x and their perception p _i are then used in module 13 (solid line) to evaluate and select the long-term candidate trajectory x _c .

This architectural embodiment according to the invention, which is 1a) has the advantage that all candidate trajectories x _i selected by the high-level decision module 13 are naturalistic and feasible and do not have to be changed afterwards.

In the 1b) The simulation architecture 1' shown is configured in such a way that it also includes the functions of module A and module B, as described above in relation to the simulation computer system according to the invention, in a module 11'. Accordingly, the combined module 11' is configured to perform the functions of module A, eg it is configured to simulate driving data from simulated vehicles in a simulation environment. This means that module A does not use the naturalistic driving data and the long-term ground truth trajectories x during use in a simulation environment. Furthermore, the combined module 11' is configured to perform the functions of module B, e.g. B. it is configured to generate perception values p _i of the simulated driving data per given time window t _i . Module 12' (module C) according to 1b) also "Gen. TP” has been trained using the training method according to the invention and is configured to generate one or more viable alternative long-term trajectories x _i per given time window t _i , to which a preference value y _i is assigned accordingly. Module 13 '(module D), according to 1b) also referred to as "HLDM", represents a computer model for high-level decision-making in a driving situation, including a complex driving situation, and was trained with the training method according to the invention to select as an action a candidate trajectory x _c that meets the predetermined threshold value for the preference value y _c exceeded to be carried out by the traffic agent.

Not shown in 1b) that the modules A and B can alternatively be provided as separate modules.

The simulation architecture 1 'of 1b) also includes the module 14' (module E), which is also referred to as "TP Exec.", to carry out the execution planning of the candidate trajectory x _c in a simulated vehicle, as well as the module 15' (module F) for the simulated "vehicle module" . For the details of the execution of the selected candidate trajectory x _c , module 15' can call up further processing steps if required, which is 1b) is not shown.

According to 1b) the combined module 11' delivers the respective simulated data and perceptions p _i to module 12' and module 13', respectively. In other words: the data of the simulated vehicles and their perception p _i are first used (dashed line) in module 12' in order to generate the realizable alternative long-term trajectories x _i . Subsequently, the data of the simulated vehicles and their perception p _i are used in module 13' (solid line) to evaluate and select the feasible long-term candidate trajectory x _c .

This simulation architecture according to the invention (embodiment 1' in 1b) provides the advantage that any long-term candidate trajectory x _c selected by the high-level decision maker of module 13 is naturalistic and feasible and does not need to be changed afterwards.

In 1b) is further shown that module 12 'and / or module 13' can be supplemented by a strategic exploration component, z. B. a component based on prediction and cost functions (PCB) or a similar component to calculate candidate trajectories that take longer than 2 seconds. Such an extension could be suitable for influencing the suggestion of alternative trajectories by module 12' and/or the selection of candidate trajectories by module 13' by making further demands on the trajectory generation/selection, e.g. B. the selection of a cost function to simulate a desired aggressiveness of the driving style.

The inventors of the present application examined a number of neural network architectures of similar construction, varying the number of neurons and the preference classes in view of the present invention. All considered networks consisted of two dense layers with Rectified Linear Unit (ReLU) activations followed by a final dense layer with softmax output.

For the training method according to the invention, the inventors used the first four hours of driving data from the DataFromSky (DFS) data set (acquired from RCE systems sro, Czech Republic), which constitutes a naturalistic driving data set for analysis. The naturalistic data included coordinate and velocity information from actual human driving trajectories on a small section of about 500 m of the A9 Autobahn in Germany. The data included in particular per Timeslot a unique vehicle identifier, global lateral and longitudinal coordinates, longitudinal velocity and longitudinal acceleration. The inventors have not considered vehicles that are visible in less than 200 time windows t _i or that are not passenger cars (but e.g. trucks) to be ego vehicles with regard to the following experimental analysis. From the first three hours of driving data, 72% were randomly selected as training data, 20% as test data and 8% as validation data.

For each resulting data image, the inventors considered each car as an ego vehicle, extracted its perception p _i as described in the detailed description of the invention above and below, and generated alternative long-term trajectories x _i as described in the detailed description of the invention above and below .

Accordingly, the inventors believed that the behavior of the driver of a ego vehicle is in large part a function of the state of the vehicle and the state of nearby vehicles. Therefore, the model was trained with features that describe both the current state of the ego vehicle and the state of the six-vehicle neighborhood. The six-car neighborhood (also called "six-car neighborhood") has already been described in the detailed description of the present invention.

The naturalistic driving data set DFS as defined above was used for training the computer model according to the invention for high-level decision-making. This data set contains a ground truth trajectory x for a series of driving situations.

• Naturgetreue Trajektorien können verschiedene (niedrige) Merkmale aufweisen, die ein Modell zu stark anpassen könnte. In diesem Fall sollte darauf geachtet werden, dass diese (niedrigen) Merkmale während des Modelltrainings möglichst nicht aufgedeckt werden.

This brings with it a number of challenges:

• Lifelike trajectories can have various (low) features that a model might overfit. In this case, care should be taken to ensure that these (low) features are not uncovered during model training.

In order to improve the prediction of a precise long-term candidate trajectory x _c , the training data should preferably contain long-term trajectories x _i with different preferences y _i , ie those similar to the ground truth trajectory x as well as others.

1. 1. Für ein bestimmtes Fahrzeug, das in den naturalistischen Fahrdaten als Ego betrachtet wird, wird der entsprechende Wahrnehmungswert p_i berechnet (wie in der detaillierten Beschreibung der Erfindung beschrieben). Die Merkmale, die beim Training als Wahrnehmungswerte berücksichtigt werden, sind in Tabelle 1 aufgeführt:

Tabelle 1: Merkmal Einheit Ego-Fahrzeug Longitudinale Geschwindigkeit m/s Longitudinale Beschleunigung m/s² laterale Beschleunigung a _lat m/s² Lager rad Für jedes Fahrzeug in einer Nachbarschaft mit sechs Fahrzeugen Rolle des Fahrzeugs (implizit durch Index) Longitudinale Geschwindigkeit s m/s Longitudinale Beschleunigung a m/s² Laterale Beschleunigung a _lat m/s² L² Abstand d M inverse TTC s^-1 Relatives Lager rad With these requirements in mind, alternative long-term trajectories x _i for training are created using the following procedure:

1. 1. For a given vehicle considered as ego in the naturalistic driving data, the corresponding perception value p _i is calculated (as described in the detailed description of the invention). The traits considered as perceptual values in training are listed in Table 1:

Table 1: feature Unit ego vehicle longitudinal speed m/s longitudinal acceleration m/s ² lateral acceleration a _lat m/s ² warehouse wheel For each vehicle in a six-vehicle neighborhood role of the vehicle (implicit by index) longitudinal velocity s m/s Longitudinal acceleration a m/s ² Lateral acceleration a _lat m/s ² L ² distance d M inverse TTC s ^-1 Relative camp wheel

• 2. Die Grundwahrheitstrajektorie x wird dann in Frenet Frame Koordinaten transformiert (für Details siehe Moritz Werling, Julius Ziegler, Sören Kammel und Sebastian Thrun. Optimale Trajektoriengenerierung für dynamische Straßenszenarien in einem Frenet Frame. Seiten 987-993, 06 2010). Dementsprechend wird die Position des Fahrzeugs in Form seiner Position s entlang der Mittellinie seiner Fahrspur und seines (seitlichen) Abstands w zu dieser Linie zu jedem Zeitpunkt ausgedrückt.
• 3. Die Grundwahrheitstrajektorie x wird darüber hinaus vereinfacht, indem sie auf sechs Entfernungsparameter reduziert wird θ̂̂ (θ̂₁ auf θ̂₆), die Folgendes beschreiben θ̂₁ s-Entfernung bis zum ersten Fahrspurwechsel, θ̂̂₂ s-Entfernung des ersten Spurwechsels, θ̂̂₃ s-Entfernung bis zum zweiten Spurwechsel, θ̂₄ s-Entfernung des zweiten Spurwechsels, θ̂̂₅ w-Entfernung des ersten Spurwechsels, und θ̂₆ w-Entfernung des zweiten Fahrspurwechsels.
• 4. Erzeugen von 50 zufälligen Langzeittrajektorien x_i durch Erzeugen einer Vielzahl von Trajektorien x_i, die gleichmäßig in Grenzen für den Abstandsparameterraum abgetastet werden 0 (z.B. ein Maximum für θ₅ und θ₆).
• 5. Generierung von 50 zufälligen Langzeittrajektorien x_i, die aus einer Normalverteilung um den Entfernungsparameter θ̂ mit denselben Einschränkungen.
• 6. Berechnen des Präferenzwertes y_i für alle Entfernungsparametersätze θ durch Erzeugen von n Trajektoriepunkten unter Verwendung von sinusbasierten Spurwechseln, wobei n eine ganze Zahl ausgewählt aus 1, 2, 3 oder mehr ist, und punktweises Vergleichen mit den jeweiligen Abstandsparametern θ̂ der Grundwahrheitstrajektorie x, vorzugsweise gemäß der folgenden Funktion $${\text{y}}_i:=1-\text{tanh}\left(\frac{1}{n}{\displaystyle \sum _{j=1}^n{\left[f\left(t_j,\theta \right)-f\left(t_j,\widehat{\theta }\right)\right]}^2}\right)$$
  

Accordingly, the future ground truth trajectory x of the vehicle is extracted as an accumulation of data of the ego vehicle over time.

• 2. The ground truth trajectory x is then transformed into Frenet frame coordinates (for details see Moritz Werling, Julius Ziegler, Sören Kammel and Sebastian Thrun. Optimal trajectory generation for dynamic road scenarios in a Frenet frame. Pages 987-993, 06 2010). Accordingly, the position of the vehicle is expressed in terms of its position s along the center line of its lane and its (lateral) distance w from that line at any point in time.
• 3. The ground truth trajectory x is further simplified by reducing it to six distance parameters θ̂̂ (θ̂ ₁ on θ̂ ₆ ), which describe θ̂ ₁ s-distance to first lane change, θ̂̂ ₂ s-distance to first lane change, θ̂̂ ₃ s-distance to second lane change, θ̂ ₄ s-distance of second lane change, θ̂̂ ₅ w-distance of first lane change, and θ̂ ₆ w-distance of second lane change.
• 4. Generate 50 random long-term trajectories x _i by generating a plurality of trajectories x _i sampled uniformly in bounds for the distance parameter space 0 (eg a maximum for θ ₅ and θ ₆ ).
• 5. Generation of 50 random long-term trajectories x _i derived from a normal distribution around the distance parameter θ̂ with the same restrictions.
• 6. Calculating the preference value y _i for all distance parameter sets θ by generating n trajectory points using sine-based lane changes, where n is an integer selected from 1, 2, 3 or more, and comparing point by point with the respective distance parameters θ̂ of the ground truth trajectory x, preferably according to the following function $${\text{y}}_i:=1-\text{tanh}\left(\frac{1}{n}{\displaystyle \sum _{j=1}^n{\left[f\left(t_j,\theta \right)-f\left(t_j,\widehat{\theta }\right)\right]}^2}\right)$$
  

The search for long-term candidate trajectories x _i may not always be successful due to limitations in the space of distance parameters 0. So it may be that in some driving situations fewer than the desired 100 alternative trajectories x _i are available.

2 Figure 12 shows a graphical representation of a ground truth trajectory x (thick black dashed line), a simplified abstract motion plan (solid thick black line), and several corresponding long-term trajectories x _i generated in accordance with the present invention. The lighter the color, the lower the respective preference value y _i . X-Axis: Position along the reference line. Y axis: Lateral position in relation to the reference line. Table 2: neurons 1 ^{pc 2nd} class balance batch epoch Correct classification rate (%) 2nd Class < 0.5 ≥ 0.5 16 8th 1000 100 82.2 · · · · · 150 81.3 · · · · · · · · · · 32 8th · · · 10 82.5 · · · · · · · · · · 500 100 82.6 · · · · · · · · 32 8th 70 30 82.0 * · · · · 82.0 ** · · · · · · · · 32 32 · · 85.5 · · · · 4th grade < 0.5 < 0.7 < 0.9 ≥ 0.9 96 16 50 10 11 29 1000 87.7
* Recursive training
** One-time training

Table 2 gives an overview of the considered calculation models and the class distribution during the training. Table 2 lists the different model architectures trained along with the lot size and the number of training epochs. Training was performed with the Adam optimizer with a learning rate of α = 10 ^-3 and exponential decay rates for the first and second moment estimates β ₁ = 0.9 and β ₂ = 0.999. The inventors used the (sparse) categorical cross-entropy as the loss function. The initialization was performed using the Xavier method.

As Table 2 shows, the different models compared all have similarly good correct classification rates in the test, ranging from 81.3 to 87.7%. The best-performing networks have a high neuron count in the second layer, while the first layer seems to have less impact.

the 3a) and 3b) further illustrate this by plotting the precision recall curve and confusion matrices of two selected models with different numbers of neurons (a) 2-class model with 32/8 neurons; b) Compare 2-class model with 32/32 neurons) in the second layer on the 4 ^th hour dataset.

In relation to 3a) the area under the curve (AUC) is 0.749 with the following ratio between the predicted label and the actual label of the generated (proposed) alternative long-term trajectories: label [label] prediction Poorly Good Poorly 55.8% 18.6% Good 3.1% 22.5%

In other words, of all the long-term alternative trajectories generated (100%), 55.8% of the alternative trajectories were correctly labeled as "bad" and only 3.1% were labeled as "poor" even though the correct label was "good". In addition, 22.5% of the alternative trajectories were correctly labeled "good" and 18.6% were labeled "good" even though the correct label was "poor".

In relation to 3b) the area under the curve (AUC) is 0.761 with the following ratio between the predicted label and the actual label of the generated (proposed) alternative long-term trajectories. label [label] prediction Poorly Good Poorly 64.8% 9.6% Good 5.9% 19.8%

In other words, of all the long-term alternative trajectories generated (100%), 64.8% of the alternative trajectories were correctly labeled as "bad" and only 5.9% were labeled as "poor" even though the correct label was "good". In addition, 19.8% of the alternative trajectories were correctly labeled "good" and 9.6% were labeled "good" even though the correct label was "poor".

This will also through 4 , which shows the examination of different neuron counts for the first and second layer for 2-class and 4-class models and the resulting accuracies.

5 shows the output of the model according to the invention for some generated trajectories that were not used for training, testing or validation. The x value is the position along the centerline of the lane and the y value is the lateral position. Vehicles, specifically cars, are shown as gray boxes (approximate dimensions as not available in the DFS dataset). Bold black line: candidate trajectory with the highest model preference y _i . Other trajectories with ground truth preference y=1 are also shown. Rejected trajectories are not shown.

• Der generative Trajektorienplaner (TP, Modul C) des Moduls 12' muss in der Lage sein, dem HLDM (Modul D) mehrere alternative Langzeittrajektorien x_i vorzuschlagen, die alle machbar und damit ausführbar sind.

According to the present invention, a generative trajectory planner (module C) (module 12, 12') is used to propose possible alternative long-term trajectories x _i to the high-level decision-maker (HLDM; module D). As mentioned earlier, this requires an extension of trajectory planning and execution, specifically in the following ways:

• The generative trajectory planner (TP, module C) of the module 12' must be able to propose several alternative long-term trajectories x _i to the HLDM (module D), all of which are feasible and therefore executable.

The TP execution of module 14' should not significantly affect the decision of module 13' (HLDM). Consider the case of a cut-in maneuver: If the TP execution of module 14' were to delay the entry process due to the gap assumption, this would effectively change the decision to hit another vehicle so that the gap is already through the TP of module 14' must be taken into account.

The alternative long-term trajectories x _i should cover the vehicle's action space well and be long enough to represent a high-level decision such as overtaking. The generated alternative long-term trajectories x _i should be expressible given a fixed set of parameters.

• Die Generierung von Langzeittrajektorien x_i in Modul 12 oder 12' kann vorzugsweise das gleiche Parametergitter für jeden Generierungslauf verwenden. Dies ermöglicht die Vorausberechnung aller Teile der TP, die nicht vom Ego oder der Umgebung abhängen. Diese Idee lässt sich auf die Vorausberechnung eines Gitters von Ego-Zuständen der Implementierungen ausweiten, wenn einige defensive Annahmen getroffen werden.

In order to keep the computational effort for the trajectory within limits, the inventors provide the following information:

• The generation of long-term trajectories x _i in module 12 or 12' can preferably use the same parameter grid for each generation run. This allows for the pre-calculation of all parts of the TP that do not depend on ego or the environment. This idea can be extended to predicting a lattice of ego states of the implementations if some defensive assumptions are made.

If the generation of long-term trajectories is problematic, an approach like PCB can preferably be used to explore the solution space more efficiently by computing partial trajectories. This combines very well with the previous approach. 6 illustrates this idea with a simplified graphical representation of a prediction-cost function-based (PCB) trajectory generation algorithm. A coarser state grid (circles, dashed lines) is searched exhaustively using forward simulations of other traffic participants. through Dashed circles symbolize unreachable states (e.g. collisions, non-navigable area). The transition between two grid points is planned using a (appropriately limited) trajectory planner with different parameterizations (continuous gray lines, only one grid sequence shown). If the distances between the state grids are homogeneous, the transitions can be precalculated.

• Die (potenziell kundenorientierte) Schnittstelle kann direkt mit den Interna des Fahrstapels gekoppelt sein: Der Kunde müsste die Entscheidungsraten und die Bedeutung der 5 möglichen Entscheidungen kennen, die sich je nach Fahrzeugzustand ändern (z. B. werden Fahrspurwechsel als Abbruch des Fahrspurwechsels interpretiert, wenn gerade ein Fahrspurwechsel durchgeführt wird).

If a customer wants to force an agent in a simulation environment to change lanes at a certain time or target a certain speed, such control can easily be achieved within the scope of the present invention by bypassing the HLDM of module 13' and issuing commands such as "Lane change left" can be injected directly into the lower parts of the driving stack. However, this approach may have certain shortcomings:

• The (potentially customer-centric) interface may be directly coupled to the internals of the driving stack: the customer would need to know the decision rates and the meaning of the 5 possible decisions that change depending on the vehicle state (e.g. lane changes are interpreted as lane change abort if a lane change is in progress).

The interface may be coupled to the HLDM abstraction layer of module 13'. In this special case, this means e.g. B. that the customer is responsible for ensuring that a requested lane change does not lead to a collision. More abstract commands such as overtaking cannot be given.

Any change in the communication between the components of the drive stack would mean a change in the interface.

A naïve approach for the proposed solution would be to let the customer directly create a trajectory to be executed by TP Execution module ([TP Execution of module]) 14'. However, this would not only keep the problems discussed above, but would also make the customer's job much more difficult, since they need a complete trajectory.

• Die Kundenschnittstelle ist von den internen Prozessen entkoppelt. Der Kunde muss keine Kenntnis von der Berechnung des Nutzens oder der Wahl der Trajektorie haben, sondern kann jede geeignete Schnittstelle nutzen, die vom Kunden DM auf jeder gewünschten Abstraktionsebene bereitgestellt wird.

In this case, the inventors propose to replace the HLDM of the module 13' with a simple customer decision component ([Customer Decision Maker],Customer DM) as available in the prior art, which exploits the utility of the generated trajectories based on the current customer requirements are calculated. This solution offers several advantages:

• The customer interface is decoupled from the internal processes. The customer does not need to know about the calculation of the utility or the choice of the trajectory, but can use any suitable interface provided by the customer DM at any desired level of abstraction.

The implementation of the customer DM is known in the prior art. For a given customer query and trajectory, the customer DM only needs to indicate how closely that trajectory matches it. This can be reduced to questions such as "Is this trajectory a lane change to the right?", "Is this trajectory accelerating to at least 100 km/h?", "Is this trajectory an overtaking maneuver?". “In more complex cases (e.g. overtaking manoeuvres), the PCB output can be communicated internally, allowing for a more abstract view of the received trajectory.

Since all alternative long-term trajectories x _i proposed to the DM are feasible, the customer request cannot lead to collisions.

The inventors of the present invention have disclosed a method for using machine learning to select the most naturalistic ones from a pool of generated viable alternative long-term trajectories x _i . This allows the classic prior art hierarchical driving stack to be modified so that the feasibility of trajectories can be considered before a naturalistic decision is made at a higher level, so that a decision once made is not compromised by components further "down" in the stack needs to be changed. It has also been shown that the present invention is successfully trained on naturalistic data and is flexible with respect to the concrete method of trajectory generation.

Claims (16)

Computer-implemented training method for a traffic agent who controls a road vehicle in a driving situation including a complex driving situation in a simulation environment, characterized in that the method comprises or consists of the following steps: a. Providing driving data per time window t over a large number of time windows t _i for one or more road vehicles, each of which is driven by a person in a realistic situation on a road, and generating a corresponding long-term ground truth trajectory x for at least some of the road vehicles as ego vehicles , b. Evaluating at least part of the corresponding basic truth trajectories x at one or more of the specified times t _i with regard to the driving criteria of rule conformity and long-term suitability in complex driving situations and assigning a corresponding basic truth preference value 9, c. Generating one or more viable alternative long-term trajectories x _i at the given time windows t _i as a function of a corresponding ground truth trajectory x and the correspondingly assigned ground truth preference value ŷ, with a corresponding preference value y _i relative to the ground truth preference value ŷ being assigned to the alternative long-term trajectories x _i accordingly will and d. Training a computer model for high-level decision-making in a driving situation that includes a complex driving situation, i) with the one or more ground truth trajectories x and the correspondingly assigned ground truth preference values ŷ and ii) with one or more corresponding feasible alternative trajectories x _i and the correspondingly assigned preference values y _i , the computer model being trained to select as an action a candidate trajectory x _c from the feasible alternative trajectories x _i to be executed by the traffic agent, the correspondingly assigned preference value y _c exceeding a predetermined threshold value, wherein the index i is independently selected from an integer of 1, 2, 3 or more. Training procedure according to claim 1 , wherein the driving data in step a) for each of the road vehicles include or consist of one or more status characteristics of the vehicle, preferably the longitudinal speed, the longitudinal acceleration and the position of the respective road vehicle in X, Y, Z coordinates to the given time windows t _i . Training procedure according to claim 1 or 2 , wherein the driving data are further processed in step a) in order to additionally include the vehicle condition characteristics of lateral acceleration, lateral speed, orientation of the vehicle and/or inverse time until collision with a vehicle, taking into account its deceleration reaction (inverse TTC) in predetermined time windows t _i . Training method according to one of Claims 1 until 3 , wherein in step a) a perception value p _i is determined per given time window t _i for one or more of the ego vehicles, the perception value p _i being a function of the driving data of the ego vehicle and the driving data of the surrounding vehicles in a six-vehicle -Neighborhood is. Training procedure according to claim 4 , wherein the perception value p _i comprises or consists of one or more state characteristics of the ego vehicle and the respective surrounding vehicles, wherein preferably the state characteristics of the ego vehicle comprise or consist of longitudinal speed, longitudinal acceleration, lateral acceleration and bearing, and the status characteristics each of the surrounding vehicles in the six-vehicle neighborhood the vehicle roll, the longitudinal velocity, the longitudinal acceleration, the lateral acceleration, the Euclidean distance d between the respective ego vehicle and a respective surrounding vehicle (L ² distance d), the inverse TTC and include or consist of relative bearing. Training method according to one of Claims 1 until 5 , wherein in step b) the computational evaluation of the criteria for compliance is designed in such a way that the conformity of the basic truth trajectory x and/or the alternative long-term trajectories x _i at a given point in time t _i with the requirements of statutory driving regulations is taken into account and the evaluation of the criteria is designed for long-term feasibility in complex driving situations in such a way that the executability of the respective trajectory is taken into account by the underlying vehicle simulation without violating physical laws and vehicle-related framework conditions. Training method according to one of Claims 1 until 6 , where the ground truth preference ŷ is set to ŷ = 1 and the preference value y _i of the alternative long-term trajectories is calculated according to the function y _i = M(x _i ) ∈ [0 ... 1]. Training method according to one of Claims 1 until 7 , wherein in step d) the predetermined threshold value for the assigned preference value y _i is set to 50% or more, alternatively 60% or more, alternatively 80% or more, alternatively 90% or more of the respective basic truth value ŷ. Training method according to one of Claims 1 until 8th , wherein in step c) the generation of one or more viable alternative long-term candidate trajectories x _i for given time windows t _i comprises or consists of i) transforming the ground truth trajectory x for the respective ego vehicles to form the position of the ego vehicles of distance parameters θ̂̂ relative to the roadway, ii) generating one or more random alternative long-term candidate trajectories x _i , and iii) calculating the preference y _i for the distance parameters θ̂ by generating one or more trajectory points and comparing them point by point with the respective distance parameters θ̂̂ of the ground truth trajectory x. Training procedure according to claim 9 , whereby according to i) the basic truth trajectory x for the respective ego vehicle is transformed into a Frenet frame, and/or according to i) the basic truth trajectory x is simplified to six distance parameters θ̂̂ (θ̂ ₁ on θ̂ ₆ ) that describe θ̂ ₁ s -Distance to first lane change, θ̂ ₂ s distance to first lane change, θ̂ ₃ s distance to second lane change, θ̂ ₄ s distance to second lane change, θ̂̂ ₅ w distance to first lane change, and θ̂̂ ₆ w distance of the second lane change, and/or according to ii) the generation of alternative long-term trajectories x _i comprises the generation of a large number of random long-term trajectories x _i that are sampled uniformly in boundaries for the distance parameter space θ and the generation of a large number of random long-term trajectories x _i that are derived from a normal distribution around the distance parameter space 0 can be chosen with the same constraints. Training procedure according to claim 10 , whereby according to iii) 5 or more, alternatively 10 or more, alternatively 20 or more, alternatively 30 or more, alternatively 40 or more, alternatively 50 or more random long-term trajectories x _i are generated, which are sampled uniformly in boundaries for the distance parameter space θ and 5 or more, alternatively 10 or more, alternatively 20 or more, alternatively 30 or more, alternatively 40 or more, alternatively 50 or more random long-term trajectories x _i are generated, which are selected from a normal distribution around the distance parameter space 0 with the same restrictions . Training method according to one of Claims 1 until 11 , wherein in step d) the computer model for high-level decision-making in a driving situation, including a complex driving situation, is trained with one or more long-term trajectories x _i having associated preference values y _i that exceed the predetermined threshold value, and with one or more long-term trajectories x _i are trained that have associated preference values y _i that fall below the predetermined threshold. Training method according to one of Claims 1 until 12 , wherein in step d) the computer model for high-level decision-making in a driving situation that includes a complex driving situation is configured to select as long-term candidate trajectory x _c the trajectory with the highest associated preference value y _c from a set of long-term candidate trajectories x _i selects that exceed the predetermined threshold to be performed by the traffic agent. A computer system for training a traffic agent navigating a road vehicle in a driving situation that includes a complex driving situation of a simulation environment that includes or consists of one or more processors, a memory device coupled to the one or more processors, and a traffic agent learning system that includes a uses a neural network model for decision-making in a driving situation involving a complex driving situation, which is stored in the storage device and configured to be executed by the one or more processors, characterized in that the traffic agent learning system is configured such that it is the computer-implemented training method according to one of Claims 1 until 13 executes Computer system for simulating a road driving environment in a driving situation that includes a complex driving situation for one or more vehicles, comprising or consisting of one or more processors, a memory device coupled to the one or more processors, and a traffic agent that uses a neural network model for decision-making in of a driving situation involving a complex driving situation stored in the storage device and configured to be executed by the one or more processors, characterized in that the traffic agent is trained according to the computer-implemented training method according to any one of Claims 1 until 13 is trained to select, as an action, a candidate trajectory x _i from the viable alternative trajectories x _i to be selected by the traffic agent are to be supplied, with the correspondingly assigned preference value y _c exceeding a predetermined threshold value. Computer system for training a traffic agent according to Claim 14 or for simulating a road traffic environment in a driving situation including a complex driving situation according to claim 15 , wherein the neural network model for decision-making in a complex driving situation is a neural network model with two classes, preferably with up to 32 neurons in the first layer and up to 32 neurons in the second layer, or a neural network model with four classes, preferably with up to 96 neurons in the first layer and up to 16 neurons in the second layer.

Images