DE102023200080A1 - Method and control device for operating a vehicle - Google Patents

Abstract

The present invention relates to a method for operating a vehicle (100), wherein a family (104) of trajectory candidates (106) for fulfilling a driving task of the vehicle (100) is generated using a deep learning-based trajectory generator (200), the trajectory candidates (106) are read in by a cost-based trajectory selector (202) and are evaluated with a cost value (210) based on predefined criteria (206), wherein the trajectory candidate (106) with the best cost value (210) is selected from the family (104) and passed to a vehicle controller of the vehicle (100) for execution.

Description

Field of the invention

The invention relates to a method for operating a vehicle, a corresponding control unit and a corresponding computer program product.

State of the art

An autonomous, semi-autonomous, automated or partially automated vehicle can independently carry out a driving task within the limits of the respective system. To do this, a planning module in the vehicle plans a trajectory to complete the driving task, taking into account traffic events in the area of the vehicle. The planned trajectory is then processed by a control module in the vehicle.

Disclosure of the invention

Against this background, the approach presented here presents a method for operating a vehicle, a corresponding control device and a corresponding computer program product according to the independent claims. Advantageous further developments and improvements of the approach presented here emerge from the description and are described in the dependent claims.

Advantages of the invention

A deep learning-based planning module of an autonomous, semi-autonomous, automated or partially automated vehicle usually outputs a single trajectory without it being possible to understand why exactly this trajectory is output by the planning module. There is a risk that the planning module may randomly output the correct trajectory, but through deep learning it may learn incorrect criteria for internally selecting this trajectory. For example, in a previously unexperienced borderline situation, these incorrect criteria can lead to an incorrect trajectory, even though the same criteria have previously led to drivable trajectories in simple traffic situations.

In the approach presented here, the deep learning-based planning module is given a driving task and the deep learning-based planning module generates several different trajectories to choose from. The deep learning is only used for planning and not for selection. The different trajectories are then evaluated according to a uniform, comprehensible scheme and the evaluations are compared with each other. The best-rated trajectory is then selected and executed or driven.

The approach presented here makes it possible to understand why and how the trajectory to be executed was selected. This ensures that in simple traffic situations the correct trajectory is not output by chance.

A method for operating a vehicle is proposed, wherein a family of trajectory candidates for fulfilling a driving task of the vehicle is generated using a deep learning-based trajectory generator, the trajectory candidates are read in by a cost-based trajectory selector and are evaluated with a cost value based on predefined criteria, wherein the trajectory candidate with the best cost value is selected from the family and passed to a vehicle controller of the vehicle for execution.

Ideas for embodiments of the present invention can be considered, among other things, as being based on the thoughts and findings described below.

A vehicle can be an autonomous, semi-autonomous, automated or partially automated vehicle. A driving task of the vehicle can be specified, for example, by a navigation system. The driving task can be, for example, to drive a section of a planned route. The section can, for example, lead from a vehicle position to an intermediate destination along the route. The intermediate destination can in particular be within a detection range of the vehicle's sensors.

A trajectory generator can read environmental information about an environment of the vehicle and objects in the environment from the vehicle's sensors. The trajectory generator can then create several trajectory candidates to drive through the environment on different paths to the intermediate destination.

As the vehicle drives through the environment, the intermediate destination can be moved further and further along the route so that the vehicle does not reach the intermediate destination until the destination of the route is reached.

A trajectory candidate can be a suggestion for a trajectory of the vehicle. The trajectory actually used can be selected from several different trajectory candidates.

A group can consist of at least two different trajectory candidates. The trajectory candidates of the group can at least start at a common point. The trajectory candidates do not necessarily all end at the same end point. The end points of the trajectory candidates can lie in the area of the intermediate goal.

A trajectory selector can examine properties of the trajectory candidates and evaluate them, taking into account specified criteria, for example with a score, grade or numerical value. The scores, grades or numerical values are summarized to form an overall rating for each trajectory candidate. The overall rating represents the effort required to drive the respective trajectory candidate. The effort can be referred to as costs. The rating for each trajectory candidate is created using a uniform rating scheme. The rating scheme covers different properties or criteria. The rating makes the trajectory candidates comparable.

The criteria can be changed. The criteria are changed globally, i.e. uniformly for all trajectory candidates. This ensures comparability.

A best cost value can correspond to the highest or lowest score, grade or numerical value depending on the evaluation scheme. The different cost values can be output to help understand the selection.

Based on the criteria, at least one driving quality of the respective trajectory candidate can be reflected in the cost value of the trajectory candidate. A driving quality can evaluate how pleasant the trajectory candidate could be to drive. For example, a curvature of the trajectory candidate and/or a jolt while driving the trajectory candidate can be evaluated. By taking the driving quality into account, a pleasant ride for passengers of the vehicle can be achieved. A strongly curved trajectory candidate can be downgraded. Likewise, a trajectory candidate with jerky speed changes can be downgraded.

One of the criteria that can be evaluated is whether the vehicle is keeping to a lane. For example, a trajectory candidate can cut a curve and violate an adjacent lane in the process. The trajectory candidate can be downgraded due to the cutting, as this makes driving behavior difficult for other road users to predict and can lead to dangerous traffic situations. The trajectory candidate can also be downgraded if the vehicle would stagger back and forth within its lane when driving along the trajectory candidate. Lane compliance can be evaluated by the distance of the trajectory candidate from the center of the lane.

Completion of the driving task can be evaluated as one of the criteria. A trajectory candidate can end before the set intermediate goal. The further away the trajectory candidate ends from the intermediate goal, the worse the trajectory candidate can be evaluated.

Individual costs based on the individual criteria can be combined to form the cost value in a deterministic and rule-based manner. The deterministic and rule-based evaluation and merging can make the evaluation comprehensible. Errors in the evaluation can thus be identified and corrected.

The cost value can be determined as a weighted sum of individual costs based on the individual criteria. The individual costs can be included in the cost value using weighting factors. This means that important criteria can have a greater influence on the cost value than less important criteria. The weighting factors can be predefined.

In addition to the family, a movement prediction of other road users can be generated using the trajectory generator. The cost value of each trajectory candidate can also be determined using the movement prediction. A movement prediction can map expected trajectories of other road users. The movement prediction can be created based on current positions or current movements of the other road users and/or past positions or a history of the movements of the other road users. The movement prediction can map when and where the other road users are expected. The trajectory candidates can be checked for conflicts with the movement prediction.

One of the criteria can be a number of collisions between the trajectory candidate and the movement prediction. A collision can be detected if a minimum distance between another road user and the vehicle is exceeded when driving along a trajectory candidate. The trajectory candidates can cross expected trajectories of the other road users. It can be taken into account whether the crossing will take place when the other road user will have already moved away from an intersection point or will only reach the intersection point after the crossing.

Before the evaluation, a selection of trajectory candidates for evaluation can be made from the family using a non-maximum suppression. By using a non-maximum suppression, very similar trajectory candidates can be removed from the family and computational effort can be saved. The selected trajectory candidates then differ significantly from one another in at least one feature.

The method is preferably computer-implemented and can be implemented, for example, in software or hardware or in a mixed form of software and hardware, for example in a control unit.

The approach presented here further creates a control unit for a vehicle, wherein the driver assistance system is designed to carry out, control or implement the steps of a variant of the method presented here in corresponding devices.

The control unit can be an electrical device with at least one computing unit for processing signals or data, at least one storage unit for storing signals or data, and at least one interface and/or a communication interface for reading in or outputting data that is embedded in a communication protocol. The computing unit can be, for example, a signal processor, a so-called system ASIC or a microcontroller for processing sensor signals and outputting data signals depending on the sensor signals. The storage unit can be, for example, a flash memory, an EPROM or a magnetic storage unit. The interface can be designed as a sensor interface for reading in the sensor signals from a sensor and/or as an actuator interface for outputting the data signals and/or control signals to an actuator. The communication interface can be designed to read in or output the data wirelessly and/or via a cable. The interfaces can also be software modules that are present, for example, on a microcontroller alongside other software modules.

Also advantageous is a computer program product or computer program with program code that can be stored on a machine-readable carrier or storage medium such as a semiconductor memory, a hard disk memory or an optical memory and is used to carry out, implement and/or control the steps of the method according to one of the embodiments described above, in particular when the program product or program is executed on a computer or device.

It is noted that some of the possible features and advantages of the invention are described herein with reference to different embodiments. A person skilled in the art will recognize that the features of the controller and the method can be combined, adapted or exchanged in a suitable manner to arrive at further embodiments of the invention.

Short description of the drawings

• 1 zeigt eine Darstellung eines Fahrzeugs in einer Verkehrssituation mit einer Schar Trajektorienkandidaten; und
• 2 zeigt eine Darstellung eines Verfahrensablaufs eines Verfahrens gemäß einem Ausführungsbeispiel.

Embodiments of the invention will now be described with reference to the accompanying drawings, wherein neither the drawings nor the description are to be construed as limiting the invention.

• 1 shows a representation of a vehicle in a traffic situation with a set of trajectory candidates; and
• 2 shows a representation of a method flow of a method according to an embodiment.

The figures are merely schematic and not to scale. Identical reference symbols denote identical or equivalent features.

Embodiments of the invention

1 shows a representation of a vehicle 100 in a traffic situation 102 with a group 104 trajectory candidates 106. The vehicle 100 is traveling with other vehicles 108 or other road users 108 on a multi-lane road. The vehicle 100 is traveling autonomously here. A trajectory planner of the vehicle 100 has generated the group 104 trajectory candidates 106 in order to fulfill a driving task of the vehicle 100. The driving task here consists of changing lanes to another lane of the road in order to overtake another vehicle 108 in the lane of the vehicle 100. The driving task is thus to reach an intermediate destination 110 in the other lane.

2 shows a representation of a procedure of a method according to an embodiment. The procedure is based on the driving task from 1 shown.

In the 1 The traffic situation 102 shown is read in by a trajectory planner 200. The initial situation 102 is shown in simplified form here. The vehicle 100 is driving in its lane and is supposed to reach its intermediate destination 110 in the adjacent lane.

In the adjacent lane, another vehicle 108 is traveling next to the vehicle 100. The other vehicle 108 is traveling slightly forward of the vehicle 100.

The trajectory planner 200 outputs three trajectory candidates 106 for the vehicle 100. The trajectory candidates 106 are evaluated in a trajectory selector 202 according to a uniform evaluation scheme 204 with the same criteria 206.

Here, the trajectory candidates 106 are evaluated, for example, according to the number of collisions and the distance to the intermediate target 110.

The number of collisions is determined using a motion prediction 208 of the other vehicle 108. The motion prediction 208 is also output by the trajectory generator 200. Here, the other vehicle 108 continues to drive in its lane.

The first trajectory candidate 106 continues in the lane of the vehicle 100 and does not change to the other lane. As a result, the first trajectory candidate 106 remains collision-free, but also misses the intermediate target 110 by one lane width.

The second trajectory candidate 106 decelerates the vehicle 100 and changes to the other lane behind the other vehicle 108. As a result, the second trajectory candidate 106 also remains collision-free and reaches the intermediate destination 110 to within less than one meter.

The third trajectory candidate 106 accelerates the vehicle 100 and moves into the other lane in front of the other vehicle 108. As a result, the third trajectory candidate 106 reaches the intermediate destination 110 to within less than one meter. The other vehicle 108 would, however, have to brake to avoid a collision.

The trajectory selector 202 maps the evaluations into a cost value 210 and selects the trajectory candidate 106 with the best cost value 210 for controlling the vehicle 100.

In the following, possible embodiments of the invention are summarized again or presented using slightly different wording.

A cost-based trajectory selection is presented.

The task of a planning module in an autonomous vehicle is to calculate a safe, drivable path while meeting additional requirements such as passenger comfort, compliance with traffic rules, and reaching a specific destination. In most systems, the planner module receives inputs from a perception module. This includes the static environment, such as road boundaries and lanes, and the state of other road users, including past ones. Assuming that the planner has information about the current and previous position, the size of the ego vehicle and other road users, and some meaningful road information, the planner is supposed to produce a path proposal. This proposal is refined and then applied by the vehicle controller to reach a predefined destination in a safe manner.

Neural network-based planner modules typically provide a single trajectory for the ego vehicle. The quality of this trajectory in terms of safety and comfort depends on the neural network's decision at inference time. So far, there is no supervised or explainable reasoning for the neural network's decision for the one planned trajectory.

Modern deep learning (DL)-based planner modules can provide multiple possible trajectories for different inputs and goals. They can generate different trajectories for the ego vehicle and the other agents in traffic. However, the reasons for selecting the most likely trajectory are unknown.

The main idea of the presented approach is to use a stochastic deep learning (DL) based planner to generate candidate trajectories and then use an evaluation mechanism to evaluate these trajectories based on various (modifiable) criteria of ride quality, e.g., curvature, judder, lane keeping, etc. The best trajectory is selected and passed to the motion controller to control the vehicle.

The calculation of the score is a deterministic, rule-based procedure that can be verified and tested, which provides an additional level of security. Due to the mechanism for selection of trajectories, there is a level of explainability that is lacking in purely machine learning-based systems. The presented method reduces the role of chance in decision making while retaining all the capabilities of modern neural network-based models. 1 shows an example of selecting a more feasible trajectory instead of the most probable trajectory, which normally follows the lane on the highway.

The following section describes the overall structure of the presented method. The planner system consists of two main parts, the trajectory generator and the point-based trajectory selector.

The trajectory generator is a path planner that can generate several possible ego-vehicle paths for the future. The path can be defined as a series of consecutive locations, where each element describes where the ego-vehicle and the other agents should be in the future. In addition, it can contain other features, such as the required speed or orientation for each frame.

There are two requirements for this module. It should be able to generate multiple trajectories for an agent and provide predictions for the whole scene (i.e. also for the future trajectories of the other agents). The first capability is usually some kind of stochastic procedure. Returning the whole scene is usually a feature of behavior prediction methods, but they are not tied to the goal, so there is no way to influence the ego route, which is a desired functionality here.

In practice, a latent variable neural network has proven to be effective, but any other stochastic model can be used. Specifically, a system is used that uses a graph neural network to encode the static and dynamic environment, with a latent neural network conditioned on the ego target positions (either predicted or externally specified) to output trajectories based on points.

The score-based trajectory selector estimates the quality of possible ego trajectories provided by the trajectory generator with respect to the predicted movements of other agents using a rule-based score.

Another way to implement the proposed functionality is to use multiple future scenarios generated by the trajectory generator, where each scenario contains a predicted trajectory for each dynamic obstacle. Instead of simple trajectories, posterior distributions for each time window for each dynamic obstacle can also be used.

Any score can be defined for ego paths, but there are some scores that provide a significant improvement in the quality and explainability of the planned path. For example, the number of collisions in the predicted future periods can be counted for each path candidate. Also, lane compliance can be evaluated based on the average distance to the lane centerline. If a goal is given by a specific position, the distance between the end of the path candidate and the goal can be easily calculated to evaluate goal compliance.

The selection module takes the trajectories provided by the trajectory generator and then optionally selects a set of significantly different trajectories by non-maximal suppression. It then calculates the scores for each trajectory. The final score of a trajectory is a weighted sum of the scores. The planner selects the trajectory with the highest score. The main steps are in 2 using a simple traffic situation. The method described works in inference time, when the already trained deep learning model runs to generate a path for the ego vehicle.

In 1 one trajectory is the most likely, another trajectory is the result of the score-based planner, and a third trajectory is the base probability.

In 2 the trajectory generator provides a prediction for the other agent(s) and this prediction is used for collision prediction in case of multiple planned ego trajectories. The second ego trajectory is safe because the ego vehicle does not change lanes until the conflicting agent has overtaken it.

Traditionally, any viable motion planning solution requires DL technology. DL-based motion planners are usually black-box models, which hinders their direct application to safety-critical problems, such as self-driving. On the other hand, using a point-based planner as a safety layer on top of the DL model ensures explainability by the quality and safety of the planned trajectory is assessed. A collision estimate or a convenient metric value can be communicated to the user (e.g. on the on-board computer screen).

The presented approach could be a good extension for any planner module that provides multiple trajectory suggestions for the ego vehicle and makes predictions about the future behavior of other road users.

Finally, it should be noted that terms such as "comprising", "including", etc. do not exclude other elements or steps, and terms such as "a" or "an" do not exclude a plurality. Reference signs in the claims are not to be considered as limitations.

Claims (12)

Method for operating a vehicle (100), wherein a family (104) of trajectory candidates (106) for fulfilling a driving task of the vehicle (100) is generated using a deep learning-based trajectory generator (200), the trajectory candidates (106) are read in by a cost-based trajectory selector (202) and are evaluated with a cost value (210) based on predefined criteria (206), wherein the trajectory candidate (106) with the best cost value (210) is selected from the family (104) and passed to a vehicle controller of the vehicle (100) for execution. Procedure according to Claim 1 in which, based on the criteria (206), at least one driving quality of the respective trajectory candidate (106) is mapped into the cost value (210) of the trajectory candidate (106). Method according to one of the preceding claims, in which keeping to a lane is evaluated as one of the criteria (206). Method according to one of the preceding claims, in which fulfillment of the driving task is evaluated as one of the criteria (206). Method according to one of the preceding claims, in which individual costs based on the individual criteria (206) are combined deterministically and rule-based to form the cost value (210). Method according to one of the preceding claims, wherein the cost value (210) is determined as a weighted sum of individual costs based on the individual criteria (206). Method according to one of the preceding claims, in which, in addition to the family (104), a movement prediction (208) of other road users (108) is generated using the trajectory generator (200), and the cost value (210) of each trajectory candidate (106) is further determined using the movement prediction (208). Procedure according to Claim 7 , in which as one of the criteria (206) a number of collisions between the trajectory candidate (106) and the motion prediction (208) is evaluated. Method according to one of the preceding claims, in which a selection of trajectory candidates (106) for the evaluation is made from the family (104) before the evaluation using a non-maximum suppression. Control device for operating a vehicle (100), wherein the control device is configured to execute, implement and/or control the method according to one of the preceding claims in corresponding devices. Computer program product which is designed to instruct a processor, when executing the computer program product, to carry out the method according to one of the Claims 1 until 9 to execute, implement and/or control. Machine-readable storage medium on which the computer program product according to Claim 11 is stored.

Images