KR102179835B1 - Method and system for path prediction considering vehicle travel intention and object selection algorithm based on vehicle sensor including wireless communication - Google Patents

Abstract

Disclosed is a vehicle sensor-based object selection algorithm including wireless communication, and a route prediction method and system in consideration of driving intention of surrounding vehicles. In one embodiment, the route prediction method includes generating fusion data using information from a plurality of sensors associated with an own vehicle and information received through external communication, and the vehicle of interest using the generated fusion data. Selecting, determining a lane change intention of the selected vehicle of interest, and generating, by the at least one processor, a predicted path of the selected vehicle of interest according to the determined lane change intention. .

Description

Object selection algorithm based on vehicle sensor including wireless communication and route prediction method and system that considers driving intention of surrounding vehicles {METHOD AND SYSTEM FOR PATH PREDICTION CONSIDERING VEHICLE TRAVEL INTENTION AND OBJECT SELECTION ALGORITHM BASED ON VEHICLE SENSOR INCLUDING WIRELESS COMMUNICATION}

The following description relates to a vehicle sensor-based object selection algorithm including wireless communication, and a route prediction method and system in consideration of driving intentions of surrounding vehicles.

An autonomous vehicle refers to a vehicle that runs on its own even if the driver does not operate the vehicle. For example, Korean Patent Laid-Open No. 10-2018-0086632 relates to an apparatus and method for determining the behavior of an autonomous vehicle. When there is an object standing in front of the autonomous vehicle or an object traveling at a low speed, the road Disclosed is an apparatus and method for determining the behavior of an autonomous vehicle that can accurately determine the driving situation of the vehicle.

On the other hand, many vehicles including ADAS (Advanced Driver Assistance System) at the level of autonomous driving level 2 have been popular, but for ADAS to work, technology that selects a target vehicle to belong to the driving path of the own vehicle through information on surrounding vehicles and driving road environment. Is required. Recently, road environment such as road curvature has been reflected to improve the ability to follow the preceding vehicle even on curved roads, but when a nearby vehicle changes lanes to the driving path of the own vehicle, the vehicle selected as the target suddenly changes, resulting in unnecessary deceleration and reduced ride comfort. There is a problem that causes. In addition, there is a problem in that there is a hardware limitation in real-time processing because there is too much data on vehicles other than the own vehicle acquired from sensors including communication equipment.

A route prediction method capable of predicting a highly reliable route by fusing information obtained from surrounding vehicles through V2X by the own vehicle, ADAS sensor and vehicle internal sensor data values, a computer device performing the method, the computer device and In combination, there is provided a computer program stored on a computer-readable recording medium and a recording medium therefor for executing the method on a computer device.

A path prediction method capable of reducing the computational load by defining a path prediction target object, a computer device that performs the method, a computer program combined with the computer device and stored in a computer-readable recording medium to execute the method on a computer device And its recording media.

A method for predicting a path of a computer device including at least one processor, wherein the at least one processor generates fusion data using information from a plurality of sensors associated with an own vehicle and information received through external communication. Generating; Selecting, by the at least one processor, a vehicle of interest using the generated fusion data; Determining a lane change intention of the selected vehicle of interest, by the at least one processor; And generating, by the at least one processor, a predicted route of the selected vehicle of interest according to the determined lane change intention.

According to one aspect, the generating of the fusion data includes: when current first information is received for a first vehicle through the external communication, generating fusion data using the first information; Generating fusion data using the second information when the second information previously received for the first vehicle is present without receiving the first information; And when the second information does not exist without receiving the first information, generating fusion data using information from a plurality of sensors.

According to another aspect, the step of generating fusion data using information from the plurality of sensors includes generating the fusion data using a weighted average of the output values of each of the plurality of sensors, and The weight to be used may be determined using an reciprocal of an error covariance for an output value of each of the plurality of sensors.

According to another aspect, the plurality of sensors includes a radar, a lidar, and a camera, and generating fusion data using information from the plurality of sensors comprises: the radar, the It may be characterized in that the fusion data is generated by detecting neighboring vehicle information using a lidar and an output value of the camera, and detecting lane information using the camera.

According to another aspect, the fusion data may include at least one of a lateral velocity of the vehicle and a lateral offset of the vehicle.

According to another aspect, the selecting of the vehicle of interest may include selecting a vehicle having a center located in a plurality of preset regions around the host vehicle as the vehicle of interest.

According to another aspect, the plurality of areas include a first area in a left lane in front of the host vehicle, a second area in a lane in front of the host vehicle, a third area in a right lane in front of the host vehicle, a fourth area in the left lane of the host vehicle, and It may be characterized in that it includes a fifth area in a right lane of the vehicle, a sixth area in a left lane behind the own vehicle, a seventh area in a rear lane of the own vehicle, and an eighth area in a right lane behind the own vehicle.

According to another aspect, a length of each of the fourth and fifth regions among the plurality of regions is determined based on the length of the own vehicle, and the length of each of the remaining regions is the speed of the own vehicle and the It may be characterized in that it is determined by the acceleration of the own vehicle and a preset threshold.

According to another aspect, the determining of the lane change intention includes maintaining the lane of the selected vehicle of interest, changing the left lane, and changing the right lane using a machine learning model pre-learned through data on actual vehicles. It can be characterized by determining an intention for one.

According to another aspect, the machine learning model may be trained using a Gaussian kernel function.

According to another aspect, the generating of the predicted path may include: generating a lane change prediction path for a first vehicle that changes lanes to a lane of the own vehicle among the selected vehicles of interest; And generating a lane maintenance prediction path for a second vehicle traveling in a lane of the own vehicle among the selected vehicles of interest.

According to another aspect, the generating of the lane change prediction route may include: predicting a lane change time required based on a lateral distance and a lateral speed required for lane change of the first vehicle; Predicting a longitudinal distance between the first vehicle and the own vehicle based on a longitudinal speed of the first vehicle and a longitudinal acceleration of the first vehicle; And predicting a lateral distance between the first vehicle and the own vehicle using a lateral distance required for a lane change of the first vehicle, a time required for the lane change, and a sinusoidal function. I can.

According to another aspect, the generating of the lane maintenance prediction route includes: a distance from the lane of the second vehicle, a heading angle error of the second vehicle, a curvature of a road, and a curvature of the road. It may be characterized in that the lane keeping prediction path is generated based on at least one of a curve rate of.

There is provided a computer-readable recording medium in which a computer program for executing the method is recorded on a computer device.

In combination with a computer device, there is provided a computer program stored on a computer-readable recording medium for executing the method on a computer device.

A computer device comprising at least one processor embodied to execute an instruction readable by the computer device, by means of the at least one processor, through information and external communication from a plurality of sensors associated with the host vehicle Generates fusion data using received information, selects a vehicle of interest using the generated fusion data, determines a lane change intention of the selected vehicle of interest, and selects the selected interest according to the determined lane change intention It provides a computer device, characterized in that generating a predicted path of the vehicle.

By fusing the information obtained from surrounding vehicles through V2X, the ADAS sensor, and the information obtained by the vehicle internal sensor data value, the own vehicle can predict a highly reliable path, and reduce the computational load by limiting the target object for path prediction.

1 is a block diagram showing an example of a computer device according to an embodiment of the present invention.
FIG. 2 is a diagram illustrating an example of a general appearance of a route prediction system according to an embodiment of the present invention.
3 is a view showing an example of the internal configuration of the sensor unit 210 according to an embodiment of the present invention.
4 is a view showing an example of the internal configuration of the intention determination unit according to an embodiment of the present invention.
FIG. 5 is a diagram illustrating an example for selecting an object candidate group that may affect the driving of an own vehicle according to an embodiment of the present invention.
6 is a diagram showing an example of a lane change intention determination according to an embodiment of the present invention.
7 is a diagram illustrating an example of an internal configuration of a path prediction unit according to an embodiment of the present invention.
8 is a diagram showing an example of a lane change route prediction according to an embodiment of the present invention.
9 is a diagram illustrating an example of predicting a lane keeping route according to an embodiment of the present invention.
10 is a diagram illustrating an example for explaining a variable for predicting a lane keeping route according to an embodiment of the present invention.
11 to 13 are flowcharts illustrating an example of a path prediction method according to an embodiment of the present invention.

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings.

The route prediction method according to the embodiments of the present invention may be performed by a computer device to be described later. For example, a computer program according to an embodiment of the present invention may be installed and driven in a computer device, and the computer device may perform a path prediction method according to an embodiment of the present invention under control of the driven computer program. I can. The above-described computer program may be combined with a computer device and stored in a computer-readable recording medium to execute a path prediction method on the computer device. The computer program described herein may have a form of an independent program package, or a form of an independent program package may be pre-installed on a computer device to be linked to an operating system or other program packages. Meanwhile, such a computer device may be disposed, installed, or located in an autonomous vehicle.

1 is a block diagram showing an example of a computer device according to an embodiment of the present invention. For example, the path prediction method according to embodiments of the present invention may be executed by the computer device 100 shown in FIG. 1.

As illustrated in FIG. 1, the computer device 100 may include a memory 110, a processor 120, a communication interface 130, and an input/output interface 140. The memory 110 is a computer-readable recording medium and may include a permanent mass storage device such as a random access memory (RAM), read only memory (ROM), and a disk drive. Here, a non-destructive large-capacity recording device such as a ROM and a disk drive may be included in the computer device 100 as a separate permanent storage device separate from the memory 110. In addition, an operating system and at least one program code may be stored in the memory 110. These software components may be loaded into the memory 110 from a computer-readable recording medium separate from the memory 110. Such a separate computer-readable recording medium may include a computer-readable recording medium such as a floppy drive, disk, tape, DVD/CD-ROM drive, and memory card. In another embodiment, software components may be loaded into the memory 110 through a communication interface 130 other than a computer-readable recording medium. For example, software components may be loaded into the memory 110 of the computer device 100 based on a computer program installed by files received through the network 160.

The processor 120 may be configured to process instructions of a computer program by performing basic arithmetic, logic, and input/output operations. Instructions may be provided to the processor 120 by the memory 110 or the communication interface 130. For example, the processor 120 may be configured to execute a command received according to a program code stored in a recording device such as the memory 110.

The communication interface 130 may provide a function for the computer device 100 to communicate with other devices (eg, storage devices described above) through the network 160. For example, requests, commands, data, files, etc., generated by the processor 120 of the computer device 100 according to a program code stored in a recording device such as the memory 110, are transmitted through the network ( 160) can be delivered to other devices. Conversely, signals, commands, data, files, etc. from other devices may be received by the computer device 100 through the communication interface 130 of the computer device 100 via the network 160. Signals, commands, data, etc. received through the communication interface 130 may be transferred to the processor 120 or the memory 110, and the file, etc. may be a storage medium (described above) that the computer device 100 may further include. Permanent storage).

The input/output interface 140 may be a means for an interface with the input/output device 150. For example, the input device may include a device such as a microphone, a keyboard, or a mouse, and the output device may include a device such as a display or a speaker. As another example, the input/output interface 140 may be a means for interfacing with a device in which input and output functions are integrated into one, such as a touch screen. The input/output device 150 may be configured with the computer device 100 and one device.

In addition, in other embodiments, the computer device 100 may include fewer or more components than the components of FIG. 1. However, there is no need to clearly show most of the prior art components. For example, the computer device 100 may be implemented to include at least a portion of the input/output device 150 described above, or may further include other components such as a transceiver and a database.

The communication method is not limited, and not only a communication method using a communication network (for example, a mobile communication network, wired Internet, wireless Internet, broadcasting network) that the network 160 may include, but also Bluetooth or NFC (Near Field Communication) Short-range wireless communication, such as, may also be included. For example, the network 160 includes a personal area network (PAN), a local area network (LAN), a campus area network (CAN), a metropolitan area network (MAN), a wide area network (WAN), and a broadband network (BBN). , Internet, and the like. In addition, the network 160 may include any one or more of a network topology including a bus network, a star network, a ring network, a mesh network, a star-bus network, a tree or a hierarchical network, etc. Not limited.

In the route prediction method according to the embodiments of the present invention, unlike the existing prior art, the information obtained from the sensor of the own vehicle is obtained by using a Basic Safety Message (BSM) from surrounding vehicles through V2X (Vehicle to Everything). It is possible to provide sensor fusion data by fusion with information from the sensor. In addition, in the route prediction method according to the embodiments of the present invention, in order to compensate for the communication delay and slow information update speed, which are disadvantages of V2X, it is checked whether the object detected from another sensor is an object that has been detected before, , It is possible to improve the reliability of sensor fusion data by using a dynamic model for a Kalman filter by using only specification information among past BSM data for a corresponding object. At this time, the number of surrounding vehicles that are the target of interest are vehicles that satisfy a specific condition and limit the number of vehicles to the maximum m (where m is a natural number, for example, 8), thereby reducing the load on calculation. In addition, in the route prediction method according to the embodiments of the present invention, based on fusion data, machine learning is used to determine whether a lane change of a neighboring vehicle is determined, and a path prediction model for a neighboring vehicle that affects the driving lane of the own vehicle. The load on the operation can be reduced by applying.

FIG. 2 is a diagram illustrating an example of a general appearance of a route prediction system according to an embodiment of the present invention. The path prediction system 200 according to the present embodiment may include a sensor unit 210, an intention determination unit 220, and a path prediction unit 230.

The sensor unit 210 may acquire information on surrounding vehicles and environments using the V2X and ADAS sensors, and may generate fusion data using the acquired information.

The intention determination unit 220 may determine the intention to change lanes of the surrounding vehicles using machine learning with the generated fusion data.

In addition, the path prediction unit 230 may process a path prediction method that varies depending on the intention determination result.

In one embodiment, the path prediction system 200 may include the computer device 100 described with reference to FIG. 1. For example, the computer device 100 may be included in the path prediction system 200 of an autonomous vehicle, and may be linked with sensors included in the path prediction system 200.

3 is a view showing an example of the internal configuration of the sensor unit 210 according to an embodiment of the present invention. The sensor unit 210 may collect data through the radar 310, the lidar 320, the camera 330, and the V2X 340, and the surrounding vehicle information detection module 350, the lane information detection module 360 ) And a sensor information fusion module 370. Depending on the embodiment, the sensor unit 210 may directly include a radar 310, a lidar 320, a camera 330, and a V2X 340, but the radar 310, the lidar 320, the camera ( At least one of the 330 and V2X 340 may be formed separately from the sensor unit 210 (eg, mounted on an autonomous vehicle) to communicate with the sensor unit 210. Here, " ρ " shown in FIG. 3 is a relative distance, " θ " is an angle, " v " is a relative velocity, " x " is a lateral coordinate, " y " is a longitudinal coordinate, and " α "" may mean acceleration, and " W _obj " may mean object width. In addition, “ W _lane ” may mean the width of a lane, “ ρ _lane ” may mean a road curvature, and “ ψ ” may mean a yaw angle.

The surrounding vehicle information detection module 350 uses the information collected through the radar 310, the lidar 320, and the camera 330 to determine the location, speed, acceleration, and/or size of the surrounding vehicle (the road and /Or width) and other surrounding vehicle information can be detected.

The lane information detection module 360 may use the camera 330 to obtain information on a driving environment such as lane information such as a curvature of a road and a lane width.

In addition, as described above, the sensor unit 210 may collect V2X information from other sensors (for example, a sensor of a nearby vehicle and/or a sensor disposed on a road) through the V2X 340. .

At this time, the V2X 340 has a disadvantage in that the update period of the data (V2X information) is slower than that of other sensors (the radar 310, the lidar 320, and the camera 330). Therefore, if V2X information is not newly received, V2X information cannot be used. However, if the currently detected object (the surrounding vehicle) is an object with a previously detected history and has received V2X information from the object, the sensor unit 210 may be configured according to time such as tire cornering stiffness and mass of the object. Unchanging specification information can be recycled. For example, according to the Society of Automotive Engineers (SAE) J2735 BSM standard, BSM part2 DF_VehicleData is a data frame that utilizes information other than part1, so it can be used with specification information in the frame. Conversely, when the V2X information is collected, the sensor unit 210 can trust and use the V2X information. This is because the reliability of V2X information is relatively higher than that of the object measured by the ADAS sensor.

In addition, according to an embodiment, the sensor unit 210 may further collect and utilize information from a vehicle speed sensor or an inertial measurement unit (IMU) built into the own vehicle.

The sensor information fusion module 370 may create fusion data by fusing the collected data. For example, the fusion data may be generated using the weighted average method presented in Equation 1 below.

"는 오차 공분산의 역수(inverse of covariance matrix)를 의미할 수 있다. 이때, 수학식 1에서는 오차 공분산의 역수(

)를 가중치로 산정하기 때문에 오차 공분산이 큰 정보는 융합 데이터 형성에 낮게 반영될 수가 있고, 오차 공분산이 낮은 정보는 크게 반영될 수가 있게 된다. 추가적으로, 센서 정보 융합 모듈(370)에서는 환경 정보를 반영하여 기계학습법에 적용시키기 위한 특징값을 생성할 수도 있다.Here, " x " may mean data. For example, “ x _camera ” refers to data collected through the camera 330, “ x _radar ” refers to data collected through the radar 310, and “ x _lidar ” refers to data collected through the _lidar 320 Can mean each. Also, " x _avg " may mean a weighted average of collected data. At this time, "calculated by weight"

"May mean an inverse of covariance matrix. In this case, in Equation 1, the inverse of the error covariance (

) Is calculated as a weight, so information with a large error covariance can be lowly reflected in fusion data formation, and information with a low error covariance can be largely reflected. Additionally, the sensor information fusion module 370 may reflect environmental information and generate a feature value for application to the machine learning method.

4 is a view showing an example of the internal configuration of the intention determination unit according to an embodiment of the present invention. As already described, the intention determination unit 220 may determine the intention to change lanes of surrounding vehicles. The intention determination unit 220 may include an interest vehicle selection module 410, a pre-training module 420, and an intention determination module 430.

The intention determination unit 220 may receive the fusion data and feature values calculated by the sensor information fusion module 370, which is the final step of the sensor unit 210, and determine the intention of the surrounding vehicles for lane change through machine learning. have. In this case, the intention determination may not proceed with all objects, but may be performed by selecting a candidate group that may affect the driving of the own vehicle. SVM (Support Vector Machine) can be used as a machine learning method used to determine intention, and as output values, for example, 0 (Lane Keeping), 1 (Left Lane Change), 2 (Right Lane Change) may be included.

FIG. 5 is a diagram illustrating an example for selecting an object candidate group that may affect the driving of an own vehicle according to an embodiment of the present invention. As shown in FIG. 5, an area that can affect the driving of the own vehicle 510 may be divided into eight areas around the own vehicle 510. In this case, each region may be described as shown in Table 1 below.

area Explanation FL Left lane in front of own vehicle F The lane ahead of the own vehicle FR Right lane in front of own vehicle HL To the left of the own vehicle HR To the right of the own vehicle RL Left lane behind own vehicle R In the rear lane of the own vehicle RR The right lane behind the own vehicle

In addition, Table 2 below describes the size of each area. Here, t is a value that can be adjusted to a certain threshold.

Explanation a

b

c

d

(Lane width)

Own vehicle length

Own vehicle acceleration

Own vehicle speed

The vehicle of interest selection module 410 described with reference to FIG. 4 may select an object of interest existing in the region of interest shown in FIG. 5 by using fusion data. In this case, the vehicle of interest selection module may select an object of interest such that at most one object is included in each area by applying a cost function as shown in Equations 2 and 3 below.

In this case, the vehicle-of-interest selection module 410 may determine an area to which the object belongs by using which area the center of the object belongs to. Therefore, the number of nearby vehicles of interest can be limited to a maximum of eight.

6 is a diagram showing an example of a lane change intention determination according to an embodiment of the present invention. When a vehicle of interest is selected, an object (Lateral Velocity of the selected vehicle of interest, a lateral offset) obtained from the fusion data can be used as an input value of the intention determination module 430. Intention determination The module 430 can learn that the lane change is made when the center of the object crosses the lane through the prior learning module 420 (Lane change time), and when the lane change is performed by adjusting the window size. It may be learned to grasp the intention of changing lanes of surrounding vehicles through prediction of whether to do so. Data on actual vehicles may be used for learning by the intention determination module 430 using the pre-learning module 420. In addition, a kernel trick can be used as a Gaussian kernel function as shown in Equation 4. Kernel tricks can be solved in a high dimension because a problem that cannot be solved in a low dimension is solved in a high dimension. In the kernel trick used in SVM, homogeneous polynomial, inhomogeneous polynomial, and the Gaussian kernel function described above are representatively used, and Gaussian kernel for machine learning model learning. Functions can be used.

The learned intention determination module 430 receives data such as the lateral velocity of the vehicle and the lateral offset of the vehicle obtained for the selected vehicle of interest, and is 0 (lane keeping) as described above. )), 1 (Left Lane Change) or 2 (Right Lane Change) can be output.

7 is a diagram illustrating an example of an internal configuration of a path prediction unit according to an embodiment of the present invention. The path prediction unit 230 may include a path prediction object selection module 710, a lane change path prediction module 720, and a lane maintenance path prediction module 730.

The path prediction object selection module 710 may select an object that may have an influence on the own vehicle. For example, the path prediction object selection module 710 may select an object predicted to travel in the same lane as the lane of the own vehicle according to the intention determination result of the intention determination unit 220.

Meanwhile, the path prediction unit 230 may predict the path of the object selected by the path prediction object selection module 710. At this time, the lane change path prediction module 720 may predict the path of the selected object as lane maintenance or lane change (left/right) according to the determination of the intention determination unit 220 that determines the lane change intention of the surrounding vehicles. . For example, when the output value of the intention determination unit 220 for the object selected by the path prediction object selection module 710 is 0, the lane maintenance path is determined, and when the output value is 1, the left lane change path is determined. , When the output value is 2, the path of the right lane change can be predicted, respectively.

)에 해당하는 거리의 합으로 계산될 수 있다.8 is a diagram showing an example of a lane change route prediction according to an embodiment of the present invention. In FIG. 8, variables and equations used in the lane change path prediction module 720 described above with reference to FIG. 7 are described. As shown in Equation 5 below, the lateral distance ( W _des ) required for the object 810 to change lanes is the distance ( W _min ) away from the adjacent lane and half of the lane width (

) Can be calculated as the sum of the distances.

Assuming that it is a constant velocity model, if the lateral distance ( W _des ) obtained through Equation 5 is divided by the lateral speed ( V _lat,obj ) of the object 810, the time required for lane change ( T _total ) is as shown in Equation 6 below. ) Can be obtained.

Assuming that the object moves at an equivalent acceleration, the longitudinal movement distance ( x ) can be obtained as shown in Equation 7 below by using the state value in the longitudinal direction from the fusion data of the sensor unit 210.

In this case, if the lateral behavior model for the object changing lanes is a ramp sinusoid function as shown in Equation 8 below, the lateral distance y of the object over time can be obtained.

The lane maintenance path prediction module 730 may generate a prediction path identical to the path of the lane on which the object is traveling to maintain the current lane.

9 is a diagram illustrating an example of predicting a lane keeping route according to an embodiment of the present invention. The information on the lane on which the object 910 is traveling may be based on information on the lane on which the own vehicle is traveling, detected by the sensor unit 210. At this time, the collected lane information is the distance between the object 910 and the lane (Lateral Offset), the heading angle error of the object 910, the curvature of the road, and/or the rate of change of the curvature of the road (Curvature rate). 9 shows an example of predicting a path for an object 910 to maintain a lane other than an existing path.

10 is a diagram illustrating an example for explaining a variable for predicting a lane keeping route according to an embodiment of the present invention. 10 illustrates the variables of Equations 9 and 10 below.

At this time, in FIG. 10, C ₀ denotes the distance between the object 910 and the lane, C ₁ denotes the driving direction tilt error of the object 910, C ₂ denotes the curvature of the road, and C ₃ denotes the rate of change of the curvature of the road. can do. In addition, in FIG. 10, “ R ” may mean a radius of curvature of a road, and “ e _y ” may mean a lateral offset.

As described above, according to embodiments of the present invention, a highly reliable route can be predicted by fusing information obtained from surrounding vehicles, ADAS sensor, and vehicle internal sensor data value through V2X, and route prediction target object It is possible to reduce the operation load by limiting

11 to 13 are flowcharts illustrating an example of a path prediction method according to an embodiment of the present invention. The path prediction method according to the present embodiment may be performed by the computer device 100 described above as an example. For example, the processor 120 of the computer device 100 may be implemented to execute a code of an operating system included in the memory 110 or a control instruction according to the code of at least one program. Here, the processor 120 includes steps 1110 to 1180, 1210 to 1240, and 1310 of the method of FIGS. 11 to 13 according to the control command provided by the code stored in the computer device 100. To 1330), the computer device 100 may be controlled. For example, steps 1110 to 1180 of FIG. 11 are performed by the sensor unit 210, steps 1210 to 1240 of FIG. 12 are performed by the intention determination unit 220, and steps 1310 to 13 of FIG. Each of 1330 may be performed by the path prediction unit 230.

In step 1110, the computer device 100 may determine whether the current V2X is received. In this case, the computer device 100 may perform steps 1120 and 1130 when V2X information is received, and step 1140 when V2X information is not received.

Steps 1120 and 1130 illustrate an example of a process in which the computer device 100 generates fusion data using the currently received V2X information. As described above, since the reliability of V2X information is relatively higher than the information on the object measured by the ADAS sensor, when V2X information is received, the computer device 100 uses the radar 310, the lidar 320, and the camera. Instead of using an ADAS sensor such as (330), fusion data can be generated using V2X information.

In step 1140, the computer device 100 may determine whether the current object is a previously detected object. At this time, the computer apparatus 100 performs steps 1150, 1160, and 1130 when the current object is a previously detected object, and step 1170 when the current object is not a previously detected object. ), step 1180, and step 1130 may be performed, respectively.

Steps 1150, 1160, and 1130 may be examples of a process in which the computer device 100 generates fusion data through a dynamic model using previously received V2X information. As described above, if the currently detected object (the surrounding vehicle) is an object with a previously detected history and has received V2X information from the object, the sensor unit 210 The specification information that does not change over time such as cornering stiffness and mass can be recycled. In other words, the computer device 100 may generate fusion data using previously received V2X information.

Steps 1170, 1180, and 1130 may be examples of a process in which the computer device 100 generates fusion data for a new object by using a kinematic model. Here, the kinematic model acquires information about the object using ADAS sensors such as radar 310, lidar 320, camera 330 and/or vehicle speed sensor built into the own vehicle, and IMU (Inertial Measurement Unit). It can be a model to do.

In this way, in fusing the data on the environment and the object around the host vehicle detected by the sensor unit 210, the reliability of the object information (V2X information) received through wireless communication is more than when the object is detected only by the ADAS sensor. Convergence data can be generated by reflecting both the fact that it is large and that it takes longer than the ADAS sensor to update object information when wireless communication is used.

In other words, the computer apparatus 100 may generate fusion data by using information from a plurality of sensors linked to the own vehicle and information received through external communication. In this case, when the current first information is received with respect to the first vehicle through external communication, the computer device 100 may generate fusion data using the first information. Also, the computer device 100 may generate fusion data using the second information when the second information previously received for the first object exists without first information being received. Also, when the first information is not received and the second information does not exist, the computer device 100 may generate fusion data by using information from a plurality of sensors. In this case, in generating fusion data using information from a plurality of sensors, the computer apparatus 100 may generate fusion data using a weighted average of output values of each of the plurality of sensors. In this case, the weight used for the weighted average may be determined using an reciprocal of the error covariance for the output values of the plurality of sensors. More specifically, the plurality of sensors may include a radar, a lidar, and a camera, and the computer device 100 detects information on surrounding vehicles using output values of a radar, a lidar, and a camera, and The fusion data can be generated by detecting lane information using. In this case, the fusion data may include at least one of a lateral speed of the vehicle and a distance to a lane of the vehicle.

In operation 1210, the computer device 100 may determine whether the object exists in the region of interest. At this time, the computer device 100 may perform step 1220 if the object exists in the region of interest, and if the object does not exist in the region of interest, it is not necessary to predict the path, and thus step 1110 again. To collect information. In this case, the computer apparatus 100 may select a vehicle existing in the region of interest as the vehicle of interest. For example, the computer apparatus 100 may select a vehicle in which the center is located in a plurality of predetermined areas around the host vehicle as the vehicle of interest. Here, the plurality of areas are a first area in the front left lane of the own vehicle, a second area in the front lane of the own vehicle, a third area in the front right lane of the own vehicle, a fourth area in the left lane of the own vehicle, A fifth area, a sixth area of the rear left lane of the host vehicle, a seventh area of the rear lane of the host vehicle, and an eighth area of the rear right lane of the host vehicle may be included. For example, the lengths of each of the fourth and fifth areas among the plurality of areas are determined based on the length of the host vehicle, and the lengths of the remaining areas are the speed of the host vehicle, the acceleration of the host vehicle, and a preset threshold. Can be determined by The width of each area may correspond to the width of the lane.

In operation 1220, the computer device 100 may determine whether the object changes lanes. As an example, in FIG. 4, the intention determination module 430 learned by the prior learning module 420 determines the intention of lane change of the object of interest (object existing in the region of interest) by the vehicle selection module 410 Examples have been described. In this case, the computer apparatus 100 may perform step 1130 when the object changes lanes, and may perform step 1140 when the object does not change lanes.

In step 1130, the computer device 100 may determine whether the object changes lanes to the own vehicle lane. In other words, when it is determined that there is an intention to change the object to the own vehicle lane, the computer device 100 may perform steps 1310 and 1330, and the intention to change the object to the own vehicle lane. If it is determined that is not present, since there is no need to pay attention to the path of the object, step 1110 may be performed again to collect information.

Steps 1310 and 1330 may be examples of a process in which the computer device 100 generates a predicted path according to a lane change of an object. In other words, the computer apparatus 100 may generate the predicted path only when the object of interest (an object existing in the region of interest) changes a lane to the lane of the own vehicle.

Meanwhile, in step 1140, the computer device 100 may determine whether the object is traveling in the own vehicle lane. At this time, the computer device 100 may perform steps 1320 and 1330 when the object is traveling in the own vehicle lane (in front of or behind the own vehicle), and the object If you are not driving in the front or rear) (for example, if you are changing lanes to a lane other than the own vehicle lane), you do not need to pay attention to the path of the object, so perform step 1110 again. To collect information.

Steps 1320 and 1330 may be examples of a process in which the computer device 100 generates a predicted path according to lane maintenance of an object. The process of generating a prediction path according to such lane maintenance has already been described with reference to FIGS. 9 and 10.

In order to determine the intention to change lanes, the computer device 100 uses a machine learning model pre-learned through data on actual vehicles to maintain the selected lane of the vehicle of interest, change the left lane, and change the right lane. You can determine your intentions. In this case, the machine learning model may be trained using a Gaussian kernel function.

Meanwhile, as already described, the computer device 100 may generate a lane change prediction path for a first vehicle that changes lanes to a lane of the own vehicle among the selected vehicles of interest in order to generate a prediction path. A lane maintenance prediction route for a second vehicle traveling in a lane of the own vehicle may be generated among the vehicles of interest. In order to generate a lane change prediction route, the computer apparatus 100 predicts a lane change time required based on a lateral distance and a lateral speed required for the lane change of the first vehicle, and determines the longitudinal speed and the first vehicle. 1 Predict the longitudinal distance between the first vehicle and the own vehicle based on the longitudinal acceleration of the vehicle, and use the lateral distance required for lane change of the first vehicle, the lane change time required, and a sinusoidal function. It is possible to predict the lateral distance between the and the own vehicle. In addition, the computer device 100 may generate a lane maintenance prediction path based on the distance to the lane of the second vehicle, the heading angle error of the second vehicle, the curvature of the road and/or the rate of change of the curvature of the road. I can.

The system or device described above may be implemented as a hardware component, a software component, or a combination of a hardware component and a software component. For example, the devices and components described in the embodiments are, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA). , A programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions, such as one or more general purpose computers or special purpose computers. The processing device may execute an operating system (OS) and one or more software applications executed on the operating system. In addition, the processing device may access, store, manipulate, process, and generate data in response to the execution of software. For the convenience of understanding, although it is sometimes described that one processing device is used, one of ordinary skill in the art, the processing device is a plurality of processing elements and/or a plurality of types of processing elements. It can be seen that it may include. For example, the processing device may include a plurality of processors or one processor and one controller. In addition, other processing configurations are possible, such as a parallel processor.

The software may include a computer program, code, instructions, or a combination of one or more of these, configuring the processing unit to behave as desired or processed independently or collectively. You can command the device. Software and/or data may be interpreted by a processing device or to provide instructions or data to a processing device, of any type of machine, component, physical device, virtual equipment, computer storage medium or device. Can be embodyed in The software may be distributed over networked computer systems and stored or executed in a distributed manner. Software and data may be stored on one or more computer-readable recording media.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, and the like alone or in combination. The medium may be one that continuously stores a program executable by a computer, or temporarily stores a program for execution or download. In addition, the medium may be a variety of recording means or storage means in a form in which a single piece of hardware or several pieces of hardware are combined, but is not limited to a medium directly connected to a computer system, and may be distributed on a network. Examples of media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magnetic-optical media such as floptical disks, and And a ROM, RAM, flash memory, and the like, and may be configured to store program instructions. In addition, examples of other media include an app store that distributes applications, a site that supplies or distributes various software, and a recording medium or storage medium managed by a server. Examples of the program instructions include not only machine language codes such as those produced by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like.

As described above, although the embodiments have been described by the limited embodiments and drawings, various modifications and variations are possible from the above description by those of ordinary skill in the art. For example, the described techniques are performed in a different order from the described method, and/or components such as a system, structure, device, circuit, etc. described are combined or combined in a form different from the described method, or other components Alternatively, even if substituted or substituted by an equivalent, an appropriate result can be achieved.

Therefore, other implementations, other embodiments and claims and equivalents fall within the scope of the following claims.

Claims (15)

In the path prediction method of a computer device including at least one processor,
Generating, by the at least one processor, fusion data using information from a plurality of sensors associated with the own vehicle and information received through external communication;
Selecting, by the at least one processor, a vehicle of interest using the generated fusion data;
Determining a lane change intention of the selected vehicle of interest, by the at least one processor; And
Generating, by the at least one processor, a predicted path of the selected vehicle of interest according to the determined lane change intention
Including,
The step of generating the fusion data,
Generating fusion data using the first information when current first information is received for the first vehicle through the external communication;
Generating fusion data using the second information when the second information previously received for the first vehicle is present without receiving the first information; And
When the first information is not received and the second information does not exist, generating fusion data using information from a plurality of sensors
Including,
Generating the fusion data using the information from the plurality of sensors,
Generating the fusion data using a weighted average of the output values of each of the plurality of sensors,
The weight used for the weighted average is determined by using an inverse number of an error covariance with respect to the output values of each of the plurality of sensors. delete delete The method of claim 1,
The plurality of sensors includes a radar, a lidar, and a camera,
Generating the fusion data using the information from the plurality of sensors,
And generating the fusion data by detecting surrounding vehicle information using the radar, the lidar, and output values of the camera, and detecting lane information using the camera. The method of claim 1,
The fusion data includes at least one of a lateral velocity of the vehicle and a lateral offset of the vehicle. The method of claim 1,
The step of selecting the vehicle of interest,
And selecting a vehicle whose center is located in a plurality of predetermined regions around the host vehicle as the vehicle of interest. The method of claim 6,
The plurality of areas are a first area in the front left lane of the own vehicle, a second area in the front lane of the own vehicle, a third area in the front right lane of the own vehicle, a fourth area in the left lane of the own vehicle, and a fifth in the right lane of the own vehicle. A route prediction method comprising: a region, a sixth region of a rear left lane of the subject vehicle, a seventh region of a rear lane of the subject vehicle, and an eighth region of a rear right lane of the subject vehicle. The method of claim 7,
The length of each of the fourth and fifth areas among the plurality of areas is determined based on the length of the host vehicle, and the length of each of the remaining areas is determined by the speed of the host vehicle, the acceleration of the host vehicle, and A route prediction method, characterized in that it is determined by a set threshold. The method of claim 1,
The step of determining the lane change intention,
And determining an intention for one of lane maintenance, left lane change, and right lane change of the selected vehicle of interest by using a machine learning model pre-learned through data on actual vehicles. The method of claim 9,
The machine learning model is a path prediction method, characterized in that the learning using a Gaussian kernel function (Gaussian kernel function). The method of claim 9,
Generating the prediction path,
Generating a lane change prediction route for a first vehicle that changes lanes to a lane of the own vehicle among the selected vehicles of interest; And
Generating a lane maintenance prediction route for a second vehicle traveling in a lane of the own vehicle among the selected vehicles of interest
Path prediction method comprising a. The method of claim 11,
Generating the lane change prediction path,
Predicting a lane change time required based on a lateral distance and a lateral speed required for lane change of the first vehicle;
Predicting a longitudinal distance between the first vehicle and the own vehicle based on a longitudinal speed of the first vehicle and a longitudinal acceleration of the first vehicle; And
Estimating a lateral distance between the first vehicle and the own vehicle using a lateral distance required for a lane change of the first vehicle, a time required for the lane change, and a sinusoidal function
Path prediction method comprising a. The method of claim 11,
The step of generating the lane maintenance prediction path,
The lane maintenance prediction path based on at least one of a distance to the lane of the second vehicle, a heading angle error of the second vehicle, a curvature of a road, and a curvature rate of the road A route prediction method, characterized in that to generate a. A computer program coupled with a computer device and stored in a computer-readable recording medium to execute a path prediction method on the computer device,
The route prediction method,
Generating fusion data using information received through external communication and information from a plurality of sensors associated with the own vehicle;
Selecting a vehicle of interest using the generated fusion data;
Determining a lane change intention of the selected vehicle of interest; And
Generating a predicted path of the selected vehicle of interest according to the determined lane change intention
Including,
The step of generating the fusion data,
When current first information is received for a first vehicle through the external communication, generating fusion data using the first information;
Generating fusion data using the second information when the second information previously received for the first vehicle is present without receiving the first information; And
When the first information is not received and the second information does not exist, generating fusion data using information from a plurality of sensors
Including,
Generating the fusion data using the information from the plurality of sensors,
Generating the fusion data using a weighted average of the output values of each of the plurality of sensors,
The weight used for the weighted average is determined using an inverse number of an error covariance for the output values of each of the plurality of sensors. In the computer device,
At least one processor embodied to execute instructions readable by the computer device
Including,
By the at least one processor,
Fusion data is generated by using information from a plurality of sensors linked to the host vehicle and information received through external communication,
Selecting a vehicle of interest using the generated fusion data,
Determine the lane change intention of the selected vehicle of interest,
Generate a predicted path of the selected vehicle of interest according to the determined lane change intention,
To generate the fusion data,
When current first information is received for the first vehicle through the external communication, fusion data is generated using the first information,
When the second information previously received for the first vehicle is present without receiving the first information, fusion data is generated using the second information,
When the first information is not received and the second information does not exist, fusion data is generated using information from a plurality of sensors,
To generate fusion data using information from the plurality of sensors,
Generating the fusion data using a weighted average of the output values of each of the plurality of sensors,
The weight used for the weighted average is determined by using the inverse of the error covariance for the output values of each of the plurality of sensors.
Computer device, characterized in that.

Images