JP2021021967A - Recognition system, vehicle control system, recognition method, and program - Google Patents

Abstract

To provide a recognition system, vehicle control system, recognition method, and program capable of further accurately specify a road surface.SOLUTION: The recognition system is loaded on a vehicle and includes a detection unit which detects object around the vehicle, and a specification unit which specifies a road surface around the vehicle on the basis of the detection result of the detection unit. The specification unit determines whether each of individual regions obtained by segmenting the detection result of the detection unit is flat or not by using a prescribed algorithm, and specifies the road surface around the vehicle by combining the determination result for each of the individual regions.SELECTED DRAWING: Figure 2

Description

The present invention relates to recognition systems, vehicle control systems, recognition methods, and programs.

Conventionally, a road shape recognition unit that recognizes a road shape, a travel route creation unit that creates a travel route using the recognized road shape, and a vehicle travel control device that realizes automatic travel according to the travel route are provided. The recognition unit extracts a coordinate information acquisition unit that acquires a plurality of coordinate information in which plane coordinates and height information are associated with each other, and a plurality of attention coordinates having a height difference of a predetermined value or more from the plurality of the coordinate information. The invention of an automatic traveling vehicle including a coordinate extracting unit for specifying a road shape and a shape specifying unit for specifying a road shape by statistically processing a plurality of extracted coordinates of interest is disclosed (Patent Document 1).

JP-A-2010-250743

In the conventional technique, a portion where the height difference is equal to or more than a predetermined value is recognized as, for example, a road shoulder or a groove, and a portion where the height difference is not is recognized as a road surface. However, in this method, there are cases where a part of the road surface is not recognized as a road surface due to the curvature of the road surface itself, or an obstacle having a small size is overlooked by increasing a predetermined value.

The present invention has been made in consideration of such circumstances, and one of the objects of the present invention is to provide a recognition system, a vehicle control system, a recognition method, and a program capable of more accurately identifying a road surface. To do.

The recognition system, vehicle control system, recognition method, and program according to the present invention have the following configurations.
(1): The recognition system according to one aspect of the present invention is a recognition system mounted on a vehicle, and is based on a detection unit that detects the position of an object existing in the vicinity of the vehicle and a detection result of the detection unit. Based on this, a specific unit that specifies the road surface around the vehicle is provided, and the specific unit uses a predetermined algorithm for each individual region that subdivides the detection result of the detection unit on a two-dimensional plane. It determines whether or not it is a flat surface, aggregates the determination results for each individual region, and specifies the road surface around the vehicle.

(2): In the aspect of (1) above, the detection unit is a rider.

(3): In the aspect of (2) above, the detection unit irradiates the periphery of the vehicle with a laser while changing the elevation angle or depression angle and the azimuth angle, and the specific unit is the elevation angle or depression angle and azimuth angle. , And the point cloud data in which the position of an object represented by at least a distance is projected onto a two-dimensional plane is subdivided into individual regions, and whether or not the data is a plane is determined by using the predetermined algorithm.

(4): In any of the above aspects (1) to (3), the specific portion makes the size of the individual region different based on the distance from the vehicle in the two-dimensional plane.

(5): In any of the above aspects (1) to (4), the specific unit acquires information indicating the distribution of objects in the vicinity of the vehicle, and is based on the information indicating the distribution of the acquired objects. Therefore, the size of the individual area is changed.

(6): In any of the above aspects (1) to (5), the specific unit acquires information on the type of road on which the vehicle exists, and the information on the acquired road type is the specific type. In the case of indicating the road of the above, the size of the individual area is increased as compared with the case where the information regarding the acquired road type does not indicate the road of a specific type.

(7): From the recognition system according to any one of (1) to (6) above and the detection result of the detection unit in the recognition system, the portion corresponding to the road surface specified by the specific unit is excluded. It is a vehicle control system including a travel control device that controls travel of the vehicle based on information.

(8): In the recognition method according to another aspect of the present invention, a computer mounted on the vehicle acquires a detection result of a detection unit that detects the position of an object existing in the vicinity of the vehicle, and is based on the detection result. Then, the road surface around the vehicle is specified, and when the specification is made, it is determined whether or not the detection result is a plane by using a predetermined algorithm for each individual region subdivided on the two-dimensional plane. , The determination result for each individual area is aggregated to specify the road surface around the vehicle.

(9): The program according to another aspect of the present invention causes a computer mounted on the vehicle to acquire a detection result of a detection unit that detects the position of an object existing in the vicinity of the vehicle, and based on the detection result. , The road surface around the vehicle is specified, and when the specification is made, the detection result is determined for each individual region subdivided on a two-dimensional plane by using a predetermined algorithm to determine whether or not the detection result is a plane. The determination results for each individual area are aggregated to specify the road surface around the vehicle.

According to the above aspects (1) to (9), the road surface can be specified more accurately.

It is a figure which shows the appearance that the recognition system and the vehicle control system are mounted on a vehicle. It is a block diagram of the object recognition device. It is a figure which shows an example of a point cloud data. It is a figure which shows the grid to be set. It is a figure which shows the point cloud data which removed the coordinates of the part determined as a road surface by the method of a comparative example. It is a figure which shows the point cloud data which removed the coordinates of the part determined as a road surface by the method of embodiment. It is a figure which shows the point cloud data which removed the coordinates of the part determined as a road surface by the method of embodiment. It is a flowchart which shows an example of the flow of processing executed by a recognition system.

Hereinafter, embodiments of the recognition system, vehicle control system, recognition method, and program of the present invention will be described with reference to the drawings.

FIG. 1 is a diagram showing a state in which a recognition system and a vehicle control system are mounted on a vehicle M. The vehicle M is equipped with, for example, a lidar (Light Detection and Ranging: LIDAR) 10 (an example of a "detection unit"), a camera 20, a radar device 30, an object recognition device 50, and a travel control device 100. To. The combination of the rider 10 and the object recognition device 50 is an example of the "recognition system", and the addition of the travel control device 100 is an example of the "vehicle control system". A detection device other than the rider may be used as the detection unit.

The rider 10 irradiates light to detect reflected light, and detects the distance to an object by measuring the time from irradiation to detection. The rider 10 can change the light irradiation direction for both the elevation angle or the depression angle (hereinafter, the irradiation direction φ in the vertical direction) and the azimuth angle (the irradiation direction θ in the horizontal direction). For example, the rider 10 scans while fixing the irradiation direction φ and changing the irradiation direction θ, then changing the irradiation direction φ in the vertical direction, fixing the irradiation direction φ at the changed angle, and changing the irradiation direction θ. While scanning, the operation is repeated. Hereinafter, the irradiation direction φ is referred to as a “layer”, one scan performed while fixing the layer and changing the irradiation direction θ is referred to as a “cycle”, and scanning for all layers is referred to as “1”. Called "scan". The layers are set with a finite number from L1 to Ln, for example (n is a natural number). The layer change is performed discontinuously with respect to the angle, for example, L0 → L4 → L2 → L5 → L1 ... So that the light emitted in the previous cycle does not interfere with the detection in this cycle. Not limited to this, the layer may be changed continuously with respect to the angle.

For example, the rider 10 outputs a data set (rider data) having {φ, θ, d, p} as one unit to the object recognition device 50. d is the distance and p is the intensity of the reflected light. The object recognition device 50 is installed at an arbitrary position in the vehicle M. In FIG. 1, the rider 10 is installed on the roof of the vehicle M, and the irradiation direction θ can be changed by 360 degrees. However, this arrangement is only an example, and is provided, for example, in the front part of the vehicle M. The rider whose irradiation direction θ can be changed by 180 degrees around the front of the vehicle M and the rider who is provided at the rear of the vehicle M and can change the irradiation direction θ by 180 degrees around the rear of the vehicle M are the vehicle M. It may be mounted on.

The camera 20 is installed at an arbitrary position capable of photographing the periphery (particularly forward or rear) of the vehicle M. For example, the camera 20 is installed above the front windshield. The camera 20 is a digital camera including an image sensor such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor), and repeatedly images the periphery of the vehicle M at a predetermined cycle.

The radar device 30 radiates radio waves such as millimeter waves around the vehicle M, and detects radio waves (reflected waves) reflected by the object to detect at least the position (distance and orientation) of the object. The radar device 30 is attached to an arbitrary position on the vehicle M. For example, the radar device 30 is mounted inside the front grill of the vehicle M.

FIG. 2 is a configuration diagram of the object recognition device 50. The object recognition device 50 includes, for example, a rider data processing unit 60, a camera image processing unit 70, a radar data processing unit 80, and a sensor fusion unit 90. The rider data processing unit 60 includes, for example, a point cloud data generation unit 61, an information acquisition unit 62, a road surface identification unit 63 (an example of a “specification unit”), a non-road surface object extraction unit 64, and a road marking line. It includes a recognition unit 65. The road surface specifying unit 63 includes, for example, a grid setting unit 63A and a plane extraction processing unit 63B. These components are realized by, for example, a hardware processor such as a CPU (Central Processing Unit) executing a program (software). Some or all of these components are hardware (circuit parts;) such as LSI (Large Scale Integration), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), GPU (Graphics Processing Unit). It may be realized by (including circuits), or it may be realized by the cooperation of software and hardware. The program may be stored in advance in a storage device (a storage device including a non-transient storage medium) such as an HDD (Hard Disk Drive) or a flash memory, or a removable storage device such as a DVD or a CD-ROM. It is stored in a medium (non-transient storage medium) and may be installed by mounting the storage medium in a drive device.

The point cloud data generation unit 61 generates point cloud data based on the rider data. The point cloud data in the present embodiment is a projection of the position of the object recognized from the rider data in the three-dimensional space onto the position on the two-dimensional plane seen from the sky. FIG. 3 is a diagram showing an example of point cloud data. The two-dimensional plane that defines the point cloud data (the two-dimensional plane represented by the X-axis and the Y-axis in the figure) is, for example, a relative two-dimensional plane seen from the rider 10. Although not shown in the figure, height information (displacement in the direction orthogonal to the X-axis and the Y-axis) is given to each coordinate of the point cloud data. The height information is calculated by the point cloud data generation unit 61 based on the continuity and scattering between the coordinates and the difference in the irradiation angle between the layers.

The information acquisition unit 62 acquires various information used when the grid setting unit 63A sets the grid.

For example, the information acquisition unit 62 acquires information indicating the distribution of objects around the vehicle M. The distribution of objects around the vehicle M is, for example, a part or all of the number of vehicles, the number of pedestrians, the number of bicycles, the number of traffic lights and pedestrian crossings, the number of intersections, etc. within the recognizable range of the recognition system. Obtain the indexed value (congestion index). For example, the congestion index shows a higher value as the above-mentioned elements are present in a higher density. The information acquisition unit 62 may calculate the congestion index by itself, or may acquire it from the camera image processing unit 70, the radar data processing unit 80, or the like.

In addition, the information acquisition unit 62 may acquire information regarding the type of road on which the vehicle M exists. The information acquisition unit 62 may acquire information on the type of road from a navigation device (not shown) mounted on the vehicle M, or derives information from the result of the camera image processing unit 70 recognizing a road sign from the camera image. You may.

The grid setting unit 63A of the road surface specifying unit 63 virtually sets a plurality of grids which are individual areas divided into two-dimensional planes that define point cloud data. FIG. 4 is a diagram showing the grid G to be set. The grid setting unit 63A sets, for example, the grid G as a rectangle (which may be square or rectangular). The grid setting unit 63A may set the grid G of the same size, or may make the size of the grid G different based on the distance from the vehicle M in the two-dimensional plane. For example, as shown in the figure, the grid setting unit 63A may increase the size of the grid G as the distance from the vehicle M increases. The grid setting unit 63A does not have to set the grid G for the recognition unnecessary area (for example, the other side of the guardrail, the area outside the road such as a building) obtained by referring to the past recognition result of the recognition system. The grid setting unit 63A may set the grid G with any polygon such as a triangle or a hexagon (which may include two or more types of polygons), or set the grid G with an indeterminate form. You may.

Further, the grid setting unit 63A may determine the size of the grid G based on the congestion index. For example, the grid setting unit 63A may reduce the size of the grid G as the congestion index increases.

Further, the grid setting unit 63A may determine the size of the grid G based on the type of road. For example, when the type of road is a specific type such as an expressway or a motorway where there are few traffic participants other than vehicles, the grid setting unit 63A determines the size of the grid G as compared with the case where it is not a specific type. It may be increased.

The plane extraction processing unit 63B performs plane extraction processing for each grid G on the point cloud data included in the grid G by a robust estimation method such as RANSAC (Random Sample Consensus), and the grid G is used as a road surface (Random Sample Consensus). It is determined whether or not there is an object on the top), and the determination result is associated with each of the grids G. The plane extraction processing unit 63B may perform another type of plane extraction processing instead of RANSAC.

RANSAC is performed, for example, by the following procedure. First, randomly select more (but not all) samples from the data set than necessary to determine the model, derive a tentative model from the selected samples by the least squares method, etc., and apply the tentative model to the data. , If there are not so many outliers, add them to the model candidates. This process is repeated several times, and the model candidate that best matches the entire data set is set as the correct model. In the present embodiment, the plane extraction processing unit 63B determines that the grid G whose correct model is a plane is a road surface.

The non-road surface object extraction unit 64 analyzes point cloud data for the grid G other than the grid G determined to be the road surface by the plane extraction processing unit 63B, and extracts the contour of the object existing on the grid G. Based on the contour, the position of the object corresponding to the contour is recognized. Alternatively, the non-road surface object extraction unit 64 extracts the contour of the object from the rider data based on the data corresponding to the grid G other than the grid G determined to be the road surface by the plane extraction processing unit 63B, and the contour is extracted. The position of the object corresponding to the contour may be recognized based on.

The road marking line recognition unit 65 pays attention to the intensity p of the reflected light in the rider data, and determines the portion where the rate of change of the intensity p is high due to the difference in color between the road surface and the road marking line such as the white line and the yellow line. Recognized as the outline of the road lane marking. As a result, the road lane marking unit 65 recognizes the position of the road lane marking such as the white line.

The processing results of the non-road surface object extraction unit 64 and the road lane marking unit 65 are output to the sensor fusion unit 90. The processing results of the camera image processing unit 70 and the radar data processing unit 80 are also input to the sensor fusion unit 90.

The camera image processing unit 70 performs various image processing on the camera image acquired from the camera 20, and recognizes the position, size, type, and the like of an object existing around the vehicle M. The image processing performed by the camera image processing unit 70 includes a process of inputting a camera image into a trained model obtained by machine learning and a process of extracting edge points and recognizing an object from a contour line in which the edge points are connected. It may be.

The radar data processing unit 80 performs various object extraction processes on the radar data acquired from the radar device 30, and recognizes the position, size, type, and the like of the objects existing around the vehicle M. The radar data processing unit 80 estimates the material of the object based on the intensity of the reflected wave from the object, for example, and thereby estimates the type of the object.

The sensor fusion unit 90 integrates the processing results input from each of the rider data processing unit 60, the camera image processing unit 70, and the radar data processing unit 80, determines the positions of objects and road marking lines, and is a travel control device. Output to 100. The process of the sensor fusion unit 90 may include, for example, a process of obtaining a logical sum, a logical product, a weighted sum, or the like for each process result.

By performing the processing as described above, the recognition system can identify the road surface more accurately. FIG. 5 is a diagram showing point cloud data in which the coordinates of the portion determined to be the road surface are removed by the method of the comparative example. The method of the comparative example is a method of applying RANSAC to the entire rider data (without dividing it into the grid G) and removing the coordinates of the region determined to be the road surface. As shown in the figure, in the method of the comparative example, many coordinates remain even in the regions A1 and A2 corresponding to the road surface. In particular, the area A1 is an uphill when viewed from the vehicle M, and it is difficult to recognize it as a road surface when RANSAC is applied to the entire data. In addition, in a general road structure, the central part of the road is high and becomes low toward the end, so in the method of the comparative example, it is judged that the road surface is not due to the height difference between the central part and the end of the road. There is a possibility that it will end up. Further, since the road has small recesses and the like, there is a possibility that the road surface may be partially recognized as not the road surface.

On the other hand, FIGS. 6 and 7 are diagrams showing point cloud data in which the coordinates of the portion determined to be the road surface are removed by the method of the embodiment. FIG. 6 shows the result when one side of the square grid G is X1, and FIG. 7 shows the result when one piece of the square grid G is X2 (X1> X2). .. As shown in these figures, according to the method of the embodiment, many coordinates are removed in the regions A1 and A2 corresponding to the road surface, whereby an object that does not actually exist is recognized as an obstacle. The possibility of getting rid of it is reduced. It should be noted that reducing one side of the grid G (that is, reducing the size of the grid G) can improve the accuracy of determining the road surface, but reducing the size of the grid G increases the processing load. Therefore, these are in a trade-off relationship. In this regard, by increasing the size of the grid G as it is farther from the vehicle M, it is possible to perform processing with a low load for a distant place where the influence is small even if there is a false recognition, and the recognition accuracy and the processing load are well balanced. Processing can be performed. In addition, by reducing the size of the grid G as the congestion index is higher, processing that prioritizes recognition accuracy is performed in places with many traffic participants such as urban areas, and processing that prioritizes reduction of processing load is performed in other places. Therefore, it is possible to perform processing with a good balance between recognition accuracy and processing load. In addition, when the road type is a specific type, by increasing the size of the grid G compared to the case where it is not a specific type, processing that prioritizes reduction of processing load is performed in places where there are few traffic participants. Since the processing is performed in a place where the recognition accuracy is prioritized, the processing with a good balance between the recognition accuracy and the processing load can be performed.

The travel control device 100 is, for example, an automatic driving control device that controls both acceleration / deceleration and steering of the vehicle M. Based on the position of the object, white line, etc. output by the object recognition device 50, the travel control device 100 automatically travels the vehicle M so as not to come into contact with the object in the set lane, or if necessary. Automatic control of lane change, overtaking, branching, merging, stopping, etc. Instead of this, the travel control device 100 may be a driving support device or the like that automatically stops when an object approaches. In this way, the travel control device 100 recognizes the position of the object (on the specified road surface) recognized by the non-road surface object extraction unit 64 for the grid G other than the grid G determined to be the road surface by the plane extraction processing unit 63B. The travel control of the vehicle M is performed based on the information output from the sensor fusion unit 90 (information excluding the corresponding portion).

FIG. 8 is a flowchart showing an example of the flow of processing executed by the recognition system. The rider 10 detects an object and repeatedly outputs the rider data to the rider data processing unit 60 (step S100).

The rider data processing unit 60 waits until the rider data for one scan is acquired (step S102). When the rider data for one scan is acquired, the process proceeds to step S106.

On the other hand, the information acquisition unit 62 operates asynchronously with, for example, the components other than the information acquisition unit 62 of the rider data processing unit 60, acquires information for setting the grid G, and becomes the road surface identification unit 63. Provided (step S104).

The point cloud data generation unit 61 generates point cloud data from the rider data (step S106). The grid setting unit 63A sets the grid G based on the information provided by the information acquisition unit 62 (step S108).

The plane extraction processing unit 63B specifies whether or not it is a road surface for each grid G (step S110). The non-road surface object extraction unit 64 recognizes an object with respect to the grid G of the non-road surface that is not specified as the road surface (step S112). Then, the rider data processing unit 60 outputs the processing results of the non-road surface object extraction unit 64 and the road marking line recognition unit 65 to the sensor fusion unit 90 (step S114). This ends the routine of the flowchart of FIG.

According to the recognition system of the embodiment described above, the detection unit (rider 10) that detects the position of an object existing around the vehicle (M) and the road surface around the vehicle based on the detection result of the detection unit. A specific unit (road surface identification unit 63) is provided, and the specific unit uses a predetermined algorithm (RANSAC) for each individual region (grid G) in which the detection result of the detection unit is subdivided on a two-dimensional plane. Since it is determined whether or not it is a flat surface by using it, and the determination results for each individual area are aggregated to specify the road surface around the vehicle, the road surface can be specified more accurately.

The recognition system may not include a part or all of the camera 20, the radar device 30, the camera image processing unit 70, the radar data processing unit 80, and the sensor fusion unit 90. For example, the recognition system may include the rider 10 and the rider data processing unit 60, and output the processing results of the non-road surface object extraction unit 64 and the road lane marking unit 65 to the travel control device 100.

Although the embodiments for carrying out the present invention have been described above using the embodiments, the present invention is not limited to these embodiments, and various modifications and substitutions are made without departing from the gist of the present invention. Can be added.

10 Rider 50 Object recognition device 60 Rider data processing unit 61 Point cloud data generation unit 62 Information acquisition unit 63 Road surface identification unit 63A Grid setting unit 63B Plane extraction processing unit 64 Non-road surface object extraction unit 65 Road marking line recognition unit 100 Traveling Control device

Claims (9)

A recognition system installed in a vehicle
A detection unit that detects the position of an object existing around the vehicle,
A specific unit that identifies the road surface around the vehicle based on the detection result of the detection unit is provided.
The specific unit determines whether or not the detection result of the detection unit is a plane by using a predetermined algorithm for each individual region subdivided on a two-dimensional plane, and aggregates the determination results for each individual region. To identify the road surface around the vehicle,
Recognition system. The detector is a rider.
The recognition system according to claim 1. The detection unit irradiates the periphery of the vehicle with a laser while changing the elevation angle or the depression angle and the azimuth angle.
The specific part is a plane using the predetermined algorithm for each individual region obtained by subdividing point cloud data in which the position of an object represented by at least an elevation angle or a depression angle, an azimuth angle, and a distance is projected onto a two-dimensional plane. Determine if there is,
The recognition system according to claim 2. The specific portion makes the size of the individual region different based on the distance from the vehicle in the two-dimensional plane.
The recognition system according to any one of claims 1 to 3. The specific unit acquires information indicating the distribution of objects in the vicinity of the vehicle, and changes the size of the individual region based on the information indicating the distribution of the acquired objects.
The recognition system according to any one of claims 1 to 4. The specific unit acquires information on the type of road on which the vehicle exists, and when the information on the acquired road type indicates a road of a specific type, the information on the acquired road type is of a specific type. Increase the size of the individual area as compared to the case where the road is not shown.
The recognition system according to any one of claims 1 to 5. The recognition system according to any one of claims 1 to 6 and
A travel control device that controls the travel of the vehicle based on information excluding the portion corresponding to the road surface specified by the specific unit from the detection result of the detection unit in the recognition system.
Vehicle control system with. The computer installed in the vehicle
Acquire the detection result of the detection unit that detects the position of the object existing around the vehicle,
Based on the detection result, the road surface around the vehicle is identified.
At the time of the above identification
The detection result is subdivided on a two-dimensional plane, and it is determined whether or not the detection result is a plane by using a predetermined algorithm for each individual region.
The judgment results for each individual area are aggregated to identify the road surface around the vehicle.
Recognition method. On the computer installed in the vehicle
Acquire the detection result of the detection unit that detects the position of an object existing around the vehicle.
Based on the detection result, the road surface around the vehicle is identified.
When making the above identification
The detection result is subdivided on a two-dimensional plane, and it is determined whether or not the detection result is a plane by using a predetermined algorithm for each individual region.
The determination results for each individual area are aggregated to identify the road surface around the vehicle.
program.