JP6082293B2 - Vehicle white line recognition device - Google Patents

Description

  The present invention relates to a vehicle white line recognition device that recognizes a white line on a road based on an image captured by an in-vehicle camera.

  In recent years, in order to improve the safety of vehicles, driving support devices that actively support driving operations of drivers have been developed. This driving support device realizes a driving support function such as lane departure prevention by recognizing a white line on the road based on a captured image etc. in front of the host vehicle and estimating a driving lane based on the recognized white line. ing.

  Such white line detection by image recognition, as disclosed in Patent Document 1, detects edge points whose luminance changes from the road surface image, and white line candidates from the point group of these edge points by Hough transform or the like. It is common to obtain a line and use a white line that divides the vehicle lane.

JP 2009-217492 A

  In order for a white line candidate line obtained from an image to be properly recognized as a white line, as disclosed in Patent Document 1, the threshold value determined from the brightness and the contrast with the road surface and the threshold value of the white line width must be satisfied. is necessary.

  However, in a situation where a normal white line is lost due to construction, etc., and a temporary white line thinner than normal is provisionally drawn over a predetermined section, when the normal white line width threshold is uniquely applied, A point group forming a thin temporary white line is erroneously determined to be noise or the like, and the state in which the white line is not recognized continues to hinder vehicle control based on the white line recognition result.

  The present invention has been made in view of the above circumstances, and an object thereof is to provide a vehicular white line recognition device capable of properly recognizing a white line even in a driving environment in which a white line thinner than usual appears. .

A white line recognition device for a vehicle according to the present invention uses a white line as a white line start point and a white line end point based on luminance changes on a plurality of search lines set in a horizontal direction on an image obtained by capturing an image of the traveling environment of the vehicle. A white line candidate point detection unit for detecting as a candidate point, a white line candidate point width calculation unit for calculating a horizontal position difference between the white line start point and the white line end point as a white line candidate point width, and the white line candidate point Of the detected point group, the ratio of points smaller than the current value of the white line width threshold that is the threshold for the white line candidate point width is calculated, and the white line width threshold is dynamically variably set according to the calculated ratio. A white line width threshold setting unit, a white line point group selection unit that compares the white line candidate point width with the white line width threshold and selects a white line point group from the white line candidate points, and a white line based on the selected white line point group And a white line calculation unit.

  According to the present invention, it is possible to properly recognize a white line even in a driving environment in which a white line that is thinner than usual appears, and to avoid an influence on vehicle control based on the result of white line recognition.

Schematic configuration diagram of a vehicle driving support device Explanatory diagram showing changes in white line width Flow chart showing white line recognition routine Explanatory drawing which shows transition of the luminance and the differential value of luminance at the white line start point and the white line end point Explanatory diagram showing how to calculate the Hough transform Illustration of Hough plane Flowchart showing white line width threshold setting routine

Embodiments of the present invention will be described below with reference to the drawings.
In FIG. 1, reference numeral 1 denotes a vehicle such as an automobile (own vehicle). The vehicle 1 recognizes an external environment by processing an image captured by an in-vehicle camera, and performs various operations for a driver based on the recognition result. A driving support system 2 that performs support is installed. The driving support system 2 includes an in-vehicle camera 3, an image processing engine 4, a control unit 5 and the like as main parts.

  In the present embodiment, the in-vehicle camera 3 is a stereo camera that captures an image of an object from different viewpoints, and includes, for example, a plurality of cameras having image sensors such as a CCD and a CMOS. Below, the vehicle-mounted camera 3 is described as a stereo camera 3. As the stereo camera 3, two left and right cameras synchronized with each other with variable shutter speed are used, and are mechanically fixed at a predetermined base length (optical axis interval) so that their optical axes are substantially parallel to each other. For example, it is unitized with the image processing engine 4. The stereo camera 3 is attached, for example, in front of the ceiling in the passenger compartment, and takes a stereo image of an object outside the vehicle from different viewpoints, and outputs image data to the image processing engine 4.

  In the following description, one image (for example, the right image) of the pair of original images captured in stereo is referred to as a reference image, and the other image (for example, the left image) is referred to as a comparison image.

  First, the image processing engine 4 divides the reference image into small areas of, for example, 4 × 4 pixels, compares the luminance or color pattern of each small area with the comparison image, searches for a corresponding area, and searches for each area. A pixel shift amount (parallax) between the left and right images is obtained, and a distribution image (distance image) including the pixel shift amount as distance information is generated. The point on the distance image is a point in the real space based on the principle of triangulation, where the vehicle width direction of the own vehicle, that is, the left-right direction is the X axis, the vehicle height direction is the Y axis, and the vehicle length direction, that is, the distance direction is the Z axis The coordinates are converted to. The image processing engine 4 recognizes white lines, side walls, three-dimensional objects, etc. of the road ahead of the host vehicle based on the captured image and its distance information, assigns different IDs to the recognized data, and assigns these to each ID. Monitoring is continuously performed between frames and transmitted to the control unit 5 as traveling environment information ahead of the host vehicle 1.

  In addition to the travel environment information from the image processing engine 4, signals from various sensors that detect the state of the host vehicle 1 are input to the control unit 5. Sensors provided in the host vehicle 1 include a vehicle speed sensor 11 that detects the host vehicle speed, a yaw rate sensor 12 that detects the yaw rate, a main switch 13 that performs ON / OFF switching of each function of the driving support control, and the steering wheel. A steering angle sensor 14 for detecting the steering angle provided on the steering shaft, an accelerator opening sensor 15 for detecting an accelerator pedal depression amount (accelerator opening) by a driver, and the like. Input to the control unit 5.

For example, when an instruction to execute an ACC (Adaptive Cruise Control) function, which is one of the functions of the driving support control, is issued through the operation of the main switch 13 by the driver, the control unit 5 recognizes the image processing engine 4. The preceding vehicle direction is read, and it is identified whether or not the preceding vehicle to be followed is traveling on the own vehicle traveling path. As a result, when a preceding vehicle to be tracked is not detected, constant speed running control is performed to maintain the vehicle speed of the host vehicle 1 at the set vehicle speed set by the driver through opening / closing control (engine output control) of the throttle valve 16. .

  On the other hand, when a preceding vehicle that is a tracking target vehicle is detected and the vehicle speed of the preceding vehicle is equal to or lower than the set vehicle speed, the following traveling control is performed in which the following distance is converged to the target inter-vehicle distance. Executed. During this follow-up running control, the control unit 5 basically converges the inter-vehicle distance to the target inter-vehicle distance through the opening / closing control of the throttle valve 16 (engine output control). Furthermore, when it is determined that sufficient deceleration cannot be obtained only by controlling the throttle valve 16 due to sudden deceleration of the preceding vehicle, the control unit 5 controls the output hydraulic pressure from the active booster 17 (automatic braking intervention). Control) to converge the inter-vehicle distance to the target inter-vehicle distance.

  Further, when execution of the lane departure prevention function, which is one of the functions of the driving support control, is instructed through the operation of the main switch 13 by the driver, the control unit 5, for example, the left and right white lines that define the own vehicle traveling lane Is set based on the vehicle speed and the vehicle traveling path is estimated based on the vehicle speed and yaw rate of the vehicle 1. Then, for example, when the control unit 5 determines that the host vehicle travel route crosses either the left or right alarm determination line within a set distance (for example, 10 to 16 [m]) ahead of the host vehicle, It is determined that the host vehicle 1 is likely to depart from the current host vehicle lane, and a lane departure warning is issued.

  Such driving support control by the control unit 5 is based on the recognition result of the white line by the image processing engine 4 as the white line recognition device. The white line in the present embodiment is a general term for lines that extend on the road and divide the vehicle lane, and the form of each line is not limited to a solid line, a broken line, etc. Is also included. Further, in the white line recognition in the present embodiment, even if the white line actually existing on the road is a double white line or the like, it is recognized by approximating it with a single straight line or a curved line on the left and right respectively.

  When recognizing a white line, the image processing engine 4 detects an edge whose luminance changes by a predetermined value or more on a plurality of search lines set in the horizontal direction (vehicle width direction) within the white line detection region set on the image. Thus, one set of white line start point Ps and white line end point Pe is detected for each search line. Specifically, for example, the image processing engine 4 changes the luminance value of each pixel from the inner side to the outer side in the vehicle width direction on each search line in each of the left and right white line detection areas set on the reference image. The edge point is detected by checking the above, and a set of white line start point Ps and white line end point Pe for each search line is detected as a point group of white line candidates based on the detected edge point. Then, the image processing engine 4 calculates a straight line component from the point group of white line candidates, and outputs it to the control unit 5 as left and right white lines that define the vehicle lane.

  At this time, the image processing engine 4 uses the threshold for the width of the white line start point Ps and the white line end point Pe to evaluate the validity of the white line candidate point as a recognition target. ) Is dynamically changed in response to the change in the arrangement state of the white line candidate points. As a result, even in a driving environment in which a temporary white line that is thinner than a normal white line is temporarily drawn due to road construction, etc., it becomes possible to properly recognize without excluding the thin white line, and the white line recognition result It is possible to continue driving support for the driver without interrupting the control based on.

  For this reason, the image processing engine 4 includes a white line candidate point detection unit, a white line candidate point width calculation unit, a white line width threshold setting unit, a white line point group selection unit, and a white line calculation unit. The image processing engine 4 having such a function unit includes a point group of white line start points and white line end points based on luminance changes on a plurality of search lines set in a horizontal direction on an image obtained by imaging the host vehicle traveling environment. Are detected as white line candidate points (white line candidate point detection unit), and a horizontal position difference between the white line start point and the white line end point is calculated as a white line candidate point width (white line candidate point width calculation unit).

  Further, the image processing engine 4 dynamically variably sets a threshold (white line width threshold) for the white line candidate point width according to the arrangement state of the point group of the white line candidate points (white line width threshold setting unit). That is, as shown in FIG. 2, the normal white line WLnormal on the road is lost due to construction or the like, and the temporary white line WLtemp thinner than normal is provisionally drawn after the section indicated by the broken line in the figure. In such a situation, the white line candidate points are continuously arranged over a predetermined section with a width narrower than the current white line width threshold, and only by setting the white line width threshold uniquely as in the prior art, It is erroneously determined as noise or the like. Therefore, in such a situation, the ratio of white line candidate points whose white line candidate point width is smaller than the current white line width threshold value is evaluated, and the white line width threshold value is changed to a value smaller than the current value according to the evaluation result. Etc. to continue white line recognition and avoid stopping the driving support function.

  In this case, the white line width threshold value may be changed by switching between a plurality of different preset threshold values, or the current threshold value may be increased or decreased according to the ratio of white line candidate points smaller than the white line width threshold value. good. Then, the image processing engine 4 compares the white line candidate point width with the white line width threshold value to select a point group constituting the white line from the white line candidate point, that is, a white line point group (white line point group selecting unit), and based on the white line point group The white line is calculated (white line calculation unit).

Hereinafter, processing related to white line recognition of the image processing engine 4 will be described with reference to the flowcharts of FIGS. 3 and 7.
The flowchart of FIG. 3 shows a white line recognition routine that is a main process of white line recognition. When this routine starts, the image processing engine 4 uses the function of the white line candidate point detection unit to set the white line start point Ps in the first step S101. In step S102, the white line end point Pe is detected. That is, for example, the image processing engine 4 determines the vehicle width based on the image center line (or the vehicle traveling direction estimated from the steering angle, etc.) in the left and right white line detection areas set on the reference image. Edge detection is performed on each search line from the inside to the outside in the direction, and the edge point indicating the white line start point Ps is searched.

  Specifically, for example, as shown in FIG. 4, the image processing engine 4 is configured such that, in a search from the inside to the outside in the vehicle width direction, the luminance of the pixels outside the vehicle width direction is relative to the luminance of the inner pixels. And a point (edge point) at which the differential value of the luminance indicating the amount of change is equal to or greater than the set threshold value on the plus side is detected as the white line start point Ps. Here, in order to simplify the calculation, the image processing engine 4 performs an edge point (i.e., first detected based on the above-described requirements) in the search from the vehicle width direction inner side to the vehicle width direction outer side on each search line. Only the edge point detected at the innermost side in the vehicle width direction based on the above requirements is detected as the white line start point Ps.

  In the detection of the white line end point Pe, the image processing engine 4, for example, in the left and right white line detection areas set on the reference image, the own vehicle travel estimated from the image center line (or the steering angle or the like). (Direction) as a reference, edge detection is performed on each search line from the inner side to the outer side in the vehicle width direction, and an edge point indicating the white line end point Pe is searched.

  Specifically, for example, as shown in FIG. 4, in the search from the inner side to the outer side in the vehicle width direction, the luminance of the pixels on the outer side in the vehicle width direction is relatively lower than the luminance of the inner pixels, and A point (edge point) at which the differential value of luminance indicating the amount of change is equal to or less than the negative threshold is detected as the white line end point Pe. Here, in order to simplify the calculation, the image processing engine 4 performs an edge point (i.e., first detected based on the above-described requirements) in the search from the vehicle width direction inner side to the vehicle width direction outer side on each search line. Only the edge point detected at the innermost side in the vehicle width direction based on the above requirements is detected as the white line end point Pe.

  In the present embodiment, the corresponding distance information on the distance image is assigned to each white line start point Ps and each white line end point Pe detected in the above-described step S101 and step S102.

  In step S103, the image processing engine 4 sets the white line width threshold WD by executing the white line width threshold setting routine of FIG. 7 that realizes the functions of the white line candidate point width calculation unit and the white line width threshold setting unit. To do. Although this white line width threshold value setting routine will be described later, in the present embodiment, the white line width threshold value WD is appropriately switched between three preset threshold values WD0, WD1, WD2 (WD0> WD1> WD2). Yes.

  After that, in step S104, the image processing engine 4 compares the white line candidate point width with the white line width threshold by the function as the white line point group selection unit, and calculates a white line from which candidate points are equal to or larger than the white line width threshold. Select a point cloud. For example, the white line point group may be selected from the point group of the white line start point Ps, or an intermediate point between the white line start point Ps and the white line end point Pe is calculated and selected from the point group of the intermediate point. Also good. In this case, it is desirable to add a variation condition in addition to the white line width condition, and to select a point group having a white line candidate point width equal to or larger than the threshold and a small variation as the white line point group.

  In subsequent step S <b> 105, the image processing engine 4 performs a process of calculating a white line from the white line point group by a function as a white line calculation unit. In the present embodiment, the image processing engine 4 executes the white line calculation process as a process of acquiring a straight line component from the white line point group by Hough transform. In the following, an example in which a linear component is obtained by Hough transform will be described. However, a linear component may be obtained by using the least square method.

Specifically, the points constituting the white line point group will be abbreviated as P. For example, as shown in FIG. 5, the image processing engine 4 performs a point P (x for each point P constituting the point group. , Z) by changing the slope θ of the straight line Lh from 0 ° to 180 ° by a predetermined angle Δθ, and based on the following equation (1), the distance from the origin O to the straight line Lh at each θ (the length of the perpendicular line) ) Find ρ.
ρ = x · cos θ + z · sin θ (1)

  Then, the image processing engine 4 votes (projects) the relationship between each θ and ρ obtained for each point P as a frequency at a corresponding location on the Hough plane (θ, ρ) shown in FIG. 6, for example. Further, the image processing engine 4 extracts a combination of θ and ρ having the largest frequency on the Hough plane (θ, ρ), and uses the θ and ρ, the Hough straight line Lh ( H) is calculated as a straight line component that approximates the point group, and this straight line component is output as a white line that divides the vehicle lane.

Next, the white line width threshold setting routine of FIG. 7 will be described.
In the first step S201, the image processing engine 4 calculates the difference between the horizontal position coordinate of the white line start point Ps and the horizontal position coordinate of the white line end point Pe as a white line candidate point width W, and then In step S202, the white line candidate point width W is compared with the current white line width threshold WD to check whether the white line candidate point width W is equal to or smaller than the white line width threshold WD. The white line width threshold WD is initially set to an initial value WD0 (for example, WD0 = 10 cm) assuming a general white line width.

  As a result, if W> WD in step S202, the process proceeds from step S202 to step S203, the white line width threshold value WD is held at the initial value WD0, and the process exits. If W ≦ WD, step S204 is executed. Of the point groups detected as white line candidate points, a ratio R of the number of candidate points less than the white line width threshold WD is calculated, and it is checked whether the ratio R is equal to or greater than a set value H. Although the setting value H is smaller than the current white line width threshold WD, the setting value H is a ratio at which candidate points constituting the white line can be considered to be continuously arranged, and is set to about H = 70%, for example.

  When R <H in step S204 and the ratio R of the number of candidate points less than the white line width threshold WD is smaller than the set value H, the process proceeds from step S204 to step S205, and the current white line width threshold WD is set to the first. The threshold WD1 is set and the routine is exited. The first threshold WD1 is a threshold for setting a white line slightly thinner than a general white line width as a recognition target, and is set to WD1 = 8 cm, for example.

  On the other hand, if R ≧ H in step S204 and the ratio R of the number of candidate points less than the white line width threshold WD is larger than the set H, the process proceeds from step S204 to step S206, where the current white line width threshold WD is A threshold value WD2 of 2 is set and the routine is exited. The second threshold WD2 is a threshold for recognizing a thin temporary white line drawn during construction or the like, and is set to WD2 = 5 cm, for example.

  A plurality of values may be set as the setting value H for switching the white line width threshold value WD, and the threshold value set for the white line width threshold value WD may be made finer than the three levels of WD0, WD1, and WD2.

  Then, after the white line width threshold value WD is set to the first threshold value WD1 or the second threshold value WD2, if the ratio R of white line candidate points where W <WD is less than the set value H, the white line width threshold value WD is If the current white line width threshold WD is the first threshold WD1 when the ratio R is equal to or greater than the set value H when the ratio R is maintained at the current value, the thinner white line is changed to a smaller second threshold WD2. If the process continues as a recognition target and the current white line width threshold WD is the second threshold WD2, the second threshold WD2 is held. On the other hand, when the white line candidate point width W is equal to or larger than the white line width threshold WD, the white line width threshold WD is reset to the initial value WD0, and the white line recognition using the initial value WD0 is continued.

  Thus, in the present embodiment, when the width of the white thousand candidate point between the white line start point and the white line end point detected on the road is smaller than the current white line width threshold value, the arrangement state of the point group of the white line candidate points Accordingly, the white line width threshold value is dynamically changed, and a white line thinner than a general white line width is set as a recognition target. As a result, even in a driving environment in which a white line that is thinner than usual appears over a predetermined section, the white line can be properly recognized, and driving support control based on the white line recognition result is not hindered.

  In the above-described embodiment, an example in which white line recognition is performed based on a pair of images captured in stereo has been described. However, the present invention is not limited to this and is captured by a monocular camera or the like. Of course, white line recognition may be performed based on the obtained image.

DESCRIPTION OF SYMBOLS 1 Vehicle 2 Driving support system 3 Stereo camera 4 Image processing engine (White line candidate point detection part, White line candidate point width calculation part, White line width threshold value setting part, White line point group selection part, White line calculation part)
5 Control unit Ps White line start point Pe White line end point W White line candidate point width WD White line width threshold WD0 Initial value WD1 First threshold WD2 Second threshold

Claims (2)

A white line candidate point detection unit that detects a point group of a white line start point and a white line end point as a white line candidate point based on luminance changes on a plurality of search lines set in a horizontal direction on an image obtained by capturing an image of the traveling environment of the vehicle. When,
A white line candidate point width calculation unit for calculating a horizontal position difference between the white line start point and the white line end point as a white line candidate point width;
Of the point group detected as the white line candidate point, a ratio of points smaller than the current value of the white line width threshold that is a threshold for the white line candidate point width is calculated, and the white line width threshold is dynamically changed according to the calculated ratio. A white line width threshold setting section variably set to
A white line point group selection unit that compares the white line candidate point width with the white line width threshold and selects a white line point group from the white line candidate points;
A white line recognition apparatus for a vehicle, comprising: a white line calculation unit that calculates a white line based on the selected white line point group. 2. The white line recognition device for a vehicle according to claim 1 , wherein the white line width threshold includes a threshold for recognizing a temporary white line thinner than a normal white line .

Images