JPH1185999A - White line detecting device for vehicle - Google Patents

Abstract

PROBLEM TO BE SOLVED: To precisely detect even a white line in a complicated shape of double broken lines or the like by retrieving the Hough space of a converted edge point and detecting the whole line from a road surface image. SOLUTION: A source image which is picked up by a CCD camera and stored in a memory for source image storage is read out and while edge points are detected over the entire screen to make positive/negative decision on them, the edge directions are calculated (S10). Then the positive edge points and negative edge points are processed separately through Hough conversion. At the same time, a dot array (pixels) in the image which are polling for a peak in the obtained Hough space are saved in a memory for polling point storage (S12) so that they can be extracted thereafter. The peak obtained in the Hough space generates a corresponding peak (S16). Points which have polled for the peak in the Hough space of a retrieved pair is read out of the polling point storage memory (S18).

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

The present invention relates to a vehicle white line detection device.

[0002]

2. Description of the Related Art Recently, a technology has been proposed in which a vehicle is equipped with a CCD camera (solid-state imaging device type camera) or the like to image a traveling road surface such as a highway, and a white line (road division line) is extracted from the obtained image. Various proposals have been made, for example,
56619 can be mentioned.

In the prior art, a point at which a gray value in an image including a white line changes from dark to light (hereinafter, referred to as a "positive edge point") and a point at which the gray value changes from light to dark (hereinafter, a "negative edge point"). ) Is detected (extracted) over the entire image,
A region where the left edge is a positive edge point, the right edge is a negative edge point, and the width of which is within a predetermined range corresponding to the width of the white line is detected as a white line candidate portion. Next, a skeleton line (center line) of the white line candidate portion is obtained, classified into patterns such as upward right or not, and Hough transform is performed on the classified pattern to detect a white line.

[0004]

The prior art relates to a white line detecting device for an automatic guided vehicle which moves by detecting a white line drawn on a factory site. Therefore, the white line has a simple composition. And the like, the pattern of the white line is also diversified, and for example, a double broken line as shown in FIG. 9 is also used.

If an attempt is made to detect such a complicated white line in accordance with the method proposed in the prior art, there may be a number of positive edge points and negative edge points in a predetermined area corresponding to the width of the white line. , Many combinations must be treated as white line candidate parts. Therefore, much time is required to process the entire image.

For this reason, in this type of technology, white lines are generally detected in real time by limiting the processing area to an image portion where the presence of white lines is predicted. However, when the processing area is limited, when a region having a characteristic similar to a white line, such as a branch line or a guardrail, is included in the region, there is a possibility that the line is erroneously recognized as a white line.

SUMMARY OF THE INVENTION Accordingly, an object of the present invention is to solve the above-mentioned inconvenience, and a vehicle capable of detecting a white line in real time even on a traveling road on which a white line having a complicated shape such as a double broken line is displayed. An object of the present invention is to provide a white line detecting device.

[0008]

In order to achieve the above object, according to the present invention, there is provided an image pickup means for picking up an image of a road surface in front of a vehicle, and a road surface image obtained by the image pickup means. , First, second, changing from light to dark or dark to light
Edge point detecting means for detecting edge points, Hough transform means for Hough transforming the detected first and second edge points, respectively, and searching for a Hough space of the transformed first and second edge points, It is configured to include a white line detection unit that detects a white line from a road surface image. Thus, a white line having a complicated shape such as a double broken line can be detected with high accuracy.

According to a second aspect of the present invention, the edge point detecting means is configured to detect the first and second edge points over the entire area of the road surface image. Thus, even in an image including a branch line, a guardrail, and the like, they are not erroneously recognized as white lines.

In the third aspect, the white line detecting means stores a point set of the converted image in the Hough space of the first and second edge points in a point set storage area. 1. Peak detecting means for detecting peaks in the Hough space of the first and second edge points respectively; Pair searching means for associating the detected peaks in the Hough space of the first and second edge points as a pair; A search unit that searches the storage unit for a point set of an image that has voted for the peak of the selected pair, and is configured to detect a white line based on the searched point set. Accordingly, in addition to the above-described effects, the white line can be detected with higher accuracy.

According to a fourth aspect of the present invention, the pair searching means is configured to fix one of the peaks in the Hough space of the first and second edge points and search for the corresponding pair. As a result, the amount of calculation can be reduced.

[0012]

Embodiments of the present invention will be described below with reference to the accompanying drawings.

FIG. 1 is a schematic diagram showing a vehicle white line detection device according to the present application in terms of hardware including a vehicle.

As shown in the figure, the apparatus includes one camera 14 mounted on the inner surface of a front windshield 12 of a vehicle 10 via an adhesive means.
4 images the road surface ahead of the vehicle 10 with a single eye. The camera 14 is a CCD (solid-state imaging device) camera,
It has 80 × width 502 pixels. The output of the CCD camera 14 is sent to an image processing ECU housed in the electronic control unit 16.

FIG. 2 shows the image processing ECU 20 (reference numeral 20).
FIG. 3 is a block diagram showing the configuration in detail).

As shown, the image processing ECU 20 includes a CP
U (processing hardware) 20a, a positive / negative edge detection and edge direction calculation unit (processing hardware) 20b for detecting a positive edge point and a negative edge point, and calculating an edge direction,
And Hough converter (processing hardware)
20c.

The image processing ECU 20 further includes a memory 20d for storing an original image, a memory 20e for storing an edge image, a memory 20f for storing a Hough transform result of positive and negative edge points, and a memory for voting images in the transformed Hough space. A voting point storage memory (the above-mentioned storage means) 20g for storing a point set (voting points) is provided. Reference numeral 22 indicates a bus.

Next, the operation of the vehicle white line detecting device according to the present invention will be described with reference to the flowchart of FIG.

The above-mentioned CCD camera 14 provides 66
An image signal is output every msec, and the program shown in the drawing is executed within that time, so that a white line is detected in real time.

First, in step S10, an original image captured by the CCD camera 14 and stored in the original image storage memory 20d is read, and edge points are detected (extracted) for the entire screen (entire area). Simultaneously with the determination, the edge direction (the direction of the local point sequence) is calculated.
The reason for finding the edge direction is to simplify the subsequent Hough transform operation.

That is, in the Hough transform performed in S14, as is well known, (ρ, θ)
Is used, the angle θ is not changed from 0 to 180 degrees, the edge direction is obtained from the edge strength in the horizontal direction and the vertical direction of the extracted edge point, and θ is changed only about 15 degrees before and after the edge direction.

As mentioned earlier, as shown in FIG. 8, a point (pixel) where dark to light changed is a positive edge point A point (pixel) where dark to light is changed is a positive edge point (the above-described first pixel). An edge point), and a point that has changed from light to dark is referred to as a negative edge point (the second edge point described above). The obtained edge image (intermediate data) is shown on the right side of S10 in the figure.

Then, the process proceeds to S12, in which the Hough transform is performed separately on the positive edge point and the negative edge point. At the same time, a point sequence (pixel) on the image voting for the peak in the obtained Hough space is stored in the voting point storage memory 20g so that it can be extracted later. The right side of the figure shows the Hough transform result, voting result, and voting point sequence performed on the positive edge point and the negative edge point.

Then, the program proceeds to S14, in which peaks are detected on the positive edge Huff space and the negative edge Huff space.

FIG. 4 is a subroutine flowchart showing the processing. This process is performed first on, for example, the Hough space of the positive edge, and then on the Huff space of the negative edge.

In the following, smoothing (averaging) is performed in S100, and the process proceeds to S102 to determine a threshold value. This is also to simplify the calculation, and to simplify the peak detection work described later. The threshold value is determined by dividing the screen into left and right regions, respectively. Since the peak is different between the case where the white line is a solid line and the case where the white line is a broken line, it is determined appropriately taking this into consideration.

More specifically, the threshold value is determined so that all the voting values (voting point sequence) of the search area in the Hough space are totaled and cells from the highest rank to a predetermined rank (for example, the tenth) remain. Note that what stores individual voting values in the Hough space (ρ-θ space) is referred to as a ρ-θ cell or simply a cell here for convenience.

Then, the process proceeds to S104, where the first ρ-θ cell is selected according to the search order.

The search order will be described with reference to FIG. 5. In general, a sequence of points constituting a traveling path is directed to an infinity point (infinity point is indicated by (0, y) in FIG. 5). In other words, the sequence of points that make up the travel path is the point at infinity (0, 0,
It is a group of straight lines passing through y). This means that the peak points are distributed on ρ = 0 cos θ + ysin θ in the ρ-θ space.

Therefore, the peak search in the ρ-θ space is performed only around this curve. By narrowing the search range in this way, it is possible to reduce the detection of linear components other than the road structure.

Further, on the image, as shown in FIG. 5, a search is made from the inside A of the traveling road toward the left and right traveling roads. Specifically, the search is performed in the ρ-θ space in the order shown in FIG. By doing so, a white line can be recognized even if the peak has a relatively low height at a position where a white line is expected to exist.

In the flow chart of FIG.
Proceeding to 106, it is determined whether the voting value is greater than or equal to the threshold value. If the result is affirmative, the process proceeds to S108, and it is determined whether the voting value is larger than all neighboring cells.
Proceeding to 0 makes that cell a local peak.

Subsequently, the flow advances to S112 to determine whether or not all the cells have been determined. If the determination is negative, the flow returns to S104 to repeat the above processing. Incidentally, S106 or S
If the determination in 108 is negative, no further determination is necessary, and the process jumps to S112. This operation is performed for all of the positive edge Huff space and the negative edge Huff space.

Returning to the flow chart of FIG.
Proceeding to 16, the peaks respectively obtained in the Hough space of the positive edge and the Huff space of the negative edge are created (searched) for the corresponding peaks.

FIG. 7 is a subroutine flowchart showing the operation.

First, referring to FIG. 8, an inner edge is first detected from both edges expected to form a white line. That is, in the case of the white line on the right side when viewed from the own vehicle, the center is set to the positive edge (the positive edge is fixed), and the corresponding (paired) negative edge is searched.

In the case of the left white line, a pair of positive edges is searched centering on the negative edge. 7 and 8 show a case where a pair is searched (created) for the left white line.

The pair search (creation) will be described with reference to FIG. 7. In S200, the first negative edge peak is selected. As shown in FIG. 8, it is designated as peak (1).

Next, the process proceeds to S202, in which the search area is set in the positive edge Huff space starting from the position of the negative edge peak, and the process proceeds to S204 to list all the positive edge peaks existing in the search area.

Next, the process proceeds to S206, and a positive edge closest to the starting point of the search area is selected from the obtained list.
That is, edges that are close on the Hough space are also close on the image. This is taken as peak (2) as shown in FIG.

Then, the process proceeds to S208, in which it is determined whether or not another negative edge peak exists between the peak (1) and the peak (2). If the result is negative, the process proceeds to S210, where the peak (1) and the peak ( 2) is paired. If the result in S208 is affirmative, S210 is skipped because the negative edge peak is fixed, which means that there is a more appropriate pair of peak (2) than (1).

Then, the program proceeds to S212, in which it is determined whether all negative edge peaks have been searched. If the result is affirmative, the program is immediately terminated.
Go to 4 and select the next negative edge peak and peak (1)
The process proceeds to S202 and the above process is repeated.

Returning to the flow chart of FIG.
Then, the process proceeds to step S18, and the voting point of the searched pair in the Hough space is read out from the voting point storage memory 22g. That is, a point sequence on the image that has voted for the peak is extracted. The obtained point sequence is shown on the right side of the figure.

Next, the process proceeds to S20, in which data evaluation, more specifically, time-series data evaluation is performed. This is S18
This is an operation of comparing the point sequence obtained in the processes up to and the past point sequence and verifying whether or not it is appropriate. Further, road parameters (position, curvature, etc.) are calculated from the obtained point sequence (described later).

Here, a case where vehicle control is performed using the white line extraction result obtained as described above will be briefly described with reference to FIG.

For example, the applicant of the present invention disclosed in
Based on the steering technique proposed in No. 23 or JP-A-9-128699, a target yaw rate is set in the electronic control unit 16 so that the vehicle 10 travels along the white line extracted from the travel road curvature and the like.

The driver steering torque detected via the torque sensor 26 disposed in the steering system is assisted so that the deviation from the output of the yaw rate sensor 24 disposed near the position of the center of gravity of the vehicle 10 is eliminated. The command value is calculated as described above, and the appropriate assist mechanism 28 (electric motor or the like) is driven based on the command value. The calculation of the travel path parameters in S22 means this.

As described above, the characteristic feature of the apparatus according to the present invention is that first, the positive and negative edges are obtained, Hough transform is performed immediately, and then the white line is detected based on the result of the Hough transform.

Referring to the drawings, for example, when processing an image obtained by capturing an image of a traveling road having a double dashed line as shown in FIG. 9, the conventional technology proposed in Japanese Patent Publication No. 6-56619 is used. As described above, if a line segment is extracted by Hough transform without discriminating the sign of the edge, an unnecessary straight line may be extracted as shown in FIG.

On the other hand, in the present invention, FIG.
As shown in A and B, since the positive and negative edges are obtained and separately subjected to the Hough transform, and then the line segment is extracted, such an inconvenience does not occur. That is, since the respective line segments (point strings) are arranged sufficiently far apart, the linear component serving as noise can be significantly reduced, so that a white line is not erroneously recognized.

Further, a feature of the apparatus according to the present invention is that the processing area is not limited, and positive and negative edges are detected in all areas (all screens) and Hough transform is performed.

Explaining the advantage, as in the prior art, when the processing area is limited to a place where a white line is likely to exist, an object having a characteristic similar to the white line such as a branch line or a guardrail is erroneously recognized as a white line. There is fear. FIG.
2A shows a case where the branch line is erroneously recognized as a white line, and FIG. 13A shows a case where the guard rail is erroneously recognized as a white line.

On the other hand, in this apparatus, since the entire screen is processed, even if a branch line or a guardrail is detected as a white line candidate, a correct white line is also detected as a white line candidate at the same time. You can choose one.

FIG. 12B shows a case where the entire screen is processed for an image including a branch line in this apparatus, and FIG. 13B shows a case where the entire screen is processed for an image including a guardrail in this apparatus. By processing the full screen,
It can be understood that erroneous recognition does not occur even for an image including a branch line or a guardrail.

Although the processing amount increases when processing the entire screen, this apparatus is configured to detect the positive and negative edge points over the entire screen and then to perform the Hough transform as described above. With such a configuration, the amount of calculation after Hough transform can be significantly reduced, and the load on the CPU can be greatly reduced. Further, since the processing up to the Hough transform can be implemented by hardware, the processing speed can be increased. For the above reasons, it is possible to process the entire screen in a short time.

In this embodiment, as described above, the image pickup means (CCD camera 14 and image processing ECU 20) for picking up an image of the road surface in front of the vehicle 10, and the image of the road surface obtained by the image pickup means, Edge point detecting means (S10 in FIG. 3) for detecting first and second edge points respectively changing from dark or dark to bright, and Hough transform means for Hough transforming the detected first and second edge points respectively ( FIG.
S12), the apparatus is configured to include a white line detecting means (S14 to S18 in FIG. 3) for searching the Huff space of the converted first and second edge points and detecting a white line from the road surface image.

Also, the edge point detecting means is configured to detect the first and second edge points respectively over the entire area of the road surface image (S10 in FIG. 3).

Further, the white line detecting means stores a point set of the converted image in the Hough space of the first and second edge points in a point set storage area (a voting point storage memory 20g, 3 (S12 in FIG. 3), the first and second
Peak detecting means (S14 in FIG. 3) for respectively detecting peaks of the edge points on the Hough space, and pair searching means for associating the detected peaks of the first and second edge points on the Hough space as pairs. 3 (S16 in FIG. 3), a search unit (S18 in FIG. 3) for searching the storage unit for a point set of an image that has voted for the peak of the searched pair, and detects a white line based on the searched point set. It was configured so that

The pair searching means fixes one of the peaks in the Hough space of the first and second edge points and searches for a pair corresponding to the peak (from S200 to S2 in FIG. 7).
14).

In the above-described embodiment, the camera is of the CCD image sensor type. However, any camera can be used as long as it can image the traveling road.

[0061]

According to the present invention, a white line having a complicated shape such as a double broken line can be detected with high accuracy.

According to the second aspect, even in an image including a branch line, a guardrail, and the like, they are not erroneously recognized as white lines.

According to the third aspect, in addition to the above-described effects, a white line can be detected with higher accuracy.

According to the fourth aspect, the amount of calculation can be reduced.

[Brief description of the drawings]

FIG. 1 is a schematic view showing an entire vehicle white line detection device according to the present application.

FIG. 2 is a block diagram illustrating details of an image processing ECU of the apparatus in FIG. 1;

FIG. 3 is a flowchart showing the operation of the apparatus in FIG. 2;

FIG. 4 is a subroutine flowchart showing a peak detecting operation on a Hough space in the flowchart of FIG. 3;

FIG. 5 is an explanatory diagram illustrating a search order in the flowchart of FIG. 4;

FIG. 6 is an explanatory diagram for explaining a search order in the flowchart of FIG. 4;

FIG. 7 is a subroutine flowchart showing an operation of creating a peak pair in the flowchart of FIG. 3;

FIG. 8 is an explanatory diagram for explaining the operation of the flowchart in FIG. 7;

FIG. 9 is an explanatory diagram of a traveling road image including a double dashed line.

FIG. 10 is an explanatory diagram showing a result of processing the image of FIG. 9 in the related art.

FIG. 11 is an explanatory diagram showing a result of processing the image of FIG. 9 by the device according to the present application.

FIG. 12 is an explanatory diagram for comparing results of processing of an image including a branch line with the apparatus according to the related art and the present application.

FIG. 13 is an explanatory diagram for comparing results obtained by processing an image including a guardrail with the apparatus according to the related art and the present application.

[Explanation of symbols]

 Reference Signs List 10 vehicle 14 CCD camera (imaging means) 20 image processing ECU 20g memory for storing voting points (storage means)

Claims (4)

[Claims] 1. A method according to claim 1, Imaging means for imaging a road surface in front of the vehicle, b. Edge point detecting means for detecting first and second edge points of a road surface image obtained by the image pickup means, which change from light to dark or dark to light, c. Hough transform means for Hough transforming each of the detected first and second edge points, d. A white line detection device for a vehicle, comprising: a white line detection unit configured to search a Hough space of the converted first and second edge points to detect a white line from the road surface image. 2. The vehicle white line detection device according to claim 1, wherein the edge point detection means detects the first and second edge points over the entire area of the road surface image. 3. The image processing apparatus according to claim 2, wherein the white line detecting means includes: e. Storage means for storing, in a point set storage area, the converted point set of the voted image in the Hough space of the first and second edge points; f. Peak detecting means for respectively detecting peaks of the first and second edge points on the Hough space; g. Pair searching means for associating the detected peaks of the first and second edge points on the Hough space as pairs, h. 2. The search device according to claim 1, further comprising: a search unit configured to search the storage unit for a point set of the image that has voted for the peak of the searched pair, wherein a white line is detected based on the searched point set. 3. The white line detection device for a vehicle according to claim 2. 4. The pair searching means according to claim 3, wherein said pair searching means fixes one of peaks of said first and second edge points in a Hough space and searches for a pair corresponding thereto. White line detector for vehicles.