KR101998584B1 - Lane detection apparatus and lane detection method - Google Patents

Abstract

A lane detecting apparatus and a lane detecting method for detecting a lane area by connecting a plurality of lane segments extracted from a top view image are provided.
According to another aspect of the present invention, there is provided a lane detecting apparatus including: an image converting unit for converting a forward image into a top view image; A filtering unit for filtering the top view image based on brightness information of each pixel in the top view image and a plurality of pixels spaced by a reference distance to both sides of the each pixel; A lane area detecting unit for extracting a plurality of lane segment from the filtered image and detecting a lane area based on the degree of similarity between the plurality of lane segment segments; And a mapping unit for mapping the detected lane area to the forward image.

Description

TECHNICAL FIELD [0001] The present invention relates to a lane detecting apparatus and a lane detecting method,

And more particularly to a lane detecting apparatus and a lane detecting method for detecting a lane in a forward image.

It is important to detect the environment information of the vehicle because the ADS or the ADS performs the vehicle control according to the surrounding environment of the driving vehicle.

The environment information of the vehicle may include presence information of the front, rear, and / or side vehicles, speed information, distance information, and information other than adjacent vehicles such as a pedestrian, a traffic signal, and the like. Further, the environment information of the vehicle may include traveling road information including lane information.

Many driver assistance systems or autonomous driving systems are provided to operate based on the information of the front lane. Therefore, in order to improve the driving convenience and the driving stability of the driver, it is necessary to detect and recognize a lane on the road ahead.

In the case of the conventional lane detection algorithm, the lane information is estimated using the Huff transform or the RANSAC based model fitting in the edge image. As a result, there is a high probability that the lane-line detection will occur due to the edge information that occurs at a place other than a number / letter or a lane on the road.

In addition, the existing lane detection algorithm displays the position of the lane using one straight line or a curved line, but the actual lane is an area having a specific size, so that a difference between the lane detection result and the actual lane may occur.

One aspect of the disclosed invention provides a lane detecting device and a lane detecting method for detecting a lane area by connecting a plurality of lane segments extracted from a top view image.

According to another aspect of the present invention, there is provided a lane detecting apparatus including: an image converting unit for converting a forward image into a top view image; A filtering unit for filtering the top view image based on brightness information of each pixel in the top view image and a plurality of pixels spaced by a reference distance to both sides of the each pixel; A lane area detecting unit for extracting a plurality of lane segment from the filtered image and detecting a lane area based on the degree of similarity between the plurality of lane segment segments; And a mapping unit for mapping the detected lane area to the forward image.

The filtering unit may include at least one of a brightness value and a brightness value of each of the pixels in the top view image and the plurality of pixels spaced apart from each other by the reference distance determined by the lane width, And the top view image may be filtered based on the brightness information.

In addition, the lane area detecting unit may perform a directional test on the pixels in the filtered image to cluster the pixels, and extract the clustered pixels as lane segments.

The lane area detecting unit estimates line segment equations of each of the plurality of lane segments using a Least Square method, and determines which of the plurality of lane segments is extrapolated using the estimated line segment equation The similarity between the lane segments can be calculated by comparing the extracted line segments with the threshold values.

In addition, the mapping unit may map the detected lane area by matching the boundary of the lane area with an edge component detected according to a lateral brightness change of a plurality of pixels constituting the forward image.

The mapping unit may map the detected lane area according to a linear model in a predetermined short-range area of the forward image, and map the detected lane area to any one of the linear model and the hyperbolic model in a remote area other than the near- And may map the detected lane area.

According to another aspect of the present invention, there is provided a lane detection method comprising: converting a forward image into a top view image; Filtering the top view image based on brightness information of each pixel in the top view image and a plurality of pixels spaced by a reference distance to both sides of each pixel; Extracting a plurality of lane segments from the filtered image; Detecting a lane area based on the degree of similarity between the plurality of lane segment segments; And mapping the detected lane area to the forward image; . ≪ / RTI >

The step of filtering the top view image may further include the step of calculating a brightness value of each of the pixels in the top view image and the plurality of pixels spaced apart from each other by the reference distance determined by the lane width, The top view image may be filtered based on the brightness information.

The step of extracting the plurality of lane segments may include grouping pixels by performing a directional test on the pixels in the filtered image, and extracting the grouped pixels into lane segments.

The step of detecting the lane area may include estimating a line segment equation of each of the plurality of lane segments using Least Square and determining which one of the plurality of lane segments The degree of similarity between the lane segments can be calculated by comparing the extrapolated line segment with the threshold value.

The mapping of the lane area may map the detected lane area by matching the boundary of the lane area with the edge component detected according to the lateral brightness change of the plurality of pixels constituting the forward image .

The step of mapping the lane area may further include mapping the detected lane area according to a linear model in a predetermined short range area of the forward image and mapping the detected lane area to a rectilinear and hyperbolic area in a remote area other than the near area, And the detected lane area may be mapped according to any one of the models.

The present invention makes it possible to reduce the lane-finding error in comparison with the prior art technique by making a lane segment and classifying the lane existing in the front without model fitting through segment clustering and outlier removal, (Center line) and dotted / solid line can be simultaneously performed. In addition, the present invention can simultaneously estimate the width of a lane in an image by localizing the left and right ends of the lane without representing one lane using one line segment.

The lane information detected by the present invention is used for the core technology of an autonomous vehicle such as a lane departure warning of a vehicle, a lane keeping assistance system, and an adaptive cruise control Can be used.

1 is a control block diagram of a lane detecting apparatus according to an embodiment of the disclosed invention.
2 is a view showing a front image obtained by the photographing unit according to an embodiment of the disclosed invention.
3 is a view showing a top view image obtained by a coordinate system converting unit according to an embodiment of the present invention.
4 is a diagram illustrating a result of filtering a top view image by a step row filter of a filtering unit according to an embodiment of the present invention.
5 is a diagram showing an image in which a lane segment is extracted by the filtering unit according to an embodiment of the present invention.
6 is a diagram showing an image in which a lane area is detected by a lane area detecting unit according to an embodiment of the present invention.
7 is a diagram illustrating a forward image in which a lane area is mapped by a lane area mapping unit according to an embodiment of the present invention.
8 is a flowchart of a lane detection method according to an embodiment of the disclosed invention.

It is to be understood that both the foregoing general description and the following detailed description of the present invention are exemplary and explanatory only and are not restrictive of the invention, as claimed, and it is to be understood that the invention is not limited to the disclosed embodiments.

The terminology used herein is for the purpose of describing the embodiments only and is not intended to limit and / or to limit the disclosed invention.

For example, the phrase "a" or "an" in this specification may include a plurality of terms, unless the context clearly dictates otherwise.

It is also to be understood that the terms " comprises "or" having "are intended to indicate that there are features, numbers, steps, operations, elements, But do not preclude the presence or addition of a number, a step, an operation, an element, a component, or a combination thereof.

Also, terms including ordinal numbers such as " first ", "second ", and the like are used to distinguish one element from another, and do not limit the one element.

In addition, terms such as "to part "," to ", "to block "," to absent ", "module ", and the like may denote a unit for processing at least one function or operation. For example, the terms may refer to at least one hardware, such as a field-programmable gate array (FPGA) / application specific integrated circuit (ASIC), at least one software stored in memory, have.

Hereinafter, an embodiment of the present invention will be described in detail with reference to the accompanying drawings. Like reference numbers or designations in the accompanying drawings may denote parts or components performing substantially the same function.

FIG. 1 is a control block diagram of a lane detecting apparatus according to an embodiment of the disclosed invention. FIG. 2 shows a front image obtained by the photographing unit according to an embodiment of the disclosed invention. FIG. 4 shows a result obtained by filtering a top view image by a step row filter of a filtering unit according to an embodiment of the disclosed invention, and FIG. 5 FIG. 6 shows an image in which a lane area is detected by the lane area detecting unit according to an embodiment of the present invention, and FIG. 7 is a diagram illustrating a forward image in which a lane area is mapped by a lane area mapping unit according to an embodiment of the present invention.

Referring to FIG. 1, a lane detecting apparatus according to an embodiment of the present invention includes a photographing unit 100 for obtaining a forward image; An image processor 200 for detecting a lane area from the obtained forward image and mapping the detected lane area to a forward image; A display unit (D) for displaying a forward image mapped with a lane area; . ≪ / RTI >

The photographing unit 100 can acquire a forward image. Here, the forward image may refer to an image including a traveling road existing in the traveling direction of the vehicle. Referring to FIG. 2, the forward image may include a traveling road, a preceding vehicle traveling on the traveling road, an object existing outside the traveling road, and the like, and may include a lane on the traveling road.

In order to acquire the forward image, the photographing section 100 may be installed inside or outside the vehicle so as to face the front of the vehicle. The photographing unit 100 mounted on the vehicle may have an inherent value such as a height from the ground surface, an angle with respect to the front (pitch angle), and the like.

In addition, the photographing unit 100 may be a monocular camera or a stereo camera, but the monocular camera will be described below as a premise.

1, the photographing unit 100 may be provided as a separate device independent of the lane detecting device.

Referring again to FIG. 1, the image processing unit 200 may detect a lane area in the forward image acquired by the photographing unit 100 and map the detected lane area on the forward image. To this end, the image processing unit 200 according to an embodiment of the disclosed invention includes an image conversion unit 210 for converting a forward image into a top view image; A filtering unit 220 for filtering the top view image; A lane area detecting unit 230 for detecting a lane area by connecting a plurality of lane segments extracted from the filtered image; A mapping unit 240 for mapping the detected lane area to a forward image; . ≪ / RTI >

The image converting unit 210 may be provided to acquire a changed image of the input image. Specifically, the image converting unit 210 may convert the forward image into a top view image. To this end, the image converting unit 210 can obtain an image according to the changed coordinate system by converting the coordinate system of the input image.

For example, the image conversion unit 210 may acquire a top view image by converting the coordinate system of the inputted forward image, or may convert the coordinate system of the inputted top view image to acquire the forward image.

If the spatial coordinate system of the three-dimensional space in which the vehicle exists is an X axis that is the traveling direction of the vehicle, a Y axis that is perpendicular to the traveling direction of the vehicle, a Y axis that is parallel to the ground, When the in-plane direction is constituted by the Z-axis, the plane coordinate system of the front image can be set with reference to the XZ plane. Specifically, the plane coordinate system of the front image has a height from the ground surface, which is an intrinsic value according to the installation position of the photographing unit 100 described above with respect to the XZ plane, an angle (pitch angle) Can be determined in consideration of the parameter value of < RTI ID = 0.0 >

The image converting unit 210 can obtain a top view image on the X-Y plane by converting the plane coordinate system of the determined forward image into the X-Y plane coordinate system. FIG. 3 illustrates a top view image obtained by changing the coordinate system of the forward image of FIG. Referring to FIG. 3, the top view image may represent visual information about the front of the front image. For example, the top view image may include a lane on the road, and the lane width of the road may be identified by w.

The filtering unit 220 may filter the top view image according to each pixel of the top view image and brightness information of a plurality of pixels spaced by a reference distance to both sides of each pixel. Here, the reference distance can be determined by the lane width.

Specifically, the filtering unit 220 calculates the brightness (brightness) including at least one of the brightness of each pixel in the top view image and the brightness values of a plurality of pixels spaced on both sides of each pixel by a reference distance determined by the lane width, The top view image can be filtered based on the information.

The filtering unit 220 according to an exemplary embodiment may filter the top view image by a step row filter according to Equation (1).

[Equation 1]

I (x-t, y) - I (x-t, y) - I (x, t)

Here, I represents the top view image, E represents the image filtered by the step row filter, x and y represent the coordinates of the pixels constituting the image on the XY plane, and t may represent the reference distance . The reference distance according to an embodiment may mean a distance of half or more of the lane width w in the top view image. Referring to Equation (1), the Step Row Filter includes a random pixel and two pixels spaced apart from the pixel by a reference distance And a larger brightness difference between two pixels spaced apart from the corresponding pixel by a reference distance, a larger brightness value can be assigned to the corresponding pixel.

Figure 4 illustrates a filtered top view image. In this way, the filtering unit 220 can extract only information about the lane from the top view image by the step row filter.

Referring again to FIG. 1, the lane area detecting unit 230 may perform binarization on the filtered top view image. Specifically, the filtering unit 220 compares the brightness value of each of the plurality of pixels constituting the filtered top view image with a predetermined threshold value, assigns a brightness value 1 to a pixel above a threshold value, Brightness value 0 can be assigned.

After performing the binarization, the lane area detecting unit 230 may perform clustering on the binarized image. Specifically, the lane area detecting unit 230 may perform a first connectivity test with respect to four pixels in adjacent directions for each of a plurality of pixels constituting the binarized image. If there is a pixel having connectivity greater than or equal to a predetermined threshold value among adjacent pixels, the lane area detecting unit 230 may perform clustering by considering the pixel as the same area.

Accordingly, the lane area detecting unit 230 can extract a plurality of lane segments. Here, the lane segment may mean a set of clustered pixels of the filtered top view image. FIG. 5 illustrates a top view image in which lane segments are extracted by clustering. FIG. 5 shows a result of clustering after binarizing the filtered top view image of FIG. 4, and the lane segment is illustrated as a color region.

Then, the lane area detecting unit 230 can detect the lane area by connecting the extracted lane segments. Specifically, the lane area detecting unit 230 can connect lane segments based on the similarity between a plurality of lane segment segments. To this end, the lane area detecting unit 230 may obtain a reference line of each of the plurality of lane segments.

Here, the reference line may mean a line segment equation obtained by applying the method of Least Squares to the lane segment. If a plurality of reference lines are obtained for each of the plurality of lane segment segments, the lane area detecting unit 230 extrapolates arbitrary two lane segments using a reference line, compares extrapolated segments with a threshold value, Can be calculated.

The lane area detecting unit 230 can detect a lane area by connecting a plurality of lane segments having a high degree of similarity. FIG. 6 illustrates a case where a lane area is detected by connecting a plurality of lane segments in FIG. 5, and it can be confirmed that a lane is detected as an area that is not a straight line.

In addition, the lane area detecting unit 230 may remove noise by comparing the detected lane area with previously stored road information. For example, the lane area detecting unit 230 can remove a lane area that does not satisfy the conditions such as the number of lanes of the previously stored lane, the lane width, the lane-to-lane distance, and the shape of the lane.

Referring again to FIG. 1, after the lane detection is completed, the lane area mapping unit 240 may map the detected lane area to the forward image. To this end, the coordinate system of the detected lane area can be converted from the X-Y coordinate system to the plane coordinate system for the forward image by the coordinate system converter 210 described above.

When the coordinate system of the lane area is converted, the lane area mapping unit 240 can map the detected lane area on the forward image. At this time, the lane area mapping unit 240 can perform localization in the adjacent area of the mapped lane area. Here, the adjacent area may mean an area belonging to a certain distance from the mapped lane area, and localization may mean that both sides of the mapped lane area coincide with the lane boundary on the forward image.

For this purpose, the lane area mapping unit 240 can set an area estimated as a lane based on the lateral brightness change of the front image within the adjacent area determined according to the mapped lane area. For example, the lane area mapping unit 240 can estimate a boundary where the brightness in the lateral direction is abruptly increased in the adjacent region and a boundary in which the brightness in the lateral direction is abruptly darkened as the lane boundary. At this time, the lane area detecting unit 230 can determine whether the brightness is rapidly brightened or rapidly darkened by comparing with a predetermined threshold value.

When the lane boundary is reached, the lane area mapping unit 240 can perform localization by matching both sides of the lane area mapped in the forward image with the lane boundary estimated from the forward image.

At this time, the lane area mapping unit 240 may apply a different mapping method depending on the distance from the viewpoint. Specifically, the lane area mapping unit 240 can map the lane area according to the straight line model in a near area within a predetermined distance from the viewpoint of the forward image. In addition, the lane area mapping unit 240 can map the lane area according to the hyperbolic model in a remote area that is an area exceeding the adjacent distance from the viewpoint of the forward image.

FIG. 7 shows a result of mapping the lane area detected in the top view image of FIG. 6 to the forward image, and means a lane area mapped with a green area.

Referring back to FIG. 1, the front image in which the lane area is mapped can be provided to the user by the display unit D. 1 illustrates a case where the lane detecting device includes the display portion D, but the display portion D may be provided as an independent device separate from the lane detecting device.

8 is a flowchart of a lane detection method according to an embodiment of the disclosed invention.

First, the front image can be transformed into a top view image by the image transform unit 210. (S100) Specifically, the image transform unit 210 transforms a front image by transforming the coordinate system of the front image You can convert to top view video. Since the coordinate system of the forward image is determined in consideration of the eigenvalues such as the height from the ground of the photographing unit 100, the angle of inclination with respect to the front, and the parameters set in the photographing unit 100, The coordinate system conversion to the top view image can be performed.

After the top view image is obtained, the filtering unit 220 may filter the top view image based on the brightness information of each pixel in the top view image and a plurality of pixels spaced by a reference distance to both sides of each pixel (S110). For example, the filtering unit 220 filters the top view image using the step row filter according to Equation (1), thereby obtaining the pixels of the top view image, two pixels spaced apart from each other by a reference distance It is possible to set the brightness of the corresponding pixel having a large difference in brightness between the pixels.

When the filtering is completed, the lane area detecting unit 230 can extract a plurality of lane segments from the filtered image (S120). To this end, the lane area detecting unit 230 extracts the filtered top view image Binarization can be done. Then, the lane area detecting unit 230 can extract lane segments through clustering of a plurality of pixels of the binarized image.

When the lane segment is extracted, the lane area detecting unit 230 can detect the lane area on the basis of the similarities among the plurality of lane segment segments. (S130) Specifically, the lane area detecting unit 230 detects, A plurality of reference lines for each segment can be obtained and a similarity between two lane segments can be calculated by comparing a line segment extrapolated from any two lane segments with a threshold value using a reference line. If the estimated degree of similarity is large, the lane area detecting unit 230 can detect the lane area by connecting the two lane segments.

Lastly, the lane area mapping unit 240 may map the detected lane area to the forward image. (S140) At this time, the lane area mapping unit 240 may map the boundary between the front image and the front image And the lane area can be mapped to the forward image by matching the set boundary with the detected lane area.

As described above, according to the lane detecting device and the lane detecting method according to the embodiment, since the lane is detected as an area, erroneous detection caused by a number or characters other than the lane on the road can be reduced. Further, detection of a lane in the form of a double line like a center line and classification of a dotted line / a solid line can be performed at the same time.

According to the lane detecting apparatus and the lane detecting method according to the other embodiments, since the both sides are localized on the forward image in the mapping of the lane area, it is possible to estimate the lane width in the image.

The foregoing detailed description should not be construed in any way as being restrictive and should be considered illustrative. The scope of the disclosed invention should be determined by rational interpretation of the appended claims, and all changes within the equivalency range of the disclosed inventions are included in the scope of the disclosed invention.

100:
200:
210: coordinate system conversion unit
220:
230: lane area detecting unit
240: lane area mapping unit
D:

Claims (12)

An image converter for converting a forward image into a top view image;
A filtering unit for filtering the top view image based on brightness information of each pixel in the top view image and a plurality of pixels spaced by a reference distance to both sides of the each pixel;
The reference distance is determined by the lane width,
A lane area detecting unit for extracting a plurality of lane segments from the filtered image through clustering and detecting a lane area based on the similarity between the plurality of lane segment segments; And
And a mapping unit for mapping the detected lane area to the forward image. The method according to claim 1,
Wherein the filtering unit comprises:
The top view image is filtered based on the brightness information including at least one of the brightness level of each of the pixels in the top view image and the brightness of the plurality of pixels spaced apart from each other by the reference distance, Of the lane detecting device. The method according to claim 1,
Wherein the lane area detecting unit comprises:
Performing a directional test on pixels in the filtered image to cluster the pixels, and extracting the clustered pixels into lane segments. The method according to claim 1,
Wherein the lane area detecting unit comprises:
Estimating a line segment equation of each of the plurality of lane segments using a Least Square method and calculating a line segment and a threshold value obtained by extrapolating any two of the plurality of lane segments using the estimated line segment equation A lane detecting device for calculating a degree of similarity between the lane segment The method according to claim 1,
Wherein the mapping unit comprises:
And maps the detected lane area by matching the boundary of the lane area with the edge component detected according to the lateral brightness change of the plurality of pixels constituting the forward image. The method according to claim 1,
Wherein the mapping unit comprises:
Maps the detected lane area according to a straight line model in a predetermined short-range area of the forward image,
And maps the detected lane area according to any one of the straight line model and the hyperbolic model in a remote area other than the near area out of the front images. Converting a forward image into a top view image;
Filtering the top view image based on brightness information of each pixel in the top view image and a plurality of pixels spaced by a reference distance to both sides of each pixel;
The reference distance is determined by the lane width,
Extracting a plurality of lane segments from the filtered image through clustering;
Detecting a lane area based on the degree of similarity between the plurality of lane segment segments; And
Mapping the detected lane area to the forward image; The lane detection method comprising: 8. The method of claim 7,
Wherein filtering the top view image comprises:
The top view image is filtered based on the brightness information including at least one of the brightness level of each of the pixels in the top view image and the brightness of the plurality of pixels spaced apart from each other by the reference distance, The lane detection method comprising: 8. The method of claim 7,
Wherein the step of extracting the plurality of lane segments comprises:
Performing a directional test on the pixels in the filtered image to cluster the pixels, and extracting the clustered pixels as lane segments. 8. The method of claim 7,
Wherein the detecting the lane area comprises:
Estimating a line segment equation of each of the plurality of lane segments using a Least Square method and calculating a line segment and a threshold value obtained by extrapolating any two of the plurality of lane segments using the estimated line segment equation And calculating the degree of similarity between the lane segments. 8. The method of claim 7,
Wherein mapping the lane area comprises:
Wherein the detected lane area is mapped by matching edges detected in accordance with a lateral brightness change of a plurality of pixels constituting the forward image and the boundary of the lane area. 8. The method of claim 7,
Wherein mapping the lane area comprises:
Maps the detected lane area according to a straight line model in a predetermined short-range area of the forward image,
Wherein the detected lane area is mapped according to any one of the straight line model and the hyperbolic model in a remote area other than the near area of the forward image.

Images