KR102384971B1 - Apparatus and method for preprocessing of image recognition - Google Patents

Abstract

The present invention relates to a pre-processing apparatus for image recognition and a pre-processing method for image recognition by extracting an image with the highest left and right average brightness difference before image recognition and using it for image recognition.

Description

Image recognition preprocessing apparatus and image recognition preprocessing method {Apparatus and method for preprocessing of image recognition}

The present invention relates to a pre-processing apparatus for image recognition and a pre-processing method for image recognition, and more particularly, to a pre-processing apparatus for image recognition and pre-processing for image recognition by extracting an image with the highest image recognition reliability before performing image recognition and using it for image recognition it's about how

Recently, a technology using image recognition has been applied to many vehicles. It is a technology such as recognizing an object to calculate a vehicle's parking space and parking the vehicle in the calculated parking space, or recognizing a lane on the road surface during driving to warn the driver if the vehicle deviates from the lane. However, depending on the driving environment, there may be cases where a specific color exists in the image. In this case, objects to be recognized, such as vehicles, pedestrians, and lanes, may not be properly recognized during the image recognition process.

Accordingly, recently, research is being conducted on a technology for securing accurate recognition performance when performing image recognition in various environments.

An object of the present invention is to provide an image recognition pre-processing apparatus and an image pre-processing method for extracting an image having the highest image recognition reliability before performing image recognition and using it for image recognition.

In order to achieve the above object, an image recognition pre-processing apparatus according to an embodiment of the present invention includes: a camera unit for acquiring an image around a vehicle; a top-view image output unit for outputting a top-view image from the image around the vehicle; a YUV image extraction unit that converts the top-view image into a YUV method and extracts a Y-channel image; an RGB image extraction unit that converts the top-view image into an RGB method and extracts an R-channel image and a B-channel image; and from each of the Y-channel image, the R-channel image, and the B-channel image, extracting a feature point to recognize an object, and calculating the left and right average brightness differences on the basis of the feature point existing in the recognized object. , a control unit for extracting an image having the largest left and right average brightness difference.

In order to achieve the above object, the image recognition pre-processing method according to an embodiment of the present invention includes the steps of obtaining an image around a vehicle and outputting a top view image; extracting a Y-channel image by converting the top-view image into a YUV method; extracting an R-channel image and a B-channel image by converting the top-view image into an RGB method; recognizing an object by extracting feature points from each of the Y-channel image, the R-channel image, and the B-channel image; calculating a left-right average brightness difference between left and right sides of the feature point present in the recognized object in each of the Y-channel image, the R-channel image, and the B-channel image; and extracting an image with the largest difference in left-right average brightness from among the Y-channel image, the R-channel image, and the B-channel image.

According to the image recognition pre-processing apparatus and image pre-processing method of the present invention, there are one or more of the following effects.

The image recognition pre-processing apparatus and image pre-processing method according to an embodiment of the present invention preprocess the image having the highest image recognition reliability among Y-channel images, R-channel images, and B-channel images before object recognition so as to be a source of image recognition. It has the effect of recognizing objects more accurately by changing the source image according to the color of the floor by processing.

Effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description of the claims.

1 is a block diagram showing the configuration of an image recognition pre-processing apparatus according to an embodiment of the present invention.
2 shows the top hat filter used by the controller and the feature points extracted by the top hat filter.
3 is a top-view image diagram and an enlarged view illustrating that the control unit recognizes a lane using a top-hat filter.
FIG. 4 is a control flowchart illustrating a flow of an image recognition pre-processing method according to an embodiment according to the configuration of FIG. 1 .

Advantages and features of the present invention and methods of achieving them will become apparent with reference to the embodiments described below in detail in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in various different forms, and only these embodiments allow the disclosure of the present invention to be complete, and common knowledge in the art to which the present invention pertains It is provided to fully inform those who have the scope of the invention, and the present invention is only defined by the scope of the claims. Like reference numerals refer to like elements throughout.

Hereinafter, the present invention will be described with reference to the drawings for explaining an image recognition pre-processing apparatus according to an embodiment of the present invention.

1 is a block diagram showing the configuration of an image recognition pre-processing apparatus according to an embodiment of the present invention. 1, the image recognition pre-processing apparatus according to an embodiment of the present invention includes a camera unit 100, a top-view image output unit 200, a YUV image extraction unit 300, an RGB image extraction unit 400, It includes a control unit 500 , a parking division extracting unit 600 , and a parking support unit 700 .

The camera unit 100 acquires an image around the vehicle. The camera unit 100 is installed on the front, rear, and both sides of the vehicle to acquire images around the vehicle.

The top-view image output unit 200 outputs a top-view image from images around the vehicle. The top-view image output unit 200 outputs a top-view image by performing image synthesis on images around the vehicle at the front, rear, and both sides of the vehicle.

The YUV image extraction unit 300 extracts the Y-channel image by converting the top-view image to the YUV method. The YUV image extraction unit 300 extracts the Y-channel image without a separate conversion process when the top-view image is already in the YUV format. The YUV image extraction unit 300 extracts the Y-channel image by extracting the Y-channel component with respect to all pixels of the YUV-type image. The YUV image extraction unit 300 converts the top-view image format to the YUV format when the top-view image is the RGB format. The YUV image extraction unit 300 converts the top-view image of the RGB method into a top-view image of the YUV method by using Equation 1.

[Equation 1]

Y = 0.257R + 0.504G + 0.098B + 16

U = -0.148R - 0.291G + 0.439B + 128

V = 0.439R - 0.368G - 0.071B + 128

Here, R, G, and B denote an R-channel component, a G-channel component, and a B-channel component, respectively, and Y and U V denote a Y-channel component, a U-channel component, and a V-channel component, respectively.

The RGB image extraction unit 400 converts the top-view image into an RGB method to extract an R-channel image and a B-channel image. The RGB image extraction unit 400 extracts the R-channel image and the B-channel image without a separate conversion process when the top-view image is already in the RGB format. The RGB image extraction unit 400 extracts an R-channel component and a B-channel component with respect to all pixels of an RGB-type image to extract an R-channel image and a B-channel image. The RGB image extraction unit 400 converts the top-view image format to the RGB format when the top-view image is the YUV format. The RGB image extraction unit 400 converts the YUV-type top-view image into an RGB-type top-view image by using Equation (2).

[Equation 2]

R = Y + 1.4075 (V -128)

G = Y - 0.3455 (U -128) - 0.7169 (V -128)

B = Y + 1.37790 (U -128)

The controller 500 recognizes an object by extracting feature points from each of the Y-channel image, the R-channel image, and the B-channel image. The control unit 500 extracts feature points from each of the Y-channel image, the R-channel image, and the B-channel image by using a top-hat filter. In this embodiment, the control unit 500 extracts the feature points of the lane. The control unit 500 extracts a feature point using a top-hat filter. The control unit 500 searches for a feature point different from the feature point by 360 degrees, and recognizes the plurality of feature points as lanes when the plurality of feature points are searched in one direction. The process of the controller 500 recognizing a lane by searching for a feature point will be described in detail with reference to FIGS. 2 and 3 , taking as an example that the controller 500 recognizes a lane.

The controller 500 calculates the left and right average luminance differences on the left and right on the basis of the feature points present in the recognized object, and extracts the image with the largest left and right average luminance difference. The left and right average brightness difference calculated by the controller 500 means image recognition reliability. The controller 500 calculates the left and right average brightness difference between the extracted feature points based on the feature points A existing on the recognized lane. The controller 500 calculates the left and right average brightness differences in the Y-channel image, the R-channel image, and the B-channel image. The controller 500 extracts an image with the largest difference in the calculated left and right average brightness from among the Y-channel image, the R-channel image, and the B-channel image. A detailed description thereof will be described with reference to FIG. 3 .

In the case of recognizing an object in an image that has not been pre-processed in the past, the recognition rate is lowered according to the color of the floor. Therefore, the image recognition pre-processing apparatus according to an embodiment of the present invention pre-processes the image having the highest image recognition reliability among the Y-channel image, the R-channel image, and the B-channel image before object recognition so as to be a source of image recognition, so that the object is more accurately has the effect of recognizing That is, the present invention allows an object to be recognized in an optimal image by changing the source image according to the color of the floor.

The reason that the control unit 500 considers only the Y-channel component, the R-channel component, and the B-channel component is that the G-channel component is already expressed by mixing the U-channel component and the V-channel component, and reflects the R-channel component and the B-channel component. This is because the U-channel component and the V-channel component are already reflected. Therefore, the control unit 500 reflects the components of all channels by considering the Y-channel component, the R-channel component, and the B-channel component.

The parking section extracting unit 600 extracts a parking section from the image with the largest left and right average brightness difference. The parking division extractor 600 detects the intersection between the parking lines by setting a region of interest (ROI) in the lane in the Y-axis direction recognized by the control unit 500, and calculates the distance between the intersections and the width of the vehicle. By comparing, it is determined whether the parking is valid and the parking section is extracted. The parking section extractor 600 recognizes a lane by applying the top-hat filter in the X-axis direction in the region of interest. The parking division extraction unit 600 extracts the intersection of the recognized lane and the previously recognized lane in the Y-axis direction. The parking segment extraction unit 600 extracts the parking segment of the vehicle by comparing the extracted distance between the intersections and the width in the Y direction of the vehicle. The parking division extracting unit 600 has the advantage of reducing the amount of computation of the algorithm by performing intersection detection only in the lane in the Y-axis direction previously recognized by the control unit 500 .

The parking support unit 700 parks the vehicle in the extracted parking section. The parking support unit 700 may include a motor-driven power steering (MDPS), acceleration means, and braking means to park the vehicle. A detailed process in which the parking support unit 700 parks the vehicle in the parking section will be omitted.

2 shows the top hat filter used by the control unit 500 and the feature points extracted by the top hat filter. Referring to FIG. 2A , a top-hat filter having a difference in brightness between the dark part Da and the bright part Br of 3 is shown. The difference in brightness of the top hat filter can be changed according to the setting. The control unit 500 uses the top-hat filter to find a bright part in the image, and extracts a feature point.

Referring to FIG. 2B , the controller 500 extracts the boundary P between the dark part Da and the bright part Br through the top hat filter. At this time, on the exemplary diagram of FIG. 3 , the extracted feature points are displayed as shown in FIG. 2( b ). The controller 500 calculates a difference in brightness between the dark part Da and the bright part Br. The controller 500 calculates the left and right average brightness difference by calculating the brightness difference for all the feature points A on the lane in the top view image.

3 is a top view image showing that the control unit 500 recognizes a lane using a top hat filter and an enlarged view thereof. 3( a ) shows a top-view image of a driving vehicle. At this time, the control unit 500 uses each top view image of the Y-channel image, the R-channel image, and the B-channel image. 3(a) shows any one of a Y-channel image, an R-channel image, and a B-channel image.

The controller 500 sets the driving direction of the vehicle as the X-axis, and sets the direction perpendicular to the X-axis as the Y-axis. The control unit 500 extracts the feature points of the lane by using the top-hat filter in the Y-axis direction. The controller 500 extracts a feature point from a region other than the vehicle region. Since the top-hat filter extracts feature points based on brightness, it is possible to extract a feature point (A) above a lane as shown in FIG. The control unit 500 searches for the lane component by searching 360 degrees at the characteristic point, thereby excluding the characteristic point B that is not above the lane when calculating the left and right average brightness difference.

3(b) is an enlarged view of an enlarged feature point on a lane. The shape of the feature point in FIG. 3(b) has already been shown below in FIG. 2(b). Referring to FIG. 3B , the control unit 500 rotates counterclockwise or clockwise around the boundary P between the dark part Da and the bright part Br of the feature point in order to detect a lane candidate from the extracted feature point. It searches 360 degrees and extracts other feature points. The control unit 500 determines whether the plurality of extracted feature points is a lane.

Referring to FIG. 3C , the controller 500 extracts a plurality of feature points to recognize a lane. The controller 500 calculates the brightness difference between the left and right sides based on the plurality of feature points A existing on the recognized lane, and calculates the left and right average brightness difference between the plurality of feature points A. In this case, the control unit 500 may exclude the feature point B that does not exist on the lane from calculating the left and right average brightness difference.

The controller 500 extracts feature points from the Y-channel image, the R-channel image, and the B-channel image, respectively. The controller 500 calculates each left and right average brightness difference from the Y-channel image, the R-channel image, and the B-channel image. The controller 500 calculates the left-right average brightness difference of the Y-channel image, the left-right average brightness difference of the R-channel image, and the left-right average brightness difference of the B-channel image.

The controller 500 extracts an image with the largest difference in the calculated left and right average brightness from among the Y-channel image, the R-channel image, and the B-channel image.

In this embodiment, the recognized object has been described as the lane, but the recognized object may be a vehicle, a pedestrian, a headlamp, etc., and is not limited to a specific object.

The operation of the image recognition pre-processing method according to an embodiment of the present invention will be described as follows. FIG. 4 is a control flowchart illustrating a flow of an image recognition pre-processing method according to an embodiment according to the configuration of FIG. 1 . Referring to FIG. 4 , the camera unit 100 acquires an image around the vehicle ( S110 ). The top-view image output unit 200 outputs a top-view image from an image around the vehicle ( S110 ).

The YUV image extraction unit 300 extracts the Y-channel image by converting the top-view image to the YUV method (S120). The YUV image extraction unit 300 converts the top view image of the RGB method into the YUV method using Equation 1, and the conversion process of the top view image has been described in FIG. 1 .

The RGB image extraction unit 400 converts the top-view image into an RGB method to extract an R-channel image and a B-channel image. The RGB image extraction unit 400 converts the top-view image to the RGB method using Equation 2, and the conversion process has been described in FIG. 1 . At this time, the Y-channel image extraction of the YUV image extraction unit 300 and the extraction of the R-channel image and the B-channel image of the RGB image extraction unit 400 may be simultaneously performed.

The controller 500 recognizes an object by extracting feature points from each of the Y-channel image, the R-channel image, and the B-channel image (S140). In this embodiment, the controller 500 recognizes a lane as an example. The control unit 500 extracts a feature point using a top-hat filter. The control unit 500 searches for a feature point different from the feature point by 360 degrees, and recognizes the plurality of feature points as lanes when the plurality of feature points are searched in one direction. This has been described in detail with reference to FIGS. 2 and 3 .

The control unit 500 calculates the left and right average brightness differences between the left and right sides of the feature points present in the recognized object in each of the Y-channel image, the R-channel image, and the B-channel image (S150). The controller 500 calculates the left and right average brightness difference at the feature point A on the recognized lane. The controller 500 calculates the left and right average brightness difference between the extracted feature points based on the feature points A existing on the recognized lane. A process in which the controller 500 calculates the left and right average brightness difference has been described with reference to FIG. 3 .

The controller 500 extracts an image with the largest left and right average brightness difference from among the Y-channel image, the R-channel image, and the B-channel image (S160).

The parking section extractor 600 calculates a parking section from the image with the largest left and right average brightness difference (S170). The parking division extraction unit 600 detects an intersection between parking lines by setting a Region Of Interest (ROI) in the lane recognized by the control unit 500, and compares the distance between the intersections with the width of the vehicle to determine whether parking is possible. It determines whether it is valid and extracts a parking section.

The parking support unit 700 parks the vehicle in the calculated parking section (S180). The parking support unit 700 may include a motor-driven power steering (MDPS), acceleration means, and braking means to park the vehicle.

In the above, preferred embodiments of the present invention have been illustrated and described, but the present invention is not limited to the specific embodiments described above, and in the technical field to which the present invention belongs, without departing from the gist of the present invention as claimed in the claims Various modifications may be made by those of ordinary skill in the art, and these modifications should not be individually understood from the technical spirit or perspective of the present invention.

100: camera unit
200: top view image output unit
300: YUV image extraction unit
400: RGB image extraction unit
500: control unit
600: parking compartment extraction unit
700: parking support department

Claims (5)

a camera unit for acquiring an image around the vehicle;
a top-view image output unit for outputting a top-view image from the image around the vehicle;
a YUV image extraction unit for extracting a Y-channel image by converting the top-view image into a YUV method;
an RGB image extraction unit that converts the top-view image into an RGB method and extracts an R-channel image and a B-channel image; and
From each of the Y-channel image, the R-channel image, and the B-channel image, a feature point is extracted to recognize an object, and the left and right average brightness differences are calculated based on the feature point existing on the recognized object, and a controller for extracting an image with the largest difference in the average brightness between the left and right. 2. The method of claim 1
The control unit extracts the feature points from each of the Y-channel image, the R-channel image, and the B-channel image by using a top-hat filter. outputting a top-view image by acquiring an image around the vehicle;
extracting a Y-channel image by converting the top-view image to a YUV method;
extracting an R-channel image and a B-channel image by converting the top-view image into an RGB method;
recognizing an object by extracting feature points from each of the Y-channel image, the R-channel image, and the B-channel image;
calculating a left and right average brightness difference between the left and right sides of the feature point present on the recognized object in each of the Y-channel image, the R-channel image, and the B-channel image; and
and extracting an image with the largest difference in left-right average brightness from among the Y-channel image, the R-channel image, and the B-channel image. 4. The method of claim 3
In the object recognition step, an image recognition pre-processing method of extracting the feature point using a top-hat filter. a camera unit for acquiring an image around the vehicle;
a top-view image output unit for outputting a top-view image from the image around the vehicle;
a YUV image extraction unit for extracting a Y-channel image by converting the top-view image into a YUV method;
an RGB image extraction unit that converts the top-view image into an RGB method and extracts an R-channel image and a B-channel image;
From each of the Y-channel image, the R-channel image, and the B-channel image, a feature point is extracted to recognize an object, and the left and right average brightness differences are calculated based on the feature point existing on the recognized object, a control unit for extracting an image with the largest difference in average brightness between the left and right;
A parking section extracting unit for extracting a parking section from the image with the largest left and right average brightness difference; and
A vehicle comprising a parking support unit for parking the vehicle in the parking compartment using a motor-driven power steering (MDPS).

Images