KR101609303B1 - Method to calibrate camera and apparatus therefor - Google Patents

Abstract

This specification discloses a method for a calibration device to determine a vehicle and a lane area on an image taken in front of the vehicle. The method includes: receiving an image of a vehicle taken in front of the vehicle; A method of detecting an edge component in an image taken in front of the vehicle, extracting a straight line by houghing the detected edge component, determining a vanishing point based on the extracted straight line, Step through the horizon; The method includes the steps of: binarizing an image photographed in front of the vehicle to accumulate a predetermined number of frames or more; calculating a horizontal axis projection histogram from the accumulated image; and determining a bonnet area through a position where the calculated histogram is above a threshold value Determining; And detecting a lane on an image taken in front of the vehicle, and then determining a lane width through the detected lane.

Description

METHOD TO CALIBRATE CAMERA AND APPARATUS THEREFOR BACKGROUND OF THE INVENTION [0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a calibration apparatus and an operation method thereof, and more particularly to a method of defining a processing region for lane recognition, vehicle detection and / or inter-vehicle distance estimation in an input image, and an apparatus used therein.

BACKGROUND OF THE INVENTION With the widespread use of automobiles today, related technologies for the convenience of drivers are rapidly developing. In particular, ADAS (Advanced Driver Assistance System) technology that assists drivers in driving, such as navigation and a vehicle black box, is increasing. In addition, demand for video-based driver assistance devices is increasing due to its versatility and economy.

Particularly, a technique of detecting a lane or a vehicle by processing a forward image photographed by a camera mounted on a vehicle has been proposed. In the conventional lane detecting method, based on a mobile device, fast and simple edge extraction and Hough and a linear lane detection method using a transform. On the other hand, a method of minimizing a search area of a lane and solving a lane blur due to an obstacle by using forward scan information of a steerable filter and a laser sensor considering the directionality of an edge There is also.

As for the recognition of the type of lane, a method of determining the dotted line and the solid line by sampling the distribution of the candidate points on the lane and looking at the candidate point intervals is also proposed.

When the entire input image of the camera is processed when the lane recognition and the vehicle detection are performed through the camera input image input from the black box or the camera mounted on the vehicle, the calculation time becomes very long, and as the processing area becomes larger, There is a problem that accuracy is lowered.

Therefore, it is a necessary process to find only the lane and the area where the vehicle is located in the entire image according to the place where the camera is installed and the angle of camera installation of the black box.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a camera calibration apparatus and an operation method thereof for vehicle detection and / or inter-vehicle distance estimation. More specifically, it is an object of the present invention to provide a method for defining a processing region of an input image for lane recognition, vehicle detection and / or inter-vehicle distance estimation in an input image, and an apparatus used therefor.

A calibration apparatus is provided in accordance with one embodiment herein. The apparatus includes: an image receiving unit that receives an image of a front of a vehicle; A method of detecting an edge component in an image taken in front of the vehicle, extracting a straight line by houghing the detected edge component, determining a vanishing point based on the extracted straight line, A horizon determining part determining the horizon through; The method includes the steps of: binarizing an image photographed in front of the vehicle to accumulate a predetermined number of frames or more; calculating a horizontal axis projection histogram from the accumulated image; and determining a bonnet area through a position where the calculated histogram is above a threshold value A bonnet area determination unit for determining the bonnet area; And a lane width estimator configured to determine a lane width through the detected lane after detecting a lane on an image taken in front of the vehicle.

There is provided a method of determining a vehicle and a lane area in an image taken by a calibration device in front of a vehicle according to another embodiment of the present disclosure. The method includes: receiving an image of a vehicle taken in front of the vehicle; A method of detecting an edge component in an image taken in front of the vehicle, extracting a straight line by houghing the detected edge component, determining a vanishing point based on the extracted straight line, Step through the horizon; The method includes the steps of: binarizing an image photographed in front of the vehicle to accumulate a predetermined number of frames or more; calculating a horizontal axis projection histogram from the accumulated image; and determining a bonnet area through a position where the calculated histogram is above a threshold value Determining; And detecting a lane on an image taken in front of the vehicle, and then determining a lane width through the detected lane.

The embodiments of the present invention have the effect of accurately specifying a processing region for lane recognition, vehicle detection, and / or inter-vehicle distance estimation in an input image. Thus, the image processing performance can be improved by adapting to the installation position or angle of the camera.

1 is a block diagram illustrating a calibration apparatus in accordance with one embodiment of the present disclosure.
2 is an exemplary view showing lane and vehicle area settings according to an embodiment of the present invention;
3 is an exemplary diagram illustrating vanishing point detection according to an embodiment of the present invention.
4 is an exemplary view illustrating the measurement of the vanishing point reliability according to the embodiment of the present invention.
FIG. 5 is an exemplary diagram illustrating vehicle bonnet area determination in accordance with one embodiment of the present disclosure. FIG.
6 is a diagram illustrating lane width determination in accordance with one embodiment of the present disclosure.
7 is a flowchart illustrating a calibration method according to one embodiment of the present disclosure.

It is noted that the technical terms used herein are used only to describe specific embodiments and are not intended to limit the invention. It is also to be understood that the technical terms used herein are to be interpreted in a sense generally understood by a person skilled in the art to which the present invention belongs, Should not be construed to mean, or be interpreted in an excessively reduced sense. Further, when a technical term used herein is an erroneous technical term that does not accurately express the spirit of the present invention, it should be understood that technical terms that can be understood by a person skilled in the art are replaced. In addition, the general terms used in the present invention should be interpreted according to a predefined or prior context, and should not be construed as being excessively reduced.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings, wherein like reference numerals refer to like or similar elements throughout the several views, and redundant description thereof will be omitted. In the following description, well-known functions or constructions are not described in detail since they would obscure the invention in unnecessary detail. It is to be noted that the accompanying drawings are only for the purpose of facilitating understanding of the present invention, and should not be construed as limiting the scope of the present invention with reference to the accompanying drawings. The spirit of the present invention should be construed as extending to all modifications, equivalents, and alternatives in addition to the appended drawings.

1 is a block diagram illustrating a calibration apparatus in accordance with one embodiment of the present disclosure.

The lane recognition and inter-vehicle distance measuring apparatus to which the calibration apparatus 100 of the present disclosure can be applied is configured to detect a front vehicle traveling in the current lane by photographing the front of the vehicle in operation and analyzing the image, Can be calculated.

The inter-vehicle distance measuring apparatus can detect (extract) an object necessary for the measurement of the inter-vehicle distance from the received image. The object may be a vehicle, a lane (solid line, broken line, etc.), or other object. For example, the inter-vehicle distance measuring device may use a shadow characteristic indicated by a preceding vehicle, a contrast characteristic of a rear wheel, or the like to determine whether an object included in the received image is a vehicle or not, It is possible to judge whether the object is a vehicle or whether it is a rear part of the vehicle by discriminating whether the image has many edge in the horizontal direction or edge in the vertical direction. In addition, the inter-vehicle distance measuring apparatus may extract feature points indicating that the vehicle is a certain search area of the image, and compare the extracted feature points with the feature data of the vehicle. In this case, the vehicle characteristic data may be characteristic data of a part of the vehicle selected from among the learned vehicle characteristic data, wherein the characteristic data of the selected part of the vehicle may be a characteristic indicative of a texture or an outline of the upper part of the vehicle.

The inter-vehicle distance measuring apparatus can learn minutiae points of various vehicles by using various modeling techniques. After the learning, a specific search area (or a candidate area for a vehicle) is searched to determine whether the object of the image is a vehicle according to how much the vehicle feature points are found.

At this time, the calibration apparatus 100 can determine the " image processing region " for detecting the vehicle and estimating the inter-vehicle distance in the inter-vehicle distance measuring apparatus. That is, a vision processing region (an area for estimating lane recognition, vehicle detection, and / or inter-vehicle distance) that varies depending on a camera installation angle or a camera angle of view can be extracted from a black box or a camera image. The calibration apparatus 100 may be included in a vehicle apparatus such as a navigation system or a black box, or may operate in conjunction with the vehicle apparatus.

As described above, the camera calibration means a process of defining a non-processing region for lane recognition, vehicle detection and / or inter-vehicle distance estimation, and the accuracy of lane recognition, vehicle detection, and inter-vehicle distance estimation depends on the accuracy of the camera calibration Is greatly influenced.

When the entire input image of the camera is processed when the lane recognition and the vehicle detection are performed through the camera input image input from the black box or the camera mounted on the vehicle, the calculation time becomes very long, and as the processing area becomes larger, There is a problem that accuracy is lowered. Therefore, it is a necessary process to find only the lane and the region of interest (ROI) in the entire image in accordance with the installation location of the camera and the angle of camera installation of the black box.

Since the vehicle or lane detection performance may vary depending on the input image from which the calibration is started, the following describes an apparatus and method for accurately performing calibration.

The calibration apparatus 100 of the present invention may include an image receiving unit 101, a horizon line determining unit 102, a bonnet area determining unit 103, and a lane width estimating unit 104.

The calibration apparatus 100 determines the horizon h1 in the input image, determines the bonnet area h2 (lower end), estimates the lane width W, and completes the calibration. Referring to Fig. 2, h1 is a horizon in an arbitrary input image (vehicle front photographing image), h2 is an upper part of a vehicle on which a camera is mounted, that is, a bonnet top, and W is a width of the detected lane.

The image receiving unit 101 receives an image of the front of the vehicle taken from a camera module installed in a part of the vehicle. The photographed image may be a plurality of images photographed at a plurality of times.

The horizon line determining unit 102 determines a horizon line in an image of the front of the vehicle. In one embodiment of the horizon line determination, the horizon line determination unit 102 detects an edge component in an image taken in front of the vehicle, performs a hough transform on the detected edge component to extract a straight line , The vanishing point is determined based on the extracted straight line, and the horizon can be determined through the position of the vanishing point. When the edge component is detected, the horizon determining unit 102 can binarize the image to detect the edge component. Then, the detected edge components are Huff transformed to extract a straight line. Next, the horizontal line determining unit 102 collects lines having a predetermined slope among the straight lines extracted as shown in FIG. As a result, the horizon determining unit 102 calculates (detects) the vanishing point from these and accumulates (stores) the position information of the vanishing point when the left and right straight lines are respectively a certain number or more. The horizon determining unit 102 may repeat the process in a new image (frame) until cumulative position information of vanishing points calculated in a plurality of frames exceeds a predefined reliability. Finally, the center of the region where the vanishing points exist or the region with the highest reliability is the position of the vanishing point, and the horizon where the vanishing point is located is determined as the horizon (h1).

The vanishing point reliability can be measured as follows. The left-right lane extracted through the Hough transform forms a vanishing point, and the reliability of the vanishing point can be measured by Equation (1) below in a region where the number of vanishing points calculated in a plurality of input images becomes maximum.

The reliability of the vanishing point increases as the value of Equation (1) becomes larger, that is, the more vanishing points become in the first region (A region) of FIG. Here, the first area (area A) is an area defined by a predetermined size (for example, 20x20 pixels), and the vanishing points calculated from a plurality of images are areas denser than a specific reference (first reference). On the other hand, the second area (area B) is defined as a size (for example, 40x40 pixels) including the first area (area A) with the same center position as the first area (area A) The vanishing points calculated from the image are regions denser than the second reference. At this time, if more vanishing points exist in the first region (region A), the vanishing point reliability value is calculated to be larger. In other words, when a vanishing point is detected and collected in a plurality of images, it is judged that the vanishing points are reliable if they are concentrated at one point (area) as much as possible.

The bonnet area determination unit 103 determines a bonnet or a hood in the image of the front of the vehicle. In one embodiment of the bonnet area determination, the bonnet area determination unit 103 binarizes the vehicle front image as shown in FIG. 5, accumulates a predetermined number of frames or more, calculates a horizontal axis projection histogram from the accumulated image , The lower end of the position where the calculated histogram appears above a certain threshold value can be determined as a bonnet region.

If a false bonnet area is detected due to a noise component, the bonnet area determination unit 103 repeatedly detects the bonnet area until a reliable bonnet area is detected in a new input image.

The determination of the bonnet area can be expressed by the following equation (2).

Here, h2 (i) denotes the bonnet area h2 detected through the projection histogram in the i-th frame, and n denotes the number of repeated frames for determining the bonnet area. That is, if the bonnet area h2 detected at a predetermined reference or more in a continuously incoming input image (n frames) is constantly detected, the bonnet area h2 can be determined to be correct, and the final h2 is detected for n It is determined as an average of the bonnet area.

The lane width estimation unit 104 determines a lane width through the detected lane after detecting a lane on an image taken in front of the vehicle. At this time, the lane width estimating unit 104 may detect a lane using a RANSAC algorithm (Random Absolute Consensus).

The RANSAC algorithm is a method for effectively removing outliers from the model fitting. For each of the extracted left and right lane candidate points, a lane hyperbolar equation is used as a fitting model, and an inlier ). At this time, the inlier threshold used in the RANSAC algorithm may be 20 cm, which is a width of one lane (the matrix PT must be converted into the distance in the image coordinates), and the number of repetitions may be preset have. For example, the number of iterations can be set to 100 in consideration of the system load and the like. The following is the process of selecting an inlier corresponding to a valid lane candidate point with the RANSAC algorithm.

1) Randomly select three different points in the extracted left lane candidate point.

2) Find the left lane equation coefficient (k, bL, c) with the least squares error as the selected lane candidate point.

3) From the calculated lane equations, select points whose distance difference of the u coordinate is less than a certain distance as an inlier.

4) If the number of inliers is larger than the previous result, store the inliers and repeat steps 1) to 3). If the processes 1) to 3) are repeated a predetermined number of times, or when the number of inliers becomes equal to the whole of the candidate points, the repetition is stopped and the selected inliers are selected as the valid lane candidate points up to that point.

5) Repeat steps 1) to 4) for the extracted right lane candidate point to select a valid lane candidate point.

Referring to FIG. 6, the lane width estimator 104 detects lane pixels in the input image, detects a width corresponding to each Y axis, detects inclination along the Y axis by the RANSAC algorithm, The lane width is determined based on the ratio. The estimated lane width can be used for estimating the lane when the lane detection fails at some point.

7 is a flow chart illustrating a calibration method in accordance with one embodiment of the present disclosure.

The calibration method may be performed by the calibration apparatus described in FIGS. 1 to 6. FIG. The calibration apparatus can determine the " image processing area " for detecting the vehicle and estimating the inter-vehicle distance in the inter-vehicle distance measuring apparatus. That is, a vision processing area (area for estimating lane recognition, vehicle detection, and inter-vehicle distance) that varies depending on the camera installation angle or camera angle of view can be extracted from the black box or camera image. The calibration apparatus 100 may be included in a vehicle apparatus such as a navigation system or a black box, or may operate in conjunction with the vehicle apparatus.

As described above, the camera calibration means a process of defining a non-processing region for lane recognition, vehicle detection and / or inter-vehicle distance estimation, and the accuracy of lane recognition, vehicle detection, and inter-vehicle distance estimation depends on the accuracy of the camera calibration Is greatly influenced.

In one embodiment, the calibration device may receive an image of the front of the vehicle (S110).

Wherein the calibration device detects an edge component in an image taken in front of the vehicle, extracts a straight line by hough-transforming the detected edge component, determines a vanishing point based on the extracted straight line, A step of determining a horizon through a location of the vanishing point may be performed (S120). For example, the step of determining the horizon may include detecting an edge component by binarizing a canny edge of the image photographed in front of the vehicle.

The step of determining the horizon may include accumulating position information of the vanishing points extracted from the plurality of images and determining the center position of the region where the position information of the accumulated vanishing points is more than the defined reliability as the vanishing point have.

At this time, the reliability is calculated by the following equation,

Here, the first area (area A) is an area defined by a predetermined size (for example, 20x20 pixels), and the vanishing points calculated from a plurality of images are areas denser than a specific reference (first reference). On the other hand, the second area (area B) is defined as a size (for example, 40x40 pixels) including the first area (area A) with the same center position as the first area (area A) The vanishing points calculated from the image are regions denser than the second reference. At this time, if more vanishing points exist in the first region (region A), the vanishing point reliability value is calculated to be larger. In other words, when a vanishing point is detected and collected in a plurality of images, it is judged that the vanishing points are reliable if they are concentrated at one point (area) as much as possible.

The calibration apparatus binarizes an image photographed in front of the vehicle and accumulates a predetermined number of frames or more, calculates a horizontal axis projection histogram from the accumulated image, and outputs the horizontal axis projection histogram to a vehicle bonnet bonnet region may be determined (S130). Wherein determining the bonnet area of the vehicle comprises determining the top of the vehicle bonnet area using the following equation:

Here, h2 (i) denotes the bonnet area h2 detected through the projection histogram in the i-th frame, and n denotes the number of repeated frames for determining the bonnet area. That is, if the bonnet area h2 detected at a predetermined reference or more in a continuously incoming input image (n frames) is constantly detected, the bonnet area h2 can be determined to be correct, and the final h2 is detected for n It is determined as an average of the bonnet area.

The calibration apparatus may perform a step of determining a lane width through the detected lane after detecting a lane on an image of the front of the vehicle (S140). For example, the calibration device may detect a lane by a Random Sample Consensus (RANSAC) algorithm on an image of the front of the vehicle.

Embodiments of the present invention include a computer-readable medium having program instructions for performing various computer-implemented operations. This medium records a program for executing the above-described lane detection method. The medium may include program instructions, data files, data structures, etc., alone or in combination. Examples of such media include magnetic media such as hard disks, floppy disks and magnetic tape, optical recording media such as CD and DVD, programmed instructions such as floptical disk and magneto-optical media, ROM, RAM, And a hardware device configured to store and execute the program. Or such medium may be a transmission medium, such as optical or metal lines, waveguides, etc., including a carrier wave that transmits a signal specifying a program command, data structure, or the like. Examples of program instructions include machine language code such as those produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter or the like.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed embodiments, but, on the contrary, It will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims.

100: Calibration device
101:
102: Horizon Line Decision Unit
103: Bonnet area determination unit
104: Lane width estimation unit

Claims (10)

An image receiving unit for receiving an image of the front of the vehicle so that an image processing region can be defined through calibration;
A method of detecting an edge component in an image taken in front of the vehicle, extracting a straight line by houghing the detected edge component, determining a vanishing point based on the extracted straight line, A horizon determining part determining the horizon through;
The method includes the steps of: binarizing an image photographed in front of the vehicle to accumulate a predetermined number of frames or more; calculating a horizontal axis projection histogram from the accumulated image; and determining a bonnet area through a position where the calculated histogram is above a threshold value A bonnet area determination unit for determining the bonnet area;
And a lane width estimator for detecting a lane on an image taken in front of the vehicle and then determining a lane width through the detected lane,
Wherein the image processing area comprises:
Wherein at least one of the horizon line, the bonnet area, and the lane width is determined through at least one of the horizon line determination unit, the bonnet area determination unit, and the lane width estimation unit,
Wherein the horizon-
Accumulating position information of vanishing points extracted from a plurality of images,
The center position of the region where the accumulated position information of the vanishing points is greater than the predetermined reliability is determined as the vanishing point,
The reliability is calculated by the following equation,

Here, the first region may be a region defined by a predetermined size in which vanishing points calculated from a plurality of images exist above a first reference, the second region has the same center position as the first region, Wherein the region is defined as a region including a region, and the vanishing points calculated in the plurality of images are regions existing above a second reference. The method according to claim 1,
Wherein the horizon line determination unit detects an edge component by binarizing an image of the front of the vehicle taken as a canny edge. delete delete An image receiving unit for receiving an image of the front of the vehicle so that an image processing region can be defined through calibration;
A method of detecting an edge component in an image taken in front of the vehicle, extracting a straight line by houghing the detected edge component, determining a vanishing point based on the extracted straight line, A horizon determining part determining the horizon through;
The method includes the steps of: binarizing an image photographed in front of the vehicle to accumulate a predetermined number of frames or more; calculating a horizontal axis projection histogram from the accumulated image; and determining a bonnet area through a position where the calculated histogram is greater than or equal to a threshold value A bonnet area determination unit for determining the bonnet area;
And a lane width estimator for detecting a lane on an image taken in front of the vehicle and then determining a lane width through the detected lane,
Wherein the image processing area comprises:
Wherein at least one of the horizon line, the bonnet area, and the lane width is determined through at least one of the horizon line determination unit, the bonnet area determination unit, and the lane width estimation unit,
Wherein the bonnet area determination unit
The upper end of the vehicle bonnet area is determined using the following equation,

Wherein the h2 (i) is a bonnet area detected through a projection histogram in an i-th frame, and n is a number of repeated frames for determining a bonnet area. 6. The method of claim 5,
Wherein the lane width estimator detects a lane by a Random Sample Consensus (RANSAC) algorithm on an image taken in front of the vehicle. A method of determining a vehicle and a lane area on an image of a front of a vehicle taken by a calibration apparatus,
Receiving an image of the front of the vehicle;
A method of detecting an edge component in an image taken in front of the vehicle, extracting a straight line by houghing the detected edge component, determining a vanishing point based on the extracted straight line, Step through the horizon;
The method includes the steps of: binarizing an image photographed in front of the vehicle to accumulate a predetermined number of frames or more; calculating a horizontal axis projection histogram from the accumulated image; and determining a bonnet area through a position where the calculated histogram is above a threshold value Determining;
Determining a lane width through the detected lane after detecting a lane on an image taken in front of the vehicle,
The step of determining the horizon comprises:
Accumulating location information of vanishing points extracted from a plurality of images;
Determining a center position of a region where the positional information of the accumulated vanishing points is more than a predetermined reliability as a vanishing point,
The reliability is calculated by the following equation,

Here, the first region may be a region defined by a predetermined size in which vanishing points calculated from a plurality of images exist above a first reference, the second region has the same center position as the first region, Wherein the image is a region defined by a size including a region, and the vanishing points calculated in the plurality of images are regions existing above a second reference. 8. The method of claim 7,
The step of determining the horizon comprises:
And binarizing canny edges of the image photographed in front of the vehicle to detect edge components. delete A method of determining a vehicle and a lane area on an image of a front of a vehicle taken by a calibration apparatus,
Receiving an image of the front of the vehicle;
A method of detecting an edge component in an image taken in front of the vehicle, extracting a straight line by houghing the detected edge component, determining a vanishing point based on the extracted straight line, Step through the horizon;
The method includes the steps of: binarizing an image photographed in front of the vehicle to accumulate a predetermined number of frames or more; calculating a horizontal axis projection histogram from the accumulated image; and determining a bonnet area through a position where the calculated histogram is greater than or equal to a threshold value Determining;
Determining a lane width through the detected lane after detecting a lane on an image taken in front of the vehicle,
Wherein determining the bonnet area of the vehicle comprises:
Determining a top end of the vehicle bonnet area using the following equation,

Where h2 (i) is the bonnet area detected through the projection histogram in the i-th frame, and n is the number of repeated frames for determining the bonnet area.

Images