JPH08320999A - Vehicle recognizing device - Google Patents

Abstract

PURPOSE: To provide a vehicle recognition device capable of recognizing a vehicle at the high accuracy of a detected position. CONSTITUTION: Picture processing is executed while noticing an interrupted point of a vertical edge detected on the side face of a vehicle in the lowermost end part of the vehicle and a symmetry property, a feature of the vehicle, to detect a vertical edge in a picture, an interrupted point of the vertical edge is detected as a vehicle lowermost end candidate by a vehicle lowermost end candidate detecting means 6, and when the vehicle lowermost end candidate of the left side e.g. has almost the same height as that of the right side, a vehicle judging means 7 judges that a detected object is a vehicle.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an apparatus for recognizing the presence of a vehicle by image processing, and an apparatus for calculating traveling information such as an inter-vehicle distance using the recognition result.

[0002]

2. Description of the Related Art As a conventional inter-vehicle distance recognizing device, for example, there is one disclosed in Japanese Patent Laid-Open No. 3-273500. In this device, first, the vehicle traveling area (the traveling lane of the vehicle) is recognized from the front image obtained by the image pickup means, and the vehicle on the image is composed of lateral edge (horizontal edge) components. Then, the vehicle is detected by searching for a lateral edge in the traveling area. Further, if the mounting position of the camera is known, the inter-vehicle distance can be calculated from the lateral edge position of the vehicle lowermost end in the image, and the degree of approach to the preceding vehicle is determined based on the inter-vehicle distance and the own vehicle speed. There is described a device that issues an alarm and warns the driver when it is determined that the vehicle is in an excessively close state.

[0003]

However, such a conventional vehicle recognition device has the following problems. First, as a first problem, there are many false detections because many lateral edges are detected on the road surface in addition to the vehicle. FIG. 28 is a diagram showing an example of a scene in which erroneous detection is likely to occur. (A) is a shadow 111 of an overpass, (b) is a white line 112 for checking an inter-vehicle distance,
(C) shows the road joint 113. (A) to (c) in the figure
In each of the scenes, there is a density difference such as dark → bright → dark or bright → dark → bright, which is judged as an edge, and the targets are distributed almost perpendicular to the traveling direction of the road. Is a target that is detected as a horizontal edge.
Next, the second problem is that the lateral edge detected at the bottom of the screen is not necessarily the bottom end of the vehicle. For example, the edge image of the scene as shown in FIG. 29A is as shown in FIG. 29B, but the edge appearing at the bottom of the screen is the road joint 121 and not the bottom end portion of the vehicle. Therefore, there is a problem that even if the distance from the lateral edge 121 to the preceding vehicle is calculated, there is a large difference from the actual distance.

The present invention has been made to solve the problems of the prior art as described above, and an object of the present invention is to provide a vehicle recognition device capable of performing vehicle recognition with high detection position accuracy.

[0005]

In order to achieve the above object, the present invention is constructed as described in the claims. FIG. 1 is a diagram corresponding to the claims of the present invention. First, FIG. 1 (a) corresponds to claim 1, and 1 is an image pickup means which is mounted on a vehicle and photographs the front or rear of its route to generate corresponding original image data.
Is a differential image generating means for generating differential image data by differentiating the generated original image data, 3 is a vehicle traveling path detecting means for detecting a vehicle traveling path based on the generated differential image data, Reference numeral 4 is a vehicle candidate detecting means for detecting a vehicle candidate based on the differential image data and original image data on the vehicle running road, and 5 is a vehicle side surface candidate for detecting a vehicle side surface candidate based on a vertical edge image in the vicinity of the vehicle candidate. Detecting means, 6 is a vehicle bottom edge candidate detecting means for detecting a vehicle bottom edge candidate based on the vehicle side surface candidate, and 7 is a vehicle determining means for making a final vehicle determination based on the vehicle bottom edge candidate. . In addition, claims 2 to 9
Is a configuration example of the vehicle side surface candidate detection means 5 in claim 1, and claims 10 to 12 are described.
Describes a configuration example of the vehicle bottom end candidate detection means 6, and claims 14 to 16 describe a configuration example of the vehicle determination means 7.

The invention described in claim 17 is based on FIG.
As shown in FIG. 1B, in the configuration of FIG. 1A, when the vehicle determination unit 7 confirms the presence of the vehicle, the vehicle bottom end coordinates are calculated based on the obtained left and right vehicle bottom end candidates. A lowermost end coordinate calculating means 8; and a distance calculating means 9 for calculating a distance to a preceding vehicle based on the lowermost end coordinate,
Is added. The eighteenth aspect describes a configuration example of the vehicle bottom edge coordinate calculating means 8.

The invention according to claim 19 is the same as in FIG.
(A) shows another configuration of the vehicle determining means 7, wherein the vehicle side determining means 7 is a vehicle side surface candidate in which a density value within a predetermined range continues in the vertical direction of the screen, In addition, the vehicle determination is performed when a plurality of vehicle side surface candidates satisfying the above condition are continuously present in the lateral direction. According to a twentieth aspect of the invention, in the invention of the nineteenth aspect, the vehicle side surface candidate detection means 5
21 is a configuration example of the above.
4 describes a configuration example of the vertical edge strength evaluation means according to claim 20, and claims 25 and 26.
Is a configuration example of the vehicle determination means 7.

[0008]

The invention described in claims 1 to 16 was made by paying attention to the point that the vertical edge detected on the side surface of the vehicle is interrupted at the lowermost end of the vehicle and the symmetry which is a characteristic of the vehicle. Is, to detect the vertical edge in the image by performing image processing, the point where the vertical edge is interrupted is the vehicle bottom end candidate, for example, when the left and right vehicle bottom end candidates are approximately the same height, It is configured to determine that the detected target is a vehicle. Therefore, it is possible to perform vehicle recognition with high detection position accuracy, as compared with the conventional vehicle determination based on only the lateral edge.

Further, in the inventions of claims 17 and 18, the vehicle bottom edge coordinates are calculated based on the vehicle bottom edge candidates obtained in the invention of claim 1, and the preceding vehicle is based on the bottom edge coordinates. It is configured to calculate the distance to. As described above, in the configuration of claim 1, since the lowermost end portion (vehicle lowermost end candidate) of the vehicle can be accurately detected, the distance to the preceding vehicle is calculated by using the distance calculation. Can be calculated accurately.

Further, the inventions according to claims 19 to 26 determine the vehicle from the vehicle side surface candidates without determining the vehicle bottom edge candidate. That is, when the vehicle side surface candidate is one in which the density value in the predetermined range continues in the vertical direction of the screen and there are a plurality of vehicle side surface candidates satisfying the condition continuously in the horizontal direction, the detected target object is detected. Is configured to be a vehicle. Therefore, even with this configuration, it is possible to perform vehicle recognition with high detection position accuracy.

[0011]

2 and 3 are diagrams of a first embodiment of the present invention. FIG. 2 is a block diagram when the vehicle recognition device according to the present invention is implemented as a vehicle approach warning device, and FIG. The conceptual diagram when mounted in a vehicle is shown. 2 and 3, reference numeral 211 denotes a laser radar range finder having a plurality of heads (for example, three heads), 212 is a video camera for capturing a front image, 213 is a vehicle speed sensor, 214 is an arithmetic unit composed of a computer or the like, Reference numeral 215 is a display / warning device such as a buzzer, a chime or a liquid crystal display. The arithmetic unit 2
Reference numeral 14 is provided with a storage device and stores original image data, differential image data, and the like in the process of calculation.

The laser radar range finder 211 emits laser light in front of the vehicle and measures the distance to the target based on the reflected light from the target. This distance measurement is performed using, for example, three heads in three directions in front of the vehicle (for example, diagonally left front, front, diagonally front right). Arithmetic unit 214
Determines the degree of approach to the target based on the vehicle speed from the vehicle speed sensor 213 and the above measurement result, and if it is determined to be excessively close, issues a warning to the driver using the display / warning device 215. . The display / warning device 215 can also display running information such as the measured inter-vehicle distance.

In order to improve the alarm accuracy in the above-mentioned device, it is necessary to clarify the target of the laser radar range finder 211, that is, the object of detection of the distance value, and to make the alarm judgment surely by the distance to the preceding vehicle. is there.
For example, it is necessary to determine which of the detection distances in front of the vehicle detected by the three heads is the value from the preceding vehicle. Therefore, in this embodiment, the image of the front of the own vehicle captured by the video camera 212 is image-processed to surely recognize the preceding vehicle and detect the distance value to the preceding vehicle.

The contents of vehicle recognition will be described below.
FIG. 4 is a flowchart showing the overall flow of the vehicle recognition calculation in this embodiment. In FIG. 4, after the process is started, initialization is performed, and then a front image is captured in step 221. Next, in step 222, a horizontal edge image is created for the obtained original image using the Sobel filter shown in FIG. In addition, in FIG. 5, (a) shows an operator for a horizontal edge, and (b) shows an operator for a vertical edge. Next, in step 223, lane marker detection is performed based on the lateral edge image. here,
Let us consider defining a lane marker model in the image plane and curve fitting the white line candidate points obtained from the lateral edge image and the model.

FIG. 6 is a diagram showing the correspondence between the coordinate systems of the three-dimensional space and the image plane.
It defines to describe the image plane as an image coordinate system. The camera coordinate system and the image coordinate system are the following (Equation 1).
Corresponds with an expression.

[0016]

[Equation 1]

However, x, y: image coordinate system X, Y, Z: camera coordinate system f: focal length of video camera FIG. 7 is a diagram showing a lane marker model described in the camera coordinate system. Yes, (a) is a plan view, (b)
Is a side view. The relationship of FIG. 7 is expressed by the following (Equation 2) formula. X = BZ ² + CZ + A−iE Y = DZ−H ₀ (Equation 2) where A: distance from origin 0 (video camera position) to the left end of the driving lane B: parameter corresponding to the curvature of the driving lane C: traveling Parameter corresponding to yaw angle β with respect to direction D: Parameter corresponding to pitch angle α E: Width of driving lane H ₀ : Height from the ground surface of video camera If converted into the coordinate system, the lane marker model in the image coordinate system is expressed by the following (Equation 3).

[0018]

(Equation 3)

However, a, b, c, d and e are parameters in the image coordinate system corresponding to A, B, C, D and E in the equation (2), respectively. Also, i is an integer,
The value is the lane marker No. in FIG. It corresponds to.
Note that FIG. 8 is an image in front of the vehicle and is shown in the image coordinate system. According to the equation (3), a plurality of lane markers appearing in the image are represented by the five parameters a to e.

FIG. 9 is a flowchart showing the contents of the lane marker detection described in step 223 of FIG. 4, and shows the detailed contents in the range of (A) to (A ′) of FIG. In FIG. 9, first, in step 271, in the newly input lateral edge image, an edge point near the lane marker model detected in the previous frame (the input image one time before) is searched, and the edge point is acquired. Output as white line candidate points. Next, in step 272, the parameters ae that minimize the error are determined for the detected white line candidate point group using the least square method. By this processing, the lane marker is updated with respect to the previous frame. Next, in step 273, the obtained i = 0 and i = 1 lane marker models are recognized as the own vehicle traveling lane, and the process ends.

Through the above processing, step 224 in FIG.
Then, vehicle detection processing is performed. FIG. 10 shows (B) to FIG.
It is a flowchart which shows the detailed content of the range of (B '). In FIG. 10, first, at step 281, a vehicle candidate is detected in the own vehicle traveling lane defined by the above-described processing. Generally, a shadow appears in the lower part of the preceding vehicle, and the position of the lateral edge caused by the shadow is detected here. The following (1) and (2) can be cited as features of the vehicle lower lateral edge (around the lower shadow of the vehicle). (1) When searching on the driving lane from the bottom of the screen, the density value changes in the order of light (road) → dark (vehicle shadow), so the lateral edge strength also reflects that order (the above). When the edge detection operator of FIG. 5 is used, a negative lateral edge, that is, an edge whose density changes from dark to light is detected). (2) In a distant place, the shadow may not be visible, and it may not always be the bottom end of the vehicle. From the above characteristics, first, the target is a negative horizontal edge point sequence,
And the y coordinate satisfying the condition of the following equation (4) is detected as the first vehicle lower end candidate y _s0 .

[0022]

[Equation 4]

Here, Ne: the number of negative lateral edge points having an absolute intensity equal to or higher than a predetermined value in the own lane area at a certain y coordinate Nw: the width (the number of pixels) of the lane area R: Ne / Nw; Function R _{t of} y: Threshold value for defining lower limit value of R Note that the above Nw is represented by a difference between when i is 0 and when it is 1 in the formula (3). That is, x ₁
−x ₀ = e (y−d), and Nw is a function of y. Further, the R _t, for example, a road width against light vehicle in the vehicle width 1.4m if 3.5m assumes highways, R _t
= 40 (%).

Next, in step 282, a second vehicle density value using the original image density value is obtained for the obtained first vehicle candidate.
Vehicle candidate determination is performed. FIG. 11 is a diagram for explaining the vehicle candidate determination process using the original image density value. In the shadow appearing in the lower part of the vehicle, there is generally a density gradation as shown in FIG. According to the figure, the original image density is minimum at the y-coordinate having the minimum value of the lateral edge strength, and therefore the y-coordinate at which the minimum value is detected is the most suitable as the vehicle bottom edge candidate. Here, the reason why it is described as “candidate” is that it is not decided as the vehicle bottom end due to the characteristic (2) of the vehicle shadow.

Next, at step 283, the function R
The y coordinate that minimizes (y) is detected, and the detected position is set as the second vehicle candidate. Here, the detected y coordinate is _defined as y _s1 . In addition, the white line for displaying the inter-vehicle distance, the road joint, etc., which are noise components in vehicle detection, are the same as
It can be easily removed by referring to the density value of the original image defined by the coordinates and the vehicle traveling lane. Next, in step 284, the vertical edge detection window is set based on y _s1 . Generally, the vertical edge that appears on the vehicle appears most strongly on the side. Further, in the vehicle vertical direction, the strongest and stable vertical edge appears near the vehicle tire. The reason for this is that the background is always the road surface. Therefore, it is efficient to set the vertical edge detection window at a height corresponding to the tire above the y-coordinate y _s1 of the vehicle lower end candidate.

The window height calculation method will be described below. FIG. 12 is a diagram for explaining the window height calculation method. Let us consider the model of FIG. 12 with the tire equivalent height of the preceding vehicle being h _t . _ht is described by the following equation (5) in the camera coordinate system (21 in FIG. 12).
01). Y = −H ₀ + h _t (Equation 5) Therefore, the relation between y _s1 and the window upper end coordinate y _t is expressed by the following (Equation 6) by using the perspective transformation represented by the above (Equation 1). Represented.

[0027]

(Equation 6)

On the other hand, the difference between the x-coordinates of the left and right lane marker models at y _s1 has the relationship of the following (Equation 7).

[0029]

(Equation 7)

When Z is eliminated from the above equations (6) and (7), the following equation (8) is obtained.

[0031]

(Equation 8)

The points x ₀ (y _s1 ), x on the lane marker model at y _t and y = y _s1 obtained here are obtained.
_The rectangle defined by ₁ (y _s1 ) is set as the vertical edge detection window. FIG. 13 is a diagram for explaining a method of setting the above vertical edge detection window,
(A) is the entire screen, (b) is the vertical edge detection window 2
111 is shown. Then, vertical edge detection is performed on the original image inside the window 2111 by the Sobel filter shown in FIG. The coordinates in the window are described in the local coordinate system (O'-x'-y '). The image coordinate system and the local coordinate system have a relationship represented by the following (Equation 9). y−y ′ = y _t , x−x ′ = x ₀ (y _s1 ) ... ( _Equation 9) Next, in step 285, for the vertical edge image of the vehicle lower portion brought by the set vertical edge detection window, , Vehicle side candidate detection processing is performed.

14A and 14B are views for explaining a first embodiment of vehicle side surface candidate detection. FIG. 14A is the vertical edge detection window described above, and FIG. 14B is the detection performed using FIG. FIG. 7 is a diagram showing a vertical edge image in which the vertical axis represents the edge strength average value I_mean on each x ′ coordinate and the horizontal axis represents x ′. The vehicle is a symmetrical object, and the background of the vehicle is almost the road surface at the window setting position. Therefore, the characteristic shown in FIG. → It indicates that a positive vertical edge reflecting the contrast of the vehicle tire density (dark) is detected, and a negative vertical edge of (dark) → (light) is detected on the right side surface of the vehicle (Fig. 14). 2121). In this average edge strength distribution I_mean, the side surface candidate can be detected by performing the processing shown in the following (Equation 10). i = 0, j = 0 for x '= 0 to x' = xm if I_mean (x ')> I_th then xl (i) = x', i = i + 1 for x '= xm to x' = xw if I_mean ( x ′) <− I_th then xr (j) = x ′, J = J + 1 (Equation 10) where xm: x coordinate of midpoint of window xw: window width i, j: threshold value ± I_th is satisfied. Counter for x-coordinates The counters i and j determine that the vehicle side candidate is not a vehicle when one or both of the counter values are 0, and the process branches to 286 in FIG. If both the vehicle side surface candidates xl (2122) and xr (2123) are detected at least on the left and right, the vehicle side surface candidate is output to the bottom edge candidate detection process of step 287 of FIG. The vehicle candidates may be output as a vehicle candidate group as shown in FIG. 14, but the average values xGL, xGR of xl (i), xr (j) may be output.
May be output. Further, in the present embodiment, the average value is calculated while maintaining the edge directionality, but the average edge strength distribution may be created by taking the absolute value of the edge strength.

Next, FIG. 15 shows a second side face candidate detection.
FIG. 6 is a diagram for explaining an example of FIG. Here, 2 in the figure
This is an embodiment of vehicle side surface candidate detection using the distribution of the sum S (x ′) of the vertical edge ternary images obtained by performing the ternary processing shown in 131 on the vertical edge images. The ternarization process is represented by the following (Formula 11). if I_ν (x ', y')> I_th then I_ν = 1 else if I_ν (x ', y') <I_th then I_ν = -1 otherwise: I_ν = 0 (Equation 11) where I_ν: vertical edge strength I_th : Threshold for ternarization In this example, the value of S is the x coordinate x of the window center.
The number of x coordinates satisfying the threshold value S _t on the left side of m and the threshold value −S _t on the right side is detected by a method similar to the expression (10).

FIG. 16 shows left and right independent threshold values I_L,
It is a figure for demonstrating the method of setting I_R. In the above-described ternarization process, the threshold value I_th is uniquely determined, but as shown in FIG. 16, a frequency distribution of edge strength is created independently from the window center x coordinate (21).
41), based on the frequency distribution, the threshold value may be automatically set as I_L and I_R independently for each frame by using a mode method which is a general method of binarization processing. . However, L shows the left side and R shows the right side.

Next, the bottom edge candidate detection processing shown in step 287 of FIG. 10 will be described. FIG. 17 is a diagram for explaining the bottom edge candidate detection processing, in which (a) shows a screen and (b) shows a window. As shown in FIG.
A vertical edge detection window 2151 having a size of ± dx in the horizontal direction and ± dy in the vertical direction is set on the basis of the average value xGL of the x coordinates of the obtained vehicle side surface candidates and the vehicle bottom end candidate coordinate y _s1 . (O "-x" -set in the window
y ”) coordinate system and the image coordinate system have the following (equation 1)
It is expressed by the formula 2).

Y−y ″ = y _s1 −dy xx−x ″ = xGk−dx (k = L, R) (Equation 12) Next, FIG. 18 illustrates the vehicle bottom end candidate detection process. It is a figure. As shown in FIG. 18, in the window, from the edge intensity distribution at the x-coordinates xl and xr of the vehicle side surface candidate, in the left-side window, y ″ coordinate yl (i), which becomes the threshold value I_th or less for the first time, right-side window. In the first step, the y ″ coordinate yr (j) that becomes the threshold value I_th or more is detected. These yl (i) and yr (j) are the vehicle bottom end candidates.

Next, the vehicle determination processing shown in step 288 of FIG. 10 will be described. Since the symmetry is one of the characteristics of the vehicle, the following vehicle determination conditions can be considered. (A) There is a predetermined number or more that satisfies the difference between yl and yr that is equal to or smaller than the predetermined value. (B) The difference between the average value of yl and the average value of yr is less than or equal to a predetermined value. That is, when the heights (y coordinate positions) of the lowermost end candidate of the left window and the lowermost end candidate of the right window are substantially the same, it can be determined that the detected target is a vehicle. Either of the above two determination conditions may be used. Therefore, if either one of the above two determination conditions is satisfied, it is determined that the vehicle is detected, and the process proceeds to the next.

If a constraint condition is further added, the vehicle bottom end candidate coordinate y _s1 and the vehicle side _face candidate coordinate x
It may be defined by the following (Equation 13) according to l and xr.

[0040]

(Equation 13)

The numerical value of the equation (13) reflects the road width of the expressway of 3.5 m, the width of the light vehicle of 1.4 m and the width of the large truck of 2.5 m. In addition, since the vehicle is within the lane, the condition of the following (Formula 14) is added. [X ₀ (y _s1 ) <xl] & [x ₁ (y _s1 )> xr] (Equation 14) The above (Equation 13) and (Equation 14) are the vehicle side surface candidate x
This means that the average value of each of the coordinate groups xl (i) and xr (j) is within the lane, and the difference between the average values is within a predetermined range.

Next, at step 289 in FIG. 10, the vehicle lower end coordinate group yl (i), y obtained at the time of the above vehicle determination.
The average value of r (j) is calculated and the calculated result is recognized as y _s as the vehicle bottom coordinate y _s . Once the vehicle bottom edge coordinate y _s is established, the approximate distance to the preceding vehicle is calculated by the following (Equation 15) formula.

[0043]

(Equation 15)

However, F: constant corresponding to focal length H ₀ : camera mounting height In the above description, the vehicle determination is performed from the vehicle candidate detection for each frame, but in the following example, An example of vehicle detection using a tracking window for tracking the vehicle after detecting the vehicle once will be described. FIG. 19 is a flowchart of vehicle detection processing using the above tracking window. In FIG. 19, the flag of step 2171 is a flag indicating whether or not the vehicle was detected in the previous frame,
When the value is "1", the process proceeds to E as the previous detection. Further, steps 2172 to 2179 are the same as those in FIG.
When it is determined that the vehicle is detected in step 2179, the configuration is the same as that in step 2181.
Then, the detected vehicle side x-coordinate and vehicle lower-end coordinate are stored in xl_old, xr_old, and ys_old, respectively, and set as reference coordinates for the next tracking window (described later).

Next, a specific tracking process will be described. FIG. 20 is a flowchart showing a specific tracking process. 21 and 22 are diagrams for explaining the flow of FIG. 20. In FIG. 20, first, in step 2181, a tracking window is set. As shown at 2191 in FIG. 21, the vehicle lower end coordinates (xl_old, ys_old) in the front frame are independently set to the left and right,
Centered on (xr_old, ys_old) ± dx, ±
A vertical edge detection window is set with a width of dy, and this is used as a tracking window.

Next, at step 2182, the vehicle side surface candidate x-coordinates are detected as xl_new (i) and xr_new (j). The method may be any of the above-mentioned methods, but here, the method using the average edge strength shown in 2192 in FIG. 21 was used. Next, in step 2183, the center of gravity of the obtained side face x-coordinate is xGL_new,
xGR_new, and in step 2184, FIG.
2193, the left and right vehicle lowermost ends y-coordinates on the side x-coordinates are detected as y-coordinates yl_new (i) and yr_new which are equal to or less than a predetermined edge strength I_thy.

The y-coordinate thus obtained is calculated in step 2185 as the absolute value of the difference between them, and it is confirmed in step 2186 that it is within a predetermined range. In step 2186, Dy is the threshold value dy.
If it is less than or equal to _th (no in 2186), it is determined that the bottom ends of the vehicle side surfaces are substantially the same (2 in FIG.
201), jump to (B ') and continue tracking. If it is determined in step 2186 that it is out of the predetermined range (2186: yes), it is considered that, for example, a vehicle interrupt has occurred (2 in FIG. 22).
202), and reset the flag in step 2187, jump to (D), and shift to a routine starting from vehicle candidate detection.

Next, FIG. 23 is a flow chart showing the detailed contents of the beam selection and the approach degree judgment shown in the step of FIG. 4, and shows portions (c) to (c ′) of FIG. In step 227 of FIG. 4, the distance data of the radar is selected (beam selection) based on the rough distance calculation result, and the radar distance value obtained in step 225 of FIG. 4 and the radar distance value obtained in step 226 of FIG. Based on vehicle speed
Determine the degree of approach. The details are shown in FIG. Figure 2
3, first, in step 2211, absolute values ΔLL to ΔLR of the difference between the radar measurement distance and the rough distance are obtained. Subsequently, in step 2212, the ΔLL to ΔLR are set.
The minimum value of the above is obtained, and the radar distance value that brings the minimum value is adopted as the distance to the preceding vehicle.

FIG. 24 is a diagram for explaining the contents of the beam selection processing (step 227 in FIG. 4) described above. Three distances (obliquely forward left, front,
It indicates that the value having the smallest difference from the approximate distance obtained by the image processing is adopted as the distance to the preceding vehicle. Next, step 221 in FIG.
In 3, a judgment is made as to whether or not the vehicle is excessively approaching based on the distance to the preceding vehicle, the own vehicle speed, and the relative speed obtained as described above.
At 214, the driver is alerted by alerting him.

In the above description, an example in which the vehicle recognition device of the present invention is applied to a vehicle approach warning device (inter-vehicle distance warning device) has been mainly described, but the vehicle recognition device of the present invention is For example, it can be applied to other systems such as an autonomous vehicle that automatically follows a preceding vehicle.

Next, another embodiment of the vehicle detection processing shown in step 224 of FIG. 4 will be described. Figure 25
6 is another flowchart of the vehicle detection process. Figure 25
In, steps 281 to 284 are the same as the steps in FIG. The difference in the present embodiment is that vehicle determination is performed by a histogram in step 285 '. That is, in step 285 ',
For the vertical edge image of the vehicle bottom, which is provided by the vertical edge detection window set in step 284,
Vehicle determination is performed based on the histogram.

A first embodiment of vehicle determination based on the histogram will be described below. After detecting the vertical edge image as described above, the ternarization processing indicated by 2131 in FIG. 15 is applied to the vertical edge image, and the sum S (x ′ ′ of the ternary image of the vertical edge obtained by the conversion is performed. ) Distribution. The above-mentioned ternarization processing is the same as that shown in the above (Formula 11). In this example, the value of S is more than the threshold value St on the left side of the x coordinate xm of the window center,
On the right side, let n
It is detected as l and nr, and if the value is equal to or larger than a predetermined value, it is determined to be a vehicle. In the ternarization process,
Although the threshold value I_th is uniquely set, the threshold value I_th is 2 in FIG.
As shown in 131, the frequency distribution of the edge strength is created independently from the x coordinate of the window center, and based on the frequency distribution, it is automatically calculated by using the mode method which is a general method of binarization processing. Left and right independently for each frame I_th
It may be set as L and I_thR.

Next, a second embodiment of vehicle determination based on a histogram will be described. FIG. 26 is a diagram for explaining the present embodiment, where (a) is a histogram image,
(B) is a run length table. In the first embodiment, the vehicle determination is performed by counting the x-coordinates that satisfy the predetermined value St or more or less continuously, independently on the left and right sides, but in the present embodiment, the vehicle determination is shown in FIG. , A histogram image is created on the memory, and the threshold S
If there is a run length that satisfies t or more or -St or less, that is, if there is a run length that is greater than or equal to the threshold value St on the left side and less than or equal to -St on the right side, the vehicle is determined.

Next, a third embodiment of vehicle determination based on a histogram will be described. In the second embodiment, the vehicle determination is performed when there is one or more run lengths on the left and right sides of the window. However, in the present embodiment, not only is the vehicle determined to be the presence of the run length, The difference is output as the accuracy (probability) that the detection target is the vehicle. That is, since the vehicle is a symmetrical target object, ideally the difference between the left and right run lengths is zero. Therefore, the closer the difference between the left and right run lengths is to 0, the greater the probability that the detection target is the vehicle.

Next, a fourth embodiment of vehicle determination based on a histogram will be described. FIG. 27 is a diagram showing an image and edge strength for explaining the fourth embodiment. On a curved road, a vertical edge may appear due to the contrast between the white line and the road surface, as shown in FIG. In this case (xs, yt)-(x_cut, ysl)
As a noise area, by adding a means for cutting the histogram caused by the white line, it is possible to extract only the histogram of the vehicle side surface candidates.

When the vehicle is determined by any of the first to fourth embodiments, the center of gravity of xl and xr is calculated to define the x coordinate of the side surface of the vehicle, and the ysl
And the vehicle side candidate coordinates xl and xr,
By adding the two conditions by the equation 3) and the equation (14),
The possibility that the detection target is a vehicle is further increased. Further, the process after the vehicle determination is performed, for example, the alarm process due to the excessive approach is the same as above.

[0057]

As described above, according to the inventions of claims 1 to 16, image processing is performed to detect a vertical edge in an image, and a point where the vertical edge is interrupted is set as a vehicle bottom edge candidate. For example, when the left and right vehicle bottom end candidates have almost the same height, the detection target is determined to be a vehicle, so that the vehicle is determined only from the lateral edge as in the conventional case. As compared with the vehicle, it is possible to obtain the effect that the vehicle can be recognized with high detection position accuracy. Further, the invention according to claim 17 and claim 18 calculates the vehicle bottom end coordinates based on the vehicle bottom end candidate obtained in the invention of claim 1 and calculates the distance to the preceding vehicle based on the bottom end coordinates. By configuring to calculate
The effect that the distance to the preceding vehicle can be accurately calculated is obtained. Further, in the invention according to claims 19 to 26, the vehicle side surface candidates are those in which the density value in a predetermined range continues in the vertical direction of the screen, and the vehicle side surface candidates satisfying the condition are in the horizontal direction. In the case where a plurality of consecutive objects exist, it is determined that the detected target is a vehicle, so that the vehicle can be recognized with high detection position accuracy.

[Brief description of drawings]

FIG. 1 is a diagram corresponding to a claim of the present invention.

FIG. 2 is a first embodiment of the present invention, and is a block diagram when the vehicle recognition device according to the present invention is implemented as a vehicle approach warning device.

3 is a first embodiment of the present invention, a conceptual diagram when the device of FIG. 2 is installed in a vehicle.

FIG. 4 is a flowchart showing the overall flow of vehicle recognition calculation in this embodiment.

FIG. 5 is an example of a sobel filter, FIG. 5A is a diagram showing an operator for horizontal edges, and FIG. 5B is a diagram showing operators for vertical edges.

FIG. 6 is a diagram showing correspondence between coordinate systems of a three-dimensional space and an image plane.

7A and 7B are diagrams showing a lane marker model described in a camera coordinate system, where FIG. 7A is a plan view and FIG. 7B is a side view.

FIG. 8 is an image view in front of the vehicle shown in an image coordinate system.

9 is a flowchart showing the contents of lane marker detection described in step 223 of FIG.

FIG. 10 is a flowchart showing detailed contents in a range of (B) to (B ′) in FIG.

FIG. 11 is a diagram for explaining vehicle candidate determination processing using original image density values.

FIG. 12 is a diagram for explaining a window height calculation method.

FIG. 13 is a diagram for explaining a method of setting a vertical edge detection window, in which (a) is a diagram showing the entire screen;
FIG. 9B is a diagram showing a vertical edge detection window 2111.

FIG. 14 is a diagram for explaining a first embodiment of vehicle side surface candidate detection, in which (a) is a vertical edge detection window,
FIG. 6B is a diagram showing the vertical edge image detected using FIG. 5 with the vertical axis representing the edge strength average value I_mean on each x ′ coordinate and the horizontal axis representing x ′.

FIG. 15 is a diagram for explaining a second embodiment of vehicle side surface candidate detection.

FIG. 16 is a diagram for explaining a method of setting left and right independent threshold values I_L and I_R.

FIG. 17 is a diagram for explaining a bottom edge candidate detection process.

FIG. 18 is a diagram for explaining a vehicle bottom end candidate detection process.

FIG. 19 is a flowchart of the entire detection process using a tracking window.

FIG. 20 is a flowchart showing a specific tracking process.

FIG. 21 is a diagram for explaining the flow of FIG. 20.

22 is a diagram for explaining the flow of FIG. 20. FIG.

FIG. 23 is a flowchart showing the detailed contents of beam selection and proximity determination shown in the step of FIG.

FIG. 24 is a diagram for explaining the beam selection process of step 227 of FIG. 4;

FIG. 25 is a flowchart of another embodiment of vehicle detection processing.

FIG. 26 is a diagram for explaining a second embodiment of vehicle determination based on a histogram.

FIG. 27 is a diagram showing an image and edge strength for explaining a fourth embodiment of vehicle determination based on a histogram.

FIG. 28 is a diagram showing an example of a scene in which a false detection is likely to occur in the conventional example.

FIG. 29 is a diagram for explaining that the horizontal edge detected at the bottom of the screen is not necessarily the bottom end of the vehicle.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 ... Imaging means 2 ... Differential image generation means 3 ... Own vehicle running road detection means 4 ... Vehicle candidate detection means 5 ... Vehicle side surface candidate detection means 6 ... Vehicle bottom end candidate detection means 7 ... Vehicle determination means 8 ... Vehicle bottom end coordinates Calculation means 9 ... Distance calculation means 211 ... Laser radar range finder 212 ... Video camera 213 ... Vehicle speed sensor 214 ... Calculation device 215 ... Display / warning device

─────────────────────────────────────────────────── ─── Continuation of the front page (51) Int.Cl. ⁶ Identification code Internal reference number FI technical display location // G05D 1/02 G06F 15/62 380

Claims (26)

[Claims] 1. An image pickup means mounted on a vehicle for photographing the front or rear of a route thereof and generating corresponding original image data; and a differentiation for generating differential image data by differentiating the generated original image data. Image generating means, own vehicle traveling road detecting means for detecting the own vehicle traveling road based on the generated differential image data, and detecting vehicle candidates based on the differential image data and original image data on the own vehicle traveling road Vehicle candidate detecting means, vehicle side candidate detecting means for detecting a vehicle side candidate based on a vertical edge image in the vicinity of the vehicle candidate, and vehicle bottom edge candidate for detecting a vehicle bottom edge candidate based on the vehicle side candidate A vehicle recognition device comprising: a detection unit; and a vehicle determination unit that determines a final vehicle based on the vehicle bottom end candidate. 2. The vehicle side surface candidate detecting means sets a vertical edge detecting window setting means for setting a window whose base is the y coordinate detected as a vehicle candidate, and a vertical edge strength for each x coordinate in the window. The vehicle recognition device according to claim 1, wherein the vehicle recognition device is configured by means for detecting a vehicle side surface candidate by evaluating. 3. The vertical edge detection window setting means,
And a means for calculating the lane width at the y-coordinate detected as a vehicle candidate and a means for calculating the image height corresponding to the tire height of the preceding vehicle at the inter-vehicle distance corresponding to the y-coordinate. The vehicle recognition device according to claim 2, wherein the vehicle recognition device is provided. 4. The vehicle side surface candidate detection means has an average value of vertical edge intensities calculated for each x coordinate in the window set by the vertical edge detection window setting means, the x coordinate being substantially the center of the window. For the left side, a value that reflects the density change from the left of the screen to light → dark exceeds a predetermined value, and for the right side, the value that reflects the density change from the left of the screen to dark → bright and exceeds the predetermined value x
The vehicle recognition device according to claim 2, wherein the vehicle recognition device is configured by means for detecting a coordinate group, and the x-coordinate group is used as a vehicle side surface candidate. 5. The vehicle side surface candidate detecting means has an average absolute value of vertical edge strengths calculated for each x coordinate within a window set by the vertical edge detecting window setting means, being equal to or larger than a predetermined value. The vehicle recognition device according to claim 2, wherein the vehicle recognition device is configured by means for detecting an x-coordinate group that provides a value, and the x-coordinate group is used as a vehicle side surface candidate. 6. The vehicle side surface candidate detecting means includes a number calculating means for calculating the number of pixels having an absolute intensity of a predetermined value or more for each x coordinate within the window set by the vertical edge detecting window setting means. 3. The vehicle recognition according to claim 2, further comprising: means for detecting an x-coordinate group that brings the number to a value greater than or equal to a predetermined value, and the x-coordinate group is used as a vehicle side surface candidate. apparatus. 7. The number calculating means comprises a means for creating a frequency distribution of edge strength in a window and a means for setting a threshold value based on the frequency distribution. The vehicle recognition device according to claim 6. 8. The vehicle side surface candidate detecting means includes a number calculating means for calculating the number of pixels having an absolute intensity of a predetermined value or more for each x coordinate within the window set by the vertical edge detecting window setting means. , The number of x-coordinate groups that produces a value greater than or equal to a predetermined value, and means for specifying only the groups that appear consecutively in the x-direction within the x-coordinate groups. 3. The vehicle side candidate is defined by.
The vehicle recognition device described in 1. 9. The vehicle side surface candidate detecting means sets a barycentric coordinate xG of the detected x coordinate group to x at a substantially center of a window.
The vehicle recognition device according to claim 2, wherein the vehicle recognition device is configured by means for independently calculating xGL and xGR from the coordinates to the left and right, and the x coordinates are used as vehicle side surface candidates. 10. The vehicle bottom edge candidate detecting means detects ay coordinate at which the vertical edge intensity distribution in the downward direction of the screen starts and becomes a predetermined value or less at the x coordinate of the vehicle side surface candidate detected by the vehicle side surface candidate detecting means. The vehicle recognition device according to claim 1, wherein the vehicle recognition device is configured by means. 11. The vehicle bottom edge candidate detection means has a predetermined width in the x direction based on the x coordinate of the vehicle side surface candidate detected by the vehicle side surface candidate detection means, and a predetermined width based on the y coordinate of the vehicle candidate. and a means for resetting a vertical edge detection window having a width in the y direction, and means for detecting ay coordinate at which the vertical edge intensity distribution in the lower direction of the screen starts and becomes a predetermined value or less in the window. The vehicle recognition device according to claim 1, wherein: 12. The vehicle bottom edge candidate detecting means is constituted by means for again detecting left and right vehicle side surface candidates again independently in the reset vertical edge detection window. Item 11. The vehicle recognition device according to item 11. 13. The vehicle determination means determines the difference between the average value of the y coordinate group yl (i) of the leftmost vehicle bottom end candidates and the average value of the y coordinate group yr (j) of the right vehicle bottom end candidates. The vehicle recognition apparatus according to claim 1, wherein the vehicle recognition apparatus is configured by a means for calculating and a means for determining that the calculation result is less than or equal to a predetermined value. 14. The vehicle determination means determines the difference between the average value of the y coordinate group yl (i) of the leftmost vehicle bottom end candidates and the average value of the y coordinate group yr (j) of the rightmost vehicle bottom end candidates. A means for calculating, a means for determining that the calculation result is less than or equal to a predetermined value, an average value of each of the x coordinate groups xl (i), xr (j) of the vehicle side surface candidate is within the lane, and The vehicle recognition apparatus according to claim 1, wherein the vehicle recognition apparatus is configured by means for determining that the difference between the average values is within a predetermined range. 15. The vehicle determining means calculates a difference between a y-coordinate group yl (i) of a leftmost vehicle bottom end candidate and a y-coordinate group yr (j) of a right vehicle bottom end candidate, 2. The device according to claim 1, which is configured by means for counting the number of sets whose calculation result is less than or equal to a predetermined value, and means for determining that the counting result is less than or equal to a predetermined value. Vehicle recognition device. 16. The vehicle determining means calculates a difference between a y-coordinate group yl (i) of the leftmost vehicle bottom end candidate and a y-coordinate group yr (j) of the right vehicle bottom end candidate, and A means for counting the number of sets whose calculation result is less than or equal to a predetermined value, a means for determining that the counting result is less than or equal to a predetermined value, and a x coordinate group xl (i), xr (j) of vehicle side surface candidates. 2. The vehicle recognition according to claim 1, further comprising: a means for determining that each average value is within a lane and a difference between the average values is within a predetermined range. apparatus. 17. An image pickup means mounted on a vehicle for photographing the front or rear of a route thereof and generating corresponding original image data, and a differentiation for generating differential image data by differentiating the generated original image data. Image generating means, own vehicle traveling road detecting means for detecting the own vehicle traveling road based on the generated differential image data, and detecting vehicle candidates based on the differential image data and original image data on the own vehicle traveling road Vehicle candidate detecting means, vehicle side candidate detecting means for detecting a vehicle side candidate based on a vertical edge image in the vicinity of the vehicle candidate, and vehicle bottom edge candidate for detecting a vehicle bottom edge candidate based on the vehicle side candidate Detection means, vehicle determination means for making a final determination of the vehicle based on the vehicle bottom edge candidate, and a case where the presence of the vehicle is confirmed by the vehicle determination means. In this case, vehicle bottom edge coordinate calculation means for calculating the vehicle bottom edge coordinates based on the obtained left and right vehicle bottom edge candidates, and distance calculation means for calculating the distance to the preceding vehicle based on the bottom edge coordinates. A vehicle recognition device comprising: 18. The vehicle recognizing device according to claim 17, wherein the vehicle bottom edge coordinate calculating means calculates an average value of y coordinates of vehicle bottom edge candidates. 19. An image pickup means mounted on a vehicle for photographing the front or rear of a route thereof and generating corresponding original image data, and a differentiation for generating differential image data by differentiating the generated original image data. Image generating means, own vehicle traveling road detecting means for detecting the own vehicle traveling road based on the generated differential image data, and detecting vehicle candidates based on the differential image data and original image data on the own vehicle traveling road Vehicle candidate detecting means, vehicle side candidate detecting means for detecting a vehicle side candidate based on a vertical edge image in the vicinity of the vehicle candidate, and the vehicle side candidate has a density value within a predetermined range in the screen vertical direction. And a vehicle determination means for determining a vehicle when a plurality of vehicle side surface candidates satisfying the condition are continuously present in the lateral direction. Vehicle recognition apparatus according to symptoms. 20. The vehicle side surface candidate detection means evaluates the vertical edge strength for each x coordinate in the vehicle side surface candidate detection window setting means installed near the y coordinate detected as the vehicle candidate. 20. The vehicle recognition apparatus according to claim 19, comprising vertical edge strength evaluation means, and means for judging whether or not the vehicle side surface is a candidate based on the evaluation result. 21. The vertical edge strength evaluation means makes a judgment based on the number of pixels having a vertical edge strength of a predetermined value or more for each x coordinate.
The vehicle recognition device described in 1. 22. The vertical edge strength evaluation means comprises means for automatically setting a threshold value for evaluating the vertical edge strength within the window. Vehicle recognition device. 23. The vehicle according to claim 20, wherein the vertical edge strength evaluation means has means for removing a vertical edge component caused by a contrast between a white line and a road surface in the window. Recognition device. 24. The means for removing a vertical edge component caused by the white line is a curve direction detecting means, and in the case of a left curve, on the left side of the window, the x-coordinate of the intersection of the upper and lower bases and the white line Means for removing vertical edge components appearing in the left side region, and in the case of a right curve, removing vertical edge components appearing in the right side region of the x coordinate on the right side of the window. The vehicle recognition device according to claim 23. 25. The vehicle determination means is arranged such that the number of pixels (histogram) having a vertical edge strength equal to or higher than the predetermined value according to claim 21 appears continuously at a predetermined value or more independently from the center of the window. 20. The vehicle recognition device according to claim 19, wherein the vehicle recognition device is configured by a detecting unit and a unit that determines a vehicle based on the detection result. 26. The vehicle determining means determines left and right independently the number of consecutive histograms of a predetermined value or more, and determines whether the difference between the left and right count results is within a predetermined range. 26. The vehicle recognition device according to claim 25, further comprising: means for outputting the value of the left-right difference as accuracy of the vehicle detection result.