JPH0793693A - Object detector for vehicle - Google Patents

Abstract

PURPOSE:To discriminate a character and a pattern on a road, etc., from an obstruction with height by measuring the motion of the edge point of an object in an image as if three-dimensional motion, and comparing travel amount with vehicle speed. CONSTITUTION:When a feature representing at least a white line or obstruction is sampled by a pre-processing means 102 under a condition where the vehicle position parameter of its own vehicle and the posture parameter of an image pickup means 101 are updated, an apparent travel amount speed measuring means 108 measures the apparent moving speed of the feature point in the image by coordinate-transforming the timewise motion of the feature point in the image to the travel amount on a road shown as a three-dimensional model based on the posture parameter of the image pickup means 101 updated by an update means 107. In such a way, when the apparent moving speed of the feature point in the image is measured by the apparent moving speed measuring means 108, an object discrimination means 109 compares the apparent moving speed with the vehicle speed, and discriminates whether or not the object such as the vehicle, etc., exists at the front in the advancing direction of the vehicle.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a vehicle object detection device for detecting an object such as a vehicle ahead of the vehicle in the traveling direction of the vehicle, for example, for automatic control of the vehicle or for preventive safe driving.

[0002]

2. Description of the Related Art A conventional object detecting device for a vehicle of this type is disclosed in, for example, Japanese Patent Application Laid-Open No. 4-36878. In the disclosed object detection device for a vehicle, after detecting the traveling road, the edge point is detected inside the traveling road,
An edge is detected as an object when it exists continuously for a certain period of time.

[0003]

However, in the case of the conventional object detecting device for a vehicle of this kind, it is simply detected whether or not an object such as a vehicle exists ahead of the traveling direction of the vehicle by detecting the presence or absence of an edge. Since the detection method depends only on the fact, there is a problem that characters and patterns drawn on the road surface may be erroneously detected as objects.

The present invention has been made by paying attention to the above-mentioned problems, and an object of the present invention is to correctly identify a difference between a character or a pattern on a road surface and an object such as a vehicle, and to identify the object. An object of the present invention is to provide an object detection device for a vehicle, which is capable of accurately detecting.

[0005]

In order to achieve the above object, the present invention, as shown in FIG. 1, is an image pickup means 101 for picking up an image of a road ahead of a vehicle in the traveling direction, and an image picked up by the image pickup means 101. A pre-processing means 102 for extracting at least a feature representing a white line or a candidate point for an obstacle, and a road model calculation means 103 for calculating a three-dimensional road model defined on three-dimensional coordinates based on three-dimensional curve parameters; Coordinate conversion means 104 for converting the three-dimensional road model calculated by the road model calculation means into image coordinates based on the posture parameter of the image pickup means, and at least the white line or obstacle candidate points extracted by the preprocessing means. The position shift detecting means 10 for detecting the position shift between the three-dimensional road model which has been subjected to coordinate conversion by the coordinate converting means and the feature expressing A change amount calculating means 106 for calculating the change amount of the three-dimensional curve parameter and the change amount of the posture parameter of the image pickup means from the position shift detected by the position shift detecting means; An updating unit 107 that updates the vehicle position parameter and the attitude parameter, and based on the attitude parameter of the imaging unit 101 updated by the updating unit 107, the temporal movement of the feature representing the edge portion
An apparent moving speed measuring means 108 for measuring the apparent moving speed of the feature points in the image by coordinate conversion into the moving amount on the road surface represented by the three-dimensional road model, and the apparent moving speed measuring means 108. It is characterized by further comprising object determining means 109 for comparing the apparent moving speed of the feature points in the image with the vehicle speed to determine whether or not an object exists in the forward direction of the vehicle.

[0006]

In the vehicle object detecting device according to the present invention, the preprocessing means 102 extracts at least the feature representing the white line or the obstacle from the image picked up by the image pickup means 101, while
Since the road model calculation means 103 calculates the three-dimensional road model defined on the three-dimensional coordinates based on the three-dimensional curve parameters, the positional deviation detection means 105 calculates the three-dimensional road model of the three-dimensional road model calculated by the road model calculation means 103. The values are converted into image coordinates by the coordinate conversion means 104 and received, while at the same time the characteristics representing at least the white line or the obstacle extracted by the preprocessing means 102 are compared and the positional deviation between them is detected. Then, from this detection result, the change amount calculating means 106 calculates the change amount of the vehicle position parameter of the own vehicle and the change amount of the attitude parameter of the image pickup means 101, and the updating means 107 calculates the vehicle position parameter of the own vehicle and the image pickup means 101. Posture parameters are updated.

Under such a condition that the vehicle position parameter of the own vehicle and the attitude parameter of the image pickup means 101 are updated, when the preprocessing means 102 extracts at least a feature representing a white line or an obstacle, the apparent moving amount speed In the measuring means 108, based on the posture parameter of the image pickup means 101 updated by the updating means 107, the temporal movement of the feature points in the image is coordinate-converted into the movement amount on the road surface represented by the three-dimensional road model. Then, the apparent moving speed of the feature point in the image is measured.

When the apparent moving speed measuring means 108 measures the apparent moving speed in this way, the object judging means 1
09 can determine whether or not an object such as a vehicle exists ahead of the traveling direction of the vehicle by comparing the apparent moving speed with the vehicle speed.

[0009]

Embodiments of the present invention will be described below with reference to the drawings.

FIG. 2 is a block diagram schematically showing the overall structure of a vehicle object detection device according to an embodiment of the present invention. The object detection device for a vehicle shown in the same figure inputs a camera 1 which is an image pickup means mounted on a vehicle to image a road ahead in the traveling direction of the vehicle, an image signal imaged by the camera 1,
An image processing unit 3 for image-processing the input image signal,
Also, it has a control unit 5 that controls the vehicle and makes an alarm decision based on the three-dimensional shape data of the road which is the measurement result obtained by the image processing unit 3 or the position of the obstacle.

The image processing unit 3 shown in FIG. 2 continuously inputs images of a two-lane road imaged by the camera 1, and from this image, the height (Dy), yaw angle (θ), and pitch of the camera 1 are input. Angle (φ), roll angle (ψ), distance (Dx) from the road edge, the camera's 5-axis posture parameters excluding the vehicle speed, the horizontal curve (ρ) of the road, and the vertical curve (μ) representing the slope of the road. Simultaneous online estimation of 3D road structure with the road shape parameters of Then, using these estimated values, the movement of the edge point of the object in the image is measured as if it were three-dimensional movement on the road surface, and the measured movement amount and the vehicle speed are compared, It distinguishes characters and patterns on the road surface from tall objects. Furthermore, even a moving object can know the relative moving speed.

FIG. 3 is a block diagram showing a detailed structure of the image processing unit 3 shown in FIG. As shown in the figure, the image processing unit 3 extracts an image input unit 11 to which an image captured by the camera 1 is input, and a feature representing an edge portion from image information input from the image input unit 11. Using the pre-processing unit 13, the road model creation unit 15 that creates a three-dimensional road model based on the road shape parameters estimated one screen before (previous time), and the camera posture parameter estimated at the previous time, A coordinate conversion unit 17 that performs coordinate conversion of the three-dimensional road model created by the road model creation unit 15 into a vehicle coordinate system, and further performs perspective conversion into an image coordinate system.
A line correspondence matching unit 19 that associates the road model, which has been subjected to coordinate conversion of the image coordinate system by 7 and the edge image of the road detected by the preprocessing unit 13 with a line correspondence relationship, and the line correspondence matching unit 19 performs line correspondence. A parameter estimation unit 21 that detects a positional deviation of the image and estimates the amount of change of each parameter of the road shape parameter and the camera posture parameter from the positional deviation.
The parameter updating unit 23 that updates the parameter estimated in 1 and the road model creating unit 15 determines the feature representing the edge portion extracted by the preprocessing unit 13 based on the camera posture parameter updated by the parameter updating unit 23. The spatio-temporal image creation unit 25 creates a spatio-temporal image coordinate-converted into the amount of movement on the road surface represented by the three-dimensional road model created by A straight line processing unit 27, which cuts out an obstacle or an edge point on the road surface and measures the apparent movement amount by the detection method, compares the apparent movement amount measured by the straight line processing unit 27 with the vehicle speed, and detects the obstacle. It is composed of an obstacle detection unit 29 that performs detection and relative speed discrimination.

The preprocessing in the preprocessing unit 13 is performed by the camera 1.
When the input image is I (x, y) and the edge image is G (x, y) in order to detect the edge of the white line of the road from the input image of the road imaged in, the following formula is calculated, Detect edge images.

[0014]

[Equation 1] By using such an arithmetic expression, the distant white line is almost horizontal on the curved road, and the edge can be extracted even when the line width is photographed with one pixel or less.

Further, the preprocessing unit 13 extracts a feature representing an edge portion of a two-dimensional shape object or a three-dimensional shape object on the road ahead of the vehicle traveling direction in the image by another known feature enhancement process. Good.

Road model creating section 15 and coordinate converting section 1
In 7, a three-dimensional road model is created based on the road shape parameters estimated at the previous time, and the behavioral parameters of the camera estimated at the previous time are used to perform coordinate conversion to the vehicle coordinate system. Perspective transformation to image coordinate system. In this embodiment, it is considered that the parameters of the curve and the gradient of the road change with time. It is assumed that the focal length of the camera 1 is known and that it is not estimated as the speed component of the vehicle. This is because the apparent white line position does not change even if it moves in parallel with the white line. Therefore, the parameters to be estimated are
The height (Dy) of the camera 1, the yaw angle (θ), the pitch angle (φ), the roll angle (ψ), the distance from the road edge (D
x) is a road shape parameter including a horizontal curve (ρ) and a vertical curvature (μ) in addition to the camera 5-axis posture parameter.

First, the coordinate system will be described with reference to FIG. As coordinate systems, three coordinate systems are defined: a road coordinate system (X, Y, Z), a vehicle coordinate system (U, V, W), and a two-dimensional image coordinate system (x, y).

The road coordinate system (X, Y, Z) represents the shape of the road based on the point where the vehicle is currently placed. As shown in FIG. 4, the origin is located on the road surface at the center of the road. In the direction of travel of the vehicle parallel to the road connection and road surface
Axis, X axis parallel to the road to the left, Y upward perpendicular to the road
It is a right-handed system that takes an axis. The center of the lens of the camera 1 is always located at Z = 0.

The vehicle coordinate system (U, V, W) has an origin at the center of the lens of the camera 1, and the W axis in the direction of the main axis of the lens.
It is a right-handed system that takes the U-axis and V-axis parallel to the imaging plane.

The image coordinate system (x, y) is defined on the image pickup plane, and the x axis is parallel to the U axis, the y axis is parallel to the V axis, and they are in opposite directions.

The point P (U, V, W) in the three-dimensional space represented by the vehicle coordinate system is represented by the perspective transformation as shown in the following equation in the image coordinate system on the image.

X = −F · U / W y = −F / v / w (2) where F is the focal length of the lens of the camera 1.

The relationship between the road coordinate system and the vehicle coordinate system corresponds to the behavior of the vehicle, that is, the attitude of the camera 1. Below, the distance from the road edge of camera 1 is Dx, and the height of camera 1 is D
The y and yaw angles are represented by θ, the pitch angle is represented by φ, and the roll angle is represented by ψ.

The vehicle coordinate system is first a road coordinate system (Dx,
After parallel translation by Dy, 0), yaw angle θ, pitch angle φ, and roll angle ψ are rotated in this order. The yaw angle θ is
The angle formed by the Z axis when the W axis is projected onto the XZ plane, the pitch angle φ is the angle between the W axis and the XZ plane, and the roll angle ψ is the rotation angle around the W axis and the U axis and the XZ plane. Is the angle with. When the rotation matrix is R, the points represented by the road coordinate system are converted into the vehicle coordinate system by the following equation.

[0025]

[Equation 2] Next, the road model created by the road model creation unit 15 will be described.

The structure of a road is determined by dividing it into a transverse curve (curve) and a longitudinal curve (gradient). The cross road is defined by an arc portion with a constant curvature, a straight portion, and a relaxation curve portion for connecting these smoothly. Further, the vertical curve is formed by smoothly connecting straight line portions having a constant gradient by a parabola.

On the three-dimensional road coordinates, both the transverse curve and the longitudinal curve are approximated by a multi-dimensional curve. The coordinates of the white line drawn on the road surface in the road coordinate system are defined by the following equation.

X = f (Z) = aZ ⁴ + bZ ³ + cZ ² + B Y = g (Z) = dZ ³ + eZ ² (4) Here, B represents the distance from the road center to the white line, 3
For lane roads, B is a positive definition for the left white line and 0 for the center. a to e are road shape parameters to be obtained. Note that the expression f (Z) in the horizontal direction is a quartic expression because it also corresponds to an S-shaped curve.

Further, the horizontal curvature ρ and the vertical curvature μ of the road
Is expressed by the following equation.

[0030]

[Equation 3] In particular, the curvature near Z = 0 is a horizontal curvature ρ = 2c and a vertical curvature μ = 2e.

In this way, if Zi (i = 1 to N) for creating a road model consisting of N points is given, the three-dimensional coordinates of the white line can be calculated. It is also possible to find the curvature at the point Z.

The road shape parameters a, b, c, d, and e shown in the above equation (4) are obtained from the previous screen (previous time),
A road model is created, and this road model is converted into a vehicle coordinate system based on the behavior parameter of the camera 1 at the previous time. Then, by observing how each point on the road model moves on the current screen (current time), the behavior variation amount and the road parameter variation amount in the vehicle coordinate system are obtained. That is, assuming that the point (x, y) on the road model has changed by (Δx, Δy) on the current screen, the fluctuations Δx, Δy
Is considered to be caused by the variation of the posture parameter of the camera and the variation of the road shape parameter, and the relationship between the variation amount of each parameter and Δx and Δy is derived. It should be noted that the parameter integrates the variation amount for each time, but since it is the variation amount based on the model, no error is accumulated.

Considering that the point (x, y) on the road model on the image has moved to (x ', y') due to the variation of the parameters, and the moving amount of the point is (Δx, Δy), the following equation is obtained. expressed.

X ′ = x + Δx y ′ = y + Δy (6) Consider the fluctuation of the posture parameter of the camera as the movement of the point in the vehicle coordinate system. That is, by obtaining the behavior variation amount in the vehicle coordinate system and combining it with the posture parameter at the previous time,
It is converted into the camera posture parameter at the current time, and a new vehicle coordinate system is created at the next time. Note that the behavior variation amount is obtained not in the road coordinate system but in the vehicle coordinate system, because the rotation angle always changes from 0, and therefore a linear solution can be obtained while maintaining the approximation accuracy.

Point P (U, V, W) in the vehicle coordinate system
Is moved to the point P '(U', V ', W'),

[Equation 4] It is represented by. Note that D _U and D _U are translation components,
α, β, and γ are rotation angles around the V axis, the U axis, and the W axis, respectively. However, the value of each parameter is assumed to be sufficiently small, and sin δ = δ, cos δ = 1, sin α = α, co
sα = 1, sinβ = β, cosβ = 1, sinγ =
γ and cos γ are set to 1, and terms of the second or higher order are ignored.

The following equation is obtained from the equations (6) and (2).

[0037]

[Equation 5] The change in image coordinates due to a slight change in each attitude parameter is
From the first-order term of the Taler expansion of the equation (5), it is expressed as the following equation.

[0038]

[Equation 6] Approximately calculating the condition of α, β, γ = 0 from the equation (8), the variation of the posture parameter of the camera is obtained as shown in the following equation.

[0039]

[Equation 7] Next, consider the variation of road parameters. From the above formulas (3) and (4), U = R ₁₁ (X−D _X ) + R ₁₂ (Y−D _Y ) + R ₁₃ Z V = R ₂₁ (X−D _X ) + R ₂₂ (Y−D _Y ) + R ₂₃ Z X = f (Z), Y = g (Z) (11), which is substituted into the equation (7), and α, β, γ = 0

[Equation 8] By calculating, the variation of the road parameter is obtained as shown in the following equation.

[0040]

[Equation 9] Therefore, by adding the equations (9) and (13), the relational expression of the movement amount of the point on the image by the posture parameter of the camera and the road parameter can be obtained.

[0041]

[Equation 10] Next, the line matching matching in the line matching matching unit 19 will be described.

The camera posture estimation method using the model includes point correspondence and line correspondence. In the present embodiment, line correspondence and line correspondence are used.
That is, the tangent line in the road model and the line component of the white line extracted from the screen are associated with each other. This is because the white line is a smooth curve and it is difficult to match the points.

An image search is necessary for association,
Here, scanning is performed on the same x-axis or the same y-axis. This is because the search process can be simplified and the speed can be increased.
Selection of x-axis scanning or y-axis scanning is performed according to the magnitude of the inclination of the associated line.

Point P on the road model in the image coordinate system
The slope of the tangent line at (x, y) is represented by ω, and ω is

[Equation 11] Ω is calculated by the following equation.

[0045]

[Equation 12] It is assumed that the white line at the current time has the same slope ω as the model at the point P ′ (x ′, y ′) near the point P. That is,
It is assumed that there is no change in inclination. It is assumed that the model obtained at the previous time has moved to the white line position at the current time. The relationship between the road model and the white line in this case is as shown in FIG.
Let p be the amount of movement on the same x-axis and q be the amount of movement on the same y-axis.
Then, the relation between Δx and Δy is Δx / p + Δy / q = 1 (17) and further q / p = −ω (18), so Δx−Δy / ω−p = 0 (19) Alternatively, −ωΔx + Δy−q = 0 (20) is obtained. The above equation represents the relationship between the (apparent) amount of movement in the direction of the coordinate axis of the straight line and the actual amount of movement. When | ω | is large (close to vertical), the equation (19) is used, and | ω |
When is small (close to horizontal), the equation (20) is used.

At some points on the road model, Pi (x
i, yi) and ωi are calculated, pi or qi is obtained from the correspondence with a point on the input image, and the least squares method is applied to obtain a simultaneous equation, and the change amount of each parameter is calculated.

The evaluation error is calculated by the above equation (14) and (1
The following equation is obtained from the equations (9) and (20).

[0048]

[Equation 13] E _i = A _1i Δa + A _2i Δb + A _3i Δc + A _4i Δd + A _5i Δe + A _6i ΔD _U + A _7i ΔD _V + A _8i Δα + A _9i Δβ + A _10i Δγ-B _i (21)

[Equation 14]

[Equation 15] Next, the operation of this embodiment will be described.

First, in the road model creating section 15, N
Create a road model consisting of points. Then, the conversion matrix of Expression (3) is created from the estimation result of the camera posture parameters (Dx, Dy, θ, φ, ψ) at the previous time. Then, the road shape parameters (a, b, c, d,
e) to road coordinate system (X, Y, Z) based on equation (4)
Calculate the three-dimensional coordinates of the road model. That is, the point sequence of the road model is created by setting Z = Zi (i = 1 to N) in the equation (4) from the road shape parameters (a, b, c, d, e) at the previous time. Next, by using the behavior parameters (Dx, Dy, θ, φ, ψ) of the camera at the previous time, the vehicle coordinate system (U, V, W) is converted by the equation (3), and the image coordinate system is further transformed by the equation (1). Convert to (x, y) and coordinate (x
i, yi) (i = i to N). Also, the slope ω of the tangent line is calculated by the equation (16). The coordinate conversion unit 17 performs each coordinate conversion.

The value calculated from the road model is (21)
Zi, Ui, Vi, Wi, ωi and x appearing in the equation
i and yi (i = 1 to N). Of these, Zi is predetermined to be, for example, an interval of 3 meters on the road coordinates. That is, if Zi is given, Ari (r = 1 in the equation (21) is obtained.
The values of 10 to 10) are determined. Next, matching with the input image is performed to obtain Bi.

The line matching matching section 19 is used in the image input section 1.
Line matching between an edge image, which is an input image obtained from the camera 1 via 1 and a model point connection, is performed. ｜ ω
When i | ≧ 1 (close to vertical), each model point (x
i, yi) is searched in the x-axis direction to find pi,
When | ωi | <1 (when it is close to horizontal), qi is obtained by searching in the y-axis direction.

When | ωi | ≧ 1, the point (xi, yi)
Consider a window having a width M _x and a height M _y centered at. Among them, a straight line having a slope ωi is generated, and the straight line having the largest sum of edge densities on the straight line is selected, and the x coordinate value of the straight line
The difference from i is pi. That is,

[Equation 16] Is calculated, the center of gravity is calculated in the vicinity of the maximum value of Ci (τ), and is defined as pi. Further, the value of Ci (τ) at this time represents the certainty, so it will be referred to as Ci again.

When | ωi | <1, qi is similarly obtained. Note that the number of pixels is suitable for a width M _x, height M _y.

As described above, since all the coefficients of the evaluation error shown in the equation (21) are obtained, each parameter is applied to the parameter estimation unit 2 by applying the least squares method.
Can be estimated in 1.

Now consider another error measure. One is to use the fact that road parameters do not change significantly with time. That is, it can be said that Δa to Δe are close to zero. Therefore, E _G1 = Δa, E _G2 = Δb, E _G3 = Δc, E _G4 = Δd, E _G5 = Δe, E _{G5 to} E _G10 = 0 (24), and the respective weights are set to Gr (r = 1 to 5). ) Is defined as a constant.

Another error relates to curvature. Road parameters c and e are 1 / the curvature of the point of Z = 0
It is 2. If the vehicle has a known speed and is traveling at a speed v (m (meter) / 1 screen time), the curvature at the point Z = vt before the t screen should be the curvature Z = 0 on the current screen. Is. Therefore, E _ρ = C _k−1 + Δc−ρ _k E _μ = e _k−1 + Δe−μ _k (25)

[Equation 17] Can be Note that c _k-1 and e _k-1 are the results of the previous screen, respectively. Further, ρ _k-1 and μ _k-1 are the curvatures obtained from the result before t screens.

Each parameter is estimated by the method of least squares.
That is,

[Equation 18] Differentiate the above equation with each variable to minimize
far. Hρ and Hμ are weighting constants. In this way, the following 10-element simultaneous linear equations

[Formula 19] Is created. Here, 1 (small letter of L) is 1-10, and m = 1-10.

This 10-element simultaneous linear equation can be easily solved.

As described above, the parameter estimation unit 21
When the change amount of each parameter is calculated by the method of least squares, the camera posture parameters (Dx, Dy, θ, φ, ψ) and the road shape parameters (a, b, c, d, e) is updated by the parameter updating unit 23. Note that the parameter at the previous time is the subscript k-1.
The updated parameter is represented by the subscript k.

First, the road shape parameters a, b, c,
For d and e, Δa to Δe may be added as they are. That is, it becomes as shown in the following equation.

[0061]

A (k) = a (K-1) + Δa, b (k) = b (K-1) + Δb, c (k) = c (K-1) + Δc, d (k) = d ( K-1) + Δd, e (k) = e (K-1) + Δe (29) Further, the camera posture parameters Dx, Dy, θ, φ, ψ
Is updated as follows: The parameters (D _U , D _U , α, β, γ) in the equation (7) are all 0 at k−1.
Since it can be considered that D _U = ΔD _U , D _U = ΔD _U , α = Δ
Let α, β = Δβ, γ = Δγ. Substituting equation (2) into equation (7),

[Equation 21] Becomes By updating the posture parameter of the camera, (U, V, W) = (U ′, V ′, W ′). By comparing formula (3) and _{(30), R (k) = SR (} k-1) D X (k) = D X (k-1) -R 11 (k) D U -R 21 _(k) obtained at _{D V D Y (k) =} D Y (k-1) -R 12 (k) D U -R 22 (k) D V (32). The relational expression of R ⁻¹ = ^RT was used. θ _(k) , φ _(k) , and ψ _(k ) are R _{31 (k)} in the equation (3 ₎ ,
It is easily obtained from R _{32 (k)} and R _{12 (k)} .

Next, a method of detecting an obstacle by a series of processing operations in the spatiotemporal image creating section 25, the straight line detecting section 27, and the object detecting section 29 will be described.

First, the principle of discriminating objects will be described.

It is assumed that the three-dimensional shape of the road ahead and the camera posture with five degrees of freedom with respect to the road are obtained by estimating the road shape and the camera posture. At this time, assuming that all the objects are on the road surface, the coordinate values of the image coordinate system and the coordinate values of the road coordinate system have a one-to-one correspondence. This is because the three-dimensional shape of the road surface is described by the road model of equation (4), and the three-dimensional coordinates are converted into the image coordinate system by equations (2) and (3).

It is assumed that a tall object is reflected in the image. Although it is not possible to determine the presence or absence of height in a two-dimensional image, apparent coordinate values can be obtained in the image coordinate system. Assuming that the point is on the road surface, the apparent coordinate value can measure how many meters ahead. For example, as shown in FIG. 6, an object with a height is observed so that points (vertices, edge points) are present on the road surface farther than the actual position.

It is assumed that the road surface is closer to the camera by the vehicle speed. At this time, in the case of a stationary object, the true movement amount matches the vehicle speed, but the apparent movement amount is observed to be larger than the vehicle speed (FIG. 6). For patterns and dirt on the road surface, the true movement amount and the apparent movement amount are equal. Furthermore, in the case of a moving object such as a preceding vehicle, the apparent amount of movement is smaller than the vehicle speed when approaching slowly, but
It has the same sign as the vehicle speed. Conversely, in the case of a distant preceding vehicle, the vehicle speed and the apparent movement amount have opposite signs. That is, the apparent movement amount of the pattern on the road surface = vehicle speed stationary object apparent movement amount> vehicle speed moving object (approaching) 0 <apparent moving amount <vehicle speed moving object (far away) 0> apparent moving amount.

The above is the principle of discrimination between a stationary object, an object approaching slowly, an object moving away, and noise and patterns on the road surface.

In the spatiotemporal image creating section 25, the preprocessing section 13
A spatiotemporal image is created by receiving the feature points extracted in step 3, the three-dimensional road model created by the road model creating unit 15, and the posture parameter updated by the parameter updating unit 23.

The creation of this spatiotemporal image will be described.

Since the road model is known (estimated), the coordinate point sequence parallel to the road and equidistant is obtained from the equation (4). As shown in FIG. 7A, in the case of two lanes, for example, two lanes are obtained in each lane or the center of each lane is calculated. The parallel lines are distinguished by the subscript j, and the point sequences in the forward direction are distinguished by the subscript i. The value of the point in the road coordinate system is the same as equation (4),

[Equation 22] Z _i is set at lm intervals, for example. L _j is 0 at the center of the road, and for example, L ₁ = -2.5 m, L ₂ = -1.25 m,
L ₃ = 0 m, L ₄ = 1.25 m, L ₅ = 2.5 m, and so on. The road shape parameters a, b, c, d, and e use the values obtained by estimating the road shape.

By converting the coordinate values calculated by the equation (33) from the equations (2) to (3), the image coordinates (x _{i, j} , y) can be obtained.
_{i, j} ) is required. At this time, the above estimation result is used as the value of the camera posture parameter.

The current time t = t _k is simply represented by k. Time k
Of the preprocessing result image G _k (x, y) of (x _{i, j} ,
y _{i, j} ), the spatiotemporal image H _j (k, i) is extracted.
To create. That is, H _j (k, i) = G _k (x _{i, j} , y _{i, j} ) (34).

The above situation is shown in FIGS.
FIG. 7C shows a conceptual diagram of the spatiotemporal image on j = 3, that is, on the center line (assumed to be a break line).

In the spatiotemporal image, the edge points moving at a constant speed appear linearly. The inclination of this straight line is the apparent movement amount.

In the straight line detector 27, the spatiotemporal straight line generator 2
An edge point on the object or the road surface is cut out from the spatiotemporal image created in step 5 by the method of straight line detection. For example, Hough conversion is performed in the section from time k to k−K. K
Is, for example, 1 second. If the Hough plane is P _j (α, β),

[Equation 23] Required by. If there is a point (α, β) in P _j (α, β) having a value equal to or greater than a certain threshold value, it is considered that a straight line has been detected. The distance from the value of β to the point is obtained (= Z _i , i
= Β), the slope β coincides with the apparent velocity.

When the apparent moving speed is obtained in this way, the object detecting unit 29 performs object detection and relative speed discrimination as follows.

As described above, the object is discriminated from the relationship between the apparent speed (movement amount) and the vehicle speed. The vehicle speed may be obtained from a vehicle speed meter, but here, a method of obtaining it from the movement amount of the center line will be described.

Assuming that the center line is a broken line, a straight line group having a constant inclination is detected as shown in FIG. 7 (c). Therefore, the value α of the straight line detection result at j = 3 is set as (the estimated value of) the vehicle speed V.

If no straight line is detected when j ≠ 3,
Judge that there is no object. When a straight line is detected, it is judged from the comparison of the sign of the inclination α and V (1) When α and V are almost equal, it is a dirt or pattern on the road surface. (2) The absolute value of α is the absolute value of V If it is larger than the value, it is judged as an object approaching at a stationary or fast relative speed. (3) If the sign of α and the sign of V are opposite, it is judged as an object moving away. (4) Other than the above, it is judged as an object approaching slowly.

In the cases of (2) and (4) above, as described above, β
The distance to the object is calculated from the value of (= Z _i i = β
Therefore, an alarm may be given according to the distance, or the vehicle speed may be automatically controlled.

[0081]

As described above, according to the present invention, by measuring the three-dimensional shape of the road and the attitude of the camera with respect to the road in advance, the movement of the edge point of the object in the image as if on the road surface. By measuring the three-dimensional movement and comparing the measured movement amount with the vehicle speed, it is possible to discriminate a character or pattern on the road surface from an obstacle with a height. Further, even with a moving obstacle, it is possible to know the approximate relative moving speed. Moreover, since only one camera signal is processed, it can be realized at low cost.

[Brief description of drawings]

FIG. 1 is a diagram corresponding to a claim of the present invention.

FIG. 2 is a block diagram schematically showing an overall configuration of a vehicle object detection device according to an embodiment of the present invention.

3 is a block diagram showing a detailed configuration of an image processing unit used in the vehicle object detection device shown in FIG.

FIG. 4 is an explanatory diagram showing a coordinate system used in the image processing unit shown in FIG.

5 is an explanatory diagram showing line-to-line matching between a road model and a white line in the line-to-line matching unit shown in FIG.

FIG. 6 is a diagram showing the principle of object detection.

FIG. 7 is a diagram showing an example of a spatiotemporal image.

[Explanation of symbols]

 101 Image pickup means 102 Pre-processing means 103 Road model calculation means 104 Coordinate conversion means 105 Positional deviation detection means 106 Change amount calculation means 107 Update means 108 Apparent movement speed measurement means 109 Object determination means

Claims (2)

[Claims] 1. An image pickup means for picking up an image of a road ahead of a vehicle in a traveling direction, a preprocessing means for extracting at least a feature representing a white line or an obstacle candidate point from an image picked up by the image pickup means, and a three-dimensional curve parameter. Road model calculation means for calculating a three-dimensional road model defined on the three-dimensional coordinates based on the above, and the three-dimensional road model calculated by the road model calculation means into image coordinates based on the attitude parameter of the imaging means. The coordinate conversion means for conversion is compared with the feature representing the candidate point of at least the white line or obstacle extracted by the preprocessing means and the three-dimensional road model coordinate-converted by the coordinate conversion means, and the two Positional deviation detecting means for detecting positional deviation, the amount of change in the three-dimensional curve parameter from the positional deviation detected by the positional deviation detecting means, and the appearance of the imaging means A change amount calculating means for calculating a change amount of a parameter, an updating means for updating the vehicle position parameter and the posture parameter based on a change amount of the both parameters, and a posture parameter of the image pickup means updated by the updating means. On the basis of the above, the apparent movement of the feature representing the edge portion is converted into a movement amount on the road surface represented by the three-dimensional road model, and the apparent moving speed of the feature point in the image is measured. An object for determining whether or not there is an object ahead of the traveling direction of the vehicle by comparing the apparent moving speed of the characteristic point in the image measured by the apparent moving speed measuring means with the vehicle speed. An object detection device for a vehicle, comprising: a determination unit. 2. The vehicle according to claim 1, wherein the object determining means discriminates various objects according to whether the apparent moving speed and the vehicle speed have the same sign or opposite signs. Object detection device.