JPH06273172A - Image processing device using stereo camera - Google Patents

Abstract

PURPOSE:To obtain an image processing device which can electrically compensate the amount of deviation of the light axis of a stereo camera, realize cost down, at the same time, allows adjustment to be made easily, and uses the stereo camera which less changes with time and has a high reliability. CONSTITUTION:The device is provided with a stereo camera 100 with a pair of video optical systems and an image processing device 200 for processing a pair of image signals G obtained from the stereo camera 100. The stereo camera 100 is provided with a means 6 for storing the amount of deviation for storing the amount of deviation (i, j) of the light axis in horizontal and up/down directions for light reception elements 2 inside the video optical system. The image processing device 200 is provided with an image memory 7 for storing a pair of image signals obtained from the stereo camera 100 and a means for compensating the amount of deviation by operating the address of the image signal on the image memory 7 according to the amount of deviation.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing apparatus using a stereo camera having a pair of video optical systems (hereinafter, simply referred to as an optical system), and particularly cost reduction by electrically performing optical axis correction. The present invention relates to an image processing apparatus using a stereo camera that realizes the above and realizes ease and high reliability.

[0002]

2. Description of the Related Art Generally, an image processing apparatus using a stereo camera is widely used, for example, for measuring a distance to an object, detecting an object as an obstacle, and measuring the position and size of the object. In this type of device, due to a manufacturing error at the time of assembly, a relative shift amount occurs between the optical axes of the left and right optical systems, and an error occurs between a pair of image signals. It is necessary to adjust the optical axis in advance. Usually, one of the pair of optical systems is used as a reference and the other is adjusted.

FIG. 2 is a perspective view showing a pair of optical system structures of a general stereo camera.
Is a pair of lenses arranged corresponding to the pair of optical axes L1 and L11, and 2 and 12 are a pair of light receiving elements that receive the image formed through the lenses 1 and 11. The light receiving elements 2 and 12 are two-dimensional array elements such as CCD image sensors. Here, the right-side optical system including the lens 1 and the light-receiving element 2 is the reference optical system, and the left-side optical system including the lens 11 and the light-receiving element 12 is the adjustment target optical system.

Α is the horizontal deviation of the optical axis L11 with respect to the lens 11, θ is the vertical deviation of the optical axis L11 with respect to the lens 11, S is the amount of deviation in the rotational direction of the image with respect to the light receiving element 12, and T is the light reception. A lateral shift amount of the image with respect to the element 12 and U is a vertical shift amount of the image with respect to the light receiving element 12.

An image signal picked up by a stereo camera as shown in FIG. 2 is sent to an image processing device (not shown) via an image signal controller as is well known, and contributes to distance measurement and the like. Further, in the conventional stereo camera, although not shown, the lens 11 and the light receiving element 12, which are the optical system on the adjustment side, are mechanically (screw) with respect to the respective deviation angles α and θ and the deviation amounts S to U. And using a spring)
An adjusting mechanism is provided.

FIG. 3 is a side view for explaining an optical axis adjusting method of a conventional stereo camera. A is an object imaged by the stereo camera, D is the position of the object A, and δ is the light to the lens 11. Angle deviation of axis L11, R is lens 1
1 is the distance from the object A, f is the focal length of the lens 11, and d is the amount of positional deviation of the object image on the light receiving element 12 due to the angular deviation δ.

Here, the relationship between the shift amount d and the focal length f,
In addition, the relationship between the position D of the object A and the distance R is expressed by the following equation.

D = ftanδ (1) D = Rtanδ (2)

Further, generally, the positional change amount X of the object A with respect to the parallel movement amount x of the image position on the light receiving element 12 is as follows.

X = (R / f) x (3)

In the equation (3), if X = D,
From the expressions (1) and (2), the following relationship holds.

Rtanδ = (R / f) x (4) x = ftanδ = d (5)

From the equations (4) and (5), tilting the optical axis L11 by the angle deviation δ means that the light receiving element 12 is ftan δ.
It can be seen that the effect is equivalent to parallel translation. That is, the amount of shift in the horizontal and vertical directions T and U on the light receiving element 12 also includes the amount of shift due to the left and right deflection angles α and θ of the lens 11.

Next, the optical system adjusting operation of the image processing apparatus using the conventional stereo camera will be described with reference to FIGS. First, an object at infinity or a reference object whose distance R is known in advance is used to image the pair of light receiving elements 2 and 12, and the optical axis is manually adjusted. For example, when an object at infinity is used, the positions of the pair of images on the light receiving elements 2 and 12 should match.
The lens 11 and the light receiving element 12 of the adjustment-side optical system are mechanically adjusted.

That is, with respect to the lens 11, the angular deviations α and θ in the vertical and horizontal directions are adjusted, and with respect to the light receiving element 12, the rotational direction R, and the vertical direction U and the horizontal direction T.
Perform parallel translation of.

At this time, practically, assuming that the angular deviation δ and the parallel movement amount x are small, the equations (1) to
As shown in the equation (5), it is replaced as the pixel deviation x in the vertical and horizontal directions. Therefore, in reality, the light receiving element 12
In addition to the adjustment of the rotation direction S, the adjustment is performed by parallel movement of the light receiving element 12 to the left and right T and up and down U, or the angle adjustment of the left and right α and up and down θ of the lens 11.

That is, the adjustment of the optical axis L11 is performed by the light receiving element 12
And the parallel movement of the left and right T and the up and down U, or the angle adjustment of the left and right α and the up and down θ of the lens 11. However, since the manual work as described above requires a great deal of labor and time, it cannot be adjusted frequently, and
Even if the adjustment is completed once, the reliability will deteriorate due to changes over time.

[0018]

As described above, the image processing apparatus using the conventional stereo camera is mechanically operated in order to adjust all the directions with respect to the optical axis L11 of one of the pair of optical systems. Since a manual adjustment mechanism using screws and springs is used, there is a problem that adjustment takes time and cost reduction cannot be realized.

Further, even if the adjustment is once performed, since the mechanical adjustment mechanism is used, for example, the adjustment screw is loosened little by little during use, and the optical axis L11 relatively changes with time, resulting in a final image. There is a problem that the processing result changes and the reliability is impaired.

The present invention has been made to solve the above problems, and realizes cost reduction, is easy to adjust, and is an image processing apparatus using a stereo camera that is highly reliable and does not change over time. The purpose is to get.

[0021]

An image processing apparatus using a stereo camera according to claim 1 of the present invention processes a stereo camera having a pair of video optical systems and a pair of image signals obtained by the stereo camera. The stereo camera has shift amount storage means for storing the shift amounts of the optical axes in the horizontal and vertical directions with respect to the light receiving elements in the video optical system. An image memory for storing a pair of image signals,
There is provided a shift amount correcting means for correcting the shift amount by operating the address of the image signal on the image memory according to the shift amount.

Further, in the image processing apparatus using the stereo camera according to claim 2 of the present invention, in claim 1, the shift amount correcting means shifts one of the image signals on the image memory by the shift amount. It is accessed using an address.

An image processing apparatus using a stereo camera according to a third aspect of the present invention is the image processing apparatus according to the first aspect, wherein the shift amount correcting means shifts one of the image signals by the shift amount. It is what you store above.

An image processing apparatus using a stereo camera according to a fourth aspect of the present invention is a stereo camera having a pair of video optical systems, and an image processing apparatus for processing a pair of image signals obtained by the stereo camera. The stereo camera includes a shift amount storage unit that stores shift amounts of the optical axis in the horizontal and vertical directions with respect to the light receiving element in the video optical system, and a shift amount correction unit that corrects the shift amount. .

An image processing apparatus using a stereo camera according to a fifth aspect of the present invention is the image processing apparatus according to the fourth aspect, wherein the shift amount correcting means outputs the output timing of one of the image signals sent to the image processing apparatus. Is to shift by the amount of shift.

[0026]

According to the first aspect of the present invention, the shift amount of the optical axis on the light receiving element is measured and stored by the electric circuit in the stereo camera, and is obtained from the stereo camera by the correction means in the image processing apparatus. Based on the shift amount information, the address relating to one of the pair of image signals in the image memory is operated to correct the shift amount of the optical axis in the processing of the image signal.

Further, according to the second aspect of the present invention, one of the image signals in the image memory is accessed at an address shifted by a shift amount.

Further, according to the third aspect of the present invention, one of the image signals from the stereo camera is stored in the image memory at an address shifted by the shift amount.

According to the fourth aspect of the present invention, the image signal whose displacement amount is corrected by the displacement amount correcting means in the stereo camera is sent to the image processing apparatus.

In the fifth aspect of the present invention, the shift amount correcting means in the stereo camera shifts the timing of one of the image signals by the shift amount and sends it to the image processing apparatus.

[0031]

【Example】

Example 1. Embodiment 1 of the present invention will be described below with reference to the drawings. 1 is a block diagram schematically showing a part of a first embodiment of the present invention (corresponding to claim 1 and claim 2),
L1, L11, 1, 11, 2 and 12 are the same as described above.

Reference numerals 3 and 13 denote a pair of amplifiers for amplifying the image signals from the light receiving elements 2 and 12, respectively, which correspond to a pair of optical systems. Reference numeral 4 denotes a light-receiving element controller that controls the scanning of the light-receiving elements 2 and 12 and also controls the scanning timing thereof. Reference numeral 5 denotes an image signal controller that controls the light-receiving element controller 4 and takes in an image signal via the amplifiers 3 and 13, and 6 The shift amount storage device stores the shift amount (i, j) of the optical axis L11 calculated by the image signal controller 5.

In this case, the shift amount (i, j) means the shift amount of the optical axis L11 with respect to the light receiving element 12 in the horizontal and vertical directions (T and U in FIG. 2), and the image memory (described later).
It corresponds to the pixel and the address in.

Therefore, the light receiving element 12 is provided with a mechanical adjusting mechanism only in the rotation direction S. still,
Although it is possible to electrically correct the deviation in the rotation direction S by calculating the deviation amount, it is desirable to perform the correction mechanically because the processing of the image is complicated and a long time is required for conversion.

Reference numeral 100 is the image signal G and the shift amount (i, j).
Is a stereo camera that outputs
The light receiving elements 2 and 12, the amplifiers 3 and 13, the light receiving element controller 4, the image signal controller 5, and the shift amount storage device 6 are provided.

Reference numeral 7 is an image memory for storing a pair of image signals G obtained from the stereo camera 100. Reference numeral 8 is a CPU for writing and reading the image signals G in the image memory 7 and for processing the image signals G. , 9 connect the image signal controller 5 and the CPU 8 to obtain the shift amount (i, j) from the CPU.
8 is an input / output interface to be taken into the device.

The CPU 8 includes a deviation amount correction means for correcting the deviation amount by operating the address of the image signal G on the image memory 7 according to the deviation amount (i, j). Reference numeral 200 denotes an image processing apparatus coupled to the stereo camera 100, which has an image memory 7, a CPU 8 and an input / output interface 9.

Next, the operation of the first embodiment of the present invention shown in FIG. 1 will be described with reference to FIG. As described above, when the stereo camera 100 is assembled, the images formed on the light receiving elements 2 and 12 include horizontal and vertical shifts including a shift in the rotation direction S and shifts due to left and right and up and down deviation angles α and θ. Misalignment between directions T and U occurs.

Of these, the correction of the deviation amount with respect to the rotation direction S is performed by a mechanical adjustment mechanism as in the conventional case, for example, so that two specific points on the image are imaged on a designated straight line. This is done by rotating the element 12.

On the other hand, the left and right and up and down deviation angles α and θ can be replaced with parallel translations in the horizontal and vertical directions T and U on the light receiving element 12, so that the address on the image memory 7 in the image processing apparatus 200 can be replaced. By the processing, the correction can be easily performed.

For example, when an object at infinity is imaged, the two-dimensional coordinate position of the image formed on each of the light receiving elements 2 and 12 is (x, y) on the reference side light receiving element 2. Assuming that (x + i, y + j) is on the adjustment side light receiving element 12, the shift amount of the light receiving element 12 is (i, j), and the correction value for the image signal G from the light receiving element 12 is − in the x direction. It becomes -j in the i and y directions.

The deviation amount (i, j) is measured by the image signal controller 5 when the rotation direction S is adjusted, and is stored in the deviation amount storage device 6. For this deviation amount (i, j), an electric digital memory may be used, a dip switch may be used, or the voltage may be adjusted by a variable resistor or the like.

When the stereo camera 100 is connected to the image signal processing apparatus 200 and powered on, the stereo camera 100 starts exchanging data with the image processing apparatus 200. First, the stereo camera 100 receives the command from the image processing device 200, and the shift amount (i, j) stored in the shift amount storage device 6 is stored.
And a sync signal and an image signal G are sent from the image signal controller.

At this time, if the shift amount storage device 6 is a digital memory, the shift amount (i, j) of the optical axis L11 is stored as a digital value, and the shift amount (i, j) is the image processing device. 200 is serially output.

The image processing apparatus 200 stores the pair of captured image signals G as data from addresses (1, 1) to (m, n) in the image memory 7, respectively. When obtaining the distance R by the principle of triangulation by the correlation calculation of the pair of image signals G, the CPU 8 in the image processing apparatus 200 is arranged on the adjustment side corresponding to the point (x, y) on the reference side image signal G. The calculation is performed by correcting the point on the image signal G as (x-i, y-j).

The correction amount (-i, -j) at this time can be easily calculated because the data of the shift amount (i, j) is taken in from the stereo camera 100 in advance. That is, when the image signal data on the adjustment side is accessed from the image memory 7, the coordinate positions (x, y) on the reference side are respectively corresponded by shifting by −i in the x direction and −j in the y direction. The data of the address to be read can be read.

Thus, in the stereo camera 100,
The deviation amount (i, j) of the optical axis of the light receiving element 12 is measured and electrically stored in the deviation amount storage device 6, and the deviation amount (i, j) is easily calculated by the deviation amount correcting means in the CPU 8. Can be corrected. Therefore, only the shift amount in the rotation direction S is manually adjusted, and time for adjusting the shift amounts in the horizontal and vertical directions T and U is not required, and the mechanical adjustment mechanism is simplified and the price is also reduced. Become cheap. Further, since there are almost no mechanically moving parts, there is no change with time and reliability is improved.

Example 2. In the first embodiment, the deviation amount correcting means in the image processing apparatus 200 corrects the address by the deviation amount (i, j) when accessing the data from the image memory 7. The address may be corrected when the image signal G is stored in 7.

In the second embodiment of the present invention (corresponding to claim 3), the deviation amount correcting means in the CPU 8 indicates the deviation amount (i, j) of the optical axis from the stereo camera 100 when the power is turned on. Since the data is being received, the data of the image signal G sent from the stereo camera 100 is stored in the image memory 7 from the beginning at the address i in the x direction and j in the y direction.
Store only the address shifted.

That is, with respect to the image signal G on the adjustment side, the data in the 1st row to the i-1th row in the x direction and the 1st column to the j-1th column in the y direction are discarded and the addresses (i, j) are stored. The corresponding data starts to be stored from the address (1,1) in the image memory 7. Below, the address (i, j +
The data corresponding to 1) is stored in the address (1, 2) in the image memory 7, and finally the data corresponding to the address (m, n) of the image signal G is stored in the address (m-i, n-j).

Therefore, in the image memory 7 relating to the image signal G on the adjustment side, there is no valid data from the row m-i + 1 to the row m, and similarly, from the column n-j + 1 to the row n.
There is no valid data up to the column. In this case, the CPU
In FIG. 8, in the calculation for obtaining the distance R and the like from the data of the image signal G, the data from the 1st row to the mi row and the data from the 1st column to the nj row in the image memory 7 are used. Will be done.

Example 3. In each of the first and second embodiments described above, the deviation amount correcting means is provided in the CPU 8 and the address correction operation is performed on the image memory 7 in the image processing apparatus 200. However, the deviation amount correcting means is provided in the stereo camera 100. May be provided. For example, in the third embodiment (corresponding to claim 4 and claim 5) of the present invention, a deviation amount correcting means is provided in the image signal controller 5, and the deviation amount (i,
j) the image signal G corrected by the amount
Send to.

That is, the shift amount correcting means in the stereo camera 100 shifts the timing of the data on the adjustment side of the image signal G by the shift amount (i, j) and the image processing apparatus 2
Image signal G sent to 00 and stored in the image memory 7
It is assumed that the data (1) has already been corrected for the deviation amount (i, j) of the optical axis L11. In this case, the image processing apparatus 200 may have the conventional configuration.

For example, if the shift amount of the optical axis L11 in the xy directions on the light receiving element 12 is (i, j), the image signals G from the first row to the i-1 row in the x direction are dummy signals. ,
Similarly, the first image signal G from the jth column to the j−1th column in the y direction is sent to the image processing apparatus 200 as a dummy signal. Then, from the image signal G at the i-th row and the j-th column, that is, the address (i, j), the image signal G corresponding to the first row and the first column, that is, the address (1,1) is sent to the image processing apparatus 200.

As a result, the image processing apparatus 200 stores the image signal G in the image memory 7 in the normal address order,
Moreover, by accessing as usual, correct data in which the shift amount (i, j) of the optical axis L11 is corrected can be obtained, and highly reliable arithmetic processing can be performed.

Further, by providing the shift amount correcting means together with the shift amount storing means 6 in the stereo camera 100, the conventional image processing apparatus 200 can be shared only by improving the stereo camera 100 side.

[0057]

As described above, according to the first aspect of the present invention, the stereo camera having the pair of video optical systems and the image processing device for processing the pair of image signals obtained by the stereo camera are provided. The stereo camera has displacement amount storage means for storing displacement amounts of the optical axis in the horizontal and vertical directions with respect to the light receiving element in the video optical system, and the image processing device includes
Since the image memory for storing a pair of image signals obtained from the stereo camera and the shift amount correcting means for correcting the shift amount by operating the address of the image signal on the image memory according to the shift amount, It is possible to electrically correct the deviation amount of the optical axis of the stereo camera, realize the cost reduction, and to obtain the image processing device using the stereo camera which is easy to adjust and has little change over time and has high reliability. There is.

According to a second aspect of the present invention, in the first aspect, the shift amount correcting means performs access using an address obtained by shifting one of the image signals on the image memory by the shift amount. Therefore, it is possible to electrically correct the deviation amount of the optical axis of the stereo camera, realize the cost reduction, and the image processing device using the stereo camera which is easy to adjust and does not change with time and has high reliability. There is an effect to be obtained.

According to a third aspect of the present invention, in the first aspect, the shift amount correcting means stores one of the image signals in the image memory at an address shifted by the shift amount. The amount of deviation of the optical axis of the stereo camera can be electrically corrected, the cost can be reduced, and the image processing apparatus using the stereo camera which is easy to adjust and does not change with time and has high reliability can be obtained. effective.

According to a fourth aspect of the present invention, the stereo camera is provided with a stereo camera having a pair of video optical systems and an image processing device for processing a pair of image signals obtained by the stereo camera. An image signal whose displacement amount has been corrected is provided with displacement amount storage means for storing displacement amounts of horizontal and vertical optical axes with respect to a light receiving element in the video optical system and displacement amount correction means for correcting the displacement amount. Since the image is transmitted to the image processing apparatus, the image processing apparatus can be used also, and the deviation amount of the optical axis of the stereo camera can be electrically corrected, and the cost can be reduced.
There is an effect that an image processing apparatus using a stereo camera that is easy to adjust and does not change with time and has high reliability can be obtained.

According to claim 5 of the present invention, in claim 4, the shift amount correcting means shifts the output timing of one of the image signals sent to the image processing apparatus by the shift amount. Therefore, the image processing device can also be used as a device, the amount of deviation of the optical axis of the stereo camera can be electrically corrected, the cost can be reduced, and the adjustment is easy and does not change with time and is highly reliable. There is an effect that an image processing device using a stereo camera can be obtained.

[Brief description of drawings]

FIG. 1 is a block diagram schematically showing a part of a first embodiment of the present invention.

FIG. 2 is a perspective view showing an optical system of a general stereo camera.

FIG. 3 is a side view for explaining a conventional optical axis adjusting method.

[Explanation of symbols]

 1, 11 Lenses 2, 12 Light receiving element 5 Image signal controller 6 Deviation amount storage device 7 Image memory 8 CPU 100 Stereo camera 200 Image processing device G Image signal T Horizontal direction U Vertical direction (i, j) Deviation amount

Claims (5)

[Claims] 1. An image processing apparatus using a stereo camera including a stereo camera having a pair of video optical systems and an image processing apparatus for processing a pair of image signals obtained by the stereo camera, wherein the stereo camera is The image processing device stores a pair of image signals obtained from the stereo camera, the image processing device having a displacement amount storage unit for storing displacement amounts of horizontal and vertical optical axes with respect to a light receiving element in the video optical system. An image processing apparatus using a stereo camera, comprising: an image memory; and a shift amount correcting unit that corrects the shift amount by operating an address of an image signal on the image memory according to the shift amount. 2. The shift amount correcting means accesses one of the image signals on the image memory by using an address shifted by the shift amount.
Image processing device using a stereo camera. 3. The image using the stereo camera according to claim 1, wherein the shift amount correcting means stores one of the image signals in the image memory at an address shifted by the shift amount. Processing equipment. 4. An image processing apparatus using a stereo camera comprising a stereo camera having a pair of video optical systems and an image processing apparatus for processing a pair of image signals obtained by the stereo camera, wherein the stereo camera is An image processing apparatus using a stereo camera having shift amount storage means for storing shift amounts of optical axes in the horizontal and vertical directions with respect to a light receiving element in the video optical system, and shift amount correcting means for correcting the shift amount. . 5. The image using the stereo camera according to claim 4, wherein the shift amount correcting means shifts the output timing of one of the image signals sent to the image processing apparatus by the shift amount. Processing equipment.