JP2011090166A - Stereo imaging apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a stereo imaging apparatus enabling stereo imaging by a single camera. <P>SOLUTION: The stereo imaging apparatus includes one camera 4, first and second plane reflecting mirrors 5 and 6, and first and second concave lenses 15 and 16. The camera 4 has a camera lens 20 of an object-side telecentric optical system. The first and second plane reflecting mirrors 5 and 6 are arranged in front of the camera lens 20. The first and second concave lenses 15 and 16 are arranged corresponding to the first and second plane reflecting mirrors 5 and 6 respectively so that their optical axes 27 and 28 may be parallel with each other, and emit light beams 53 and 54 incident from an object side in parallel toward the first and second plane reflecting mirrors 5 and 6. The first and second plane reflecting mirrors 5 and 6 are arranged so that light beams incident from the first and second concave lenses 15 and 16 may be mutually made to be incident in parallel with the camera lens 20 as a principal light beam 51 on the object side. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a stereo imaging device.

  Stereo imaging devices that capture a pair of left and right parallax images are known.

Japanese Patent Laid-Open No. 2002-300603

  In the conventional stereo imaging apparatus, the distance from the camera to the subject can be measured by stereo viewing using parallax when the same subject at the same time is imaged by two cameras.

For example, as shown in FIG. 6, the distance (the distance between the lenses 103 and 104) between two cameras 101 and 102 arranged on the left and right (up and down in the figure) is _Lab , the camera focal length is f, and the subject Q If the coordinate positions on the imaging surfaces G _a and G ₂ are P _a (X _a , Y _a ) and P _b (X _b , Y _b ), respectively, when Y _a and Y _b are substantially equal, the camera 101 , 102 (lenses 103, 104) to the subject Q is calculated by the following equation.

d = L _ab × f / (X _a −X _b )
The imaging plane _G a, the coordinates position in the _{G b} is the optical axis _OA a, OA _b and each imaging plane _G a, the intersection _O a and _{G b,} the _{O b} is the origin, the left (the top in FIG. (Direction) is the positive direction of the x axis, and the vertical value is the coordinate value in each plane coordinate system with the positive direction of the y axis. In the example of FIG. 6, _Xa is a positive value and _Xb is a negative value.

  However, in the conventional stereo imaging device, one subject must be simultaneously imaged by two cameras. That is, two cameras are necessary and the imaging timings of both need to be matched.

  SUMMARY An advantage of some aspects of the invention is that it provides a stereo imaging device capable of performing stereo imaging with a single camera.

  In order to achieve the above object, a stereo imaging apparatus according to the present invention includes one camera, first and second planar reflecting mirrors, and first and second lenses. The camera has a camera lens that constitutes an object-side telecentric optical system or a double-sided telecentric optical system. The first and second planar reflecting mirrors are disposed in front of the camera lens. The first and second lenses are arranged corresponding to the first and second planar reflecting mirrors, respectively. The imaging surface of the camera has a first region where a mirror image is formed via a first plane reflecting mirror and a second region where a mirror image is formed via a second plane reflecting mirror.

  The first and second lenses are arranged so that their optical axes are parallel to each other, and emit light incident from the object side in parallel toward the first and second planar reflecting mirrors. The first and second planar reflecting mirrors are arranged so that the light rays incident from the first and second lenses are incident on the camera lens in parallel with each other as principal light rays on the object side.

  In the above configuration, the first image formed in the first region and the second image formed in the second region have parallax with each other. That is, stereo imaging can be performed with a single camera, and stereo matching processing can be performed by using these first and second images.

  The light beam incident on the first lens is emitted in a direction parallel to the optical axis (first optical axis) of the first lens, reflected by the first planar reflecting mirror, and image pickup optics. It enters the camera lens as a chief ray parallel to the optical axis of the system (camera lens). Similarly, the light beam incident on the second lens is emitted in a direction parallel to the optical axis (second optical axis) of the second lens, reflected by the second plane reflecting mirror, and imaged. It enters the camera lens as a chief ray parallel to the optical axis of the optical system. For this reason, even if it is a case where the 1st plane reflective mirror is made to adjoin or separate in the optical axis direction with respect to a camera lens, the 1st field of an imaging surface does not change. Similarly, even when the second planar reflecting mirror is brought close to or away from the camera lens in the optical axis direction, the second region 26 on the imaging surface does not change. That is, even if the distance (baseline) between the first optical axis and the second optical axis is changed, neither the first area nor the second area on the imaging surface is affected, and the same range. Maintained.

  Therefore, the baseline can be increased or decreased by moving the second plane reflecting mirror without changing the size of the second plane reflecting mirror and maintaining the range of the second region. That is, the change in the base line does not cause an increase in the size of the planar reflecting mirror (an increase in the size of the entire apparatus). In addition, by setting the base line long, the parallax of a distant object increases and the measurement accuracy can be improved.

  According to the present invention, stereo imaging is possible with a single camera.

It is a block diagram which shows the functional structure of the stereo imaging device of one Embodiment of this invention. It is a perspective view which shows typically the example of arrangement | positioning of the camera unit of FIG. It is a top view which shows typically the optical structure of the stereo imaging device of FIG. It is a flowchart which shows the process which the image calculating apparatus of FIG. 1 performs. It is a top view which shows typically the optical structure of a modification. It is a top view which shows typically the optical structure for demonstrating the measurement of the distance by a stereo vision method.

  Hereinafter, an embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing a functional configuration of the stereo imaging device of the present embodiment, FIG. 2 is a perspective view schematically showing an arrangement example of the camera unit of FIG. 1, and FIG. 3 is an optical diagram of the stereo imaging device of FIG. FIG. 4 is a flowchart showing processing executed by the image arithmetic apparatus shown in FIG. 1.

  As shown in FIG. 1, the stereo imaging device 1 of the present embodiment includes a camera unit 2 and an image arithmetic device 3.

  The camera unit 2 includes one camera 4, a plurality of (two in the present embodiment, first and second planar reflectors) planar reflectors 5 and 6 fixed in front of the camera 4, and each plane. And a plurality of lenses respectively corresponding to the reflecting mirrors 5 and 6 (first and second concave lenses 15 and 16 corresponding to the first and second planar reflecting mirrors in this embodiment). The image calculation device 3 executes a distance calculation process by stereo vision using the reception unit 7 that receives imaging data from the camera 4, a storage unit 8 that temporarily stores the received imaging data, and the stored imaging data. And an image processing arithmetic unit 9 for performing the processing.

  For example, as shown in FIG. 2, the camera unit 2 is fixed in a space on the roof of the cab 11 of the truck 10. The camera 4 images the front area of the vehicle input via the concave lenses 15 and 16 and the plane reflecting mirrors 5 and 6 at predetermined time intervals, and transmits the captured image data to the image calculation device 3.

  As shown in FIG. 3, an imaging optical system 21 including a camera lens 20 is disposed in the lens barrel of the camera 4. The camera lens 20 is a lens of an object side telecentric optical system disposed on the object side of the aperture stop 50. In the telecentric optical system, the aperture stop 50 exists at the focal position V of the lens, and the principal ray 51 on the object side is parallel to the optical axis 22 of the imaging optical system 21 (camera 4).

  In the camera unit 2, the first and second concave lenses 15 and 16 are arranged so that their optical axes (the first optical axis 27 and the second optical axis 28) are parallel to each other, and from the object side. The incident light rays 53 and 54 are emitted in parallel toward the first and second reflecting mirrors 5 and 6. The reflecting surfaces (mirror surfaces) 16 and 18 of the first and second reflecting mirrors are disposed in front of the camera lens 20 and receive the light rays incident through the first and second concave lenses 15 and 16 by the imaging optical system 21. The principal ray 51 on the object side parallel to the optical axis 22 is incident on the camera lens 20.

  On the optical axis 22 of the imaging optical system 21, an imaging surface 24 of an imaging element (for example, CCD) 23 is arranged perpendicular to the optical axis 22. In the present embodiment, of the range of the field angle 2θ on the image side of the imaging optical system 21, the range imaged through the reflecting surface (mirror surface) 17 of the first planar reflecting mirror 5 is centered on the focal position V. The range from the optical axis 22 to the angle θ in one direction and the range imaged through the reflecting surface (mirror surface) 18 of the second planar reflecting mirror 6 is from the optical axis 22 around the focal position V in the other direction. The range is set up to the angle θ. In other words, the imaging surface 24 includes a first region 25 where a mirror image is formed via the first plane reflecting mirror 5 and a second region where a mirror image is formed via the second plane reflecting mirror 6. 26 and divided into left and right. In the following description, the ranges reflected by the first and second reflecting surfaces 15 and 16 and directed forward in the imaging direction through the first and second concave lenses 15 and 16 are respectively defined as the first visual field and the second visual field. This is called a visual field.

  In the above configuration, the light beam incident on the first concave lens 15 is emitted from the first concave lens 15 in a direction parallel to the first optical axis 27, reflected by the first planar reflecting mirror 5, The light enters the camera lens 20 as a principal ray parallel to the optical axis 22 of the imaging optical system 21. Similarly, a light beam incident on the second concave lens 16 is emitted from the second concave lens 16 in a direction parallel to the second optical axis 28, reflected by the second planar reflecting mirror 6, and imaged. The light enters the camera lens 20 as a principal ray parallel to the optical axis 22 of the optical system 21. For this reason, even when the first planar reflecting mirror 5 is moved closer to or away from the camera lens 20 in the direction of the optical axis 22, the first region 25 of the imaging surface 24 does not change. Similarly, even when the second planar reflecting mirror 6 is moved closer to or away from the camera lens 20 in the direction of the optical axis 22, the second area 26 of the imaging surface 24 does not change. Therefore, even if the distance (baseline) B between the first optical axis 27 and the second optical axis 28 is changed, both the first region 24 and the second region 25 of the imaging surface 24 are affected. And remain in the same range.

  Stereo matching processing can be performed using the image of the first region 25 and the image of the second region 26. For stereo matching processing, in stereo vision, for a plurality of images with parallax, corresponding point search processing for finding a pair of pixels corresponding to the same part in the subject, and triangulation principle is applied to the corresponding points. Distance measurement processing for obtaining the distance to the subject.

  Next, processing executed by the image arithmetic device 3 will be described with reference to FIG.

  The image processing calculation unit 9 captures an image (image data) received by the receiving unit 7 from the camera 4 and temporarily stored in the storage unit 8 (step S1), and stores the captured image in the first area 25. A corresponding image (first image) and an image (second image) corresponding to the second region 26 are divided (step S2).

  Next, the stereo matching process is executed using the first and second images (step S3).

  If the influence of the radial distortion due to the concave lenses 15 and 16 is large, it is preferable to perform a correction process of the radial distortion on the captured image during the stereo matching process. For the correction process of the radiation distortion, a general method of performing calibration in advance, checking the characteristics of the optical system (concave lenses 15 and 16), and identifying parameters necessary for correction may be used.

  Further, the correction processing of the radial distortion can be omitted by using a plurality of lens groups that correct the radial distortion by an optical system, such as a low distortion lens, instead of the concave lenses 15 and 16.

  According to the present embodiment, the first image formed in the first region 25 and the second image formed in the second region 26 have parallax with each other. That is, stereo imaging can be performed with the single camera 4, and stereo matching processing can be performed by using these first and second images.

  Further, in an imaging optical system in which a general convex lens is used as a camera lens instead of the object side telecentric optical system and the concave lens is not provided, the imaging area on the imaging surface increases as the plane reflecting mirror is separated from the camera lens. Decrease. That is, when the plane reflecting mirror is separated from the camera lens while the imaging area is maintained in the same range, the plane reflecting mirror needs to be enlarged. For this reason, if an attempt is made to set a large baseline, the plane reflecting mirror must be enlarged, resulting in an increase in the size of the entire apparatus.

  On the other hand, in this embodiment, the light beam incident on the first concave lens 15 is emitted from the first concave lens 15 in a direction parallel to the first optical axis 27, and the first planar reflecting mirror 5. And enters the camera lens 20 as a principal ray parallel to the optical axis 22 of the imaging optical system 21. Similarly, a light beam incident on the second concave lens 16 is emitted from the second concave lens 16 in a direction parallel to the second optical axis 28, reflected by the second planar reflecting mirror 6, and imaged. The light enters the camera lens 20 as a principal ray parallel to the optical axis 22 of the optical system 21. For this reason, even when the first planar reflecting mirror 5 is moved closer to or away from the camera lens 20 in the direction of the optical axis 22, the first region 25 of the imaging surface 24 does not change. Similarly, even when the second planar reflecting mirror 6 is moved closer to or away from the camera lens 20 in the direction of the optical axis 22, the second area 26 of the imaging surface 24 does not change. That is, even if the distance (baseline) B between the first optical axis 27 and the second optical axis 28 is changed, both the first region 24 and the second region 25 of the imaging surface 24 are affected. And remain in the same range.

  Accordingly, the base line B can be increased or decreased by moving the second planar reflecting mirror 6 without changing the size of the second planar reflecting mirror 6 and maintaining the range of the second region 25. it can. That is, the change in the base line B does not cause an increase in the size of the planar reflecting mirror 6 (an increase in the size of the entire apparatus). Further, by setting the base line B long, the parallax of a distant object increases, and the measurement accuracy can be improved.

  Moreover, although the case where the camera lens of the object side telecentric optical system was used was demonstrated in the said embodiment, it may replace with this and the camera lens of a both-side telecentric optical system may be used. In the case of a bilateral telecentric optical system, an image side camera lens 30 is provided in the camera unit 2 in addition to the object side camera lens 20 as shown in FIG. The image-side camera lens 30 causes light rays emitted from the object-side camera lens 20 to enter the imaging surface 24 as image-side principal rays 55 parallel to the optical axis 22 of the imaging optical system 21. That is, both the object side principal ray 51 and the image side principal ray 55 are parallel to the optical axis 22 of the imaging optical system 21.

  In the above embodiment, the camera field of view is divided into two by the two plane reflecting mirrors 5 and 6, but the present invention is not limited to this, and the number of sets of concave lenses and plane reflecting mirrors is increased. As a result, the number of divisions of the camera view can be increased. For example, when three sets of concave lenses and plane reflecting mirrors are provided to divide the camera field of view into three and there are three fields of view (first to third fields of view), a region where these three fields of view overlap is generated. In this overlapped region, for example, even when the subject is present in the occlusion (invisible region) of the first visual field, the subject may be imaged by the second visual field and the third visual field. That is, when three or more sets of concave lenses and planar reflecting mirrors are arranged, three or more images having parallax can be obtained, and thus occlusion can be reduced as a whole.

  The stereo imaging apparatus of the present invention is useful for image processing using stereo vision.

1: Stereo imaging device 2: Camera unit 3: Image calculation device 4: Camera 5: First plane reflecting mirror 6: Second plane reflecting mirror 15: First concave lens 16: Second concave lens 20: Camera lens ( Object side camera lens)
21: Imaging optical system 22: Optical axis of imaging optical system 24: Imaging surface 25: First region 26: Second region 27: Optical axis of first concave lens 28: Optical axis of second concave lens 30: Camera Lens (camera lens on the image side)
50: aperture stop 51: principal ray on the object side 53: ray incident on the first concave lens 54: light ray incident on the second concave lens 55: principal ray on the image side

Claims (2)

One camera having a camera lens constituting the object side telecentric optical system or the both side telecentric optical system;
First and second planar reflecting mirrors disposed in front of the camera lens;
First and second lenses disposed corresponding to the first and second planar reflecting mirrors, respectively,
The imaging surface of the camera includes a first region where a mirror image is formed via the first plane reflecting mirror, and a second region where a mirror image is formed via the second plane reflecting mirror. Have
The first and second lenses are arranged so that their optical axes are parallel to each other, and emit light rays incident from the object side in parallel toward the first and second planar reflecting mirrors,
The first and second planar reflecting mirrors are arranged so that light rays incident from the first and second lenses are incident on the camera lens as parallel principal rays on the object side. The stereo imaging device characterized by the above-mentioned. The stereo imaging device according to claim 1,
A stereo comprising an image processing calculation unit that performs a stereo matching process using the first image formed in the first region and the second image formed in the second region. Imaging device.

Images