JP2004163271A - Noncontact image measuring apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a noncontact image measuring apparatus capable of performing highly accurate photographic surveying in an object space under condition that a reference point can not be placed in the object space by enabling on-site calibration which eliminates both confining conditions on scale distances or altitude reference points in the object space and the need for a ground reference point. <P>SOLUTION: The noncontact image measuring apparatus comprises both a half mirror 20 for correspondingly directing both the photographing optical axis 16 of a digital camera 10 and the laser optical axis 18 of a laser range finder 14 toward points of measurement in the space to be measured and a personal computer 22 for computing external orientation elements and internal orientation elements of the digital camera 10 on the basis of acquired image coordinates and measured distances on six points of measurement or more in the space to be measured photographed from two locations or more by the digital camera 10 and to which the distances are measured by the laser range finder 14. The noncontact image measuring apparatus enables three-dimensional measurement on the space to be measured. <P>COPYRIGHT: (C)2004,JPO

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to a non-contact image measurement device capable of realizing simple digital photogrammetry using a high-resolution digital camera for consumer use or the like.
[0002]
[Prior art]
The change from analog to digital in photogrammetry is the transition from film to CCD (Electrical Coupling Device) sensor, eliminating the time loss inevitable in conventional film cameras of film development and printing, and real-time Image acquisition is now possible. In recent years, in particular, with the rapid development of sensor technology, the resolution of CCD sensors has been rapidly increasing, and a wide variety of high-resolution consumer digital cameras have been released. Is expected in many fields.
[0003]
Conventionally, techniques for associating a distance with a captured image include acquiring a grayscale image covering a wide real space, acquiring distance information from a camera for an arbitrary point in the image, and joining the images. A technique of an image creating apparatus with distance information in which a panoramic image is created by using a panoramic image and distance information of an object whose distance is measured in the panoramic image is recorded (Patent Document 1).
On the other hand, in photogrammetry which deals with obtaining the spatial position of a subject, that is, three-dimensional coordinates, there are types such as ground photogrammetry and aerial photogrammetry, but the principle on which they are based is the same. That is, it is a projection image. More specifically, photogrammetry measures the plane coordinates of a photograph, which is a central projection image, and further utilizes the geometrical condition of light connecting the subject, lens, and film (or image sensor) to determine the spatial position of the subject, that is, That is, to obtain three-dimensional coordinates. The measurement always contains some errors, and for the photographic side amount, the optimal value of the unknown variable must be obtained from the measured value containing various errors, and correction of lens distortion, focal length, principal point position, etc. It is considered that the internal orientation based on the internal orientation element of the parameter and the external orientation based on the external orientation element of the position of the photographing point and the inclination of the camera are important (Non-Patent Document 1).
[0004]
[Patent Document 1]
JP 2000-227963 [Non-patent document 1]
Published by the Japan Society of Photogrammetry (Revised in April 1997), published by the Analysis Photogrammetry Committee, "Analysis Photogrammetry"
[0005]
[Problems to be solved by the invention]
From the above background, the inventors have studied the application of a high-resolution digital camera for consumer use to digital photogrammetry, and have further developed software for digital photogrammetry using a consumer digital camera.
[0006]
However, in many methods at present, the internal orientation elements need to be acquired in advance, and further, to acquire the actual coordinates of the object, a scale distance or an elevation reference point is required in the object space. Is a bottleneck in the practical application of simple digital photogrammetry using a consumer digital camera.
[0007]
Note that, in Patent Document 1, distance information is simply associated with the specification of a captured image. When a panoramic image is created by joining 10 images, distance information for 10 points is recorded in the image. It will be. In Patent Document 1, as described on page 6, a projective transformation formula is used for superimposing images, but this transform formula is strictly called a two-dimensional projective transform formula, and Is a plane. Therefore, conversion to a real space having three-dimensional coordinates by this equation involves an error, and this error can be a large error that cannot be overlooked in the field of photogrammetry. On the other hand, in the field of photogrammetry, accurate three-dimensional coordinates in real space are required rather than representation on a computer monitor, and the level required in terms of accuracy is completely different.
[0008]
The present invention has been made to solve the above-described conventional problems, and enables on-site calibration that eliminates the need for a scale distance or a constraint condition of an elevation reference point and a ground reference point on a target space. Accordingly, it is an object of the present invention to provide a non-contact image measurement device that enables high-accuracy photogrammetry in a target space where a reference point cannot be set in the target space.
[0009]
[Means for Solving the Problems]
That is, the present invention has the following configuration in order to solve the above-mentioned problems.
The present invention provides a digital imaging unit for photographing a measurement target space including a measurement point, an image coordinate calculation unit for obtaining image coordinates of a measurement point in a captured image, and a non-contact measurement of a distance to a measurement point in the measurement target space A laser rangefinder, an optical axis matching unit for causing the imaging optical axis of the digital imaging unit and the laser optical axis of the laser rangefinder to coincide with each other and head toward a measurement point in the measurement target space, and two or more positions Calculating means for calculating an external orientation element and an internal orientation element of the digital imaging means, based on the determined image coordinates and the measured distance for six or more measurement points in the measurement target space which have been photographed and distance-measured; A storage means for storing a measurement distance, an image coordinate, an external orientation element, and an internal orientation element, thereby enabling three-dimensional measurement of a measurement target space. It is a measuring device.
Here, in order to accurately perform photogrammetry, it is necessary to correctly know the distance from the center of the imaging means to the measurement point in the measurement target space. For this reason, the camera and the laser range finder are not set so that the laser and the optical axis of the camera are parallel to each other as in the above-mentioned Patent Document 1, but in the present invention, the optical axis matching means such as a half mirror is used. The optical axis of the digital imaging means and the optical axis of the laser distance meter are completely matched. Further, in the present invention, the distance is measured by photographing from two or more positions, and six or more measurement points (preferably, for example, easily recognized on a monitor image) in the measurement target space by the digital imaging means, the laser distance meter, and the coordinate calculation means. In the following, this measurement point is referred to as a pseudo reference point, that is, referred to as a “pseudo reference point”. The external orientation element and the internal orientation element can be calculated on the basis of the distance measured by the laser range finder, and thus the calibration of the digital imaging means can be performed.
Therefore, the pseudo reference point can be appropriately set in the measurement target space without separately setting a reference point in the measurement target space, so that natural terrain and buildings which could not be measured because the conventional reference point could not be provided. Can be easily and accurately performed. That is, the inventor has developed an image measurement system that integrates a 3 million pixel digital camera for consumer use, a laser range finder, and a personal computer, and sets a reduced distance or an altitude reference point constraint condition and a ground reference point on the target space. This enables on-site calibration that eliminates the need. Therefore, it is possible to construct a system capable of performing precise three-dimensional measurement and three-dimensional expression of a target space on site.
[0010]
The optical axis coincidence unit is a half mirror provided on an optical axis between the measurement target space and the digital imaging unit, and transmits the half mirror to define a measurement target space of the digital imaging unit. It is preferable to have the half mirror for making the optical axis toward the measurement point coincide with the laser optical axis reflected by the half mirror and toward the measurement point in the measurement target space. This makes it possible to optically and precisely match the optical axes of the digital imaging means and the laser range finder with a simple structure of a half mirror.
Monitor means for displaying an image photographed by the digital imaging means on a screen (for example, as a finder image) and displaying a laser beam irradiation point of the laser distance meter on the display screen; and optical axes of the digital imaging means and the laser distance meter And an optical axis adjusting means for adjusting at least one of the optical axes. In the non-contact image measurement device, when performing photogrammetry, the optical axis is adjusted in advance and then brought to the survey site, but it is desirable that the optical axis can be adjusted again at the site. Thereby, a real-time image is acquired, calibration at the imaging site (on-site calibration) and an image in real time are acquired, and three-dimensional measurement (3D measurement) of the target space can be performed.
In addition, the digital imaging unit, the laser range finder, the storage unit, the half mirror, the monitor unit, the optical axis adjusting unit, and the data processing unit are provided in an integrated housing, and the housing is biaxially rotated in the turning direction and the elevation direction. It is preferable to provide a supporting means that supports as much as possible (provides a dedicated tripod or the like that is not capable of biaxial rotation). In this way, the oblique distance from the digital imaging means to the measurement point in the measurement target space can be measured, and the digital imaging means, the laser distance meter, the monitoring means, and the like are housed in an integrated housing and integrated. This makes it possible to acquire images in real time, and perform on-site calibration and 3D measurement more easily and accurately. Further, the non-contact image measurement device of the present invention performs photographing and distance measurement of the target space from two positions. In addition, it is possible to perform surveying while looking at the monitor while maintaining the state where the optical axes of the digital imaging means and the laser distance meter are aligned. Therefore, handling of the non-contact image measuring device itself is easy and the surveying accuracy is improved.
[0011]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
FIG. 1 is an explanatory block diagram of the non-contact image measurement device according to the embodiment.
As shown in FIG. 1, the non-contact measurement device captures an image of a measurement target space including a measurement point and has a digital camera (digital imaging means) 10 having a zoom function and an image coordinate of the measurement point in the captured image. A coordinate calculation means 12 such as a personal computer 22 to be determined, a laser distance meter 14 for measuring a distance to a measurement point in a measurement target space in a non-contact manner, a photographing optical axis 16 of the digital camera 10 and the laser distance meter 14 A half mirror (optical axis matching means) 20 for causing the laser optical axis 18 to coincide with each other and head toward the measurement point in the measurement object space, taking an image with the digital camera 10 from two or more positions, and using the laser distance meter 14 External orientation elements of the digital camera 10 based on the obtained image coordinates and the measurement distance for the six or more measurement points in the measurement target space where the distance was measured. A personal computer to calculate the internal orientation parameters and: have (PC example of an arithmetic unit) 22, and makes it possible to three-dimensional measurement of the measurement object space.
[0012]
Further, the non-contact image measuring device is a memory (storage means) for storing an image taken by the digital camera 10, a measurement distance of the laser range finder 14, a calculated image coordinate, a calculated external orientation element, an internal orientation element, and the like. 24, a monitor (monitor means) 26 composed of a liquid crystal display element for displaying an image captured by the digital camera 10 on a screen and displaying a laser beam irradiation point of the laser rangefinder 14 on the display screen; And an optical axis adjusting means comprising a rotatable mirror 32 capable of adjusting at least one of the optical axes of the laser range finder 14.
[0013]
The half mirror 20, which is the optical axis matching means, is a half mirror 20 provided on the optical axis 16 of the digital camera 10 facing the measurement target space, and the half mirror 20 (the front surface 20f, the back surface 20r). The laser beam emitted from the laser range finder 14 is reflected by the surface 20f of the half mirror 20, and the imaging optical axis 16 of the imaging field of view toward the measurement point in the measurement object space of the digital camera 10 Is a half mirror 20 for matching the laser optical axis 18 toward the measurement point.
[0014]
In the non-contact image measuring apparatus according to the embodiment, the digital camera 10, the laser distance meter 14, the memory 24, the half mirror 20, the monitor 26, the mirror 32 of the optical axis adjusting unit, and the calculating unit 22 are provided in an integrated housing 28. At the same time, the housing 28 is supported to be biaxially rotatable in a turning direction (a turning direction axis 30A) and a vertical elevation direction (an elevation direction axis 30B) on a horizontal plane (a tripod capable of biaxial rotation is used). ) A support structure 30 is provided.
Although the laser emission direction of the laser range finder 14 provided in the housing 28 is parallel to the imaging optical axis of the digital camera 10, the laser A mirror 32 is provided so as to be rotatable about the axis, and the laser light 14A emitted from the laser distance meter is reflected by the mirror 32 to guide the laser light 14A to the surface 20f of the half mirror 20. By adjusting the angle position of the mirror 32 and adjusting the optical axis 18 of the laser beam 14A, the laser optical axis 18 and the photographing optical axis 16 can be matched.
[0015]
The personal computer 22 for data processing for displaying an image on the monitor 26 includes a hard disk drive (HDD) 24A as a memory (storage means) 24, a random access memory (RAM) 24B, and an arithmetic processing means. A central processing unit (CPU) 22A is provided on a motherboard 22B and housed in the housing 28.
Specifically, the digital camera has 3.14 million pixels, the laser distance meter 12 has a distance accuracy of ± 3 mm up to 40 m, and a video capture board 22C connected to a motherboard 22B for transferring a finder image.
[0016]
A method for obtaining an external orientation element and an internal orientation element in the non-contact image measurement device according to the embodiment will be described.
When measuring an image, it is necessary to correctly know the distance from the center of the digital camera to the target space. Therefore, the half mirror 20 is positioned so that the laser optical axis 18 and the optical axis (photographing optical axis) 16 of the digital camera 10 completely match. Used. The monitor 26 can also be used as a finder of the digital camera 10.
[1] First, an image of the measurement target space by the digital camera 10 is taken into the personal computer 22 from a first position where the measurement target space can be seen.
[2] Then, the housing 28 accommodating the system of the non-contact image measurement device is rotated on the support structure 30 with respect to six characteristic points (pseudo reference points) in the measurement target space, and the laser distance meter for the points is provided. The distance is measured by 14.
[3] It is possible to aim at the measurement point through the monitor (finder) 26 and use the zoom function of the digital camera 10 to make the point irradiated with the laser beam 14A exactly coincide with the measurement point. Further, in order to prevent an error due to the rotation of the system, the center of rotation of the system is coincident with the center of the camera axis.
[4] The image coordinates of the measurement point can be obtained by image processing by the personal computer 22 or manually. The obtained image coordinates are recorded in the memory 24 of the personal computer 22 in association with the distance.
[5] After acquiring the image coordinates and the distances for the six or more measurement points (feature points), the personal computer 22 calculates the calibration of the digital camera (the external orientation element and the internal orientation element of the digital camera). .
[6] The system of the non-contact image measurement device such as the digital camera 10 is moved from the first position, installed at the second position, and the operations from [1] to [5] are performed.
[7] From now on, as in the conventional photogrammetry, three-dimensional coordinates of a measurement target space are calculated for stereo images taken and measured at two positions by a bundle adjustment method, and the existing CG software (computer graphics Software)) to enable 3D representation.
[0017]
Next, the calculation of the calibration of the digital camera (the external orientation element and the internal orientation element of the digital camera) will be described.
The non-contact image measurement apparatus according to the present embodiment enables an on-site calibration method (calibration at the site to be measured) that does not require installation of a ground reference point, and is obtained by the non-contact image measurement apparatus. The left and right stereo images obtained at the two positions are simultaneously adjusted using the distance as a constraint condition. The left and right camera positions are three-dimensional coordinates (X0L, Y0L, Z0L), (X0R, Y0R, Z0R), the left and right camera postures. (ΩL, ψL, κL), (ωR, ψR, κR), camera focal length f, principal point position x0, y0, distortion count p1, conversion coefficients a1, a2 from sensor to screen coordinates, and pseudo reference point The dimensional coordinates are unknown quantities. The pseudo reference point is a point in the measurement target space at which the distance used for calibration (calibration) is measured. For on-site calibration, six or more points are included in the measurement target space. A pseudo reference point is required. In this case, 36 conditional expressions are obtained for 30 unknown quantities.
[0018]
explain in detail. In the above case, in image acquisition, six external orientation elements (the position of the imaging point and the inclination of the camera) are unknown quantities. Since the photographing is performed from two places, twelve external orientation elements for the left and right cameras are unknown quantities. Six internal orientation elements (principal point positions x0 and y0, focal length f, affine transformation coefficients a1 and a2, and lens distortion coefficient p1) are unknown quantities. Strictly speaking, these values are only small for the left and right cameras. Although slightly different, we will assume that these values do not change this time. Therefore, a total of 18 left and right external orientation elements and internal orientation elements are unknown quantities.
[0019]
Next, distance measurement is performed for six pseudo reference points in the measurement target space. Since the three-dimensional coordinates for the pseudo reference points are also unknown quantities, 6 × 3 = 18 unknown quantities are used. And a total of 36 unknowns are obtained.
On the other hand, since two collinear conditional expressions are obtained for one pseudo reference point, 12 for six pseudo reference points from one photograph are solved simultaneously for the left and right images. As a result, a total of 24 conditional expressions are obtained.
[0020]
Further, since the distance measurement is performed for the same pseudo reference turning point in the target space at the left and right camera positions, one distance conditional expression (distance constraint expression) is obtained for one pseudo reference turning point. Thus, six distance conditional expressions can be obtained with one image, and 6 + 6 = 12 distance conditional expressions can be obtained by simultaneously processing the left and right images.
Therefore, the number of conditional expressions is 24 + 12 = 36, where the number of unknowns and the number of equations are equal, and the problem can be solved.
[0021]
In the above case, the unknown quantity is 30 pieces. This assumes that, for example, one of the pseudo reference points is the origin (0, 0, 0) in order to reduce the number of unknown quantities. Then, since the three-dimensional coordinates for the pseudo reference point are known, 33 is obtained by subtracting 3 from the unknown quantity. In addition, if the X-axis and the Y-axis are determined in the measurement target space, three unknown quantities can be further geometrically reduced, and finally, 30 unknown quantities are obtained.
Normally, 30 unknowns can be solved if there are 30 equations, but in this case, nonlinear equations will be treated. Solve by the least squares method. The non-contact measurement is enabled by setting the condition of the distance, but the point of the solution is that the left and right cameras, that is, the images from two places are handled at the same time.
[0022]
The inventor performed stereo photography of a test target T as shown in FIG. 3 in order to verify the application of the non-contact image measurement device embodying the present invention to image sensing. The test target T was provided with 42 circular black circle D targets, and the accuracy of three-dimensional coordinates of the measured 42 black circles D (side points) was verified. The test target T has a length of 1.8 m, a width of 1.55 m, a depth Z (central two rows) of 0.33 m, and is provided with seven vertical and six black circles D at equal intervals. Further, a reflective seal E is attached to the center of each black circle D, and the center of the reflective seal E is measured with a first-class total station (distance accuracy ± 1 mm, angular accuracy ± 2 ″) to indicate the black circle D. True value.
The image coordinates of each test target T are obtained by subjecting the captured image to binarization processing, extracting only the black circle D, and determining the area center of gravity of the black circle.
FIG. 4 shows a shooting situation in stereo shooting.
[0023]
On the other hand, FIG. 5 shows a result obtained by performing on-site calibration by the non-contact image measurement device embodying the present invention, obtaining a mean square error (RMSE1) for the 42 side points (black circles), and a reference value. The figure shows the result obtained by performing camera calibration by the conventional bundle adjustment method using nine points and calculating the mean square error (RMSE2) for 33 side points.
As shown in FIG. 5, the results obtained by the non-contact image measurement device according to the present invention are slightly lower in both the plane and the depth accuracy than the results obtained by the conventional bundle adjustment method using the reference points. The reason for this is presumed that the result of the non-contact image measurement device is the accuracy of the measurement distance and the imperfection of the coincidence between the optical axis and the laser axis. Considering that there is no need for labor and time-consuming ground reference point installation work, that on-site 3D (three-dimensional) measurement is possible, and that on-site modeling is also possible, It can be understood that this is an extremely effective method in the field of image sensing using a digital camera.
[0024]
【The invention's effect】
As described above, according to the present invention, it is possible to perform on-site calibration that does not require the restriction condition of the scale distance or the elevation reference point and the ground reference point on the target space, and place the reference point in the target space. High-accuracy photogrammetry can be performed in a target space with no conditions
In the present invention, a monitor means for displaying a photographed image and a photographing optical axis by the digital photographing means and displaying a laser beam irradiation point of the laser distance meter on the displayed image, and adjusting at least an optical axis of the laser distance meter. And an optical axis adjusting means for performing the operation, the imaging optical axis and the laser beam irradiation point of the laser distance meter can be displayed on the monitor means. Real-time image acquisition and calibration at the imaging site (on-site calibration) and real-time image acquisition, enabling three-dimensional measurement (3D measurement) of the target object Become.
A data processing unit for storing the captured image data and the distance data in the memory unit and displaying the image on the monitor unit; a digital imaging unit, a laser distance meter, a memory unit, a half mirror, a monitor unit, and an optical axis adjusting unit , And data processing means are provided in an integral housing, and support means (using a dedicated tripod capable of rotating in two axes) is provided for supporting the housing so as to be capable of rotating in two directions in a turning direction and an elevation direction. In this way, the oblique distance from the digital imaging means to the measurement point in the measurement target space can be measured, and the digital imaging means, the laser distance meter, the monitoring means, and the like are housed in an integrated housing. And integrated into a real-time image via a data processing means such as a video capture board. And possible 3D measurement even more easily and accurately.
[Brief description of the drawings]
FIG. 1 is an explanatory block diagram of a non-contact image measurement device according to an embodiment of the present invention.
FIG. 2 is an explanatory diagram of attachment of the non-contact image measurement device of FIG. 1 to a support unit.
FIGS. 3A and 3B are external explanatory views of a test target for verifying the non-contact image measurement device according to the present invention, wherein FIG. 3A is an overall view of the test target and FIG. 3B is an explanatory view of an individual target (black circle). It is.
FIG. 4 is an explanatory diagram of a shooting situation for verifying the non-contact image measurement device according to the present invention.
FIG. 5 shows a result of performing on-site calibration by a non-contact image measurement device embodying the present invention and a result of performing camera calibration by a conventional bundle adjustment method.
[Explanation of symbols]
10. Digital video camera (digital imaging means)
12 Coordinate calculation means 14 Laser distance meter 16 Photographing optical axis 18 Laser optical axis 20 Half mirror (optical axis matching means)
22 Personal computer (arithmetic means)
22A Central processing unit (CPU)
22B Motherboard 24 Memory (storage means)
24A Hard Disk Drive (HDD)
24B random access memory (RAM)
26 Memory 28 Housing 30 Support structure 30A Revolving direction axis 30B Elevation direction axis 32 Mirror D Test target (black circle)
T test target

Claims (4)

Digital imaging means for photographing a measurement target space including a measurement point,
Image coordinate calculating means for obtaining image coordinates of a measurement point in a captured image;
A laser distance meter that measures the distance to the measurement point in the measurement target space without contact,
Optical axis matching means for causing the photographing optical axis of the digital imaging means and the laser optical axis of the laser distance meter to coincide with each other so as to be directed to a measurement point in the measurement target space,
Calculating means for calculating an external orientation element and an internal orientation element of the digital imaging means based on the obtained image coordinates and measurement distances for the six or more measurement points in the measurement target space which are photographed from two or more positions and distance measured; ,
A non-contact image measurement apparatus, comprising: a storage unit for storing a captured image, a measurement distance, an image coordinate external orientation element, and an internal orientation element, and enabling three-dimensional measurement of a measurement target space. The optical axis matching unit is a half mirror provided on the optical axis between the measurement target space and the digital imaging unit, and transmits through the half mirror to a measurement point of the measurement target space of the digital imaging unit. 2. The non-contact image measurement device according to claim 1, further comprising the half mirror for matching an optical axis going to the laser axis reflected by the half mirror to a measurement point in a measurement target space. 3. Monitor means for displaying an image captured by the digital imaging means on a screen and displaying a laser beam irradiation point of a laser distance meter on the display screen,
3. The non-contact image measuring apparatus according to claim 1, further comprising: an optical axis adjusting unit configured to adjust at least one of an optical axis of the digital imaging unit and an optical axis of the laser range finder. A digital imaging unit, a laser range finder, a storage unit, a half mirror, a monitor unit, an optical axis adjusting unit, and a data processing unit are provided in an integrated housing, and the housing is rotatable in two directions in a turning direction and an elevation direction. The non-contact image measuring apparatus according to claim 1, further comprising a supporting unit that supports the shaft.

Images