JP2010014699A - Shape measuring apparatus and shape measuring method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a shape measuring apparatus and shape measuring method for measuring a shape even when a part or the whole of an object to be measured comes out of an imaging region of a plurality of imaging means when the object to be measured is imaged using the plurality of imaging means such as cameras. <P>SOLUTION: This shape measuring apparatus picks up an image while moving the plurality of imaging means that have a predetermined positional relation and have different view points, and measures the distance to the object to be measured using the image. This apparatus includes an auxiliary imaging means that images the imaging region of the plurality of imaging means far from the point where the view points of the plurality of imaging means start to overlap and a region including the periphery of the imaging region, and moves integrally with the plurality of imaging means. When the image is picked up while the plurality of imaging means and the auxiliary imaging means are moved and the whole shape of the object to be measured is measured, the position of the object to be measured captured by the plurality of imaging means is tracked using the image obtained from the auxiliary imaging means. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a shape measuring apparatus based on the principle of photogrammetry, and moves a plurality of image pickup means, measures the distance to a measured object at each position of the image pickup means, and integrates the measurement results to obtain the object. The present invention relates to a shape measuring apparatus that measures the shape of a measurement object.

  When reproducing an object with computer graphics (CG), it is necessary to create its shape as three-dimensional data.

  If the target is an imaginary object, the designer or operator creates the shape data using various CG creation tools.

  In addition, when trying to convert a sample of a shape or prototype or an actual object into data, it is possible to create shape data by itself by looking at a picture or video, or by using some shape measuring device. There is a method of creating using measurement data.

  However, it takes a very heavy load to create the shape data by all manual operations. On the other hand, when creating measurement data by the latter shape measuring device, several devices are commercially available depending on the object and the required accuracy. For example, one of the simplest devices is to use a plurality of images with different viewpoints based on the principle of photogrammetry to determine the distance to each common point in the image. Software that reads and processes different images.

  When a stereo camera is used, camera parameters such as the distance between the two cameras, the focal length of each camera, and the direction of the optical axis are obtained in advance, and the image from each camera is subject to an object on the image. The distance to each point of the measurement object is obtained by the principle of triangulation. When performing shape measurement using such a stereo camera, the distance to each point of the object to be measured is obtained for the range that can be imaged at once, and the shape can be measured, but the shape of the entire object to be measured is known. In order to achieve this, it is necessary to image the object to be measured from all directions and to integrate the respective data into the same three-dimensional coordinate system.

  As a system that automates this, the stereo camera movement measurement system of Non-Patent Document 1 is realized. This is to capture the measurement object while moving the two-lens stereo camera, and calculate the 3D coordinates of the point to be obtained from the shape information obtained in each frame (the captured image obtained each time the stereo camera is moved) It is.

  As described above, the shape data of the entire object to be measured can be easily obtained, but if a part or the whole of the object to be measured goes out of the imaging area during imaging, the frame before and after that There arises a problem that errors occur in the shape information and coordinates of the measured object to be obtained.

  In order to prevent this, there is a method to increase the surrounding information of feature points of the measured object, which will be described later, that is, to extract feature points by performing edge detection and multi-resolution analysis using a wider range of information. It is done. However, if the information is increased, the calculation load increases and real-time processing cannot be performed. Real-time processing of about 15 fps (frame / second) is required to obtain the entire shape while moving the camera. For example, when the KLT method (Non-Patent Document 2) widely used in moving image analysis is used, real-time processing at about 15 fps is easy, but the SIFT method (non- In Patent Document 3), processing at 15 fps is difficult.

As described above, even when a part or the whole of the measured object goes out of the imaging area during imaging, the calculation load is not increased, and the shape information and coordinates of the measured object obtained from the frames before and after that are used. It was difficult to measure the shape without causing an error.

  In order to solve the above-mentioned problem, it is conceivable that the lens used for the stereo camera has a wider angle and the object is less likely to be removed from the imaging region, but at the same time, the accuracy of the shape measurement also decreases.

Hiroki Unten, Tomohito Masuda, Toru Mitsuhashi, Makoto Ando: "Examination of VR model automatic generation method by moving stereo camera", Transactions of the Virtual Reality Society of Japan, 12, 1, pp. 127-135 (2007) Bruce D. Lucas and Takeo Kanade.An Iterative Image Registration Technique with an Application to Stereo Vision.International Joint Conference on Artificial Intelligence, pages 674-679, 1981 David G. Lowe, "Distinctive image features from scale-invariant keypoints," International Journal of Computer Vision, 60, 2, pp. 91-110,2004

  In the present invention, when an object to be measured is imaged using an imaging means such as a plurality of cameras, the shape can be measured even when a part or the whole of the object to be measured is outside the imaging area of the plurality of imaging means. It is to provide a shape measuring apparatus and a shape measuring method.

  The invention according to claim 1 of the present invention captures an image while moving a plurality of imaging means having different viewpoints in a predetermined positional relationship, and measures the shape by measuring the distance to the object to be measured using the image. In the apparatus, the imaging region of the plurality of imaging units far beyond the point where the fields of view of the plurality of imaging units start to overlap and the region including the periphery of the imaging region are captured and moved together with the plurality of imaging units. An auxiliary imaging unit that captures images while moving the plurality of imaging units and the auxiliary imaging unit and measures the entire shape of the object to be measured, using the images obtained from the auxiliary imaging unit. The shape measuring apparatus is characterized in that the position of an object to be measured captured by an imaging means is tracked.

  The invention according to claim 2 of the present invention captures an image while moving a plurality of imaging means having different viewpoints in a predetermined positional relationship, and measures the shape by measuring the distance to the object to be measured using the image. In the apparatus, the imaging region of the plurality of imaging units far beyond the point where the fields of view of the plurality of imaging units start to overlap and the region including the periphery of the imaging region are captured and moved together with the plurality of imaging units. An auxiliary imaging unit that captures an image while moving the plurality of imaging units and the auxiliary imaging unit, and measures the entire shape of the object to be measured, using the image obtained from the auxiliary imaging unit. The shape measuring apparatus according to claim 1, wherein a change in a relative position and direction of the plurality of imaging units with respect to an object position is determined.

The invention according to claim 3 of the present invention captures an image while moving a plurality of imaging means having different viewpoints in a predetermined positional relationship, and measures the shape by measuring the distance to the object to be measured using the image. In the method, the imaging region of the plurality of imaging units far beyond the point where the fields of view of the plurality of imaging units start to overlap and the region including the periphery of the imaging region are captured and moved together with the plurality of imaging units. The auxiliary imaging means is used to take an image while moving the plurality of imaging means and the auxiliary imaging means, and the measured object captured by the plurality of imaging means using the image obtained from the auxiliary imaging means This is a shape measuring method characterized by tracking a position.

  The invention according to claim 4 of the present invention captures an image while moving a plurality of imaging means having different viewpoints in a predetermined positional relationship, and measures the shape by measuring the distance to the object to be measured using the image. In the method, the imaging region of the plurality of imaging units far beyond the point where the fields of view of the plurality of imaging units start to overlap and the region including the periphery of the imaging region are captured and moved together with the plurality of imaging units. An image is picked up while moving the plurality of image pickup means and the auxiliary image pickup means using the auxiliary image pickup means, and the plurality of image pickup means for the position of the object to be measured using an image obtained from the auxiliary image pickup means. It is a shape measurement method characterized by determining a change in relative position and direction.

  According to the shape measuring apparatus and the shape measuring method of the present invention, it is possible to track a feature point even when an object to be measured has deviated from the imaging area of the imaging means for shape measurement, and quickly shape the entire object. Can be measured.

The schematic diagram which shows one Embodiment of the shape measuring apparatus of this invention Schematic diagram showing the relationship of the imaging area of each imaging means The figure which shows the relationship between the to-be-measured object imaged in each imaging area, and the feature point extracted on a to-be-measured object Schematic diagram showing the relationship between the measured object imaged in each imaging area and the feature points extracted on the measured object Flow chart of shape measurement Flowchart diagram of another shape measurement method

  The best mode of the shape measuring apparatus and measuring method of the present invention will be described with reference to the drawings.

  FIG. 1 is a schematic diagram showing an embodiment of a shape measuring apparatus according to the present invention, and FIG. 2 is a schematic diagram showing a relationship between imaging regions of each imaging means in the shape measuring apparatus according to the present invention.

  FIG. 1A shows an example in which auxiliary imaging means 13 which is another imaging means according to the present invention is incorporated in addition to a set of imaging means 11 and 12 for shape measurement, and the upper part shows a side view. The lower part is a diagram showing a plane, and the auxiliary imaging unit 13 is installed between the pair of imaging units 11 and 12. FIG. 2B shows an example in which auxiliary imaging means 23, which is another imaging means according to the present invention, is incorporated in a frame 24 of an existing stereo camera including a pair of imaging means 21 and 22, and the upper side is a side view. In the figure, the lower part is a diagram showing a plane, and the auxiliary imaging means 23 is installed outside the frame 24 of the stereo camera in which a pair of imaging means 21 and 22 are incorporated.

  FIG. 2 shows a field A1 of the imaging means (11 or 21), a field B1 of the imaging means (12 or 22), and a field C1 of the field C1 of the auxiliary imaging means (13 or 23). Region C is shown. The imaging area D where the imaging area A and the imaging area B overlap is an area where the distance to the object to be measured can be measured. Here, the auxiliary imaging means is at a position far from the object to be measured from the point L where the visual field A1 of the imaging means (11 or 21) and the visual field B1 of the imaging means (12 or 22) begin to overlap, and the visual field is A1, B1. It is installed so that it crosses the outside line.

  FIG. 3 shows the relationship between the measured object imaged in each imaging area and the feature points extracted on the measured object. A part of the imaging area A and the imaging area B of each imaging unit overlaps, and the imaging area C is an area including the imaging area A, the imaging area B, and the periphery of the imaging area. .

  The image pickup means shown in FIG. 1 (a) or (b) is moved to pick up images shown by F-1, F-2, F-3, and F-4. F-1, F-2, F-3, and F-4 are in the order of captured frames (shown by skipping several frames as appropriate), and the imaging means (11 or 21), (12 or 22), ( 13 or 23) is moved and imaged, so that the position of the measured object to be imaged changes. For the sake of explanation, only one round object to be measured with a leaf-like object is shown as a representative for the purpose of explanation.

  The feature points on the measured object obtained for acquiring the shape information are a, b, c, d, and e, respectively. These feature points are obtained from edge detection, corner detection, and the relationship with surrounding pixels such as brightness and color difference from the surrounding pixels. In F-1, all five points are in the imaging area of all imaging means, but in F-2, points a and d and point e are outside the imaging area B, and in F-3, all five points are imaged. The point a, the point d, and the point e are also out of the imaging region A. Thereafter, in F-4, four points excluding the point a enter the imaging region of all the imaging systems, and the point a also enters the imaging region A and the imaging region C.

  When imaging is performed in this manner, for example, if a feature point d′-e ′ in the surrounding situation similar to the feature point indicated by d−e is in the vicinity thereof, d, e if the information used for feature point extraction is not sufficient May deviate from d ′ and e ′ when they deviate from the imaging area. For example, in F-2, what the imaging area A of the imaging means (11 or 21) recognizes as de is regarded as corresponding to that in the imaging area B of the imaging means (12 or 22). ―E 'is mistakenly recognized.

  However, in the present invention, the image pickup means (11 or 21) and the auxiliary image pickup means (13 or 23) for picking up an image including all areas of the image pickup area of the image pickup means (12 or 22) and the surrounding area are added. . Further, the camera parameters such as the distance between all the imaging means, the focal length, the direction of the optical axis, and the F value are obtained in advance, so that the coordinates on the image of the imaging area A and the imaging area B of each feature point are obtained. The relationship between the coordinates on the image, the coordinates on the image of the imaging area A and the coordinates on the image of the imaging area C, and the coordinates on the image of the imaging area B and the coordinates on the image of the imaging area C Can be obtained.

  This is schematically shown in FIG. As shown in F-1, the image of the imaging area A or the imaging area B on the plane at a certain distance from the imaging means is considered as an image projected onto the imaging area C. That is, the image of the imaging area A or the imaging area B can be regarded as a part of the image of the imaging area C, and the imaging area C even in the case where a feature point appears from each imaging area as shown in F-2. Tracking can be continued as long as it is on the image, and when these feature points return to the imaging region A and the imaging region B again, correct information can be obtained instantaneously without mistaking the feature points.

  The feature points are extracted by a method such as edge detection or corner detection in each of the images in the imaging region A, the imaging region B, and the imaging region C, and points corresponding to each image using a method such as block matching. Ask for. After that, by using corresponding point matching between frames together, it is possible to track feature points within each image and between images.

  FIG. 5 is a flowchart of shape measurement according to the present invention.

  FIG. 3 and FIG. 3 show the flow of shape measurement in an example in which the auxiliary imaging means 23 which is another imaging means according to the present invention is incorporated in the existing stereo camera 24 including the pair of imaging means 21 and 22 in FIG. It shows below using FIG.

  After START (S1), the setup of each imaging means, that is, the imaging area (view angle) and the installation position are determined (S2). Camera parameters such as the distance between two imaging means and auxiliary imaging means of a stereo camera as a plurality of imaging means, the focal length of each camera, and the direction of the optical axis are obtained (calibration) (S3). After completing the above preparation, an image is taken (S4). Next, feature points are extracted from the image of the measured object imaged by each imaging means (S5) (here, the feature points of points a, b, c, d, e in FIG. 3 are obtained from each imaging means). Next, image association between the imaging area A, the imaging area B, and the imaging area C obtained from each imaging means (matching each feature point in each imaging area) is performed (S6). Next, the distance between each imaging means and the feature points of the object to be measured is calculated to obtain shape data (S7, S8).

  Further, if the entire shape (S9) has not been completed, the respective imaging means are moved together to change the viewpoint (S10) and imaged (S11). In the same manner as described above, feature points are extracted from the images picked up by the respective image pickup means (S12), the distance between the image pickup means and the feature points is calculated, and shape data is acquired (S13, S14). Next, the same feature point captured this time and the feature point captured last time are tracked (S15), the feature points are associated between the frames in the same manner as described above (S16), and the shape data obtained in each frame is obtained. Integrate (connect to one coordinate system (inter-frame alignment)) (S17).

  In the feature point tracking (S15), some or all of the points a, b, c, d, and e are outside the imaging area A or the imaging area B as shown in F-2 and F-3 in FIG. 4, the image of the imaging area A or B can be regarded as a part of the image of the imaging area C as shown in FIG. it can.

  In this way, imaging, feature point extraction, feature point tracking, feature point association, distance calculation, and shape data acquisition are performed for all shapes (S9) while moving a plurality of imaging means and auxiliary imaging means together. The measurement ends (S18).

  In the above shape measurement, the method of measuring the shape by tracking the positions of the measured objects captured by the plurality of imaging means using the image obtained from the auxiliary imaging means has been described. Next, a method for measuring the shape of the object to be measured by determining changes in the relative positions and directions of the plurality of image capturing means with respect to the position of the object to be measured using an image obtained from the auxiliary imaging means will be described.

  After START (K1), the setup of each imaging means, that is, the imaging area (view angle) and the installation position are determined (K2). Camera parameters such as the distance between two imaging means and auxiliary imaging means of a stereo camera as a plurality of imaging means, the focal length of each camera, and the direction of the optical axis are obtained (calibration) (K3). After completing the above preparation, an image is taken (K4). Next, feature points are extracted from the image of the measured object imaged by each imaging means (K5) (here, the feature points at points a, b, c, d, e in FIG. 3 are obtained from each imaging means). Next, the image association between the imaging area A, the imaging area B, and the imaging area C obtained from each imaging means is performed (matching the feature points in each imaging area) (K6). Next, the relative positions and directions of the plurality of imaging means with respect to the feature points of the measurement object are determined, and shape data is acquired (K7, K8).

Further, when the entire shape (K9) has not been completed, the respective imaging means are moved together (K10) to take an image (K11) in order to change the viewpoint. A feature point is extracted from the image captured by each imaging unit in the same manner as described above (K12), changes in the relative positions and directions of the plurality of imaging units with respect to the object to be measured are determined, and shape data is determined based on the changes. Is acquired (K13, K14). Next, the feature point captured this time and the feature point captured last time are tracked (K15), the feature points are correlated between the frames in the same manner as described above (K16), and the shape data obtained in each frame is obtained. Integrate (connect to one coordinate system (inter-frame alignment)) (K17). When the process is completed for all shapes (Y of K9), the process is ended (END) (K18).

  As described above, according to the shape measuring apparatus according to the present invention, the plurality of the plurality of shape measuring imaging means separately from the plurality of shape measuring imaging means by the auxiliary imaging means that captures an area including the imaging regions of the plurality of shape measuring means. Even when the object to be measured has deviated from the imaging area of the imaging means, it is possible to track the feature point and measure the shape. Further, since the auxiliary imaging means according to the present invention is used for feature point tracking and distance information is obtained by the plurality of imaging means, shape measurement can be performed without reducing shape measurement accuracy.

11, 12... A set of shape measurement imaging means 13... Auxiliary imaging means 21, 22... A set of shape measurement imaging means 23. Field of view B1 of the imaging means 11 or 21 Field of view C1 of the imaging means of 12 or 22 Field of view A of the auxiliary imaging means of 13 or 23 Image area B of the imaging means of 11 or 21 .. Imaging area C of imaging means 12 or 22 Imaging area D of imaging means 13 or 23 Imaging area L where imaging area A and imaging area B overlap Field of view A1 and field of view B1 overlap Starting point

Claims (4)

  In an apparatus for capturing an image while moving a plurality of imaging means having different viewpoints in a predetermined positional relationship, measuring the distance to the object to be measured using the image, and measuring the shape, the field of view of the plurality of imaging means is The plurality of imaging units includes an auxiliary imaging unit that captures an imaging region of the plurality of imaging units beyond the point where overlap starts and an area including the periphery of the imaging region and moves integrally with the plurality of imaging units. When measuring the overall shape of the object to be measured while moving the image capturing means and the auxiliary imaging means, the positions of the objects to be measured captured by the plurality of imaging means are measured using the images obtained from the auxiliary imaging means. A shape measuring device characterized by being tracked.   In an apparatus for capturing an image while moving a plurality of imaging means having different viewpoints in a predetermined positional relationship, measuring the distance to the object to be measured using the image, and measuring the shape, the field of view of the plurality of imaging means is The plurality of imaging units includes an auxiliary imaging unit that captures an imaging region of the plurality of imaging units beyond the point where overlap starts and an area including the periphery of the imaging region and moves integrally with the plurality of imaging units. When measuring the overall shape of the object to be measured by moving an image and moving the auxiliary imaging means, the plurality of imaging means relative to the position of the object to be measured using the image obtained from the auxiliary imaging means A shape measuring apparatus characterized by determining a change in a general position and direction.   In the method of capturing an image while moving a plurality of imaging means having different viewpoints in a predetermined positional relationship, measuring the distance to the object to be measured using the image, and measuring the shape, the field of view of the plurality of imaging means is Using the auxiliary imaging means that images the imaging area of the plurality of imaging means beyond the point where overlap starts and the area including the periphery of the imaging area, and moves together with the plurality of imaging means, the plurality of imaging means A shape measurement characterized in that an image is captured while moving the imaging means and the auxiliary imaging means, and the positions of the measured objects captured by the plurality of imaging means are tracked using images obtained from the auxiliary imaging means. Method.   In the method of capturing an image while moving a plurality of imaging means having different viewpoints in a predetermined positional relationship, measuring the distance to the object to be measured using the image, and measuring the shape, the field of view of the plurality of imaging means is Using the auxiliary imaging means that images the imaging area of the plurality of imaging means beyond the point where overlap starts and the area including the periphery of the imaging area, and moves together with the plurality of imaging means, the plurality of imaging means An image is picked up while moving the image pickup means and the auxiliary image pickup means, and a change in relative position and direction of the plurality of image pickup means with respect to the position of the object to be measured is determined using the image obtained from the auxiliary image pickup means. A shape measuring method characterized by:

Images