JP5297779B2 - Shape measuring apparatus and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To automatically obtain measurement values including an initial value necessary for three-dimensional shape measurement by automatically determining an erroneous corresponding point in an overlapping image. <P>SOLUTION: The shape measuring device 1 includes: image pickup parts 2 to 9 picking up images of a measurement object 18 in an overlapping image pickup area; a feature point correspondence part 21 for performing the correspondence of the positions of the feature points of the measurement object 18 picked up by the image pickup parts 2 to 9; a parallax measurement part 22 obtaining the parallax of the feature points subjected to the correspondence by the feature point correspondence part 21 in the overlapping image; a three-dimensional coordinate calculation part 23 obtaining the three-dimensional coordinates of the feature points of the measurement object 18 on the basis of the parallax obtained by the parallax measurement part 22; an erroneous corresponding point determination part 24 determining the erroneous corresponding point on the basis of the parallax obtained by the parallax measurement part 22 and the three-dimensional coordinates obtained by the three-dimensional coordinate calculation part 23 or the form of the measurement object 18 in a three-dimensional coordinate system; and a three-dimensional shape measurement part 25 obtaining the three-dimensional shape of the measurement object 18 on the basis of the positions or the like of the feature points excluding the point determined as the erroneous corresponding point by the erroneous corresponding point determination part 24. <P>COPYRIGHT: (C)2010,JPO&amp;INPIT

Description

  The present invention relates to a shape measurement technique for measuring a three-dimensional shape of a measurement object based on overlapping images obtained by photographing the measurement object from a plurality of photographing positions, and in particular, initial values necessary for measurement of the three-dimensional shape. The present invention relates to a technique for automatically acquiring measured values.

  Conventionally, the theory of photogrammetry has been studied. In recent years, a technique for measuring the three-dimensional shape of an object to be measured from overlapping images taken from a plurality of photographing positions using the theory of photogrammetry has been disclosed. In order to measure the three-dimensional position of the measurement object, it is necessary to associate six or more points in the left and right images, but this processing must be performed manually or automatically by attaching a mark to the measurement object. was there.

  Further, in order to measure the three-dimensional shape of the measurement object, stereo matching is performed on the pixels of the measurement object. For stereo matching, least-square matching (LSM) for searching while deforming a template image, a normalized correlation method, or the like is used. This process requires many points and lines associated with the left and right images, but manually setting initial values of these points and lines is cumbersome and involves skills.

  For example, Patent Documents 1 and 2 disclose techniques for solving such problems. In the invention described in Patent Document 1, a pair of first photographed images obtained by photographing a measurement object provided with a reference feature pattern from different directions and a measurement object provided with no reference feature pattern are first. Based on a pair of second captured images captured from the same direction as the captured direction of one captured image, a feature pattern is extracted by taking the difference between the first captured image and the second captured image obtained in each direction.

  According to this aspect, since an image of only the feature pattern can be created, the position of the feature pattern can be automatically and accurately detected. Also, by increasing the number of feature pattern points, it is possible to automatically detect the corresponding surface in the left and right images.

Further, in the invention described in Patent Document 2, position correction is performed between the photographing position of the measurement target and the reference position of the measurement target determined by the design data, and the three-dimensional shape of the measurement target is compared with the design data. , The miscorresponding points associated with each other are deleted. According to this aspect, the three-dimensional shape measurement process can be automated.
JP 10-318732 A JP 2007-212430 A

  In view of such a background, the present invention provides a technique for automatically acquiring measurement values including initial values necessary for measurement of a three-dimensional shape by automatically determining erroneous correspondence points in overlapping images. With the goal.

  According to the first aspect of the present invention, the imaging unit that images the measurement object in the overlapping imaging region from a plurality of imaging positions, and the position of the feature point of the measurement object in the overlapping image captured by the imaging unit Based on the parallax obtained by the parallax measurement unit, a parallax measurement unit that obtains parallax in an overlapping image of the feature points associated by the feature point association unit, and a feature point of the measurement target Based on the three-dimensional coordinate calculation unit for obtaining the three-dimensional coordinates, the parallax obtained by the parallax measurement unit, and the three-dimensional coordinates obtained by the three-dimensional coordinate calculation unit or the form of the measurement object in the three-dimensional coordinate system An incorrect corresponding point determining unit for determining an erroneous corresponding point, and the position of the feature point excluding the point determined as an erroneous corresponding point by the erroneous corresponding point determining unit and the plurality of photographing positions of the measurement object. 3D coordinates of feature points Others are shape measuring apparatus characterized by comprising: a three-dimensional shape measurement unit for obtaining a three-dimensional shape of the measurement object.

  According to the first aspect of the present invention, it is possible to automatically acquire a measurement value including an initial value necessary for measuring a three-dimensional shape by automatically determining an erroneous correspondence point in an overlapping image.

  According to a second aspect of the present invention, in the first aspect of the present invention, based on the calibration subject provided with a reference point and the position of the corresponding reference point in a duplicate image obtained by photographing the calibration subject, the imaging And a photographing position / orientation measuring unit that obtains the photographing position and orientation of the unit.

  According to the second aspect of the present invention, the photographing position and orientation of the photographing unit are determined using the first method in which the calibration subject is photographed before photographing the measurement object and the position of the photographing unit is obtained in advance. Can be sought.

  The invention described in claim 3 is the invention described in claim 1, further comprising at least one of a horizontal angle measuring unit and a vertical angle measuring unit for measuring a shooting position of the shooting unit.

  According to the third aspect of the present invention, it is not necessary to obtain the photographing position and orientation of the photographing unit based on the image obtained by photographing the calibration subject.

  According to a fourth aspect of the present invention, in the first aspect of the present invention, the calibration subject provided with a reference point in a predetermined positional relationship and the position of the corresponding reference point in the overlapping image obtained by photographing the calibration subject. And a photographing position / orientation measuring unit for obtaining a photographing position and a posture of the photographing unit.

  According to the invention described in claim 4, by using the second method of photographing the calibration subject together with the measurement object and obtaining the position of the photographing unit and the three-dimensional position of the measurement object in parallel, The photographing position and posture of the photographing unit can be obtained.

  According to a fifth aspect of the present invention, in the invention according to any one of the second to fourth aspects, the feature point association unit is a feature in at least a pair of stereo images captured by separating the imaging unit by a predetermined baseline length. The point positions are associated with each other.

  According to the fifth aspect of the present invention, the three-dimensional shape of the measurement object can be measured based on at least a pair of stereo images.

  The invention according to claim 6 is the invention according to any one of claims 2 to 4, wherein the miscorresponding point determination unit creates a histogram of parallax and corresponds to parallax that is not within a predetermined range from the average of parallax. The feature point to be determined is determined as an erroneous correspondence point.

  According to the sixth aspect of the present invention, it is possible to determine an erroneous correspondence point at a position away from the measurement object.

  The invention according to claim 7 is the invention according to any one of claims 2 to 4, further comprising a feature projection unit that projects the feature onto the measurement object.

  According to the seventh aspect of the invention, since the feature enters the portion where the feature of the measurement object is poor, many feature points can be detected.

  The invention according to claim 8 is the invention according to claim 4, wherein when the parallax measurement unit obtains the parallax of the feature point associated by the feature point association unit, the position near the feature point is determined. The parallax of the feature point is estimated based on the parallax of the known reference point.

  According to the invention described in claim 8, when associating the position of the feature point in the overlapped image, it is possible to search for the corresponding point in the horizontal direction or the vertical direction from the position corresponding to the estimated parallax of the feature point.

  The invention according to claim 9 is the invention according to claim 4, wherein the miscorresponding point determination unit determines whether the feature point is a miscorresponding point based on a parallax of a reference point whose position near the feature point is known. It is characterized by determining whether or not.

  According to the ninth aspect of the present invention, it is possible to determine a miscorresponding point based on the parallax of the reference point near the feature point.

  The invention according to claim 10 is the invention according to claim 1, wherein the miscorresponding point determination unit determines an miscorresponding point based on at least one of horizontal parallax and vertical parallax.

  According to the tenth aspect of the present invention, it is possible to perform shooting with vertical parallax removed, shooting with horizontal parallax removed, or shooting with both vertical and horizontal parallax removed.

  The invention described in claim 11 is the invention described in claim 1, wherein the three-dimensional shape measurement unit measures a three-dimensional shape using the inside of the contour of the measurement object as a processing region.

  According to the invention described in claim 11, an unnecessary three-dimensional shape is not formed in the contour portion of the measurement object.

  The invention according to claim 12 is characterized in that, in the invention according to any one of claims 2 to 4, when the miscorresponding point determination unit determines that it is a miscorresponding point, it corresponds to the miscorresponding point. The point designation is canceled.

  According to the twelfth aspect of the present invention, it is possible to obtain the three-dimensional coordinates of the feature points excluding the miscorresponding points.

  According to a thirteenth aspect of the present invention, in the first aspect of the invention, the feature point association unit obtains a parallax of a feature point based on a known depth amount, and determines an erroneous correspondence point based on the parallax. It is characterized by doing.

  According to the thirteenth aspect of the present invention, it is possible to determine an erroneous correspondence point when the depth amount is known.

  The invention according to claim 14 is the invention according to claim 1, further comprising a contour extracting unit that extracts a contour of the measurement object in the overlapping image, and the feature point association unit is extracted by the contour extracting unit. The feature points are associated with each other in a region inside the contour.

  According to the fourteenth aspect of the present invention, since the feature points are not extracted from the non-overlapping regions in the overlapping image, the erroneous corresponding points in the vicinity of the contour are not extracted.

  The invention according to claim 15 is the invention according to claim 2 or 4, wherein the miscorresponding point determination unit is obtained by the three-dimensional coordinate calculation unit from the center of gravity of a reference point provided in the calibration subject. The feature point is determined to be a miscorresponding point based on the distance to the three-dimensional coordinates of the point.

  According to the fifteenth aspect of the present invention, it is possible to determine an erroneous correspondence point at a position away from the measurement object.

The invention according to claim 16 is the invention according to claim 1, wherein the three-dimensional shape measurement unit performs a process of removing the miscorresponding point performed when the initial value is obtained by the miscorresponding point determination unit. Then, it is configured so as to be performed when a measurement value is obtained thereafter.

According to invention of Claim 16 , the reliability of the measurement at the time of calculating | requiring a measured value in a three-dimensional shape measurement part improves.

The invention according to claim 17 is a feature point associating step for associating the positions of the feature points of the measurement object in the overlapped images taken from a plurality of photographing positions, and the feature points associated in the feature point associating step. A parallax measurement step for obtaining a parallax in an overlapping image, a three-dimensional coordinate calculation step for obtaining a three-dimensional coordinate of a feature point of the measurement object based on the parallax obtained in the parallax measurement step, and a three-dimensional coordinate calculation step. The position of the feature point excluding the erroneous correspondence point determination step for determining the erroneous correspondence point based on the form of the measurement object in the three-dimensional coordinate system, and the point determined as the erroneous correspondence point in the erroneous correspondence point determination step and based on said plurality of photographing positions, and a three-dimensional shape measuring step of obtaining a three-dimensional shape of the three-dimensional coordinates or the measurement object feature points of the measurement object, wherein The erroneous correspondence point determination step includes a form determination step, and in the form determination step, a first irregular triangle network having a specific mesh interval is created using the feature points associated in the feature point correspondence step as initial values. A second process for removing a part having a side length exceeding a specific threshold from the irregular triangle network, and an irregular triangle network separated from a main part as a result of the second process. From among the above, a third process for creating a reference form by removing a lump whose labeling number of connections is equal to or less than a specific threshold value, and the three-dimensional coordinates of the feature points remaining after the third process is completed And a fourth process for determining that the corresponding feature point is a miscorresponding point when the distance between the reference form and the reference form exceeds a specific threshold .

According to the invention described in claim 17 , the measurement value including the initial value necessary for the measurement of the three-dimensional shape can be automatically acquired by automatically determining the erroneous correspondence point in the overlapping image.

  According to the present invention, it is possible to automatically give a measurement value including an initial value necessary for measuring a three-dimensional shape by automatically determining an erroneous correspondence point in an overlapping image.

  According to the present invention, the first method for obtaining the position of the photographing unit in advance of photographing the object to be measured and photographing the calibration object in advance and the position of the photographing unit together with the object to be photographed for photographing the calibration object. The second method can be applied in parallel. Examples applied to both of them will be described below.

1. First Embodiment Hereinafter, an example of a shape measuring apparatus and a program in which the present invention is applied to a first method will be described with reference to the drawings. In this first method, since the position and orientation of the photographing unit are first calculated from the calibration subject, two or more photographing units are fixed. The advantage of this method is that even when a moving object (for example, a living body) is measured, the measurement object can be captured and measured in an instant. In addition, once the position and orientation of the photographing unit is obtained from the calibration subject, three-dimensional measurement can be performed at any time by placing the measurement object in the space.

(Configuration of shape measuring device)
FIG. 1 is a top view of a shape measuring apparatus employing the first method. The shape measuring apparatus 1 includes photographing units 2 to 9, feature projecting units 10 to 13, a relay unit 14, a calculation processing unit 15, a display unit 17, and an operation unit 16. The shape measuring apparatus 1 measures the shape of the measuring object 18 disposed in the center of the imaging units 2 to 9.

  For the imaging units 2 to 9, for example, a video camera, a CCD camera (Charge Coupled Device Camera) for industrial measurement, a CMOS camera (Complementary Metal Oxide Semiconductor Camera), or the like is used. The imaging units 2 to 9 are arranged around the measurement object 18. The imaging units 2 to 9 shoot the measurement object 18 in overlapping imaging areas from a plurality of imaging positions.

  The imaging units 2 to 9 are arranged in the horizontal direction or the vertical direction with a predetermined baseline length apart. Note that an imaging unit may be added and arranged in both the horizontal direction and the vertical direction. The shape measuring apparatus 1 measures the three-dimensional shape of the measuring object 18 based on at least a pair of overlapping images. Accordingly, the number of the photographing units 2 to 9 can be appropriately set to one or a plurality depending on the size and shape of the photographing subject.

  For example, a projector or a laser device is used for the feature projection units 10 to 13. The feature projection units 10 to 13 project a pattern such as a random dot pattern, dot spot light, or linear slit light onto the measurement target 18. Thereby, a feature enters a portion where the feature of the measurement object 18 is poor. The feature projection units 10 to 13 are arranged between the imaging units 2 and 3, between the imaging units 4 and 5, between the imaging units 6 and 7, and between the imaging units 8 and 9. In addition, when the measurement object 18 has a characteristic, or when a pattern can be applied, the characteristic projection units 10 to 13 can be omitted.

  The imaging units 2 to 9 are connected to the relay unit 14 via an interface such as Ethernet (registered trademark), camera link, or IEEE 1394 (Institut of Electrical and Electronic Engineers 1394). As the relay unit 14, a switching hub or an image capture board is used. Images captured by the imaging units 2 to 9 are input to the calculation processing unit 15 via the relay unit 14.

  The calculation processing unit 15 is a personal computer (PC), or a PLD (Programmable Logic hardware using PLD such as Field Programmable Gate Array) or ASIC (Application Specific Integrated Circuit). The calculation processing unit 15 is operated by the operation unit 16, and the processing contents and calculation results of the calculation processing unit 15 are displayed on the display unit 17. A keyboard and a mouse are used for the operation unit 16, and a liquid crystal monitor is used for the display unit 17. In addition, the operation unit 16 and the display unit 17 may be configured integrally with a touch panel type liquid crystal monitor.

  FIG. 2 is a block diagram of the shape measuring apparatus. The calculation processing unit 15 includes an imaging position / orientation measurement unit 20, a feature point association unit 21, a parallax measurement unit 22, a three-dimensional coordinate calculation unit 23, an incorrect corresponding point determination unit 24, and a three-dimensional shape measurement unit 25. These may be implemented as a module of a program that can be executed by a PC, or may be implemented as a PLD such as an FPGA.

  The imaging position / orientation measurement unit 20 measures external orientation elements (imaging positions and orientations) of the imaging units 2 to 9 based on an image obtained by imaging the calibration subject 19 shown in FIG. Note that if the internal orientation elements (principal point, focal length, and lens distortion) of the imaging units 2 to 9 are not known, the imaging position / orientation measurement unit 20 also obtains them simultaneously. The calibration subject 19 is a cubic calibration box in which a plurality of reference points are arranged.

  A color code target is used as the reference point (see Japanese Patent Application Laid-Open No. 2007-64627). The color code target has three retro targets (retroreflective targets). First, the photographing position / orientation measurement unit 20 binarizes an image obtained by photographing the calibration subject 19, thereby detecting a retro target and obtaining its center-of-gravity position (reference point image coordinates). The photographing position / orientation measurement unit 20 labels each reference point based on the color code target color scheme (color code). Thereby, the position of the corresponding reference point in the overlapping image is known.

  Then, the photographing position / orientation measurement unit 20 calculates the external orientation elements of the photographing units 2 to 9 by using the relative orientation method, the single photograph orientation method, the DLT method, or the bundle adjustment method. These may be used alone or in combination. The first method will be described in detail in this embodiment, and the second method will be described in detail in the seventh embodiment.

  In this first method, since the position and orientation of the photographing units 2 to 9 are first calculated by the calibration subject 19, two or more photographing units 2 to 9 are fixed. The advantage of this method is that even when a moving object (for example, a living body) is measured, the measurement object 18 can be captured and measured in an instant. In addition, once the position and orientation of the photographing units 2 to 9 are obtained from the calibration subject 19, three-dimensional measurement can be performed at any time by placing the measurement object 18 in the space.

  The feature point association unit 21 extracts feature points of the measurement object 18 from at least a pair of stereo images, and associates the positions of the feature points in the stereo image. When the photographing units 2 to 9 are arranged in the horizontal direction, the feature point association unit 21 searches for the position of the feature point in the horizontal direction, and when the photographing units 2 to 9 are arranged in the vertical direction. When the position of the feature point is searched in the vertical direction and the photographing units 2 to 9 are arranged in the horizontal direction and the vertical direction, the position of the feature point is searched in the horizontal direction and the vertical direction.

  The feature point association unit 21 includes a background removal unit 26, a feature point extraction unit 27, and a corresponding point search unit 28. The background removal unit 26 generates a background removal image in which only the measurement object 18 is copied by subtracting the background image from the processed image in which the measurement object 18 is copied.

  The feature point extraction unit 27 extracts feature points from the background removed image. At this time, feature points are extracted from the left and right stereo images in order to limit the search range of corresponding points. As a feature point extraction method, a differential filter such as Sobel, Laplacian, Prewitt, or Roberts is used.

  The corresponding point search unit 28 searches for a corresponding point corresponding to the feature point extracted in one image in the other image. As a corresponding point search method, template matching such as a residual sequential detection method (Sequential Similarity Detection Method: SSDA), a normalized correlation method, a direction code matching method (Orientation Code Matching: OCM), or the like is used.

  The parallax measurement unit 22 obtains the parallax of the feature points associated by the feature point association unit 21. The parallax measurement unit 22 obtains the horizontal parallax when the photographing units 2 to 9 are arranged in the horizontal direction, and obtains the vertical parallax when the photographing units 2 to 9 are arranged in the vertical direction. When 2 to 9 are arranged in the horizontal direction and the vertical direction, the horizontal parallax and the vertical parallax are obtained.

  The three-dimensional coordinate calculation unit 23 calculates the feature points of the measurement object 18 based on the external orientation elements measured by the imaging position / orientation measurement unit 20 and the image coordinates of the feature points associated by the feature point association unit 21. Calculate 3D coordinates.

  The miscorresponding point determination unit 24 includes the parallax (at least one of horizontal parallax and vertical parallax) obtained by the parallax measurement unit 22 and the measurement target in the three-dimensional coordinate or the three-dimensional coordinate system obtained by the three-dimensional coordinate calculation unit 23. A miscorresponding point is determined based on the form of the object 18. The miscorresponding point determination unit 24 includes a parallax determination unit 29, a space determination unit 30, and a form determination unit 31. When the miscorresponding point determination unit 24 determines that it is a miscorresponding point, the designation of the feature point corresponding to the miscorresponding point is canceled.

  The parallax determination unit 29 creates a histogram of parallax obtained by the parallax measurement unit 22, and determines a feature point corresponding to parallax that is not within a predetermined range from the average value of parallax as an erroneous correspondence point. Since the three-dimensional coordinates of the feature points are calculated based on the parallax, parallax greatly deviating from the average value of parallax is highly likely to be an erroneous correspondence point.

  The space determination unit 30 defines a space of a predetermined distance from the center of gravity of the reference point of the calibration subject 19 as a measurement space, and the three-dimensional coordinates of the feature points calculated by the three-dimensional coordinate calculation unit 23 protrude from the measurement space. In the case that the feature point is detected, the feature point is determined as an erroneous correspondence point.

  The form determination unit 31 creates a reference form of the measurement object 18 from the three-dimensional coordinates of the feature points calculated by the three-dimensional coordinate calculation part 23, and based on the distance between the reference form and the three-dimensional coordinates of the feature points, Determine the miscorresponding points.

  The three-dimensional shape measurement unit 25 performs stereo matching on the pixels in the predetermined region using the feature point group excluding the points determined as the erroneous correspondence points by the erroneous correspondence point determination unit 24, and Find the 3D shape. For stereo matching, LSM that searches while deforming a template image, a normalized correlation method, or the like is used. The three-dimensional shape is displayed on the display unit 17 as a point cloud of feature points or an irregular triangular network (TIN).

  Furthermore, the process of removing the miscorresponding point performed when the miscorresponding point determination unit 24 obtains the initial value can be similarly performed when the three-dimensional shape measuring unit 25 subsequently obtains the measurement value. .

(Processing of shape measuring device)
Hereinafter, detailed processing of the shape measuring apparatus will be described with reference to FIG. FIG. 3 is a flowchart of the program of the shape measuring apparatus. The program for executing this flowchart can be provided by being stored in a recording medium such as a CDROM.

  First, a processed image and a background image are input (step S10). Next, the background removal unit 26 removes the background of the measurement target 18 (step S11). FIG. 4 is a drawing substitute photo (A) showing a processed image, a drawing substitute photo (B) showing a background image, and a drawing substitute photo (C) showing a background removed image. The background removed image is generated by subtracting the background image from the processed image. Background removal is performed on the left and right images. Note that this process may not be performed if a background image cannot be obtained.

  Next, feature points are extracted from the left and right images by the feature point extraction unit 27 (step S12). By extracting feature points from the left and right images, the search range for corresponding points can be reduced.

The feature point extraction unit 27 performs preprocessing such as reduction processing, brightness correction, and contrast correction as necessary (step S12-1). Next, the edge strength is calculated from the preprocessed left and right images by the Sobel filter (step S12-2). FIG. 5 shows Sobel filters in the x and y directions. When the luminance values of nine pixels corresponding to the matrix of the Sobel filter are I _{1 to} I ₉ from the upper left to the lower right, the intensity in the x direction is dx, and the intensity in the y direction is dy, the target pixel (center The edge intensity Mag of the pixel is calculated by the following equation (1).

  FIG. 6 is a drawing-substituting photograph (A) showing an input image subjected to 1/4 compression, and a drawing-substituting photograph (B) showing an edge strength image obtained by a Sobel filter. The feature point extraction unit 27 performs post-processing such as thinning the edge intensity image in FIG. 6B (step S12-3). By thinning, the edge becomes one pixel width. As a result, since the positions of the feature points finally extracted are thinned out, the feature points are extracted without deviation in the image.

  Next, the feature point extraction unit 27 performs binarization processing (step S12-4). In order to automatically determine the binarization threshold, a histogram of edge strength is created. FIG. 7 is a histogram of edge strength. The feature point extraction unit 27 uses, as a binarization threshold, an edge intensity corresponding to a position where the cumulative frequency counted from the edge having the stronger edge intensity is 50% of the total number of edge pixels in the created histogram.

  In the case of FIG. 7, the number of pixels of the edge is 56986 pixels, and the edge strength at the 28493th pixel where the cumulative frequency counted from the stronger side is 50% of the total number of edge pixels is 52. Therefore, the binarization threshold is 52. FIG. 8 is a drawing substitute photograph showing the result of binarization with the threshold 52, and FIG. 9 is a drawing substitute photograph showing the feature points extracted in the left and right images.

  Next, the corresponding point search unit 28 associates feature points in the left and right images (step S13). The corresponding point search unit 28 creates a template image centered on each feature point in the left image, and searches for a corresponding point having the strongest correlation with the template image in a predetermined region in the right image.

  FIG. 10 is an explanatory diagram (A) for explaining the template creation method, an explanatory diagram (B) for explaining the search line determination method, and an explanatory diagram (C) for explaining the search width determination method. As shown in FIG. 10A, the template image is composed of 21 pixels × 21 pixels centered on the feature point of interest. Since the vertical parallax is removed from the stereo image, a search line is provided in parallel to the x-axis as shown in FIG. Also, as shown in FIG. 10C, the corresponding point search unit 28 detects the leftmost feature point and the rightmost feature point on the search line of the right image, and the leftmost feature point and the rightmost feature point. Corresponding points are searched using the search range up to the feature point of.

  As a result, the point having the strongest correlation with the template image becomes the corresponding point. When the photographing units 2 to 9 are arranged in the vertical direction, the search line is parallel to the y axis. When the photographing units 2 to 9 are arranged in the vertical direction and the horizontal direction, the search lines are the x axis and the y axis.

  Next, the miscorresponding point determination unit 24 determines a feature point (miscorresponding point) that is incorrectly associated (step S14). The miscorresponding point determination process includes determination based on parallax (step S14-1), determination based on measurement space limitation (step S14-2), and determination based on the form of the measurement object 18 (step S14-3). is there. These determination processes are performed in stages, but may be an aspect in which at least one process is performed.

  Hereinafter, the determination method based on parallax will be described (step S14-1). In the stereo method, the three-dimensional coordinates of feature points are calculated based on parallax. Therefore, the parallax of the feature point in the background tends to be different from the parallax of the feature point of the measurement object 18. FIG. 11 is an explanatory diagram for explaining the parallax of a miscorresponding point. As shown in FIG. 11, the parallax (−10) at the miscorresponding point is significantly different from the other parallaxes (−50, −50, −40, −60). For this reason, the parallax determination unit 29 creates a histogram of the parallax measured by the parallax measurement unit 22, and determines a feature point having a parallax greatly deviating from the average parallax value as an erroneous correspondence point.

  FIG. 12 is a parallax histogram. When the average parallax is μ and the standard deviation is σ, if the parallax is not in the section [μ−1.5σ, μ + 1.5σ] that is 1.5 times the standard deviation σ from the average μ, it corresponds to that parallax. The feature point is determined to be a miscorresponding point. Note that the value of 1.5 is an initial value obtained by experiments with a plurality of samples, and can be changed in advance.

  FIG. 13 is a drawing-substituting photograph (A) showing a point group of feature points before parallax determination, and a drawing-substituting photograph (B) showing a point group of feature points after parallax determination. As illustrated in FIG. 13, by performing the parallax determination, the feature points of the background that are far away from the measurement target 18 are removed as erroneous correspondence points.

  Next, a determination method based on measurement space limitations will be described (step S14-2). The measurement space is defined as a space centered on the center of gravity of the reference point of the calibration subject 19. FIG. 14 is a drawing-substituting photograph showing the reference point of the calibration subject and its center of gravity. It is assumed that the calibration subject 19 is placed at the center of gravity of the measurement target 18.

  First, the space determination unit 30 calculates the barycentric coordinates by taking the average of all the reference point coordinates. A space 50 cm from the center of gravity is defined as a measurement space. If the distance from the center of gravity of the reference point to the three-dimensional coordinates of the feature point obtained by the three-dimensional coordinate calculation unit 23 exceeds 50 cm, the space determination unit 30 determines that the feature point is a miscorresponding point. . The threshold value of 50 cm can be changed in advance based on the size of the measurement object 18. FIG. 15 is a drawing-substituting photograph showing feature points removed by space determination.

  Next, the determination method according to the form of the measurement object 18 will be described (step S14-3). The above-described parallax determination and space determination are effective when there is a large error, and the miscorresponding point located in the contour portion of the measurement object 18 is not removed. Therefore, the form determination unit 31 creates a reference form of the measurement object 18, and determines a miscorresponding point based on the distance between the reference form and the three-dimensional coordinates of the feature points.

  FIG. 16 is a drawing substitute photo (A) showing a TIN created at a mesh interval of 4 mm, and a drawing substitute photo (B) showing a TIN created at a mesh interval of 10 mm. First, the form determination unit 31 uses the feature points remaining until the space determination (step S14-2) as an initial value of the LSM, and creates a TIN that is the basis of the reference form. As shown in FIG. 16A, when the mesh interval of TIN is small, an uneven surface is formed due to the remaining miscorresponding points. Therefore, as shown in FIG. 16B, the form determination unit 31 increases the mesh interval of the TIN and creates a rough shape (rough surface) of the measurement object 18.

  Next, the form determination unit 31 removes TIN having a long side. The threshold of the side length is set to 40 mm, which is four times that when the created mesh interval is 10 mm. The threshold value of 4 times the mesh interval to be created is a value obtained by experiments with a plurality of samples, and this threshold value can be changed in advance.

  FIG. 17 is a drawing-substituting photograph showing the result of removing TIN having a long side. As shown in FIG. 17, as a result of removing the TIN having a long side, a TIN lump separated from the main part is generated. Since the separated TIN lump is often a background or a hair portion with few features even if the side length is short, the reliability of the surface of the measurement object 18 is low. Therefore, the form determination unit 31 performs labeling for each TIN chunk, and removes the TIN chunk based on the number of connections. FIG. 18 is a drawing-substituting photograph showing the TIN labeling result. The form determination unit 31 creates a final rough surface (reference form) by removing a TIN lump whose labeling number is 10 or less. Note that the threshold of the number of labeling connections can be changed in advance.

  Next, when the distance from the created reference form to the three-dimensional coordinates of the remaining feature points exceeds a predetermined threshold value, the form determination unit 31 determines that the feature point is a miscorresponding point. FIG. 19 is a drawing-substituting photograph (A) showing a point form and a point group of feature points, and a drawing-substituting photograph (B) showing a point group of feature points after form determination. First, the form determination unit 31 searches for the closest point among the vertices of the TIN forming a rough surface from the remaining points of the feature point group. When the distance from the vertex of the nearest TIN to the feature point exceeds a predetermined threshold value, the form determination unit 31 determines that the feature point is a miscorresponding point. The threshold is set to 20 mm, which is twice the created mesh interval of 10 mm. The threshold value of twice the mesh interval to be created is a value obtained by experiments with a plurality of samples, and this threshold value can be changed in advance.

  When the determination of the miscorresponding point regarding the initial value is completed, the three-dimensional shape of the measurement object is measured by the three-dimensional shape measurement unit 25 (step S15). The three-dimensional shape measurement unit 25 performs stereo matching using the feature point group shown in FIG. 19B as an initial value of LSM, and measures a three-dimensional shape (dense surface). FIG. 20 is a drawing-substituting photograph showing the result of dense surface measurement. In FIG. 20, the processing area of LSM is a convex area in which the feature point group after removing the miscorresponding points is surrounded by a convex hull. For this reason, an unnecessary TIN is generated from the head to the shoulder, but a TIN having a specified mesh interval is generated in the portion of the random dot pattern projected by the feature projection units 10 to 13.

  FIG. 21 is a drawing-substituting photograph showing measurement results of a three-dimensional shape in a plurality of models. When a plurality of models shown in FIG. 21 are synthesized, an unnecessary TIN that connects the head to the shoulder formed in the front model hides the original surface formed in the side model. Therefore, the processing area of the LSM is limited so as not to generate unnecessary TIN formed from the head to the shoulder.

  FIG. 22 is a diagram illustrating restrictions on the processing area of LSM. As shown in FIG. 22, the enclosing line of the feature point group is created. The encircling line is created by creating a convex hull line and repeating the process of cutting the processing area inward while looking at the distance and angle between the constituent points.

  It should be noted that the processing for removing the miscorresponding point performed when the initial value is obtained by the miscorresponding point determination unit 24 can be similarly performed when the three-dimensional shape measuring unit 25 subsequently obtains the measured value. .

  FIG. 23 is a drawing-substituting photograph showing the measurement result of adjusting the LSM processing area, and FIG. 24 is a drawing-substituting photograph showing the synthesis result of the plurality of models in FIG. As shown in FIGS. 23 and 24, it can be seen that unnecessary TIN is removed from the head to the shoulder.

  Hereinafter, specific processing when the photographing position / orientation measuring unit 20 adopts the relative orientation method in the first method of photographing the calibration subject before photographing the measurement object and obtaining the position of the photographing unit in advance will be described. To do.

  According to the relative orientation method, an external orientation element can be obtained based on six or more corresponding reference points copied in the overlapping image. If the three-dimensional position of the reference point is known, the absolute coordinates of the photographing units 2 to 9 are obtained by absolute orientation.

FIG. 25 is an explanatory diagram for explaining relative orientation. In the relative orientation, an external orientation element is obtained from six or more corresponding points (pass points) in the two left and right images. In the relative orientation, a coplanar condition that two rays connecting the projection centers O ₁ and O ₂ and the reference point P must be in the same plane is used. The following formula 2 shows the coplanar conditional expression.

As shown in FIG. 25, the origin of the model coordinate system is taken as the left projection center O ₁ , and the line connecting the right projection center O ₂ is taken as the X axis. For the scale, the base length is the unit length. At this time, the parameters to be obtained are the rotation angle κ ₁ of the left camera, the rotation angle φ _{1 of} the Y axis, the rotation angle κ _{2 of the} right camera, the rotation angle φ ₂ of the Y axis, and the rotation angle φ _{2 of} the X axis. There are five rotation angles ω ₂ . In this case, since the rotation angle ω ₁ of the X axis of the left camera is 0, there is no need to consider it. Under such a condition, the coplanar conditional expression of Expression 2 becomes as shown in Expression 3, and each parameter can be obtained by solving this expression.

  Here, the following relational expression for coordinate transformation is established between the model coordinate system XYZ and the camera coordinate system xyz.

Using these equations, an unknown parameter (external orientation element) is obtained by the following procedure.
(1) The initial approximate values of unknown parameters (κ ₁ , φ ₁ , κ ₂ , φ ₂ , ω ₂ ) are normally 0.
(2) The coplanar conditional expression of Equation 3 is Taylor-expanded around the approximate value, and the value of the differential coefficient when linearized is obtained by Equation 4, and an observation equation is established.
(3) A least square method is applied to obtain a correction amount for the approximate value.
(4) The approximate value is corrected.
(5) Using the corrected approximate value, the operations (1) to (4) are repeated until convergence.

When the relative orientation converges, connection orientation is further performed. Connection orientation is a process of unifying the inclination and scale between a plurality of models to make the same coordinate system. When this processing is performed, a connection range represented by the following formula 5 is calculated. As a result of the calculation, if ΔZ _j and ΔD _j are equal to or less than a predetermined value (for example, 0.0005 (1/2000)), it is determined that the connection orientation has been normally performed.

(Advantages of the first embodiment)
According to the first embodiment, it is possible to automatically acquire the initial value of the LSM or normalized correlation method used for stereo matching by automatically determining the erroneous correspondence points in the left and right overlapping images.

  Further, by limiting the processing region for stereo matching, only the original shape of the measurement object 18 can be formed, and a plurality of models can be synthesized.

  Further, the photographing position and orientation of the photographing units 2 to 9 can be measured by the calibration subject 19. After obtaining the photographing position and orientation, three-dimensional measurement can be performed at any time by placing the measuring object 18 in the space. Further, even a dynamic measurement object 18 having movement can be measured if it is simultaneously photographed by the photographing units 2 to 9.

2. Second Embodiment The second embodiment estimates the parallax of a feature point or determines a miscorresponding point based on the parallax of a reference point near the feature point. The first embodiment adopts the first method described above. It is a modification of the determination method based on parallax performed by the error corresponding point determination unit in the embodiment.

FIG. 28 is an explanatory diagram illustrating a parallax estimation method for feature points. As shown in FIG. 28, the feature point F of the measurement object and the reference points P1 to P3 are copied in the left stereo image and the right stereo image. At this time, the parallax D between the feature point F of the left image and the feature point F ′ on the right image is represented by the reference point when the parallaxes of the reference points P1 to P3 are D _{1 to} D ₃ as shown in the following formula 6. It is estimated by the average parallax of P1 to P3.

  When searching for the position of the feature point F ′, the feature point association unit 21 searches in the horizontal direction from the position corresponding to the estimated parallax D. When the photographing units 2 to 9 are arranged in the vertical direction, the feature point association unit 21 searches in the vertical direction from a position corresponding to the estimated parallax D.

  Furthermore, the miscorresponding point may be determined using the parallax estimation method as described above. That is, the parallax determination unit 29 determines an erroneous correspondence point of the feature point based on the average parallax of the reference points near the feature point. It is also possible to weight the distance from the feature point to the reference point and determine the miscorresponding point of the feature point.

(Advantage of the second embodiment)
According to the second embodiment, when associating the positions of the feature points in the left and right images, the corresponding points can be searched in the horizontal direction or the vertical direction from the position corresponding to the estimated parallax of the feature points. Further, an erroneous correspondence point can be determined based on the parallax of the reference point near the feature point.

3. Third Embodiment Hereinafter, a modification of the first embodiment will be described. In the third embodiment, parallax of feature points is obtained based on a known depth amount, and an erroneous correspondence point is determined. It is one of the modifications of the determination method by parallax in 1st Embodiment which employ | adopted the 1st system mentioned above.

  In the case of stereo photography, the optical axes of the two cameras are parallel, the distance a (focal length) from the principal point to the CCD surface is equal, and the CCD is disposed perpendicular to the optical axis. For this reason, if the distance (baseline length) between the optical axes of the camera is 1, the relationship of the following Expression 7 is established between the coordinates of the feature points F1 (x1, y1) and F2 (x2, y2) on the image. It holds. However, the origin of the entire coordinate system (x, y, z) is taken as the principal point of one camera lens.

  According to Equation 7, the depth amount z is inversely proportional to the parallax x2-x1. Therefore, the miscorresponding point determination unit 24 can determine the miscorresponding point based on the known depth amount.

FIG. 29 is an explanatory diagram illustrating a method for determining an erroneous corresponding point based on the depth amount. It is assumed that the depth amount Z between the feature points F1 and F2 is known. At this time, the difference D ₁ -D ₂ between the parallax D ₁ of the feature point F ₁ and the parallax D ₂ of the feature point F ₂ is obtained by the following equation 8 where the focal length is a and the baseline length is l.

If any of the feature point F1 and F2 are not erroneous corresponding points are, D ₁ -D ₂ in which the difference in parallax between the parallax and the feature point F2 of the feature point F1, was determined from the number 8 on the basis of the known depth amount Z , The feature point F1 or F2 can be determined as a miscorresponding point.

(Advantage of the third embodiment)
According to the third embodiment, when the depth amount is known, it is possible to determine a miscorresponding point.

4). Fourth Embodiment A modification of the first embodiment will be described below. The fourth embodiment further includes a contour extraction unit that extracts the contour of the measurement object.

  FIG. 30 is a block diagram of a shape measuring apparatus according to the fourth embodiment. The shape measuring apparatus 1 includes a contour extracting unit 37 between the background removing unit 26 and the feature point extracting unit 27. First, the contour extraction unit 37 creates an edge intensity image of the measuring object 18 using a differential filter such as Sobel, Laplacian, Plewwit, and Roberts.

  Next, the contour extraction unit 37 performs a labeling process on the image obtained by thinning the edge strength image, and obtains the length of the edge from the number of connected pixels. A contour edge is extracted by binarizing a value obtained by multiplying the edge strength and the edge length. FIG. 31A is a drawing-substituting photograph showing a contour edge. The contour edge 38 has a strong edge strength and a large amount of linear components compared to the internal features of the measurement object 18.

  The feature point extraction unit 27 extracts feature points in a region inside the contour edge 38 extracted by the contour extraction unit 37. FIG. 31B is a drawing-substituting photograph (B) showing the feature point extraction region. The extraction region 39 is a region having a predetermined ratio (for example, 80%) of the lateral width of the contour edge. This value of 80% can be changed in advance.

(Advantage of the fourth embodiment)
FIG. 32 is a drawing-substituting photograph showing a difference in photographing range of stereo images. As shown in FIG. 32, in the stereo image, there is a left image shooting range 40 that does not appear in the right image and a right image shooting range 41 that does not appear in the left image. Since the feature points extracted in the photographing regions 40 and 41 cannot be associated with the feature points, they are all false correspondence points. However, according to the fourth embodiment, since the feature point extraction region is limited, no erroneous corresponding point is extracted.

5. Fifth Embodiment In the fifth embodiment, as the method for obtaining the position and orientation of the photographing unit in the first embodiment, the photographing position of the photographing unit is determined by at least one of the horizontal angle measuring unit and the vertical angle measuring unit. A means for measuring is adopted.

  FIG. 26 is a top view of a shape measuring apparatus having a horizontal angle measuring unit. The horizontal angle measuring unit 32 is a rotary encoder, and includes a horizontal angle scale 33 and an encoder 34. The photographing units 2 to 9 can be fixed at an arbitrary position of the horizontal angle scale 33, and the photographing units 2 to 9 and the horizontal angle scale 33 are integrally supported so as to be rotatable clockwise and counterclockwise. . On the other hand, the encoder 34 is fixed at an arbitrary position on the circumference of the horizontal angle scale 33.

  The horizontal angle scale 33 includes imaging unit detection patterns 33a to 33d and horizontal angle detection slits 33e. The imaging unit detection patterns 33a to 33d are detachable at arbitrary positions on the circumference of the horizontal angle scale 33, and are mounted at positions where the imaging units 2 to 9 are fixed. For example, the image capturing unit detection patterns 33a to 33d are arranged between the image capturing units 2 and 3, the image capturing units 4 and 5, the image capturing units 6 and 7, the image capturing units 8 and 9, and the image capturing units 2 and 4. , 6 and 8 are installed.

  When the imaging units 2 to 9 and the horizontal angle scale 33 rotate, the encoder 34 detects the imaging unit detection patterns 33a to 33d. At this time, the encoder 34 sets the horizontal angle of the position of the pattern detected first among the imaging unit detection patterns 33a to 33d to zero. Then, the encoder 34 detects the horizontal angle from there to the other imaging unit detection patterns 33a to 33d by the horizontal angle detection slit 33e.

  When the imaging units 2 to 9 adjust the height so as to rotate on the same horizontal plane, the imaging positions of the imaging units 2 to 9 are obtained based only on the horizontal angle detected by the encoder 34. The encoder 34 outputs the horizontal angles of the photographing units 2 to 9 to the calculation processing unit 15. The calculation processing unit 15 obtains a relative three-dimensional shooting position of the shooting units 2 to 9 based on the input horizontal angle of the shooting units 2 to 9. Further, when obtaining the photographing positions of the real scale photographing units 2 to 9, the calculation processing unit 15 inputs the diameter of the horizontal angle scale 33.

  Further, the shape measuring apparatus 1 may include a vertical angle measuring unit that rotates in the vertical direction integrally with the imaging units 2 to 9 instead of the horizontal angle measuring unit 32. In this case, the calculation processing unit 15 obtains the photographing positions of the photographing units 2 to 9 based on the vertical angle detected by the vertical angle measuring unit.

  Furthermore, the shape measuring apparatus 1 may be configured to include a vertical angle measuring unit that rotates integrally with the imaging units 2 to 9 and the horizontal angle measuring unit 32 in the vertical direction. In this case, the calculation processing unit 15 obtains the photographing position of the photographing units 2 to 9 based on the horizontal angle and the vertical angle of the photographing units 2 to 9.

(Advantage of the fifth embodiment)
According to the fifth embodiment, it is not necessary to obtain the shooting positions and postures of the shooting units 2 to 9 based on the image obtained by shooting the calibration subject 19. That is, the shape measuring apparatus 1 does not need to include the photographing position / orientation measuring unit 20.

6). Sixth Embodiment In the first embodiment that employs the first method, the example has been described in which the photographing position / orientation measurement unit 20 obtains the photographing position and orientation of the photographing unit using the relative orientation method. In the sixth embodiment, single photograph orientation and DLT method will be described as examples of specific processing that can be adopted instead of the relative orientation method.

6-1. Single photograph orientation FIG. 27 is an explanatory diagram for explaining single photograph orientation. Single photo orientation uses the collinear condition that holds the reference point in a single photo, and uses the camera position O (X ₀ , Y ₀ , Z ₀ ) and camera posture (ω , Φ, κ). The collinear condition is a condition that the projection center, the photographic image, and the ground target points (Op ₁ P ₁ , Op ₂ P ₂ , Op ₃ P ₃ ) are on a straight line. The camera position O (X ₀ , Y ₀ , Z ₀ ) and the camera posture (ω, φ, κ) are external orientation elements.

  First, the camera coordinate system is x, y, z, the photographic coordinate system x, y, and the ground coordinate system is X, Y, Z. It is assumed that photographing is performed in a direction in which the camera is sequentially rotated counterclockwise by ω, φ, and κ counterclockwise with respect to the positive direction of each coordinate axis. Then, the three-dimensional coordinates of the reference points corresponding to the four image coordinates (at least three points) are substituted into the quadratic projective transformation equation shown in Equation 9, and the observation equations are established to obtain the parameters b1 to b8.

Using the parameters b1 to b8 in Equation 9, an external orientation element is obtained from Equation 10 below.

6-2. DLT method The DLT method approximates the relationship between the photographic coordinates and the three-dimensional coordinates of the target space by a cubic projective transformation equation. The basic formula of the DLT method is the following formula 11. For details of the DLT method, refer to “Shunji Murai: Analytical Photogrammetry, p46-51, p149-155” and the like.

  Eliminating the denominator of Equation 11 yields the linear form of Equation 12.

  Furthermore, when Equation 12 is transformed, the following Equation 13 is obtained.

By substituting the three-dimensional coordinates of six or more reference points into Equation 13 and solving using the least square method, eleven unknown variables L _{1 to} L ₁₁ that determine the relationship between the photograph coordinates and the target point coordinates are obtained. You can get it. Note that L _{1 to} L ₁₁ include external orientation elements.

(Advantage of the sixth embodiment)
According to the sixth embodiment, when single photograph orientation is adopted, it is advantageous for measurement from a small number of images, and when DLT is adopted, there is an advantage that calculation processing is simpler than single photograph orientation. is there.

7). Seventh Embodiment The seventh embodiment is based on the second method in which the position of the photographing unit is photographed in parallel with the object to be measured and the reference scale (subject for calibration) is obtained in parallel. It is a modification of the processing method of the imaging position and orientation measurement unit 20 in the embodiment.

  The second method is a method in which a measurement object to be measured and a calibration subject are simultaneously photographed, and the position and orientation of the photographing unit are obtained to perform three-dimensional measurement. In this case, there is no need to fix the photographing unit, and there may be one to a plurality of photographing units, and measurement is possible if the number of photographing is two or more. The advantage of this method is that the position of the photographing unit is free and can be measured even from one unit, so that the configuration can be simplified. In addition, since the position and orientation of the photographing unit are obtained at the same time as the measurement object is photographed, it is not necessary to obtain the position and orientation of the photographing unit in advance. The calibration subject to be imaged together uses a fixed length such as a reference scale, or a fixed coordinate.

  The shooting part does not need to be fixed basically and can be placed anywhere. FIG. 33 is a top view of the shape measuring apparatus according to the seventh embodiment, and FIG. 34 is a top view of a modification of the shape measuring apparatus according to the seventh embodiment. FIG. 33 shows the relationship between the reference rule 19, the image capturing unit 2 and the measurement object 18, the measurement processing unit 15, the operation unit 16, and the display unit 17 in a mode in which a single image capturing unit moves while moving a place. Yes. On the other hand, FIG. 34 shows a modification in a mode in which two cameras are configured as a stereo camera or multiple cameras are configured as a multi-camera.

  In this case, the measurement processing unit 15, the operation unit 16, and the display unit 17 can be configured by only the photographing unit and the PC by using the PC. If there is no pattern on the object, the pattern is projected by a projector or a pattern is applied to the object. As a method for obtaining the photographing position and orientation of the photographing unit, a relative orientation method, a single photo orientation method, a DLT method, or a bundle adjustment method is used, and these may be used alone or in combination.

  If the photographing position and orientation (external orientation element) of the photographing unit are obtained by single photograph orientation or the DLT method, the external orientation element can be obtained based on the relative positional relationship of the reference points captured in one photograph. it can.

  FIG. 35 is a drawing substitute photo (A) showing a left image obtained by photographing a reference scale and a measurement object, and a drawing substitute photo (B) showing a right image. As shown in FIG. 35, the measurement object 18 is photographed together with a reference rule 35 (calibration subject) in which color code targets 36a to 36d are arranged in a relative positional relationship.

  The imaging position / orientation measurement unit 20 illustrated in FIG. 2 acquires the overlapping image illustrated in FIG. The photographing position / orientation measurement unit 20 binarizes the overlapping images to obtain the barycentric positions (image coordinates of the reference points) of the color code targets 36a to 36d. The photographing position / orientation measurement unit 20 reads a color code from the color scheme of the color code targets 36a to 36d, and labels each color code target 36a to 36d.

  With this label, the correspondence of the reference points in the overlapping image can be known. The photographing position / orientation measuring unit 20 uses the mutual orientation method, single photograph orientation or DLT method, or bundle adjustment method based on the image coordinates of the reference point and the three-dimensional relative positional relationship of the reference points. Obtain the shooting position and posture of ~ 9. A combination of these also requires a highly accurate position and orientation.

  The second method, which is the processing method of the photographing position / orientation measurement unit 20 employed in the present embodiment, is a modification of the first method employed in the first embodiment, and for other configurations and processes, the first method is used. The configuration and processing shown in the sixth embodiment can be adopted.

(Advantage of the seventh embodiment)
According to the seventh embodiment, the second method is adopted, and the position and orientation of the photographing unit can be obtained and three-dimensionally measured by photographing the measurement object 18 and the calibration subject at the same time. The imaging unit can be configured from one unit to any number of units, and since it is not necessary to fix the imaging unit, the configuration can be simplified.

  The present invention can be used for a shape measuring apparatus and a program for measuring a three-dimensional shape of a measurement object.

It is a top view of the shape measuring apparatus which employ | adopted the 1st system. It is a block diagram of the shape measuring device concerning a 1st embodiment. It is a flowchart of a shape measuring apparatus. A drawing substitute photo (A) showing a processed image, a drawing substitute photo (B) showing a background image, and a drawing substitute photo (C) showing a background removed image. It is a Sobel filter in the x and y directions. They are a drawing substitute photo (A) showing an input image subjected to 1/4 compression, and a drawing substitute photo (B) showing an edge strength image by a Sobel filter. It is a histogram of edge intensity. 5 is a drawing substitute photograph showing the result of binarization with a threshold value 52. FIG. It is a drawing substitute photograph showing the feature points extracted in the left and right images. It is explanatory drawing (A) explaining the template preparation method, explanatory drawing (B) explaining the determination method of a search line, and explanatory drawing (C) explaining the determination method of search width. It is explanatory drawing explaining the parallax of a miscorresponding point. It is a parallax histogram. They are a drawing substitute photograph (A) showing a point group of feature points before parallax determination and a drawing substitute photograph (B) showing a point group of feature points after parallax determination. It is a drawing substitute photograph showing the reference point and the center of gravity of the calibration subject. It is a drawing substitute photograph which shows the feature point removed by space determination. They are a drawing substitute photo (A) showing TIN created with a mesh interval of 4 mm and a drawing substitute photo (B) showing TIN created with a mesh interval of 10 mm. It is a drawing substitute photograph which shows the result of removing TIN having a long side. It is a drawing substitute photograph which shows the labeling result of TIN. It is drawing substitute photograph (A) which shows the point form of a reference | standard form and a feature point, and drawing substitute photograph (B) which shows the point cloud of the feature point after form determination. It is a drawing substitute photograph which shows the result of dense surface measurement. It is a drawing substitute photograph which shows the measurement result of the three-dimensional shape in a some model. It is a figure which shows the restriction | limiting of the process area | region of LSM. It is a drawing substitute photograph which shows the measurement result which adjusted the process area | region of LSM. It is a drawing substitute photograph which shows the synthetic | combination result of the several model of FIG. It is explanatory drawing explaining a relative orientation. It is a top view of the shape measuring apparatus which concerns on 5th Embodiment. It is explanatory drawing explaining a single photograph orientation. It is explanatory drawing explaining the parallax estimation method of a feature point. It is explanatory drawing explaining the determination method of the miscorresponding point based on depth amount. It is a block diagram of the shape measuring apparatus which concerns on 4th Embodiment. It is drawing substitute photograph (A) which shows an outline edge, and drawing substitute photograph (B) which shows the extraction area | region of a feature point. It is a drawing substitute photograph which shows the difference in the imaging | photography range of a stereo image. It is a top view of the shape measuring apparatus which concerns on 7th Embodiment. It is a top view of the modification of the shape measuring apparatus which concerns on 7th Embodiment. It is drawing substitute photograph (A) which shows the left image which image | photographed the reference | standard scale and the measuring object, and drawing substitute photograph (B) which shows a right image.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Shape measuring apparatus, 2-9 ... Imaging | photography part, 10-13 ... Feature projection part, 14 ... Relay part, 15 ... Calculation processing part, 16 ... Operation part, 17 ... Display part, 18 ... Measuring object, 19 ... Calibration subject, 20 ... shooting position / orientation measurement unit, 21 ... feature point correspondence unit, 22 ... parallax measurement unit, 23 ... three-dimensional coordinate calculation unit, 24 ... miscorresponding point determination unit, 25 ... three-dimensional shape measurement unit, 26: Background removal unit, 27: Feature point extraction unit, 28 ... Corresponding point search unit, 29 ... Parallax measurement unit, 30 ... Space determination unit, 31 ... Form determination unit

Claims (17)

A feature point association unit for correlating the position of the feature point of the measurement object in the overlapping image captured from a plurality of shooting positions ;
A parallax measurement unit that obtains parallax in the overlapping images of the feature points associated by the feature point association unit;
Based on the parallax obtained by the parallax measurement unit, a three-dimensional coordinate calculation unit for obtaining a three-dimensional coordinate of the feature point of the measurement object;
Based on the form of the measurement object in the three-dimensional coordinate system obtained by the three-dimensional coordinate calculation unit, an erroneous corresponding point determination unit that determines an erroneous corresponding point;
Based on the position of the feature point excluding the point determined to be an erroneous correspondence point by the erroneous correspondence point determination unit and the plurality of imaging positions, the three-dimensional coordinates of the characteristic point of the measurement object or the three-dimensional of the measurement object A three-dimensional shape measuring unit to find the shape
With
The miscorresponding point determination unit includes a form determination unit,
The form determination unit
A first process for creating an irregular triangle network having a specific mesh interval with the feature points associated by the feature point correspondence unit as an initial value;
A second process for removing a portion having a side length exceeding a specific threshold from the irregular triangle network;
As a result of the second processing, from the irregular triangle network separated from the main part, a third processing for creating a reference form by removing a lump whose labeling connection number is a specific threshold value or less;
When the distance between the three-dimensional coordinates of the feature point remaining at the stage where the third process is completed and the reference form exceeds a specific threshold, the corresponding feature point is determined as a miscorresponding point And the fourth process
Shape measuring apparatus and performs. A calibration object with a reference point;
The imaging position / orientation measurement unit for obtaining an imaging position and orientation of the imaging unit based on a position of a corresponding reference point in an overlapping image obtained by imaging the calibration subject. Shape measuring device.   The shape measuring apparatus according to claim 1, further comprising at least one of a horizontal angle measuring unit and a vertical angle measuring unit for measuring a shooting position of the shooting unit. A calibration object with a reference point in a predetermined positional relationship;
The imaging position / orientation measurement unit for obtaining an imaging position and orientation of the imaging unit based on a position of a corresponding reference point in an overlapping image obtained by imaging the calibration subject. Shape measuring device.   5. The shape measurement according to claim 2, wherein the feature point association unit associates positions of feature points in at least a pair of stereo images captured by separating the imaging unit by a predetermined baseline length. apparatus.   5. The miscorresponding point determination unit creates a parallax histogram, and determines a feature point corresponding to a parallax that is not within a predetermined range from an average of parallax as a miscorresponding point. The shape measuring device described in 1.   The shape measuring apparatus according to claim 2, further comprising a feature projecting unit that projects a feature onto the measurement object.   The parallax measurement unit estimates the parallax of the feature point based on the parallax of the reference point whose position near the feature point is known when obtaining the parallax of the feature point associated by the feature point association unit The shape measuring apparatus according to claim 4.   The shape measurement according to claim 4, wherein the miscorresponding point determination unit determines whether or not the feature point is a miscorresponding point based on a parallax of a reference point whose position near the feature point is known. apparatus.   The shape measurement apparatus according to claim 1, wherein the miscorresponding point determination unit determines an miscorresponding point based on at least one of horizontal parallax and vertical parallax.   The shape measuring apparatus according to claim 1, wherein the three-dimensional shape measuring unit measures a three-dimensional shape using the inside of the outline of the measurement object as a processing region.   The specification of a feature point corresponding to the miscorresponding point is canceled when the miscorresponding point determination unit determines that it is a miscorresponding point. Shape measuring device.   The shape measuring apparatus according to claim 1, wherein the feature point association unit estimates a parallax of a feature point based on a known depth amount, and determines an erroneous corresponding point based on the parallax.   The image processing apparatus further includes a contour extraction unit that extracts a contour of the measurement object in the overlapping image, and the feature point association unit associates the position of the feature point in a region inside the contour extracted by the contour extraction unit. The shape measuring apparatus according to claim 1.   The miscorresponding point determination unit determines the feature point as an incorrect corresponding point based on a distance from the center of gravity of a reference point provided on the calibration subject to the three-dimensional coordinate of the feature point obtained by the three-dimensional coordinate calculation unit. The shape measuring apparatus according to claim 2 or 4, wherein   The three-dimensional shape measuring unit is configured to perform the process of removing the erroneous corresponding point performed when the initial value is obtained by the erroneous corresponding point determining unit when the measured value is subsequently obtained. The shape measuring apparatus according to claim 1. A feature point correspondence step for correlating the position of the feature point of the measurement object in the overlapping image taken from a plurality of photographing positions ;
A parallax measurement step for obtaining a parallax in an overlapping image of the feature points associated in the feature point association step;
Based on the parallax obtained in the parallax measurement step, a three-dimensional coordinate calculation step for obtaining a three-dimensional coordinate of the feature point of the measurement object;
Based on the form of the measurement object in the three-dimensional coordinate system obtained in the three-dimensional coordinate calculation step, an erroneous corresponding point determining step for determining an erroneous corresponding point;
Based on the position of the feature point excluding the point determined as the erroneous correspondence point in the erroneous correspondence point determination step and the plurality of imaging positions, the three-dimensional coordinates of the characteristic point of the measurement object or the three-dimensional of the measurement object 3D shape measurement step to find the shape
With
The erroneous corresponding point determination step includes a form determination step,
In the form determining step,
A first process for creating an irregular triangle network having a specific mesh interval with the feature points associated in the feature point correspondence step as initial values;
A second process for removing a portion having a side length exceeding a specific threshold from the irregular triangle network;
As a result of the second processing, from the irregular triangle network separated from the main part, a third processing for creating a reference form by removing a lump whose labeling connection number is a specific threshold value or less;
When the distance between the three-dimensional coordinates of the feature point remaining at the stage where the third process is completed and the reference form exceeds a specific threshold, the corresponding feature point is determined as a miscorresponding point A program for executing the fourth process .

Images