JP5352997B2 - Three-dimensional shape measuring apparatus, three-dimensional shape measuring method and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a means for determining an abnormal output point of an image, in measurement of a three-dimensional shape by a phase shift method. <P>SOLUTION: The three-dimensional shape measuring device includes a protrusion, an imaging part, an operation part and a determination part. The protrusion irradiates an object to be measured with pattern light whose brightness distribution is changed in a sine wave shape. The imaging part images an image by the pattern light projected to the object to be measured from a direction different from a projection direction of the protrusion. The imaging part acquires an image group comprising a plurality of images having each phase of the pattern light shifted at a fixed interval between each image. The operation part extracts a plurality of image sets comprising a smaller number of images than the image group from the image group, and determines a parameter depending on a light source relative to each image set. The determination part determines whether an abnormal output point is included in some image in the image group, based on a plurality of parameters determined from each image set respectively. <P>COPYRIGHT: (C)2009,JPO&amp;INPIT

Description

  The present invention relates to a three-dimensional shape measuring apparatus using a phase shift method and its peripheral technology.

Conventionally, a phase shift method as shown in Patent Document 1 is known as a method of measuring the three-dimensional shape of a measurement object. In this phase shift method, a fringe pattern having a sinusoidal intensity distribution is projected a plurality of times on the surface of the object to be measured to obtain the phase of the sine wave, and the three-dimensional shape is restored based on the information.
US Pat. No. 6,075,605

  By the way, in the measurement of the three-dimensional shape by the phase shift method, it is assumed that a plurality of images are taken under the same conditions except for the phase pattern. Therefore, for example, when an abnormality occurs in any of a plurality of images due to a disturbance such as a change in an external illumination environment or an abnormality in the three-dimensional shape measurement apparatus, the measurement accuracy is greatly reduced. On the other hand, since it is very difficult to determine the presence / absence of the abnormality from only individual images taken by the phase shift method, there has been a demand for improvement thereof.

  Therefore, an object of the present invention is to provide means for determining an abnormal output point of an image in measurement of a three-dimensional shape by a phase shift method.

A three-dimensional shape measurement apparatus according to one aspect includes a projection unit, an imaging unit, a calculation unit, and a determination unit. The projection unit irradiates the measurement target with pattern light whose luminance distribution changes in a sine wave shape. The imaging unit captures an image of pattern light projected onto the measurement object from a direction different from the projection direction of the projection unit, and acquires an image group including n images in which the phase of the pattern light is shifted from one image to another. To do. The calculation unit extracts _n C ₃ image sets from the _n images, and obtains a parameter depending on the light source from each of the extracted image sets . The determination unit determines whether or not an abnormal output point is included in any of the images in the image group based on the consistency between the parameters depending on the plurality of light sources obtained from each image set . n is an integer of 4 or more.

  The three-dimensional shape measuring apparatus according to the one aspect described above is expressed as a three-dimensional shape measuring method, and a program and a storage medium for causing a computer to execute the three-dimensional shape measuring method are specific embodiments of the present invention. It is effective as

  According to the present invention, an abnormal output point of an image can be determined based on parameters obtained from a plurality of image sets.

  FIG. 1 shows a configuration example of a three-dimensional shape measuring apparatus according to an embodiment.

  The three-dimensional shape measurement apparatus includes a stage 12 on which a measurement object 11 is placed on a reference plane, a projection unit 13, an imaging unit 14, a CPU 15, a storage unit 16, a monitor 17, and an input unit 18. have. Here, the projection unit 13, the imaging unit 14, the storage unit 16, the monitor 17, and the input unit 18 are each connected to the CPU 15.

  The projection unit 13 is arranged so that the optical axis is inclined with respect to the reference plane of the stage 12, and projects the pattern light toward the measurement object 11 of the stage 12. The projection unit 13 includes a light source 21, an illumination optical system 22, a pattern formation unit 23, and a projection optical system 24. The illumination optical system 22 polarizes the illumination light from the light source 21 in the optical axis direction. The pattern forming unit 23 gives a pattern to the light flux parallel to the optical axis direction by the illumination optical system 22. The pattern forming unit 23 forms a striped (sine lattice) pattern light having a sinusoidal luminance change in one direction. The pattern forming unit 23 can shift the phase of the sine grating in the direction perpendicular to the optical axis of the projection unit 13. The pattern light formed by the pattern forming unit 23 is applied to the measurement object 11 on the stage 12 via the projection optical system 24.

  The imaging unit 14 is arranged so that the optical axis is perpendicular to the reference plane of the stage 12, and captures an image of pattern light projected on the measurement object 11 on the stage 12. The imaging unit 14 includes an imaging optical system 25 that forms an image of reflected light from the measurement object 11 and an imaging element 26 that captures an image of the measurement object 11.

  The CPU 15 is a processor that comprehensively controls the operation of the three-dimensional shape measuring apparatus. For example, the CPU 15 operates the projection unit 13 and the imaging unit 14 according to a program to capture an image of the measurement object 11 on which the pattern light is superimposed. Moreover, CPU15 calculates | requires the three-dimensional shape of the measuring object 11 by using the data of a some image.

  Moreover, CPU15 functions as the calculating part 15a and the determination part 15b by execution of a program. Here, the calculation unit 15a obtains parameters (amplitude value, offset value, and phase of pattern light) at each position on the screen corresponding to the imaging range of the imaging unit 14 based on the luminance change of the plurality of images. Moreover, the determination part 15b determines whether an abnormal output point is contained in said screen based on the parameter which the calculating part 15a calculated | required.

  The storage unit 16 is configured by a nonvolatile storage medium such as a flash memory. The storage unit 16 can store data of an image captured by the imaging unit 14 and point cloud data described later calculated by the CPU 15. The storage unit 16 stores a program executed by the CPU 15.

  The monitor 17 displays an image captured by the imaging unit 14 in accordance with an instruction from the CPU 15. Further, the input unit 18 receives various inputs including a measurement start instruction.

  Next, the principle of measuring a three-dimensional shape by the phase shift method in this embodiment will be briefly described. In the present embodiment, an example in which the shape of the measurement object 11 is obtained by the 4-bucket method will be described.

In the present embodiment, pattern light of a sine lattice is projected onto a stationary measurement object 11, and the measurement object 11 is imaged four times by shifting the phase of the pattern light by π / 2. When attention is paid to the luminance values (I ₁ , I ₂ , I ₃ , I ₄ ) at the coordinates (x, y) in the four captured frames, the relative lightness difference always changes the phase difference of the pattern light. Show. Therefore, the phase α of the projected pattern light at the coordinates (x, y) can be obtained by the following equation (1).

そして、画面内の各位置で位相αをそれぞれ求めた後に、位相αが等しい点を連結して等位相線を得る。この等位相線は、測定対象物１１を所定の平面で切断したときの断面形状を表している。そのため、投影部１３および撮像部１４の位置関係が既知であれば、三角測量の原理によって、この等位相線から測定対象物１１の各位置の高さを示す三次元的な距離画像のデータ（点群データ）を得ることができる。

And after calculating | requiring the phase (alpha) in each position in a screen, the point with equal phase (alpha) is connected and an equiphase line is obtained. This equiphase line represents a cross-sectional shape when the measurement object 11 is cut along a predetermined plane. Therefore, if the positional relationship between the projection unit 13 and the imaging unit 14 is known, three-dimensional distance image data indicating the height of each position of the measurement object 11 from this equiphase line (in accordance with the principle of triangulation) ( Point cloud data) can be obtained.

By the way, in the measurement of the three-dimensional shape described above, any one of the luminance values (I ₁ , I ₂ , I ₃ , I ₄ ) is caused by a disturbance such as a change in the external illumination environment or an abnormality of the three-dimensional shape measurement apparatus. If there is an abnormality, the correct phase cannot be obtained. Therefore, in this embodiment, the abnormal output point is determined in the screen in the following manner. Hereinafter, an operation example of the three-dimensional shape measurement apparatus of the present embodiment will be described with reference to the flowchart of FIG.

  Step S101: In response to a measurement start instruction from the input unit 18, the CPU 15 irradiates the measurement object 11 with pattern light and acquires a 4-frame image group. At this time, the CPU 15 images the measurement object 11 by shifting the phase of the pattern light by π / 2 for each frame so that the phase of the pattern moves by one cycle in four frames.

  Step S102: The CPU 15 obtains the phase α at each position in the screen from the above equation (1) using the luminance value of the image group (S101) of four frames. The phase α at each position in the screen obtained in S102 is recorded in the storage unit 16 by the CPU 15.

  Step S103: The CPU 15 extracts three frames from the four-frame image group (S101) and generates four different image sets.

  Step S104: In each of the four image sets (S103), the CPU 15 uses the three-frame images included in the image set to determine the light source-dependent parameters (pattern light amplitude value, pattern light offset value, pattern light (Phase).

  As an example, for an image set including a first frame, a second frame, and a third frame, the CPU 15 in S104 calculates each parameter in the following manner.

Here, the luminance values I ₁ , I ₂ , and I ₃ at the coordinates (x, y) in the first frame, the second frame, and the third frame can be expressed as the following Expression (2), respectively.

式（２）において、「ａ」はパターン光の振幅値を示している。また、「ｂ」はパターン光のオフセット値を示している。また、「φ」はパターン光の位相を示している。

In Equation (2), “a” indicates the amplitude value of the pattern light. “B” indicates an offset value of the pattern light. “Φ” indicates the phase of the pattern light.

  Then, when the above equation (2) is solved for a, b, and φ, it can be transformed as equation (3).

そのため、ＣＰＵ１５は、第１フレームから第３フレームまでの座標（ｘ，ｙ）の輝度値Ｉ₁，Ｉ₂，Ｉ₃を用いて式（３）を解くことで、画面内の座標（ｘ，ｙ）における各パラメータの値（パターン光の振幅値ａ、パターン光のオフセット値ｂ、パターン光の位相φ）を求める。勿論、ＣＰＵ１５は、同じ画像組における画面内の他の座標についても同様に各パラメータの値を求める。そして、上記の画像組から求めた各位置のパラメータの値は、ＣＰＵ１５によって記憶部１６に記録される。

Therefore, the CPU 15 solves the equation (3) using the luminance values I ₁ , I ₂ , and I ₃ of the coordinates (x, y) from the first frame to the third frame, so that the coordinates (x, The value of each parameter (pattern light amplitude value a, pattern light offset value b, pattern light phase φ) in y) is obtained. Of course, the CPU 15 similarly determines the value of each parameter for other coordinates in the screen in the same image set. Then, the value of the parameter at each position obtained from the image set is recorded in the storage unit 16 by the CPU 15.

  Note that, for the other three image sets, the CPU 15 obtains the values of the respective parameters by the calculation almost similar to the expressions (2) and (3).

  Step S105: The CPU 15 detects an abnormal output point in the screen using a parameter (S104) depending on the light source obtained for each image set. For example, the CPU 15 in S105 detects an abnormal output point in the screen by the following processing (A) or (B). Note that the position of the abnormal output point in the screen detected in S105 is recorded in the storage unit 16 by the CPU 15.

  (A) The CPU 15 obtains a variance value using parameters (a, b, φ) obtained from four image sets at each position in the screen. Specifically, focusing on the coordinates (x, y) in the screen, the values of a, b, and φ are obtained one by one from each image set (S104), and therefore correspond to the coordinates (x, y). There are four values for a, b, and φ. Therefore, the CPU 15 can obtain the respective variance values of a, b, and φ from the four sets of values of a, b, and φ with respect to the coordinates (x, y) in the screen.

  Then, the CPU 15 detects the coordinates (x, y) in the screen as an abnormal output point when the variance value of any of the above a, b, and φ is larger than the threshold value. That is, when there is no abnormality in any of the four frames imaged in S101, the values of the four sets of a, b, and φ are substantially the same. On the other hand, if there is an abnormality in any of the four frames imaged in S101, the four sets of a, b, and φ values vary, and the variance values of a, b, and φ exceed the threshold value.

  (B) The CPU 15 obtains the average value of the phase φ obtained from the four image sets at each position in the screen. Thereafter, the CPU 15 obtains a difference between the average value of the phase φ at the same position in the screen and the phase α obtained in S102. And CPU15 detects the position in the screen where said difference becomes larger than a threshold value as an abnormal output point. That is, when there is no abnormality in any of the four frames imaged in S101, the value of the phase α and the value of the four phases φ are almost the same. On the other hand, if there is an abnormality in any of the four frames imaged in S101, the value of the four φs varies, so the difference between the average of the phase φ and the phase α exceeds the threshold value.

  Step S106: The CPU 15 determines whether or not an abnormal output point is detected in S105. If the above requirement is satisfied (YES side), the process proceeds to S107. On the other hand, if the above requirement is not satisfied (NO side), the process proceeds to S108.

  Step S107: The CPU 15 executes an exception process accompanying detection of an abnormal output point. Specifically, the CPU 15 executes at least one of the following (1) to (3). Note that the CPU 15 may issue a warning to the outside by the display on the monitor 17 or the sound of a speaker (not shown) in accordance with the transition to the exception processing in S107.

  (1) The CPU 15 generates determination result data indicating the position (S105) of the abnormal output point. As an example, the CPU 15 generates text data indicating the coordinates of abnormal output points in the screen and image data indicating the position distribution of abnormal output points in the screen. The CPU 15 records the determination result data in the storage unit 16 or displays it on the monitor 17.

  Note that after the determination result data is generated, the CPU 15 may shift to the process of S108 (illustration of the shift to S108 is omitted in FIG. 2). In this case, the CPU 15 may delete the abnormal output point data in the final point cloud data.

(2) The CPU 15 interpolates the measurement value of the abnormal output point using the measurement values around it. Specifically, the CPU 15 uses the luminance values (I ₁ , I ₂ , I ₃ , I ₄ ) of the coordinates (x, y) corresponding to the position (S105) of the abnormal output point as the luminance values of the surrounding pixels. Interpolate with. And CPU15 calculates | requires the phase (alpha) of a coordinate (x, y) using said luminance value after interpolation. Thereafter, the CPU 15 proceeds to the processing of S108 (illustration of the transition to S108 is omitted in FIG. 2).

  (3) The CPU 15 instructs the projection unit 13 and the imaging unit 14 to re-shoot the image. In this case, the CPU 15 proceeds to S101 and repeats the above operation (the illustration of the transition to S101 is omitted in FIG. 2).

  Step S108: The CPU 15 generates point cloud data of the measurement object 11 based on the phase α (S102) at each position in the screen. Note that the point cloud data generated in S108 is recorded in the storage unit 16 by the CPU 15. This is the end of the description of FIG.

  Hereinafter, the effect of this embodiment is demonstrated. In the three-dimensional shape measuring apparatus according to the present embodiment, parameters (a, b, φ) depending on the light source from the three-frame image among the four-frame images captured under the same conditions except for shifting the phase of the pattern light. The presence / absence of an abnormal output point in the screen is determined by checking the consistency of the parameters. Therefore, the three-dimensional shape measuring apparatus of the present embodiment can obtain highly reliable three-dimensional measurement data.

<Supplementary items of the embodiment>
(1) In the above embodiment, an example in which the parameter depending on the light source is obtained for the entire photographing range has been described. However, the parameter depending on the light source may be obtained for a part of the photographing range. At this time, the area for obtaining the parameter depending on the light source may be set automatically by the CPU 15 analyzing the image, or may be set by allowing the user to specify a range.

  As an example, the CPU 15 has a low-contrast area where the contrast is less than or equal to a threshold in an arbitrary frame, a high-brightness area where the luminance value of the image is equal to or higher than the first luminance threshold, A low luminance region that is equal to or lower than the luminance threshold is extracted. Then, the CPU 15 may obtain the parameter depending on the light source for at least one of the low contrast region, the high luminance region, and the low luminance region. As a result, the calculation load on the CPU 15 can be reduced by setting only the portion considered to have low data reliability as the calculation target.

  In addition, the first luminance threshold and the second luminance threshold may be set with reference to a range in which the luminance level output with respect to the amount of light incident on the imaging unit 14 has linearity. In other words, when the output characteristic of the imaging unit 14 has a knee region on the high luminance side or the low luminance side, the CPU 15 may obtain a parameter depending on the light source in a portion indicating a luminance value corresponding to the knee region. . Further, the CPU 15 matches the first luminance threshold and the second luminance threshold with the saturation level of the output characteristic of the imaging unit 14 to obtain a parameter that depends on the light source in a portion where overexposure or underexposure occurs. May be.

  As another example, the CPU 15 displays a captured image on the monitor 17 and receives a range designation input for obtaining a parameter depending on the light source from the input unit 18. And CPU15 calculates | requires the parameter depending on a light source in the partial area | region corresponding to said range designation | designated. Thereby, the calculation load of CPU15 can be reduced by making only the part considered that the reliability of data is low from a user's experience as a calculation object. Further, if the user designates the outer edge of the measurement object 11 on the screen, the part of the measurement object 11 that has not been photographed can be excluded from the calculation target. FIG. 3 schematically shows an example of the display screen when the above range is specified.

(2) Although the case of the 4-bucket method has been described in the above embodiment, the present invention is naturally applicable to the case of a multi-bucket method in which imaging of four frames or more is performed by measuring a three-dimensional shape by the phase shift method. At this time, there are _n C ₃ combinations for extracting an arbitrary three-frame image set from n frames, but the CPU 15 calculates a light source-dependent parameter (a, b, φ) for each of these image sets, What is necessary is just to confirm the consistency of parameters between image sets.

  (3) In the above embodiment, an example has been described in which the CPU 15 implements the functions of the calculation unit 15a and the determination unit 15b by software in a program. However, these configurations may be realized by hardware by an ASIC. Absent.

  (4) Note that the program stored in the storage unit 16 in the above embodiment may be a firmware program updated by version upgrade or the like. That is, the function of the three-dimensional shape measurement apparatus of the present embodiment may be provided by updating the firmware program of the existing three-dimensional shape measurement apparatus.

  It should be noted that the present invention can be implemented in various other forms without departing from the spirit or main features thereof. Therefore, the above-described embodiment is merely an example in all respects and should not be interpreted in a limited manner. The present invention is defined by the claims, and the present invention is not limited to the text of the specification. Further, all modifications and changes belonging to the equivalent scope of the claims are within the scope of the present invention.

The figure which shows the structural example of a three-dimensional shape measuring device Flow chart showing an operation example of the three-dimensional shape measurement apparatus of the present embodiment Schematic diagram showing an example of the display screen when the range is specified

Explanation of symbols

DESCRIPTION OF SYMBOLS 11 ... Measuring object, 13 ... Projection part, 14 ... Imaging part, 15 ... CPU, 16 ... Memory | storage part, 17 ...
Monitor, 18 ... Input section

Claims (11)

A projection unit that irradiates a measurement object with pattern light whose luminance distribution changes in a sinusoidal shape;
An image of the pattern light projected onto the measurement object from a direction different from the projection direction of the projection unit is captured, and an image group including n images in which the phase of the pattern light is shifted from each other is obtained. An imaging unit to
_N C ₃ different image sets are extracted from the _n images, and a calculation unit for obtaining a parameter depending on the light source from each of the extracted image sets ;
A determination unit that determines whether or not an abnormal output point is included in any image of the image group, based on consistency between parameters that depend on the plurality of light sources obtained from each image set ;
Equipped with a,
N is an integer of 4 or more,
A three-dimensional shape measuring apparatus characterized by that. The three-dimensional shape measuring apparatus according to claim 1,
The three-dimensional shape measuring apparatus, wherein the parameter includes any one of an amplitude value of the pattern light, an offset value of the pattern light, and a phase of the pattern light. In the three-dimensional shape measuring apparatus according to claim 2,
The determination unit performs the determination based on a variance value between parameters depending on a plurality of the light sources . In the three-dimensional shape measuring apparatus according to claim 2,
The calculation unit obtains a first phase based on a luminance change at the same point of each image of the image group, and obtains a second phase obtained by averaging a plurality of phases obtained from the image set ,
The determination unit performs the determination based on a difference between the first phase and the second phase at the same point. In the three-dimensional shape measuring apparatus according to any one of claims 1 to 4,
The determination unit performs the determination with respect to at least one of a partial region in which the contrast is a threshold value or less in the image and a partial region in which the luminance value is outside a predetermined range in the image. Original shape measuring device. In the three-dimensional shape measuring apparatus according to any one of claims 1 to 4,
An input unit that receives a designation input of a partial area of the image;
The three-dimensional shape measuring apparatus according to claim 1, wherein the determination unit performs the determination with respect to the designated and input partial region. In the three-dimensional shape measuring apparatus according to any one of claims 1 to 5,
When the determination unit determines that the abnormal output point is included, the determination unit outputs determination result data indicating a position of the abnormal output point. In the three-dimensional shape measuring apparatus according to any one of claims 1 to 5,
When the determination unit determines that the abnormal output point is included, the determination unit interpolates measurement data of the abnormal output point with surrounding measurement data. In the three-dimensional shape measuring apparatus according to any one of claims 1 to 5,
When the determination unit determines that the abnormal output point is included, the determination unit instructs the imaging unit to reacquire the image group. The pattern light whose luminance distribution changes in a sine wave shape is irradiated onto the measurement object, and an image of the pattern light projected onto the measurement object from a direction different from the projection direction of the projection unit is captured, and between the images Obtaining an image group consisting of n images whose phase of the pattern light is shifted from each other;
_N C ₃ image sets are extracted from the _n images, a light source-dependent parameter is obtained from each of the extracted image sets, and the parameters depending on the plurality of light sources are based on the consistency among the parameters. Determine whether any image in the image group contains an abnormal output point ,
N is an integer of 4 or more,
A three-dimensional shape measuring method characterized by that.   A program for causing a computer to execute the three-dimensional shape measurement method according to claim 10.

Images