JP2006329684A - Image measuring instrument and method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To efficiently carry out an assigned operation for a measuring portion by selecting a feature point. <P>SOLUTION: A solid imaging element 21 images a measuring object 30 from different view points to acquire an image in every of the view points. An image processor 23 extracts the feature point from an image of the measuring object 30 in the image, and searches a point corresponding to the feature point from the image of the measuring object 30 in the image different from the image from which the feature point is extracted, a prescribed mark is displayed as to the point of which the point corresponding to the feature point out of the feature points on the image is obtained by the search, a selection result when one or more of the feature points are selected is acquired out of the feature points displayed with the mark, and three-dimensional measurement is executed by triangulation based on a parallex at the time when picking up the image in every parallex, using the feature point concerned in the selection result as the measuring object. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an image measurement technique, and more particularly to a technique for performing three-dimensional measurement by triangulation using images obtained by imaging a measurement object from different viewpoints.

  2. Description of the Related Art Stereo measurement apparatuses that perform three-dimensional measurement by triangulation by acquiring images by imaging a measurement object from a plurality of viewpoints and specifying a measurement location on the acquired images are used. In such an apparatus, when characteristics such as the shape of the measurement object are unknown, the user manually performs measurement by specifying a measurement location.

  With regard to such a measuring apparatus, for example, in Patent Document 1, an image is acquired by a pair of imaging systems, an arbitrary position in the image displayed on the LCD display is designated by a measurement point input apparatus, and three-dimensional measurement is performed. An apparatus for performing the above-described calculation is disclosed.

  According to the device disclosed in Patent Document 1, there may be a case where an error occurs in the designation of the measurement location due to individual differences in the designation of the measurement location manually performed by the user, and accurate measurement cannot be performed. It can be considered that a long time is required for the operation. Therefore, it is possible to specify a measurement location without causing a measurement location specification error by extracting a feature point on the image by image processing and selecting the extracted feature point by the user.

In addition, as a technique related to the present invention, for example, Non-Patent Document 1 discloses an image processing technique for extracting feature points common to a plurality of images.
Japanese Patent Laid-Open No. 5-18748 Satoshi Kanazawa and Kenichi Kanaya, “Extraction of image feature points for computer vision”, Journal of the Institute of Electronics, Information and Communication Engineers, Institute of Electronics, Information and Communication Engineers, December 2004, Vol. 87, No. 12, p. 1043-1048

  In the apparatus disclosed in Patent Document 1 listed above, correspondence between feature points of a plurality of images is not taken into consideration, so that it is inaccurate by selecting feature points that cannot be dealt with (cannot be measured). There are problems such as troublesome measurement and a problem that it may take time to confirm that the feature points are supported.

  In addition, when extracting feature points by image processing, depending on the image, a large number of feature points may be extracted that are too sufficient for measurement, and there is a problem that the feature point selection operation is rather complicated.

  The present invention has been made in view of the above-described problems, and a problem to be solved is to make it possible to efficiently specify a measurement location by selecting a feature point.

  An image measurement apparatus according to one aspect of the present invention includes an imaging unit that captures an image of a measurement object from different viewpoints and acquires an image for each viewpoint, and a feature that extracts a feature point from the image of the measurement object in the image Point extraction means, search means for searching for a point corresponding to the feature point from the image of the measurement object in the image different from the image from which the feature point is extracted, and the feature point of the feature point on the image Mark display means for displaying a predetermined mark for the point corresponding to the feature point obtained by the search, and when one or more feature points are selected from the feature points displaying the mark A feature point selection result acquisition means for acquiring a selection result, and a measurement means for performing three-dimensional measurement by triangulation based on parallax at the time of capturing an image for each viewpoint, using the feature point according to the selection result as a measurement target; And characterized in that it has a, to solve the problems described above by this feature.

  In the above-described image measuring device according to the present invention, the search is performed by matching processing between the images for each viewpoint, and the mark display unit determines the evaluation value that is the processing result of the matching processing. Accordingly, at least one of the color, shape, and size of the mark to be displayed may be changed.

  The image measurement apparatus according to the present invention described above further includes an extraction condition instruction acquisition unit that acquires an instruction regarding a condition for extracting the feature point, and the feature point extraction unit includes the feature that matches the condition. You may comprise so that a point may be extracted.

  At this time, the instruction on the condition for extracting the feature point includes the feature point form, the feature value indicating the degree of the feature of the feature point, and the characteristics of the pixels constituting the image. It may be at least one of the pixel information to be represented.

Here, the indication of the form of the feature point may be an independent point, an intersection, a pole, an inflection point, or an end point.
At this time, the instruction of the pixel information may be information indicating the luminance or color difference of the pixel.

  Further, in the above-described image measuring device according to the present invention, the feature point selection result acquisition unit is a pointer indicating any one of the feature points on which the mark is displayed on the image and is movable. A pointer may be displayed, and the feature point indicated by the pointer when the pointer is moved may be acquired as the selection result.

  An image measurement method according to another aspect of the present invention captures a measurement object from different viewpoints, acquires an image for each viewpoint, and extracts feature points from the image of the measurement object in the image. The point corresponding to the feature point is searched from the image of the measurement object in the image different from the image from which the feature point is extracted, and the point corresponding to the feature point on the image is the point corresponding to the feature point. A predetermined mark is displayed with respect to what is obtained by the search, and a selection result when one or more feature points are selected from the feature points on which the mark is displayed is obtained, and the selection result is related to The characteristic point is to perform three-dimensional measurement by triangulation based on the parallax at the time of capturing an image for each viewpoint, and the above-described problems are solved.

  According to the present invention, as described above, the operation of specifying the measurement location by selecting the feature points can be performed efficiently, so that a stereo image measurement device capable of accurate and easy measurement can be provided. The effect of becoming.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. Here, an embodiment in which the present invention is implemented with a measurement endoscope apparatus capable of stereo measurement will be described.
FIG. 1 shows the overall configuration of an endoscope apparatus for measurement embodying the present invention. In the figure, an endoscope apparatus for measurement 10 includes an endoscope insertion portion 11 configured to be detachable from an optical adapter capable of stereo measurement, a control unit 12 that houses the endoscope insertion portion 11, A remote controller 13 for giving operation instructions necessary for various operation control of the entire system of the measurement endoscope apparatus 10 and a liquid crystal monitor (hereinafter, a processing menu) for displaying an endoscope image and operation control contents (hereinafter, processing menu). , LCD) 14, a normal endoscopic image, or a face-mount display (hereinafter referred to as FMD) 17 that can stereoscopically view the endoscopic image as a pseudo stereo image, and image data to FMD 17. The FMD adapter 18 to be supplied is provided.

Next, the system configuration of the measurement endoscope apparatus 10 will be described with reference to FIG.
As shown in FIG. 2, the control unit 12 includes a video signal processing circuit 22 and an image processing device 23. The endoscope insertion unit 11 freely curves a pair of lens systems 20L and 20R at the tip, a solid-state image sensor 21 as an imaging means, an illumination device (not shown), and the tip of the endoscope insertion unit 11. And a bending device (not shown). The pair of lens systems 20L and 20R are provided at a predetermined distance in the lateral direction, and image the same subject (measurement object 30).

  An imaging signal from the solid-state imaging device 21 is converted into image data by the video signal processing circuit 22 and then supplied to the image processing device 23. The image processing device 23 performs image processing, measurement, recording, and the like on the supplied image data in response to an operation on the remote controller 13. The video signal processing circuit 22 further synthesizes the supplied image data with the graphic data generated by the image processing device 23, converts the combined image data into a video signal such as an NTSC signal, and supplies it to the LCD 14. To do. The LCD 14 displays a composite image of the endoscopic image and the graphic image represented by the supplied video signal.

Next, how to obtain the three-dimensional coordinates of the measurement point by stereo measurement will be described with reference to FIG.
First, the coordinates of the measurement points on the original image captured by the left and right optical systems are (X _L , Y _L ) and (X _R , Y _R ), respectively, and the three-dimensional coordinates of the measurement target points are (X , Y, Z).

Here, the distance between the optical centers of the left optical system and the right optical system is D, and the focal length is F.
t = D / (X _L −X _R )
If t is defined, according to the principle of triangulation,
X = t × X _R + D / 2
Y = t × Y _R
Z = t × F
It becomes.

  As described above, when the coordinates of the measurement point on the original image are determined, the three-dimensional coordinates of the measurement point can be obtained from the known parameters D and F. By obtaining the three-dimensional coordinates of several points in this way, the distance between two predetermined points, the distance between a line connecting the two predetermined points and another one point, the area, the depth, the surface shape, etc. Various measurements can be performed. This is stereo measurement.

  Next, a procedure for performing stereo measurement using the measurement endoscope apparatus 10 will be described with reference to FIGS. 4, 5 </ b> A, and 5 </ b> B. FIG. 4 is a flowchart showing a stereo measurement procedure. 5A and 5B show examples of display screens on the LCD 14 at the time of stereo measurement by the measuring endoscope apparatus 10, and the screen examples of FIG. 5A are lacking in turbine blades which are engine parts of aircraft. FIG. 5B shows the shape of an icon used for designating the characteristics of the measurement location in this measurement. ing.

  In FIG. 4, first, in step SlOl, the user operates the endoscope insertion unit 11 and the remote controller 13 to capture an image of a measurement target and acquire an image. In the screen example shown in FIG. 5, the left image 210 and the right image 211, which are acquired images, represent the measurement objects 300 and 301 captured by the left optical system and the right optical system, respectively. Has been.

  Here, when measuring the captured measurement objects 300 and 301, the user performs a predetermined operation on the remote controller 13. If this operation is detected in the determination procedure of step S102 of FIG. 4, the determination result is Yes, and the procedure proceeds to step S103. In step S102, if the operation described above is not detected, the determination result is No, the procedure returns to step S101, and the above-described procedure is repeated.

  After giving the measurement execution instruction described above, the user further operates the remote controller 13 to select the measurement mode icon 200 in the measurement screen, and the measurement mode (for example, the measurement type in the measurement endoscope apparatus 10) (for example, Specify distance measurement between two points, area measurement, etc.). In step S103, this selection content is acquired, and predetermined settings necessary for operating the measurement endoscope apparatus 10 in the selected measurement mode are performed.

  Next, the user further operates the remote controller 13 to select the select mode icon 220 in the measurement screen and gives an instruction to start selecting feature points. If this instruction is detected in the determination procedure of step S104, the determination result is Yes, and the procedure proceeds to step S105.

  If the selection of the select mode icon 220 is not detected in step S104, the determination result is No, the procedure proceeds to step S120, the user's operation content for the remote controller 13 is acquired, and the manual operation is performed. In step S121, the image processing device 23, which functions as a measurement unit, obtains the designation of the measurement location, and performs an operation for calculating the three-dimensional coordinates for the designated measurement location as described above. Based on this, the calculation for the three-dimensional measurement in the measurement mode acquired in step S103 is executed.

  When the select mode icon 220 is selected as described above, in step S105, the image processing device 23 that also functions as a feature point extracting unit first displays the measurement object 300 appearing on the left image 210. Processing for extracting feature points such as edges in the image is executed. Points 310 to 318 in the left image 210 of FIG. 5A indicate feature points extracted by this extraction processing. The feature points are extracted using a known method such as the method disclosed in Non-Patent Document 1 described above.

  Next, in step S <b> 106, the image processing device 23 functioning as a search unit searches for a point corresponding to the extracted feature point from the right image 211 (for example, disclosed in Non-Patent Document 1 described above). Template matching). Then, in the subsequent step S107, the image processing device 23 that also functions as a mark display means, among the corresponding points found in the right image 211 and the feature points in the left image 210 on the measurement screen being displayed on the LCD 14, A process of displaying a solid white circle mark (◯ mark) on a case where a corresponding point has been found is executed. The circles indicated by the feature points 310 to 314 in the left image 210 of FIG. 5A and the corresponding points 320 to 324 in the right image are displayed by this processing.

  In the processing executed in step S107, the color, shape, size, etc. of the circles can be changed according to the evaluation value of the processing result of the matching processing in step S106. For example, the higher the evaluation value, the darker the ◯ mark may be displayed. Further, in the processing executed in step S107, the feature points in the left image 210 for which the corresponding points cannot be found are feature points 315 to 318 in the left image 210 in FIG. 5A. In addition, a broken-line white circle may be displayed, or no mark may be displayed.

  In this way, by changing the display of the circles according to the evaluation value of the processing result of the matching process, the user can specify the measurement location after confirming the feature points that can be measured.

  After the display process in step S107, the user confirms whether or not a circle is present at a desired measurement location. Here, when there is no circle at the desired measurement location, or when a circle is displayed. If it is difficult to select a desired measurement location due to excessive, the remote controller 13 is operated to make a request to change the feature point extraction condition. If the request is detected in the determination procedure of step Sl08, the determination result is Yes, and the procedure proceeds to step S109. Here, when the request is not detected (that is, when a circle is appropriately present at a desired measurement location), the determination result is No, and the procedure proceeds to step S110.

  An instruction to change the setting of conditions for extracting feature points from the captured image is given, and in step S109, the image processing apparatus 23 functioning as an extraction condition instruction acquisition unit acquires this instruction.

Here, setting of feature point extraction conditions will be described.
In this embodiment, as a method for designating feature point extraction conditions, the following items can be selected.
[1] A feature value indicating the degree of the feature that the feature point has, for example, a parameter value such as a threshold value can be designated. For example, as shown in (1) of FIG. 5B, by selecting the parameter setting icon 230, the lower limit value setting icon 231 and the upper limit value setting icon 232, and setting the parameters, the range of the set parameters ( For example, feature points are extracted based on sharpness in edge extraction. Thereby, the number of extracted feature points can be adjusted.
[2] Designate the feature point form to be extracted. For example, as shown in (2) of FIG. 5B, when any one of the independent point form icon 240, the intersection form icon 241, the extreme point form icon 242, the inflection point form icon 243, and the end point form icon 244 is selected. Only feature points having the specified form are extracted. Thereby, the number of extracted feature points can be adjusted.
[3] Select the type of pixel information representing the characteristics of the pixels constituting the image that is the target of feature point extraction. For example, as shown in (3) of FIG. 5B, when either the luminance icon 250 or the color difference icon 251 is selected, feature points are extracted for the type (luminance or color difference) of the selected pixel information. Thereby, the number of extracted feature points can be adjusted.

  In FIG. 5A, a feature point extraction condition change request and an item switching instruction for setting the feature point extraction condition described above are performed by selecting an option icon 221. Each icon illustrated in FIG. 5B is an option icon. The display is switched in the option display area 222 in response to a click operation on 221.

  Returning to the description of FIG. 4, when the feature point extraction condition setting change instruction is acquired in step S109 as described above, the procedure returns to step S105 and the procedure after the feature point extraction processing by the image processing device 23 is performed. Are repeated, and feature points that match the feature point extraction condition changed by the change instruction are extracted.

  On the other hand, the user who has not made the feature point extraction condition change request in the determination procedure of step Sl08 operates the remote controller 13 and selects some of the feature points displayed with circles. In step S110, the image processing device 23 acquires the selection result of the feature points.

Selection of the feature point by the user is performed by the following operation.
When the user performs a predetermined pointer movement operation with the remote controller 13, pointers (pointers) 205 and 206 indicating the feature points on which the circles are displayed move between the circles. Here, the pointer 205 is displayed on the left image 210, and the pointer 206 is displayed on the right image 211. In this way, the image processing device 23 displays the pointer on each of the left image 210 and the right image 211 on the LCD 14 so that the user can confirm whether or not matching at a desired measurement location has been appropriately performed. Can be easily performed.

  Note that the image processing device 23 is displaying an enlarged image on the LCD 14 around the pointer 205 in the left image 210 of the measurement screen or the pointer 206 in the right image 211 of the measurement screen during the feature point selection operation in step S110. Are displayed on the zoom window 204 in the measurement screen. With this enlarged image, the user can confirm whether or not the desired measurement location is correctly specified. The magnification of the enlarged image can be adjusted by a predetermined magnification changing operation of the remote controller 13.

  After confirming that the measurement location is correctly specified, the user performs a predetermined selection operation on the remote controller 13 to confirm the selection of the feature point, whereby the measurement location is specified, and as a feature point selection result acquisition unit The functioning image processing device 23 acquires the selection result of the feature points. The designation of the measurement location is repeated a predetermined number of times determined according to the measurement mode. For example, the measurement location is designated twice for the measurement between two points.

  Finally, in step S111, the image processing device 23 functioning as a measurement unit performs an operation for calculating the three-dimensional coordinates as described above for the designated measurement location, and is acquired in step S103 based on the operation result. The calculation for the three-dimensional measurement in the measurement mode that has been performed is executed.

As described above, the stereo measurement in the measurement endoscope apparatus 10 is completed.
As described above, in the measurement endoscope apparatus 10 according to the present embodiment, the measurement point is designated and measured from the feature point extracted by the feature point extraction unit, and thus individual differences may occur in the designation of the measurement point. Absent. Furthermore, since the measurement location is designated simply by selecting the measurement location from the extracted feature points, the measurement location designation operation can be performed in a short time.

  Furthermore, in the measurement endoscope apparatus 10 according to the present embodiment, only the feature points that have been found to correspond in the left and right images during stereo measurement can be selected by the user as measurement locations. Accordingly, it is possible to prevent erroneous specification of a point that cannot be measured as a measurement location, thereby improving the efficiency of the operation for specifying the measurement location.

  In the above-described embodiment, the example in which the present invention is implemented by the measurement endoscope apparatus has been described. However, the present invention can also be applied to a general image measurement apparatus. In addition, the present invention is not limited to the above-described embodiments, and various improvements and changes can be made without departing from the scope of the present invention.

1 is a diagram showing an overall configuration of an endoscope apparatus for measurement embodying the present invention. It is a figure which shows the system configuration | structure of the endoscope apparatus for measurement shown in FIG. It is explanatory drawing of how to obtain | require the three-dimensional coordinate of the measurement point by stereo measurement. It is the figure which showed the procedure of the stereo measurement by the endoscope apparatus for measurement shown in FIG. 1 with the flowchart. It is a figure which shows the example of a screen of a measurement screen. It is a figure which shows the example of the shape of the icon used in order to designate the characteristic of a measurement location.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Endoscope apparatus for measurement 11 Endoscope insertion part 12 Control unit 13 Remote controller 14 Liquid crystal monitor (LCD)
17 Face Mount Display (FMD)
18 FMD adapter 20L, 20R Lens system 21 Solid-state imaging device 22 Video signal processing circuit 23 Image processing device 30, 300, 301 Measurement object 200 Measurement mode icon 204 Zoom window 205, 206 Pointer 210 Left image 211 Right image 220 Select mode icon 221 Option icon 222 Option display area 230 Parameter setting icon 231 Lower limit value setting icon 232 Upper limit value setting icon 240 Independent point form icon 241 Intersection form icon 242 Extreme point form icon 243 Inflection point form icon 244 End point form icon 250 Luminance icon 251 Color difference icon 310, 311, 312, 313, 314, 315, 316, 317, 318 Feature point 320, 321, 322, 323, 324応点

Claims (8)

Imaging means for imaging a measurement object from different viewpoints and acquiring an image for each viewpoint;
Feature point extracting means for extracting feature points from the image of the measurement object in the image;
Search means for searching for a point corresponding to the feature point from the image of the measurement object in the image different from the image from which the feature point is extracted;
Mark display means for displaying a predetermined mark on a point corresponding to the feature point among the feature points on the image obtained by the search;
Feature point selection result acquisition means for acquiring a selection result when one or more feature points are selected from among the feature points on which the mark is displayed;
Measuring means for performing three-dimensional measurement by triangulation based on parallax at the time of capturing an image for each viewpoint, with the feature point according to the selection result as a measurement target;
An image measuring device comprising: The search is performed by a matching process between the images for each viewpoint,
The mark display means changes at least one of the color, shape, and size of the mark to be displayed according to an evaluation value that is a processing result of the matching process.
The image measurement apparatus according to claim 1, wherein Further comprising extraction condition instruction acquisition means for acquiring an instruction about a condition for extracting the feature points;
The feature point extracting means extracts the feature points that match the condition;
The image measurement apparatus according to claim 1, wherein   The instruction about the condition for extracting the feature point includes: a feature point form, a feature value representing the feature level of the feature point, and pixel information representing the characteristics of the pixels constituting the image. The image measuring device according to claim 3, wherein the image measuring device is at least one of them.   The image measurement apparatus according to claim 4, wherein the indication of the feature point form is an independent point, an intersection point, a pole point, an inflection point, or an end point.   The image measurement apparatus according to claim 4, wherein the instruction of the pixel information is information indicating a luminance or a color difference of the pixel.   The feature point selection result acquisition means displays a movable pointer indicating one of the feature points on which the mark is displayed on the image and moves the pointer. The image measurement apparatus according to claim 1, wherein the feature point indicated by the pointer is acquired as the selection result. Capture the object to be measured from different viewpoints and acquire images for each viewpoint.
Extracting feature points from the image of the measurement object in the image;
Searching for a point corresponding to the feature point from the image of the measurement object in the image different from the image from which the feature point is extracted;
A point corresponding to the feature point on the image obtained by the search is displayed with a predetermined mark;
Obtaining a selection result when one or more feature points are selected from the feature points on which the mark is displayed;
Performing three-dimensional measurement by triangulation based on parallax at the time of capturing an image for each viewpoint, with the feature point according to the selection result as a measurement target,
An image measurement method characterized by that.