JP4373038B2 - Measurement program - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention , Measurement images recorded by a measurement endoscope device that performs measurement by image processing using endoscopic images can be handled on a personal computer. Measurement program About.
[0002]
[Prior art]
In general, in order to examine a test object in more detail with an endoscope, it is necessary to measure the test object, and in order to satisfy such a requirement, a conventional test using an endoscope is required. Various proposals for measuring means capable of measuring a specimen are disclosed.
[0003]
For example, in the proposal described in Japanese Patent Laid-Open No. 10-248806, a measurement endoscope apparatus using stereo measurement is shown. In addition, in the proposal described in Japanese Patent Application No. 2000-101122 filed earlier by the present applicant, a measurement endoscope that automatically selects and executes a different measurement method depending on the type of the optical adapter. A mirror device is shown.
[0004]
In the measurement endoscope apparatus described in the former Japanese Patent Laid-Open No. 10-248806, an optical adapter having two optical systems necessary for imaging and measuring a test object can be attached to and detached from the endoscope body. The image data of the two lens systems in the optical adapter is formed on one image sensor, and measurement is performed by image processing using at least the obtained endoscopic image. Optical data of the optical adapter Processing for reading information from a recording medium on which recording is performed, processing for correcting optical data based on the position error of the imaging system of the endoscope main body, processing for converting the coordinates of an image measured based on the corrected optical data, And a measurement processing means for performing processing for obtaining three-dimensional coordinates of an arbitrary point by matching two images based on the two images that have undergone coordinate conversion.
In the measurement endoscope apparatus having the above configuration, two images are obtained on the basis of two pieces of image information obtained by coordinate conversion of two images of a test object (subject) captured by the image sensor via the optical adapter. The three-dimensional coordinates of an arbitrary point on the test object are obtained by the matching of This makes it possible to realize a measurement endoscope apparatus that is inexpensive and has excellent measurement accuracy.
[0005]
On the other hand, the measurement endoscope apparatus described in the latter Japanese Patent Application No. 2000-101122 has a plurality of connecting portions provided at the distal end portion of the endoscope and a plurality of subject images that can be attached to and detached from the connecting portions. In a measurement endoscope apparatus that connects one type of optical adapter and one of the optical adapters and measures an image signal of the image sensor by image processing, display data associated with the plurality of optical adapters in advance The menu display process for performing the selection operation and the measurement processing means for performing the measurement process based on the selection result of the menu display process are provided.
[0006]
In the measurement endoscope apparatus configured as described above, when an optical adapter is selected on the menu, a measurement method corresponding to the optical adapter is automatically selected. It is possible to execute measurement processing corresponding to the selected measurement method only by pressing the measurement execution switch.
[0007]
In addition, the result of performing the calibration process for each of the plurality of optical adapters in the embodiment described in the above publication is stored as measurement environment data in a compact flash (R) memory card, which is an external storage medium. A configuration in which the measurement environment data suitable for the corresponding optical adapter is used by selecting an appropriate optical adapter is disclosed.
[0008]
[Problems to be solved by the invention]
Usually, a measurement image is a video endoscope body, a measurement optical adapter having an objective lens, a camera control unit (CCU) for converting a signal output from the video endoscope into a TV signal, and a TV signal as a digital signal. Depending on the type of recording device or the individual used for converting to and recording, the characteristics of the image appearance change. For this reason, the measurement endoscope device must be set correctly without making various measurement environment settings so that it is suitable for the combination of the devices when the measurement image is taken. I can't.
[0009]
Therefore, in an apparatus or system that performs measurement, it is also important to create data necessary for correcting the measurement image before the process of actually calculating the measurement value, and this process is an apparatus that records the image. Since the processing related to the measurement described above has been incorporated in the image recording device because it has a characteristic that depends on the image, conventionally, when re-measurement is performed on a recorded image, measurement is performed only on this device. Could not do. Or, if there is no individual difference of the recording device, it was possible to measure the image recorded by another device on another device, but again to perform re-measurement on the measurement endoscope device It was limited.
[0010]
Recently, as the types of endoscope bodies, optical adapters, CCUs, and image recording devices have increased to suit various measurement applications, various combinations of devices (measurement environments) Opportunities to perform remeasurement with recorded measurement images are increasing.
[0011]
There are also many commercial software programs for PCs that not only capture and display recorded images on a personal computer, but also facilitate these operations.
[0012]
Therefore, it has become common to manage many types of measurement images with such PC software, but since there was no means and processing to enable measurement environment settings on a PC, It was not possible to remeasure the measurement image captured on the computer.
[0013]
The present invention has been made in view of the above circumstances, and the measurement image recorded by the measurement endoscope apparatus can be re-measured on a personal computer to improve the work efficiency of the inspection. Measurement program The purpose is to provide.
[0014]
In addition, it is necessary to set the measurement environment suitable for the images recorded in various measurement environments to the personal computer, but it is possible to prevent measurement in the wrong measurement environment. By enabling re-measurement in a correct measurement environment with simple operation, re-measurement can be performed immediately without being aware of the measurement environment, and the operability at that time can be improved. Measurement program It is also intended to provide.
[0015]
[Means for Solving the Problems]
The measurement program of the present invention includes stereo image data including two images having parallax imaged through a stereo measurement adapter, and calibration data indicating optical characteristics of the stereo measurement adapter; Measurement environment information indicating the type of the calibration data; Image file with Acquireable image acquisition Step and A determination step for determining whether or not calibration data is added to the image file captured by the image acquisition step, and the determination step includes adding the calibration data to the image file captured by the image acquisition step. When determining that the calibration data has been added by the determination step, reading the calibration data added to the image file from the image file captured by the image acquisition step, and the determination step However, when it is determined that the calibration data is not added to the image file captured by the image acquisition step, the calibration data corresponding to the measurement environment information is at least one arbitrary reference with reference to the measurement environment information. The calibration data is recorded on the recording medium. Determining whether or not calibration data corresponding to the measurement environment information is recorded on the recording medium, and reading calibration data corresponding to the measurement environment information from the recording medium. Read out Based on the calibration data, The image file captured by the image acquisition step has Correcting the geometric distortion included in the stereo image data to generate corrected stereo image data; performing a matching process between the two images included in the corrected stereo image data; and the matching process And the results of Read out The method includes a step of calculating three-dimensional coordinates at an arbitrary point of the corrected stereo image data based on the calibration data, and a step of measuring a subject using the three-dimensional coordinates.
[0016]
DETAILED DESCRIPTION OF THE INVENTION
Embodiments of the present invention will be described below with reference to the drawings.
[0017]
1 to 15 relate to an embodiment of the present invention, FIG. 1 is a perspective view showing a system configuration of a measurement endoscope apparatus, and FIG. 2 shows an electrical circuit configuration of the measurement endoscope apparatus of FIG. FIG. 3 is a perspective view showing the configuration of the remote controller of FIG. 1, FIG. 4 is a perspective view showing the configuration of the endoscope tip portion of the endoscope insertion portion of FIG. 1 with a stereo measurement adapter, and FIG. 4 is a cross-sectional view taken along line AA in FIG. 4, FIG. 6 is a diagram showing an endoscopic image with the stereo measurement adapter in FIG. 4, and FIG. 7 is a diagram showing a mask shape image of the stereo measurement adapter in FIG. 8 is a perspective view showing a configuration of the endoscope distal end portion of the endoscope insertion portion of FIG. 1 with a normal optical adapter attached, FIG. 9 is a cross-sectional view taken along line AA of FIG. 8, and FIG. FIG. 11 is a diagram showing an example of a thumbnail screen displayed, and FIG. 12 is a stereo image. FIG. 13 is a diagram showing a display example of a measurement screen by a normal optical adapter, FIG. 14 is a first flowchart showing an example of a control operation characteristic of this measurement software, and FIG. These are the 2nd flowcharts which show the example of control operation which becomes the feature of this measurement software.
[0018]
(Constitution)
The system configuration of the measurement endoscope apparatus 10 according to the present embodiment will be described. As shown in FIG. 1, the measurement endoscope apparatus 10 incorporates an imaging element and a plurality of stereo measurement optical adapters and a normal measurement use. An endoscope insertion unit 11 configured to be detachably mountable with an optical adapter, a control unit 12 that houses the endoscope insertion unit 11, and various operation controls of the entire system of the measurement endoscope apparatus 10 are executed. A remote controller 13 for performing necessary operations, a liquid crystal monitor (hereinafter referred to as LCD) 14 for displaying an endoscopic image or operation control content (for example, a processing menu), and a normal endoscopic image, or A face mount display (hereinafter referred to as FMD) 17 capable of stereoscopically viewing the endoscopic image as a pseudo stereo image, and an FMD display for supplying image data to the FMD 17. It is configured to include a descriptor 18.
[0019]
Further, the system configuration of the apparatus will be described in detail with reference to FIG. As shown in FIG. 2, the endoscope insertion portion 11 is connected to an endoscope unit 24, and this endoscope unit 24 is mounted in the control unit 12, for example, as shown in FIG. Although not shown, the endoscope unit 24 includes a light source device for obtaining illumination light necessary for imaging, and an electric bending device for bending the endoscope insertion portion 11 electrically freely. It is configured.
[0020]
An imaging signal from the solid-state imaging device 43 (see FIG. 5) at the distal end of the endoscope insertion portion is input to a camera control unit (hereinafter referred to as CCU) 25. The CCU 25 converts the supplied imaging signal into a video signal such as an NTSC signal and supplies it to a main processing circuit group in the control unit 12.
[0021]
The main circuit group mounted in the control unit 12 includes, for example, as shown in FIG. 2, a CPU 26, a ROM 27, a RAM 28, a PC card interface (hereinafter, referred to as a CPU card) that controls various functions based on the main program. PC card I / F) 30, USB interface (hereinafter referred to as USB I / F) 31, RS-232C interface (hereinafter referred to as RS-232CI / F) 29, audio signal processing circuit 32 and video signal processing circuit 33.
[0022]
The RS-232CI / F 29 is connected to the CCU 25, the endoscope unit 24, and the remote controller 13, respectively, and controls the operation of the CCU 25, the endoscope unit 24 and the operation based on the operation by the remote controller 13 that gives an operation instruction. It is for performing communication necessary for each.
[0023]
The USB I / F 21 is an interface for electrically connecting the control unit 12 and the personal computer 21. When the USB I / F 21 is connected via the USB I / F 21, the personal computer 21 side also includes an internal unit in the control unit 12. It is possible to perform various instruction control such as an instruction to display an endoscopic image and image processing at the time of measurement, and input of control information and data necessary for various processes between the control unit 12 and the personal computer 21. Output can be performed.
[0024]
The PC card I / F 30 is configured such that a PCMCIA memory card 22 and a compact flash (R) memory card 23 are detachably connected. That is, when any one of the above memory cards is inserted, the control processing by the CPU 26 reproduces data such as control processing information and image information stored in the memory card as a recording medium, and the PC card I / F 30 Or data such as control processing information or image information can be supplied to the memory card via the PC card I / F 30 and recorded.
[0025]
The video signal processing circuit 33 displays the video signal from the CCU 25 and the operation menu generated by the control of the CPU 26 so as to display a composite image obtained by synthesizing the endoscopic image supplied from the CCU 25 and the graphic operation menu. By combining the display signal based on the display signal and performing processing necessary for display on the screen of the LCD 14 and supplying it to the LCD 14, a composite image of the endoscopic image and the operation menu is displayed on the LCD 14. In the video signal processing circuit 33, it is also possible to simply perform processing for displaying an endoscopic image or an image such as an operation menu alone.
[0026]
The audio signal processing circuit 32 is supplied with an audio signal that is generated by being collected by the microphone 20 and recorded on a recording medium such as a memory card, or an audio signal obtained by reproducing the recording medium such as a memory card. Processing (amplification processing or the like) necessary for reproducing the audio signal is performed and output to the speaker 19. Thereby, an audio signal is reproduced by the speaker 19.
[0027]
The CPU 26 executes a program stored in the ROM 27 and controls various circuit units so as to perform processing according to the purpose, thereby controlling the operation of the entire system.
[0028]
Next, the configuration of the remote controller 13 and an example of the program operation control of the CPU 26 based on the operation will be described with reference to FIG.
[0029]
The remote controller 13 used in the measurement endoscope apparatus 10 of the present embodiment has been improved to further improve operability during use such as measurement.
[0030]
As shown in FIG. 3, the remote controller 13 includes a joystick 47, a lever switch 48, a freeze switch 49, a store switch 50, and a measurement execution switch 51 on at least the upper surface. In other words, the remote controller 13 is easy to operate for the user. Is adopted.
In the remote controller 13 having the above-described configuration, the joystick 47 is a switch that performs a bending operation of the endoscope distal end portion, and can give an operation instruction freely in any direction of 360 degrees. The lever switch 48 is a switch for performing a pointer operation when performing various menu operations for graphic display and measurement, and is configured in substantially the same shape as the joystick switch 47. The freeze switch 49 is a switch used when displaying the endoscope moving image displayed on the LCD 14 as a still image. The store switch 50 is a switch used to record the still image on the PCMCIA memory card 22 (see FIG. 2) when the still image is displayed by pressing the freeze switch 49. The measurement execution switch 51 is a switch used when executing measurement software.
[0031]
The freeze switch 49, the store switch 50, and the measurement execution switch 51 are configured to employ, for example, an on / off pressing type. Further, functions other than those described above can be assigned to the lever switch 48.
[0032]
For example, these functions may be assigned to the lever switch 48 so that the zoom up function of the image can be executed when the lever switch 48 is tilted to the right and the zoom DOWN function can be executed when the lever is tilted to the left. Also, normally, when measurement is performed with a zoom image, measurement cannot be performed correctly because the magnification of the image changes. In such a case, when the measurement execution switch 51 is pressed, the CPU 26 receives this operation signal, cancels the instantaneous zoom function, and controls to execute the measurement after the image is frozen. As another method, control may be performed so that the image can be measured as it is in consideration of the zoom magnification.
[0033]
Next, the configuration of the stereo measurement adapter used in the measurement endoscope apparatus 10 of the present embodiment will be described with reference to FIGS.
[0034]
4 and 5 show a state in which the stereo measurement adapter 37 is attached to the endoscope front end 39. The stereo measurement adapter 37 is connected to the male screw 54 of the endoscope front end 39 by the female screw 53 of the fixing ring 38. It is fixed by screwing.
[0035]
In addition, a pair of illumination lenses 36, two objective lenses 34, and an objective lens 35 are provided at the tip of the stereo measurement adapter 37. The two objective lenses 34 and 35 form two images on the image sensor 43 provided in the endoscope distal end portion 39. The obtained imaging signal is supplied to the CCU 25 through the electrically connected signal line 43a and the endoscope unit 24, converted into a video signal by the CCU 25, and then supplied to the video signal processing circuit 33. As a result, for example, an image as shown in FIG. 6 is displayed on the LCD 14.
[0036]
The measurement endoscope apparatus 10 according to the present embodiment, when performing stereo measurement, uses, for example, a recording in which optical data of the stereo measurement adapter 37 is recorded using an endoscope image obtained by imaging a white subject as illustrated in FIG. Stereo measurement processing of the object to be measured is executed based on optical data taken from a medium (for example, a compact flash (R) memory card).
[0037]
Stereo measurement by the measurement endoscope apparatus 10 includes first processing for reading optical information from a recording medium (for example, a compact flash (R) memory card) on which optical data of the stereo measurement adapter 37 is recorded, and the endoscope. A position error is calculated from the second process for reading the position information of the image pickup device 43 and the stereo measurement adapter 37 at the distal end portion 39, and the position information and the position information of the main endoscope and the stereo measurement adapter 37 obtained at the time of production. A fourth process for correcting the optical data from the position error, a fifth process for converting the coordinates of an image measured based on the corrected optical data, and a coordinate-converted image This is performed by executing at least a sixth process for obtaining an arbitrary point three-dimensional coordinate by matching two images based on the above.
[0038]
For example, the CPU 26 executes the first to fourth processes once for the stereo measurement adapter 37 and controls the result to be recorded on the compact flash (R) memory card 23 as measurement environment data. Thereafter, when performing stereo measurement, the CPU 26 controls to load the measurement environment data on the RAM and execute the fifth, sixth, and seventh processes.
[0039]
When performing the second process of reading the positional information between the imaging device 43 and the stereo optical adapter 37 at the endoscope tip 39, the shape of the mask provided on the optical adapter (not shown) is captured as shown in FIG. This is done by comparing the shape and position of the mask during production. In this case, the capture of the mask shape is performed by capturing a white image (projecting white paper or the like). The brightness of the white image at this time is determined by the gain of the CCU 25 and the shutter speed.
[0040]
Normally, the gain of the CCU 25 and the shutter speed of the image sensor 43 are controlled so as to automatically become optimum conditions. However, when capturing the mask shape, the gain of the CCU 25 is low and the shutter speed of the image sensor 43 is high. It tends to be set, and the image becomes dark and the mask shape cannot be taken clearly, which adversely affects the measurement accuracy. Therefore, in the present embodiment, the control is performed by the CPU 26 so that the gain of the CCU 25 and the shutter speed are fixed. Thereby, the shape of the mask can be captured reliably, and the measurement accuracy does not deteriorate.
[0041]
When preparation for measurement is completed, for example, when measuring the length of the crack 44 in FIG. 6, the measurement point is designated so that the crack 44 is traced by a broken line on the left image. Each time a new measurement point is designated, the CPU 26 searches for the corresponding point on the right image, obtains a three-dimensional coordinate at each point from the coordinate of the measurement point and the corresponding point, and is adjacent to the three-dimensional coordinate. The distance between two matching points is calculated, and the total of them is calculated to display the length of the crack 44 on the LCD 14 as the total length.
[0042]
Next, the configuration of the normal optical adapter used in the measurement endoscope apparatus 10 of the present embodiment will be described with reference to FIGS.
[0043]
8 and 9 show a state in which the normal optical adapter 42 is attached to the endoscope distal end portion 39. The normal optical adapter 42 is connected to the male screw 54 of the endoscope distal end portion 39 by the female screw 53 of the fixing ring 38. It is fixed by screwing.
[0044]
In addition, a pair of illumination lens 41 and objective lens 40 are provided at the tip of the normal optical adapter 42. The objective lens 40 forms an image on the image sensor 43 provided in the endoscope front end 39. The obtained image pickup signal is supplied to the CCU 25 through the signal line 43a and the endoscope unit 24 that are electrically connected in the same manner as the stereo measurement adapter 37, and is converted into a video signal by the CCU 25 before the video. By being supplied to the signal processing circuit 33, for example, an image as shown in FIG.
[0045]
The measurement endoscope apparatus 10 according to the present embodiment normally performs a measurement using an optical adapter by using a method based on comparative measurement. In other words, the comparative measurement is a method of measuring with reference to the understood dimensions in the screen.
[0046]
For example, when the diameter of the circle shown in FIG. 10 is known, a pointer is placed at both ends of the diameter of the circle and a length L1 45 between the two points is input. The dimension L2 46 to be obtained is obtained as a ratio by the arithmetic processing by the CPU 26 from the size on the screen of L1. At this time, the distortion correction is performed based on the information of the distortion characteristic of the lens, and adjustment is performed so as to obtain the dimension more accurately. The distortion characteristics of the lens are recorded in advance on the ROM 27, and the CPU 26 performs comparative measurement by loading data corresponding to the selected normal optical adapter 42 onto the RAM 2.
[0047]
Further, in this measurement endoscope, in order to save the settings of a plurality of stereo measurement adapters, the measurement environment data is controlled to be recorded on a removable removable flash memory card 23.
[0048]
The measurement environment data includes measurement environment information, calibration data, a conversion table, and an inverse conversion table.
[0049]
Since the measurement environment information is information indicating the type of measurement environment data, that is, the type of calibration data or conversion table, for example, the following information is included.
[0050]
・ Stereo measurement adapter type and individual identification number
・ Type of endoscope insertion part and individual identification number
・ TV type such as NTSC
・ CCU type
・ Video capture circuit types
Since the type of CCU and the type of video capture circuit are often provided as a module of the measurement endoscope apparatus, in this case, these may be read as the type of the measurement endoscope apparatus.
[0051]
The calibration data consists of optical data indicating the optical characteristics measured in the production process of the stereo measurement adapter, corrected to information suitable for the measurement endoscope actually used in the measurement. Contains content.
[0052]
・ Position information for the combination of the endoscope bell used in the production process data measurement and this optical adapter ・ Position information for the combination of the endoscope actually used for measurement and this optical adapter
A correction formula in which displacement correction is performed from the position information of the above two mounting conditions with respect to the geometric distortion correction formulas of the two optical systems obtained by data measurement in the production process.
The position coordinates of the optical axis in which the positional deviation correction has been performed based on the position information of the two mounting conditions described above with respect to the position coordinates of the optical axes of the two lenses obtained by the data measurement in the production process.
・ Distance between the optical axes of two lenses determined by data measurement in the production process
・ Focal distance of two lens systems obtained by data measurement of production process
The conversion table included in the measurement environment data is a conversion table for performing geometric distortion correction on a measurement image photographed by the tree endoscope apparatus using the calibration data, and this table is used. And a corrected image after distortion correction can be created.
[0053]
The inverse conversion table included in the measurement environment data is an inverse conversion table for obtaining coordinates on the original image before correction with respect to the coordinates on the corrected image.
[0054]
Further, in this measurement endoscope, the captured image is recorded on the PCMCIA memory card 22 together with calibration data and measurement environment information used for measurement under the control of the CPU 26.
[0055]
Further, in this measurement endoscope, the PCMCIA memory card 22 for recording an image of calibration data and measurement environment information included in one selected from a plurality of measurement environment data registered in the apparatus, It is also possible to record on a removable memory card that is separate from the Compact Flash (R) memory card for recording the measurement environment data.
[0056]
In the present embodiment, a measurement endoscope recorded on a PCMCIA memory card 22 by a measurement endoscope apparatus, capturing a measurement image with calibration data and measurement environment information on a personal computer, and taking the image is measured. The present invention relates to software that can perform the same measurement on a personal computer by setting the same measurement environment as the apparatus, and this embodiment will be described below.
[0057]
The software for the personal computer according to the present embodiment selects the first process for capturing and managing the image recorded by the measurement endoscope apparatus 10 into the personal computer, and selecting one of the managed images. The second process for calling the measurement function, the third process for managing the calibration data recorded on the recording medium and the measurement environment information in the personal computer, and the fourth process for associating the calibration data suitable for the measurement image And a fifth process for creating a conversion table for correcting geometric distortion included in the image, and a sixth process for creating a corrected image by performing coordinate conversion on the image taken for measurement using the conversion table Seventh processing for matching the left and right two images at an arbitrary point based on the corrected image, and obtaining the three-dimensional coordinates from the two left and right coordinates obtained by the matching, the position coordinates of the optical axis, and the focal length And processing the wafer, which is constituted by a ninth process of obtaining a measurement of distance and area between two points in the three-dimensional coordinates of a plurality of arbitrary points, the measurement is performed by executing these.
[0058]
When a memory card with recorded measurement images is attached to a personal computer and images are selected by this software, these images are stored in folders on the personal computer managed by this software by the first process. Is duplicated. When this process ends, the thumbnail 55 of the image copied earlier is displayed in the thumbnail screen of the software as shown in FIG. If a symbol indicating a stereo measurement image or a comparative measurement image is recorded in the header of the copied image file, a stereo measurement icon 56 and a comparative measurement icon 57 are displayed on each thumbnail 55 accordingly. Is displayed.
[0059]
Then, an image to be measured is selected from the thumbnail screen, a measurement execution menu (not shown) is selected from the menu bar 58, the measurement execution button 60 is pressed from the toolbar 59, or the thumbnail is displayed. When the stereo measurement icon 56 or the comparative measurement icon 57 is pressed, each measurement function is called by the second process, and when preparation for measurement is completed, a stereo measurement screen as shown in FIG. 12 or as shown in FIG. The comparative measurement screen is displayed.
[0060]
FIG. 12 shows various menu buttons 61 for executing various measurement functions, a message display field 63 for displaying messages concerning operations, various numerical value display fields 62 for displaying measurement results (numerical values), and measurement. An example is shown in which two measurable visual field areas 64 are cut out from the image and pasted and displayed on this measurement screen.
[0061]
FIG. 13 shows an example in which various menu buttons 65, a message display field 67, various numerical value display fields 66, and a measurement image 68 are displayed as in FIG.
[0062]
The measurement endoscope apparatus registers a plurality of measurement environment data, and the total capacity of the conversion table and the inverse conversion table included in each measurement environment data is, for example, 12 Mbytes. Therefore, if three stereo measurement adapters are registered in this apparatus, 12 Mbytes × 3 = 36 Mbytes. In measurement endoscope devices, it takes time to create measurement environment data due to restrictions such as the CPU, and various measurements are often performed using several types of optical adapters. Several pieces of measurement environment data that can be recorded within the capacity range of a flash ROM or a compact flash (R) memory card that has been created have been created in advance.
[0063]
On the other hand, the processing speed of a commercially available personal computer is several times to several tens of times faster than a measurement endoscope apparatus. Thus, when measurement environment data is created on a personal computer, it is completed in several seconds to several tens of seconds. In addition, because of the nature of managing a large number of images on a PC, it is necessary to handle a large number of types of measurement environment data. Therefore, every time you select an image and perform measurement, you can create measurement environment data from the calibration data attached to the image, so you only have to secure the capacity of one measurement environment data. Get better.
[0064]
In addition, the calibration data required for creating measurement environment data is often very small in size, such as several tens of Kbytes, making it easy to save multiple calibration data in a personal computer. It becomes possible to register. Therefore, even if calibration data is not attached to an image, there is a convenience that measurement environment data can be created and remeasured by selecting an appropriate one from the calibration data list.
[0065]
Here, a plurality of measurement environment data and calibration data registered in the personal computer can be easily registered by the following two methods.
[0066]
The first method is to select a measurement image with measurement environment information and calibration data, and execute the measurement with the same measurement environment as the selected image in the already registered measurement environment data. If the same file is not registered, the measurement environment information and calibration data attached to the image are automatically copied into the folder managed by this software. sign up.
[0067]
In the second method, the measurement endoscope apparatus selects an appropriate measurement environment data registered here, and records the measurement environment information and calibration data on a removable recording medium. Prepare in advance. Alternatively, an appropriate one selected from the measurement environments registered by this software is prepared, and the measurement environment information and calibration data recorded on a removable recording medium are prepared. Mount the recording medium prepared in this way on a personal computer, select the appropriate measurement environment information and calibration data recorded here with this software, copy it to the folder managed by this software, and copy it. Register.
[0068]
As described above, if the measurement environment is supported by this software, you can easily set the measurement environment for any measurement image recorded by various measurement endoscope devices, and you can perform remeasurement quickly. .
[0069]
(Function)
With reference to FIG. 14 and FIG. 15, a control operation that is a feature of the software of the present embodiment will be described in detail.
[0070]
A measurement image recorded by the measurement endoscope apparatus is captured by this software, and one appropriate thumbnail is selected from the screen of FIG. 11 by a mouse or key operation. Then, when the measurement execution menu in the menu bar 58 is selected, the measurement execution button 60 is pressed from the toolbar 59, or the stereo measurement icon 56 or the comparative measurement icon 57 on the thumbnail is pressed, as shown in FIG. The routine program is executed.
[0071]
When the routine shown in FIG. 14 is executed, in the determination process of step S111, it is determined whether or not the image is recorded by the measurement endoscope apparatus with reference to the header of the selected image file. If it is an image recorded in step S112, it is determined in step S112 whether or not the image is a stereo measurement image by referring to the header of the image.
[0072]
Here, in the case of a stereo measurement image, it is determined whether or not measurement environment information is attached to the header of this image file in the determination process of step S113. If this is recorded, the determination of step S114 is performed. It is determined whether or not the calibration data attached to the image is supported by this software in the process, and if it is supported, the process proceeds to step S116. If the calibration data attached to the image in the determination process of step S114 is not supported by this software, it is displayed on the screen of FIG. 11 to display that stereo measurement cannot be performed and to select another image. Return.
[0073]
In the determination process in step S116, it is determined whether or not the measurement environment data recorded in the measurement environment database already registered in the personal computer has the same measurement environment information as this image. If there is the same item, the process proceeds to step S118. If it is determined in step S116 that there is no same item, the measurement environment information and the calibration data attached to the image are registered in the database, and the process proceeds to step S118.
[0074]
In the determination process in step S118, it is determined whether or not the measurement environment data is the same as the measurement environment data that has already been created and set. If the measurement environment information is the same, in the determination process in step S119, It is determined whether or not the difference between the position information of the visual field area included in and the position information of the visual field area included in the current measurement environment data is smaller than a predetermined threshold value.
[0075]
If it is smaller than the threshold value in step S119, the currently set measurement environment data is read, and the stereo measurement process in step S121 is performed.
[0076]
If the determination process in step S118 is different from the current measurement environment data, or if the determination process in step S119 is greater than the threshold value, the measurement environment data is obtained using the calibration data attached to the header of the image file. Is newly created, and the stereo measurement process of step S121 is performed.
[0077]
When preparation for stereo measurement is completed in step S121, the screen of FIG. 12 enters a standby state. When measurement is performed on this screen, the measurement result is displayed.
[0078]
When this screen is closed, the screen returns to the screen of FIG. 11 and another image can be selected.
[0079]
On the other hand, when the measurement environment information and the calibration data are not attached to the measurement image in the determination process in step S113, the process proceeds to S123 in order to manually select the measurement environment data suitable for this image so that the measurement can be performed. The routine program shown in FIG. 15 is executed.
[0080]
In step S201, the user selects whether to select appropriate data from already registered calibration data, use calibration data recorded on another recording medium, or end the measurement.
[0081]
When selecting an appropriate calibration data from already registered calibration data, a list of calibration data registered in the measurement environment database is displayed in step S203, and an appropriate one is displayed to the user in step S204. Let them choose one. Next, in step S205, it is determined whether or not the measurement environment information is attached to the image. If so, the content of the measurement environment information recorded in the image file in the determination process in step S206 and the user selected It is determined whether or not the content of the measurement environment information in the calibration data matches. If they match in step S206, the process proceeds to step S209.
[0082]
If they do not match in step S206, it is displayed that the selected calibration data cannot be used, and the flow returns to step S201 to select another calibration data.
[0083]
On the other hand, if the measurement environment information is not attached to the image in step S205, the user reconfirms the content of the measurement environment information in the selected calibration data, and the process proceeds to step S209.
[0084]
In step S209, measurement environment data is created using the calibration data selected from the list, and stereo measurement processing is executed.
[0085]
If it is determined in step S201 that the calibration data recorded in another recording medium, not from the database, is used, the recording medium in which the calibration data is recorded is inserted into the personal computer, and is stored in step S212. Display a list of calibration data.
[0086]
Then, after the user selects an appropriate one from these, it is determined whether or not the calibration data selected in the determination process in step S214 corresponds to this software. If it is determined in step S214, it is determined whether or not measurement environment information is attached to the image in step S215. If so, the content of the measurement environment information recorded in the image file in the determination process in step S217. And whether or not the content of the measurement environment information of the calibration data selected by the user matches. If they match in step S217, the process proceeds to step S219.
[0087]
If the measurement environment information is not attached to the image in step S215, the user reconfirms the content of the measurement environment information in the selected calibration data, and the process proceeds to step S219.
[0088]
Then, the calibration data and measurement environment information selected in step S219 are registered in the database, and in step S220, the measurement environment data is created using the calibration data, and the stereo measurement process is executed.
[0089]
On the other hand, when the calibration data selected in step S214 is not supported by this software, or when the content of the measurement environment information in the image does not match the content of the measurement environment information of the selected calibration data in step S217. Displays that the selected calibration data cannot be used, and returns to step S201 to select another calibration data.
[0090]
When step S209 or step S220 is completed, measurement environment information and calibration data may be recorded in the header of the image file as desired by the user before performing stereo measurement.
[0091]
Furthermore, if the image selected in step S112 in FIG. 14 is not a stereo measurement image, it is determined in the determination process in step S124 whether the image is a comparative measurement by referring to the header of the image, and is a comparative measurement image. In this case, the distortion correction information of the optical adapter attached to the image is read in step S125, and comparison measurement processing is executed.
[0092]
On the other hand, if it is determined in step S124 that the selected image is neither a stereo measurement image nor a comparative measurement image, the measurement process is ended and the screen shown in FIG. 11 is returned to select another image, or step S127 is selected. Let the user select a measurement method.
[0093]
If the measurement method selected in step S127 is comparative measurement, the initial value distortion correction information is read and comparative measurement processing is executed.
[0094]
If stereo measurement is selected in step S127, the process proceeds to step S123, and the program shown in FIG. 15 is executed to select appropriate calibration data from the list so that measurement can be performed.
[0095]
Note that if the image selected in step S111 is not an image recorded by the measurement endoscope apparatus, or if the measurement is terminated in step S128, the screen returns to the screen in FIG. 11 so that another image can be selected. To.
[0096]
(Effect 1)
In this embodiment, just by selecting a measurement image and executing measurement, the correct information necessary for measurement is read and the measurement environment is set, and it is correct on a general personal computer, not on a measurement endoscope device. Measurement is possible.
[0097]
Moreover, an appropriate measurement program is executed simply by selecting a measurement image and pressing a measurement execution menu or a measurement button regardless of the type of measurement.
[0098]
(Effect 2)
When performing measurement using images with measurement environment information and calibration data, the current setting is used when the measurement environment information indicating the type of device used to capture the image matches the current setting. If it is used for measurement, the accuracy of the self-image taken to inspect the position of the field of view changes for some reason, and the positional relationship information also changes accordingly. May decrease.
[0099]
In the present embodiment, even if the measurement environment information indicating the type of device used to capture the image matches the current setting, it is confirmed whether the position information of the visual field area has changed. Therefore, not only reduces the case where the accuracy is reduced due to such a cause, but also allows the measurement environment to be automatically set and measured using calibration data attached to the image beforehand. It is safe, quick, and ready to make measurements easily.
[0100]
(Effect 3)
Since the calibration data has a capacity of several tens of Kbytes and the measurement environment information is often several Kbytes, it may be desired to record only the measurement environment information in the image in order to reduce one image file.
[0101]
Also, if measurement environment information or calibration data is recorded in a file other than the image file, the measurement environment information or calibration data file recorded in association with the image for some reason is lost or deleted. It is also possible to be trapped.
[0102]
In the case of a conventional measurement endoscope system, such an image cannot be remeasured even on the apparatus, or the measurement environment setting operation until actual remeasurement is complicated.
[0103]
On the other hand, in this embodiment, the measurement environment can be easily set on a personal computer simply by selecting from a list of already registered measurement environment data or a list of calibration data recorded on another memory card. Settings can be made and remeasurement can be performed quickly.
[0104]
Furthermore, since it is confirmed whether or not the data selected from the list is compatible with the image to be measured, there is an effect of reducing the case where measurement is performed in an inconsistent state.
[0105]
In the present invention according to the above embodiment, the measurement environment can be set on the personal computer and re-measurement can be performed on the image recorded on the removable recording medium by the measurement endoscope apparatus. Therefore, it is possible to prevent the measurement from being performed on an image that is not a measurement image. The user can perform measurement corresponding to the correct optical adapter with a simple operation without worrying about the type of equipment of the optical adapter, improving operability at the time of re-measurement and improving inspection efficiency. It becomes possible to improve.
[0106]
Although it supports various measurement environments, measurement environment data is recreated as needed, so only one set of measurement environment data capacity can be secured on the recording medium (HD) in the PC. Therefore, it is not necessary to specifically limit the specifications of the personal computer HD for this purpose.
[0107]
In addition, when performing measurement with a measurement image with the first type of calibration data, the calibration data can be automatically loaded into the managed calibration data group. Re-measurement can be performed without much awareness of existence.
[0108]
In addition to the measurement image, only the calibration data can be recorded on the recording medium, or it can be imported into a personal computer, so it is possible to remeasure even an image without calibration data. .
[0109]
In addition, there is no need to perform re-measurement on the device that actually captured the image, so re-measurement can be performed on another device by attaching the recording medium on which the image is recorded to another device and performing the same simple operation. Can be performed.
[0110]
As a result, the state of the measurement environment in various measurement endoscope devices can be reproduced on a single personal computer, so the measurement image can be easily re-measured on the personal computer familiar to the inspector. Furthermore, by using a computer network system, it becomes possible to easily share data of images and measurement results among a plurality of inspectors, and the inspection environment is greatly improved.
[0111]
In other words, according to the present invention, a measurement image is taken into a personal computer together with measurement environment information and calibration data, an appropriate image is selected from the plurality of stored images, and a stereo measurement function is executed. The measurement environment is automatically set from the measurement environment information and calibration data attached to the, and the measurement process can be immediately executed on the personal computer. At this time, the measurement environment information and the calibration data attached to the image are automatically duplicated in the calibration data group database and managed separately from the image.
[0112]
In addition, a process of searching for the same calibration data group already managed in the personal computer by referring to the measurement environment information recorded in the measurement image, and automatically using the search result data. If measurement is performed with an image that does not have calibration data due to the capacity of the medium on which the image is recorded, etc., by providing a means to change the setting of the measurement environment, the same measurement information as the selected measurement image is used. If certain measurement environment data exists in the apparatus, measurement environment data can be automatically created from already existing calibration data, and measurement processing can be executed by using this.
[0113]
Furthermore, by providing a means to determine whether it matches the current measurement environment setting by referring to the measurement environment information recorded in the measurement image, the current setting can be made without creating measurement environment data again. Measurement processing can be executed using the measured measurement environment data, and the time for preparation for measurement can be shortened.
[0114]
Furthermore, even if measurement environment information and calibration data are not recorded in the measurement image for some reason, a measure is provided so that the measurer can check whether the measurement environment of the selected image is recorded. Measurement can be performed without consciousness. In this case, a list of calibration data already existing in the personal computer can be displayed, and measurement can be performed by selecting an appropriate one from the list. It becomes.
[0115]
In addition to exporting the image to the recording medium and importing from the recording medium, the calibration data in the personal computer can be copied to the recording medium, or the calibration data in the recording medium can be imported to the database in the personal computer. Therefore, it is possible to construct a database that can manage only the calibration data independently. As a result, even when re-measurement is performed on an image without calibration data and measurement environment data, re-measurement can be easily performed on such an image if appropriate calibration data is loaded into the database via a recording medium. It becomes possible.
[0116]
【The invention's effect】
As described above, according to the present invention, the measurement image recorded by the measurement endoscope apparatus can be re-measured on the personal computer, and the work efficiency of the inspection can be improved.
[Brief description of the drawings]
FIG. 1 is a perspective view showing a system configuration of a measurement endoscope apparatus according to a first embodiment of the present invention.
2 is a block diagram showing an electrical circuit configuration of the measurement endoscope apparatus of FIG. 1. FIG.
3 is a perspective view showing the configuration of the remote controller of FIG. 1. FIG.
4 is a perspective view showing a configuration of an endoscope distal end portion of the endoscope insertion portion of FIG. 1 with a stereo measurement adapter attached. FIG.
5 is a cross-sectional view taken along line AA in FIG.
6 is a view showing an endoscopic image with the stereo measurement adapter of FIG. 4 attached. FIG.
7 is a diagram showing an image of a mask shape of the stereo measurement adapter of FIG. 4;
8 is a perspective view showing a configuration of an endoscope distal end portion of the endoscope insertion portion of FIG. 1 with a normal optical adapter attached thereto.
9 is a cross-sectional view taken along line AA in FIG.
10 is a view showing an endoscopic image with the normal optical adapter of FIG. 9 attached. FIG.
FIG. 11 is a diagram showing a display example of a thumbnail screen
FIG. 12 is a diagram showing a display example of a measurement screen by a stereo optical adapter
FIG. 13 is a diagram showing a display example of a measurement screen by a normal optical adapter
FIG. 14 is a first flowchart showing a control operation example that is a feature of the measurement software;
FIG. 15 is a second flowchart showing a control operation example that is a feature of the measurement software;
[Explanation of symbols]
10. Measurement endoscope device
11 ... Endoscope insertion part
12 ... Control unit
13 ... Remote controller
14 ... Liquid crystal monitor (LCD)
17 ... Face cloak display (FMD)
18 ... FMD adapter
19 ... Speaker
20 ... Microphone
21 ... Personal computer
22 ... PCMCIA memory card
23 ... Compact Flash (R) memory card
24 ... Endoscope unit
25 ... Camera Control Unit (CCU)
26 ... CPU (control unit)
27 ... ROM
28 ... RAM
29 ... RS-232CI / F
30 ... PC card I / F
31 ... USB I / F
32. Audio signal processing circuit
33 ... Video signal processing circuit
34, 35, 40 ... Objective lens system
36, 41 ... Illumination lens
37 ... Stereo optical adapter
38 ... Fixing ring
39 ... End of endoscope
42 ... Normal optical adapter
43 ... Image sensor
44 ... Crack
45 ... L1 (known dimensions)
46 ... L2 (unknown dimension)
47 ... Joystick
48 ... Lever switch
49 ... Freeze switch
50 ... Store switch
51 ... Measurement execution switch
53 ... Female thread
54 ... Male thread
55 ... Thumbnail
56 ... Stereo measurement icon
57 ... Comparison measurement icon
58 ... Menu bar
59 ... Toolbar
60 ... Measurement execution button
61, 65 ... various menu buttons
62, 66 ... Various numerical value display fields
63, 67 ... Message display field
64 ... Measurable field of view
68 ... Measurement image

Claims (4)

Stereo image data including two images having parallax imaged through a stereo measurement adapter, calibration data indicating the optical characteristics of the stereo measurement adapter, and measurement environment information indicating the type of the calibration data An image acquisition step capable of acquiring an image file having ;
A determination step for determining whether or not calibration data is added to the image file captured by the image acquisition step;
When the determination step determines that the calibration data is added to the image file captured by the image acquisition step, the determination step is added to the image file that is determined to have the correction data added. Reading calibration data from the image file captured by the image acquisition step;
When the determination step determines that calibration data is not added to the image file captured by the image acquisition step, the calibration data corresponding to the measurement environment information is at least 1 with reference to the measurement environment information. Determining whether or not two arbitrary calibration data are recorded on a recording medium, and
When it is determined that calibration data corresponding to the measurement environment information is recorded on the recording medium, reading calibration data corresponding to the measurement environment information from the recording medium;
Correcting the geometric distortion included in the stereo image data included in the image file captured by the image acquisition step based on the read calibration data, and generating corrected stereo image data;
Performing a matching process between the two images included in the corrected stereo image data;
Calculating three-dimensional coordinates at an arbitrary point of the corrected stereo image data based on the result of the matching process and the read calibration data;
And a step of measuring a subject using the three-dimensional coordinates. The step of generating the corrected stereo image data includes:
Creating a conversion table used for coordinate conversion in the stereo image data from the read calibration data;
The measurement program according to claim 1, further comprising: converting the stereo image data based on the conversion table so as to remove geometric distortion from the stereo image data. The calibration data added to the image file that has been determined that the calibration data has been added in the determination step has the optical characteristics measured in the production process of the stereo measurement adapter within the connection of the stereo measurement adapter. 3. The measuring program according to claim 1, wherein the measuring program is corrected as an optical characteristic suitable for an endoscope insertion portion. Determining the type of measurement process executable on the stereo image data;
The measurement program according to claim 1, further comprising a step of executing a measurement process corresponding to the determined type on the corrected stereo image data.

Images