WO2013175816A1 - Distance measurement apparatus - Google Patents

Abstract

A distance measurement apparatus is provided with: an imaging optical system (10) for forming an object image; a pupil colour separation filter (14) for subjecting a pupil of the imaging optical system (10) to colour separation; a colour imaging element (11) which subjects the formed object image to photoelectric conversion and outputs an image; a pixel-interpolation calculation unit (35) for generating interpolation pixels of pixels arranged in a plurality of colour images obtained by separating the colours of the image outputted from the imaging element (11); a pixel deviation detector (33) which synthesizes the colour images obtained by colour separation and the interpolation pixels generated by the pixel-interpolation calculation unit (35) to generate a plurality of interpolated colour images, and detects the relative deviation amount of the object image in the plurality of interpolated colour images; and a distance calculation unit (34) for generating information related to object distance on the basis of the detected deviation amount.

Description

Distance measuring device

The present invention relates to a distance measuring device for acquiring distance information of an object.

For example, Japanese Patent Application Laid-Open No. 2001-174696 describes a color imaging apparatus that performs pupil division by color by interposing a pupil color division filter having different spectral characteristics for each partial pupil in the photographing optical system. . A subject image formed on a color imaging device through the pupil color division filter is captured by the color imaging device and output as an image signal. This image signal is color separated, and distance information is generated by detecting a relative shift amount between pupil color divided color signals. This distance information may be the distance to the subject itself, but may be both focusing information of the focusing deviation direction and the focusing deviation amount, for example, when aiming at autofocus (AF).

A distance measuring device capable of acquiring such distance information can be used for various devices and devices such as a microscope and a digital camera. For example, when used for a microscope, the height measurement of a subject (also referred to as a sample) Will be able to

Specifically, assuming that the colors to be pupil-color-divided are red (R) and blue (B) (see FIGS. 3 to 7 according to the present invention, etc.) By measuring at points, it is possible to determine the relative height distance between the measured points. Furthermore, it is also possible to obtain an absolute distance based on the in-focus position.

For example, it is assumed that the height of the subject 6 as shown in FIG. 8 according to the present invention is measured by a microscope using a pupil color division filter as shown in FIG. 3 according to the present invention. Here, the upper surface of the subject 6 is as shown in FIG. 9 according to the present invention.

When the imaging optical system of the microscope is focused on the surface of the white plate 6a shown in FIG. 8, R in the optical object image 6i of the white printed matter “1” on the black box-like object 6b formed on the imaging device. The subject image IMGr of the component and the subject image IMGb of the B component become a double image as shown in FIG. 10 according to the present invention. Accordingly, the image of the R signal and the image of the B signal in the color image obtained by imaging with the imaging device are the same as in FIG.

A case will be considered in which the shift amount of this RB double image is measured in the line A portion of FIG. When a Bayer array sensor as shown in FIG. 2 according to the present invention is used as an imaging element of a microscope, an RB double image at the line A part is as shown in FIG. Here, FIG. 21 is a view showing an example of the RB double image formed on the Bayer array sensor.

When the pixel value of the R pixel and the pixel value of the B pixel are respectively acquired from the basic array of 2 × 2 pixels arranged from 1 to 6 in the horizontal direction shown in FIG. 21, for example, the result is as shown in FIG. FIG. 22 is a diagram showing the pixel value of the R pixel and the pixel value of the B pixel acquired from the RB double image of FIG. 21. Here, a black circle connected by a solid line indicates a pixel value of the R pixel, and a black square connected by a dotted line indicates a pixel value of the B pixel.

For example, the data shown in FIG. 22 is subjected to, for example, ZNCC (Zero-mean Normalized Cross-Correlation) (see Equation 1 according to the embodiment of the present invention) to obtain correlation values. Calculate the pixel shift amount from.

Since the amount of displacement calculated from this ZNCC is in pixel coordinate units, interpolation of sub-pixels (for example, subpixel estimation methods such as equiangular straight line fitting and parabola fitting) are performed to calculate the amount of displacement with higher accuracy. . The shift amount calculated in this way can be proportionally converted based on the configuration of the imaging optical system and the pupil color division filter, so the distance in the height direction from the in-focus position can be obtained.

However, since the horizontal positions (and vertical positions) of the R and B pixels are discrete on the Bayer array, the waveform shape of R and the waveform shape of B are as shown in FIG. The difference between the two can not be detected correctly.

Further, for example, in Japanese Patent Application Laid-Open No. 2008-122835, when calculating the degree of correlation between two data strings, the DC component or AC component is extracted, and the correlation is accurately detected by aligning the signal levels of the extracted components. Describes a correlation calculation method, a correlation calculation device, a focus detection device, and an imaging device.

However, in the case shown in FIG. 22, since the waveform shape of R and the waveform shape of B are originally different, sufficient effects can be obtained even with the technique described in Japanese Patent Application Laid-Open No. 2008-122835. I can not think that I can do it.

Then, when z is the subject height (mm) and x is the displacement amount (pixel) of the RB double image, a proportional conversion equation obtained by the configuration of the imaging optical system and the pupil color division filter is, for example, the present invention In the case of “z = 0.15x” shown in Formula 2 according to the embodiment of the present invention, and the required accuracy in the z direction (distance measurement resolution) is 0.05 mm, the pixel shift amount is 0 It needs to be detected with an accuracy of .33 pixels or less.

As described above, in the case of an imaging element in which pixels of each color are discretely arranged, for example, in a Bayer array, the waveform shape of each color image may differ, and at this time, the detection accuracy of the amount of deviation decreases. It will Therefore, there is a need for a technique for further improving the detection accuracy of the amount of deviation, and hence the distance measurement accuracy.

The present invention has been made in view of the above circumstances, and when acquiring information on a subject distance based on a plurality of color images obtained by imaging pupil color-divided light, distance measurement accuracy is higher. It aims to provide a distance measuring device that can

A distance measurement device according to an aspect of the present invention includes an imaging optical system for forming an image of an object, and an optical path of the imaging optical system, wherein plural partial pupils of the imaging optical system have different spectral characteristics. A pupil color division optical system that divides the pupil of the imaging optical system by color, and photoelectric conversion of an object image formed by the imaging optical system via the pupil color division optical system, and a plurality of pixels are arrayed Pixels of a color imaging element for outputting the selected image, and a plurality of color images obtained by color separation of the image output from the imaging element, which are arranged in a plurality of color images for different partial pupils Generating a plurality of interpolated color images by combining a pixel interpolation unit that generates the interpolation pixels, a plurality of color images related to the different partial pupils, and the interpolation pixels generated by the pixel interpolation unit; Subject to multiple interpolated color images A distance information generation unit that detects relative displacement of the body image and generates information related to the subject distance based on the displacement, and the pixel interpolation unit detects at least the displacement. The interpolation pixel is generated in a certain shift detection direction.

BRIEF DESCRIPTION OF THE DRAWINGS The block diagram which shows the structure of the distance measuring device in Embodiment 1 of this invention. FIG. 6 is a diagram for explaining a pixel array of the imaging device in the first embodiment. FIG. 6 is a view for explaining an example of the arrangement of a pupil color division filter according to the first embodiment; FIG. 7 is a plan view showing the state of subject light flux focusing when an object at a longer distance side than the in-focus position is imaged in the first embodiment. FIG. 7 is a view showing, for each color component, the shape of blur formed by light from one point on the subject that is on the far side of the in-focus position in the first embodiment. FIG. 7 is a plan view showing the state of subject light flux focusing when the subject on the near distance side of the in-focus position is imaged in the first embodiment. FIG. 7 is a view showing, for each color component, the shape of blur formed by light from one point on an object on the near distance side of the in-focus position in the first embodiment. FIG. 2 is a perspective view showing a subject in the first embodiment. FIG. 2 is a view showing the upper surface of a black box-like object in the subject of the first embodiment. FIG. 7 is a view showing a state in which the optical subject image of the R component and the optical subject image of the B component of the white printed matter formed on the image pickup element in the first embodiment are shifted. FIG. 7 is a diagram showing pixel values of R and B images of white printed matter “1” for pixel arrangement on line A, and interpolated pixel values and composite values in the first embodiment. The block diagram which shows the structure of the distance measuring device in Embodiment 2 of this invention. FIG. 16 is a diagram showing pixel values of R and B images of white printed matter “1” and a shifted pixel value and a combined value in the pixel arrangement on the line A in the second embodiment. The block diagram which shows the structure of the distance measuring device in Embodiment 3 of this invention. FIG. 14 is a diagram for explaining optical conditions of the imaging optical system in the third embodiment. The block diagram which shows the structure of the distance measuring device in Embodiment 4 of this invention. FIG. 16 is a view showing a state in which the user designates the measurement position of the subject on the screen of the monitor in the fourth embodiment. The block diagram which shows the structure of the distance measuring device in Embodiment 5 of this invention. FIG. 17 shows a state of a subject image of an R component and a B component in an original image formed on an imaging surface of an imaging element and a partially enlarged view of a line A in the fifth embodiment. In the fifth embodiment, a diagram showing a state of a subject image of an R component and a B component when the shift image formed on the imaging surface of the imaging device is inclined and slightly larger than the original image and line A Partially enlarged view. The figure which shows an example of RB double image conventionally imaged on a Bayer array sensor conventionally. FIG. 23 is a diagram showing pixel values of an R pixel and pixel values of a B pixel acquired from the RB double image in FIG. 21 conventionally.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.

Embodiment 1
FIGS. 1 to 11 show Embodiment 1 of the present invention, and FIG. 1 is a block diagram showing the configuration of the distance measuring device 1.

The distance measurement device 1 generates information on the distance to the subject based on an image obtained by capturing an image of the subject. Examples of application of the distance measuring device 1 include, for example, industrial microscopes, digital cameras, digital video cameras, mobile phones with cameras, PDAs with cameras (PDAs with cameras), personal computers with cameras, surveillance cameras, endoscopes And the like, but of course are not limited thereto.

In addition, below, the distance measurement apparatus 1 applied to the imaging device supposing an industrial microscope is demonstrated.

First, the subject 6 is an object whose distance is to be measured, and is also referred to as a sample, for example, in the field of a microscope.

The distance measuring device 1 measures the distance to the subject 6 and includes, for example, a lens barrel 2, a controller 3, a personal computer (PC) 4, and a monitor 5.

The lens barrel 2 includes an imaging optical system 10, an imaging device 11, a pupil color division filter 14, a ring illumination 16, an imaging device control unit 21, a zoom control unit 22, a focus control unit 23, and a focus driving mechanism. And 24.

The imaging optical system 10 is for forming an optical subject image on the imaging element 11 and includes, for example, a zoom lens 12, an aperture 13, and an objective lens 15. Here, the zoom lens 12 is for changing the focal length of the imaging optical system 10 to perform zooming (change of imaging magnification). The stop 13 changes the passing range of the light beam passing through the imaging optical system 10 to adjust the brightness of the subject image formed on the imaging element 11. The pupil diameter of the imaging optical system 10 is also changed by changing the aperture diameter of the diaphragm 13. Furthermore, the objective lens 15 is an optical element mainly responsible for the power in the imaging optical system 10.

The imaging device 11 is configured by arraying a plurality of pixels (imaging pixels), and photoelectrically converts an object image formed by the imaging optical system 10 through the pupil color division filter 14 to convert the plurality of pixels (image pixels). Is an output of an arrayed image. Here, the color image pickup device 11 receives an object image for each of a plurality of wavelength bands (for example, but not limited to RGB) light, photoelectrically converts them, and outputs as an electric signal. It is an element.

The configuration of the color imaging device may be a single-plate imaging device provided with an on-chip device color filter, or may be a three-plate system using dichroic prisms for performing color separation into RGB color light, or the same. May be an imaging device of a method capable of acquiring RGB imaging information according to the position in the depth direction of the semiconductor at the pixel position of the semiconductor, or which imaging information of a plurality of wavelength bands can be acquired in pixel units It does not matter.

However, in the present embodiment, it is assumed that the imaging device 11 is a single-plate imaging device provided with an element color filter of primary color Bayer arrangement as shown in FIG. 2 on a chip, for example. Here, FIG. 2 is a diagram for explaining a pixel array of the image pickup device 11.

In the color filter of the primary color Bayer arrangement, as shown in FIG. 2, a B filter for configuring a B pixel and an R filter for configuring an R pixel are arranged in one diagonal direction, and the other diagonal direction The basic array has a filter configuration in which 2 × 2 pixels in which G filters for forming G pixels are arranged are arranged as a basic array.

In addition, although the imaging device 11 can be widely applied to an imaging device such as a CMOS sensor or a CCD sensor, it is not necessary to read out all pixels in order to measure the distance to the subject. It is preferable to employ a CMOS sensor that can read out a desired pixel.

The pupil color division filter 14 is disposed on the light path of the imaging optical system 10, and a pupil which color-divides the pupil of the imaging optical system 10 by providing plural partial pupils in the pupil of the imaging optical system 10 with different spectral characteristics. It is a color division optical system. Accordingly, the pupil color division filter 14 can also be called a band-limiting filter because it performs band limiting of light to be transmitted for each partial pupil.

Specifically, for example, the pupil color division filter 14 is configured as shown in FIG. FIG. 3 is a diagram for explaining an example of the configuration of the pupil color division filter 14.

In the pupil color division filter 14 shown in FIG. 3, the pupil of the imaging optical system 10 is divided into a first partial pupil and a second partial pupil. For example, the left half is a G (green) component and R The RG filter 14r that passes the (red) component and blocks the B (blue) component, and the GB filter 14b that passes the G and B components and blocks the R component on the right half. Therefore, the pupil color division filter 14 passes all the G components contained in the light passing through the aperture (and hence the pupil) of the diaphragm 11 of the imaging optical system 10, and passes the R component only in the partial pupil of the left half of the aperture. The B component is allowed to pass through only the partial pupil of the other right half of the aperture.

In addition, in order to prevent the occurrence of light quantity loss due to the mismatch of the spectral characteristics and the deterioration of the distance measurement accuracy, the RGB spectral transmission characteristics of the pupil color dividing filter 14 and the RGB spectral of the element filter of the image sensor 11 (see FIG. 2) The transmission characteristics are preferably identical or as close as possible.

The ring illumination 16 is an illumination device that illuminates the subject 6 with illumination light. The ring illumination 16 has a plurality of light sources such as LEDs arranged in a ring around the periphery of the optical path that does not shield the optical path of the imaging optical system 10, and the shadow of the illumination hardly occurs on the subject. Is equipped. Here, the ring illumination 16 is provided because the industrial microscope is assumed as described above, but another illumination device may be used, or natural light may be used without providing the illumination device. I do not care.

The imaging element control unit 21 controls the imaging element 11 and performs drive control of the imaging element 11 and readout control from the imaging element 11. Further, when the imaging device 11 is an analog imaging device, the imaging device control unit 21 also performs A / D conversion of a signal read from the imaging device 11.

The zoom control unit 22 performs control to move the zoom lens 12 in the optical axis direction and to change the focal length of the imaging optical system 10.

The focus control unit 23 controls the focus drive mechanism 24 so that the optical image of the subject 6 formed by the imaging optical system 10 is positioned on the imaging surface of the imaging device 11 (that is, focused). .

The focus drive mechanism 24 performs drive to adjust the focus position of the imaging optical system 10 based on the control signal from the focus control unit 23. For example, in the case of an industrial microscope, the focus drive mechanism 24 moves the lens barrel 2 itself in the direction of the optical axis, that is, moves the lens barrel 2 itself in a direction away from the subject 6 or in a direction approaching the subject 6. Configured as Further, in the case of a digital camera, for example, the focus drive mechanism 24 is configured as a mechanism for moving the focus lens included in the objective lens 15 described above in the optical axis direction.

The controller 3 is connected to the lens barrel 2 described above, and controls the entire system of the distance measuring device 1. The controller 3, the memory 32, the pixel shift detection unit 33, and the distance calculation unit 34 and a pixel interpolation operation unit 35.

The system controller 31 controls the whole of the distance measuring device 1 in an integrated manner, including each operation unit and the like in the controller 3 and each control unit and the like in the lens barrel 2. The system controller 31 also performs processing such as color separation of an image output from the imaging device 11 into a plurality of color images.

The memory 32 is for temporarily buffering the image signal received from the lens barrel 2, and includes, for example, an SDRAM or the like.

The pixel interpolation operation unit 35 interpolates interpolation pixels of pixels arranged in a plurality of color images relating to different partial pupils among a plurality of color images obtained by color separation of the image output from the imaging device 11, It is a pixel interpolation unit that generates at least a relative shift amount in a shift detection direction which is a direction to detect the shift amount. Specifically, the pixel interpolation operation unit 35 of the present embodiment obtains R, G, B color images obtained by color separation of one image read out from the image sensor 11 and A / D converted by the system controller 31. The interpolation calculation is performed on the R-color image and the B-color image to be subjected to shift detection among the above to generate an R-interpolated pixel and a B-interpolated pixel.

The pixel shift detection unit 33 generates a plurality of interpolation color images by combining a plurality of color images relating to different partial pupils and the interpolation pixels generated by the pixel interpolation operation unit 35, and a plurality of interpolation color images The relative displacement amount of the subject image in the above is detected, and is a part of the distance information generation unit. Specifically, the pixel shift detection unit 33 of the present embodiment combines the R-interpolated color image by combining the R-color image color-separated by the system controller 31 with the R-interpolated pixels generated by the pixel interpolation operation unit 35. A B-interpolated color image is generated by combining the B-color image generated and generated by the system controller 31 with the B-interpolated pixel generated by the pixel interpolation operation unit 35. Then, the pixel shift detection unit 33 detects the shift amount of the subject image in the generated R interpolation color image and B interpolation pixel.

Based on the displacement amount detected by the pixel displacement detection unit 33, the distance calculation unit 34 may be information related to the object distance (the distance to the object itself may be, or the direction and amount of displacement of focusing etc. The former is suitable for measuring, for example, the shape (height) of the subject, and the latter is suitable for generating, for example, focusing on the subject. Part of

The PC 4 is connected to the controller 3 described above, and has a control application 41 which is software that doubles as a user interface of the distance measuring device 1.

The monitor 5 displays an image signal of the subject 6 transmitted from the controller 3 via the PC 4 and application information on the control application 41.

Therefore, the user operates the distance measuring device 1 by using an input device (a keyboard, a mouse or the like) provided in the PC 4 while observing the display on the monitor 5.

Next, spatial positional deviation of each color image acquired in the distance measuring device 1 as described above will be described with reference to FIGS. 4 to 7. FIG. Here, FIG. 4 is a plan view showing how the subject light flux is collected when an object at a longer distance than the in-focus position is imaged, and FIG. 5 is a view on the object at a longer distance than the in-focus position. FIG. 6 is a view showing the shape of blur formed by light from a point for each color component, FIG. 6 is a plan view showing a state of subject light flux condensing when an object at a near distance side from the focusing position is imaged Reference numeral 7 is a diagram showing, for each color component, the shape of a blur formed by light from one point on the subject that is on the near distance side of the in-focus position.

First, when the subject is in the in-focus position, the light emitted from one point on the subject is collected at one point on the imaging device 11 regardless of which color component it is, and the subject as a point image Form an image. Therefore, no positional displacement occurs between colors, and a subject image without color blur is formed.

On the other hand, when the object OBJf is on the far side of the in-focus position, for example, the light emitted from one point on the object OBJf causes the G component to be changed as shown in FIGS. 4 and 5. A circular blurred object image IMGg is formed, a right half semicircular blurred object image IMGr is formed for the R component, and a left half semicircular blurred object image IMGb is formed for the B component. Therefore, when the object OBJf at a longer distance side than the in-focus position is imaged, a blurred image is formed in which the R component subject image IMGr is shifted to the right and the B component subject image IMGb is shifted to the left. The left and right positions of the R component and the B component in are opposite to the left and right positions of the R component transmission region (RG filter 14r) and the B component transmission region (GB filter 14b) in the pupil color division filter 14 when viewed from the imaging device 11. is there. Further, as shown in FIG. 5, the G component object image IMGg is a blurred image spanning the R component object image IMGr and the B component object image IMGb. Then, the more the subject OBJf moves away from the in-focus position to the far side, the larger the blur, and the distance between the center of gravity Cr of the subject image IMGr of the R component and the center of gravity Cb of the subject image IMGb for the B component The distance between the center of gravity Cr of the subject image IMGr and the center of gravity Cg of the object image IMGg of G component, and the distance between the center of gravity Cg of the object image IMGg of G component and the center of gravity Cb of the object image IMGb for B component Will be increased.

On the other hand, when the object OBJn is closer than the in-focus position, for example, the light emitted from one point on the object OBJn causes circular blurring of the G component as shown in FIGS. 6 and 7. A subject image IMGg to be formed is formed, a subject image IMGr having a semicircular blur in the left half is formed for the R component, and a subject image IMGb having a semicircular blur on the right half is formed for the B component. Therefore, when imaging the object OBJn on the near distance side of the in-focus position, a blurred image is formed in which the R component subject image IMGr is shifted to the left and the B component subject image IMGb is shifted to the right. The left and right positions of the R component and the B component in are the same as the left and right positions of the R component transmission region (RG filter 14r) and the B component transmission region (GB filter 14b) in the pupil color division filter 14 when viewed from the imaging device 11. is there. In addition, the object image IMGg of the G component is a blurred image that straddles the object image IMGr of the R component and the object image IMGb of the B component (see FIG. 7) as well at this short distance side. Then, the more the subject OBJn moves away from the in-focus position toward the near distance side, the larger the blur becomes, and the distance between the center of gravity Cr of the subject image IMGr of the R component and the center of gravity Cb of the subject image IMGb for the B component The distance between the center of gravity Cr of the subject image IMGr and the center of gravity Cg of the object image IMGg of G component, and the distance between the center of gravity Cg of the object image IMGg of G component and the center of gravity Cb of the object image IMGb for B component Will be increased.

Therefore, for the combination of two color images having different barycentric positions when passing through the pupil region of the imaging optical system 10, the object distance can be calculated by calculating the amount of deviation (the amount of phase difference) based on the correlation. Is possible. In the present embodiment, the amount of deviation between the subject image IMGr of the R component and the subject image IMGb of the B component with the largest separation distance between the barycentric positions is detected. In this case, the accuracy is higher than that in the case of detecting the displacement between the R component subject image IMGr and the G component subject image IMGg, or the G component subject image IMGg and the B component subject image IMGb. It is thought that it will be possible to detect

Next, it is assumed that the subject 6 has a shape as shown in FIG. 8 in the present embodiment. Here, FIG. 8 is a perspective view showing the subject 6.

That is, in the subject 6, it is assumed that a black box-like object 6b having a height is disposed on the white plate 6a. Then, a white print “1” as an example is formed on the upper surface of the black box-like object 6 b. FIG. 9 is a view showing the upper surface of the black box-like object 6 b in the subject 6.

When the imaging optical system 10 is focused on the surface of the white plate 6a, the subject image of the R component in the optical subject image 6i of the white printed matter “1” on the black box-like object 6b formed on the imaging device 11 The object image IMGb of the ING Gr and B component is a double image as shown in FIG. Here, FIG. 10 is a view showing a state in which the optical subject image IMGr of the R component and the optical subject image IMGb of the B component of the white printed matter formed on the imaging device 11 are deviated.

Therefore, among the color images obtained by imaging the white printed matter “1” on the black box-like object 6 b by the imaging device 11, the R signal image (R image) and the B signal image (B image) are also It becomes a double image similar to FIG.

The R image and the B image are transmitted to the PC 4 through the controller 3 and displayed on the monitor 5 by the control application 41 (note that, as an image to be displayed on the monitor 5, the deviation between the R image and the B image is It may be a corrected image or an image picked up without passing through the pupil color division filter 14 (in the case of simply observing the subject, it is preferable to be an image without such deviation). Furthermore, on the monitor 5, an operation screen related to the control application 41 is also displayed.

Then, it is assumed that an instruction input for measuring the height of the upper surface of the black box-like object 6b on which the white printed matter “1” is formed is performed by the user operating the mouse or the like.

A case is considered in which the amount of deviation between the R image and the B image is measured at the horizontal line A portion shown in FIG. Further, hereinafter, the horizontal direction is referred to as an X direction, and the vertical direction is referred to as a Y direction.

To simplify the description, only [(32 pixels in the X direction) × (2 pixels in the Y direction formed by the R-Gr row and the next Gb-B row in the Bayer arrangement) on the line A] will be considered. Here, Gr is a G pixel disposed between R pixels in the horizontal line, and Gb is a G pixel disposed between B pixels in the horizontal line.

FIG. 11 is a diagram showing the pixel values of the R image and the B image of the white printed matter “1” for the pixel array on the line A, and the interpolation pixel value and the composite value.

In the actual imaging element 11, there is a physically insensitive area between the pixels (for example, an area covered with a light shielding film or the like to suppress generation of electric charge in the electric circuit), but in the present embodiment In the following description, the aperture ratio of each pixel is assumed to be 100%.

Then, regarding the pixel value, for example, the pixel value when the R image is formed on the entire aperture (100%) of the R pixel is 100, and the pixel value when the R image is not formed at all on the R pixel is 0 As the intermediate value, it is assumed that the pixel value is in proportion to the percentage of the aperture of the R pixel where the R image is formed.

First, in the example shown in FIG. 11, the shift amount in the X direction of the R image and the B image of the white printed matter “1” is 3.4 pixels with the pixel pitch in the X direction as a unit. Therefore, it is most desirable that the amount of displacement obtained as a detection result be equal to the actual displacement amount of 3.4 pixels.

As described above, since the imaging element 11 has a Bayer array, G pixels (Gr pixels or Gb pixels) are disposed between R pixels and B pixels. Therefore, the acquired pixel values of the R pixel and the pixel values of the B pixel are not only discretely arranged by one pixel in the horizontal direction, but are further arranged to be shifted by one pixel each in the horizontal direction and the vertical direction It becomes.

When the R pixel value at coordinate X is R (X) and the B pixel value is B (X), the acquired pixel value is R (9) = R (11) = R (13) = R (15) = 100 for other R (X) = 0, B (12) = 60, B (14) = B (16) = B (18) = 100, B (20) = 40 for other It is assumed that B (X) = 0.

Here, the obtained data is used as it is, for example, as shown in the following Equation 1, ZNCC (Zero-mean Normalized Cross-Correlation)
[Equation 1]

When the correlation value of the amount of deviation is obtained by the above, the results shown in Table 1 below are obtained.

[Table 1]

The image shift amount in the left column of Table 1 corresponds to 2 pixels (more exactly 2) corresponding to the R pixel and the B pixel being arranged by 1 pixel skipping (every 2 pixels) in the horizontal direction. Pixel pitch is used as a unit.

Further, although the normalized cross correlation operation (ZNCC) is used to obtain the correlation value of the amount of deviation here, it is of course not limited to this, and a sum of squared difference (SSD) or a sum of absolute difference (SAD) Other operations such as) may be used.

From Table 1, the correlation value 0.92 at which the image shift amount is 1 is the highest correlation value, and the second highest correlation value is the correlation value 0.80 at which the image shift amount is 2. Therefore, the true shift amount is estimated to be intermediate between the image shift amounts 1 and 2.

Therefore, in order to calculate the shift amount with higher accuracy, sub-pixel interpolation is performed using the obtained correlation value. As the sub-pixel interpolation calculation, as described above, conformal straight line fitting, parabola fitting, etc. are known, but here, for example, it is assumed that equiangular straight line fitting is performed using the obtained correlation value ratio. .

In this case, from the ratio of the correlation value at which the image shift amount is 0, 1, 2
Interpolation value = (0.34-0.80) / (2 x (0.34-0.92)) = 0.4
The shift amount is calculated to be 1.4 pixels with the pixel pitch in the phase difference detection direction as a unit (hereinafter referred to simply as "pixel" or "pixel", the shift amount, pixel spacing, etc. The unit is more precisely the pixel pitch).

As described above, the R pixel spacing and the B pixel spacing used in the calculation are twice the actual pixel spacing (when considered as one pixel regardless of the color component) on the imaging device 11, and Since the R pixel and the B pixel are shifted by one pixel in the horizontal direction, if this point is corrected,
An amount of shift = 1.4 × 2 + 1 = 3.8 pixels is obtained.

As described above, since the actual shift amount is 3.4 pixels, a detection error of 0.4 pixels is generated in 3.8 pixels which is an image shift amount calculated when the obtained data is used as it is. It will be occurring.

Next, the case where interpolation calculation is performed by the pixel interpolation calculation unit 35 to generate interpolation pixels will be described.

The pixel interpolation operation unit 35 interpolates and estimates an R pixel value corresponding to the Gr pixel position in the R-Gr row and a B pixel value corresponding to the Gb pixel position in the Gb-B row. In this embodiment, in order to simplify the description, an average value of R pixel values on both sides is calculated as an R interpolation pixel value, and an average value of B pixel values on both sides is calculated as a B interpolation pixel value. (Refer to the corresponding part in FIG. 11).

The pixel shift detection unit 33 combines the original R pixel value and the R interpolation pixel value calculated by the pixel interpolation operation unit 35 to generate an R interpolation color image (see R pixel composite value in FIG. 11). Similarly, the original B pixel value and the calculated B interpolation pixel value are combined to generate a B interpolation color image (see the B pixel composite value in FIG. 11).

Then, using the generated interpolation color image (R interpolation color image and B interpolation color image), when the pixel displacement detection unit 33 determines the correlation value of the displacement amount by ZNCC as shown in the above equation 1, The results as shown in Table 2 below are obtained.

[Table 2]

The image shift amount in the left column of Table 2 corresponds to one pixel (more precisely, one pixel pitch) corresponding to arrangement of the interpolation color image in the horizontal direction every one pixel (every one pixel). Unit).

From Table 2, the correlation value 0.97 at which the image shift amount is 4 pixels is the highest correlation value, and the second highest correlation value is the correlation value 0.94 at which the image shift amount is 3 pixels. Therefore, the true shift amount is estimated to be intermediate between 3 pixels and 4 pixels.

Therefore, the pixel shift detection unit 33 performs sub-pixel interpolation using the obtained correlation value in order to calculate the shift amount with higher accuracy. Here, in the same manner as described above, it is assumed that isometric straight line fitting is performed using the ratio of the obtained correlation values.

Then, from the ratio of the correlation value when the image shift amount is 3 pixels, 4 pixels, 5 pixels,
Interpolation value = (0.94-0.86) / (2 x (0.86-0.97)) =-0.36
The shift amount is calculated as (4-0.36) = 3.64 pixels.

An image shift amount of 3.64 pixels calculated using the interpolation color image has a detection error of 0.24 pixels with respect to an actual shift amount of 3.4 pixels. Become. That is, it is possible to calculate the amount of deviation with higher accuracy than the detection error 0.4 in the case without the above-described interpolation (when the obtained data is used as it is). The detection error of 0.24 pixels also satisfies the accuracy of the shift amount of 0.33 pixels or less as described in the above-mentioned background art.

In the above description, the interpolation in the pixel interpolation operation unit 35 is performed by using the average value of two pixels on both sides in the X direction, but a method of estimating pixel values based on more pixels adjacent in the X direction may be used Instead of the X direction, a method of estimation using pixel values in the vertical direction (Y direction) or an oblique direction may be used. Furthermore, an estimation method using a super resolution technique or the like may be used based on a plurality of images.

The distance calculation unit 34 uses the proportional conversion formula obtained based on the configuration of the imaging optical system 10 and the pupil color division filter 14 to calculate the relative shift amount calculated by the pixel shift detection unit 33, and the height of the subject. Convert to directional distance information.

Here, assuming that the subject height is z (mm) and the deviation amount is x (pixel), a proportional conversion equation obtained by the configuration of the imaging optical system 10 and the pupil color division filter 14 is, for example, Equation 2 below
[Equation 2]
z = 0.15x
Shall be The subject height z is an example of “information regarding subject distance”.

At this time, when the displacement amount x = 3.64 calculated by the pixel displacement detection unit 33 is substituted, 0.546 mm is obtained as the subject height z. Since the subject height z obtained from the actual displacement amount x = 3.4 is 0.51 mm, the measurement height error of the distance measuring device 1 of the present embodiment is 0.036 mm.

This numerical value indicates the amount of deviation from the in-focus position, and when the lower surface of the black box-like object 6b (that is, the surface of the white plate 6a shown in FIG. 8) is at the current in-focus position, It indicates the height of the black box-like object 6b.

In the case where the control application 41 can set a plurality of measurement points, the user calculates and displays the difference between the plurality of set measurement points, thereby the user can obtain a desired point of the observation subject. It is possible to obtain height information of

In the present embodiment (and the following embodiments), various arithmetic units and the like are described as the hardware configuration in the controller 3, but the present invention is not limited to this. (In this case, the controller 3 may be omitted because the PC 4 can also function as the controller 3).

Furthermore, since the industrial microscope is assumed above, the example in which the distance measuring device 1 is configured to include the lens barrel 2, the controller 3, the PC 4, and the monitor 5 has been described, but the distance measuring device 1 For example, in the case of a digital camera or the like, these may be integrated. That is, when the distance measuring device 1 is a digital camera or the like, the lens barrel 2 is a lens barrel of a camera or a lens driving mechanism, the controller 3 and the PC 4 are a CPU of a camera Liquid crystal monitor etc.

According to the first embodiment, the number of samplings can be increased in the direction of calculating the relative shift amount. Specifically, it is possible to increase R pixels and B pixels to be subjected to shift amount detection in the horizontal direction which is the shift amount calculation direction. This makes it possible to improve the detection accuracy of the amount of deviation. As a result, the distance measurement accuracy with respect to the subject can be improved.

In addition, since the increase in the number of samplings is performed by interpolation calculation of pixels, there is an advantage that a mechanism or the like for moving the imaging device 11 (see Embodiment 2 and the like below) is unnecessary. At this time, if only necessary portions of the image data obtained from the imaging device 11 are subjected to interpolation calculation, it is also possible to reduce the calculation load. In addition, when the interpolation pixel value is calculated from the average value of adjacent pixel values, the calculation is simple and high-speed processing is also possible.

Thus, according to the distance measuring device of the present embodiment, when acquiring information on the subject distance based on color images of multiple colors obtained by imaging the pupil color-divided light with a color imaging device, the distance measurement accuracy Can be made higher.

Second Embodiment
12 and 13 show Embodiment 2 of the present invention, and FIG. 12 is a block diagram showing the configuration of the distance measuring device 1, and FIG. 13 is an R of white printed matter “1” for the pixel arrangement on line A. It is a figure which shows the pixel value of an image and B image, and a shift pixel value and a synthetic | combination value.

In the second embodiment, the same parts as those of the first embodiment described above are denoted by the same reference numerals and descriptions thereof will be omitted, and only differences will be mainly described.

In the first embodiment, the interpolation pixels are obtained by calculation, but in the present embodiment, the interpolation pixels are obtained by shifting the image sensor 11 and performing imaging a plurality of times.

That is, in the distance measuring device 1 of this embodiment, the element shift unit 17 and the element shift control unit 25 are added to the lens barrel 2 with respect to the configuration shown in FIG. The configuration is such that the interpolation operation unit 35 is removed.

Here, the element shift unit 17 and the element shift control unit 25 are pixel interpolation units, and move the image pickup device 11 in parallel in the direction of detecting deviation in a plane perpendicular to the optical axis of the imaging optical system 10 It has become.

The element shift unit 17 is for minutely moving the imaging element 11 in a direction (for example, a horizontal pixel array direction) in which at least the amount of deviation is detected. Specifically, it is conceivable to adopt a mechanical system capable of moving the imaging element 11 using a piezoelectric element or the like as the element shift unit 17. When the distance measuring device 1 is a digital camera or the like equipped with a device driving type camera shake correction mechanism, this camera shake correction mechanism may be used as the device shift unit 17.

The element shift control unit 25 controls driving of the element shift unit 17.

Also in the present embodiment, the subject is shaped as shown in FIG. 8 and FIG. 9 etc., and the obtained RB double image is as shown in FIG. It is assumed that the subject conditions such that the actual displacement amount of is 3.4 pixels are the same as in the first embodiment described above.

Next, an operation at the time of measuring information on a subject distance in the present embodiment will be described.

When the user inputs an instruction to start measurement via the control application 41, the system controller 31 generally controls each control system related to image capturing of the distance measuring device 1, and first captures an original image with the imaging element 11. Let

The original image thus taken is temporarily stored in the memory 32.

When the photographing of the original image is completed, next, the system controller 31 transmits, to the element shift control unit 25, an instruction to shift the imaging element 11.

When receiving this command, the element shift control unit 25 generates a drive signal and transmits the drive signal to the element shift unit 17.

When the element shift unit 17 receives this drive signal, it shifts the image sensor 11 in the direction to detect the shift amount, here, right shift by one pixel (more exactly by one horizontal pixel pitch) in the X direction of the pixel array. Let As a result of this shift, for example, the R pixel at X-direction coordinate 1 moves to X-direction coordinate 2, and the other pixels also move to a position at which the X-direction coordinate increases by one.

When the right shift for one pixel is completed in this way, the system controller 31 generally controls each control system related to image capturing of the distance measuring device 1 to cause the imaging element 11 to capture a shift image.

The shift image thus captured is also temporarily stored in the memory 32.

The pixel displacement detection unit 33 combines the R pixel value of the original image stored in the memory 32 and the R pixel value of the shift image to generate an R interpolation color image (see the R pixel composite value in FIG. 13). Similarly, the B pixel value of the original image and the B pixel value of the shifted image are combined to generate a B-interpolated color image (see the B pixel combined value in FIG. 13).

Then, using the generated interpolation color image (R interpolation color image and B interpolation color image), when the pixel displacement detection unit 33 determines the correlation value of the displacement amount by ZNCC as shown in the above equation 1, The results as shown in Table 3 below are obtained.

[Table 3]

The image shift amount in the left column of this Table 3 corresponds to one pixel (more precisely, one pixel pitch) corresponding to arrangement of the interpolation color image in each pixel (each pixel) in the horizontal direction. Unit).

From Table 3, the correlation value 0.97 in which the image shift amount is 3 pixels is the highest correlation value, and the second highest correlation value is the correlation value 0.92 in which the image shift amount is 4 pixels. Therefore, the true shift amount is estimated to be intermediate between 3 pixels and 4 pixels.

Therefore, the pixel shift detection unit 33 performs sub-pixel interpolation using the obtained correlation value in order to calculate the shift amount with higher accuracy. Here, in the same manner as described above, it is assumed that isometric straight line fitting is performed using the ratio of the obtained correlation values.

Then, from the ratio of the correlation value at which the image shift amount is 2, 3, and 4 pixels,
Interpolation value = (0.74-0.92) / (2 x (0.74-0.97)) = 0.39
The shift amount is calculated as (3 + 0.39) = 3.39 pixels.

The image shift amount 3.39 pixels calculated using the interpolation color image obtained by the shift of the imaging device 11 is 0.01 pixels with respect to the actual shift amount 3.4 pixels. It means that a detection error has occurred. That is, the amount of deviation can be calculated with extremely high accuracy than the detection error of 0.4 when the obtained data as described above is used as it is (in the case shown in Table 1). Further, the amount of deviation can be calculated more accurately than the detection error of 0.24 pixels when the interpolation color image obtained by the interpolation of the first embodiment is used (in the case shown in Table 2). The detection error of 0.01 pixel satisfies the accuracy of the shift amount of 0.33 pixel or less as described in the above-mentioned background art.

The distance calculation unit 34 uses the proportional conversion formula obtained based on the configurations of the imaging optical system 10 and the pupil color division filter 14 to calculate the amount of deviation calculated by the pixel deviation detection unit 33, and calculates the distance in the height direction of the subject. Convert to information.

Also in the present embodiment, assuming that the proportional conversion equation is as shown in the above-mentioned equation 2, if the displacement amount x = 3. 39 calculated by the pixel displacement detection unit 33 is substituted, 0.509 mm as the subject height z Is obtained. The measured height error of the distance measuring device 1 of the present embodiment is 0.001 mm in comparison with the subject height z = 0.51 mm obtained from the actual displacement amount x = 3.4.

In the above, the case where two still images are acquired by shifting one pixel is described, but for example, three still images may be acquired by shifting 2/3 pixels, and still more still images It may be acquired.

In addition, since it is not necessary to arrange the pixels of the original image and the pixels of the interpolation image at equal intervals, if the condition that the shifted image does not completely match the original image is satisfied (for example, The multiple, the pixel pitch, etc., and the shift amount may be properly selected.

And although acquisition of an image is performed like a still image in the above-mentioned, while making the image pick-up element 11 move continuously (shift), one or more shift images are sequentially acquired like a moving image photography succeeding an original image It does not matter if you do.

Further, in the present embodiment, the embodiment in which the interpolation pixel is obtained by shifting the imaging element 11 and performing imaging a plurality of times has been described, but the imaging optical system 10 may be shifted to perform imaging a plurality of times. . In this case, the element shift unit 17 and the element shift control unit 25 shift the imaging optical system 10.

Alternatively, both of the imaging optical system 10 and the imaging element 11 may be shifted to perform imaging a plurality of times. In this case, the element shift unit 17 and the element shift control unit 25 shift the imaging optical system 10 and the imaging element 11.

According to the second embodiment, substantially the same effect as that of the first embodiment described above is obtained, and the image sensor 11 is moved in parallel by the device shift unit 17 and the device shift control unit 25 which are device moving units. A plurality of color images obtained by acquiring a plurality of images at different positions in the detection direction from the imaging device 11 and performing color separation on the plurality of acquired images, the color images relating to the same color Interpolated color images are generated by combining the pixel shift detection unit 33. As a result, since it is possible to obtain an actual measurement value more accurate than the pixel value obtained by the interpolation operation, it is possible to obtain information on the subject distance with higher accuracy.

Third Embodiment
14 and 15 show Embodiment 3 of the present invention, and FIG. 14 is a block diagram showing the configuration of the distance measuring device 1, and FIG. 15 is a diagram for explaining the optical conditions of the imaging optical system 10. It is.

In the third embodiment, the same parts as those in the first and second embodiments described above will be assigned the same reference numerals and descriptions thereof will be omitted, and only differences will be mainly described.

In the present embodiment, as in the second embodiment described above, the interpolation pixel is obtained by shifting the image sensor 11 and performing imaging a plurality of times, but the shift amount of the image sensor 11 at this time It is set based on the optical conditions.

That is, the distance measuring device 1 of this embodiment has a configuration in which an element shift amount calculation unit 36 is added to the controller 3 in addition to the configuration shown in FIG. 12 of the second embodiment described above.

The element shift amount calculation unit 36 is a movement amount calculation unit that calculates the parallel movement amount (shift amount) of the imaging element 11 during acquisition of two images based on the optical conditions of the imaging optical system 10. .

Therefore, the element shift unit 17 and the element shift control unit 25 which are element movement units translate the imaging optical system 10 in parallel by the amount of parallel movement calculated by the element shift amount calculation unit 36.

The imaging optical system 10 has optical conditions as schematically shown in FIG. Here, in FIG. 15, the optical axis of the imaging optical system 10 is shown as O.

Further, D indicates the diameter of the diaphragm of the diaphragm 13 in the imaging optical system 10 (therefore, D / 2 is the radius from the optical axis O to the diaphragm 13). LG indicates the distance (gravity center distance) from the optical axis O to the gravity center position of the RG filter 14r in the pupil color division filter 14, for example. However, the distance from the optical axis O to the gravity center position of the GB filter 14 b (gravity center distance) is also LG.

Also in this embodiment, as shown in FIG. 3, filters in which the RG filter 14 r and the GB filter 14 b are respectively formed in a semicircular shape (in FIG. 15, R transmission at the upper side and B transmission at the lower side) are used. ,
LG = 4 × (D / 2) / (3π) = (2D) / (3π)
It becomes. Therefore, the gravity center distance LG depends on the size of the aperture diameter D.

Further, the focal length of the imaging optical system 10 is f. The focal length f is an amount that changes as the zoom lens 12 moves.

Assuming that the angle at which the diaphragm radius D / 2 of the diaphragm 13 is viewed from the focal position is θ, the numerical aperture NA (Numerical Aperture) of the imaging optical system 10 is
NA = sin θ
Is represented by As is apparent from FIG. 15, the numerical aperture NA is more precisely the object side NA, and depends on the size of the aperture diameter D of the aperture 13 (the smaller the aperture diameter D, the smaller the NA. As the aperture diameter D increases, the NA also increases.

Further, the distance between the focal position and the imaging surface 11 a of the imaging device 11 and the multiplier of the optical magnification are assumed to be Z.

The shift amount between the R and B subject images on the imaging surface 11a (located at an optical distance from the focal position Z) of the imaging element 11 is the image of the barycentric position of the RG filter 14r on the imaging surface 11a and the GB filter 14b. The distance to the image at the center of gravity, which is X in the figure (therefore, X / 2 is the distance from the optical axis O to the image of the center of gravity of the RG filter 14r or the image of the center of gravity of the GB filter 14b) .

From the geometrical relationship as shown in FIG. 15, the proportional relationship shown in the following Equation 3 is obtained.

[Equation 3]
(X / 2): Z = LG: f
When the distance measuring device 1 is used, for example, when the observation magnification is changed, in the case of an actual imaging optical system 10, the zoom lens 12 is controlled and the diaphragm 13 is adjusted at the same time. Therefore, when the observation magnification is changed, optical conditions such as the focal length f, the barycentric distance LG, and the NA value in the imaging optical system 10 change.

Furthermore, since the optical lens constituting the imaging optical system 10 is generally configured to obtain a desired characteristic by combining a plurality of lenses, R subject image and B subject image at an arbitrary zoom magnification It is difficult to express the displacement amount X by a simple mathematical expression.

On the other hand, the method of determining the resolution (therefore, this resolution is the resolution in the optical axis direction) required when performing height measurement of a subject (specimen) using, for example, an industrial microscope is described below (1 Or (2).

(1) A certain resolution (for example, resolution of 0.05 mm or more) can be obtained regardless of observation magnification (2) Depth of focus determined based on the optical condition of the set observation magnification (DOF: Depth The element shift amount calculation unit 36 obtains the resolution (for example, 1/2 DOF, more generally (k × DOF) with k as a predetermined coefficient) based on the Of Focus). Using the parameters and the optical parameters as input values, the shift amount of the image sensor 11 for obtaining the resolution of the required accuracy as shown in (1), (2) etc. in the height direction of the subject 6 is acquired.

As a method of acquiring the shift amount by the element shift amount calculation unit 36, the above-mentioned respective parameters may be input by arithmetic operation, or look-up tables (LUTs) for the respective parameters may be prepared in advance. It may be stored in the amount calculator 36, and this LUT may be referred to at the time of distance measurement.

For example, in the case where the resolution of (2) is adopted to create the LUT, the following procedure is performed.

First, based on the configuration of the imaging optical system 10, one of various optical conditions such as the zoom magnification, the numerical aperture NA, and the aperture diameter D is determined.

Then, the depth of focus (DOF) is determined based on this optical condition.

Next, the required resolution (k × DOF) in (2) is determined based on the depth of focus (DOF).

Further, the detection accuracy of the deviation required to detect the resolution (k × DOF) is determined, and a value equal to or less than the determined deviation (for example, a predetermined value of 1 or less (the predetermined value If it is too small, it is preferable to set a predetermined value close to 1 or a value close to 1) in order to obtain a high accuracy that is excessively high than the required accuracy.

Such a procedure is repeated while changing the optical conditions of the imaging optical system 10, for example, the zoom magnification.

As a result, a LUT of the shift amount of the image sensor 11 according to various optical conditions that can be taken by the imaging optical system 10 is created.

When the actual shift amount of the imaging device 11 is determined, not only the optical conditions of the imaging optical system 10 but also the pixel pitch of the imaging device 11 (the imaging device 11 is shifted with an accuracy finer than the pixel pitch) Also, limitations in accordance with various conditions of the entire imaging system (the entire imaging system including the imaging optical system 10, the pupil color division filter 14, the imaging element 11 and the like) are imposed.

An example of the shift amount of the imaging device 11 determined in this manner is as shown in Table 4 below.

[Table 4]
Optical magnification Shift amount 0.7 times 1 pixel 1.0 times 2/3 pixels 2.0 times 1/2 pixels Here, the “pixel” as a unit in the shift amount is more accurately shifted as described above It is a pixel pitch in the direction.

Therefore, for example, when the optical magnification is 0.7, the R pixel and the B pixel are shifted by one pixel (for example, to the right) after capturing the original image (in this case, the position of the G pixel at the original image capturing) R pixel and B pixel are located in the image), the shift image is photographed in this state, the original image and the shift image are combined to generate the interpolation color image, and the relative color image is generated based on the generated interpolation color image. The amount of deviation will be calculated.

In addition, when the optical magnification is 1.0, after capturing the original image, the image is shifted by 2/3 pixels (for example, to the right), and in this state, the shifted image is captured, and further by 2/3 pixels (for example, Shift to the right (in this case, it is shifted by 4/3 pixels from the original image shooting position), shoot the shifted image in this state, combine the original image and the two shifted images An interpolation color image is generated, and the amount of deviation is calculated based on the generated interpolation color image.

Furthermore, when the optical magnification is 2.0 times, after capturing the original image, it is shifted by 1/2 pixel (for example, to the right), and in this state, the first shifted image is captured, and then 1/2 pixel Shift the image by a half (for example, to the right) (shift amount of one pixel from the original image capture position), capture a second shift image in this state, and then shift it for The third shift image is captured in this state, and the four images, ie, the original image and the first to third shift images, are synthesized to obtain an interpolation color. An image is generated, and a relative displacement amount is calculated based on the generated interpolated color image.

Then, the element shift amount calculation unit 36 refers to, for example, the LUT shown in Table 4, and when the current optical magnification of the imaging optical system 10 is less than 0.85, for example, 0.7 times the optical magnification The shift amount is determined with reference to the column, and when the optical magnification is 0.85 or more and less than 1.5, the shift amount is determined with reference to the optical magnification column of 1.0. When it is 5 times or more, the shift amount is determined with reference to the optical magnification column of 2.0 times and the like.

According to the third embodiment, substantially the same effect as that of the second embodiment described above is obtained, and the amount of shifting the imaging element 11 is determined based on the optical conditions of the imaging optical system 10. It is possible to efficiently obtain a shifted image for generating an interpolated color image with appropriate accuracy, without being accompanied by insufficient accuracy or excessive accuracy.

And, when the shift amount in the plane perpendicular to the optical axis of the imaging device 11 is calculated based on the value obtained by multiplying the predetermined value by the depth of focus which is the amount in the optical axis direction, it is suitable for industrial microscopes, for example. It becomes composition.

Further, if a method of referring to the LUT is adopted, it is not necessary to perform calculation every time the optical condition of the imaging optical system 10 is changed, and processing load can be reduced to improve responsiveness.

Fourth Embodiment
16 and 17 show Embodiment 4 of the present invention, and FIG. 16 is a block diagram showing the configuration of the distance measuring device 1. FIG. 17 shows the user designating the measurement position of the subject on the screen of the monitor 5. It is a figure which shows a mode that it does.

In the fourth embodiment, the same parts as those in the first to third embodiments described above will be assigned the same reference numerals and descriptions thereof will be omitted, and mainly different points will be mainly described.

The present embodiment is basically the same configuration as Embodiment 2 described above, but the user can specify which part of the subject is to be subjected to distance measurement, and the distance on the imaging device 11 Only the part to be measured can be read out.

Therefore, the imaging device 11 in this embodiment adopts a configuration that can read out any pixel, and a specific example is a CMOS sensor or the like. Because this can not be done, it is not adopted in this embodiment).

The distance measuring device 1 of this embodiment has a configuration in which a reading area calculation unit 37 is added to the controller 3 in addition to the configuration shown in FIG. 12 of the second embodiment described above.

The read area calculation unit 37 is a pixel area setting unit that calculates and sets a pixel area for detecting a shift amount by the pixel shift detection unit 33 when capturing a shift image and reading out pixel information from the imaging device 11 .

Then, the imaging element control unit 21 drives the imaging element 11 so as to read out only the pixel values of the pixel area calculated by the reading area calculation unit 37 for the shift image.

Next, an operation at the time of measuring information on a subject distance in the present embodiment will be described.

When the user inputs an instruction to start measurement via the control application 41, the system controller 31 generally controls each control system related to image capturing of the distance measuring device 1, and first captures an original image with the imaging element 11. Let

The original image thus taken is temporarily stored in the memory 32.

When the photographing of the original image is completed, the photographed original image is transmitted to the PC 4 via the controller 3 and displayed on the screen 5 a of the monitor 5 by the control application 41. At this time, a message such as "Please specify a measurement part" may be displayed on the screen 5a of the monitor 5 and displayed.

Then, as shown in FIG. 17, the user operates the input device such as a mouse to move the pointer 5p on the screen 5a, and a portion to be measured (in the example shown, white printed matter “1” in the black box 6b By performing the confirmation operation on the substantially central portion of (1), one point (measurement point) on the measurement target portion is designated. The designated measurement point is transmitted from the control application 41 to the read area calculation unit 37.

When the information on the measurement point is received, the reading area calculation unit 37 calculates a pixel area to be read from the image pickup device 11 at the time of shift imaging in the subsequent stage.

Specifically, the calculation of the pixel area includes the line including the measurement point and the line adjacent to the line (in this selection method, one line is a line including an R pixel and the other line is a B pixel). For example, it is possible to calculate, as a region, R pixels and B pixels included in a constant range on the left and right around the measurement point in the included line).

A more specific example will be described.

First, it is assumed that the image sensor 11 is a sensor having a pixel array of 4000 pixels in the X direction and 3000 pixels in the Y direction. Then, the coordinates of the pixel on the image pickup device 11 are set such that the upper left corner is (X: Y) = (1: 1) and the lower right corner is (X: Y) = (4000: 3000).

In such a coordinate system, the measurement point designated by the user is (2000: 1500), and this measurement point is the B pixel in the Gb-B row.

Then, the read area operation unit 37 selects one of the 1499th line and the 1501th line adjacent to the line including the measurement point as a line including the R pixel. Here, for example, line 1499 is selected.

Further, the reading area calculation unit 37 selects, for example, the range of 32 pixels (specifically, the range of the X coordinate of 1985 to 2016) centered on the measurement point in the 1500th line, and the selected pixel range and X Further, a pixel range (a range of 32 pixels in this example) in line 1499 having the same coordinates is further selected. The pixel range selected here corresponds to the line A described in the second embodiment, that is, the line A is set so as to include the measurement points designated by the user.

As a result of such processing, the R pixel and the B pixel included in the pixel range selected by the read area operation unit 37 are as follows.

R pixel B pixel (1985: 1499) (1986: 1500)
(1987: 1499) (1988: 1500)
· ·
· ·
· ·
(2015: 1499) (2016: 1500)
In this way, the reading area calculation unit 37 sets the R pixel and the B pixel included in the selected pixel range as the pixel area, and transmits the information of the set pixel area to the imaging element control unit 21.

Then, the imaging element control unit 21 generates a read address from the imaging element 11 so that only the received pixel area, that is, a total of 32 pixels of the R pixel of 16 pixels and the B pixel of 16 pixels is read out. Read control 11.

In this way, with regard to the shift image, only the pixels necessary for detecting the amount of deviation are read out from the image sensor 11 while the image sensor 11 is shifted as described above. The pixel values thus read out are stored in the memory 32 as described above.

The subsequent calculation of the shift amount and the like are the same as in the second embodiment described above.

In the above description, in order to simplify the description, the R pixel and the B pixel in the pixel range of 32 pixels in the X direction centered on the designated measurement point are uniquely read out, but the original image is After photographing, based on the state of the subject in the original image, only the area more suitable for the correlation calculation may be set as the pixel area to be read out.

For example, for a line including a designated measurement point in the original image and a line adjacent to the line, edge detection of R pixel and B pixel in the vicinity of the measurement point is performed, and R pixels in a region including the detected edge And B pixels may be set in the pixel area.

Also, have the user specify two measurement points in the horizontal direction (more generally, in the direction in which the pupil is divided by the pupil color division filter 14), and R pixels on the line segment connecting the two specified points And B pixels may be set in the pixel region.

In the above description, only the R pixel and the B pixel included in the selected pixel range are targeted for readout, and the time required for readout is shortened as much as possible. However, the readout time is somewhat longer (for example, about twice) If it does not matter, the entire selected pixel range (that is, a pixel range that also includes Gr pixels and Gb pixels not used for the shift amount calculation) may be set as the pixel region to be read. In this case, there is an advantage that the readout control of the imaging device 11 becomes somewhat easier.

According to the fourth embodiment, higher-speed arithmetic processing can be performed while maintaining the subject height measurement function with high accuracy similar to that of the second embodiment described above.

Fifth Embodiment
18 to 20 show Embodiment 5 of the present invention, and FIG. 18 is a block diagram showing a configuration of the distance measuring device 1, and FIG. 19 shows an image formed on the imaging surface 11a of the imaging device 11. FIG. 20 shows a state of a subject image IMGr of R component and a subject image IMGb of B component in a subject image 6i according to an image, and a partially enlarged view of line A, FIG. 20 forms an image on the imaging surface 11a of the imaging device 11. FIG. 17A is a partially enlarged view of line A and FIG. 18B is a diagram showing a state of a subject image IMGr of an R component and a subject image IMGb of a B component when a subject image 6i related to a shift image is inclined and slightly larger than an original image;

In the fifth embodiment, the same parts as those in the first to fourth embodiments described above will be assigned the same reference numerals and descriptions thereof will be omitted, and only different points will be mainly described.

This embodiment has basically the same configuration as that of the second embodiment described above, but before and after shifting the imaging device 11, whether or not the relative movement of the subject 6 with respect to the imaging device 11 occurred except for the shifting It is determined whether or not there is movement, and since it is not possible to perform accurate distance measurement, unnecessary arithmetic processing is not performed.

That is, the distance measurement device 1 of this embodiment has a configuration in which a shift image comparison operation unit 38 is added to the controller 3 in addition to the configuration shown in FIG. 12 of the second embodiment described above.

The shift image comparison operation unit 38 acquires an original image acquired before shifting the imaging element 11 and a shift amount after moving the imaging element 11 at least in a pixel area where the shift amount is detected by the pixel shift detection unit 33. It is an image comparison unit that compares the match between a pixel shift image that is an image.

Specifically, FIG. 19 shows a subject image 6i related to the original image formed on the imaging surface 11a of the imaging element 11, and on the line A, an image portion AR of the subject image IMGr of R component; In the image portion AB of the B component subject image IMGb, an amount of deviation as shown in the drawing occurs.

On the other hand, FIG. 20 shows a subject related to a shift image (an image obtained by shifting the image sensor 11 by, for example, one pixel in the X direction after capturing the original image) formed on the imaging surface 11a of the image sensor 11 The image 6i is shown. As shown, the subject image 6i is somewhat rotated clockwise in the XY plane and slightly larger in size than the original image of FIG. Here, the reason why the size of the subject image 6i slightly increases is considered to be that the subject 6 has moved somewhat in the direction in which the subject 6 approaches the lens barrel 2 (upward in the height direction).

Since the subject image 6i is thus rotated and enlarged, the amount of deviation occurring in the image portion AR of the R component subject image IMGr and the image portion AB of the B component subject image IMGb on the line A is shown in FIG. It is larger than the 19 original images.

An interpolation color image is generated by combining the original image captured before and after the relative movement of the subject 6 with respect to the imaging device 11 (but excluding the movement due to the shift) and the shift image as described above. Even if the amount of deviation is calculated based on the interpolated color image, an accurate amount of deviation can not be obtained, that is, accurate distance measurement can not be performed.

Therefore, when the shift image comparison operation unit 38 detects an image deviation that is inappropriate for generating the interpolation color image by comparing the original image and the shift image, generation of the interpolation color image, interpolation, The generation of information on the subject distance by the pixel shift detection unit 33 and the distance calculation unit 34 based on the color image is stopped.

Next, an operation at the time of measuring information on a subject distance in the present embodiment will be described.

After capturing the original image, the pixel shift detection unit 33 calculates the amount of shift on the line A based on the R and B color images of the original image stored in the memory 32.

Next, after shifting the imaging device 11 as described in the second embodiment described above, a shift image is taken.

Then, based on the R-color image and the B-color image related to the shift image stored in the memory 32, the pixel shift detection unit 33 calculates the shift amount on the line A.

The shift image comparison operation unit 38 compares the shift amount obtained based on the original image with the shift amount obtained based on the shift image, and determines whether the difference between the two is -1 or more and +1 or less. Determine if

When the difference obtained by the shift image comparison operation unit 38 is -1 pixel or more and +1 pixel or less, the system controller 31 determines that the careless motion has not occurred and sets the original image and the shift image. The pixel shift detection unit 33 causes the pixel shift detection unit 33 to perform processing of generating an interpolated color image by combining the above and a highly accurate shift amount detection based on the interpolated color image. Further, the system controller 31 causes the distance calculation unit 34 to perform processing of generating information on the subject distance based on the detected shift amount.

On the other hand, when the difference obtained by the shift image comparison operation unit 38 is smaller than -1 pixel or larger than +1 pixel, the system controller 31 determines that an inadvertent movement has occurred and performs control. The user is notified of that via the application 41.

Then, the system controller 31 cancels the generation process of the interpolation color image by the pixel shift detection unit 33 and the detection process of the shift amount based on the interpolation color image. Therefore, the generation of the information on the subject distance based on the displacement amount obtained from the interpolation color image by the distance calculation unit 34 is also automatically stopped.

Furthermore, the system controller 31 performs processing of the pixel shift detection unit 33 and processing of the distance calculation unit 34 based on only the original image instead of highly accurate processing based on the interpolated color image, and generates information on the subject distance. Let

In addition, when notifying the user that careless movement has occurred, information on the subject distance obtained only from the original image and information on the subject distance obtained only from the shift image are combined as a reference value. You may make it notify. In this case, the system controller 31 further causes the processing of the pixel shift detection unit 33 and the processing of the distance calculation unit 34 to be performed based on only the shift image to generate information on the subject distance.

In the above description, in consideration of not using G pixels in the pixel array for distance detection, it is determined whether the difference is −1 or more and +1 or less, but the determination condition is The present invention is not limited to the above, and other determination conditions in consideration of subject conditions etc. may be adopted, and the determination conditions may be variably controlled according to the subject conditions.

Also, in the above description, it was determined whether or not there was an inadvertent movement based on the respective displacement amount calculation results at the same coordinates of the original image and the shift image, but it is not limited to this technology. Other known techniques such as a comparison method of detecting the movement vector of the subject may be used as appropriate.

According to the fifth embodiment, substantially the same effect as that of the second embodiment described above can be obtained, and it is determined whether or not the relative movement of the subject 6 with respect to the imaging device 11 occurs carelessly. When it is determined that a certain movement has occurred, the synthesis of the corresponding image is stopped, and the subsequent processing based on the interpolated color image is also stopped, so that the unnecessary load increase due to unnecessary calculation processing is suppressed and erroneous It is possible to prevent giving information on the subject distance to the user.

In addition, when it is determined that the careless movement has occurred, the user is notified of the fact, so that the user can be given an opportunity to perform precise measurement again.

Of course, the configurations of the above-described third to fifth embodiments based on the second embodiment may be combined as appropriate.

Furthermore, although the above description mainly describes the distance measuring device, a control method may be used to control the distance measuring device as described above, or a control program for causing a computer to control the distance measuring device as described above, It may be a non-transitory recording medium readable by a computer that records the control program.

The present invention is not limited to the above-described embodiment as it is, and in the implementation stage, the constituent elements can be modified and embodied without departing from the scope of the invention. In addition, various aspects of the invention can be formed by appropriate combinations of a plurality of constituent elements disclosed in the above-described embodiment. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, the constituent elements in different embodiments may be combined as appropriate. As a matter of course, various modifications and applications are possible without departing from the scope of the invention.

This application is based on Japanese Patent Application No. 2012-120044 filed on May 25, 2012 as a basis for claiming priority, and the above disclosure is made of the present specification and claims. It shall be quoted in the drawings.

Claims (8)

1. An imaging optical system for forming an image of an object;
   A pupil color division optical system which is disposed on the optical path of the imaging optical system and which divides the pupil of the imaging optical system by providing different partial characteristics to the plurality of partial pupils of the imaging optical system;
   A color imaging element that photoelectrically converts an object image formed by the imaging optical system via the pupil color division optical system, and outputs an image in which a plurality of pixels are arranged;
   A pixel interpolation unit that generates interpolation pixels of pixels arranged in a plurality of color images related to different partial pupils among a plurality of color images obtained by color separating an image output from the image pickup device;
   A plurality of interpolated color images are generated by combining the plurality of color images relating to the different partial pupils and the interpolation pixels generated by the pixel interpolation unit, and the relativeness of the subject images in the plurality of interpolation color images A distance information generation unit that detects an amount of deviation and generates information on a subject distance based on the amount of deviation;
   Equipped with
   The distance measuring device, wherein the pixel interpolation unit generates the interpolation pixel in a shift detection direction which is at least a direction in which the shift amount is detected.
2. The pixel interpolation unit includes an element moving unit configured to move at least one of the imaging optical system and the imaging element in parallel in the deviation detection direction in a plane perpendicular to the optical axis of the imaging optical system. The interpolation pixel is generated by translating at least one of the imaging optical system and the imaging device by the unit to obtain a plurality of images having different positions in the shift detection direction from the imaging device,
   The distance information generation unit generates the interpolation color image by combining color images of a plurality of color images obtained by color-separating a plurality of acquired images and relating to the same color. The distance measuring device according to claim 1, characterized in that:
3. The apparatus further comprises a movement amount calculation unit that calculates the parallel movement amount of at least one of the imaging optical system and the imaging element while acquiring a plurality of images based on the optical conditions of the imaging optical system,
   The distance measuring apparatus according to claim 2, wherein the element moving unit moves in parallel at least one of the imaging optical system and the image pickup element by the parallel movement amount calculated by the movement amount calculating unit. .
4. The distance measurement apparatus according to claim 3, wherein the optical condition of the imaging optical system used by the movement amount calculation unit to calculate the parallel movement amount includes a focal depth of the imaging optical system.
5. The movement amount calculation unit calculates a parallel movement amount of at least one of the imaging optical system and the imaging device based on a value obtained by multiplying the focal depth by a predetermined coefficient. Distance measuring device.
6. The image processing apparatus further includes a pixel area setting unit configured to set a pixel area for detecting the displacement amount by the distance information generation unit.
   3. The distance measurement according to claim 2, wherein after at least one of the imaging optical system and the imaging element is moved in parallel, only the pixel area set by the pixel area setting unit is read out from the imaging element. apparatus.
7. At least in a pixel area where the distance information generation unit detects the shift amount, an image acquired before translating at least one of the imaging optical system and the imaging element and a shift that is an image acquired after translating the image And an image comparison unit that compares the image with the image;
   The distance information generation based on the interpolation color image when the image comparison unit compares the image and the shift image and detects an image deviation that is inappropriate for generating the interpolation color image. The distance measuring apparatus according to claim 2, wherein generation of information on the subject distance by the unit is stopped.
8. The pixel interpolation unit includes a pixel interpolation operation unit that generates the interpolation pixel by performing an interpolation operation on a color image obtained by color separation of one image. Distance measuring device as described.

Images