JP2005148454A - Three-dimensional confocal laser scanning microscope system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To obtain a highly accurate three-dimensional image by using accurate displacement information of an actuator for driving an objective lens, and also to realize a three-dimensional laser microscope system capable of obtaining a stable image by controlling a vibration level based on the acceleration calculated from the displacement information of the actuator. <P>SOLUTION: In the three-dimensional confocal laser scanning microscope system which is provided with a confocal scanner for obtaining sliced images of a sample as confocal images, a video camera for converting the confocal images to video signals, an image processing device for converting the video signals to image data, the actuator for shifting the focal point of the objective lens of the microscope in the direction of the optical axis, and a control means for generating a scanning waveform signal for scanning the objective lens in the direction of the optical axis via this actuator, and which is configured so as to be able to obtain sliced images in the direction of depth of the sample, the image processing device is configured so as to synthesize the sliced images to form a three-dimensional image based on the displacement signal outputted from the actuator. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to a three-dimensional confocal laser microscope system, and more particularly to an improvement for accurately capturing a displacement of an actuator that moves an objective lens in an optical axis direction and obtaining a highly accurate and stable three-dimensional image. .

  The confocal microscope can obtain a slice image without making a thin section of the sample, and can construct an accurate three-dimensional stereoscopic image of the sample from the slice image. It is used for observation, morphology observation, or LSI surface observation in the semiconductor market (see, for example, Patent Document 1).

JP 2002-72102 A

  FIG. 4 is a configuration diagram of the confocal microscope described in Patent Document 1. The video rate camera 1, the confocal scanner 2, the microscope 3, the actuator 4 and the objective lens 5 are arranged on the same optical axis. The confocal scanner 2 has a nipou disk having a large number of pinholes and a corresponding microlens array, and is a compact add-on type employing a nipou disk system composed of a simple optical system.

  The confocal scanner 2 is attached to the camera port of the microscope 3. The confocal microscope uses a laser beam to input an image of the sample to the confocal scanner 2 via the objective lens 5, the actuator 4 and the microscope 3. The confocal scanner 2 obtains a confocal image of the sample and inputs it to the video rate camera 1.

  FIG. 5 is a time chart of various signals handled by the confocal microscope of FIG. The video rate camera 1 converts the confocal image into a video signal 101, and inputs the video signal 101 to the confocal scanner 2, the signal input terminal of the synchronization interface box 9, and the image input terminal of the image processing device 6. The confocal scanner 2 performs rotation synchronization control of the Niipou disc in synchronization with the video signal 101.

  When a video tape deck is used for the image processing apparatus 6, the video tape deck records the video signal 101 input from the image input terminal and the start signal 103 input from the audio input terminal simultaneously on the video tape for a long time. To do. On the video tape, the confocal image that changes in real time and the scanning start timing of the focal position of the objective lens 5 are recorded simultaneously.

  The synchronous interface box 9 extracts either the even-numbered pulse train or the odd-numbered pulse train of the video signal 101 and creates an internal A signal. The arbitrary waveform generator 7 generates a trigger signal 102 which is an H level pulse signal, inputs the trigger signal 102 to the trigger input terminal of the synchronous interface box 9, and uses it for the start timing of scanning of the focal plane.

  The synchronization interface box 9 creates an internal B signal in synchronization with the falling edge of the trigger signal 102. The internal B signal is a pulse signal having a pulse width time of H level of about 35 msec and slightly longer than the video rate time of the video rate camera 1. The synchronous interface box 9 generates a start signal 103 by performing an AND operation on the inverted signal of the internal A signal and the internal B signal, and inputs the start signal 103 to the synchronous input terminals of the image processing device 6 and the arbitrary waveform generator 7.

  The image processing device 6 starts capture for converting and recording the video signal 101 into image data in synchronization with the rising edge of the start signal 103 from the synchronization input terminal. Based on the video signal 101 from the signal input terminal, the synchronization interface box 9 controls rotation synchronization of the Niipou disc by the confocal scanner 2, the start timing of acquisition of the video signal by the image processing device 6, and the objective lens of the optical control system. All the timings for starting the scanning of the focal position are synchronized.

  The arbitrary waveform generator 7 starts scanning the focal position of the objective lens 5 by the optical control system in synchronization with the rise of the start signal 103. The arbitrary waveform generator 7 generates a scanning signal 104 and inputs it to the controller 8. The scanning signal 104 is a sawtooth signal that increases linearly from L level to H level in a certain time. The controller 8 inputs the scanning signal 104 to the actuator 4. The actuator signal 105 is an actual actuator position signal, and after having fully extended, there is an overshoot after returning to the origin at once, and during this time, there is a dead zone.

  The actuator 4 is attached between the objective lens revolver of the microscope 3 and the objective lens 5, and the length in the focal direction of the image changes in proportion to the level of the scanning signal 104 by piezo driving, and the focal position of the objective lens 5 To control. The confocal microscope acquires a slice image of the sample by scanning the focal plane based on the scanning signal 104.

  According to such a configuration, the rotation synchronization control of the Niipou disc, the acquisition timing of the video signal by the image processing device, and the scanning start timing of the focal position of the lens by the optical control system are all synchronized with the video signal, The position accuracy of the confocal image is improved, and variations in individual acquisition times are eliminated when acquiring a plurality of slice images, so that a highly reliable slice image can be obtained.

However, in the conventional three-dimensional confocal laser microscope system, the displacement of the actuator that moves the objective lens in the optical axis direction uses an estimated value based on the voltage value applied to the actuator.
For this reason, if an error occurs between the set value of the displacement and the actual value, the position information in the depth direction of the sample is not accurate, and the constructed three-dimensional image lacks accuracy.
Further, when the actuator returns from the maximum displacement to the minimum displacement, the maximum acceleration occurs, and the huge acceleration becomes vibration. For example, when the frequency matches the natural frequency of the actuator itself or the microscope housing, a resonance phenomenon occurs. In other words, since the acceleration could not be controlled, there was a problem that the sample surface became unstable due to vibration and could not be observed.

  The present invention is intended to solve the problem of such a conventional confocal microscope system, and obtains a highly accurate three-dimensional image using accurate displacement information of an actuator that drives an objective lens. At the same time, it is an object of the present invention to realize a three-dimensional laser microscope system capable of suppressing vibrations by controlling acceleration calculated from displacement information of an actuator and obtaining a stable image.

  The present invention is a three-dimensional confocal laser microscope system configured as follows.

(1) a confocal scanner that acquires a slice image of a sample as a confocal image;
A video rate camera for converting the confocal image into a video signal;
An image processing device for converting the video signal into image data;
An actuator that moves the focal position of the objective lens of the microscope in the direction of the optical axis;
Control means for generating a scanning waveform signal for scanning the objective lens in the optical axis direction via this actuator,
A three-dimensional confocal laser microscope system configured to acquire a slice image in a depth direction of a sample,
The three-dimensional confocal laser microscope system, wherein the image processing device generates a three-dimensional image by synthesizing the slice images based on a displacement signal output from the actuator.

(2) a confocal scanner that acquires a slice image of the sample as a confocal image;
A video rate camera for converting the confocal image into a video signal;
An image processing device for converting the video signal into image data;
An actuator that moves the focal position of the objective lens of the microscope in the direction of the optical axis;
Control means for generating a scanning waveform signal for scanning the objective lens in the optical axis direction via this actuator,
A three-dimensional confocal laser microscope system configured to acquire a slice image in a depth direction of a sample,
The control means calculates an acceleration when the actuator is displaced based on a displacement signal output from the actuator, and controls so that the acceleration does not exceed a preset value. Focus laser microscope system.

(3) The control means includes:
A waveform calculation device for obtaining the waveform of the scanning waveform signal by calculation; and
An arbitrary waveform generator that stores this scanning waveform and generates it in synchronization with the scanning waveform generation trigger signal;
An actuator driver for driving the actuator based on a scanning waveform generation trigger signal output from the arbitrary waveform generator;
A signal controller that generates each trigger signal in synchronization with the video signal of the video rate camera and gives it to each unit;
An A / D converter for A / D converting the displacement signal and outputting displacement data to the image processing device and the waveform calculation device;
The three-dimensional confocal laser microscope system according to claim 1 or 2, characterized by comprising:

(4) When the calculated acceleration exceeds a preset value, the control means sets the return time from the maximum displacement to the minimum displacement of the actuator as an integral multiple of the vertical synchronization signal period included in the video signal. The three-dimensional confocal laser microscope system according to claim 2, wherein the three-dimensional confocal laser microscope system is increased to

(5) The scanning waveform is a triangular wave, and a discontinuous change portion of the triangular wave is a waveform generated by S-shaped control. 3D confocal laser microscope system.

(6) The scanning waveform is a step wave, and a discontinuous change portion of the step wave is a waveform generated by S-shaped control. The three-dimensional confocal laser microscope system described.

The present invention has the following effects.

According to the first aspect of the present invention, an accurate displacement of the actuator that moves the objective lens in the optical axis direction can be obtained, and a highly accurate dimensional image can be obtained by synthesizing slice images based on the displacement information. A three-dimensional confocal laser microscope system that can be created can be realized.

According to the second aspect of the present invention, the acceleration at the time of displacement of the actuator is calculated based on the displacement signal output from the actuator, and the acceleration is controlled so that the acceleration does not exceed a preset value. It is possible to realize a three-dimensional confocal laser microscope system that can reduce and obtain a stable image.

According to the third aspect of the present invention, the displacement data of the actuator is generated by the A / D converter, and by using this displacement data, the image processing apparatus can obtain a highly accurate three-dimensional image and the actuator. Since the acceleration associated with the displacement can be suppressed within a predetermined value, vibration can be suppressed and a stable image can be obtained.

According to the fourth aspect of the present invention, the acceleration generated when the actuator moves from the maximum displacement to the minimum displacement can be suppressed within a predetermined value, and vibration can be reduced and a stable image can be obtained.
Further, by setting the return time from the maximum displacement of the actuator to the minimum displacement to be an integral multiple of the vertical synchronization signal period, it is possible to set the return time that effectively uses the update time of the camera frame.

According to the fifth and sixth aspects of the present invention, by generating the discontinuous change portion of the scanning waveform by the S-shaped control, the acceleration of the portion can be reduced.

  Hereinafter, the present invention will be described in detail with reference to the drawings. FIG. 1 is a block diagram showing an embodiment of a three-dimensional confocal laser microscope system according to the present invention. In FIG. 1, the same parts as those in FIG. 4 are denoted by the same reference numerals, and the description thereof is omitted.

In FIG. 1, 10 is a signal controller, 11 is a waveform calculator, 12 is an actuator driver, 13 is an A / D converter, and 20 is a sample.
The signal controller 10 receives a video signal output from a video rate camera (hereinafter simply referred to as a camera) 1 and outputs the video signal as it is, and also extracts a vertical synchronization signal from the video signal, and based on this, various signals are extracted. Generate a trigger signal.
When the arbitrary waveform generator 7 a receives the scanning waveform generation trigger signal from the signal controller 10, the arbitrary waveform generator 7 a generates a scanning waveform sent from the waveform calculator 11 and stored in advance, and sends it to the actuator driver 12.

The waveform calculation device 11 calculates a triangular wave or a step wave from the scanning period of the objective lens 5 when observing the sample 20 and the scanning distance in the optical axis direction, and outputs it as a scanning waveform to the arbitrary waveform generator 7a. .
The actuator driver 12 generates a drive signal for driving the actuator 4 based on the scanning waveform signal output from the arbitrary waveform generator 7a.
The actuator 4 scans the objective lens 5 in the direction of the optical axis in accordance with the output waveform of the arbitrary waveform generator 7a, and continuously acquires cross-sectional slice images of the observation sample by the image processing device 6a in synchronization with the scanning.

  The actuator 4 is provided with a displacement sensor (not shown) for sensing its own displacement. The displacement sensor outputs a displacement signal and feeds it back to the actuator driver 12, whereby the position of the actuator 4 is controlled. At this time, an error may occur between the target position (estimated position) and the actual position.

The actuator driver 12 outputs the acquired displacement signal to the A / D converter 13. The A / D converter 13 generates displacement data indicating the displacement of the actuator 4, that is, the displacement of the objective lens 5 in the optical axis direction by A / D converting the displacement signal in synchronization with the image capture trigger.
The image processing device 6a obtains an accurate position of the slice image in the optical axis direction from the displacement data, and synthesizes the slice images to generate a three-dimensional image. Thereby, a highly accurate three-dimensional image is obtained.
Here, the part constituted by the waveform calculation device 11, the arbitrary waveform generator 7a, the actuator driver 12, the signal controller 10, and the A / D converter 13 is referred to as control means.

Next, the operation of such a configuration will be described with reference to the time chart of each signal shown in FIG.
When the video signal (including the vertical synchronization signal) shown in FIG. 2A is sent from the camera 1 to the control signal device 10, the signal controller 10 sends the video signal as it is to the image processing device 6a. A vertical synchronization signal is extracted from the video signal and sent to the confocal scanner 2, and various trigger signals, that is, a scanning waveform generation trigger signal [FIG. 2 (c)], an image capture trigger signal [FIG. 2 (e)]. ] Is generated.

  When the signal controller 10 receives the image capture start signal shown in FIG. 2B, the signal controller 10 generates the first vertical synchronization signal after the image capture start signal becomes LOW as the scanning waveform generation trigger signal [FIG. )] To the arbitrary waveform generator 7a and a vertical synchronization signal to the image processing device 6a as an image capture trigger signal shown in FIG. The image capture start signal [FIG. 2B] is a signal that the operator inputs to the signal controller at an arbitrary timing from a host controller such as a personal computer, and its pulse width is the vertical synchronization signal of the video signal. It is more than twice the period.

When the arbitrary waveform generator 7a receives the scanning waveform generation trigger signal from the signal controller 10, the arbitrary waveform generator 7a generates a step wave shown in FIG. 12 to send.
In FIG. 2 (d), a step waveform in which the voltage gradually increases with the passage of time and returns to the base voltage at a predetermined voltage value is used as a scanning waveform signal. This scanning waveform is a triangular wave. Also good.

  The actuator 4 is driven by a drive signal from the actuator driver 12 and scans the objective lens 5 in the optical axis direction according to the waveform of FIG. In the image processing apparatus 6a, an image of the sample 20 is acquired in synchronization with this.

  In the above series of operations, the displacement sensor provided in the actuator 4 outputs a displacement signal of each step wave (Z1, Z2,..., Zn), which is converted into A / D by the A / D converter 13. Transform to generate displacement data. Using this displacement data, the actual position of the slice image with respect to the observation sample is obtained, and a three-dimensional image is synthesized. Thereby, a highly accurate three-dimensional image is obtained.

Next, acceleration control when the actuator is displaced will be described.
The waveform calculation device 11 calculates the absolute value of the acceleration in one cycle of the scanning waveform from the displacement data output from the A / D converter 13, and when the value exceeds a preset value, the absolute value of the acceleration is less than the set value. Control to become.

The acceleration is calculated by the following formula.
Velocity V = first derivative of displacement = dz / dt
Acceleration a = first derivative of speed = dv / dt
Here, z is displacement and t is time.

The operation of such a configuration will be described with reference to the time chart of each signal shown in FIG. FIG. 3 is a time chart showing how the acceleration of the actuator is controlled.
In FIG. 3, FIG. 3A is a vertical synchronization signal (video signal) sent from the camera 1 to the control signal device 10. FIG. 3B shows a step-like scanning waveform calculated by the waveform calculation device 11.

The step waveform changes in synchronization with the fall of the vertical synchronization signal. In this discontinuous change portion, S-shaped control is performed to reduce the acceleration. For this reason, the acceleration of this part becomes constant and changes in a rectangle as shown in FIG. FIG. 3C shows the acceleration obtained from the displacement data and time.

The relationship between speed and acceleration in S-shaped control is expressed by the following equation.
Displacement z = (1/2) · A · t ²
Velocity v = first derivative of displacement z = A · t
Acceleration a = first derivative of velocity v = A
Here, t is time and A <(2πf) ² .

  When the absolute value of the calculated acceleration exceeds a preset value, control is performed to reduce the absolute value. Specifically, since the maximum acceleration is obtained when the actuator 4 returns from the maximum displacement to the minimum displacement, the return time of the actuator 4 is set so that the absolute value of the acceleration does not exceed the set value as shown in FIG. increase. At this time, even if the minimum displacement is returned during the imaging of one frame of the camera, it is necessary to wait until the imaging of the next frame starts. Accordingly, if the time for returning from the maximum displacement to the minimum displacement is set to an integral multiple of the vertical synchronization signal period (twice in FIG. 3D), the return time that effectively uses the frame update time can be set.

  In FIG. 3 (b), a step wave that gradually increases with time and returns to the base voltage (minimum displacement) at a predetermined voltage value (maximum displacement) is used as the scanning waveform. The scanning waveform may be a triangular wave. In this case, S-shaped control is performed at a discontinuous change portion (turning point) of the triangular wave in order to reduce the acceleration of this portion.

As described above, as shown in FIG. 3E, acceleration, that is, vibration can be rapidly reduced, and a stable image can be acquired.

The above-described image processing device 6a and waveform calculation device 11 may be realized by a personal computer.
Further, the present invention is not limited to the above-described embodiments, and includes many changes and modifications without departing from the essence thereof.

It is a block diagram which shows one Example of the three-dimensional confocal laser microscope system which concerns on this invention. It is a time chart of each signal concerning the present invention. It is a time chart which showed a mode that the acceleration of an actuator was controlled. It is a block diagram which shows an example of the conventional confocal microscope. It is a time chart of each signal which the confocal microscope of FIG. 4 handles.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Video rate camera 2 Confocal scanner 3 Microscope 4 Actuator 5 Objective lens 6a Image processing apparatus 7a Arbitrary waveform generator 10 Signal controller 11 Waveform calculation apparatus 12 Actuator driver 13 A / D converter 20 Sample

Claims (6)

A confocal scanner that acquires a slice image of the sample as a confocal image;
A video rate camera for converting the confocal image into a video signal;
An image processing device for converting the video signal into image data;
An actuator that moves the focal position of the objective lens of the microscope in the direction of the optical axis;
Control means for generating a scanning waveform signal for scanning the objective lens in the optical axis direction via this actuator,
A three-dimensional confocal laser microscope system configured to acquire a slice image in a depth direction of a sample,
The three-dimensional confocal laser microscope system, wherein the image processing device generates a three-dimensional image by synthesizing the slice images based on a displacement signal output from the actuator. A confocal scanner that acquires a slice image of the sample as a confocal image;
A video rate camera for converting the confocal image into a video signal;
An image processing device for converting the video signal into image data;
An actuator that moves the focal position of the objective lens of the microscope in the direction of the optical axis;
Control means for generating a scanning waveform signal for scanning the objective lens in the optical axis direction via this actuator,
A three-dimensional confocal laser microscope system configured to acquire a slice image in a depth direction of a sample,
The control means calculates an acceleration when the actuator is displaced based on a displacement signal output from the actuator, and controls so that the acceleration does not exceed a preset value. Focus laser microscope system. The control means includes
A waveform calculation device for obtaining the waveform of the scanning waveform signal by calculation; and
An arbitrary waveform generator that stores this scanning waveform and generates it in synchronization with the scanning waveform generation trigger signal;
An actuator driver for driving the actuator based on a scanning waveform generation trigger signal output from the arbitrary waveform generator;
A signal controller that generates each trigger signal in synchronization with the video signal of the video rate camera and gives it to each unit;
An A / D converter for A / D converting the displacement signal and outputting displacement data to the image processing device and the waveform calculation device;
The three-dimensional confocal laser microscope system according to claim 1 or 2, characterized by comprising:   When the calculated acceleration exceeds a preset value, the control means sets the return time from the maximum displacement to the minimum displacement of the actuator to be an integral multiple of the vertical synchronization signal period included in the video signal. The three-dimensional confocal laser microscope system according to claim 2, wherein the three-dimensional confocal laser microscope system is increased.   5. The three-dimensional image according to claim 1, wherein the scanning waveform is a triangular wave, and a discontinuous change portion of the triangular wave is a waveform generated by S-shaped control. Focus laser microscope system. 5. The scanning waveform according to claim 1, wherein the scanning waveform is a step wave, and a discontinuous change portion of the step wave is a waveform generated by S-shaped control. Dimensional confocal laser microscope system.

Images