TWI577331B - You can self-correct the method of marking visible light with visible light - Google Patents

Description

Self-correcting method for indicating invisible fluorescence with visible light

The invention relates to a method for self-correcting to indicate invisible fluorescence by visible light, in particular to an instant projection, in particular to direct projection of visible light in a fluorescent position, so that the operator can project by observing the visible light spot. The area where the fluorescent dye is located is observed in the area.

The fluorescence spectrum excited by Indocyanine Green (ICG), which is commonly used in clinical practice, is a near-infrared part, which is invisible light, and it is impossible to directly observe the position of the fluorescent dye directly by the naked eye. Therefore, the problem with the traditional ICG fluorescence development technology is that the ICG fluorescence is invisible, and all the images must be seen through the screen, so that the operator must look at the screen to know where the fluorescent light is, and there is no way to simultaneously See the surgical area; in addition, because the fluorescent image is black and white, the background light is very weak, so the surgeon must compare the control screen with the surgical position to know the approximate fluorescent position. This is also the current US The biggest inconvenience of the existing fluorescent development system is that it is an indirect imaging system. In response, it has been proposed to use Google Glass's wearable infrared imaging device to allow physicians to see infrared light (Reference: Liu et al, Surgery, Volume 149, Number 5) in an attempt to solve the problem of looking up the screen. However, this design does not take into account the operator's operability and comfort.

At present, the fluorescent developing system on the market is also trying to improve the above-mentioned inconvenience. For example, the HyperEye Medical System (HEMS) is equipped with a color camera and an infrared camera at the same time. The image fusion (Fusion) technology allows the surgeon to see the colored background and the fluorescent area on the screen (Reference) :InTech Article by Masaki Yamamoto et al. However, this still can't get rid of the indirect image operation.

In addition, the Republic of China Patent No. I283593 proposes a method for automatically controlling the laser light displacement of a laser skin treatment apparatus, which does not disclose a method for treating when the distance between the skin area and the camera is changed; and the Republic of China patent No. M289636, a device for controlling laser light displacement of a laser skin treatment device is proposed. The patent application scope of the patent clearly discloses that the composition comprises an initial mirror and a final mirror and an adjustment mirror group interposed therebetween. The mirror set consists of two lenses for angle adjustment, which shows complex structural adjustments.

Since the traditional surgeon relies on the eyes to operate his own surgery, this is also the most intuitive way of operation. However, the prior art has not been able to provide visualization of the fluorescent light. Therefore, the general practitioner cannot meet the point that the user can directly see the light in the surgical field when actually using it to facilitate the operation of the operation.

The main object of the present invention is to overcome the above problems encountered in the prior art and to provide a direct observation that allows the operator to get rid of the screen without having to confirm the position of the surgical site and the screen display position, and can observe it in an instant manner. Fluorescent dye fluid moves, by instantly projecting visible light in the fluorescent position, allowing the surgeon to observe the area projected by the visible light spot and observe that the location of the fluorescent dye can be self-corrected and visible invisible. Fluorescent method.

A secondary object of the present invention is to provide a method of self-correcting the projection scale in conjunction with a zoom lens.

Another object of the present invention is to provide a method for adjusting the angle of a visible light source only by passing through the initial reflection mirror and the final reflection mirror, without the need to adjust the mirror assembly.

For the above purposes, the present invention is a self-correcting method for indicating invisible fluorescence by visible light, comprising at least the following steps: (A) placing a calibration plate at a corrected initial position and a fluorescent The excitation light source illuminates the calibration plate; (B) uses an image sensor to capture the image with the optical imaging module contained therein, and moves the calibration plate to align its center position with the center of the image, and then input the image into an image analysis Selecting a reference point for calibration in the processor; (C) setting a spot position controller to a neutral position, and projecting a spot of visible light through an optical element to project a visible light spot on the calibration plate; and (D) utilizing The image sensor captures an image by using the optical imaging module, and inputs the image into the image analysis processor to obtain a visible light spot, and adjusts an initial position of the spot position controller by analyzing an error between the reference point and the visible spot. The error is reduced to a visible light spot, which can be completely aligned with the reference point on the calibration plate through the center position of the calibration plate, and the field of view of the optical imaging module is recorded. And the driving position setting of the spot position controller, the spot position controller outputs a standard circle, and the image sensor is captured and input into the image analyzing processor to record the position of the standard circle in the image. Complete the mapping relationship between visible light spot and image space.

In the above embodiment of the present invention, when the image of the optical imaging module is changed to reduce the image, the spot position controller outputs the standard circle, and the image sensor captures the standard circle at the image position, and The query space corrects all the data and compares it with the pre-corrected standard circle position to find the position with the smallest distance error from the standard circle. The internal difference method is used to find the mapping relationship between the visible light spot and the image space to obtain the driving setting of the corresponding spot position controller.

In the above embodiment of the present invention, the spatial correction system selects a plurality of visual field distances in the farthest and closest field of view of the optical imaging module, respectively performs the spot position controller correction, and records the standard of each position. Round position.

In the above embodiment of the present invention, the spot position controller comprises a point-like visible light source generator and the optical element, the optical element further comprising a horizontal direction scanner provided with a primary mirror and a final mirror a vertical direction scanner, wherein the point-shaped visible light source outputted by the point-shaped visible light source generator moves through the horizontal mirror to move left and right to the initial mirror, and the vertical direction scanner moves up and down to adjust the mirror to adjust The visible light spot is projected onto the correction plate.

In the above embodiment of the invention, the optical imaging module is a zoom lens.

In the above embodiment of the present invention, the calibration plate includes nine calibration reference points, and the visible light spot of the spot position controller can be fully aligned with the nine reference points by adjusting the driving position of the spot position controller.

S101~s108‧‧‧Steps

1‧‧‧ calibration board

11‧‧‧ Reference point

3‧‧‧Image sensor

31‧‧‧Optical imaging module

4‧‧‧Image Analysis Processor

5‧‧‧ spot position controller

51‧‧‧ spot visible light source generator

511‧‧‧ spot visible light source

52‧‧‧Optical components

521‧‧‧Horizontal scanner

5211‧‧‧first mirror

522‧‧‧Vertical direction scanner

5221‧‧‧End mirror

53‧‧‧ visible light spot

54‧‧‧ standard round

6‧‧‧Digital imagery

61‧‧‧Fluorescent area

62‧‧‧Scanning area

Figure 1 is a schematic diagram of the calibration process of the present invention.

Figure 2 is a schematic diagram of the correction architecture of the present invention.

Fig. 3 is a schematic view showing the structure of the spot position controller of the present invention.

Fig. 4 is a schematic view showing the visible light spot of the present invention projected on a correction plate.

Figure 5 is a schematic representation of the reference point of the visible light spot on the calibration plate of the present invention. Figure.

Figure 6 is a schematic diagram of the scanning sequence of the present invention.

Figure 7 is a schematic diagram showing the relationship between the spot position controller, the visible light spot state and the image of the present invention.

Please refer to FIG. 1 to FIG. 7 , which are respectively a schematic diagram of the calibration process of the present invention, a schematic diagram of the calibration architecture of the present invention, a schematic diagram of the structure of the spot position controller of the present invention, and a visible light spot of the present invention projected on the calibration plate. The schematic diagram, the visible light spot of the present invention is completely aligned with the reference point on the calibration plate, the scanning sequence diagram of the present invention, and the spot position controller of the present invention, the relationship between the visible spot state and the image relationship. As shown in the figure: the present invention is a self-correcting method for indicating invisible fluorescence in visible light, comprising at least a system calibration program and a dynamic self-correction program, wherein the system calibration program displays a spatial correspondence of the camera for correcting images and visible light. The position and size problem of the spot projection is as follows: First, in step s101, a correction plate 1 is placed at a correction initial position, and a fluorescent excitation light source 2 is illuminated by the calibration plate 1.

In steps s102 and s103, an image sensor 3 is used to capture an image of the optical imaging module 31 therein, and the calibration plate 1 is moved to align its center position with the center of the image, and then the image is input into an image analysis. A reference point 11 for calibration is selected in the processor 4.

In step s104, a spot position controller 5 is set at a neutral position (x=0, y=0), and the spot position controller 5 includes a point-like visible light source generator 51 and an optical element 52 (such as the third As shown in the figure), the optical element 52 further comprises A horizontal direction scanner 521 having an initial mirror 5211 and a vertical direction scanner 522 provided with a final mirror 5221, and the point visible light source generator 51 outputs a point visible light source 511 through which the optical element is transmitted. 52 projecting a visible light spot 53 on the correction plate 1 (as shown in FIG. 4); wherein the point visible light source 511 is moved left and right by the horizontal direction scanner 521 to the initial mirror 5211, and the vertical direction The mirror 5221 that moves up and down the scanner 522 adjusts the position at which the visible light spot 53 is projected onto the correction plate 1.

In step s105, the image sensor 3 captures the image by the optical imaging module 31, and inputs the image into the image analysis processor 4 to obtain the visible light spot 53 and analyzes the error between the reference point 11 and the visible light spot 53. Adjusting the initial setting of the spot position controller 5 to reduce the error to the visible light spot 53. The reference point 11 on the calibration plate 1 can be completely aligned by the center position of the calibration plate 1, and the optical imaging module 31 can be recorded. The field of view and the driving setting of the spot position controller 5, in step s106 to s108, the spot position controller 5 projects a standard circle 54 and is captured by the image sensor 3 and input to the image analysis processing. In the device 4, the position of the standard circle 54 in the image is recorded, and the relationship between the visible light spot and the image space is completed. The calibration plate 1 includes nine calibration reference points 11 for adjusting the driving position of the spot position controller 5 so that the projected visible light spot 53 can be completely aligned with the nine reference points 11 (as shown in FIG. 5). Show). As such, a new method of self-correcting invisible fluorescent light in visible light is constructed by the above disclosed process.

The optical imaging module 31 is a zoom lens, and can be used with a zoom lens to perform a dynamic self-correction program, including a space. Correction and self-correction.

The spatial correction system selects a plurality of visual field distances in the farthest and closest field of view of the optical imaging module, respectively performs the spot position controller correction, and records the standard circle position of each position.

The self-correction system firstly drives the spot position controller to output a standard circle when the user adjusts the lens magnification, that is, changes the field of view of the optical imaging module to reduce the image range, and the image sensor captures the standard circle image. Position, and query all the spatial correction data, compare with the pre-corrected standard circle position, find the position with the smallest distance error from the standard circle, and use the internal difference method to find the mapping relationship between the visible light spot and the image space to obtain the corresponding The drive position setting of the spot position controller.

When used, the actual scanning mode of operation is as shown in Figures 6 and 7. The image sensor captures a digital image 6 containing a fluorescent region 61 of the fluorescent reaction, the fluorescent region containing no Consecutive independent blocks.

The gray area value of the fluorescent area 61 is larger than the area where the background is not fluorescent. The area where the gray level value is greater than or equal to the threshold value set by the user is selected, the area is marked as the fluorescent area 61, and the gray level value is set to 255, other grayscale values are less than the threshold value are marked as the background area, and the grayscale value is set to 0, in order to avoid too small fluorescent areas or noise, so the area will be used as the screening condition for the fluorescent area. Excluding the small fluorescent area, and finally converting all the fluorescent areas into a minimum rectangular range that can surround the fluorescent area as the scanning area 62, and the spot position controller only needs to repeat the numerical sequence as shown in FIG. Scan this area to save scan time and increase the frequency of scan updates.

The threshold value mentioned above is determined by the physician according to experience or the experience of the physician. Established in a database, the system analyzes a recommended value and then fine-tunes by the physician; the screening condition is based on the size of the fluorescent area, and the value varies from the size of the observed area, the organ type, and the physician. The value is usually determined by the physician, depending on how many fine fluorescent areas are seen.

The invention enables the surgeon to get rid of the screen for direct observation, and does not need to confirm the position of the surgical site and the marked portion of the screen repeatedly, and can observe the movement of the fluorescent dye fluid in an instant manner, by instantly projecting visible light in the fluorescent position, The operator can observe the position of the fluorescent dye by observing the area projected by the visible light spot, so as to solve the problem that the current fluorescent development technology must rely on the screen to indirectly see the position of the fluorescent image. Therefore, the present invention can effectively solve the instant projection of invisible light, and establish a method for self-correcting the projection scale with the zoom lens. When the patient moves or the fluorescent area changes, the image sensor has an auto focus to change the photography. The mechanism of automatically adjusting the visible light marking area when the area or the size is used, and the invention only needs to adjust the angle of the visible light source through the primary reflection mirror and the final reflection mirror, and the utility model can reduce the cost without adjusting the mirror group.

In summary, the present invention is a self-correcting method for indicating invisible fluorescence by visible light, which can effectively improve various disadvantages of the conventional use, and can enable the operator to get rid of the screen for direct observation without marking the surgical site and the screen. The position is confirmed by repeated alignment, and the movement of the fluorescent dye fluid can be observed in an instant manner. By projecting the visible light in the fluorescent position, the operator can observe the fluorescent dye by observing the area projected by the visible light spot. Position, in order to solve the current fluorescence development technology must rely on the screen to indirectly see the location of the fluorescent image, so that the invention can be more progressive, more practical, more in line with the user's needs, indeed meet the invention patent For the requirements of the application, the patent application is filed according to law.

However, the above is only the preferred embodiment of the present invention, and the scope of the present invention is not limited thereto; therefore, the simple equivalent changes and modifications made in accordance with the scope of the present invention and the contents of the invention are modified. All should remain within the scope of the invention patent.

S101~s108‧‧‧Steps

Claims (6)

[Item 1] A method for self-correcting to indicate invisible fluorescence in visible light, comprising at least the following steps:
(A) placing a calibration plate at a corrected initial position and illuminating the calibration plate with a fluorescent excitation source;
(B) using an image sensor to capture an image with its included optical imaging module, and moving the calibration plate to align its center position with the center of the image, and then input the image into an image analysis processor for correction. Reference point
(C) setting a spot position controller to a neutral position, and projecting a spot of visible light through an optical element onto the calibration plate; and (D) utilizing the image sensor to image the optical imaging mode The group captures the image, and inputs the image into the image analysis processor to obtain a visible light spot. By analyzing the error between the reference point and the visible spot, the initial setting of the spot position controller is adjusted to reduce the error to the visible spot through the calibration plate. The center position is completely aligned with the reference point on the calibration plate, and the field of view of the optical imaging module and the driving setting of the spot position controller are recorded, and the spot position controller outputs a standard circle through which the image is sensed. The device captures and inputs to the image analysis processor to record the position of the standard circle in the image, and completes the mapping relationship between the visible light spot and the image space. [Item 2] The method for self-correcting the invisible fluorescent light by visible light according to the first aspect of the patent application scope, wherein when the image field of the optical imaging module is changed to reduce the image, the spot position controller output standard circle is first driven. The image sensor captures the standard circle at the position of the image, and queries the space to correct all the data, compares it with the pre-corrected standard circle position, finds the position with the smallest distance error from the standard circle, and finds the position with the smallest distance error from the standard circle. The visible light spot is mapped to the image space to obtain the driving setting of the corresponding spot position controller. [Item 3] The method for self-correcting visible light indicating invisible fluorescence according to item 2 of the patent application scope, wherein the spatial correction system selects a plurality of visual field distances in the farthest and closest field of view of the optical imaging module The spot position controller correction is performed separately, and the standard circle position of each position is recorded. [Item 4] The method for self-correcting the invisible fluorescent light by visible light according to the first aspect of the patent application, wherein the spot position controller comprises a spot-shaped visible light source generator and the optical element, the optical element further comprising a a horizontal direction scanner of the initial mirror, and a vertical direction scanner provided with the final mirror, and the spotted visible light source output by the point visible light source generator is moved to the left and right by the horizontal direction scanner And the mirror that moves up and down with the vertical direction scanner to adjust the position where the visible light spot is projected on the correction plate. [Item 5] The method of self-correcting the invisible fluorescent light by visible light according to the first aspect of the patent application scope, wherein the optical imaging module is a zoom lens. [Item 6] The method for self-correcting the invisible fluorescent light by visible light according to the first aspect of the patent application, wherein the calibration plate includes nine calibration reference points, and by adjusting the driving setting of the spot position controller, The visible light spot that it projects can be perfectly aligned with the nine reference points.