JP5877104B2 - Image processing system - Google Patents

Description

  The present invention relates to an image processing system for displaying a moving image and a still image side by side in a single screen.

  The electronic endoscope system has a function of capturing a moving image captured by the electronic endoscope. A specific configuration of this type of electronic endoscope system is described in Patent Document 1, for example.

  The electronic endoscope system described in Patent Document 1 normally displays a moving image as a parent screen, and when the freeze switch is operated, the parent screen is switched to a still image captured when the switch is operated and displayed. At the same time, the moving image that has been displayed as the parent screen is displayed as a child screen. Thereby, the surgeon can observe a moving image and a still image simultaneously in one screen.

JP 2002-272679 A

  By the way, when the moving image is displayed on the child screen, the display size is reduced as compared with the parent screen, so that the absolute movement (movement amount) of the lesioned part or the like in the moving image is reduced. For example, since the movement of a vocal cord or a part close to the heart is relatively large, it cannot be said that it is difficult to distinguish between a moving image that has been reduced and reduced in movement and a still image. On the other hand, since the large intestine, stomach, and the like are not actively moving parts, the moving image displayed on the child screen may appear as if it is a still image. In this case, it is difficult to discriminate between a moving image and a still image, and there is a possibility that an operator may mistake the moving image and the still image. For this reason, for example, problems such as continuation of in-body cavity observation and treatment may occur while the surgeon considers a still image as a moving image.

  The present invention has been made in view of the above circumstances. An object of the present invention is to provide an image processing system suitable for preventing a surgeon from mixing a moving image with a still image.

  An image processing system according to an embodiment of the present invention that solves the above-described problems is generated by a first operation unit, a light emitting unit that can cause the first operation unit to emit light, and a predetermined captured image generation unit. Capture means capable of capturing a moving image, image composition means for combining the moving image transferred from the captured image generation means and the still image captured by the capture means in a single screen so that they can be displayed simultaneously. When the operation of the first operation unit is performed, the capture unit captures a moving image, the light emission unit causes the first operation unit to emit light in a predetermined color, and the image synthesis unit performs synthesis after synthesis. An identifier of a predetermined color is superimposed on a still image arranged in the image.

  The operator can identify a still image in the composite image by visually observing an identifier having the same color as the light emission color of the first operation unit, so that the moving image and the still image are not mistaken.

  The image processing system according to an embodiment of the present invention may further include a second operation unit for designating an image to be a parent screen. In this case, the image synthesizing unit is configured such that one of the moving image and the still image specified by the operation of the second operation unit is a parent screen, and the other image is a child screen having a display size smaller than the parent screen. Only when the moving image and the still image are combined and the still image is designated as the parent screen, an identifier of a predetermined color is superimposed on the still image arranged in the combined image.

  The identifier is superimposed on a region surrounding a still image, for example. A specific example of the identifier is a mask displayed in the same color as the emission color of the first operation unit.

  In addition, the image composition unit can superimpose information on the elapsed time from the start of the period when the still image is designated as the parent screen on the still image arranged in the synthesized image.

  In addition, an image processing system according to an embodiment of the present invention includes an electronic endoscope capable of capturing a captured image and a predetermined captured image generation unit, and captures a captured image input from the electronic endoscope. It is good also as a system provided with the signal processing apparatus which processes and produces | generates a moving image. In this case, the first operation unit is provided in at least one of the electronic endoscope and the signal processing device, for example.

  According to an embodiment of the present invention, an image processing system suitable for preventing a surgeon from mixing a moving image with a still image is provided.

It is a block diagram which shows the structure of the endoscope system in one Embodiment of this invention. It is a schematic diagram which shows the operation part of the endoscope in one Embodiment of this invention. It is a figure which shows the example of a display screen displayed on the display screen of the monitor in one Embodiment of this invention. It is a figure which shows the example of a display screen displayed on the display screen of the monitor in one Embodiment of this invention.

  Hereinafter, an electronic endoscope system according to an embodiment of the present invention will be described with reference to the drawings.

  FIG. 1 is a block diagram illustrating a configuration of an electronic endoscope system 100 according to the present embodiment. As shown in FIG. 1, the electronic endoscope system 100 includes an electronic scope 1 and a video processor 2. The electronic scope 1 and the video processor 2 are optically and electrically connected.

  The video processor 2 includes a light source unit 41. The light source unit 41 generates illumination light by a surface sequential method, and includes a light source, a condensing lens, a color filter, a color filter rotation control circuit, a light amount diaphragm, a diaphragm control circuit, and the like. The light source is a white light source, such as a halogen lamp, a xenon lamp, or a white LED. White light emitted from the light source is collected by the condenser lens and passes through the color filter. Under the control of the color filter rotation control circuit, the color filter rotates in synchronization with a timing pulse or a vertical synchronization signal when writing the video signal to the frame memory, and the white light incident from the condenser lens is red (R). , Green (G), blue (B) light sequentially. The illumination light color-separated by the color filter is reduced to an appropriate light amount by a light amount diaphragm operated under the control of the aperture control circuit, and is incident on the incident end of the light guide 21 formed of an optical fiber bundle. Note that the imaging method may be a simultaneous imaging method instead of the frame sequential method.

  Illumination light supplied from the light source unit 41 propagates through the inside of the light guide 21 and is emitted from the emission end of the light guide 21 disposed in the distal end portion of the flexible tube of the electronic scope 1. An objective optical system 11 and a light distribution optical system 12 are arranged in the distal end portion of the flexible tube. An imaging element 13 (such as a CCD image sensor or a CMOS image sensor) is disposed at the subsequent stage of the objective optical system 11. In addition, a signal line in which a plurality of transmission paths are bundled is drawn from the image sensor 13 to a connection portion with the video processor 2. Although the objective optical system 11 is shown as a single piece in FIG. 1, it is actually composed of a plurality of pieces. Furthermore, this embodiment also supports optical zoom.

  The illumination light emitted from the light guide 21 reaches the target part via the light distribution optical system 12. The illumination light is reflected by the target site and forms an image on the imaging surface of the imaging device 13 via the objective optical system 11. The illumination light received by the image sensor 13 is photoelectrically converted and then transmitted to the video signal processing circuit 30 of the video processor 2.

  The surgeon can change the magnification of the electronic zoom by operating a switch of the operation unit 22 of the electronic scope 1 or a switch for electronic zoom provided on the operation panel 42 of the video processor 2. The CPU 39 of the video processor 2 receives an operation signal for the electronic zoom switch, and transmits an electronic zoom control signal to the image processing circuit 34 based on the operation signal. The image processing circuit 34 performs electronic zoom processing on the video signal based on the received control signal.

  Further, in the electronic endoscope system 100, in addition to the electronic zoom which is a zoom function mainly performed on the video processor 2 side, an optical zoom which is a zoom function mainly performed on the electronic scope 1 side is also executed. be able to. In the operation unit 22 of the electronic scope 1, a motor 14 and a rotary encoder 15 that detects the number of rotations of the motor 14 are provided as optical zoom means. The motor 14 is connected to the gear 10 via a torque wire. The gear 10 uses the driving force of the motor 14 to move the movable lens in the objective optical system 11 in the optical axis direction, and changes the focal length with the imaging surface of the imaging element 13 without changing the in-focus position. . Information such as the input to the motor 14 and the rotational speed of the motor 14 detected by the rotary encoder 15 is fed back to the motor control circuit 16.

  For example, in the case of a PWM (Pulse Width Modulation) drive type that modulates by changing the duty ratio of the pulse wave, the PWM frequency and the duty ratio are fed back to the motor control circuit 16. Since the rotational speed of the motor 14 corresponds to the amount of movement of the movable lens in the objective optical system 11, the optical zoom is performed by feedback control based on information such as the rotational speed of the motor 14 detected by the rotary encoder 15. Can be adjusted to a desired magnification. The surgeon can change the magnification of the optical zoom by operating the switch for the operation unit 22 or the switch for optical zoom provided on the operation panel 42 or the like. The CPU 18 of the electronic scope 1 receives an operation signal for the optical zoom switch, and transmits a control signal to the motor control circuit 16 based on the received operation signal. The motor control circuit 16 drives the motor 14 by the number of rotations corresponding to the desired optical zoom magnification in accordance with the control signal received from the CPU 18. The drive of the motor 14 is transmitted to the gear 10, and the gear 10 moves the movable lens in the objective optical system 11 in the optical axis direction by the number of rotations of the motor 14.

  The CPU 18 is connected to the memory 17 and the timing circuit 19 in addition to the motor control circuit 16. The memory 17 stores information related to the electronic scope 1 such as functions that can be executed by the electronic scope 1, the number of pixels of the image sensor 13, and a frame rate. Therefore, the CPU 18 reads information necessary for image processing from the memory 17 and designates a setting value necessary for each block. When the electronic scope 1 is connected to the video processor 2, the CPU 18 includes information stored in the memory 17 such as identification information of the electronic scope 1 and property information specific to the image sensor 13 in the video processor 2. Information necessary for processing is transmitted to the CPU 39.

  When receiving information related to the electronic scope 1 from the CPU 18, the CPU 39 instructs the timing circuit 37 to change the processing timing of each block in the video processor 2 based on the received information, or the image processing circuit 34. The RGB gain value is adjusted, and control is performed so that the color optimized for the currently used electronic endoscope system is reproduced. Further, the CPU 18 receives information related to image processing in the video processor 2 stored in the memory 44 from the CPU 39, and controls the drive circuit 20 via the timing circuit 19 based on the received information to control the image sensor 13. To drive.

  The video signal processing circuit 30 receives the video signal output from the image sensor 13, and performs various signal processing such as clamping, knee, γ correction, interpolation processing, and AGC (Auto Gain Control) on the received signal. . The video signal processing circuit 30 converts the processed signal into a digital signal sequence, performs RGB color conversion, and outputs R, G, and B signals to the R memory 31, G memory 32, and B memory 33, respectively. To do. The R memory 31, the G memory 32, and the B memory 33 buffer the input video signal in units of frames based on the timing pulse output from the timing circuit 37. The R memory 31, the G memory 32, and the B memory 33 convert the signal into a monitor display format such as NTSC (National Television System Committee) or PAL (Phase Alternation Line) under the control of the timing circuit 37, and perform image processing. Output to the circuit 34.

  The image processing circuit 34 enhances blood vessels for enhancement, brightness correction, noise reduction, mask processing, and the like for R, G, and B video signals received from the R memory 31, G memory 32, and B memory 33. Are subjected to D / A conversion and output to the adder 35. The character generation circuit 38 outputs various character information such as a patient name, a practitioner's findings and comments, which are input from the keyboard 3 and the operation panel 42, to the adding unit 35. The adder 35 adds a character information signal to the video signal input from the image processing circuit 34 so that the character information input from the character generation circuit 38 is displayed at a predetermined position in the screen. The video signal to which the character information is added is output to the amplifier 36, amplified, and then output to the monitor 8, the printer 4, the PC 5, and the VTR 6 that are peripheral devices.

  2A and 2B are schematic views of the operation unit 22 of the electronic scope 1. FIG. In the present embodiment, four switches 51, 52, 54, and 55 are provided in the operation unit 22, and different functions are assigned to the switches. In addition to the functions of the electronic scope 1 and the video processor 2, the switches 51, 52, 54, 55 measure the printer 4, PC 5, VTR 6, electrocardiogram, pulse, blood pressure, etc. connected to the video processor 2. Functions of peripheral devices such as the device 7 and the monitor 8 can also be executed. For example, a still image generated in the video processor 2 is transmitted to the printer 4 for printing, a still image or a moving image generated in the video processor 2 is stored in the PC 5 or VTR 6, and output from the measuring device 7. The measurement result can be displayed on the monitor 8.

  The operation unit 22 is provided with a right / left bending angle knob 57 and an up / down bending angle knob 58 for turning the distal end of the flexible tube of the electronic scope 1 in the vertical and horizontal directions. When the surgeon rotates the right and left bending angle knob 57 and the left and right bending angle knob 58, the bending range of the distal end portion of the flexible tube of the electronic scope 1 can be controlled to change the imaging range. The left and right bending angle knob 57 and the left and right bending angle knob 58 are respectively provided with lock levers 56 and 59 for fixing the bending state of the distal end portion of the flexible tube by each angle knob.

  The operation unit 22 is provided with an air / water supply switch 53 and a suction switch 60. The air / water supply switch 53 is used to expand the patient's lumen by supplying air from the air / water supply port provided at the distal end of the flexible tube of the electronic scope 1, and to secure the field of view, or to detect the surface of the objective lens due to body fluid or bleeding. When the observation performance of the electronic scope 1 is deteriorated due to contamination, water is sprayed to remove the dirt on the lens surface, and air is blown to blow off water droplets on the objective lens surface to restore the field of view or treatment object This method is used when air is blown onto the surface of the object to obtain a clear observation image of the treatment object. The suction switch 60 is used when air is sucked in order to adjust the amount of air in the body cavity of the patient, or body fluid or blood is sucked out.

  One end of the treatment instrument channel (treatment instrument channel port 61) is provided at the distal end side of the operation unit 22, and the other end of the treatment instrument channel (the other treatment instrument channel opening) is provided at the distal end of the flexible tube. Is provided. Various treatment tools such as a biological forceps, a cell collection brush, a foreign substance removal forceps, a cleaning pipe, and an injection needle are inserted into the treatment tool channel port 61 as necessary. The inserted treatment tool is guided to a treatment target site such as a lesioned portion through the treatment tool channel and the other treatment tool channel port.

  Next, light emission of the switches 51, 52, 54, and 55 of the operation unit 22 will be described. The switches 51, 52, 54, and 55 can emit light in accordance with light emission control of LEDs or the like by the light emitting circuit 23. The functions that can be assigned to the switches 51, 52, 54, and 55 and the emission colors are associated with each other at a ratio of 1: 1. Information on the correspondence relationship between the function and the emission color is stored in the memory 17. For convenience, the connection between the light emitting circuit 23 and other components is omitted.

  When the electronic scope 1 and the video processor 2 are connected, the CPU 18 of the electronic scope 1 communicates with the CPU 39 of the video processor 2. The CPU 39 reads the identification information of the video processor 2 stored in the memory 44 and transfers it to the CPU 18. When a peripheral device is connected to the video processor 2, the CPU 39 also transfers identification information of the connected peripheral device to the CPU 18. Based on the received identification information, the CPU 18 determines a function that can be executed between the video processor 2 and peripheral devices, and the correspondence between the determination result and the function stored in the memory 17 and the emission color. Based on this, the function is assigned to each switch of the operation unit 22 and the emission color is designated.

  In the present embodiment, each of the switches 51, 52, 54, and 55 is assigned an enhancement function for performing freeze, printing by a printer, recording by a VTR, and blood vessel enhancement in an image. Further, red, green, blue, and yellow light emission colors are designated as the light emission colors for the switches 51, 52, 54, and 55, respectively.

  In addition, each switch of the operation panel 42 can emit light with a different emission color for each function by a light emitting circuit 43 (for example, LED). The CPU 18 controls the light emission circuit 43 in cooperation with the CPU 39 and designates red, green, blue, and yellow as emission colors for the freeze switch, print switch, recording switch, and enhancement switch in the operation panel 42. That is, the light emission colors of the switch of the operation unit 22 that executes the same function and the switch of the operation panel 42 match.

  For example, when the switch 51 of the operation unit 22 is pressed, the CPU 18 causes the switch 51 to emit red light, and transmits a capture signal to the still image processing circuit 34 </ b> C mounted on the image processing circuit 34 in cooperation with the CPU 39. . When the freeze switch in the operation panel 42 is pressed, the CPU 39 causes the freeze switch to emit light in red and transmits a capture signal to the still image processing circuit 34C. The switch that emits red light may be only the switch pressed by the operator, or may be both the switch 51 and the freeze switch.

  The still image processing circuit 34C captures the image generated by the image processing circuit 34 when the capture signal is received, and captures until a freeze release operation (for example, the switch 51 or the freeze switch is pressed again). The processed image is output to the adding unit 35. That is, from the image processing circuit 34, a moving image generated using video signals sequentially input from the preceding circuit and a still image captured by the still image processing circuit 34C are output.

  Further, the image processing circuit 34 adds a child screen designation signal for designating a child screen to the moving image signal and outputs the signal to the adding unit 35. On the other hand, the still image processing circuit 34C superimposes an identifier of the same color as the light emission color (red) of the switch 51 and the freeze switch on the still image signal and designates the parent screen for the still image signal. A designation signal is added and output to the adder 35. In this embodiment, a mask Mr shown in FIG. 3A described later is employed as an identifier so as not to give unnecessary discomfort to the surgeon who has used a conventional endoscope system. Note that the mask in the present embodiment surrounds an image and does not substantially cover part of the image. The adding unit 35 performs image composition processing so that the still image specified as the parent screen is arranged over the entire display screen area, and the moving image specified as the child screen is arranged at one corner of the four corners of the display screen area. Do. FIG. 3A illustrates a composite image displayed on the display screen of the monitor 8.

  As shown in FIG. 3A, on the display screen of the monitor 8, a still image is displayed on the entire display screen as a parent screen, and a moving image is displayed on the upper right corner of the display screen as a child screen. A general black mask Mk is superimposed on the moving image, while a red mask Mr which is the same as the emission color of the switch 51 and the freeze switch is superimposed on the still image. The surgeon can identify a still image corresponding to the freeze operation by visually observing the red mask Mr having the same emission color as that of the switch 51 or the freeze switch. Therefore, there is no possibility that the surgeon mistakes a moving image and a still image.

  The operation unit 22 is provided with a parent screen switching switch 62. When the parent screen switching switch 62 is pressed, the CPU 18 cooperates with the CPU 39 to transmit a switching signal to the image processing circuit 34 and the still image processing circuit 34C. When receiving the switching signal, the image processing circuit 34 and the still image processing circuit 34C switch the screen designation signal to be added to the image. Specifically, when a child screen designation signal is added, switching to the parent screen designation signal is performed, and when a parent screen designation signal is added, switching to the child screen designation signal is performed. Therefore, for example, when the parent screen switching switch 62 is pressed while the screen shown in FIG. 3A is displayed on the display screen of the monitor 8, the display screen of the monitor 8 displays the moving image as the parent screen. In addition to being displayed on the entire display screen, the still image is switched to a screen displayed in the upper right corner of the display screen as a sub screen (see the display screen example in FIG. 3B).

  Here, since the moving image becomes the parent screen, the absolute movement (movement amount) of the lesioned part or the like in the moving image becomes larger than that in the child screen. In this case, since there is substantially no possibility that the surgeon mistakes the moving image and the still image, the mask for the still image is also changed to general black (mask Mk). In order to more reliably prevent the moving image and the still image from being mixed, in the display screen example of FIG. 3B, the red mask Mr having the same emission color as that of the switch 51 and the freeze switch is applied to the still image. Superposition may be continued. A switch having the same function as the parent screen switching switch 62 may also be provided in the operation panel 42.

  By the way, it is not desirable to continue observing a still image for a long time during the treatment because a burden is imposed on the patient. Therefore, in this embodiment, a time count function is added. FIG. 4 is a diagram illustrating a display screen example of the monitor 8 during execution of the time count function.

  The operation unit 22 is provided with a time count switch 63 for turning on / off the time count mode. When the time count mode is turned on by the time count switch 63, the CPU 18 cooperates with the CPU 39 to display information on the elapsed time from the start of the period during the period when the still image is designated as the parent screen. Send to. The character generation circuit 38 outputs the transmitted character information of the elapsed time to the adding unit 35. The adding unit 35 superimposes the character information of the elapsed time on the still image, whereby the screen shown in FIG. 4 is displayed on the display screen of the monitor 8. The operator can grasp how long the still image is observed by visually recognizing the elapsed time in the display screen. For example, the surgeon can proceed with the operation while considering time distribution and the like.

  The above is the description of the exemplary embodiments of the present invention. Embodiments of the present invention are not limited to those described above, and various modifications are possible within the scope of the technical idea of the present invention. For example, the embodiment of the present application also includes contents appropriately combined with embodiments explicitly shown in the specification or obvious embodiments. For example, after the electronic scope 1 and the video processor 2 are connected, the function and emission color assigned to each switch of the operation unit 22 of the electronic scope 1 can be changed. The surgeon can use the operation panel 42 of the video processor 2 or the keyboard 3 connected to the video processor 2 as input means, and can change the function and emission color assigned to each switch of the operation unit 22.

  Further, the CPU 39 of the video processor 2 can store the still image captured by the still image processing circuit 34C in an external storage device such as the PC 5, the VTR 6, or a file system. The CPU 39 also reads out still images stored in these external storage devices, and uses the read still images as a parent screen or a child screen instead of the still images captured by the still image processing circuit 34C. 8 display screens. Also in this case, the mask Mr is superimposed on the still image in order to prevent the surgeon from mixing the moving image with the still image. That is, the still image on which the mask Mr is superimposed may be an image captured in the past.

  In the present embodiment, the mask Mr surrounding the image is exemplified as the identifier (see FIGS. 3 and 4), but the form of the mask is not limited to this. For example, the mask may be in a form that does not completely surround the image (for example, lines superimposed on the upper and lower ends of the image).

1 Electronic scope 2 Video processor 3 Keyboard 4 Printer 5 PC
6 VTR
7 Measuring device 8 Monitor 18, 39 CPU
22 Operation Units 23 and 43 Light Emitting Circuit 34C Still Image Processing Circuit 42 Operation Panels 51, 52, 54, and 55 Switch 100 Electronic Endoscope System

Claims (6)

A first operation unit;
A light emitting means capable of causing the first operation unit to emit light;
Capture means capable of capturing a moving image generated by a predetermined captured image generation means;
Image composition means for combining the moving image transferred from the captured image generation means and the still image captured by the capture means in a single screen so that they can be displayed simultaneously;
With
When the operation of the first operation unit is performed,
The capture means captures the moving image,
The light emitting means causes the first operation unit to emit light in a predetermined color,
The image processing system, wherein the image synthesizing unit superimposes the identifier of the predetermined color on a still image arranged in the synthesized image after synthesis. A second operation unit for designating an image to be a main screen;
The image composition means includes
One of the moving image and the still image specified by the operation of the second operation unit is a parent screen, and the other image is a child screen having a display size smaller than the parent screen. Combining the still image, and
The identifier of the predetermined color is superimposed on the still image arranged in the composite image only when the still image is designated as a parent screen. Image processing system.   The image processing system according to claim 1, wherein the identifier is superimposed on a region surrounding the still image.   The image processing system according to any one of claims 1 to 3, wherein the identifier is a mask displayed in the same color as the light emission color of the first operation unit.   The image synthesizing unit superimposes information on an elapsed time from the start of the period for which the still image is designated as a parent screen on a still image arranged in the synthesized image. The image processing system according to claim 3 or claim 4 quoting claim 2. An electronic endoscope for acquiring a captured image;
A signal processing device that has the predetermined captured image generation means, generates a moving image by processing a captured image input from the electronic endoscope;
With
The image processing system according to any one of claims 1 to 5, wherein the first operation unit is provided in at least one of the electronic endoscope and the signal processing device. .

Images