KR20210122472A - Method, apparatus and system for enhancing image quality of endoscope - Google Patents

Abstract

The present invention relates to a method, apparatus and system for improving the quality of an endoscopic image, and more particularly, by identifying factors impeding the quality of an image captured by the endoscope and controlling the endoscope to improve the quality of the image, The present invention relates to an endoscopic image quality improvement method, apparatus, and system capable of generating an improved quality endoscope image.
In the present invention, there is provided an endoscopic image quality improvement system for improving the quality of an image generated by using an endoscope, comprising: an image acquisition unit configured to acquire the image; an image quality evaluation unit configured to determine a type of a first image quality inhibiting element existing in a first region of the image and a location of the first image quality inhibiting element in the image; and a control unit controlling the image quality evaluation unit to respond to the first image quality impeding factor identified by the image quality evaluation unit.

Description

Method, apparatus and system for improving endoscopic image quality

The present invention relates to a method, apparatus and system for improving the quality of an endoscopic image, and more particularly, by identifying factors impeding the quality of an image captured by the endoscope and controlling the endoscope to improve the quality of the image, The present invention relates to an endoscopic image quality improvement method, apparatus, and system capable of generating an improved quality endoscope image.

Various types of endoscopes are used to diagnose or treat various organs of a patient's stomach, small intestine, and large intestine.

However, in the image taken by the endoscope, a bright area where the light of the light source is specularly reflected or a dark area where the light of the light source is insufficient may occur, and also bubbles, debris ( Debris), bleeding (blood), etc., may cause a problem in which it is difficult to accurately identify some areas of the endoscopic image.

On the other hand, conventional techniques for adjusting the brightness of the endoscopic image by measuring the brightness of the endoscopic image and adjusting the light amount of the light source have been attempted. It was difficult to understand the cause and location of various image quality degradations occurring in a specific area of the image because it was limited to adjusting the brightness of the image. there was no

In addition, a technique for improving the image quality by applying a specific algorithm to the acquired endoscopic image is being attempted, but even in this case, some problems can be improved, and the solution to the image quality deterioration factors of various causes is still insufficient in many ways. did. More specifically, an image processing algorithm can improve a dark area photographed due to a lack of a light source, but it is difficult to solve a dark area that is too dark and saturated with an image processing algorithm alone.

Accordingly, there is a continuous need for a method to identify factors that impair endoscopic image quality due to various causes, which are difficult to solve due to brightness control of endoscopic images or application of image processing algorithms, and to generate improved quality endoscopic images. An appropriate solution for this has not yet been proposed.

Korean Patent Laid-Open Publication No. 10-2011-0094283 (2011.08.23)

The present invention has been devised to solve the above problems, and it identifies factors that impair endoscopic image quality due to various causes that are difficult to solve due to brightness control of an endoscopic image or application of an image processing algorithm, and generates an improved quality endoscopic image. An object of the present invention is to disclose a method, apparatus and system for improving endoscopic image quality.

Other detailed objects of the present invention will be clearly grasped and understood by experts or researchers in the technical field through the specific contents described below.

An endoscopic image quality improvement system according to an embodiment of the present invention for solving the above problems is an endoscopic image quality improvement system for improving the quality of an image generated using an endoscope, and an image acquisition unit for acquiring the image ; an image quality evaluation unit configured to determine a type of a first image quality inhibiting element existing in a first region of the image and a location of the first image quality inhibiting element in the image; and a control unit controlling the image quality evaluation unit to respond to the first image quality inhibiting factor.

In this case, the control unit may determine the type and corresponding action of a corresponding device for improving the first image quality inhibiting factor in consideration of the type and location information of the first image quality inhibiting factor.

Furthermore, when a bright part or a dark part occurs in the first area, the controller may change the direction of the light source or change the amount of light.

Also, when a bright part or a dark part occurs in the first area, the controller may change or add a type of light source.

In addition, when a bright or dark region occurs in the first region, the controller may change the direction or position of a probe, a distal tip, or a camera of the endoscope.

In addition, when bubbles, debris, or bleeding occurs in the first region, the control unit changes the direction or position of a probe, distal tip, or camera of the endoscope. can

In addition, when bubbles, debris, or bleeding occurs in the first region, the controller may control to eject the cleaning solution from the distal tip of the endoscope.

In addition, when an element obstructing the view exists in the same area in the first image at the first time point and the second image at the second time point in the image, the control unit ejects a cleaning solution to the camera lens of the endoscope to the lens. can be controlled to clean.

In addition, the image quality evaluator may calculate the type and location information of the first image quality inhibiting factor using a learning technique.

In addition, the image quality evaluation unit may divide the image and evaluate the image quality of some images, collect the image quality evaluations of the partial images, and derive an image quality evaluation result for the entire image.

In addition, an endoscopic image quality improvement method according to another aspect of the present invention provides an endoscopic image quality improvement method for improving the quality of an image generated using an endoscope, comprising: acquiring the image; an image quality evaluation step of identifying a type of a first image quality inhibiting element existing in a first region of the image and a location of the first image quality inhibiting element in the image; and a control step of controlling to respond to the first image quality inhibiting factor identified in the image quality evaluation step.

The method, apparatus and system for improving endoscopic image quality according to an embodiment of the present invention identify factors that inhibit endoscopic image quality due to various causes that are difficult to solve due to brightness control of an endoscopic image or application of an image processing algorithm, and provide improved quality. It is possible to create an image of the endoscope.

BRIEF DESCRIPTION OF THE DRAWINGS The accompanying drawings, which are included as a part of the detailed description to help the understanding of the present invention, provide embodiments of the present invention, and together with the detailed description, explain the technical spirit of the present invention.
1 is a block diagram of an endoscope system according to an embodiment of the present invention.
2 is a block diagram of an endoscopic image quality improvement system according to an embodiment of the present invention.
3 is a diagram illustrating a first image quality inhibiting element identified by the endoscopic image quality improvement system according to an embodiment of the present invention.
4 is a view for explaining a corresponding action for a first image quality inhibiting element in the endoscopic image quality improvement system according to an embodiment of the present invention.
5 is a flowchart of a method for improving endoscopic image quality according to an embodiment of the present invention.

The present invention can apply various transformations and can have various embodiments. Hereinafter, specific embodiments will be described in detail based on the accompanying drawings.

The following examples are provided to provide a comprehensive understanding of the methods, apparatus and/or systems described herein. However, this is merely an example and the present invention is not limited thereto.

In describing the embodiments of the present invention, if it is determined that the detailed description of the known technology related to the present invention may unnecessarily obscure the gist of the present invention, the detailed description thereof will be omitted. In addition, the terms to be described later are terms defined in consideration of functions in the present invention, which may vary according to intentions or customs of users and operators. Therefore, the definition should be made based on the content throughout this specification. The terminology used in the detailed description is for the purpose of describing embodiments of the present invention only, and should not be limiting in any way. Unless explicitly used otherwise, expressions in the singular include the meaning of the plural. In this description, expressions such as “comprising” or “comprising” are intended to indicate certain features, numbers, steps, acts, elements, some or a combination thereof, and one or more other than those described. It should not be construed to exclude the presence or possibility of other features, numbers, steps, acts, elements, or any part or combination thereof.

In addition, terms such as first and second may be used to describe various components, but the components are not limited by the terms, and the terms are for the purpose of distinguishing one component from other components. used only as

Hereinafter, exemplary embodiments of an endoscopic image quality improvement method, apparatus and system according to the present invention will be described in detail with reference to the accompanying drawings.

First, FIG. 1 illustrates a configuration diagram of an endoscope system 1 according to an embodiment of the present invention. As can be seen in FIG. 1, the endoscope system 1 according to an embodiment of the present invention controls the endoscope insertion unit 2 having a photographing means such as a camera, and the driving of the endoscope insertion unit 2, Furthermore, the endoscope controller 4 receives the image signal generated by using the photographing means such as a camera from the endoscope insertion unit 2 and provides it to users such as medical staff through the image display unit 5 such as a monitor. can

Although FIG. 1 is drawn in a specific endoscopy form for convenience, the contents of this patent are applicable to various endoscopy systems 1 such as flexible endoscopy and rigid endoscopy.

At this time, in the endoscope insertion unit 2 in the endoscope system 1 according to an embodiment of the present invention, not only an image capturing means such as a camera, but also a light source, a probe, and a terminal tip ( distal tip) and the like may be provided.

In addition, the endoscope control unit 4 may be implemented integrally with other devices such as the image display unit 5 or may be implemented separately as a separate device or system.

In addition, in the above description, the case where medical staff mainly use the endoscope for medical examination has been described as an example, but the present invention is not necessarily limited thereto. Applicable.

2 is a block diagram of an endoscopic image quality improvement system 100 according to an embodiment of the present invention. As described above, the endoscopic image quality improvement system 100 according to an embodiment of the present invention may be implemented by being integrated as a part of the endoscopic system 1 .

As can be seen in FIG. 2 , the endoscopic image quality improvement system 100 according to an embodiment of the present invention is an endoscopic image quality improvement system 100 for improving the quality of an image generated using an endoscope, An image acquisition unit 110 for acquiring an image, an image quality evaluation unit for identifying a type of a first image quality inhibiting element present in a first region of the image and a location of the first image quality inhibiting element in the image ( 120) and the control unit 130 for controlling the image quality evaluation unit 120 to respond to the first image quality inhibiting factor.

In this case, the control unit 130 determines and controls the type and corresponding action of a corresponding device for improving the first image quality inhibiting factor in consideration of the type and location information of the first image quality inhibiting factor, thereby controlling the endoscope. It is possible to identify factors that impair endoscope image quality due to various causes that are difficult to solve by adjusting the brightness of the image or applying an image processing algorithm, and generate an improved quality endoscope image.

Hereinafter, the endoscopic image quality improvement system 100 according to an embodiment of the present invention will be described in more detail by dividing it for each component.

First, the image acquisition unit 110 acquires an image generated by using a photographing means such as the camera.

More specifically, the image acquisition unit 110 may acquire the endoscopic image generated through image processing on the image signal generated by the photographing means such as the camera, etc., but the present invention is not necessarily limited thereto.

Next, the image quality evaluation unit 120 detects the type of the first image quality inhibiting element existing in the first region of the image and the location of the first image quality inhibiting element in the image.

More specifically, there may be various types of the first image quality inhibiting element, but image quality elements such as brightness, saturation, and contrast of an image may be inhibited, or elements that obstruct the view of an image may be included. , information about the location of the first area showing a degree of difficulty in obtaining an image may include information on the location of the first area or the location of elements that obstruct the view of the image.

As a more specific example, in the present invention, as the first image quality inhibiting factor, a light from a light source is typically reflected and a bright part with an excessive amount of light, a dark part with an insufficient amount of light can be mentioned, and also bubbles. ), debris, and blood may also be included in the type of the first image quality inhibitory factor.

More specifically, FIG. 3 exemplifies an endoscopic image in which a bright portion (FIG. 3(A)) where the light of the light source is reflected is present in the first region of the image.

As can be seen in FIG. 3 , when a highlight is generated in the first region of the image, the image quality evaluation unit 120 evaluates the image and the type of the first image quality inhibiting factor present in the image ( FIG. 3 ). In the image of 3, a bright part) is determined, and a position (region (A) in the image of FIG. 3 ) where the first image quality inhibiting element exists in the image is calculated.

To this end, the image quality evaluation unit 120 may calculate the type and location information of the first image quality inhibitory factor using a learning technique such as deep learning or machine learning. do.

In addition, the image quality evaluation unit 120 divides the image and evaluates the image quality for some images, collects the image quality evaluation for the partial images, and derives the image quality evaluation result for the entire image. may be

Next, the control unit 130 controls the first image quality inhibiting factor identified by the image quality evaluation unit 120 to respond to the image quality improvement.

In this case, the control unit 130 may determine the type and corresponding action of a corresponding device for improving the first image quality degrading factor in consideration of the type and location information of the first image quality degrading factor.

For a more specific example, the control unit 130 may control or operate a light source, a camera, a probe, a distal tip, etc. provided in the endoscope, for example, the type and direction of the light source. , the intensity, the direction of the camera, the direction of the probe or the direction of the distal tip can be controlled.

Accordingly, in the case of the prior art, in contrast to the limited improvement of the image by using only the brightness of the entire image or a specific image processing algorithm, in the endoscopic image quality improvement system 100 according to an embodiment of the present invention, the image quality related It is possible to generate an improved quality endoscope image by controlling various devices of the endoscope including a light source by identifying the presence and location of various types of image quality-impairing factors.

More specifically, when a bright part or a dark part occurs in the first region of the image, the control unit 130 changes the direction of the light source or changes the amount of light in consideration of the position of the bright part or dark part of the first region, An attempt can be made to resolve the luminance or gloom.

Furthermore, when a plurality of light sources exist in the endoscope, an attempt may be made to eliminate the bright or dark areas by changing the area irradiated by each light source or by changing the amount of light.

In addition, when a bright part or a dark part occurs in the first area of the image, the control unit 130 changes or adds the type of light source (white light, narrowband light source, fluorescent light source, infrared light source, etc.) to the image. It may be possible to obtain additional information from

In addition, when a bright part or a dark part occurs in the first region of the image, the controller 130 considers the position of the bright part or the dark part of the first region to use a probe and a distal tip of the endoscope. Alternatively, you can change the orientation or position of the camera.

Also, when bubbles, debris, or bleeding occurs in the first region of the image, the controller 130 controls the first region to contain bubbles, debris, or The direction or position of a probe, distal tip, or camera of the endoscope may be changed in consideration of the location of the bleeding.

Furthermore, when bubbles, debris, or bleeding occurs in the first region of the image, the controller 130 controls the first region to contain bubbles, debris, or In consideration of the location of the bleeding, it is possible to control to eject the washing solution from the distal tip of the endoscope.

In addition, in the endoscopic image quality improvement system 100 according to an embodiment of the present invention, there is an element obstructing the view in the same area in the first image of the first viewpoint and the second image of the second viewpoint in the image. In this case, the controller 130 may control to wash the lens by jetting a cleaning solution to the camera lens of the endoscope.

That is, when an element obstructing the view is photographed in the same area even though the field of view photographed by the camera is changed due to movement of the camera or enlargement or reduction of the lens, foreign substances are present in the lens of the camera Therefore, the control unit 130 may control to remove foreign substances present in the lens of the camera by jetting a cleaning solution to the camera lens of the endoscope to wash the lens.

More specifically, in FIG. 4 , an action corresponding to the first image quality inhibiting factor in the endoscopic image quality improvement system 100 according to an embodiment of the present invention is described.

For example, in FIG. 4 , when the arm part is present in the first area ((P1 of FIG. 4 )), the control unit 130 is inserted into the endoscope insertion part 2 as shown in FIG. 4 . The direction of the light source may be changed in consideration of the location of the first region ((P1) of FIG. 4) from the location of the provided light source ((P0) of FIG. 4) (eg, (R1 of FIG. 4)) ) direction). Furthermore, the control unit 130 may change the amount of light or the type of light source from the position of the light source (P0 in FIG. 4 ) to the position of the first region (P1 in FIG. 4 ). .

In addition, when bubbles exist in the second region (P2 in FIG. 4 ) in FIG. 4 , the control unit 130 controls the position of the second region ( In consideration of (P2) of FIG. 4), the cleaning solution is ejected from the distal tip of the endoscope insertion part 2 in the direction of the second region ((R2) direction of FIG. 4) to remove the air bubbles. may be

At this time, the control unit 130 may eject the washing solution in consideration of the distance to the second region ((D2 of FIG. 4)) or eject the washing solution after moving the endoscope insertion unit 2 have.

In addition, the endoscopic image quality improvement system 100 according to an embodiment of the present invention may be applied to a stored still image, a moving image, or an image acquired while using the endoscope.

Furthermore, the endoscopic image quality improvement system 100 according to an embodiment of the present invention can be applied not only to the endoscope for medical examination, but also to various images generated from the capsule endoscope, the surgical endoscope, and the industrial endoscope.

Accordingly, in the endoscopic image quality improvement system 100 according to an embodiment of the present invention, it is possible to identify and improve endoscopic image quality impeding factors of various causes that are difficult to solve by adjusting the brightness of the endoscopic image or applying an image processing algorithm. It is possible to produce quality endoscopy images.

Also, FIG. 5 shows a flowchart of a method for improving the quality of an endoscopic image according to an embodiment of the present invention.

As can be seen in FIG. 5 , the endoscopic image quality improvement method according to an embodiment of the present invention is an endoscopic image quality improvement method for improving the quality of an image generated using an endoscope, and an image acquisition step of acquiring the image (S110), the image quality evaluation step (S120) of determining the type of the first image quality inhibiting element existing in the first region of the image and the location of the first image quality inhibiting element in the image, and the image quality evaluation It may include a control step (S130) of controlling to respond to the first image quality inhibiting factor identified in step (S120).

Since the endoscopic image quality improvement method according to an embodiment of the present invention is similar to the above-described endoscopic image quality improvement system 100 and its operation principle and operation, a person skilled in the art can easily implement and implement it, so a more detailed description will be omitted. and may refer to a series of descriptions of the endoscopic image quality improvement system 100 described above.

In addition, the computer program according to another aspect of the present invention is characterized in that it is a computer program stored in a computer-readable medium to execute each step of the method for improving the quality of the endoscopic image quality salpin above on a computer. The computer program may be not only a computer program including a machine code generated by a compiler, but also a computer program including a high-level language code that can be executed in a computer using an interpreter or the like. In this case, the computer is not limited to a personal computer (PC) or a notebook computer, and includes a central processing unit (CPU) such as a server, smart phone, tablet PC, PDA, mobile phone, etc. to process any information that can execute a computer program. includes the device.

The medium may continue to store a computer executable program, or may be temporarily stored for execution or download. In addition, the medium may be various recording means or storage means in the form of a single or several hardware combined, it is not limited to a medium directly connected to any computer system, and may exist distributed on a network. Examples of the medium include a hard disk, a magnetic medium such as a floppy disk and a magnetic tape, an optical recording medium such as CD-ROM and DVD, a magneto-optical medium such as a floppy disk, and those configured to store program instructions, including ROM, RAM, flash memory, and the like. In addition, examples of other media may include recording media or storage media managed by an app store that distributes applications, sites that supply or distribute various other software, and servers.

Accordingly, the method, apparatus, and system for improving endoscopic image quality according to an embodiment of the present invention identify and improve endoscopic image quality impeding factors due to various causes that are difficult to solve due to brightness control of an endoscopic image or application of an image processing algorithm. It is possible to create an endoscope image of the highest quality.

The above description is merely illustrative of the technical idea of the present invention, and those of ordinary skill in the art to which the present invention pertains may make various modifications, changes, and substitutions within the scope not departing from the essential characteristics of the present invention. It will be possible. Therefore, the embodiments disclosed in the present invention and the accompanying drawings are for explanation rather than limiting the technical spirit of the present invention, and the scope of the technical spirit of the present invention is not limited by these embodiments and the accompanying drawings. . The protection scope of the present invention should be interpreted by the claims below, and all technical ideas within the scope equivalent thereto should be construed as being included in the scope of the present invention.

1: Endoscope system
2: endoscope insertion part
4: endoscope control unit
5: video display unit
100: endoscopic image quality improvement system
110: image acquisition unit
120: image quality evaluation unit
130: control unit

Claims (11)

In the endoscopic image quality improvement system for improving the quality of an image generated using an endoscope,
an image acquisition unit for acquiring the image;
an image quality evaluation unit configured to determine a type of a first image quality inhibiting element existing in a first region of the image and a location of the first image quality inhibiting element in the image; and
and a control unit controlling the image quality evaluation unit to respond to the first image quality impeding factor identified by the image quality evaluation unit. According to claim 1,
In the control unit,
Considering the type and location information of the first image quality inhibiting factor,
The endoscopic image quality improvement system, characterized in that it determines the type of the corresponding device and the corresponding action for improving the first image quality inhibiting factor. 3. The method of claim 2,
When a bright or dark part occurs in the first region,
The control unit is an endoscopic image quality improvement system, characterized in that to change the direction of the light source or change the amount of light. 3. The method of claim 2,
When a bright or dark part occurs in the first region,
The control unit is an endoscopic image quality improvement system, characterized in that for changing or adding a type of light source. 3. The method of claim 2,
When a bright or dark part occurs in the first region,
The control unit is an endoscope image quality improvement system, characterized in that for changing the direction or position of a probe (probe), distal tip (distal tip) or camera of the endoscope. 3. The method of claim 2,
When bubbles, debris, or bleeding occurs in the first region,
The control unit is an endoscope image quality improvement system, characterized in that for changing the direction or position of a probe (probe), distal tip (distal tip) or camera of the endoscope. 3. The method of claim 2,
When bubbles, debris, or bleeding occurs in the first region,
The control unit is an endoscope image quality improvement system, characterized in that the control to eject the cleaning solution from the distal tip (distal tip) of the endoscope. 3. The method of claim 2,
When an element obstructing the view is present in the same area in the first image of the first viewpoint and the second image of the second viewpoint in the image,
The control unit is an endoscope image quality improvement system, characterized in that the control to clean the lens by jetting a cleaning solution to the camera lens of the endoscope. According to claim 1,
The image quality evaluation unit,
The endoscopic image quality improvement system, characterized in that by using a learning technique to calculate the type and location information of the first image quality inhibiting factor. According to claim 1,
In the image quality evaluation unit,
After dividing the image and evaluating the image quality for some images,
The endoscopic image quality improvement system, characterized in that by collecting the image quality evaluation of the partial images, and deriving the image quality evaluation result for the entire image. In the endoscopic image quality improvement method for improving the quality of an image generated using an endoscope,
an image acquisition step of acquiring the image;
an image quality evaluation step of identifying a type of a first image quality inhibiting element existing in a first region of the image and a location of the first image quality inhibiting element in the image; and
and a control step of controlling to respond to the first image quality impeding factor identified in the image quality evaluation step.

Images