JP5528255B2 - Endoscopic image processing system - Google Patents

Description

  The present invention relates to an endoscope image processing system that supports detection of a characteristic lesion site in an image captured by an endoscope.

  An endoscope unit having an electronic endoscope is used for observing the inside of the body not irradiated with light. In the endoscope unit, the insertion tube is inserted into the body, the illuminated living tissue is imaged, and the captured subject image can be observed on the monitor.

  The appearance of a lesion site in the body is often different from that of a normal site, and an image displayed on the monitor of the endoscope unit may be used for diagnosis. Judgment as to whether or not it was a lesion site depended on the skill level of the observer.

  In addition, in in-vivo observation with an endoscope, it is difficult to adjust the direction in which a specific part is observed, and therefore it may be possible to obtain only images that are difficult to determine the difference in appearance. In such a case, it is difficult for a skilled observer to distinguish between a lesion site and a normal site.

  Therefore, an image analysis apparatus has been proposed that detects a lesion more accurately by detecting a two-dimensional gray value gradient even when it is difficult to detect an edge in an image obtained by observing a subject from above (Patent Document). 1).

  However, in the image analysis apparatus described in Patent Document 1, it is easy to estimate a circular lesion because a portion having an isotropic gradient is detected, but a portion having a gradient gradient other than a circle is detected. It was difficult to do. In addition, the image analysis apparatus described in Patent Document 1 has a problem in that false detections due to the influence of noise that can be mixed into pixel signals increase.

JP 2007-244519 A

  Therefore, an object of the present invention is to provide an endoscopic image processing system that detects a part estimated as various types of lesions while reducing the influence of noise.

  An endoscope image processing system according to the present invention includes a receiving unit that receives an image signal composed of pixel signals corresponding to a plurality of pixels that constitute a received light image that is a received light image, and a plurality of image blocks on the received light image. And a setting unit for setting a pixel at a first position in the image block as a reference pixel and a plurality of pixels located in a first direction from the reference pixel as a first pixel in the image block A selection unit that selects, for each image block, a plurality of pixels located in a second direction intersecting the first direction from a reference pixel, and a reference pixel and a plurality of first pixels The difference between the pixel signals of the two pixels whose interval is the first interval is defined as the first direction difference, and the pixel signal of the two pixels whose interval is the second interval among the reference pixel and the plurality of second pixels. Difference in the second direction A first calculation unit that calculates for each image block, a frequency in which the first direction difference belongs to the first class in the first direction, and a second direction difference belongs to the first class in the second direction A first feature based on a predetermined calculation method using a counter that counts the frequency for each image block and a frequency in which the first and second direction differences belong to the first class in the first and second directions, respectively. A second calculation unit that calculates the amount for each image block, a memory that stores a first feature amount that is calculated in advance based on images of a normal site and a first lesion site in the body, and a second calculation unit Discrimination for discriminating whether the image of the image block is an image of the normal site and the first lesion site by comparing the first feature value calculated by the above and the first feature value stored in the memory It is characterized by providing a part.

  The counter has a frequency in which the first direction difference belongs to a second class in a first direction different from the first class in the first direction and a second direction different from the second class in the second direction. It is preferable that at least one of the frequencies to which the second direction difference belongs to the second class is counted, and the second calculation unit calculates the first feature value using the frequency counted by the counter.

  In addition, the selection unit selects a plurality of pixels located in a third direction different from the first and second directions from the reference pixel, which are arranged in the image block, as a third pixel for each image block, and performs the first calculation. The unit calculates, for each image block, a difference between pixel signals of two pixels having a third interval between the reference pixel and the plurality of third pixels as a third direction difference, and the counter has a third direction. The frequency for which the difference belongs to the first class in the third direction is calculated for each image block, and the second calculation unit calculates the frequency for which the third direction difference belongs to the first class in the third direction for each image block. It is preferable that the second calculation unit calculates the first feature amount for each image block using the frequency counted by the counter.

  The second calculation unit calculates the second feature amount for each image block based on a predetermined calculation method using the frequency counted by the counter, and the memory stores the images of the normal site and the first lesion site. The second feature amount calculated in advance based on the second feature amount is stored, and the determination unit compares the second feature amount calculated by the second calculation unit with the second feature amount stored in the memory, thereby generating an image. It is preferable to determine whether or not the block image is an image of a normal site and a first lesion site.

  Further, the image size of the image of the normal part and the first lesion part used for calculation of the first feature amount stored in the memory is determined as the reference size, and the image size of the image block is determined as the reference size. It is preferable.

  In addition, the image size of the image of the normal site and the first lesion site used for calculation of the first feature value stored in the memory is set as the reference size, and the first feature value stored in the memory is A predetermined calculation method using the frequencies in which the first and second direction differences in the images of the normal site and the first lesion site whose image size is the reference size belong to the first class in the first and second directions, respectively. And the image size of the image block is determined to be a first size different from the reference size, and the second calculation unit calculates the frequency and the frequency of the first direction difference belonging to the first class in the first direction. It is preferable to normalize the frequency in which the direction difference of 2 belongs to the first class in the second direction using the reference size and the first size.

  In addition, a control unit that controls the setting unit, the selection unit, the first and second calculation units, the counter, and the determination unit is provided, and the normal part used for calculating the first feature value stored in the memory and the first part The image size of the image of the lesion site is determined as the reference size, and the first feature amount stored in the memory is the first and first images in the images of the normal site and the first lesion site whose image size is the reference size. Each of the two direction differences is calculated based on a predetermined calculation method using the frequencies belonging to the first class in the first and second directions, and the setting unit sets the received light image as one of a plurality of sizes as the image size. The second calculation unit can calculate the frequency of the first direction difference belonging to the first class in the first direction and the image size of the image block different from the reference size. The second direction difference is the second direction The frequency belonging to the first class is normalized using the reference size and the image size of the image block, and when the control unit determines that the image of any image block is not the image of the first lesion site, the setting unit The image size of the image block is changed, the reference unit and the first and second pixels are selected again by the selection unit, the first and second direction differences are calculated again by the first calculation unit, and the first and second counters are calculated by the counter. The frequency where the direction difference of 2 belongs to the first class is counted again, the first feature amount is calculated by the second calculation unit, and the image of the image block is the image of the normal site and the first lesion site by the discrimination unit. It is preferable to determine again whether or not.

  Further, it is preferable that the determination unit causes the image size of the image block to be set to the maximum size among the plurality of sizes at the time of initial setting in the setting unit after the reception unit receives the image signal.

  In addition, it is preferable that the determination unit causes the setting unit to change the image size of the image block to be small when the determination unit determines that the image of any image block is not the image of the first lesion site.

  In addition, the memory stores a range of first feature values determined based on a plurality of first feature values calculated in advance based on a plurality of images of a normal site and a first lesion site, and a determination unit Is based on whether or not the first feature value calculated by the second calculation unit is included in the range of the first feature value stored in the memory, the image of the image block is a normal part and a first lesion part It is preferable to discriminate whether or not it is an image.

  The memory stores a plurality of first feature amounts calculated in advance based on a plurality of images of the normal site and the first lesion site, and the determination unit calculates the first feature calculated by the second calculation unit. Classifying whether a feature amount is a normal site, a first lesion site, or a site other than the normal site and the first lesion site by a predetermined classification method using a plurality of first feature values stored in a memory Accordingly, it is preferable to determine whether or not the image of the image block is an image of a normal site and a first lesion site.

  In addition, the predetermined calculation method is principal component analysis, and the first and second direction differences among the first and second direction differences calculated for the images of a plurality of normal sites and first lesion sites received in advance are used. It is preferable that the first principal component calculated for the frequency belonging to the first class in the direction of 2 is calculated as the first feature amount.

  The memory stores the first feature amount when the image of the image block is an image of a second lesion site different from the first lesion site, and the determination unit is calculated by the second calculation unit. It is preferable to determine whether or not the image of the image block is the image of the second lesion site by comparing the first feature amount and the first feature amount stored in the memory.

  Further, it is preferable that the first calculation unit calculates the first and second direction differences using a luminance signal component or a red term component in the pixel signal.

  The selection unit preferably selects the reference pixel with the center of the image block as the first position.

  ADVANTAGE OF THE INVENTION According to this invention, it becomes possible to estimate whether the biological tissue image | photographed with the endoscope is a normal site | part or a lesion site | part, reducing the influence of noise.

1 is a block diagram schematically showing an internal configuration of an endoscope unit including an endoscope processor having an endoscope image processing system to which a first embodiment of the present invention is applied. FIG. It is a block diagram which shows roughly the internal structure of the lesioned part estimation circuit of 1st Embodiment. It is an arrangement view showing positions of a reference pixel and first to eighth pixels in an image block. It is a graph which shows the normal area | region and the 1st-3rd lesion area | region on the coordinate plane which makes the 1st, 2nd feature-value 2 axes | shafts. It is an image displayed on a monitor when there is a site estimated to be a lesion site. It is a graph which shows a 1st brightness | luminance gradient histogram. It is a graph which shows a 2nd brightness | luminance gradient histogram. The coordinates of the first and second feature amounts calculated for the images of the actual normal site and the first to third lesion sites, and the normal region and the first to third lesion regions defined by each coordinate are shown. It is a graph. It is a flowchart which shows the setting control of a 1st, 2nd function, a normal area | region, and the 1st-3rd lesion area | region. It is a 1st flowchart which shows the lesioned part estimation process performed by a lesioned part estimation circuit. It is a 2nd flowchart which shows the lesioned part estimation process performed by a lesioned part estimation circuit. It is a 3rd flowchart which shows the lesion part estimation process performed by a lesion part estimation circuit. It is a block diagram which shows roughly the internal structure of the lesioned part estimation circuit of 2nd Embodiment. It is the graph which plotted the coordinate of the 1st, 2nd feature-value calculated with respect to the image of an actual normal site | part and the 1st-3rd lesion site | part.

Embodiments of the present invention will be described below with reference to the drawings.
FIG. 1 is a block diagram schematically showing an internal configuration of an endoscope unit including an endoscope processor having an endoscope image processing system to which the first embodiment of the present invention is applied.

  The endoscope unit 10 includes an endoscope processor 20, an electronic endoscope 40, and a monitor 11. The endoscope processor 20 is connected to the electronic endoscope 40 and the monitor 11.

  Illumination light for illuminating the subject is supplied from the endoscope processor 20 to the electronic endoscope 40. The subject irradiated with the illumination light is imaged by the electronic endoscope 40. An image signal generated by imaging of the electronic endoscope 40 is sent to the endoscope processor 20.

  In the endoscope processor 20, predetermined signal processing is performed on the image signal obtained from the electronic endoscope 40. The image signal subjected to the predetermined signal processing is transmitted to the monitor 11, and an image corresponding to the transmitted image signal is displayed on the monitor 11.

  Next, the configuration of the endoscope processor 20 will be described. The endoscope processor 20 includes a light source system 21, first and second video signal processing circuits 22a and 22b, a lesion portion estimation circuit 30, an SDRAM 23, an EEPROM 24 (memory), a system controller 25, and the like.

  Illumination light emitted from the light source system 21 enters a light guide 41 provided in the electronic endoscope 40. The illumination light is transmitted to the tip of the insertion tube 42 by the light guide 41. The transmitted illumination light is applied to the subject in the distal direction of the insertion tube 42.

  An optical image of the reflected light of the subject irradiated with the illumination light is imaged on the light receiving surface of the image sensor 43 by an objective lens (not shown) provided at the distal end of the insertion tube 42. The image sensor 43 is controlled so as to generate an image signal of one frame every fixed period, for example, 1/60 seconds.

  A plurality of pixels (not shown) are two-dimensionally arranged on the light receiving surface of the image sensor 43. Each pixel is covered with one of RGB color filters (not shown). In each pixel, a pixel signal corresponding to the amount of received light component transmitted through the color filter is generated. The image signal is composed of pixel signals generated in all pixels.

  The generated image signal is converted into a digital signal by an A / D converter (not shown) provided in the electronic endoscope 40. The image signal converted into the digital signal is transmitted to the first video signal processing circuit 22a. The first video signal processing circuit 22 a is connected to the SDRAM 23 via the bus 26. The image signal is stored in the SDRAM 23.

  Note that the second video signal processing circuit 22 b, the lesioned part estimation circuit 30, and the EEPROM 24 are also connected to the bus 26, and image signals and necessary data are transmitted via the bus 26.

  The first video signal processing circuit 22a reads an image signal stored in the SDRAM 23, and predetermined signal processing including color interpolation processing and matrix processing is performed on the read image signal. By performing color interpolation processing, all pixels have RGB primary color signal components. Further, a luminance signal component and a color difference signal component are generated from the RGB primary color components by matrix processing. The image signal that has undergone predetermined signal processing is stored in the SDRAM 23.

  The image signal subjected to the predetermined signal processing by the first video signal processing circuit 22a is read from the SDRAM 23 to the second video signal processing circuit 22b. In the second video signal processing circuit 22b, predetermined signal processing such as superimposing is performed on the image signal. The image signal subjected to the predetermined signal processing is converted into an analog signal and output to the monitor 11.

  As will be described later, when the lesion portion estimation function provided in the endoscope processor 20 is executed, signal processing for instructing a portion estimated to be a lesion portion is additionally performed by the second video signal processing circuit 22b. Applied.

  An image corresponding to the received image signal is displayed on the monitor 11. As described above, the image sensor 43 is driven so as to generate an image signal every 1/60 seconds, and the image signal is also transmitted to the monitor 11 every 1/60 seconds. A moving image is displayed on the monitor 11 by switching the image to be displayed every 1/60 seconds.

  As described above, the endoscope processor 20 is provided with a lesion portion estimation function. When the lesion portion estimation function is executed, the position estimated as the lesion site in the image displayed on the monitor 11 is displayed surrounded by a dotted line as will be described in detail later.

  Next, the configuration of the lesion site estimation circuit 30 and the lesion site estimation function executed by the lesion site estimation circuit 30 will be described. FIG. 2 is a block diagram showing the internal configuration of the lesion site estimation circuit 30. The lesion estimation circuit 30 includes a division circuit 31 (setting unit), a selection circuit 32 (selection unit), a luminance gradient calculation circuit 33 (first calculation unit), a histogram creation circuit 34 (counter), a normalization circuit 35, and a feature. An amount calculation circuit 36 (second calculation unit) and a determination circuit 37 (determination unit) are included.

  When the lesion estimation function is executed, an image block is set by the dividing circuit 31. An image block is a small area obtained by dividing a captured image into a matrix. Specifically, a plurality of rectangular areas having a specific image size constituted by a plurality of pixels arranged in a matrix in the effective shooting range of the image sensor 43 are determined as image blocks.

  The number of pixels in the row direction and the column direction of the image block is determined to be equal. The image size of the image block can be changed to any one of the first, second, and third sizes. Each size is determined such that first size> second size> third size.

  When the image block is set by the dividing circuit 31, addresses of a plurality of pixels constituting each image block are transmitted from the dividing circuit 31 to the selection circuit 32 as image block data. When the selection circuit 32 receives the image block data, the selection circuit 32 selects a single image block.

  The luminance signal component of the pixel in the selected single image block is read from the SDRAM 23 via the bus 26. As described below, the luminance signal components of the reference pixel and the plurality of first to eighth pixels in the image block are read out. The read luminance signal component is transmitted to the luminance gradient calculation circuit 33.

  As shown in FIG. 3, the pixel located at the center in the image block IB is defined as the reference pixel sp. In addition, pixels arranged every two pixels upward from the reference pixel sp are defined as the first pixel p1. In addition, pixels arranged every two pixels from the reference pixel sp in the right direction are defined as the second pixels p2. In addition, pixels arranged every two pixels downward from the reference pixel sp are defined as third pixels p3. In addition, pixels arranged every two pixels from the reference pixel sp toward the left are defined as fourth pixels p4.

  In addition, pixels arranged every two pixels from the reference pixel sp in the upper right direction are defined as fifth pixels p5. In addition, pixels arranged every two pixels from the reference pixel sp toward the lower right are defined as the sixth pixel p6. In addition, pixels arranged every two pixels from the reference pixel sp toward the lower left are defined as the seventh pixel p7. Further, pixels arranged every two pixels from the reference pixel sp toward the upper left direction are defined as the eighth pixels p8.

  As described above, the luminance signal components in the reference pixel sp and the first to eighth pixels p1 to p8 are transmitted to the luminance gradient calculating circuit 33. The luminance gradient calculation circuit 33 calculates first to eighth luminance gradients described below.

  The first luminance gradient is a difference between luminance signal components in two pixels, which are two pixels among the reference pixel sp and all the first pixels p1, and are separated from each other by an interval of two pixels. is there. For example, the difference in luminance signal component between the first pixel p1 and the reference pixel sp at a position separated by two pixels upward from the reference pixel sp, or two pixels and four pixels separated from the reference pixel sp upward. This is the difference between the luminance signal components at the two first pixels p1 at the position.

  Similarly, the second luminance gradient is a difference between luminance signal components in two pixels of the reference pixel sp and all the second pixels p2 that are separated from each other by an interval of two pixels. It is. Similarly, the third luminance gradient is a difference between luminance signal components in two pixels of the reference pixel sp and all the third pixels p3, which are separated from each other by an interval of two pixels. It is. Similarly, the fourth luminance gradient is a difference between luminance signal components in two pixels of the reference pixel sp and all the fourth pixels p4, which are separated from each other by an interval of two pixels. It is.

  Similarly, the fifth luminance gradient is the difference between the luminance signal components in two pixels of the reference pixel sp and all the fifth pixels p5 that are separated from each other by an interval of two pixels. It is. Similarly, the sixth luminance gradient is a difference between luminance signal components in two pixels of the reference pixel sp and all the sixth pixels p6 that are separated from each other by an interval of two pixels. It is. Similarly, the seventh luminance gradient is a difference between luminance signal components in two pixels of the reference pixel sp and all of the seventh pixels p7 that are separated from each other by an interval of two pixels. It is. Similarly, the eighth luminance gradient is a difference between luminance signal components in two pixels of the reference pixel sp and all the eighth pixels p8 that are separated from each other by an interval of two pixels. It is.

  The calculated first to eighth luminance gradients are transmitted to the histogram creation circuit 34 as data. The histogram creation circuit 34 creates a histogram for each of the first to eighth luminance gradients.

  As an example, creation of a histogram of the first luminance gradient will be described. The signal level of the luminance signal component can be represented by 9 bits, that is, 0 to 511. The histograms are -256 to -224, -224 to -175, -174 to -125, -124 to -75, -74 to -25, -24 to 24, 25 to 74, 75 to 124, 125 to 174, The frequencies of the first to eleventh classes in the range of 175 to 224 and 225 to 256 are counted.

  The counted frequencies of the first to eleventh classes are transmitted as data to the normalization circuit 35. In the normalization circuit 35, the frequencies of the first to eleventh classes are normalized. For normalization, the image size of the image block IB is read from the dividing circuit 31, and a reference size described later is read from the EEPROM 24. A value obtained by dividing the reference size by the image size of the image block IB is calculated as a normalization coefficient. Normalization is performed by multiplying the frequencies of the first to eleventh classes by a normalization coefficient.

  The normalized frequency is transmitted to the feature quantity calculation circuit 36 as data. In the feature amount calculation circuit 36, the frequencies fr1.1 to fr1.11 to fr1.11 to the eighth luminance gradient histogram of the first to eleventh luminance gradient histograms in the normalized first luminance gradient histogram. The first and second feature amounts are calculated using the frequencies fr8.1 to fr8.11.

  In order to calculate the first and second feature amounts, the first and second functions f1 (fr1.1,..., Fr1.11,... Fr8.1,... Fr8.11) (hereinafter referred to as f1) , F2 (fr1.1,..., Fr1.11,... Fr8.1,..., Fr8.11) (hereinafter referred to as f2) are read from the EEPROM 24.

  The frequency fr1.1 to fr1.1 of the normalized first luminance gradient histogram fr1.1 to fr1.11 to the frequency fr8.1 to 8.11 of the eighth luminance gradient histogram. By substituting 11 into the first and second functions f1 and f2, the first and second feature quantities are calculated.

  The first and second feature amounts are transmitted as data to the determination circuit 37. In the determination circuit 37, it is determined whether the image formed on the image block IB is an image of a normal part and first to third abnormal parts based on the first and second feature amounts.

  Whether the image formed on the image block IB is an image of which part is calculated on the coordinate plane having the first and second feature amounts as two axes is calculated by the first and second calculated values. The determination is made based on the coordinates determined by the feature amount.

  As shown in FIG. 4, a normal area ah and first to third lesion areas a1 to a3 are defined on a two-dimensional coordinate plane having the first feature quantity as the horizontal axis and the second feature quantity as the vertical axis. It is done. An image corresponding to a region including a point having the first and second feature amounts calculated by the feature amount calculation circuit 36 as coordinates is estimated as an image formed on the image block IB.

  For example, when the calculated first and second feature quantities are located in the normal area ah, it is estimated that an image of a normal part is formed on the selected image block IB. In addition, when the calculated first and second feature amounts are located in two areas of the first and second lesion areas, the first and second lesion sites are displayed on the selected image block IB. Is estimated to be formed.

  A region including the first and second feature amounts is stored in the SDRAM 23 as data. When the image determination in the determination circuit 37 for the selected image block IB ends, a single image block IB that has not been subjected to image determination is selected by the selection circuit 32.

  The same processing as described above is performed on the selected image block IB, and image discrimination is performed. When image discrimination is performed for all image blocks IB, the system controller 25 (see FIG. 1) determines whether there is an image block IB on which images of the first to third lesion sites are formed. . Note that the operation of each part of the endoscope processor 10 is controlled by the system controller 25.

  When the images formed in all the image blocks IB are images that are estimated to be normal parts, the image size of the image block IB is changed to the second size based on the control of the system controller 25. After the image size is changed, image discrimination is performed for all image blocks IB, as in the first size.

  Even in the image block IB having the second size as the image size, when an image estimated to be the first to third lesion sites is not detected, the image size of the image block is changed to the third size. The Thereafter, image discrimination is performed as in the first and second sizes.

  When the image size is the first size or the second size and an image estimated as the first to third lesion sites is formed on any one of the image blocks IB, the image size is not changed. The estimation of the lesioned part with respect to the image signal of the target frame is stopped.

  After the estimation of the lesioned part is stopped or the image discrimination for the image block IB having the third size is completed, the second video signal processing circuit 22b executes signal processing for displaying the estimation result. By the signal processing of the estimation result display, an image (see FIG. 5) in which the area corresponding to the image block IB estimated as the first to third lesion sites is surrounded by the frame line 11f is created.

  Note that the color of the frame line is changed according to the type of the estimated lesion site. For example, when the first, second, and third lesion sites are estimated, the border line is drawn in red, green, and blue.

  Until the lesion part estimation function is turned off, the above-described lesion part estimation process is performed on the image signals of all frames.

  Next, the reference size, the first and second functions, the normal region, and the first to third lesion regions that are stored in the EEPROM 24 and used for lesion portion estimation will be described.

  These data are determined based on the images of the actual normal site and the first to third lesion sites, and are stored in the EEPROM 24 when the endoscope processor 10 is manufactured.

  First, a reference size is determined. The reference size is set to an appropriate size that can distinguish the normal site and the first to third lesion sites. Next, a plurality of images of the actual normal site and the first to third lesion sites are collected.

  It should be noted that the image is collected by selecting a part of the image from a whole image of the endoscope taken in advance by a medical worker or the like and classifying whether the image is a normal site or the first to third lesion sites. Executed. Note that the image size of the selected image is converted to a reference size. Alternatively, the frequency for forming the first to eighth histograms may be normalized by the reference size as in the case of executing the lesion portion estimation function without converting the image size.

  First to eighth luminance gradients are calculated for each collected image. Based on the calculated first to eighth luminance gradients, first to eighth luminance gradient histograms are created for each image. Each histogram has a different tendency depending on the image. That is, even if the individual histograms are different or different images are used for normal parts, similar histograms are created. The same applies to the first to third lesion sites.

  For example, as shown in FIGS. 6 and 7, the histograms of the first and second luminance gradients differ depending on the normal site and the first to third lesion sites. Similarly, the histograms of the third to eighth luminance gradients differ depending on the normal site and the first to third lesion sites.

  Next, by performing principal component analysis on the frequencies of the first to eleventh classes for each of the first to eighth luminance gradient histograms, it is suitable for distinguishing between a normal site and first to third lesion sites. First and second functions f1 and f2 which are calculation methods of the first and second principal components are determined.

  That is, an 88-dimensional (8 × 11) feature space using the frequencies of the first to eleventh classes for each of the first to eighth luminance gradient histograms as variables is converted into a normal region, first to first features by principal component analysis. The first and second principal components that can represent the distribution of the three lesion sites are converted into a two-dimensional feature space.

  Based on the first to eighth luminance gradient histograms created for each of the plurality of images of the normal site and the plurality of images of the first to third lesion sites, the first and second principal components for the respective images are Calculated as the first and second feature amounts.

  The calculated first and second feature quantities are plotted on a coordinate plane having the first and second feature quantities as two axes (see FIG. 8). A region including the first and second feature amounts corresponding to the image of the normal part is determined as the normal region ah. In addition, regions including the first and second feature amounts corresponding to the images of the first to third lesion sites are determined as the first to third lesion regions a1 to a3.

  The reference size determined as described above, the first and second functions f1, f2, the normal region ah, and the first to third lesion regions a1 to a3 are stored in the EEPROM 24, and are used for the lesion portion estimation function. Used.

  Next, setting control executed by the first and second functions, the normal region, and the first to third lesion region setting devices (not shown) will be described with reference to the flowchart of FIG. Setting control is executed by a start input to the setting device.

  In step S100, it is determined whether there is an input for selecting an image and an input for classifying. For example, an input for selecting a partial image by the operator is performed on the entire collected images of a plurality of endoscopes, and the selected image is in any one of a normal site and first to third lesion sites. It is determined whether there is an input for classifying whether there is any.

  If there is an image selection input and a classification input, the process proceeds to step S101. If there is no input, step S101 is skipped and the process proceeds to step S102.

  In step S101, image data corresponding to the selected partial image is extracted, and the extracted image data and classified part data are stored in a working memory (not shown) of the setting device. After storing in the working memory, the process proceeds to step S102.

  In step S102, it is determined whether or not an input for ending image selection has been made. If the selection is not completed, the process returns to step S100. Thereafter, the processing from step S100 to step S102 is repeated until selection end is input. When an input to end image selection is made, the process proceeds to step S103.

  In step S103, the first to eighth luminance gradients are calculated for each input image. When the calculation of the brightness gradient is finished, the process proceeds to step S104.

  In step S104, first to eighth luminance gradient histograms for each image are created. When the first to eighth luminance gradient histograms are created, the process proceeds to step S105.

  In step S105, normalization of the first to eighth luminance gradient histograms is performed. That is, the frequency of each class in the first to eighth luminance gradient histograms is multiplied by a coefficient obtained by dividing the reference size by the image size of the original image. After normalization, the process proceeds to step S106.

  In step S106, principal component analysis is performed on the normalized first to eighth luminance gradient histograms created for each image, whereby the first and second principal components are converted into the first and second principal components. First and second functions f1 and f2 to be calculated as feature amounts are created. After creating the first and second functions f1, f2, the process proceeds to step S107.

  In step S107, the created first and second functions f1 and f2 are stored in a recording memory (not shown). After storing in the memory, the process proceeds to step S108.

  In step S108, first, first and second feature amounts for each image stored in S101 are calculated. Note that the first and second feature amounts for each image are substituted by substituting the frequencies in the first to eighth luminance gradient histograms for each image into the first and second functions f1 and f2 stored in step S107. Is calculated.

  Next, the first and second feature amounts corresponding to the calculated images are plotted in a two-dimensional coordinate space having the first and second feature amounts as axes. Based on the corresponding points of each plotted image, the normal range and the first to third lesion ranges are determined. The normal range and the first to third lesion ranges are determined based on an appropriate calculation method based on the corresponding points of each image. Alternatively, the operator may determine the range for each actually plotted point. After determining the normal range, the process proceeds to step S109.

  In step S109, the normal range and the first to third lesion ranges determined in step S108 are stored in the recording memory. After storing in the memory, the setting control is terminated.

  A normalized histogram is used for calculating the normal range and the first to third lesion ranges. Moreover, the normalized histogram is equivalent to the histogram of the image of the normal site and the first to third lesion sites whose image size is the reference size.

  Next, the lesion part estimation process executed by the lesion part estimation circuit 30 will be described with reference to the flowcharts of FIGS. The lesion part estimation process is started each time an image signal of one frame is received in a state where execution of the lesion part estimation function is input.

  In step S200, the image size of the image block IB is determined to be the first size. After the image size IB is determined, the process proceeds to step 201.

  In step S201, the image block IB having the image size determined in step S200 or step S218 described later is set. After setting the image block IB, the process proceeds to step S202.

  In step S202, a single image block IB that has not been subjected to the processing in steps S203 to S213 is selected from the plurality of image blocks IB set in step S201. After selecting a single image block IB, the process proceeds to step S203.

  In step S203, the reference pixel sp and the first to eighth pixels p1 to p8 in the image block IB selected in step S202 are selected. After selecting the reference pixel sp and the first to eighth pixels p1 to p8, the process proceeds to step S204.

  In step S204, the luminance signal components of the reference pixel sp and the first to eighth pixels p1 to p8 selected in step S203 are read from the SDRAM 23. After reading the luminance signal component, the process proceeds to step S205.

  In step S205, the first to eighth luminance gradients are calculated using the luminance signal component read in step S204. After calculating the luminance gradient, the process proceeds to step S206.

  In step S206, first to eighth luminance gradient histograms are created using the first to eighth luminance gradients calculated in step S205. After creating the first to eighth luminance gradient histograms, the process proceeds to step S207.

  In step S207, the first to eighth luminance gradient histograms created in step S206 are normalized. That is, the frequency of each class in the first to eighth luminance gradient histograms is multiplied by a normalization coefficient obtained by dividing the reference size by the image size of the image block IB. After normalization of the histogram, the process proceeds to step S208.

  In step S208, first and second feature amounts are calculated. That is, the first and second feature quantities are calculated by substituting the frequencies of the classes in the first to eighth luminance gradient histograms normalized in step S207 into the first and second functions f1 and f2. Is done. After calculating the first and second feature amounts, the process proceeds to step S209.

  In step S209, it is determined whether or not the first and second feature values calculated in step S208 belong to the normal range. If it is in the normal range, step S210 to step S212 are skipped and the process proceeds to step S213. If not in the normal range, the process proceeds to step S210.

  In step S210, it is determined whether or not the first and second feature amounts calculated in step S208 belong to the first lesion range. If it is the first lesion range, step S211 and step S212 are skipped and the process proceeds to step S213. If it is not the first lesion range, the process proceeds to step S211.

  In step S211, it is determined whether or not the first and second feature amounts calculated in step S208 belong to the second lesion range. If it is the second lesion range, step S212 is skipped and the process proceeds to step S213. If it is not the second lesion range, the process proceeds to step S212.

  In step S212, it is determined whether or not the first and second feature amounts calculated in step S208 belong to the third lesion range. If it is not the third lesion range, step S213 is skipped and the process proceeds to step S214. If it is the third lesion range, the process proceeds to step S213.

  In step S213, a part signal corresponding to the part determined in steps S209 to S212 is added to the image data of the image block. After adding the part signal, the process proceeds to step S214.

  In step S214, it is determined whether or not image estimation processing has been performed on all the image blocks IB set in step S201. If image estimation processing has not been performed on all image blocks, the process returns to step S202. Thereafter, the processes in steps S202 to S214 are repeated until the image estimation process is performed on all the image blocks IB. If image estimation processing has been performed on all image blocks, the process proceeds to step S215.

  In step S215, it is determined whether or not there is an image block IB estimated to be the first to third lesion sites. If there is an image block estimated to be the first to third lesion sites, the process proceeds to step S218. If none of the image blocks is estimated to be the first to third lesion sites, the process proceeds to step S216.

  In step S216, it is determined whether or not the image size of the image block IB is the third size. If the image size of the image block IB is the third size, the process proceeds to step S218. If the image size of the image block IB is not the third size, the process proceeds to step S217.

  In step S217, the image size of the image block IB is changed to one size smaller. That is, when the image size of the image block IB is the first size, it is changed to the second size, and when the image size of the image block IB is the second size, it is changed to the third size. After changing the image size, the process returns to step 201.

  As described above, when there is an image block IB estimated to be the first to third lesion sites in step S215, or when the image size of the image block IB is the third size in step S216, step S218 is performed. Proceed to In step S218, the image signal stored in the SDRAM 23 is subjected to image processing for superimposing the frame line 11f on the entire image at the location of the image block IB estimated to be the first to third lesion sites. After the frame line superimposition process, the lesion part estimation process performed on the image signal of one frame is terminated.

  As described above, according to the endoscopic image processing system of the first embodiment, it is possible to determine whether or not there are lesion sites having various appearances based on the captured in-vivo images.

  As described above, the luminance gradient histogram in each direction differs depending on the appearance of a normal site or a lesion site. Therefore, as in this embodiment, a feature amount calculation method using a luminance gradient histogram in each direction in an actual image is determined, the feature amount calculated for the actual image is stored as a database, and an image block It is possible to determine whether or not the lesion site has various appearances by comparing the feature amount calculated every time with a stored database.

  Further, in the first embodiment, it is possible to reduce the influence of noise mixed in the pixel signal with respect to the estimation of the lesion site. Noise may be mixed in some pixel signals, and the luminance gradient using the pixel signal mixed with noise has an error for noise. However, in this embodiment, even if an error for noise is mixed in a part of the brightness gradient, the image is discriminated using many brightness gradients, so that the influence of noise is reduced.

  In the first embodiment, since the luminance gradient histogram is normalized, it is possible to estimate a lesion site for image blocks of various sizes. When normalization is not performed, an image size other than the same image size as the image size of the normal region and the first to third lesion regions used in the calculation of the normal range and the first to third lesion ranges The lesion site could not be estimated for the image. On the other hand, in the present embodiment, normalization is performed, so that it is possible to estimate a lesion site regardless of the size.

  Next, an endoscope image processing system according to a second embodiment of the present invention will be described. The endoscopic image processing system according to the second embodiment is different from the first embodiment in a method for determining whether or not a lesion site uses the first and second feature amounts. Hereinafter, a description will be given focusing on differences from the first embodiment. In addition, the same code | symbol is attached | subjected to the site | part which has the same function as 1st Embodiment.

  In the second embodiment, the configuration and functions of the parts of the endoscope processor 20 other than the EEPROM 240 and the lesion site estimation circuit 300 are the same as those in the first embodiment (see FIG. 13).

  Unlike the first embodiment, the EEPROM 240 has an actual normal region and first to second functions used for determining the first and second functions f1 and f2 instead of the normal region and the first to third lesion regions. The first and second feature amounts calculated for a plurality of images of the third lesion site are stored. That is, as shown in FIG. 14, the first and second feature amounts calculated for the image of the normal site and the lesion site are stored without defining the normal region and the first to third lesion regions. ing.

  The configuration of the lesion site estimation circuit 300 in the second embodiment is the same as that in the first embodiment (see FIG. 13). On the other hand, in the lesion site estimation circuit 300 in the second embodiment, only the function of the discrimination circuit 370 is different from that in the first embodiment. The function of each part of the lesioned part estimation circuit 300 other than the determination circuit 370 is the same as that of the first embodiment.

  Therefore, the first and second feature amounts of the image of the image block IB selected by the feature amount calculation circuit 36 are transmitted to the determination circuit 370 as data. As in the first embodiment, the determination circuit 370 determines whether the image formed on the image block IB is an image of a normal site and first to third lesion sites.

  In order to discriminate images, the first and second feature quantities calculated for the images of actual normal sites and lesion sites stored in the EEPROM 240 are read out to the discriminating circuit 370. In the discrimination circuit 370, the first and second features calculated by the pattern classification method such as the NN method using the first and second feature values calculated for the images of the actual normal site and the lesion site. It is determined to which part the quantity is classified. In addition, when the distance from the collection of the first and second feature amounts calculated for the actual image is more than a certain determination value, the classification may be impossible.

  Also with the endoscope image processing system according to the second embodiment configured as described above, it is possible to determine whether or not there are lesion sites with various appearances based on the captured in-vivo images.

  In the first and second embodiments, the luminance gradients in the eight directions are calculated in the upper, lower, left, right, upper right, lower right, lower left, and upper left directions. It is not limited to the direction. It may be a luminance gradient in another direction. Further, the direction of the luminance gradient to be calculated is not limited to eight directions. The effects similar to those of the first and second embodiments can be obtained as long as at least two directions intersecting each other or three or more directions.

  However, the greater the number of directions, the better the detection accuracy, but the calculation speed is reduced. Therefore, it is preferable that the number of directions is determined in accordance with the calculation speed of the circuit to be used.

  Moreover, although it is the structure by which the difference of the brightness | luminance signal component of the pixels which left | separated by the 2 pixel space | interval is calculated as a 1st-8th brightness | luminance gradient, an space | interval is not limited to 2 pixels. Further, the pixel interval may be changed depending on the direction of the luminance gradient. However, when the pixel interval varies depending on the direction of the luminance gradient, the images of the actual normal site and the first to third lesion sites used for calculating the first and second functions stored in the EEPROMs 24 and 240 are displayed. It is necessary to match the pixel interval according to the direction of the luminance gradient at.

  In the first and second embodiments, in the first to eighth luminance gradient histograms, the luminance gradient is divided into the first to eleventh classes, but is divided into at least two or more classes. Thus, the same effects as those of the first and second embodiments can be obtained.

  In the first and second embodiments, the first and second feature amounts are calculated. However, only the first feature amount may be calculated, or three or more feature amounts may be calculated. It may be a calculated configuration. If the normal part and the first to third lesion parts can be discriminated, the calculated feature amount is not limited to two.

  Further, in the first and second embodiments, the histogram calculated based on the image of the image block is normalized, but may not be normalized. Even if it is not normalized, it is possible to estimate whether or not the image is a lesion site with respect to the image of the image block of the reference size. However, as long as normalization is possible as described above, the lesion site can be estimated without changing the optical magnification, regardless of the size of the lesion site.

  In the first and second embodiments, the estimated lesion sites are the first to third lesion sites, but the number of lesion sites that can be identified is not limited to three. It suffices if it is possible to determine whether or not there is one or more lesion sites.

  In the first and second embodiments, the calculation method for the first and second feature amounts is determined by principal component analysis. However, the frequency of each class in the first to eighth luminance gradient histograms is determined. As the variables, any known feature amount calculation method for calculating a feature amount capable of expressing the features of each variable for each image may be used.

  In the first and second embodiments, the image size of the image block IB can be changed in three stages of the first to third sizes, but can be changed in any number of stages of two or more. Good.

  In the first and second embodiments, the image size of the image block IB immediately after execution of the lesion portion estimation function is the first size that is the maximum size in the adjustable range, but other sizes are used. Also good. However, as in the first and second embodiments, by reducing the size in order from the maximum size, the time required for estimation can be reduced compared to the configuration in which the size is increased in order from the minimum size. Is possible.

  In the first and second embodiments, the image size of the image block IB can be changed in three stages of the first to third sizes. It is possible to execute.

  In the first and second embodiments, the position of the reference pixel in the image block is determined at the center of the image block IB, but is not limited to the center. Any position of the image block IB may be used.

  In the first and second embodiments, the difference between the luminance signal components between the two pixels is calculated as the luminance gradient, but the R primary color gradient is calculated using the R primary color signal component instead of the luminance signal component. Then, the first and second feature amounts may be calculated based on the first to eighth R primary color gradients instead of the first to eighth luminance gradient histograms. The R primary color signal component contains more unevenness information in the body than the other primary color signal components, and even if it is used instead of the luminance signal component, it can be sufficiently used for estimating a lesion site.

10 Endoscope unit 11f Frame line 23 SDRAM
24, 240 EEPROM
25 system controller 30, 300 lesion part estimation circuit 31 division circuit 32 selection circuit 33 luminance gradient calculation circuit 34 histogram creation circuit 35 normal circuit 36 feature amount calculation circuit 37, 370 discrimination circuit 40 electronic endoscope 43 imaging element ah normal region a1 ˜a3 1st to 3rd lesion area IB image block sp reference pixel p1 to p8 1st to 8th pixel

Claims (14)

A receiving unit that receives an image signal constituted by pixel signals corresponding to a plurality of pixels constituting a received light image that is a received light image;
A setting unit for setting a plurality of image blocks on the received light image;
In the image block, the plurality of pixels arranged in the first direction from the reference pixel as the first pixel, the pixel at the first position in the image block as the reference pixel. A selection unit configured to select, for each image block, a plurality of the pixels located in a second direction intersecting with the first direction from the reference pixel as second pixels;
A difference between pixel signals of two pixels having a first interval among the reference pixel and the plurality of first pixels is defined as a first direction difference, and the reference pixel and the plurality of second pixels are compared. A first calculation unit that calculates, for each image block, a difference between pixel signals of two pixels having a second interval in the second direction difference;
A counter that counts, for each image block, the frequency that the first direction difference belongs to the first class in the first direction and the frequency that the second direction difference belongs to the first class in the second direction. When,
Based on a predetermined calculation method using the frequencies in which the first and second direction differences belong to the first class in the first and second directions, the first feature amount is calculated for each image block. A second calculation unit for calculating;
A memory for storing the first feature amount calculated in advance based on images of a normal site and a first lesion site in the body;
By comparing the first feature value calculated by the second calculation unit and the first feature value stored in the memory, the image of the image block is the normal part and the first feature value. A discriminator for discriminating whether the image is a lesion site;
A control unit that controls the setting unit, the selection unit, the first and second calculation units, the counter, and the determination unit;
The image size of the image of the normal site and the first lesion site used for the calculation of the first feature amount stored in the memory is determined as a reference size,
The first feature amount stored in the memory is such that the first and second direction differences in the images of the normal site and the first lesion site whose image size is the reference size are the first and second direction differences, respectively. , Calculated based on the predetermined calculation method using the frequency belonging to the first class in the second direction,
The setting unit can set the received light image to the image block having one of a plurality of sizes as an image size,
When the image size of the image block is an image size different from the reference size, the second calculation unit calculates the frequency of the first direction difference belonging to the first class in the first direction and the second Normalize the frequency belonging to the first class in the second direction using the reference size and the image size of the image block;
The control unit causes the setting unit to change the image size of the image block when the determination unit determines that the image of any image block is not the image of the first lesion site, and causes the selection unit to change the image block The reference pixel and the first and second pixels are selected again, the first calculation unit is caused to calculate the first and second direction differences again, and the counter is configured to determine the first and second direction differences. The frequency belonging to the first class is counted again, the second calculation unit is caused to calculate the first feature amount, and the image of the image block is displayed on the normal part and the first lesion part on the determination unit. An endoscopic image processing system characterized by re-determining whether or not the image is an image. It said counter includes a first different from the first class of the first direction of the first class and the different first direction of the second class to the first power of directions difference belongs, and said second direction Counting at least one of the frequencies to which the second direction difference belongs to a second class in two directions;
The endoscope image processing system according to claim 1, wherein the second calculation unit calculates the first feature amount using the frequency counted by the counter. The selection unit selects, for each image block, a plurality of pixels located in a third direction different from the first and second directions from the reference pixel, which are arranged in the image block, as third pixels. And
The first calculation unit determines, for each image block, a difference between pixel signals of two pixels having a third interval among the reference pixel and the plurality of third pixels as a third direction difference. To
The counter calculates, for each image block, a frequency at which the third direction difference belongs to the first class in the third direction;
The second calculation unit counts the frequency for which the third direction difference belongs to the first class in the third direction for each image block,
3. The internal view according to claim 1, wherein the second calculation unit calculates the first feature amount for each of the image blocks by using the frequency counted by the counter. Mirror image processing system. The second calculation unit calculates a second feature amount for each image block based on the predetermined calculation method using the frequency counted by the counter,
The memory stores the second feature amount calculated in advance based on images of the normal site and the first lesion site,
The determination unit compares the second feature amount calculated by the second calculation unit with the second feature amount stored in the memory, so that the image of the image block is the normal region. It is discriminate | determined whether it is an image of the said 1st lesion site | part. The endoscopic image processing system of Claim 2 or Claim 3 characterized by the above-mentioned.   The image size of the image of the normal site and the first lesion site used for the calculation of the first feature amount stored in the memory is determined as a reference size, and the image size of the image block is the reference size The endoscope image processing system according to any one of claims 1 to 4, wherein the endoscope image processing system is determined in size. The image size of the image of the normal site and the first lesion site used for the calculation of the first feature amount stored in the memory is determined as a reference size,
The first feature amount stored in the memory is such that the first and second direction differences in the images of the normal site and the first lesion site whose image size is the reference size are the first and second direction differences, respectively. , Calculated based on the predetermined calculation method using the frequency belonging to the first class in the second direction,
The image size of the image block is set to a first size different from the reference size,
The second calculation unit calculates the frequency that the first direction difference belongs to the first class in the first direction and the frequency that the second direction difference belongs to the first class in the second direction, The endoscope image processing system according to any one of claims 1 to 4, wherein normalization is performed using a reference size and the first size.   The determining unit causes the image size of the image block to be set to a maximum size among the plurality of sizes at the time of initial setting in the setting unit after the receiving unit receives the image signal. The endoscope image processing system according to claim 1.   The discriminating unit causes the setting unit to change the image block to reduce the image size when the discriminating unit discriminates that the image of any image block is not the image of the first lesion site. The endoscopic image processing system according to claim 7, wherein: The memory stores a range of the first feature amount determined based on a plurality of the first feature amounts calculated in advance based on a plurality of images of the normal site and the first lesion site. And
The determining unit is configured to determine whether the first feature amount calculated by the second calculation unit is included in a range of the first feature amount stored in the memory. It is discriminate | determined whether the image of this is an image of the said normal site | part and the said 1st lesion site | part. The endoscope image processing system of any one of Claims 1-8 characterized by the above-mentioned. The memory stores a plurality of the first feature amounts calculated in advance based on a plurality of images of the normal site and the first lesion site,
The discriminating unit is configured such that the first feature amount calculated by the second calculation unit is the normal part, the first feature amount by a predetermined classification method using a plurality of first feature amounts stored in the memory. Whether or not the image of the image block is an image of the normal site and the first lesion site by classifying whether the site is one lesion site, or a site other than the normal site and the first lesion site The endoscope image processing system according to any one of claims 1 to 8, wherein the endoscope image processing system is determined.   The predetermined calculation method is principal component analysis, and the first and second direction differences calculated for images of the plurality of normal sites and the first lesion site received in advance are the first and second direction differences. 11. The first principal component obtained for the frequency belonging to the first class in the first and second directions is calculated as the first feature amount. The endoscope image processing system according to item. The memory stores the first feature amount when the image of the image block is an image of a second lesion site different from the first lesion site,
The determination unit compares the first feature amount calculated by the second calculation unit with the first feature amount stored in the memory, so that the image of the image block is the second feature amount. It is discriminate | determined whether it is an image of the lesion site | part of Claim 1. The endoscopic image processing system of any one of Claims 1-11 characterized by the above-mentioned.   13. The first calculation unit according to claim 1, wherein the first calculation unit calculates the first and second direction differences using a luminance signal component or a red term component in the pixel signal. The endoscope image processing system according to item. The endoscope image processing system according to any one of claims 1 to 13, wherein the selection unit selects the reference pixel with the center of the image block as the first position.

Images