JPH1014864A - Diagnosis support device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To yield a stable auto-diagnostic result by calculating a characteristic quantity being lesser influenced by difference in the image picking-up conditions. SOLUTION: A characteristic quantity calculating block 74 in a diagnosis support processing execution program 70 of a diagnosis supporting device calculates the inter-ROI characteristic quantity by combining two or more ROI's set in one endoscopic image, and thereby evaluates the relative difference with a normal mucous membrane between differing disease indications. That is, judgement and classification not being influenced by the dispersion of image picking-up conditions can be made practicable by determining the relative value of the characteristic quantity calculated from a plurality of ROI's in the same image.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a diagnosis support apparatus, and more particularly, to a diagnosis support apparatus having a feature of automatically discriminating and classifying the type of a lesion based on image data from an endoscope apparatus.

[0002]

2. Description of the Related Art In recent years, an elongated insertion portion has been inserted into a body cavity,
2. Description of the Related Art An endoscope apparatus capable of observing an organ in a body cavity or the like on a monitor screen using a solid-state imaging device or the like as an imaging unit on a monitor screen to perform inspection or diagnosis is widely used.

[0003] Further, ultrasonic waves are applied to the organs in the body cavity,
Ultrasound endoscope devices that can observe or inspect or diagnose the condition of the internal organs of the body cavity on a monitor screen based on the reflection or transmittance of the ultrasonic waves are also widely used.

On the other hand, there has been proposed an image display apparatus using a medical image database for managing a large number of image data obtained by such an endoscope apparatus, an ultrasonic diagnostic apparatus and the like.

[0005] Endoscope images picked up by the endoscope device and the ultrasonic endoscope device are recorded on a recording medium such as a magneto-optical disk, and an image file device is connected so that the image file device can be effectively used for later diagnosis. Is used as a system.

In the field of medical imaging, for example, as shown in Japanese Patent Application Laid-Open No. 7-37056, a computer is used for an image recorded in an image file device.
A diagnosis support device capable of performing an automatic diagnosis has been proposed.

[0007] The diagnosis support apparatus uses a certain amount of feature calculated from a region of interest (ROI) in an image, and uses threshold processing or a statistical / non-statistical discriminator to determine what kind of lesion the image to be diagnosed has. The diagnosis is supported by presenting to the physician whether the classification is made.

[0008]

However, in the conventional diagnosis support apparatus, diagnosis support is performed by simply using feature amounts individually calculated from ROIs set in a large number of images for an endoscope image. However, it cannot be said that the calculation of the feature amount according to the feature of the lesion is necessarily performed. The reason is as follows.

In endoscopic images, doctors find and diagnose lesions based on many features such as the color tone on the mucous membrane surface, the regularity and coarse density of structural patterns (textures).

[0010] However, in an endoscopic image, there is a large variation in image capturing conditions. For example, even when the same lesion is imaged, the color tone in the captured image may change because the irradiation condition of the observation light changes for each image depending on the observation angle, the distance, and the like. In addition, the influence of individual differences in mucous membrane color and the like may be considered. Further, the characteristics of the texture change greatly depending on the observation distance. Therefore, the values of the feature amounts in the normal part and the lesion part calculated from the different endoscope images approach each other, and hinder the diagnosis support.

[0011] On the other hand, it is considered that the discovery and diagnosis of lesions are often based mainly on the difference in the characteristics between normal mucosa and lesion mucosa. For example, the discovery of a lesion from a relative difference in some characteristic between a normal part and a lesion part in one endoscopic image (for example, a color tone is different from that of the surrounding mucous membrane) determines the type of the difference (for example, Diagnosis is made based on the red color).

This point is not taken into account in the conventional diagnosis support apparatus, and it has not been possible to calculate relative feature values in a plurality of ROIs.

Further, in an endoscope image, no consideration has been given to a feature amount that effectively uses local information depending on the position of each pixel on the image.

Further, in order to calculate a feature amount from an endoscope image, preprocessing such as noise removal is performed in a stage prior to application of the feature amount calculation method. However, these preprocessing methods only apply a known basic method, and a plurality of methods specialized for extracting feature amounts from an endoscope image, and further, if necessary, No consideration has been given to using the above methods in an appropriate order and combination.

The present invention has been made in view of the above circumstances, and calculates a feature amount that is less affected by differences in imaging conditions, calculates a feature amount that stores information that depends on the position of a pixel on an image, and An object of the present invention is to provide a diagnosis support apparatus that can obtain a stable automatic diagnosis result by providing a preprocessing method for calculating a feature amount useful for diagnosis from an endoscope image.

[0016]

According to the present invention, there is provided a diagnosis support apparatus comprising: image input means for inputting image data comprising at least one signal; storage means for storing the input image data; A region-of-interest setting unit for setting at least one region of interest for the recorded image data; a characteristic amount calculating unit for calculating a characteristic amount from the region of interest set by the region of interest setting unit; Discriminating and classifying means for performing discrimination and classification of lesions using the feature amount calculated by the calculating means.

In the diagnosis support apparatus according to the present invention, the characteristic amount calculating means calculates a characteristic amount from the region of interest set by the region of interest setting means, and the discriminating and classifying means is calculated by the characteristic amount calculating means. By performing classification and classification of lesions using the feature amount, a feature amount or the like that is less affected by a difference in imaging conditions is calculated, and by using this feature amount,
It is possible to obtain a stable automatic diagnosis result.

[0018]

Embodiments of the present invention will be described below with reference to the drawings.

FIGS. 1 to 40 relate to a first embodiment of the present invention. FIG. 1 is a block diagram showing a configuration of a diagnosis support apparatus. FIG. 2 is a diagram showing a diagnosis support processing execution program in the diagnosis support apparatus of FIG. FIG. 3 is a functional block diagram showing the configuration, and FIG.
FIG. 4 is a configuration diagram showing a conceptual configuration of a multi-window screen in the diagnosis support apparatus of FIG. 4, FIG. 4 is a configuration diagram showing a configuration of a multi-window screen developed by operating icons on the screen of FIG. 3, and FIG. FIG. 6 is a first explanatory diagram illustrating the configuration of a window on the screen, and FIG. 6 is a second explanatory diagram illustrating the configuration of a window on the multi-window screen of FIG.
FIG. 7 is a configuration diagram showing a specific configuration of a multi-window screen in the diagnosis support apparatus of FIG. 1, and FIG. 8 is file management data for managing an endoscope image file recorded on the hard disk of FIG. FIG. 9 is an explanatory diagram for explaining management data provided for each endoscopic examination of each endoscopic image of the endoscopic image file recorded on the hard disk in FIG. 1, and FIG. 7 is a configuration diagram showing a configuration of an image management data window developed by a click operation of an image display icon, FIG. 11 is a configuration diagram showing a configuration of a search window developed by a click operation of a call button in FIG. 10, and FIG. FIG. 13 is a configuration diagram showing a configuration of a search result display window expanded by a click operation of the condition search start button in FIG. 11, and FIG. 13 is a click operation of an image list button in FIG. FIG. 14 is a configuration diagram showing a configuration of an image list display window expanded by the operation of FIG. 13, FIG. 14 is a configuration diagram showing a configuration of an image display window expanded by a click operation of the display button of FIG. 13, and FIG. FIG. 16 is a configuration diagram showing a configuration of an image processing window developed by a click operation, FIG. 16 is a configuration diagram showing a configuration of a processing result image display window developed by a click operation of an execution button of FIG. 15, and FIG. 17 is a database of FIG. Configuration diagram showing the configuration of the database managed by the management block,
FIG. 18 is a configuration diagram showing a configuration of an ROI setting window expanded by a click operation of the ROI setting icon of FIG. 7, and FIG. 19 is a configuration of a calculated feature menu window expanded by a click operation of a feature selection button of FIG. FIG. 20 is a block diagram showing the configuration of a batch selection menu window developed by clicking the batch selection button in FIG. 19, and FIG. 21 is a site designation developed by clicking the site setting button in FIG. FIG. 22 is a configuration diagram showing a configuration of a menu window, FIG. 22 is a configuration diagram showing a configuration of a feature value calculation result display window developed by a click operation of a feature value calculation execution button in FIG. 18, and FIG. 23 is a diagram based on the ROI setting icon in FIG. FIG. 24 is an explanatory diagram for explaining a method of setting an ROI for an original image and a processing result image. FIG. FIG. 25 is a flowchart showing the flow of a feature amount calculation process using an output button, FIG. 25 is an explanatory diagram illustrating sampling pixels extracted from G image data in the feature amount calculation process of FIG. 24, and FIG. FIG. 27 is a configuration diagram showing the configuration of an inter-ROI mutual feature value menu window developed by operation.
FIG. 28 is an explanatory diagram for explaining the inter-ROI mutual feature calculated in the inter-ROI mutual feature menu window of FIG. 26;
26 is a flowchart for explaining the operation of automatic combination in the calculation of the inter-ROI mutual feature in the inter-ROI mutual feature menu window of FIG. 26, and FIG. 29 is a feature developed by clicking the feature amount calculation icon of FIG. FIG. 30 is a configuration diagram showing the configuration of the calculation window, and FIG. 30 is an RO diagram developed by clicking the ROI designation button in FIG.
FIG. 31 is a block diagram showing a configuration of a discrimination / classification window developed by a click operation of a discrimination / classification icon in FIG. 7, and FIG. 32 is a block diagram showing a configuration of a data set creation button in FIG. FIG. 33 is a configuration diagram showing a configuration of a data set creation window to be expanded, FIG. 33 is a configuration diagram showing a configuration of a class-specific ROI list window expanded by a click operation of a class-specific ROI selection button in FIG. 31, and FIG. FIG. 35 is a configuration diagram showing a configuration of a data set list window expanded by a click operation of an existing data set call button.
FIG. 36 is a configuration diagram showing a configuration of a discrimination / classification execution window developed by a click operation of a discrimination / classification execution button 1;
FIG. 3 is a configuration diagram showing a configuration of a discrimination / classification result display window developed by a click operation of the execution button in FIG. 35;
7 is a configuration diagram showing a configuration of a report creation window expanded by a click operation of a report creation icon in FIG. 7, and FIG. 38 is a configuration diagram showing a configuration of a feature report creation window expanded by a click operation of a selection button in FIG. FIG. 39 and FIG. 39 are configuration diagrams showing a configuration of a feature amount report display window developed by a click operation of the display button of FIG. 38. FIG. 40 is a configuration of a graph display window developed by a click operation of the graph creation button of FIG. FIG.

As shown in FIG. 1, a diagnosis support apparatus 1 according to the present embodiment includes, for example, a video processor 2 that obtains an image signal from an electronic endoscope (not shown) and converts it into a video signal. An observation monitor 3 for projecting a video signal, an input unit 4 for converting a video signal from the video processor 2 into image data for signal processing, image data signal-processed by the input unit 4, and lossless or irreversible compression of this image data Server unit 5 for storing the compressed image data, and searching and displaying the image data or the compressed image data stored in the server unit 5, a region of interest (ROI) setting process, a feature amount calculation process, a discrimination classification process, etc. And a conference unit 6 for performing a series of diagnosis support processes.

The input unit 4 is an A / D converter 1 for converting an analog RGB video signal as a video signal from the video processor 2 into image data as a digital signal.
1, an image processing unit 12 having a memory for storing image data and generating an image file to which management information from the video processor 2 has been added, and a LAN (Local Area Network) cable 4
a LAN controller 13 for transmitting the image data to the server unit 5 via the control unit a, and a controller 14 for communicating image data management information and the like with the video processor 2 and controlling the image processing unit 12 and the LAN controller 13.

The video processor 2 has a video signal output terminal,
An analog RGB video signal output terminal and a communication signal output terminal are provided. The video signal output terminal is connected to the observation monitor 3, the analog RGB video signal output terminal is connected to the input terminal of the A / D converter 11, and the communication signal output terminal is connected. The end is connected to the controller 14.

The output terminal of the A / D converter 11 is
It is connected to the data signal end of the image processing unit 12.

A control signal terminal and a data signal terminal of the controller 14 are connected to a control signal terminal of the image processing unit 12 and the LAN controller 13 by a bus line 14a. The control of the image processing unit 12 and the LAN controller 13 by the controller 14 is performed by a signal via the bus line 14a.

An image observed by an electronic endoscope (not shown) converted into a video signal by the video processor 2 is, for example,
The image is displayed on the observation monitor 3 as an observation image. When the operator of the video processor 2 determines that the above-described observation image needs to be recorded, the video signal described above is converted into an analog RGB video signal by A / A.
The output from the A / D converter 1
Numeral 1 performs a predetermined quantization process on an analog RGB video signal, converts the analog RGB video signal into a digital RGB video signal, and outputs the digital RGB video signal to the image processing unit 12 as observation image data.

The image processing section 12 stores observation image data input from the A / D converter 11 under the control of the controller 14.

The controller 14 performs various data processing such as reduction processing on the observation image data stored in the image processing unit 12, further adds management information to form an image file, and temporarily stores the image file in the image processing unit 12. Or to the LAN controller 13. When the image file subjected to various data processing is temporarily stored in the image processing unit 12 as described above, the controller 14 outputs the image file from the image processing unit 12 to the LAN controller 13 at a predetermined timing. It has become.

The server unit 5 has the L of the input unit 4
A LAN controller 21 for receiving an image file sent from the AN controller 13, a memory 22 for temporarily storing the image file received by the LAN controller 21, and a large-capacity storage medium for storing the image file received by the LAN controller 21; Hard disk 2
, A compression device 25 for reversibly or irreversibly compressing the image file received by the LAN controller 21 and sending the compressed image data to the hard disk driver 24, a LAN controller 21, a memory 22, and a hard disk driver. And a controller 26 for controlling the compression device 25. The hard disk driver 24 records the image file and the compressed image data on the hard disk 23, and the LAN controller 21
The image file or the compressed image data recorded on the hard disk 23 can be transmitted to the conference unit 6 via a.

The LAN controller 13 of the input unit 4
End of the LAN cable 4a is the LAN of the server unit 5.
It is connected to the controller 21. The LAN cable 4a is a so-called 10baseT cable, and is capable of performing bidirectional data communication of 10 Mbit / sec within a range of 100 m or less using a twisted pair wire, and performs control between a plurality of devices and transmission and reception of data. Is what you can do.

A control signal terminal and a data signal terminal of the controller 26 of the server unit 4 are connected to control signal terminals of the memory 22, the LAN controller 21, and the hard disk drive 24 by a bus line 26a.

In the server unit 5, the image file from the LAN controller 13 of the input unit 4 is input by the LAN controller 21 via the LAN cable 4a.
The controller 26 temporarily stores the image file input via the LAN controller 21 in the memory 22.
The hard disk drive 24 stores the image file on the hard disk 23, for example.

The conference unit 6 stores an image file or compressed image data from the LAN controller 21 of the server unit 5 via the LAN cable 5a, and stores the image file or compressed image data received by the LAN controller 31. An image processing unit 32, a decompression device 33 that decompresses the compressed image data stored by the image processing unit 32, and image data that is a digital signal in the image file stored by the image processing unit 32 and decompressed by the decompression device 33. D that performs inverse quantization on image data and converts it into an analog RGB video signal
/ A converter 34, an observation monitor 35 for displaying analog RGB video signals converted by the D / A converter 34, and a controller 36 for controlling the image processing unit 32
And

The data signal end of the image processing unit 32 is D / A
The output terminal of the D / A converter 34 is connected to an observation monitor 35. A control signal end and a data signal end of the controller 36 are connected to the image processing unit 32 and the LA by a bus line 36a.
It is connected to a control signal terminal with the N controller 31.
The control signal end and the data signal end of the controller 36 are:
The bus line 36b is connected to a control signal terminal for a CPU 41 described later.

The conference unit 6 has a CPU 41 for controlling the controller 36 and a keyboard 42 for inputting, for example, a request for searching for an image file to the server unit 5 and for inputting various information along with the image file. And the signal of the keyboard 42 and the CP
A keyboard interface (hereinafter, referred to as a keyboard I / F) 43 for matching with the signal of U41, and a search monitor 4 for displaying information input by the keyboard 42
4, a mouse 45 for giving an instruction to move the cursor coordinates on the screen of the search monitor 44 to an arbitrary position, and a mouse interface (hereinafter referred to as a mouse I / F for matching the signal of the mouse 45 with the signal of the CPU 41). 46)
A hard disk 47 which records an execution program of the CPU 41 and various data such as image data of a menu screen of the search monitor 44; and a hard disk interface (hereinafter referred to as a hard disk I / F) for matching a signal of the hard disk 47 with a signal of the CPU 41. 4)
8, a working memory 49 used as various processing work areas of the CPU 41, and an image processing unit 5 including a memory for storing digital RGB video signals to be displayed on the search monitor 44.
0 and a printer 51 for printing information from the CPU 41
And a printer interface (hereinafter, referred to as a printer I / F) 5 for matching the printer 51 and the CPU 41.
2 is provided.

Further, the conference unit 6
A password storage unit 61 that stores a password out of information input from the keyboard 42 by the PU 41, a password monitoring unit 62 that determines the level of the password stored in the password storage unit 61, and a password monitoring unit 62 based on the determination result. And a control restriction unit 63 for restricting the control of the CPU 41.

The control signal end and the data signal end of the CPU 41 are connected to the hard disk I / F 4 by a bus line 41a.
8, the mouse I / F 46, the keyboard I / F 43, the working memory 49, the image processing unit 50, and the control signal terminal and the data signal terminal of the password monitoring unit 61. Then, the CPU 41 controls the hard disk I / F 48, the mouse I / F 46, and the keyboard I / F 46 via the bus line 41a.
The F43, the printer I / F 52 and the working memory 49 are controlled.

The mouse I / F 46 detects a signal corresponding to the physical relative movement amount of the mouse 45 and outputs the signal to the work memory 49. The work memory 49 stores the movement amount described above.

The keyboard I / F 43 includes a keyboard 42
Is output to the working memory 49, and the working memory 49 stores the above-described character information and the like.

The hard disk I / F 48 reads a program executed by the CPU 41 and image data for the search monitor 44 such as a menu screen from the hard disk 47 and outputs the read data to the work memory 49. Image data and the like are stored.

The printer I / F 52 transmits information transmitted from the CPU 41 to the printer 51 and performs printing.

As described above, the CPU 41 loads the program stored in the hard disk 47 into the work memory 44 when the power is turned on, and operates according to the program.

As described above, the image file stored in the hard disk 23 of the server unit 5 is transferred to the hard disk driver 14 under the control of the controller 26.
And stored in the memory 22 once or directly to L
Output to the AN controller 21. An image file is input from the LAN controller 21 to the LAN controller 31 of the conference unit 6 via the LAN cable 5a.

In the conference unit 6,
The LAN controller 31 is controlled by the controller 36.
Outputs an image file to the image processing unit 32. The image file from the hard disk 23 of the server unit 5 stored in the image processing unit 32 is separated into digital observation image data and management information under the control of the controller 36, and the digital observation image data is stored in the image processing unit 32. The management information is sent to the CPU 41 via the bus line 36b.

The image processing section 32 stores the above-described observation image data. As described above, the observation image data, which is a digital signal, stored in the image processing unit 32 is converted into an analog RGB video signal by the inverse quantization of the D / A converter 34, and is output to the observation monitor 35. ing. The observation monitor 35 projects the input analog RGB video signal as described above. If the observation image data is compressed image data, it is output to the D / A converter 34 after being expanded by the expansion device 33.

The CPU 41 includes a cursor by a mouse 45, character information by a keyboard 42, a hard disk 4
7 is processed so that the image data for the search monitor 44 such as the menu screen from 7 and the management information are combined or displayed alone, and are stored in the image processing unit 50 as image data.

In the observation image data of the image file recorded on the hard disk 23, for example, a color observation image is divided into 640 dots in the horizontal direction and 480 dots in the vertical direction. 8
It is composed of a predetermined number of bytes quantized to be bits.

Next, details of the diagnosis support processing according to the present embodiment will be described. The diagnosis support processing is executed by the CPU 41 using a diagnosis support processing execution program recorded on the hard disk 47. The diagnosis support processing is operated by input using the keyboard 42 and the mouse 45 in a multi-window environment displayed on the search monitor 44.

As shown in FIG. 2, the diagnosis support processing execution program 70 includes an image management block 71, a database management block 72, an ROI setting block 73, a feature amount calculation block 74, a discrimination and classification block 75, a report creation block 76, and an image It consists of a processing block 77.

Subsequently, the diagnosis support processing execution program 70
The outline of each block that constitutes will be described.

The image management block 71 manages and searches for endoscope images used in the diagnosis support processing.

The database management block 72 manages a database for storing and managing images used in the diagnosis support processing, ROIs to be set, calculated feature amounts, and the like.

The ROI setting block 73 sets an ROI on the endoscope image for calculating a characteristic amount and making a classification.

The feature value calculation block 74 calculates a feature value by applying a feature value calculation method to the ROI set by the ROI setting block 73.

The discrimination / classification block 75 performs a discrimination / classification process using the feature amount calculated in the feature amount calculation block 74.

The report creation block 76 creates a report by displaying a list of the processing results obtained by the feature amount calculation block 74 and / or the discrimination and classification block 75 and plotting them on a graph.

The image processing block 77 applies, for example, image processing such as noise removal processing and structural component enhancement processing to the endoscope image which is the original image used for the diagnosis support processing.

The processing result image by the image processing block 77 is recorded on the hard disk 23 in the server unit 5 like the original image, and can be used as data to be subjected to diagnosis support processing.

Here, in the conference unit 6, each block can operate in parallel by multi-processing on a multi-window.

FIG. 3 is an explanatory diagram for explaining an example of a multi-window. Initially, icons 81, 82, and 83 for opening windows for executing various functions are displayed on the monitor screen 80 of the search monitor 44.
The operator clicks with the mouse pointer 84 that moves in conjunction with the mouse 45 in order to open the window of the target function. The click indicates an operation in which the mouse pointer 84 is overlaid on an icon or the like and the button of the mouse 45 is pressed.

The icons clicked by the mouse pointer 84 open one or more windows for executing the corresponding function. FIG. 4 is an explanatory diagram for explaining an example of an opened window. Icons 81, 82, and 83 in FIG. 3 indicate windows 85, 86, and 87 as shown in FIG.
4 on the monitor screen 80.

Thereafter, on the monitor screen 80, operations such as inputting on a window on which the mouse pointer 84 is superimposed (hereinafter, such a window is referred to as an active window, and a window 86 is an active window in FIG. 4). Is performed. Also, by moving the mouse pointer 84, an arbitrary window can be selected as an active window.

FIGS. 5 and 6 are explanatory diagrams for explaining an example of the configuration of the above-mentioned window. In FIG. 5, a window 90 includes a button 91 for designating the operation of a function and the like. An output display area 92 for displaying an output from the system side in a function related to 90, a menu bar 93 for selecting an item based on the function, and a scroll bar 9 for scrolling the menu bar 93
4. An input display area 95 for displaying an input from the keyboard 42 and an end button 96 for closing the window 90 when the operation is completed.

When the mouse pointer 84 is overlaid on the button 91 and clicked, a corresponding function can be operated. It is also possible to control so that another new window (not shown) is opened when the button 91 is selected, for example.

In the output display area 92, messages such as texts and numerical data are displayed from the system side.

When any input is made from the keyboard 42, the input information can be displayed in the input display area 95.

When all the operations are completed, the window 90 can be closed by placing the mouse pointer 84 on the end button 96 and clicking on it.

In the menu bar 93, a desired one is selected by superimposing the mouse pointer 84 on a bar (the part numbered 1, 2, 3,... In FIG. 5) indicating each item and clicking. be able to. At this time, an operation such as reverse display is applied to the clicked bar, so that the operator can be notified of which is specified.

When the number of selectable items is large with respect to the display range of the menu bar 93 (in the example of FIG. 5, for example, when there are ten items), the scroll bar 94 is used. be able to. That is,
When the mouse pointer 84 is overlaid on the upper scroll button 94a of the scroll bar 94 and the button of the mouse 45 is pressed, the menu bar 93 is scrolled upward, and when pressed over the lower scroll button 94b of the scroll bar 94, the menu bar 93 is scrolled downward. In the example of FIG. 5, it is possible to display and select item 6 and subsequent items by using the downward scroll button 94b.

The scroll pointer 94c is a guide for knowing which part of the scroll bar 94 is displayed on the menu bar 93 in all items.

As shown in FIG. 6, it is also possible to arrange a menu bar item in a two-dimensional manner and provide a scroll bar for scrolling in the horizontal direction. That is, in FIG. 6, the window 100 includes a menu bar 101 in which items are arranged two-dimensionally, a scroll bar 102 for scrolling vertically and a scroll bar 103 for scrolling horizontally.

The operator can select any item by scrolling the menu window 100 in the horizontal and vertical directions in the same manner as the operation on the scroll bar 94 in FIG.

The window according to the present embodiment has buttons, menu bars, scroll bars, output display areas, and input display areas of any size and number, which operate as described above. In the following description, unless otherwise specified, it is assumed that when the end button is clicked, the corresponding window is closed.

FIG. 7 is an explanatory diagram for explaining the calling operation of each function of the diagnosis support processing according to the present embodiment.

First, on the search monitor 44, a main screen 110 on which icons serving as a main menu of the diagnosis support processing are arranged is displayed. This main screen 110
Is an image display icon 111 for calling an image display function,
Database icon 112 for calling the database customization function, ROI setting icon 113 for calling the ROI setting function, feature amount calculation icon 114 for calling the feature amount calculation function, discrimination classification icon 115 for calling the discrimination classification function, report creation icon 116 for calling the report creation function And an image processing icon 117 for calling an image processing function. The above icons 111-
In the diagnosis support processing execution program, the functions executed by the 117 are the image management block 71 for the image display function, the database management block 72 for the database customization function, and the ROI setting function for the ROI setting function, as shown in FIG. The setting block 73 operates the feature amount calculation block 74 for the feature amount calculation function, the discrimination and classification block 75 for the discrimination and classification function, the report creation block 76 for the report creation function, and the image processing block 77 for the image processing function.

Each function can be operated in parallel by multiprocessing. In addition, each function has a subroutine for each operation, and an arbitrary subroutine can be called from a different function as needed.

Next, the operation of each block in the diagnosis support processing execution program will be described in detail.

(Image management block) Image management block 7
As described above, reference numeral 1 mainly manages and searches for an endoscope image used in the diagnosis support processing. The endoscope image file recorded on the hard disk 23 of the server unit 5 is managed for each endoscopic examination of a target patient.

As shown in FIG. 8, the endoscope image file recorded on the hard disk 23 is managed for each patient ID and examination date (and patient name) by file management data. Numbers are assigned to the units for microscopy. Inspection No. Is set for each examination and the patient ID is set for each patient, so that there is no duplication.

FIG. 9 is an explanatory diagram for explaining management data provided for each endoscopic examination of each endoscope image in the endoscope image file. In this example, examination N
o. It is assumed that seven pieces of endoscope image data are managed as 1. As the management data, in addition to the above-mentioned patient ID, patient name, and examination date, for example, examination site, diagnosis name,
It can include data such as notes and attending physicians. These data are recorded on the hard disk 23 and used as keys in an image search process described later.

By selecting the image display icon 111 with the mouse pointer 84 (see FIG. 7), the image management block 71 opens an image management data window 120 as shown in FIG. The image management window 120 includes an input display area 121 for inputting various data, a call button 122 and an end button 123 used when re-editing management data that has already been given.

The operator places the mouse pointer 84 on the input field of the desired management data and inputs necessary information from the keyboard 42. Inspection No. If only the input button 122 is clicked and the call button 122 is clicked, the management data of the corresponding endoscopy is displayed in the input display area 121. After the management data is called, the operator can edit desired management data.

In FIG. 10, when the call button 122 is clicked with the mouse 45, a search process is executed.
In this search processing, a search window 130 is first opened as shown in FIG. The search window 130 includes an input display area 131 for inputting a search key, a condition search start button 132 for designating start of a condition search using the input search key, and a full search for displaying all endoscopy data. It consists of an all search start button 133 and an end button 134 for designating the start. In this example, it is assumed that a patient ID, a patient name, an examination site, and a diagnosis name can be used as search keys.

When performing an all search in the search process, the operator clicks the all search button 133 with the mouse 45.
Further, when performing a condition search, the operator is required to enter the input display area 13.
The user inputs a desired key to the number 1 using the keyboard 42, and clicks the condition search start button 132 with the mouse 45.

A search is executed by these operations.
If there is endoscopic examination data obtained as a search result, a new search result display window 140 as shown in FIG. 12 is opened to display the search result.

A search result display window 140 includes a menu bar 141 for displaying management data (in this example, an examination number, a patient ID, a patient name, and an examination date) of the endoscopic examination data as a search result, and a scroll bar. A bar 142, an image list button 143 for designating a list display of endoscopic images recorded in a desired endoscopic examination, and a confirm button 144 used when selecting an endoscopic examination in each process described later. And an end button 145. After the operator designates a desired endoscopic examination by clicking on the menu bar 141, and further clicks the image list button 143, a list of endoscopic images recorded in the corresponding endoscopic examination is displayed in FIG. An image list display window 150 as shown is opened.

For the endoscopic image recorded based on the endoscopic examination displayed in the search result window 140, the display of the endoscopic image, image processing, setting of the ROI, and features described later are performed. Each process such as the amount calculation and the classification is applied.

The image list display window 150 is shown in FIG.
As shown in the figure, a data display area 151 for displaying management data in the endoscopic examination data and the number of recorded endoscope images, and the like, and an image No. assigned to each recorded endoscope image. And the like, a menu bar 152, a scroll bar 153, and a display start for starting display of an endoscopic image selected by clicking on the menu bar 152, by specifying a desired endoscopic image by displaying the same. It comprises a button 154 and an end button 155.

After the operator clicks and selects an arbitrary item (here, an endoscope image) on the menu bar 152,
By clicking the display button 154, an image display window 160 for displaying the selected endoscope image as shown in FIG. 14 is opened.

As shown in FIG. 14, the image display window 160 is used for image selection and the like in an image display area 161, a data display area 162 for displaying management data and the like, and an ROI setting process described later. Confirm button 1
63 and an end button 164.

The operator can check whether the final search result is a desired endoscope image by looking at the endoscope image displayed on the image display window 160.

When all the operations on the image management data window 120 shown in FIG. 10 have been completed, the image management data window 120 is closed by clicking the end button 123 with the mouse 45.

When the display button 154 is clicked in FIG. 13, the image display window 160 is opened and the control is performed so that the selected endoscope image is displayed on the observation monitor 35. Good.

The above-described search processing can be executed at any time when an endoscope image search is required from each block shown in FIG. That is, a search instruction button for instructing to open the search window 130 in FIG. 11 can be set on a window (not shown) for executing a function in each block.

(Image Processing Block) Next, the operation of the image processing block 77 will be described. Image processing icon 1
By selecting 17 with the mouse pointer 84 (see FIG. 7), the image processing block 77 opens an image processing window 170 as shown in FIG. The image processing block 77 applies desired image processing to the endoscope image recorded on the hard disk 23 as described above.

The operation of the image processing block 77 is designated on the image processing window 170. The image processing window 170 includes a menu bar 171 including a list of image processing to be executed, a scroll bar 172, and a search button 173 for designating execution of the above-described search processing.
And an execution button 174 and an end button 175 for designating the start of execution of one or more image processes designated on the menu bar 171.

The operator designates a desired image processing name on the menu bar 171 by clicking. Image processing includes, for example, noise removal processing, structural component enhancement processing, and binarization processing.

By clicking the search button 173, the above-described search window 130 is opened (see FIG. 11). After the above-described series of search processing, the operator clicks the menu bar 141 and the confirm button 144 in the search result display window 140 to select the endoscope image to which the selected image processing is applied, and press the end button 1
By clicking on 34, the image processing window 170
(See FIG. 12).

After the selection of the image processing name to be applied and the endoscope image are completed, the processing is executed by clicking an execution button 174 in FIG. Menu bar 17
Each image processing displayed on 1 is stored in the hard disk 47 as a subroutine program. The processing result image that is the execution result is displayed on the newly opened processing result image display window. As shown in FIG. 16, the processing result image display window 180 for displaying the processing result image includes a processing result image display area 181, an image management data display area 182, and a hard disk 2 for the processing result image.
Recording start button 183 for instructing the start of recording to No. 3
And an end button 184.

The image management data display area 182 displays the management data described above, the processing name of the applied image processing, and the like. When the recording start button 183 is clicked, the processing result image is recorded on the hard disk 23 together with the management data.

If the recording to the hard disk 23 is not performed, the processing result image display window 180 may be closed by clicking the end button 184. In this case, the processing result image is rejected.

(Database Management Block) Next, the database management block 72 will be described. The database management block 72 is a database for managing the endoscopy, the endoscopic image, the set ROI, the processing result in each function, and the like in the diagnosis support apparatus 1.

The database managed by the database management block 72 is, as shown in FIG.
0, patient management data 191, processing result image management data 1
92, image processing method data 193, image management data 19
4. ROI management data 195, site data 196, finding data 197, classification data 198, feature amount calculation method data 199, inter-ROI mutual feature amount data 200, ROI feature amount data 201, discriminant classification method data 202, data set name data 203, used feature data 204, classification class data 205, class-specific sample data 20
6. Discrimination classification coefficient data 207, discrimination classification result data 2
08, discriminating boundary value data 209,
Each data is stored as a file on the hard disk 23
Is stored in

The examination data 190 is a database for managing data given for each of the above-mentioned endoscopic examinations, which is a management unit of an endoscope image.

The patient management data 191 is a reference database that provides data such as a patient name, a date of birth, and a patient ID that are assigned to each patient.

The image management data 194 is a database that manages data given to each endoscope image managed under each endoscopic examination.

The processing result image management data 192 is a database for managing data added to the processing result image to which the above-described image processing has been applied.

The image processing method data 193 is a reference database for giving the type of image processing method.

[0108] The ROI management data 195 is a database for managing data given to each of the ROIs set by a series of processes described later for the endoscope image or the processing result image.

The site data 196 is a reference database for giving a diagnosis site to the ROI management data.

The finding data 197 is a reference database for giving a finding attribute to the ROI management data.

The classification data 198 is a reference database for giving a diagnosis classification to the ROI management data.

The feature amount calculation method data 199 is a reference database that gives the name of the feature amount calculation process applied to the set ROI.

The inter-ROI mutual feature data 200 is a database for managing the inter-ROI mutual feature calculated by a series of processes described later.

The ROI feature data 201 is a database for managing ROI feature calculated by a series of processes described later.

The discrimination / classification method data 202 is a reference database that gives the names of the discrimination / classification processing methods using the calculated feature amounts.

The data set name data 203 is a database for managing a discriminant classification system described later based on the discriminant classification method.

The used feature data 204 is a database for managing the names of the features used in the generation of the discriminant classification system.

The classification class data 205 is a database for managing the class (diagnosis classification in this embodiment) to be classified by the generated data set of the discrimination classification system.

The class-specific sample data 206 is a database for managing the ROI used as the teacher data by the data set in generating the discriminant classification system.

The discriminant classification coefficient data 207 is a database for managing each coefficient value in the generated discriminant classification system.

The discrimination / classification result data 208 is a database for managing the result obtained by applying the discrimination / classification system generated to the set ROI.

The discrimination boundary value data 209 is a database for managing a value that is a discrimination classification reference of the generated discrimination classification system.

Here, the reference database includes, as items, names of diagnostic sites (esophagus, stomach, large intestine, etc.), findings (redness, fading, bumps, etc.), and diagnostic classifications (normal, ulcers, early cancer, etc.). It is a database. The reference database is a database that can be customized by an operator at any time, such as addition, change, and deletion of items.

On the other hand, in databases other than the reference database, items are generated when data to be managed is generated. For example, when an endoscopic image is recorded by a new endoscopic examination, a new item is additionally generated in the examination data 190 and the image management data 194. Also, when data is updated or deleted, the corresponding item is updated or deleted.

Each data is based on its related contents,
The connection is made by referring to the primary numbers serving as the respective keys. The primary number is a number assigned one-to-one for each item in each database. Thus, various data such as examinations, patients, images, ROIs, and the like are associated.

The database management block 72 controls a series of operations such as opening, reading, rewriting, storing, and closing each data when the appearance of data to be made into a database or a change in data contents occurs.

The database icon 112 shown in FIG.
By clicking, the patient management data 191, the site data 196, the finding data 197, and the classification data 19
8 enables customization such as addition, change, and deletion of items. In this case, a database customization window (not shown) is opened, and the operator performs operations such as data to be customized, addition, change, and deletion on the window, and ends the operation by clicking the confirmation button and the end button. Database management block 72
Rewrites the corresponding data by clicking on the confirm button.

(ROI Setting Block) Next, the operation of the ROI setting block 73 will be described. When the execution of the ROI setting process is selected by clicking the ROI setting icon 113 in FIG. 7, the image management data window 120 shown in FIG. 10 is opened. Hereinafter, ROI is performed according to the above-described procedure.
The endoscope image for which the setting is made is selected on the menu bar 141 on the search result display window 140 as shown in FIG. Subsequently, by clicking the confirm button 144, a new ROI setting window is opened.

As shown in FIG. 18, an ROI setting window 220 includes an ROI display area 221 for performing image display and ROI drawing work, an ROI number selection button 222 for designating an ROI number to be set, and a feature amount to be calculated. A feature amount selection button 223 for designation, a part setting button 224 for setting a part, a finding, and a classification for the set ROI, a finding setting button 225, a classification setting button 226, and calculation of a feature amount Button 227, a ROI deletion button 228 used for re-setting the ROI, a confirm button 229 for instructing registration of the set ROI in the database, an end button 230, and a drawing line of the ROI in the image display area. Display designation buttons 231 and R for designating display or non-display of I made from the non-display designation button 232.

The operator proceeds with the operation by clicking each function button on the ROI setting window 220 using the mouse 45. In this embodiment, it is assumed that a maximum of five ROIs can be set on one endoscope image.

First, an endoscope image to be set as an ROI is displayed in the image display area 221 of the ROI setting window 220. After confirming the displayed endoscope image, the operator presses the ROI number selection button 2 with the mouse pointer.
Click 22. The ROI number selection button 222 indicates the numerals 1 to 5, and the RO to which each is set
Corresponds to I numbers (1) to (5).

Subsequently, on the image display area 221,
The ROI is drawn by operating the mouse 45. The drawing of the ROI may be controlled, for example, by moving the mouse pointer 84 while holding down the button of the mouse 45 and displaying the locus thereof. One of the corresponding ROI numbers (1) to (5) is superimposed and displayed on the image display area 221 on the drawn ROI.

Further, the setting of the ROI is not limited to the drawing of an arbitrary shape, and may be set by a rectangle such as a square or a rectangle. The size (size) of the rectangle in the horizontal and vertical directions is variable by designating by the operator or by selecting from a group of initial setting values increased by, for example, 5 pixels. In this case, it is assumed that a “rectangle / arbitrary drawing selection button” and a “rectangle size designation button” (both not shown) are provided on the ROI setting window shown in FIG. The operator moves the mouse pointer 84,
It is possible to set a rectangular ROI at a desired position on the image displayed in the image display area 221. On the image display area 221, a rectangle is displayed at the same time as the mouse pointer 84, and the rectangle is controlled to move with the movement of the mouse pointer 84.

The shape of the ROI is not limited to a rectangle.
An arbitrary fixed ROI such as a circle or another polygon may be set to an arbitrary size.

Next, a feature value to be calculated in the set ROI is designated. This operation is performed by clicking the feature amount selection button 223. Feature amount selection button 223
When is clicked, the calculation feature amount menu window 240 shown in FIG. 19 opens.

The calculation feature amount menu window 240
A menu bar 241 for displaying names of various processing methods and names of feature amounts calculated from the respective methods, and a scroll bar 242
, A mutual feature calculation button 243 for specifying a mutual feature between ROIs to be described later, a collective selection button 244 for specifying batch selection of the feature also described later, Quantity menu window 24
It is composed of a decision button 245 for closing 0.

[0137] The menu bar 241 displays the diagnosis support device 1.
Are displayed, and an item including a name of a feature amount obtained by applying each processing method is displayed. There are various processing methods such as a method of evaluating a color tone based on R, G, and B image data of each pixel included in a set ROI, a method of breaking a texture using a density co-occurrence matrix, and the like. Can be considered. The calculation is designated by clicking a corresponding item on the menu bar 241 using the mouse 45.

As described above, by making it possible to designate an arbitrary feature amount from among the feature amounts that can be calculated, it is possible to save the calculation time and the storage area of the calculation result due to the calculation of the unnecessary feature amount. Becomes possible.

In the diagnosis support processing according to the present embodiment, not only is it possible to specify the feature to be calculated not only by clicking each item of the menu bar 241 but also by using the collective selection button 244. Can be done.

When the collective selection button 244 is clicked, a new collective selection menu window is opened.

As shown in FIG. 20, a collective selection menu window 250 includes a combination number button 251 for selecting a combination of feature amounts to be calculated, and a feature amount item designated by the combination number button 251. A feature amount display area 252 to be displayed, an all-selection button 253 for designating calculation of all feature amounts, a confirm button 254, and an end button 255 for ending the operation on the collective selection menu window 250 Have been.

In the collective selection menu window 250,
Several combinations (three in the present embodiment) of the feature amounts used in the subsequent classification and classification processing are set in advance. That is, by clicking the buttons numbered 1 to 3 of the combination number buttons 251 and the confirm button 254, it is possible to always calculate the minimum necessary feature amount. Therefore,
It is possible to realize a significant saving in the calculation time required for calculating the unused feature amounts and the storage area for storing the calculated values.

When the all-selection button 253 is clicked, all the feature values on the menu bar 251 are selected. As a result, when all the feature amounts are required, the labor for clicking each item indicating the feature amount to be calculated is omitted.

It should be noted that the above-described specification of the calculated feature amount is as follows.
Since it can be performed at the time of executing a feature amount calculation process and a discrimination / classification process, which will be described later, it may be omitted (no designation) at the time of setting the ROI.

Subsequently, the part of the ROI to be set is specified. For example, when the site in the endoscopy is the stomach, the site is specified in the setting of the ROI.
Details such as the pylorus are given. In the ROI setting window 220 shown in FIG. 18, by clicking the region setting button 224, a new region designation menu window is opened.

As shown in FIG. 21, the part designation menu window 260 includes a menu bar 261 for displaying items in the part data 196 of FIG. 17, a scroll bar 262, and a decision button 263 for confirming the part setting. I have. The confirm button 263 also functions as an end button, and closes the site designation menu window 260 when clicked. The operator selects the name of the desired part by clicking the menu bar 261, and confirms by clicking the confirm button 263.

The ROI setting window 22 shown in FIG.
By clicking the “0” finding setting button 225 and the classification setting button 226, the finding and classification of the ROI to be set are specified in the same manner as the designation of the part. To specify a finding and a classification, a finding specification menu window and a classification specification menu window (both not shown) are opened. At this time, the point that each menu window is different from the contents of the part designation menu window 260 is that the menu bar 26
1 shows the finding data 197 and the classification data 198, respectively.
(See FIG. 17).

In the designation of a part, a finding, and a classification, "not set" is given as an initial value. This makes it possible to omit the specification of each item and perform only the drawing of the ROI.

When the setting of one ROI is completed by the above series of operations, the user clicks the OK button 229 to set the ROI.
New data is added to the OI management data 195 (see FIG. 17).

Although the operation of each button and the like has been described in order for the sake of simplicity, the ROI setting includes the ROI number selection button 222 and the confirmation button 229.
May be clicked at the start of ROI setting and at the end of setting, respectively, and the work during that may be performed in any order.

When the setting of the ROI is completed, the feature amount can be calculated. The calculation of the feature amount is performed by the operation of a feature amount calculation block 74 described later, and not only the click of the feature amount calculation execution button 227 but also the click of the feature amount calculation icon 114 and the call from the determination classification block 75 (both are performed). This will be described later.

In the ROI setting window 220 in FIG. 18, when the feature amount calculation execution button 227 is clicked, if the feature amount to be calculated is selected, the feature amount calculation result display window 270 shown in FIG. 22 is displayed. open. If the feature to be calculated has not been selected, a message such as "Please select the feature to be calculated" is displayed.

As shown in FIG. 22, the feature amount calculation result display window 270 includes a display area 271 for displaying the value of the feature amount calculated for each ROI, a scroll bar 272 and an end button 273. Has become. In the display area 271, the inspection NO. , Patient ID, patient name, image No. , The number of the ROI, and the like, information about the ROI is displayed, and a value as a calculation result is sequentially displayed from the ROI for which the calculation is completed. In addition, when a plurality of ROI setting windows 220 are opened to set ROIs in parallel for a plurality of different endoscope images, a large number of ROIs are set.
When calculating the feature amount from the scroll bar 272
Can be used. After checking the calculation result, the operator clicks the end button 273 to close the feature amount calculation result window 270. The calculation result of the feature amount is added to the ROI feature amount data 201 (FIG. 17).
reference).

In the example of FIG. 22, the calculated feature values are displayed on one feature value calculation result display window 270. However, control is performed such that a different feature value calculation result display window is opened for each feature value. You may.

The ROI setting window 22 shown in FIG.
0, the ROI display designation button 231 and the ROI
When each of the non-display designation buttons 232 is clicked, control is performed so that the drawing lines of the ROI in the image display area 221 are displayed and erased. Thereby, the operator can check the endoscope image without the drawing line of the ROI at any time.

Note that the calculation of the characteristic amount is performed by the image processing block 7.
7 can be used as the processing result image. In this case, the set drawing position, region, findings, classification, and the like of the ROI are the same as those of the original image to which the image processing is not applied. Therefore, RO in the original endoscope image
When I is set or changed, control is performed so that the same data is added to the processing result image.
Further, when the ROI is set or changed in the processing result image, control may be performed so that the same data is added to the original image.

Conventionally, in the original image and the processing result image, R
When setting an OI or cutting out the same part for comparison and displaying it on one screen, the operator needs to separately specify information such as a coordinate position and an ROI size for each image. Was.

FIG. 23 is an explanatory diagram for explaining a method of setting an ROI for an original image and a processing result image in the present embodiment. In FIG. 23, (a) is an original image,
(B) shows the processing result images.

In the original image shown in FIG.
When a is determined, it is assumed that the ROI 1b is automatically set for the processing result image of FIG.
The ROI 1a and the ROI 1b have the same position, size, and shape of the ROI on the image.

On the contrary, if the ROI 1b in the processing result image of FIG. 23B is set first, the corresponding FIG.
RO1a is similarly set for the original image of (a).

That is, when an ROI is set in any one of the original image and one or more processing result images corresponding to the original image, it is considered that the same ROI is set in each of the related images. , So that necessary ROI management data and the like are added.

(Characteristic Value Calculation Block) Next, a characteristic value calculation method in the characteristic value calculation block 74 will be described. In the present embodiment, as a feature amount calculating method, G
A texture breaking method based on abor (Gabor) features is applied.

First, Gabor features will be described. The Gabor feature is a feature amount based on modeling of the human visual system, and is a method of analyzing frequency components in which local information depending on a position (coordinate) in an image is stored.
Since a more detailed explanation is described in the document “Fingerprint recognition by Gabor feature; Hamamoto et al .: Proceedings of the 26th Image Engineering Conference 1995, P.91 to P.94”, the outline is described here. I do.

The Gabor feature is a feature amount calculated from a value obtained by a convolution operation between the Gabor filter and the original image. The Gabor filter is obtained by multiplying a two-dimensional Gaussian surface by a plane wave transmitted in one direction on a two-dimensional plane, and has a standard deviation σx in the Gaussian surface.
And σy, the traveling direction θk of the plane wave, and the wavelength λm of the plane wave. The standard deviations σx and σy are closely related to the wavelength λm and can be a function of the wavelength λm, denoted as σx (λm) and σy (λm), respectively. The Gabor filter is a two-dimensional filter including a real part Re (f) and an imaginary part Im (f).

(Equation 1)

(Equation 2) Is defined by Note that the Gabor filter is obtained by the equation (1).
By changing σx (λm), σy (λm), λm and θk in (2), various characteristics can be realized.

Here, the wavelength λm defines the frequency component band extracted from the image by the Gabor filter, and the direction θk defines the direction.

Gabor features are obtained from the convolution of a Gabor filter with an image. Image size N × N
Multi-valued image I (i, j) (0≤i≤N-1,0≤j≤N-
1), and the sampling pixel on the image is (X, Y). The convolution of the image with the Gabor filter for a particular θk, λm is

(Equation 3) Given by Then, by using g in the equation (3), Gab
or the feature value h (X, Y, θk, λm)

H (X, Y, θk, λm) = | g (X, Y, θk, λm) | (4) where | z | is the absolute value of the complex number α + iβ (α ² + β ² )
Represents ^1/2 .

The Gabor feature quantity h described above
(X, Y, θk, λm) are created for each combination number (L) of a plurality of θk and λm, and calculated for M sampling pixels. The obtained feature value is used as a feature quantity (feature vector) having a feature dimension number M × L.

The direction θk can be determined if the filter is designed in the n direction.

Equation 5 θk = π (k−1) / n (k = 1,..., N) (radian) (5) For example, if there are four directions, 0, π / 4, π / 2, 3
π / 4 (radian) may be set. The wavelength λm is 2
^1/2 , 2 × 2 ^1/2 , 4 × 2 ^1/2 , 8 × 2 ^1/2 . Further, σx (λ) and σy (λ) may be determined, for example, as σx (λm) = σy (λm) = 0.5 × λm. It is also possible to make σx (λm) ≠ σy (λm).

FIG. 24 shows Gabor in this embodiment.
9 is a flowchart for explaining a feature amount calculation method based on features. In step S101, G image data Ig in the ROI set on the endoscope image is obtained. Here, for simplicity, the shape of the ROI has a size of N ×
It is assumed to be a rectangle of N. In step S102, it is determined whether or not each of the pre-processing such as noise removal, inverse γ correction, and shading correction is applied to the acquired G image data Ig. If so, the process proceeds to step S103. If not, the process proceeds to step S103. Proceed to S104.

In step S103, preprocessing such as noise removal by median filtering, inverse γ correction, shading correction and the like is applied to the G image data Ig. Note that each of the preprocessings may be selectively applied.

In step S104, the G image data Ig is subjected to Ga filtering by the Gabor filter described above.
Calculate the bor feature. In the present embodiment, the above-mentioned Gabor feature is defined as follows: θk = (0, π / 4, π / 2, 3π / 4) (a) λm = (3 × 2 ^1/2 , 6 × 2 ^1/2 ) (B) It is assumed that L = 8 Gabor filters designed for σx (λm) = σy (λm) = 0.5 × λm (c) are used to calculate M sampling pixels from G image data Ig.

FIG. 25 is an explanatory diagram for explaining how to obtain M sampling pixels in the present embodiment. Here, as shown in FIG. 25, G of size N × N
For the image Ig, a total of 25 sampling pixels of 5 × 5 at intervals D are set, and the Gabor feature by the L Gabor filters is calculated.

Returning to FIG. 24, in the subsequent step S105, the M × L (=
120) Gabor features hij (i = 1, 2,...,
M; j = 1, 2,..., L) are set as feature amounts, each value is displayed in a feature amount calculation result window (see FIG. 22), and stored in the ROI feature amount data 201.

As described above, by using the Gabor feature by applying the Gabor filter to the endoscope image, it is possible to calculate a feature amount storing local information depending on the position on the image. Using the obtained feature amounts, the texture indicating the structural pattern of the mucous membrane surface in the endoscopic image can be satisfactorily analyzed and classified.

The way of giving each of the direction θk, the wavelength λm, and the standard deviations σx and σy for designing the Gabor filter is not limited to the example of the present embodiment, but is adapted to the image to be applied. Etc. may be determined as appropriate. Also,
The directions θk may be set at unequal intervals.

It is of course possible to normalize the characteristics of the Gabor filter to be used. In this case, for example, each Gab is set so that the frequency response to the DC component in the frequency domain is 0 and the maximum value of the absolute value of the frequency response in the pass band is α (α is an arbitrary constant).
or filter may be changed. Further, such normalization may be applied to the calculated Gabor feature.

Further, in the present embodiment, the G image in the ROI is used as the target of the feature amount calculation.
And the same characteristic amount may be calculated from the B image and the B image. Also,
For example, the present invention can be applied to an image obtained by applying an operation to R, G, and B image data such as a luminance image.

Furthermore, the calculation of the Gabor feature is not limited to being applied to the M sampling pixels, but may be obtained from all the pixels included in the ROI.

Further, the shape of the ROI is not limited to a rectangle, and a similar feature can be obtained from the ROI in an arbitrary drawing shape.

Next, R in the feature amount calculation block 74
The inter-OI mutual feature will be described. The inter-ROI mutual feature indicates a feature calculated by a combination of two or more ROIs set in one endoscopic image.

For example, assuming that the difference between ROIs (1) and (2) and the average value of log (G / R) is used as the mutual feature, the inter-ROI mutual feature is calculated using the log (G) calculated from each ROI. G = R1−v2, which is the difference between the average values of G / R) v1 and v2. In addition to the difference calculation, the ratio v1 / v
2 and the like (for example, (v1-
1) A formula such as xv2 may be used.) 3 or more ROIs
May be used.

By introducing the mutual feature between ROIs, it is possible to evaluate the relative difference between different lesions and normal mucosa. For example, lesion 1 and lesion 2
When the respective color tones are compared with normal mucous membranes, the observer may determine that the former is a white tone and the latter is a red tone. This is based on the relative evaluation of the lesion site in one endoscopic image and the normal mucosa around it.

On the other hand, the condition for irradiating the observation light with the light source, which is the condition for capturing the endoscope image, differs for each endoscope image. Therefore, even when lesions of the same white tone or red tone are imaged, the compositions of the R, G, and B image data that constitute them differ. Therefore, since it is considered that the imaging conditions are relatively uniform in the field of view of one endoscope image, the relative values of the feature amounts calculated from a plurality of ROIs in the same image are obtained.
Discrimination and classification not affected by variations in imaging conditions can be performed.

In this embodiment, the mutual feature calculation button 24 is displayed on the calculation feature menu window 240 shown in FIG.
When 3 is clicked, the inter-ROI mutual feature menu window 280 shown in FIG. 26 is opened.

Inter-ROI Mutual Feature Menu Window 2
Reference numeral 80 denotes a menu bar 281 for designating the type of feature to be calculated, a scroll bar 282, a new bar 283 for designating the type of operation between ROIs, a scroll bar 284, and a reference class described later. A menu bar 285 for specifying, a scroll bar 286, and a batch selection button 287 for batch selection of mutual feature values
And a determination button 288 for determining the inter-ROI mutual feature to be calculated and closing the inter-ROI mutual feature menu window 280.

In the menu bar 281, similarly to the menu bar 241 in FIG. 19, a list of feature amounts that can be calculated is displayed, and the type of the feature amount to be calculated by clicking on the operator is set. At this time, if there is a feature amount set in the menu bar 241 in FIG. 19, control may be performed so that the same feature amount is also automatically selected in the menu bar 281.

The scroll bar 282 can scroll the menu bar 281 as needed.

The menu bar 283 is used to set the type of the above-described inter-ROI operation, and a plurality of operations can be selected.

The scroll bar 284 can scroll the menu bar 283 as needed.

The menu bar 285 sets a reference classification which is a classification of the ROI to be a reference in calculating the inter-ROI mutual feature amount. In the example described above, lesion 1 and lesion 2
When calculating the mutual feature value between ROIs for each normal mucosa, the reference classification is set to normal. A plurality of reference classifications may be selectable.

The scroll bar 286 can scroll the menu bar 285 as needed.

The collective selection button 287 is the same as the one shown in FIG.
RO to be calculated in the same manner as the operation of the batch selection button 244 of
A batch selection of mutual feature values between I is designated.

After completing a series of operations in the inter-ROI mutual feature menu window 280, the operator determines the inter-ROI mutual feature calculated by clicking on the confirm button 288, and then sets the inter-ROI mutual feature menu window 2
80, and returns to the ROI setting window 220 of FIG.

When the calculation of the mutual feature between ROIs is set, the mutual feature between ROIs is calculated by clicking the feature value calculation execution button 227 in FIG.

FIG. 27 is an explanatory diagram for explaining the calculation of the mutual feature value between ROIs. In the endoscope image 290, five ROIs are set and the classification buttons 226 are respectively set.
(See FIG. 18). In the example shown in FIG. 27, it is assumed that the reference classification is normal and the mutual operation is a difference operation.
Also, the feature quantity in each ROI (1) to (5) is represented by v
1 to v5.

As described above, the inter-ROI mutual feature value is obtained by calculating the feature value calculated from the ROI classified into the same classification as the reference classification and the feature value calculated from the ROI classified into another classification. Things. Therefore, in this case, the mutual feature between ROIs is obtained as follows. However, it is assumed that the inter-ROI mutual feature value is not calculated between the ROIs to which the reference classification is given.

V1-v2 v1-v4 v1-v5 v3-v2 v3-v4 v3-v5 These combinations can be automatically determined using the classification assigned to each ROI as a key. FIG.
It is a flowchart for demonstrating the operation | movement of the automatic combination in mutual feature-value calculation between ROI.

In step S1, a variable i for counting the number of an ROI having the same classification (normal in this example) as the reference classification is initialized to 1, and the flow advances to step S2.

In the following step S2, the i-th RO
It is determined whether the classification of I is normal. If the determination is true, the process proceeds to step S3, and if false, the process proceeds to step S9.

In step S3, a variable j for counting the number of the ROI to be calculated as the difference between the feature quantity and the i-th ROI determined to be the reference classification by the determination in step S2 is initialized to 1. Proceed to step S4.

In step S4, it is determined whether or not i = j. If the determination result is true, step S7
If false, the process proceeds to step S5.

In step S5, the j-th ROI
It is determined whether the classification is normal. If the determination result is true, the process proceeds to step S7, and if false, the process proceeds to step S6.

In step S6, the mutual feature value between ROIs in the i-th and j-th ROIs is calculated.
Proceed to step S7.

In step S7, j is set to j + 1, and the flow advances to step S8.

In step S8, it is determined whether or not j> 5, that is, whether or not processing has been completed for all ROIs (five in this example). If the determination is true, the process proceeds to step S9, and if false, the process proceeds to step S4.

In step S9, i is set to i + 1, and the flow advances to step S10.

In step S10, it is determined whether or not i> 5, that is, all ROIs (five in this example)
It is determined whether the process for has been completed. If the result of the determination is true, the process is terminated; otherwise, the process proceeds to step S2.

When the feature amount calculation execution button 227 is clicked, if an inter-ROI mutual feature amount to be calculated is selected, an inter-ROI feature amount calculation result display window (not shown) is opened. The configuration of the inter-ROI feature amount calculation result window is substantially the same as the above-described feature amount calculation result display window (FIG. 22), for example, (1)-(2).
For example, the only difference is that the ROIs are displayed in such a manner as to be understood.

The calculation result of the mutual feature between ROIs is
It is added to the inter-interest feature data 200 (see FIG. 17).

On the other hand, the feature amount calculation icon 114 (FIG. 7)
Clicking opens a new feature amount calculation window.

As shown in FIG. 29, the feature amount calculation window 300 includes an ROI designation button 301 for selecting an ROI for calculating the feature amount, a feature amount calculation execution button 302 for executing the feature amount calculation, and End button 303
Consists of

When the operator clicks on ROI designation button 301, an ROI designation window 310 shown in FIG. 30 opens.

As shown in FIG. 30, a ROI designation window 310 has a menu bar 311 for displaying and selecting a list of ROIs already set,
A scroll bar 312, an all-ROI designation button 313 for designating all set ROIs as targets of feature amount calculation, and an R for confirming the set ROI.
An OI image display button 314, a feature amount selection button 315 for selecting the type of the feature amount to be calculated, a ROI limitation button 316 for limiting the ROI displayed on the menu bar 311 by a site, a finding, or a classification, and an ROI designation. The confirmation button 317 is used to confirm and close the ROI designation window 310.

In the menu bar 311, the inspection No. ,
Patient ID, patient name, image No. Alternatively, data for specifying the ROI, such as the number of the ROI, is displayed, and the ROI is designated by clicking an arbitrary item.

When ROI image display button 314 is clicked, image display area 221 in ROI setting window 220 for displaying an endoscope image including the ROI specified on menu bar 311 (see FIG. 18)
A new window (not shown) according to is opened. At this time, data such as the site, findings, and classification may be displayed together with the endoscope image.

When the feature amount selection button 315 is clicked, the same processing as when the feature amount selection button 223 in the above-described ROI setting window 220 is clicked is performed.

When the ROI limitation button 316 is clicked, the display items on the menu bar 311 are limited by the operator. That is, only the ROI to which the specified data is attached is displayed using the site, the finding, or the classification as a key.

When a series of operations are completed, the operator clicks a confirm button 317 to confirm the designation of the ROI for calculating the characteristic amount, and closes the ROI designation window 290.

When the feature amount calculation execution button 303 in FIG. 29 is clicked, the feature amount is calculated for the designated ROI. The subsequent operation is the same as when the feature amount calculation execution button 227 in the ROI setting window 220 is clicked.

[0220] When the end button 303 is clicked, the feature amount calculation window 300 is closed.

(Determination and Classification Block) Next, the operation of the determination and classification block 75 will be described.

As the discriminant classification method, various methods are known as a statistical method or a non-statistical method.
The former includes a linear discriminant function and ECM rules, and the latter includes a discriminator using a neural network. In this embodiment, Fisher's linear discriminant function (two-class classification), which is widely used as a known technique, is handled.

The discrimination / classification processing is performed by using a discrimination / classification unit (this embodiment) based on one or more characteristic amounts in a plurality of different classes (in the present embodiment, the class given to the set ROI is a class). In this embodiment, a linear discriminant function is created, and it is determined which class the data to be discriminated and classified corresponds to. When creating a discriminant classifier, one sample data (called teacher data) to which a correct class is given in advance is used.
A plurality of combinations of one or more feature values (called feature vectors) are used.

In the present embodiment, for example, type I early cancer and adenoma in the stomach are classified as classes. In addition, the feature amount constituting the feature vector is defined as a mutual feature amount between ROIs, and log (G / R) and lo
It is assumed that the difference between the average values of g (B / R) is used.

In the present embodiment, the class to be classified and classified, the type of the feature used as the feature vector, the combination of the ROI used as the teacher data, and each coefficient in the classifier are called a data set. .

The operator selects the discrimination / classification icon 75 (FIG. 7).
By clicking, a new classification window is opened.

As shown in FIG. 31, the discrimination / classification window 320 includes a menu bar 321 for displaying and selecting the type of the discrimination / classification technique to be applied, and a scroll bar 32.
2, a data set creation button 323 for operating a data set creation process to be described later, a discrimination classification execution button 324 for executing discrimination classification, and an end button 32
It consists of five.

The menu bar 321 displays a list of executable discrimination and classification methods, and the operator selects a discrimination and classification method to be executed by clicking.

[0229] When the data set creation button 323 is clicked, a new data set creation window opens.

As shown in FIG. 32, a data set creation window 330 includes a menu bar 331 for displaying and selecting a class (here, a classification) for setting a class to be classified and classified, and a scroll bar 332. ,
A menu bar 333 for selecting a feature amount to be used as a feature vector, scroll bars 334a and 334b for scrolling the menu bar 333 in the vertical and horizontal directions, and a class-specific ROI selection for specifying teacher data A button 335, a data set name input area 336 for giving a name to the data set to be created, a create execution button 337 for executing the creation of the data set, and existing data for calling the already created data set It consists of a set call button 338 and an end button 339.

The operator selects a class to be determined and classified on the menu bar 331, and selects a feature amount to be used as a feature vector on the menu bar 333.

Clicking on a class-specific ROI selection button 335 opens a new class-specific ROI list window. This class-specific ROI list window is opened by the same number as the number of classes to be classified and classified.

As shown in FIG. 33, a class-specific ROI list window 340 is used to display a menu bar 341 for displaying and selecting a set list of ROIs, a scroll bar 342, and the name of the corresponding class. Output display area 343 and an all select button 3 for collectively selecting all ROIs displayed on the menu bar 341
44 and an enter button 345.

On the menu bar 341, the set R
In the OI, a list of classes to which the corresponding class is attached is displayed, and a click specifies that the class is used as teacher data. By clicking the select all button 344, all ROIs displayed on the menu bar 341 can be collectively designated as teacher data.

After the designation of the ROI to be used as the teacher data is completed, the class-specific ROI list window is closed by clicking the confirm button 345.

By performing the above series of processes several times for the class to be determined, designation of teacher data is completed.

A name for a data set created by input from the keyboard 42 can be given. The data set name is displayed on the output display area 336 in FIG.

The operations on the data set creation window 330 described above can be performed in an arbitrary order.

Subsequently, a data set is created by clicking a create execution button 337. Fish
In the linear discriminant function of er, the same number of coefficients as the number of features constituting the feature vector and the boundary value of the discriminant classification are calculated.

Also, an existing data set call button 3
Clicking on 38 opens a new dataset list window.

As shown in FIG. 34, the data set list window 350 displays a list of data set names that have been created, and a menu bar 351 for selecting the data set, a scroll bar 352, a confirm button 353, and an end button 354. Consists of

The operator selects one data set name on the menu bar 351 and clicks the OK button 353 to set the menu bar 331, menu bar 333, and output display area 336 in the data set creation window 330. Can be the same as the selected data set. The same applies to each display item in the class-specific ROI list window 340.

Thereafter, a new data set can be created by appropriately changing the reproduced setting conditions. This makes it possible to make adjustments such as to stop using feature amounts that have a low degree of contribution to the discriminant classification and to slightly change the combination of ROIs used as teacher data.

After a series of processing is completed, the end button 339 is clicked to open the data set creation window 3
Close 30.

Also, the discrimination / classification execution button 324 shown in FIG.
By clicking, discrimination and classification using the created data set is executed. Execute discrimination classification button 324
Clicking opens a new discrimination / classification execution window.

As shown in FIG. 35, a discrimination / classification execution window 360 includes a menu bar 361 capable of displaying and selecting a list of already created data set names,
A scroll bar 361, a menu bar 363 for displaying and selecting all set ROIs,
A scroll bar 364, a corresponding class selection button 365 for limiting the ROI displayed on the scroll bar 363 to a ROI to which the same class as the teacher data is assigned, and an execution for starting the execution of the discrimination classification A button 366 and an end button 367 are provided.

[0247] The operator selects an arbitrary data set displayed on the menu bar 361 as a discriminating classifier.
Also, any ROI displayed on the menu bar 363
Is selected as a classification target.

When the corresponding class selection button 365 is clicked, only the ROIs to which the same class as the teacher data is assigned are displayed. This makes it easy to confirm the correct answer rate and the like of the discrimination classification based on the created data set.

When the execute button 366 is clicked, discrimination / classification processing using the selected data set and ROI is executed, and a new discrimination / classification result display window is opened.

As shown in FIG. 36, the discrimination / classification result display window 370 includes an output display area 371 for displaying the used data set name, an output display area 372 for displaying the discrimination / classification boundary value, and a discrimination / classification. An output display area 373 for displaying the name of the class set as the target, an output display area 374 for displaying the information of the ROI set as the classification target and the determination result, a scroll bar 375 and an end button 376 I have.

The boundary value of the discriminant classification is a coefficient that gives a boundary separating two classes in the Fisher's linear discriminant function. If the value of the result calculated from the feature vector of the ROI to be discriminated and classified is larger or smaller than this boundary value, the ROI is classified into two classes. When the calculation result is equal to the boundary value, it is only necessary to determine in advance which class to classify.

In the output display area 374, the inspection No. relating to the ROI from which the discrimination / classification result was obtained is displayed. , Patient ID,
Image No. , ROI numbers and the like and the discrimination classification result are sequentially displayed.

After confirming the discrimination / classification result, the operator clicks the end button 376 to close the discrimination / classification result display window 370.

The discrimination / classification result display windows are opened by the number of the selected data sets, and the discrimination / classification results for each data set are displayed. After performing the desired process in the series of operations described above, the discrimination / classification window is closed by clicking the end button 325 in the discrimination / classification window 320 of FIG.

In the diagnosis support processing according to the present embodiment, when a feature used as a feature vector has not been calculated in a ROI designated as teacher data or data to be classified and classified, the feature is automatically calculated. Control is performed to call the amount calculation block 74. Accordingly, it is possible to set a large number of ROIs in advance and collectively calculate the minimum feature amount required for discrimination and classification.

(Report Creation Block) Next, the operation of the report creation block 76 will be described. Clicking on the report creation icon 116 (FIG. 7) opens a report creation window. The report in the present embodiment is assumed to be composed of a data group created based on each processing result in feature amount calculation and discrimination classification.
The contents are the value of the feature amount calculated from the set ROI,
It is based on the result of classification and classification by the data set.

As shown in FIG. 37, the report creation window 380 includes a selection button 381 for selecting the content of a report to be created, and an end button 382. The selection button 381 is used to click on a desired one of the processing results in the feature amount calculation or discrimination classification.
It is possible to create both reports simultaneously in parallel.

When a report is created for the feature amount calculation result using the select button 381, a new feature amount report creation window is opened.

As shown in FIG. 38, a feature amount report creation window 390 includes a menu bar 391 for displaying and selecting a list of classes, ie, classifications, for which a report is to be created, a scroll bar 392, and a feature amount to be used. Menu bar 393 for displaying and selecting a list of
And scroll bars 394a and 394b, a display button 395 for instructing the start of display of a report, and a graph creation button 396 and an end button 397 for instructing the start of display of a graph described later.

The operator selects a class and a feature to be used for report creation on the menu bars 391 and 393.

Subsequently, by clicking the display button 395, a new feature amount report display window is opened. As shown in FIG. 39, the feature amount report display window 400 is opened as many as the number of classes selected in the menu bar 391, and the output display area 4 for displaying class names.
01 and inspection No. , Patient ID, image No. , ROI number, etc. and menu bar 39 in each ROI
3, a menu bar 402, a scroll bar 403a and 403b, and a print start button 404 for instructing a start of printing of a report, and a finish button 40 are displayed.
It consists of five.

When the operator selects a desired item on the menu bar 402 and clicks the print start button 404, the contents can be output to the printer 51.

[0263] A collective selection button for selecting all items in the menu bar 402 may be provided.

When the end button 405 is clicked, the feature amount report display window 400 is closed.

In FIG. 38, a graph creation button 3
Clicking on 96 opens a new graph display window. In this case, the number of feature amounts in the menu bar 393 is two.

As shown in FIG. 40, a graph display window 410 includes an output display area 411 for displaying a legend of each data class plotted on the graph, and a characteristic amount selected on the vertical and horizontal axes, respectively. Display area 412 for displaying a graph given with
1 and a print start button 413 and an end button 4 for instructing printing of display contents in the graph display area 412.
It consists of fourteen.

In the graph display area 412, selected feature amounts of each class are plotted on two-dimensional coordinates.
The scale in the display is automatically set based on the minimum value and the maximum value of the calculation result of each feature amount. Also, in order to facilitate recognition of the class to which the plotted data belongs,
The plot colors are different for each class. In this example, the number of classes is set to three, plotted using red, blue, and green points, respectively, and the correspondence is displayed in the output display area 401 as a legend.

Further, the average value or the variance of the calculation results of each feature amount in each class may be displayed together.

When the print start button 413 is clicked, the contents in the output display area 411 and the graph display area 412 are output to the printer 51.

By clicking the end button 414, the graph display window 410 is closed.

The feature report display window 40
When the “0” and the graph display window 410 are open at the same time, control is performed so that the correspondence between the items in the menu bar 402 and the plots in the graph display area 412 can be understood. For example, when an item on the menu bar 402 is clicked, the corresponding plot may be highlighted.

When a report for the discrimination / classification result is created with the selection button 381 in FIG. 37, a discrimination / category result report creation window (not shown) is opened, and the selected data set, discrimination / classification result, etc. May be appropriately selected and output to the printer 51.

As described above, the diagnosis support apparatus 1 according to the present embodiment
Then, the RO set in one endoscope image by the feature amount calculation block 74 of the diagnosis support processing execution program 70
By calculating the mutual feature value between ROIs by using two or more combinations in I, the relative difference between different lesions with respect to the normal mucosa is evaluated, and multiple Rs in the same image are evaluated.
By determining the relative value of the feature amount calculated from the OI, it is possible to perform the classification that is not affected by the variation in the imaging condition.

The feature calculation method based on the Gabor feature described in the first embodiment uses the ROI
For example, M × L feature amounts obtained by applying L Gabor filters to M sampling pixels in the G image data Ig included in G image data Ig are used for texture analysis and discrimination classification. However, the present invention is not limited to this, and as a further utilization method of the Gabor feature, a new feature amount more effective for diagnosis may be calculated from the obtained Gabor feature.

That is, as a first modification of the first embodiment, a series of processing shown in steps S101 to S104 in FIG.
bor features hij (i = 1, 2,..., M; j = 1, 2,
, L) is the same as in the first embodiment.

In this first modification, the obtained G
The abor feature is defined as follows: h (1) = (h11, h12,..., h1L) h (2) = (h21, h22,..., h2L) ::::: h (M) = ( hM1, hM2, ..., hML). h (i) (i = 1, 2,..., M) can be considered as vectors each having L elements. Therefore, assuming the probability distribution that the elements of each vector occur,
It is considered that a parameter defining the parameter is calculated, and the obtained value is used as a new feature amount.

In the first modification of the first embodiment, the probability distribution in which each element of the vector h (i) occurs is assumed to be an L-dimensional normal distribution, and a parameter defining a multidimensional normal distribution is assumed. And the covariance matrix Σ.

Average vector μ = (μ1, μ2,..., ΜL)
Each element of

(Equation 6) Required by This indicates the average value (expected value) of the Gabor feature calculated by the i-th Gabor filter in the M sampling pixels. The covariance matrix Σ is

(Equation 7) Defined by In equation (7), σpq is

(Equation 8) Is the variance / covariance defined by In the covariance matrix Σ, from the property of σpq = σqp, if L variances (diagonal components) and ((L × (L−1)) / 2) covariances are used as feature amounts, Good.

Based on the mean vector μ and the variance / covariance σpq obtained as described above, the number of dimensions (L + L + (L × (L−
1)) / 2) values are set as feature amounts, displayed in a feature amount calculation result window (see FIG. 22), and ROI feature amount data 2
01.

Therefore, the first embodiment of the first embodiment
In the modified example, as described above, assuming a probability distribution in which the value of the feature amount calculated from each pixel in the ROI occurs, a parameter defining the distribution is calculated, and a new feature amount is obtained. Thus, it is possible to calculate a feature amount more effective for diagnosis. Further, an effect of reducing the number of feature dimensions can be obtained.

Further, in the first modification of the first embodiment, the probability distribution in which the feature amount based on the Gabor feature occurs has been described. The feature assuming the probability distribution is not limited to this. Absent. That is, for example, log (G / R), lo obtained from each sampling pixel
As for the value of g (B / R), the type of the probability distribution that occurs may be assumed, and a parameter defining the same may be calculated to obtain a new feature amount. Further, the form of the probability distribution in which the feature value occurs is not limited to the normal distribution, but is a binomial distribution, χ
Various things such as ^two distributions are conceivable.

As a second modification of the first embodiment, a method of calculating a statistic based on R, G, and B image data in a set ROI is used as a feature amount calculating method.

In an endoscope image composed of a plurality of color signals, the color tone and distribution of each pixel constituted by each color signal are useful information for diagnosis. In the second modification of the first embodiment, attention will be paid to this point, and a description will be given of a method of calculating a feature amount that is suitable for use in discrimination and classification.

In the second modification of the first embodiment, each pixel in the endoscope image is 0 to 25, respectively.
It is assumed that the image data is composed of R, G, and B image data having a value of 5. As the statistics in the ROI set on such an endoscope image, for example, the following (1) to (10) can be considered. Note that the number of pixels for calculating each statistic is N here. Also, ri, gi and bi are:
R, G of the i-th (i = 1, 2,..., N) pixel
And the value of the B data.

(1) Average value of RGB data: μr, μ
g, μb The calculation formulas are:

(Equation 9) Becomes

(2) Variance of RGB data: σ ² _r ,
σ ² _g , σ ² _b

(Equation 10) Becomes

(3) Standard deviation of RGB data: σ _r ,
σ _g , σ _b σ _r , σ _g , σ _b are given by the positive square roots of the respective variances σ ² _r , σ ² _g , σ ² _b .

(4) Covariance of RGB data: σ ² _rg ,
σ ² _rb , σ ² _gb

[Equation 11] Becomes

(5) Correlation coefficient σ ² _rg / (σ _r σ _g ), σ ² _rb / (σ _r σ _b ), σ ² _gb / (σ _g ) of RGB data
σ _b ).

(6) The average value / standard deviation μr / σ _r , μg / σ _g , and μb / σ _b of each of the RGB data are given.

(7) log (G / R) and log (B /
Average value of R): μ (log (G / R)), μ (log (B / R)

(Equation 12) Becomes

(8) log (G / R) and log (B /
R) variance: σ ² _{log (G / R)} , σ ² _{log (B / R)}

(Equation 13) Becomes

(9) log (G / R) and log (B /
R) standard deviation: σ _{log (G / R)} , σ _{log ( B / R)} σ _{log (G / R)} , σ _{log (B / R)} are the respective variances σ ²
It is given by the positive square root of _{log (G / R)} and σ ² _{log (B / R)} .

(10) log (G / R) and log (B
/ R) covariance: σ ² _{log (G / R) log (B / R )}

[Equation 14] Becomes

Further, by calculating R / A, G / A and B / A as A = ri + gi + bi (where ri, gi and bi are the values of R, G and B data of the i-th pixel, respectively), The color tone of a pixel can be represented by a vector in the RGB color space. It is also possible to use those statistics as features. That is, the following (11) to (1)
The statistic of 6) can be used as a feature amount.

(11) Average values of R / A, G / A, B / A: μ _{r / a} , μ _{g / a} , μ _{b / a}

(Equation 15) Becomes

(12) Dispersion of R / A, G / A, B / A:
σ ² _{r / a} , σ ² _{g / a} , σ ² _{b / a}

(Equation 16) Becomes

(13) Standard deviations of R / A, G / A and B / A: σ _{r / a} , σ _{g / a} , σ _{b / a} σ _{r / a} , σ _{g / a} , σ _{b / a} , Their variances σ ² _{r / a} , σ ²
It is given by the positive square root of _{g / a} and σ ² _{b / a} .

(14) Covariance of R / A, G / A, B / A: σ ² _{r / a, g / a} , σ ² _{r / a, b / a} , σ ² _{g / a, b / a} The calculation formulas are:

[Equation 17] Becomes

(15) Correlation coefficients of R / A, G / A, B / A σ ² _{r / a, g / a} / (σ _{r / a} σ _{g / a} ), σ ² _{r / a, b / a} / (σr _/ aσ
_{b / a} ) and σ ² _{g / a, b / a} / (σ _{g / a} σ _{b / a} ).

(16) Average value / standard deviation of R / A, G / A, B / A μ _{r / a} / σ _{r / a} , μ _{g / a} / σ _{g / a} , μ _{b / a} / σ _b given by _{/ a} .

As described above, in the second modification of the first embodiment, by calculating the above-described statistics from the ROI, it is possible to obtain feature amounts effective for diagnosis.

FIGS. 41 to 43 relate to the second embodiment of the present invention. FIG. 41 is a pattern diagram showing the structure pattern of the texture of an image, and FIG. 42 explains the correction of the structure pattern of the texture of FIG. FIG. 43 is an explanatory view, and FIG.
FIG. 8 is a pattern diagram showing a modified example of the texture structure pattern of FIG.

Since the second embodiment is almost the same as the first embodiment, only different points will be described, and the same components will be denoted by the same reference numerals and description thereof will be omitted.

The second embodiment of the present invention has the same configuration as the first embodiment, and corrects a feature amount for extracting local information depending on a position in an image such as a Gabor feature. About the method.

The Gabor feature is, as described above, in the direction θ.
k is a value calculated by a Gabor filter set by wavelengths λm and σx (λm) and σy (λm). On the other hand, there are cases where it is difficult to define directions such as up, down, left, and right, because images obtained under conditions where it is difficult to determine imaging conditions, such as imaging with an endoscope, are rotated and moved. Many. Therefore, the calculated value of the feature amount depending on the change in the direction θk may not be meaningful.

In the present embodiment, a description will be given of a correction that can obtain a feature amount that favorably contributes to diagnosis even in an image in which rotational movement occurs and an image in which it is difficult to define the direction.

In the present embodiment, as a feature amount calculating method, a feature amount based on the Gabor feature described in the first embodiment is calculated.

In the present embodiment, it is assumed that the Gabor filter for calculating the Gabor characteristic is obtained from the numerical value groups (a), (b) and (c) described in the first embodiment. That is, there are four directions θk,
There are two types of wavelengths λm, and there are eight types of Gabor filters.

FIGS. 41 and 42 are explanatory diagrams for explaining the correction in this embodiment. Here, in a certain pixel, Gabor obtained by changing the direction θk with respect to the wavelength λm
The values of the Gabor features calculated by the filters are respectively (hm1, hm2, hm3, hm4).

In the patterns to be classified into the same class in the discriminant classification, it is expected that the distribution of the values of the Gabor features in the different directions at the same wavelength λm will be similar. For example, consider a case where a Gabor feature is obtained from a texture as shown in FIG. FIG. 42A shows the elements of the structure pattern of the texture shown in FIG.

For the pixel P (x, y) in the texture structure pattern shown in FIG. 42 (a), the Gabor feature by θk in the four directions at the same λm is shown in FIG.
, That is, hm1 becomes the maximum. on the other hand,
When the rotational movement as shown in FIG. 42 (c) occurs, the pixel P ′ (x, x) corresponding to the pixel P (x, y) after the rotational movement
The Gabor feature in y) is θ in FIG.
2 is the largest.

In this example, in the Gabor filter at the same wavelength λm, the direction θk at which the Gabor characteristic is maximized is determined by the structural pattern to be textured.
This indicates that it is possible to correct the influence of the rotational movement on the calculated feature value.

That is, as shown in FIGS. 42 (a) and 42 (c), even when a change occurs due to the rotational movement of the texture structure pattern, the direction θk at which the obtained Gabor feature value becomes the maximum is newly set to θ1. Then, rearrangement as shown in FIG. 42 (d) may be performed. In this example,
Gabor features obtained at pixel P (x, y) (h
m1, hm2, hm3, hm4) to (hm2, hm3, hm4, hm
The order can be changed to the order of 1). Further, the amount of rotation may be determined not only based on the direction giving the maximum value, but also based on the tendency of the values of hm1 to hm4.

Gabo in the direction θk described above
The reordering of r features is applied at all sampled pixels in the ROI. This is a certain wavelength λ
m, an average value of Gabor features calculated by a Gabor filter defined by different directions θk (four directions in this example) is obtained, and rearrangement is performed on θk that gives the maximum value. If higher accuracy in correcting the rotational movement is desired, more directions θk (for example, six directions) may be determined.

A plurality of wavelengths λm (two types in this example) are determined. The order of rearrangement is determined at a specific wavelength (reference wavelength) among the wavelengths, and the wavelength of λm is determined based on other wavelengths.
Sorting in the same order is applied to the or feature. An arbitrary λm can be selected as the reference wavelength. For example, the reference wavelength may be set to a wavelength that well reflects a target texture structure pattern.

By applying the above-described correction, it is possible to apply a feature amount calculation method that stores local information depending on the position to a pattern (texture) having a rotational movement.

In an endoscopic image, preservation of direction may not be important because of irregularities in the texture structure. For example, as in the example of the texture shown in FIG. 43, the directionality of the entire texture is not so regular, and the features found in the structural pattern itself that constitutes the texture may be important. For such a texture, the R
Rather than applying a correction to the rotational movement of the entire OI, the calculated G is calculated independently for each sampling pixel.
Applying rearrangement of the abor features will provide a desirable feature amount. In this case, the sorting of the Gabor features described with reference to FIG. 42 may be performed based on the calculated maximum value of each of the 25 pixels. By applying this correction, it is possible to calculate a new feature amount independent of the propagation direction of the frequency component from the Gabor feature. The setting of the reference wavelength may be appropriately selected.

In some cases, preservation of direction is not important at all due to irregularities in the texture structure. In such a case, Gabor features hm1 to hmn (n is the number of directions θk) obtained in the order defined in the direction θk at each sampling point may be rearranged and used according to the value.

FIGS. 44 to 46 relate to the third embodiment of the present invention. FIG. 44 is a block diagram showing the processing configuration of the pre-processing block, and FIG. 45 is a diagram for explaining the operation of the pre-processing block of FIG. FIG. 46 is a second explanatory diagram for explaining the operation of the preprocessing block of FIG.

Since the third embodiment is almost the same as the first embodiment, only different points will be described, and the same components will be denoted by the same reference numerals and description thereof will be omitted.

The third embodiment of the present invention has the same configuration as that of the first embodiment, and is characterized by a preprocessing method for obtaining a good value of a feature by a feature calculation method.

In FIG. 44, the preprocessing method block 50
0 denotes a noise removal block 501 for removing noise in the endoscope image, an inverse γ correction block 502 for applying inverse γ correction, and a color shift with respect to the image captured by the frame sequential endoscope. It comprises a correction block 503 and a shading correction block 504 for correcting a global change in brightness due to shading. Each block can be applied independently and selectively, and can be used in any combination and order.

Next, the operation of each block will be described. The noise removal block 501 removes noise in each of the R, G, and B image data. Specifically, for example, filtering by a median filter with a mask size of 3 × 3 or a uniform weighting filter may be applied to all or any of the R, G, and B image data. Processing targets are R, G and B
The present invention can be applied not only to image data but also to all image data used by a feature amount calculation method following the subsequent stage, such as luminance image data (the same applies to preprocessing in other blocks).

The inverse gamma correction block 502 comprises R, G, and B
A pre-process for removing the γ correction applied to each image data is applied.

The color misregistration correction block 503 corrects a color misregistration occurring in an image picked up by a frame sequential endoscope. Specifically, it can be realized by a series of processes described below.

In the color misregistration correction, in the ROI to be processed, the R image and the B image based on the G image
The number of pixels shifted in each of the horizontal and vertical directions is estimated, and each shift is corrected. The correction of the deviation of the R image from the G image is performed as follows.

FIGS. 45 and 46 are diagrams for explaining processing for estimating the number of pixels corresponding to the shift of each image and correction for the estimated shift.

In FIG. 45, the size of the G image is x1 ×
The size of the y1, R image is x2 × y2, and x2 = x1 + 2
Xdx, y2 = y1 + 2 x dy. Here, dx and dy are values determined in consideration of the range in which the displacement has occurred in the horizontal and vertical directions, respectively.

As a result, although the R image data and the G image data are obtained from the same site, the size of the R image data is slightly larger than that of the R image data.

First, the starting point Sr of the R image data in FIG. 45 is set to coordinates (0, 0). On the other hand, the start point Sg of the G image data is set to (x, y), and the pixel is moved by one pixel within a range of 2 dx in the horizontal direction and 2 dy in the vertical direction from the coordinates (0, 0). That is, Sg (x, y) (0 ≦ x ≦ 2
dx, 0 ≦ y ≦ 2dy). Each time the pixel is moved by one pixel in the horizontal or vertical direction, the cross-correlation between the R image data and the G image data is calculated, and Sg (xmax, ymax) that gives the maximum value is calculated.

For example, dx = dy = 5, xmax = 2, ym
If ax = 7, xmax−dx = −3, ymax−
Since dy = 2, the deviation of the R image data from the G image data is −3 pixels in the horizontal direction (3 pixels in the left direction toward the image) and 2 pixels in the vertical direction (2 pixels in the image lower direction).
Pixel).

Therefore, as shown in FIG. 46, when the R image data is shifted by three pixels in the horizontal direction and by two pixels in the vertical direction, the correction for the color shift is completed.

By applying the same processing to the B image data, all of the R, G, and B image data are corrected to have no color shift.

Next, the shading correction block 504
Will be described. Shading indicates a global change in lightness and darkness that appears on a captured image due to the irradiation angle of illumination light, the shape of an imaging target, and the like. In the present embodiment, the global brightness change in each of the R, G, and B image data in the ROI is represented by a quadratic surface approximation (hereinafter, referred to as a quadratic surface approximation).
This curved surface is referred to as a shading surface), and shading correction using this is applied.

First, a method of calculating a shading surface will be described. The quadratic surface S (x, y) is

S (x, y) = ax ² + bxy + cy ² + dx + ey + f (18) Here, a, b, c, d, e, and f are constants for defining a quadric surface, x and y are x and y coordinates in a three-dimensional space, and S (x, y) is a point. The values of the quadric surface at (x, y) are shown. In the image, x and y indicate the position of the pixel on the image, and S (x,
y) corresponds to the value of the pixel.

As an example, a quadratic surface approximation for R image data of size N × N will be described. Each pixel in the R image data (Ir) has a value r (x,
y). Where 0 ≦ x <N and 0 ≦ y
<N. Value S (x, y) on shading surface
And the square error D between each pixel value r (x, y) on the R image data Ir is

D (x, y) = (S (x, y) −r (x, y)) ² (19) Focusing on this difference, 0 ≦ x <N and 0 ≦ y <
N, the sum Dsum of the square error D (x, y) in equation (19) is

(Equation 20) Let us consider finding constants a, b, c, d, e and f that give the minimum value. Substituting equation (18) into equation (20) gives

(Equation 21) Becomes Constants a, b, c, d, and e that give the minimum value of Dsum
The combination of the values of f and f can be calculated from six simultaneous equations in which equation (21) is partially differentiated with respect to each, and "= 0" (that is, an extreme value is obtained).
Expressing the resulting system of equations in matrix form,

(Equation 22) Becomes Here, Σ in equation (22) is

(Equation 23) Shall be shown.

Constants a, b, c, and D giving the minimum value of Dsum
d, e, and f can be obtained by finding an inverse matrix of a 6 × 6 matrix relating to x and y on the left side in equation (22) and multiplying the right side from the left, and substituting this into equation (18). To form a shading surface.

Next, a method of shading correction using the calculated shading surface will be described.

First, in order to store the value of the DC component, the average value rmean of the pixel r (x, y) in the R image data Ir
Ask for. Also, for each of the pixels r (x, y),

Rs (x, y) = r (x, y) / (ax ² + bxy + cy ² + dx + ey + f) (24) is calculated, and the average value rs _mean of the obtained rs (x, y) is obtained. ,

[Mathematical formula-see original document] rsn (x, y) = rs (x, y) / rs _mean (25) is obtained. Finally,

[Mathematical formula-see original document] rr (x, y) = rsn (x, y) * rs _mean (26) is calculated, and this is defined as R image data after shading correction. Equation (25) normalizes the sum of the values of rsn (x, y) to 1, and may be omitted in a series of processing. In that case, r s (X, y)
By multiplying by s _mean , rr (x, y) can be obtained.

By applying the series of processes described above to the G image data and the B image data in the same manner, it is possible to obtain image data after shading correction for each of them. Further, a shading surface may be obtained from any one of the R, G, and B image data, and the obtained constants a, b, c, d, e, and f may be used as a shading surface of another image. .

The shading correction block 504
The processing content of the shading correction applied in the above is not limited to the above description. For example, image data obtained by low-pass filtering may be substituted for the shading curved surface.

As described above, by applying the feature value calculation method to image data to which each preprocessing is applied in an arbitrary combination and order, it is possible to calculate feature values more effective for diagnosis. . The combination and order of application of each preprocessing may be appropriately determined according to the feature amount calculation method. In addition, designation by a user may be possible. Furthermore, you may apply each preprocessing repeatedly. For example, in the color shift correction block 503, it is also possible to apply the shading correction again to the R, G, and B images after the shading correction is applied. In this case, more accurate color shift correction can be performed.

Further, in the preprocessing method block 500, another preprocessing method may be appropriately added. For example, threshold processing, edge enhancement processing, binarization processing, labeling processing, and the like for image data are included.

[Supplementary Notes] (Supplementary note 1) Image input means for inputting image data consisting of at least one signal, storage means for storing the input image data, and image data recorded by the storage means A region-of-interest setting unit for setting at least one region of interest, a characteristic amount calculating unit for calculating a characteristic amount from the region of interest set by the region-of-interest setting unit, and a characteristic amount calculating unit. A diagnostic support device comprising: a discriminating and classifying means for performing discriminating and classifying lesions using a feature amount.

(Additional Item 2) The additional characteristic item 1, wherein the characteristic amount calculating means calculates a new second characteristic amount based on the first characteristic amount calculated from one region of interest. Diagnosis support device.

(Additional Item 3) The additional item 1 or 2 wherein the characteristic amount calculating means calculates a new second characteristic amount based on the first characteristic amount calculated from a plurality of regions of interest. The diagnostic support device according to the above.

(Additional Item 4) The diagnostic support apparatus according to any one of additional items 1, 2 and 3, wherein the first characteristic amount is a characteristic amount based on a color tone.

(Additional Item 5) The diagnosis support apparatus according to any one of additional items 1, 2 and 3, wherein the first characteristic amount is a characteristic amount based on texture.

(Additional Item 6) The additional item 3, 4, or 5, wherein the second feature value is a value obtained by a difference operation on the first feature value calculated from the plurality of regions of interest. The diagnosis support device according to any one of the above.

(Additional Item 7) Any one of additional items 3, 4, and 5, wherein the second feature value is a value based on a ratio of the first feature value calculated from the plurality of regions of interest. The diagnostic support device according to any one of the above.

(Additional Item 8) The additional item 1, 2, 3, 4, 5, wherein the feature amount calculating means calculates a statistic based on the image data signal in the region of interest.
The diagnosis support apparatus according to any one of 6, 7, and 8.

(Additional Item 9) The diagnosis support apparatus according to additional item 5, wherein the characteristic amount calculating means calculates a characteristic amount based on a local frequency component distribution.

(Additional Item 10) The additional item 2, 3, 4, 5, 6, 7, or 16, wherein the second feature value is based on a probability distribution in which the value of the first feature value occurs. 10. The diagnosis support device according to any one of 8 and 9.

(Supplementary item 11) The second feature value is an average value and / or a variance and / or a covariance and / or a standard deviation and / or a standard value of the first feature value in the region of interest.
Alternatively, the diagnosis support device according to claim 10, wherein the diagnosis support device is a correlation coefficient.

(Additional item 12) The additional item 1, 2, 3, 4, 5, 6, 7, 8, or 17, wherein the characteristic amount calculated by the characteristic amount calculating means is a normalized value. 12. The diagnosis support apparatus according to any one of 9, 10, and 11.

(Additional Item 13) The additional items 1, 2, 3, 4, 5, 6, 7, 8, and 9 wherein the characteristic amount calculated by the characteristic amount calculating means is a corrected value. , 10, 1
13. The diagnosis support device according to any one of 1 and 12.

(Additional Item 14) The diagnostic support apparatus according to additional item 13, wherein the correction for the feature amount is a correction relating to a direction.

(Additional Item 15) The diagnosis support apparatus according to Additional Item 13, wherein the correction for the feature amount is a correction relating to rotational movement of an image.

(Additional Item 16) The feature quantity calculating means may be Ga
Additional features 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 1 to calculate a feature amount based on the bor feature.
The diagnosis support device according to any one of 1, 12, 13, 14 and 15.

(Additional Item 17) The feature value calculated by the feature value calculation means may be an average value and / or a standard deviation and / or a variance and / or a covariance and / or a phase based on a signal constituting the image data. 9. The diagnosis support apparatus according to claim 8, wherein the number is a value obtained by dividing the number of relations and / or the standard deviation by the average value.

(Additional Item 18) An additional item 1, wherein an image processing means for applying image processing to the image data is provided at a stage preceding the characteristic amount calculating means.
2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12,
The diagnosis support apparatus according to any one of 13, 14, 15, 16, and 17.

(Additional Item 19) The image processing means may apply at least one image processing of noise removal processing and / or γ correction removal processing and / or color shift correction processing and / or shading correction processing. 19. The diagnosis support device according to claim 18, wherein

(Appendix 20) The application of the image processing to the image processing means and / or the order of application are made different according to the type of the characteristic amount calculated by the characteristic amount calculating means. 20. The diagnostic support device according to claim 19, wherein

(Additional Item 21) The additional items 1, 2, 3, 4, 5, 6, 7, 8 wherein the shape of the region of interest set by the region of interest setting means is an arbitrary drawing line. ,
9, 10, 11, 12, 13, 14, 15, 16, 1
The diagnosis support device according to any one of 7, 18, 19, and 20.

(Additional Item 22) The additional items 1, 2, 3, and 4 are characterized in that the region of interest set by the region of interest setting means has a predetermined shape and its size is variable. , 5, 6, 7, 8, 9, 10, 1
1, 12, 13, 14, 15, 16, 17, 18, 19
21. The diagnosis support apparatus according to any one of 20.

(Additional Item 23) The diagnostic support apparatus according to additional item 22, wherein the shape of the region of interest set by the region of interest setting means is a polygon or a circle.

(Additional Item 24) The shape of the region of interest set by the region of interest setting means is an arbitrary drawing line.
Or an additional item 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 1 provided with a region-of-interest shape selection means for selecting a predetermined shape.
1, 12, 13, 14, 15, 16, 17, 18, 1
The diagnosis support device according to any one of 9, 20, 21, 22 and 23.

(Additional Item 25) The additional item 1, 2, 3, 4, 5, 6, 7, wherein the image data is an endoscope image and / or an ultrasonic image composed of a plurality of color signals. ,
8, 9, 10, 11, 12, 13, 14, 15, 16,
17, 18, 19, 20, 21, 22, 23 or 24
The diagnosis support device according to any one of the above.

(Additional Item 26) The diagnostic support apparatus according to Additional Item 25, wherein the endoscope image is composed of RGB data.

(Additional Item 27) An image processing method characterized by extracting a feature amount relating to the directionality of a pattern from image data.

(Additional Item 28) A first step of extracting a characteristic amount relating to the directionality of a pattern from image data, and a second step of generating a first characteristic vector having the extracted characteristic amount as an element. The image processing method according to claim 27, further comprising: a step; and a third step of generating a second feature vector from the first feature vector.

(Additional Item 29) The image processing method according to Additional Item 28, wherein the second feature vector is a parameter that defines a probability distribution of occurrence of each element in the first feature vector. .

(Additional Item 30) Additional item 2 is characterized in that at least one of value normalization and / or direction correction is applied to the extracted feature amount.
30. The image processing method according to any one of 7, 28 and 29.

(Additional Item 31) The correction regarding the direction is
31. The image processing method according to claim 30, wherein the method relates to rotational movement of the image.

(Appendix 32) If the extracted feature quantity is G
The image processing method according to any one of additional items 27, 28, 29, 30, and 31, wherein the image processing method is based on the abor feature.

(Additional Item 33) A first step of generating a multi-dimensional surface approximating a global change in brightness for image data, and a second step of correcting the image data using the multi-dimensional surface An image processing method comprising:

(Additional Item 34) The image processing method according to Additional Item 33, wherein the correction is shading correction.

(Additional Item 35) The additional items 27, 28, 29, and 3 wherein the image data is an endoscope image.
The image processing method according to any one of 0, 31, 32, 33, and 34.

(Supplementary note 36) Image input means for inputting original image data composed of at least one signal, image processing means for applying image processing to the original image data input by the image input means, A storage unit for storing the original image data input by the image input unit and the processing result image data by the image processing unit; and storing the original image data and / or the processing result image data recorded by the storage unit. On the other hand, there is provided a region of interest setting means for setting at least one region of interest, and a managing means for managing information of the region of interest set for the original image data and the processing result image data. Diagnosis support device.

(Additional Item 37) The management means manages the original image and the processing result image for the original image as one image group, and the region of interest setting means sets at least one original image and / or 37. The image processing method according to claim 36, wherein a region of interest having the same shape, position, and size as the region of interest set for the processing result image is added to another image in the image group.

[0382]

As described above, according to the diagnosis support apparatus of the present invention, the characteristic amount calculating means calculates the characteristic amount from the region of interest set by the region of interest setting means, and the discriminating and classifying means calculates the characteristic amount. Is used to perform classification and classification of lesions using the feature amounts calculated by the above. Therefore, a feature amount or the like that is less affected by differences in imaging conditions is calculated, and a stable automatic diagnosis result can be obtained by using this feature amount. This has the effect.

[Brief description of the drawings]

FIG. 1 is a configuration diagram showing a configuration of a diagnosis support device according to a first embodiment of the present invention;

FIG. 2 is a functional block diagram showing a configuration of a diagnosis support processing execution program in the diagnosis support device of FIG. 1;

FIG. 3 is a configuration diagram showing a conceptual configuration of a multi-window screen in the diagnosis support apparatus of FIG. 1;

4 is a configuration diagram showing a configuration of a multi-window screen developed by operating an icon on the screen of FIG. 3;

FIG. 5 is a first explanatory diagram illustrating a configuration of a window on the multi-window screen of FIG. 4;

FIG. 6 is a second explanatory diagram illustrating a configuration of a window on the multi-window screen of FIG. 4;

FIG. 7 is a configuration diagram showing a specific configuration of a multi-window screen in the diagnosis support apparatus of FIG. 1;

FIG. 8 is an explanatory diagram for explaining file management data for managing an endoscope image file recorded on the hard disk in FIG. 1;

FIG. 9 is an explanatory diagram illustrating management data assigned to each endoscopic examination of each endoscopic image of the endoscopic image file recorded on the hard disk of FIG. 1;

FIG. 10 is a configuration diagram showing a configuration of an image management data window expanded by a click operation of the image display icon in FIG. 7;

11 is a configuration diagram showing a configuration of a search window expanded by a click operation of a call button in FIG.

FIG. 12 is a configuration diagram showing a configuration of a search result display window expanded by a click operation of a condition search start button in FIG. 11;

FIG. 13 is a configuration diagram showing a configuration of an image list display window developed by a click operation of the image list button in FIG.

FIG. 14 is a configuration diagram showing a configuration of an image display window developed by a click operation of a display button in FIG. 13;

FIG. 15 is a configuration diagram showing a configuration of an image processing window developed by a click operation of the image processing icon in FIG. 7;

16 is a configuration diagram showing a configuration of a processing result image display window developed by a click operation of an execution button in FIG. 15;

FIG. 17 is a configuration diagram showing a configuration of a database managed by the database management block in FIG. 2;

18 is a configuration diagram showing a configuration of an ROI setting window expanded by a click operation of the ROI setting icon in FIG. 7;

FIG. 19 is a configuration diagram showing a configuration of a calculation feature menu window developed by a click operation of a feature selection button in FIG. 18;

20 is a configuration diagram showing a configuration of a batch selection menu window that is developed by a click operation of a batch selection button in FIG. 19;

21 is a configuration diagram showing a configuration of a site designation menu window which is developed by a click operation of a site setting button in FIG. 18;

FIG. 22 is a configuration diagram showing a configuration of a feature value calculation result display window developed by a click operation of a feature value calculation execution button in FIG. 18;

23 is an explanatory diagram illustrating a method of setting an ROI for an original image and a processing result image using the ROI setting icon in FIG. 7;

24 is a flowchart showing the flow of a feature amount calculation process using the mutual feature amount calculation button in FIG. 19;

25 is an explanatory diagram illustrating sampling pixels extracted from G image data in the feature amount calculation processing in FIG.

26 is a configuration diagram showing the configuration of an inter-ROI mutual feature menu window developed by a click operation of the mutual feature calculation button in FIG. 19;

FIG. 27 is an explanatory diagram for explaining an inter-ROI mutual feature calculated in the inter-ROI mutual feature menu window of FIG. 26;

FIG. 28 is a flowchart for explaining the operation of automatic combination in the calculation of inter-ROI mutual features in the inter-ROI mutual features menu window of FIG. 26;

29 is a configuration diagram showing a configuration of a feature amount calculation window developed by a click operation of the feature amount calculation icon in FIG. 7;

FIG. 30 is a configuration diagram showing a configuration of an ROI designation window which is developed by a click operation of the ROI designation button in FIG. 29;

FIG. 31 is a configuration diagram showing a configuration of a discrimination / classification window which is developed by a click operation of the discrimination / classification icon in FIG. 7;

FIG. 32 is a configuration diagram showing a configuration of a data set creation window which is developed by clicking a data set creation button in FIG. 31;

FIG. 33 is a configuration diagram showing a configuration of a class-specific ROI list window expanded by a click operation of a class-specific ROI selection button in FIG. 31;

FIG. 34 is a configuration diagram showing a configuration of a data set list window expanded by a click operation of an existing data set call button in FIG. 32;

FIG. 35 is a configuration diagram showing a configuration of a discrimination / classification execution window developed by a click operation of the discrimination / classification execution button in FIG. 31;

36 is a configuration diagram showing a configuration of a discrimination / classification result display window developed by a click operation of an execution button in FIG. 35;

FIG. 37 is a configuration diagram showing a configuration of a report creation window expanded by a click operation of the report creation icon in FIG. 7;

FIG. 38 is a configuration diagram showing a configuration of a feature amount report creation window developed by a click operation of a selection button in FIG. 37;

39 is a configuration diagram showing a configuration of a feature amount report display window developed by a click operation of a display button in FIG. 38;

40 is a configuration diagram showing the configuration of a graph display window developed by clicking the graph creation button in FIG. 38;

FIG. 41 is a pattern diagram showing a structure pattern of an image texture according to the second embodiment of the present invention;

FIG. 42 is an explanatory diagram illustrating correction of the structure pattern of the texture in FIG. 41;

FIG. 43 is a pattern diagram showing a modification of the structure pattern of the texture shown in FIG. 41;

FIG. 44 is a block diagram showing a processing configuration of a pre-processing block according to the third embodiment of the present invention.

FIG. 45 is a first explanatory view illustrating the operation of the preprocessing block in FIG. 44;

FIG. 46 is a second explanatory diagram illustrating the operation of the preprocessing block in FIG. 44;

[Description of Signs] 1 ... Diagnosis support device 2 ... Video processor 3, 35 ... Observation monitor 4 ... Input unit 5 ... Server unit 6 ... Conference unit 11 ... A / D converter 12, 32, 50 ... Image processing unit 13, 21 , 31L LAN controller 14, 26, 36 Controller 22 Memory 23, 47 Hard disk 24 Hard disk driver 25 Compressor 33 Expander 34 D / A converter 41 CPU 42 Keyboard 43 I / F 44 search monitor 45 mouse 46 mouse I / F 48 hard disk I / F 49 working memory 50 image processing unit 51 printer 52 printer I / F 61 password storage unit 62 password monitoring unit 63 control Restriction unit 70: Diagnosis support processing execution program 1 ... image management block 72 ... database management block 73 ... ROI setting block 74 ... feature calculation block 75 ... discriminant classifier block 76 ... reporting block 77 ... image processing block

Claims (1)

[Claims] An image input unit for inputting image data comprising at least one signal; a storage unit for storing the input image data; and at least one image data for the image data recorded by the storage unit. A region-of-interest setting unit for setting a region of interest; a characteristic amount calculating unit for calculating a characteristic amount from the region of interest set by the region-of-interest setting unit; a lesion using the characteristic amount calculated by the characteristic amount calculating unit And a discriminating and classifying means for performing discriminating and classifying.