CN110472676A - Stomach morning cancerous tissue image classification system based on deep neural network - Google Patents
Stomach morning cancerous tissue image classification system based on deep neural network Download PDFInfo
- Publication number
- CN110472676A CN110472676A CN201910716776.6A CN201910716776A CN110472676A CN 110472676 A CN110472676 A CN 110472676A CN 201910716776 A CN201910716776 A CN 201910716776A CN 110472676 A CN110472676 A CN 110472676A
- Authority
- CN
- China
- Prior art keywords
- sample
- training
- image
- module
- model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000013528 artificial neural network Methods 0.000 title claims abstract description 20
- 210000002784 stomach Anatomy 0.000 title claims abstract description 14
- 238000012549 training Methods 0.000 claims abstract description 62
- 230000001575 pathological effect Effects 0.000 claims abstract description 36
- 238000007781 pre-processing Methods 0.000 claims abstract description 16
- 239000013598 vector Substances 0.000 claims abstract description 14
- 206010028980 Neoplasm Diseases 0.000 claims abstract description 12
- 201000011510 cancer Diseases 0.000 claims abstract description 11
- 238000000605 extraction Methods 0.000 claims abstract description 11
- 238000003062 neural network model Methods 0.000 claims abstract description 10
- 230000000694 effects Effects 0.000 claims abstract description 5
- 238000000034 method Methods 0.000 claims description 26
- 201000011591 microinvasive gastric cancer Diseases 0.000 claims description 23
- 230000008569 process Effects 0.000 claims description 16
- 238000004364 calculation method Methods 0.000 claims description 9
- 238000012795 verification Methods 0.000 claims description 9
- 238000013145 classification model Methods 0.000 claims description 6
- 238000005457 optimization Methods 0.000 claims description 6
- 238000012545 processing Methods 0.000 claims description 6
- 230000002496 gastric effect Effects 0.000 claims description 5
- 230000002093 peripheral effect Effects 0.000 claims description 4
- 238000013527 convolutional neural network Methods 0.000 description 12
- 230000006870 function Effects 0.000 description 10
- 210000001519 tissue Anatomy 0.000 description 10
- 210000003855 cell nucleus Anatomy 0.000 description 6
- 238000013135 deep learning Methods 0.000 description 6
- 238000003745 diagnosis Methods 0.000 description 6
- 230000004927 fusion Effects 0.000 description 5
- 208000005718 Stomach Neoplasms Diseases 0.000 description 4
- 239000000284 extract Substances 0.000 description 4
- 206010017758 gastric cancer Diseases 0.000 description 4
- 230000007246 mechanism Effects 0.000 description 4
- 238000011160 research Methods 0.000 description 4
- 201000011549 stomach cancer Diseases 0.000 description 4
- 230000034994 death Effects 0.000 description 3
- 231100000517 death Toxicity 0.000 description 3
- 238000013461 design Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 238000010801 machine learning Methods 0.000 description 3
- 238000003759 clinical diagnosis Methods 0.000 description 2
- 238000004195 computer-aided diagnosis Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 238000006073 displacement reaction Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000002474 experimental method Methods 0.000 description 2
- 238000005065 mining Methods 0.000 description 2
- 230000000877 morphologic effect Effects 0.000 description 2
- 238000010827 pathological analysis Methods 0.000 description 2
- 230000007170 pathology Effects 0.000 description 2
- 238000005070 sampling Methods 0.000 description 2
- 238000012706 support-vector machine Methods 0.000 description 2
- 238000009825 accumulation Methods 0.000 description 1
- 231100000504 carcinogenesis Toxicity 0.000 description 1
- 210000004027 cell Anatomy 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 210000000805 cytoplasm Anatomy 0.000 description 1
- 238000003066 decision tree Methods 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 238000013136 deep learning model Methods 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000013399 early diagnosis Methods 0.000 description 1
- 238000003708 edge detection Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 201000006585 gastric adenocarcinoma Diseases 0.000 description 1
- 210000004907 gland Anatomy 0.000 description 1
- 230000000762 glandular Effects 0.000 description 1
- 238000007490 hematoxylin and eosin (H&E) staining Methods 0.000 description 1
- 238000010191 image analysis Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 239000012535 impurity Substances 0.000 description 1
- 208000015181 infectious disease Diseases 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 230000003902 lesion Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 210000004877 mucosa Anatomy 0.000 description 1
- 230000008520 organization Effects 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 238000002601 radiography Methods 0.000 description 1
- 238000010008 shearing Methods 0.000 description 1
- 238000000638 solvent extraction Methods 0.000 description 1
- 239000013589 supplement Substances 0.000 description 1
- 210000004876 tela submucosa Anatomy 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 238000000844 transformation Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- Molecular Biology (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Mathematical Physics (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The present invention relates to a kind of stomach morning cancerous tissue image classification system based on deep neural network, the system include: that WSI pathological section obtains module, for obtaining stomach morning carninomatosis reason sectioning image；Image pre-processing module, for being pre-processed to stomach morning carninomatosis reason sectioning image；Data set division module, for the data set after image preprocessing to be divided into training set and verifying collection, training set is used for model training, and verifying collection is for verifying modelling effect；Characteristic extracting module, the picture feature based on the bis- branching networks structure extraction training sets of GoogleNet or verifying collection, output phase answer the feature vector of picture；Classifier training module obtains stomach morning cancer disaggregated model based on neural network model training.
Description
Technical Field
The invention relates to a histology image classification system for early gastric cancer based on a deep neural network, and relates to the technical field of image classification.
Background
Early gastric cancer refers to gastric cancer in which the lesion invades only the mucosa or submucosa. According to statistics, nearly 100 million cases of gastric adenocarcinoma occur each year worldwide, with the third leading cause of cancer death worldwide, and also a major cause of infection-complicated cancer deaths. Among them, the stomach early cancer accounts for 15% to 57% of the cases of stomach cancer, so it has important significance for early diagnosis and effective treatment of stomach early cancer to reduce the death rate of stomach cancer. Histomorphological features are currently the most recognized and reliable diagnostic criteria for tumors. Generally, the pathological diagnosis process of the stomach precancer comprises the steps of firstly extracting suspected pathological tissues, then carrying out hematoxylin and eosin (H & E) staining on the suspected pathological tissues to prepare pathological sections, carrying out comparison and identification on cell morphology on the pathological sections by a pathologist under a high-power microscope to confirm diagnosis, and greatly depending on pathological knowledge and diagnosis experience accumulation of the pathologist in high efficiency and accuracy. At present, with the rapid development of pathological subjects and the increase of diagnosis requirements, the number of pathological sections is increased explosively, and long-time pathological image interpretation under a microscope causes high working strength and high pressure of pathologists, so that clinical requirements are difficult to meet. Meanwhile, the accuracy of diagnosis is easily influenced by subjective factors of doctors.
In 1963, Lockwick et al published a method of digitizing X-ray films, CAD based medical images began to be generated, and research in this field also began to be active. With the development of computer technology and the popularization of digital pathology, a computer pathology auxiliary system becomes a new tool for pathological diagnosis, and the research of the current CAD system has better effect on the aspect of radiography, but has less research on the aspect of image identification of early gastric cancer pathological sections. The related research is mainly based on the traditional image processing algorithm, the machine learning algorithm and the deep learning algorithm. The traditional image processing algorithm mainly adopts methods such as edge detection, texture feature, morphological filtering, shape model construction, template matching and the like, firstly carries out preprocessing and denoising on an image, then extracts features and selects the features, and then accurately analyzes the image according to the obtained information such as shape, color, texture, surrounding organization relationship and the like. Pathological image recognition based on a machine learning algorithm generally needs to segment key parts such as cell nucleuses of images, and a special feature extractor is designed according to clinical diagnosis bases and the morphological features and statistical features of the cell nucleuses obtained through calculation to extract features of the segmented key parts. The extracted features are then input to a classifier, such as a Support Vector Machine (SVM), KNN (k-nearest neighbor), and a decision tree, for recognition. Due to the strong nonlinear modeling capability of the deep learning network and the characteristics of large information amount and rich features of the medical images, the deep learning algorithm is widely applied to the medical images. However, the method needs a large amount of pathological images labeled with high quality as training samples to train a pre-designed CNN (Convolutional neural network), so that deep features can be extracted from a deep learning model, and then training is performed based on the features, and finally, the trained models are used for automatically identifying the pathological images.
Conventional image analysis-based methods typically need to be designed according to domain-specific medical knowledge in conjunction with specific application scenarios. And the extracted features are limited, the designed features have weak adaptability and high application cost. The machine learning method analyzes tasks in a data-driven manner, can automatically learn relevant model features and data characteristics from large-scale data sets of specific problems, but generally requires complex feature engineering, cannot be expanded on more data quickly, and is difficult to adapt to different fields and applications. The deep learning method is an end-to-end learning mode, does not need manual decision and design features, and has excellent performance. But face the problems of large data dependency and high cost of collection of large and accurate medical image annotation data. In addition, medical pathological images often have large picture sizes and high resolution, and effective information of the pathological images is difficult to be fully learned by a general neural network structure. Due to the definition of the parameters, the input of the neural network is limited, typically 224x224 or 299x299 pixels. In order to adapt to the input of the neural network, two processing methods are picture scaling and clipping respectively, wherein the picture scaling and clipping lose local detail information of an image, and the picture clipping lacks global feature information of an original image, so that the neural network is easy to over-fit. The distribution among the processed data set samples is uncertain and is likely to be unbalanced, and if all samples are uniformly learned without distinguishing, the deep neural network is likely to learn insufficiently under the limited data set, and further, the respective results are inaccurate.
Disclosure of Invention
In view of the above problems, an object of the present invention is to provide a histological image classification system for early gastric cancer based on deep neural network, which can obtain more accurate classification results under a limited data set, and simultaneously avoid the over-fitting phenomenon and reduce the generalization error.
To achieve the above object, an embodiment of the present invention provides a histological image classification system for early gastric cancer based on a deep neural network, including:
the WSI pathological section acquisition module is used for acquiring a gastric precancer pathological section image;
the image preprocessing module is used for preprocessing the stomach early cancer pathological section image;
the data set dividing module is used for dividing the data set subjected to image preprocessing into a training set and a verification set, wherein the training set is used for model training, and the verification set is used for verifying the effect of the model;
the feature extraction module is used for extracting the picture features of the training set or the verification set based on the GoogleNet double-branch network structure and outputting the feature vectors of the corresponding pictures;
and the classifier training module is used for training to obtain the stomach early cancer classification model based on the neural network model.
Further, the feature extraction module performs feature extraction based on a dual-branch network structure of GoogleNet, wherein the GoogleNet dual-branch network structure adopts a dual-input model, a main branch is used for extracting the features of the zoomed whole picture, a parallel auxiliary branch is used for extracting the local detail features of the randomly cut part of the original picture, and finally the global features and the local features are integrated and spliced to output the feature vectors of the corresponding pictures.
Further, the classifier training module specifically comprises the following processes: and determining the type of the sample as a simple sample or a difficult sample according to the prediction probability value (determining the difficulty degree of the sample) of the corresponding real category of the sample in the current batch after each iteration is finished by adopting a mini-batch SGD optimization algorithm, and obtaining the early gastric cancer classification model based on neural network training by adopting a difficult sample weight endowing strategy training mode of increasing the weight calculated in a loss function on the difficult sample and reducing the weight on the simple sample.
Further, the difficult sample weighting strategy is:
let D be (x)i，yi) Representing the entire training set, xiIs the ith picture, yiIs its corresponding tag, p (y)i|xi) Is the prediction probability value of the current batch belonging to the correct category of the sample x, and each picture generates an n-dimensional probability vector through a neural network modelP (y) for sample i labeled ki|xi) Namely, it is All p (y) participating in the training sample t-1 times before are storedi|xi) T represents the current training iteration number; the difficulty of the sample is determined by the average prediction probability of the corresponding class of the sample in the training processIt is decided that,the larger the sample is, the more accurate the sample can be predicted with higher confidence, namely the sample is a simple sampleIn the same way, the method has the advantages of,relatively small, difficult samples, weight in the loss calculationIf W is adopted to represent the parameters of the network model, the calculation formula of the loss function is L ═ Σiwi·lossi(W) + λ R (W), wherein WiIs the weight magnitude of the sample in the computation of the loss function, lossi(W) is the calculated loss of the ith sample under the model parameter W, and λ R (W) is the regularization term.
Further, the early cancer pathological section image is obtained by a digital camera set under a high power microscope.
Furthermore, the image preprocessing module cuts the whole WSI into image blocks patch with set size, removes peripheral tissues and tissue edges in the cut patch and removes the patch with distortion during shooting, and then carries out random rotation, displacement, cutting and/or overturning processing on the patch to carry out image enhancement.
Due to the adoption of the technical scheme, the invention has the following advantages:
1. in view of the characteristics of high resolution and large size of medical images, the invention is based on the strong extraction capability of a convolutional neural network in deep learning on the image characteristics, the image characteristics are extracted through a double-branch CNN, the global characteristics are extracted from main branches, the local detail characteristics lost in the down-sampling process of the main branches are supplemented by collateral auxiliary branches, and finally the model is predicted based on the integrated global characteristics and local characteristics, so that the prediction precision can be effectively improved;
2. because the labeling process of the high-quality labeled pathological image is complex and the cost is high, the invention provides a deep learning optimization algorithm based on sample weighting, the weight of the sample is adjusted according to the prediction result of the sample in the training process, and the learning of the difficult sample is emphasized, so that the convergence speed of the random gradient decline is accelerated, and the overall performance of the model is improved;
in conclusion, with the popularization of computer-aided diagnosis technology, the method has wide application prospect and can be widely applied to classification and identification of the histological images of the early gastric cancer.
Drawings
FIG. 1 is a schematic diagram of a Googlenet-based dual-branch network structure according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of a sample weighting mechanism according to an embodiment of the present invention;
fig. 3 is a schematic diagram of a histological image classification system for early gastric cancer according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be obtained by a person skilled in the art without any inventive step based on the embodiments of the present invention, are within the scope of the present invention.
The invention provides a double-branch feature fusion CNN framework based on difficult sample mining for classifying the pathological images of the early gastric cancer. In clinical diagnosis, accurate diagnosis of pathological images of early gastric cancer requires focusing on microscopic characteristics such as whether or not the cell nucleus is deformed and the degree of deformation, and observing the morphology of glandular tissues and the overall distribution state of surrounding cell nuclei. Therefore, in order to make the deep neural network model fully utilize the local characteristic information and the global characteristic information of the image and obtain higher accuracy and better generalization capability, the invention designs a model architecture with two branches, wherein a main branch is used for extracting the characteristics of the zoomed whole image, a parallel auxiliary branch is used for extracting the local detail characteristics of the random cut part of the original image, and then the characteristics are fused for classification. In order to enable the network to be sufficiently learned under a limited data set, the invention adopts a sample weighting strategy, namely, samples which are not sufficiently learned currently are weighted higher in the calculation of the loss function, samples which are processed by the neural network model are weighted lower, and all samples are learned in a distinguishing way to improve the classification performance of the model. The difficulty degree of the sample is determined according to the prediction probability of the sample belonging to the real class in the training process. Obviously, the relatively large prediction probability corresponding to the real category is a simple sample, and the relatively small prediction probability corresponding to the real category is a difficult sample, and the following describes the principle of the dual-branch feature fusion CNN architecture and the sample weight assignment mechanism provided by the present invention in detail;
1. dual-branch feature fusion CNN architecture
As shown in fig. 1, in order to effectively extract and integrate global feature information and local detail feature information of a pathological image, a dual-branch network structure based on GoogleNet is proposed, and the dual-branch network structure based on GoogleNet is characterized by one more branch and is a dual-input model. The branch network in the design framework has universality for the convolutional neural network with any structure, and different convolutional neural networks can be adopted according to the requirements of specific tasks. The down-sampled and scaled image (with size of 224 × 224) can be directly used as the input of the backbone network branch to extract the global structural features of the whole tissue. The side branch can use the image block which is not subjected to the down-sampling scaling step and randomly cut into 224 × 224 size as input, so as to supplement the lost local detail features in the main branch, such as the morphology, distribution and other information of cell nucleus, and then integrate and splice the global features and the local features for the classification prediction of the model.
2. Sample weighting mechanism
As shown in fig. 2, if a sample has a lower probability of the corresponding real category output by the model during the iteration process, the sample is a difficult sample for the model (for example, the probability is 0.35 at t-1 iteration, the probability is 0.45 at t-th iteration, and as the training times increase, the probability value of the corresponding real category is always smaller and not significantly improved), that is, a sample that is difficult to learn by the model; accordingly, if a sample is always predicted correctly with high confidence, it is clear that the sample is a relatively simple sample (e.g., probability t-1 is 0.88, t is 0.95). In the training process, for samples for which the model can predict accurately with high confidence, such samples may be too simple for the model to contain a small amount of information, and thus contribute little to the improvement of the model performance in the subsequent training. And the distribution of the difficult samples and the simple samples in the training samples is unknown, so a sample weighting mechanism is provided, the weight of each training sample in the loss function is adjusted according to the training result of each round, and the mining of the difficult samples is focused instead of indiscriminate unified training of all samples, so that the performance of the model is further improved.
In the optimization process of the neural network, a simple and universal mini-batch SGD optimization algorithm is adopted. After each iteration is finished, calculating the weight of the sample according to the predicted value of the sample in the current batch, specifically, making D ═ xi，yi) Representing the entire training set, xiIs the ith picture, yiIs its corresponding label (label is the true category of the picture). p (y)i|xi) The prediction probability value of the current batch which belongs to the correct class of the sample x, namely the probability of the corresponding class output by the last layer of the neural network, is shown as an example in an n-classification experiment, and each picture can generate an n-dimensional probability vector through a neural network modelP (y) for sample i with true label ki|xi) Namely, it ist represents the number of current training iterations,all p (y) participating in the training sample t-1 times before are storedi|xi). Then the average prediction probability valueDetermines the difficulty of the sample (The larger the score, the easier the sample is for the classifier, the weight w in the computation of the loss functioniInversely proportional thereto, i.e. of samplesThe smaller, meaning that the sample is not correctly identified by the neural network model, should be taken into account in the subsequent training process. If W is adopted to represent model parameters, the calculation formula of the loss function is L ═ Σiwi·lossi(W) +2R (W), wherein WiIs the weight magnitude of the sample in the computation of the loss function, lossi(W) is the calculated loss of the ith sample under the model parameter W, and λ R (W) is the regularization term to reduce model overfitting.
Based on the above principle, as shown in fig. 3, the present embodiment provides a histological image classification system for early gastric cancer based on a deep neural network, which includes a WSI (white slide image full-slice digital image) pathological section acquisition module, an image preprocessing module, a data set partitioning module, a feature extraction module, and a classifier training module; wherein,
the WSI pathological section acquisition module is used for acquiring the gastric precancer pathological section image, wherein the gastric precancer pathological section image is acquired by a high-performance digital camera arranged under a high-power microscope, and convenience is provided for image observation under a large screen and computer-aided diagnosis. The image of the pathological section of the early gastric cancer is complex, and the pathological section of the early gastric cancer contains cell nucleuses, cytoplasm, interstitium, gland tissue gaps, impurities in the process of preparing the section and the like.
And the image preprocessing module is used for processing the early gastric cancer pathological section image to remove background and noise interference. Because the image resolution of the pathological section of the early gastric cancer is very high, the pathological section is difficult to be directly applied to a neural network model, and peripheral tissues, background, noise and the like can also interfere with classification. Therefore, the image preprocessing module cuts the whole WSI into image blocks with set sizes, namely, the patch, removes peripheral tissues and tissue edges in the cut patch and the patch with distortion in shooting, and then carries out subsequent processing on the patch. In order to enhance the robustness of the algorithm under limited data, the image preprocessing module further performs image enhancement by adopting transformations such as random rotation, displacement, shearing, overturning and the like, and the specific algorithm is the existing algorithm and is not described herein again.
And the data set dividing module is used for dividing the whole data set after image preprocessing into a training set and a verification set, wherein the training set is used for model training, the verification set is used for verifying the effect of the model, the dividing principle is that the data set is divided randomly according to different WSIs, the approximate proportion of the training set to the verification set is 4: 1, and if the number of the pictures in each category is approximately the same, the data distribution of each category is kept consistent.
The feature extraction module is configured to extract picture features through a two-branch feature fusion CNN architecture, that is, a two-branch network structure based on GoogleNet, and output corresponding feature vectors, where the CNN can automatically and efficiently extract features of each layer of a picture, in order to effectively extract global features and local features of an integrated image, in this embodiment, the two-branch feature fusion CNN is designed to perform feature extraction, and then cascade the features extracted by two branches and output the feature vectors corresponding to the picture, where a specific feature cascade process is the prior art and is not described herein again, and the image blocks directly subjected to downsampling and scaling (224 × 224) and randomly cut into 224 × 224 size are used as inputs.
The classifier training module is used for training to obtain a stomach early cancer classification model for performing WSI image diagnosis of the stomach early cancer, wherein a softmax layer of a neural network is equivalent to a classifier, softmax outputs probability vectors corresponding to classes according to extracted feature vectors, the numerical distribution range of the probability is 0 to 1, the class corresponding to the maximum output probability is a final classification prediction result of the model, and the specific process of the classifier training module is as follows: and (3) determining the type of the sample as a simple sample or a difficult sample according to the prediction probability value of the sample in the current batch after each iteration is finished by adopting a mini-batch SGD optimization algorithm, and obtaining the early gastric cancer classification model based on neural network training by adopting a difficult sample weighting strategy training mode of increasing the weight calculated in a loss function for the difficult sample and reducing the weight for the simple sample.
Specifically, the specific process of the difficult sample weighting strategy is as follows:
let D be (x)i，yi) Representing the entire training set, xiIs the ith picture, yiIs its corresponding tag. p (y)i|xi) The prediction probability value of the current batch which belongs to the correct class of the sample x, namely the probability of the corresponding class output by the last layer of the network, by taking an n classification experiment as an example, the model finally outputs a probability vector with dimensions of n multiplied by k, n is the total number of samples in a mini-batch, k is the number of classes, and each picture can generate an n-dimensional probability vector through a neural network modelP (y) for sample i with true label ki|xi) Namely, it ist represents the number of current training iterations,all p (y) participating in the training sample t-1 times before are storedi|xi). The difficulty of the sample is determined by the average prediction probability of the corresponding class of the sample in the training processIt is decided that,the larger the sample, the more accurate the sample can be predicted with higher confidence, i.e. the sample is a simple sample, and similarly,the smaller are difficult samples, since the simple samples have a smaller contribution to the gradient after the network after being correctly identified, whereas the difficult samples have a larger contribution to the gradient. The performance of the model is more dependent on the difficult samples, in a certain courseOverfitting of the model can be avoided in degree.The larger the score, the easier the sample is for the classifier, the weight w in the computation of the loss functioniInversely proportional thereto, i.e. of samplesThe smaller, meaning that the sample is not correctly identified by the model, should be taken into account in the subsequent training process. Weights in loss calculationIf W is adopted to represent the parameters of the network model, the calculation formula of the loss function is L ═ Σiwi·lossi(W) + λ R (W), wherein WiIs the weight magnitude of the sample in the computation of the loss function, lossi(W) is the calculated loss of the ith sample under the model parameter W, and λ R (W) is the regularization term to reduce model overfitting.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solutions of the present invention and not for limiting the scope of protection thereof, and although the present application is described in detail with reference to the above embodiments, those of ordinary skill in the art should understand that: numerous variations, modifications, and equivalents will occur to those skilled in the art upon reading the present application and are within the scope of the claims appended hereto.
Claims (6)
1. A histological image classification system for early gastric cancer based on a deep neural network, the system comprising:
the WSI pathological section acquisition module is used for acquiring a gastric precancer pathological section image;
the image preprocessing module is used for preprocessing the stomach early cancer pathological section image;
the data set dividing module is used for dividing the data set subjected to image preprocessing into a training set and a verification set, wherein the training set is used for model training, and the verification set is used for verifying the effect of the model;
the feature extraction module is used for extracting the picture features of the training set or the verification set based on the GoogleNet double-branch network structure and outputting the feature vectors of the corresponding pictures;
and the classifier training module is used for training to obtain the stomach early cancer classification model based on the neural network model.
2. The histological image classification system of gastric precancer according to claim 1, wherein the feature extraction module performs feature extraction based on a dual-branch network structure of GoogleNet, wherein the GoogleNet dual-branch network structure adopts a dual-input model, a main branch is used for extracting features of the zoomed whole picture, a parallel auxiliary branch is used for extracting local detail features of a randomly cut part of the original picture, and finally, the global features and the local features are integrated and spliced to output feature vectors of the corresponding picture.
3. The histological image classification system for early gastric cancer according to claim 1, wherein the classifier training module comprises the following specific processes:
and determining the type of the sample as a simple sample or a difficult sample according to the prediction probability value of the corresponding real category of the sample in the current batch after each iteration is finished by adopting a mini-batch SGD optimization algorithm, and obtaining the early gastric cancer classification model based on neural network training by adopting a difficult sample weighting strategy training mode of increasing the weight calculated in a loss function on the difficult sample and reducing the weight on the simple sample.
4. The histology image classification system for early gastric cancer according to claim 3, wherein the difficult sample weighting strategy is:
let D be (x)i，yi) Representing the entire training set, xiIs the ith picture, yiIs its corresponding tag, p (y)i|xi) Is the prediction probability value of the current batch belonging to the correct category of the sample x, and each picture passes through a neural network modelAn n-dimensional probability vector is generatedP (y) for sample i labeled ki|xi) Namely, it is
All p (y) participating in the training sample t-1 times before are storedi|xi) T represents the current training iteration number;
the difficulty of the sample is determined by the average prediction probability of the corresponding class of the sample in the training processIt is decided that,the larger the sample, the more accurate the sample can be predicted with higher confidence, i.e. the sample is a simple sample, and similarly,relatively small, difficult samples, weight in the loss calculation
If W is adopted to represent the parameters of the network model, the calculation formula of the loss function is L ═ Σiwi·lossi(W) + λ R (W), wherein WiIs the weight magnitude of the sample in the computation of the loss function, lossi(W) is the calculated loss of the ith sample under the model parameter W, and λ R (W) is the regularization term.
5. The histological image classification system for early gastric cancer according to any of claims 1 to 4, wherein the pathological section images of early gastric cancer are obtained by a digital camera under high power microscope.
6. The histological image classification system for early gastric cancer according to any of claims 1 to 4, wherein the image preprocessing module cuts the whole WSI into image patches with a set size, removes peripheral tissues and tissue edges in the cut patches and patches with distortion during shooting, and performs random rotation, shift, cut and/or flip processing on the patches for image enhancement.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910716776.6A CN110472676A (en) | 2019-08-05 | 2019-08-05 | Stomach morning cancerous tissue image classification system based on deep neural network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910716776.6A CN110472676A (en) | 2019-08-05 | 2019-08-05 | Stomach morning cancerous tissue image classification system based on deep neural network |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110472676A true CN110472676A (en) | 2019-11-19 |
Family
ID=68509954
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910716776.6A Pending CN110472676A (en) | 2019-08-05 | 2019-08-05 | Stomach morning cancerous tissue image classification system based on deep neural network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110472676A (en) |
Cited By (27)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111091166A (en) * | 2020-03-25 | 2020-05-01 | 腾讯科技（深圳）有限公司 | Image processing model training method, image processing device, and storage medium |
| CN111461243A (en) * | 2020-04-08 | 2020-07-28 | 中国医学科学院肿瘤医院 | Classification method, classification device, electronic equipment and computer-readable storage medium |
| CN111626335A (en) * | 2020-04-29 | 2020-09-04 | 杭州火烧云科技有限公司 | Improved hard case mining training method and system of pixel-enhanced neural network |
| CN111815609A (en) * | 2020-07-13 | 2020-10-23 | 北京小白世纪网络科技有限公司 | Pathological image classification method and system based on context awareness and multi-model fusion |
| CN111814835A (en) * | 2020-06-12 | 2020-10-23 | 理光软件研究所（北京）有限公司 | Training method and device of computer vision model, electronic equipment and storage medium |
| CN111985536A (en) * | 2020-07-17 | 2020-11-24 | 万达信息股份有限公司 | Gastroscope pathological image classification method based on weak supervised learning |
| CN112070079A (en) * | 2020-07-24 | 2020-12-11 | 华南理工大学 | X-ray contraband package detection method and device based on feature map weighting |
| CN112308119A (en) * | 2020-10-15 | 2021-02-02 | 中国医学科学院北京协和医院 | Immunofluorescence classification method and device for glomerulonephritis |
| CN112396123A (en) * | 2020-11-30 | 2021-02-23 | 上海交通大学 | Image recognition method, system, terminal and medium based on convolutional neural network |
| CN112488234A (en) * | 2020-12-10 | 2021-03-12 | 武汉大学 | End-to-end histopathology image classification method based on attention pooling |
| CN112560957A (en) * | 2020-12-17 | 2021-03-26 | 北京邮电大学 | Neural network training and detecting method, device and equipment |
| CN112784801A (en) * | 2021-02-03 | 2021-05-11 | 紫东信息科技(苏州)有限公司 | Text and picture-based bimodal gastric disease classification method and device |
| CN113053496A (en) * | 2021-03-19 | 2021-06-29 | 深圳高性能医疗器械国家研究院有限公司 | Deep learning method for low-dose estimation of medical images |
| CN113096131A (en) * | 2021-06-09 | 2021-07-09 | 紫东信息科技(苏州)有限公司 | Gastroscope picture multi-label classification system based on VIT network |
| CN113256572A (en) * | 2021-05-12 | 2021-08-13 | 中国科学院自动化研究所 | Gastroscope image analysis system, method and equipment based on restoration and selective enhancement |
| CN113299388A (en) * | 2021-05-12 | 2021-08-24 | 吾征智能技术(北京)有限公司 | System for cross-modal medical biological characteristic cognitive diseases based on fever with rash |
| CN113327666A (en) * | 2021-06-21 | 2021-08-31 | 青岛科技大学 | Multi-label local-to-global learning method of chest radiography disease multi-classification network |
| CN113506287A (en) * | 2021-07-27 | 2021-10-15 | 陕西师范大学 | Full-view pathological section image classification method, device, equipment and storage medium |
| CN113591919A (en) * | 2021-06-29 | 2021-11-02 | 复旦大学附属中山医院 | AI-based analysis method and system for prognosis of postoperative recurrence of early hepatocellular carcinoma |
| CN113593707A (en) * | 2021-09-29 | 2021-11-02 | 武汉楚精灵医疗科技有限公司 | Stomach early cancer model training method and device, computer equipment and storage medium |
| WO2022193276A1 (en) * | 2021-03-19 | 2022-09-22 | 深圳高性能医疗器械国家研究院有限公司 | Deep learning method for low dose estimation of medical image |
| CN115171882A (en) * | 2022-07-07 | 2022-10-11 | 广东工业大学 | Intelligent medical auxiliary diagnosis method and system based on multi-prior embedded Y-type network |
| CN115601585A (en) * | 2022-09-15 | 2023-01-13 | 中科聚信信息技术（北京）有限公司（Cn） | Agricultural pest and disease diagnosis method and device based on picture analysis |
| CN115861212A (en) * | 2022-11-25 | 2023-03-28 | 中国医学科学院肿瘤医院 | System and application for distinguishing related diseases based on gastric mucosa biopsy sample |
| US11798165B2 (en) | 2020-10-28 | 2023-10-24 | Sony Group Corporation | Tumor detection and segmentation for DPI AI platform routes |
| CN117173485A (en) * | 2023-09-18 | 2023-12-05 | 西安交通大学医学院第二附属医院 | Intelligent classification system method and system for lung cancer tissue pathological images |
| CN117274282A (en) * | 2023-10-08 | 2023-12-22 | 首都医科大学附属北京朝阳医院 | Medical image segmentation method, system and equipment based on knowledge distillation |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107368670A (en) * | 2017-06-07 | 2017-11-21 | 万香波 | Stomach cancer pathology diagnostic support system and method based on big data deep learning |
| CN108898175A (en) * | 2018-06-26 | 2018-11-27 | 北京工业大学 | Area of computer aided model building method based on deep learning gastric cancer pathological section |
| CN109086836A (en) * | 2018-09-03 | 2018-12-25 | 淮阴工学院 | A kind of automatic screening device of cancer of the esophagus pathological image and its discriminating method based on convolutional neural networks |
| CN109300121A (en) * | 2018-09-13 | 2019-02-01 | 华南理工大学 | A kind of construction method of cardiovascular disease diagnosis model, system and the diagnostic model |
| CN109376777A (en) * | 2018-10-18 | 2019-02-22 | 四川木牛流马智能科技有限公司 | Cervical cancer tissues pathological image analysis method and equipment based on deep learning |
-
2019
- 2019-08-05 CN CN201910716776.6A patent/CN110472676A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107368670A (en) * | 2017-06-07 | 2017-11-21 | 万香波 | Stomach cancer pathology diagnostic support system and method based on big data deep learning |
| CN108898175A (en) * | 2018-06-26 | 2018-11-27 | 北京工业大学 | Area of computer aided model building method based on deep learning gastric cancer pathological section |
| CN109086836A (en) * | 2018-09-03 | 2018-12-25 | 淮阴工学院 | A kind of automatic screening device of cancer of the esophagus pathological image and its discriminating method based on convolutional neural networks |
| CN109300121A (en) * | 2018-09-13 | 2019-02-01 | 华南理工大学 | A kind of construction method of cardiovascular disease diagnosis model, system and the diagnostic model |
| CN109376777A (en) * | 2018-10-18 | 2019-02-22 | 四川木牛流马智能科技有限公司 | Cervical cancer tissues pathological image analysis method and equipment based on deep learning |
Cited By (42)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111091166A (en) * | 2020-03-25 | 2020-05-01 | 腾讯科技（深圳）有限公司 | Image processing model training method, image processing device, and storage medium |
| CN111461243A (en) * | 2020-04-08 | 2020-07-28 | 中国医学科学院肿瘤医院 | Classification method, classification device, electronic equipment and computer-readable storage medium |
| CN111626335A (en) * | 2020-04-29 | 2020-09-04 | 杭州火烧云科技有限公司 | Improved hard case mining training method and system of pixel-enhanced neural network |
| CN111626335B (en) * | 2020-04-29 | 2023-06-30 | 杭州火烧云科技有限公司 | Improved difficult-case mining training method and system for pixel-enhanced neural network |
| CN111814835A (en) * | 2020-06-12 | 2020-10-23 | 理光软件研究所（北京）有限公司 | Training method and device of computer vision model, electronic equipment and storage medium |
| CN111814835B (en) * | 2020-06-12 | 2024-07-05 | 理光软件研究所（北京）有限公司 | Training method and device for computer vision model, electronic equipment and storage medium |
| CN111815609B (en) * | 2020-07-13 | 2024-03-01 | 北京小白世纪网络科技有限公司 | Pathological image classification method and system based on context awareness and multi-model fusion |
| CN111815609A (en) * | 2020-07-13 | 2020-10-23 | 北京小白世纪网络科技有限公司 | Pathological image classification method and system based on context awareness and multi-model fusion |
| CN111985536A (en) * | 2020-07-17 | 2020-11-24 | 万达信息股份有限公司 | Gastroscope pathological image classification method based on weak supervised learning |
| CN111985536B (en) * | 2020-07-17 | 2024-02-09 | 万达信息股份有限公司 | Based on weak supervised learning gastroscopic pathology image Classification method |
| CN112070079A (en) * | 2020-07-24 | 2020-12-11 | 华南理工大学 | X-ray contraband package detection method and device based on feature map weighting |
| CN112308119A (en) * | 2020-10-15 | 2021-02-02 | 中国医学科学院北京协和医院 | Immunofluorescence classification method and device for glomerulonephritis |
| CN112308119B (en) * | 2020-10-15 | 2021-11-05 | 中国医学科学院北京协和医院 | Immunofluorescence classification method and device for glomerulonephritis |
| US11798165B2 (en) | 2020-10-28 | 2023-10-24 | Sony Group Corporation | Tumor detection and segmentation for DPI AI platform routes |
| CN112396123A (en) * | 2020-11-30 | 2021-02-23 | 上海交通大学 | Image recognition method, system, terminal and medium based on convolutional neural network |
| CN112488234A (en) * | 2020-12-10 | 2021-03-12 | 武汉大学 | End-to-end histopathology image classification method based on attention pooling |
| CN112488234B (en) * | 2020-12-10 | 2022-04-29 | 武汉大学 | End-to-end histopathology image classification method based on attention pooling |
| CN112560957A (en) * | 2020-12-17 | 2021-03-26 | 北京邮电大学 | Neural network training and detecting method, device and equipment |
| CN112560957B (en) * | 2020-12-17 | 2023-04-07 | 北京邮电大学 | Neural network training and detecting method, device and equipment |
| CN112784801A (en) * | 2021-02-03 | 2021-05-11 | 紫东信息科技(苏州)有限公司 | Text and picture-based bimodal gastric disease classification method and device |
| CN113053496A (en) * | 2021-03-19 | 2021-06-29 | 深圳高性能医疗器械国家研究院有限公司 | Deep learning method for low-dose estimation of medical images |
| WO2022193276A1 (en) * | 2021-03-19 | 2022-09-22 | 深圳高性能医疗器械国家研究院有限公司 | Deep learning method for low dose estimation of medical image |
| CN113053496B (en) * | 2021-03-19 | 2023-08-29 | 深圳高性能医疗器械国家研究院有限公司 | Deep learning method for low-dose estimation of medical image |
| CN113299388A (en) * | 2021-05-12 | 2021-08-24 | 吾征智能技术(北京)有限公司 | System for cross-modal medical biological characteristic cognitive diseases based on fever with rash |
| CN113256572A (en) * | 2021-05-12 | 2021-08-13 | 中国科学院自动化研究所 | Gastroscope image analysis system, method and equipment based on restoration and selective enhancement |
| CN113299388B (en) * | 2021-05-12 | 2023-09-29 | 吾征智能技术(北京)有限公司 | Cross-modal medical biological characteristic cognitive disease system based on fever with rash |
| CN113096131A (en) * | 2021-06-09 | 2021-07-09 | 紫东信息科技(苏州)有限公司 | Gastroscope picture multi-label classification system based on VIT network |
| CN113327666A (en) * | 2021-06-21 | 2021-08-31 | 青岛科技大学 | Multi-label local-to-global learning method of chest radiography disease multi-classification network |
| CN113327666B (en) * | 2021-06-21 | 2022-08-12 | 青岛科技大学 | Multi-label local-to-global learning method of chest radiography disease multi-classification network |
| CN113591919A (en) * | 2021-06-29 | 2021-11-02 | 复旦大学附属中山医院 | AI-based analysis method and system for prognosis of postoperative recurrence of early hepatocellular carcinoma |
| CN113591919B (en) * | 2021-06-29 | 2023-07-21 | 复旦大学附属中山医院 | Analysis method and system for prognosis of early hepatocellular carcinoma postoperative recurrence based on AI |
| CN113506287A (en) * | 2021-07-27 | 2021-10-15 | 陕西师范大学 | Full-view pathological section image classification method, device, equipment and storage medium |
| CN113593707B (en) * | 2021-09-29 | 2021-12-14 | 武汉楚精灵医疗科技有限公司 | Stomach early cancer model training method and device, computer equipment and storage medium |
| CN113593707A (en) * | 2021-09-29 | 2021-11-02 | 武汉楚精灵医疗科技有限公司 | Stomach early cancer model training method and device, computer equipment and storage medium |
| CN115171882A (en) * | 2022-07-07 | 2022-10-11 | 广东工业大学 | Intelligent medical auxiliary diagnosis method and system based on multi-prior embedded Y-type network |
| CN115601585A (en) * | 2022-09-15 | 2023-01-13 | 中科聚信信息技术（北京）有限公司（Cn） | Agricultural pest and disease diagnosis method and device based on picture analysis |
| CN115861212B (en) * | 2022-11-25 | 2023-07-14 | 中国医学科学院肿瘤医院 | System and apparatus for distinguishing related disorders based on gastric mucosa biopsy samples |
| CN115861212A (en) * | 2022-11-25 | 2023-03-28 | 中国医学科学院肿瘤医院 | System and application for distinguishing related diseases based on gastric mucosa biopsy sample |
| CN117173485B (en) * | 2023-09-18 | 2024-02-13 | 西安交通大学医学院第二附属医院 | Intelligent classification system method and system for lung cancer tissue pathological images |
| CN117173485A (en) * | 2023-09-18 | 2023-12-05 | 西安交通大学医学院第二附属医院 | Intelligent classification system method and system for lung cancer tissue pathological images |
| CN117274282A (en) * | 2023-10-08 | 2023-12-22 | 首都医科大学附属北京朝阳医院 | Medical image segmentation method, system and equipment based on knowledge distillation |
| CN117274282B (en) * | 2023-10-08 | 2024-04-19 | 首都医科大学附属北京朝阳医院 | Medical image segmentation method, system and equipment based on knowledge distillation |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110472676A (en) | Stomach morning cancerous tissue image classification system based on deep neural network | |
| Jia et al. | Detection of cervical cancer cells based on strong feature CNN-SVM network | |
| CN108364288B (en) | Segmentation method and device for breast cancer pathological image | |
| Sebai et al. | MaskMitosis: a deep learning framework for fully supervised, weakly supervised, and unsupervised mitosis detection in histopathology images | |
| CN112017191A (en) | Method for establishing and segmenting liver pathology image segmentation model based on attention mechanism | |
| Veta et al. | Detecting mitotic figures in breast cancer histopathology images | |
| CN113034505B (en) | Glandular cell image segmentation method and glandular cell image segmentation device based on edge perception network | |
| Pan et al. | Mitosis detection techniques in H&E stained breast cancer pathological images: A comprehensive review | |
| CN110838100A (en) | Colonoscope pathological section screening and segmenting system based on sliding window | |
| CN112990214A (en) | Medical image feature recognition prediction model | |
| CN112348059A (en) | Deep learning-based method and system for classifying multiple dyeing pathological images | |
| CN115775226B (en) | Medical image classification method based on transducer | |
| Nateghi et al. | Automatic detection of mitosis cell in breast cancer histopathology images using genetic algorithm | |
| CN115063592B (en) | Multi-scale-based full-scanning pathological feature fusion extraction method and system | |
| CN114782948B (en) | Global interpretation method and system for cervical fluid-based cytological smear | |
| Kanwal et al. | Quantifying the effect of color processing on blood and damaged tissue detection in whole slide images | |
| CN117036288A (en) | Tumor subtype diagnosis method for full-slice pathological image | |
| CN116884623A (en) | Medical rehabilitation prediction system based on laser scanning imaging | |
| Alahmari et al. | A review of nuclei detection and segmentation on microscopy images using deep learning with applications to unbiased stereology counting | |
| Ning et al. | Multiscale context-cascaded ensemble framework (MsC 2 EF): application to breast histopathological image | |
| Sohail et al. | Deep object detection based mitosis analysis in breast cancer histopathological images | |
| CN113012167A (en) | Combined segmentation method for cell nucleus and cytoplasm | |
| Peng et al. | Fully convolutional neural networks for tissue histopathology image classification and segmentation | |
| Yancey | Deep Feature Fusion for Mitosis Counting | |
| Wang et al. | A novel dataset and a deep learning method for mitosis nuclei segmentation and classification |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20191119 |
|
| RJ01 | Rejection of invention patent application after publication |