CN114898172A - Classification and modeling method of diabetic retinopathy based on multi-feature DAG network - Google Patents

Abstract

The invention discloses a method for classifying and modeling diabetic retinopathy based on a multi-feature DAG network. First, different methods are used to extract the index features of diabetic retinopathy, including hemorrhagic spot features, fundus neovascularization features and retinal varices features; In order to optimize the DAG network and constantly change the training scheme to train the network, the extracted features can be fused with multiple features, and the local features can be composed of complex local or global features to restore the object; finally, the normal and lesion classification is performed by the softmax classifier. The present invention uses the DIARETDB1 data set and the Dalian Third People's Hospital data to evaluate the performance of the classification model, and the results show that the classification accuracy rates of the present invention are 98.7% and 98.5% for the DIARETDB1 data set images and hospital data, respectively.

Description

Classification and modeling method of diabetic retinopathy based on multi-feature DAG network

technical field

The invention relates to the field of medical image processing, in particular to a method for classifying and modeling diabetic retinopathy based on a multi-feature DAG network.

Background technique

Diabetic retinopathy is one of the most serious blinding eye diseases. Diabetic retinopathy is divided into stage VI, the first three stages are non-proliferative stages, and the last three stages are proliferative stages. In addition to systemic symptoms such as polydipsia, increased urine sugar and blood sugar, diabetic patients also have fundus changes that are mainly characterized by spot hemorrhage, neovascularization and varicose blood vessels in the retina of both eyes. Diagnosis of diabetes and estimation of prognosis are of great importance. In the past, ophthalmologists manually evaluated the fundus images based on the characteristics of diabetic retinopathy to detect whether the retina was diseased. However, because some patients are in the pre-diabetic retinopathy and the characteristics are not obvious, it is easy to miss the optimal treatment time due to inaccurate evaluation. Therefore, a fast and accurate automatic recognition and classification system for diabetic retinopathy images is needed, which can also identify those diabetic retinopathy images with inconspicuous features.

At present, the classification of retinal images has been achieved by extracting features, such as extracting features of microaneurysm, exudate, and vitreous hemorrhage. feature extraction. At the same time, the existing research only extracts one or two kinds of feature information, so that the classification model cannot learn in detail and comprehensively, resulting in a low classification accuracy.

SUMMARY OF THE INVENTION

In order to solve the above-mentioned technical problems existing in the prior art, the present invention provides a method for classifying and modeling diabetic retinopathy based on a multi-feature DAG network.

The technical solution of the present invention is: a method for classifying and modeling diabetic retinopathy based on a multi-feature DAG network, which is carried out according to the following steps:

Step 1: Preprocess each retinal image in the training set to obtain a training set of feature images

Step 1.1 Obtain the characteristic image of retinal hemorrhage

Extract the green channel image in the RGB color mode of the retina image, grayscale the green channel image and divide it into several sub-blocks, count the cumulative distribution histogram of each sub-block, and set a finite threshold T _c in the histogram:

T _c =max(1,T _d ×h×w/S)

where T _d is the iterative adaptive soft threshold, S is the total pixels of the image, h and w are the length and width of the image;

Compare the gray value in the histogram with the set finite threshold T _c , and distribute the gray value area in the histogram that exceeds the finite threshold T _c evenly under the histogram and ensure that the total area of the histogram remains unchanged. Use the method of linear interpolation to optimize the processing to make the retinal hemorrhage feature prominent, that is, to obtain the retinal hemorrhage feature image;

Step 2.2 Obtain the characteristic image of fundus neovascularization

First, each pixel in the retinal image is convolved with 8 masks M _q , _q =1, 2, . Response, taking the maximum value G of the maximum response as the output of the edge magnitude image:

G ₌ max{|M0 _| ,|M1|,| _M2 |,| _M3 |,| _M4 |,| _M5 |,| _M6 |,| _M7 |}

Finally, the image is binarized according to the adaptive soft threshold to make the features of new blood vessels stand out, that is, the feature image of fundus new blood vessels is obtained;

Step 3.3 Obtain retinal varices feature images

Divide the retinal image into N sub-blocks, and iteratively calculate the clustering center C _v of retinal blood vessels in the retinopathy image and the corresponding degree of membership D _pv :

In the formula, N is the total number of sub-blocks, C is the number of clusters of clusters, x _p , p=1, 2,...., N represents the p-th sub-block, ||*|| represents the measure of any distance, m∈[1,∞) belongs to a weighting index; C _k represents the cluster center of the k-th class; when the membership degree satisfies the following iteration termination conditions, the iteration is stopped, and the local optimal value J _m is calculated to make the retinal varicose characteristics Prominent to obtain a characteristic image of retinal varicose vessels;

where l is the number of iteration steps, and ε is the error threshold;

Step 3.4 uses the acquired retinal feature image set as the feature image training set;

Step 2: Input the feature images in the feature image training set to the DAG network for training

Step 2.1 Establish an optimized DAG network. The optimized DAG network consists of a trunk, two branches, an add layer, a pooling layer avpool, and a full connection layer Full connect. The trunk is divided into five groups, each of which is It consists of a convolutional layer Conv, a normalization layer BN and an activation function layer relu. The two branches are both convolutional layers skipConv, and one trunk and two branches are linked with the add layer at the same time;

Step 2.2 Input the image of the feature image training set to the optimized DAG network, realize multi-feature fusion and learning, and output the multi-feature fusion result F _i ^add :

F _i ^add =(X _i +Y _i +Z _i )*K=X _i *K+Y _i *K+Z _i *K

In the formula, K represents the convolution kernel of the convolution layer, * represents the convolution, X _i represents the retinal hemorrhage spot feature, Y _i represents the fundus neovascularization feature; Z _i represents the retinal varicose feature;

Step 3: Send the multi-feature fusion result F _i ^add to the softmax classifier, and calculate the predicted probability of being normal or diseased according to the following formula, so as to achieve effective classification of diabetic retinopathy and normal;

In the formula, R is the prediction probability, i=1,2,...,M represents the ith image, M is the total number of retinal images, and e is a parameter;

When Y∈(0,0.5), the image is judged as normal, and when Y∈[0.5,1), the image is judged as lesion.

The present invention first uses different methods to extract the index features of diabetic retinopathy, including hemorrhagic spot features, fundus neovascularization features and retinal varices features; secondly, an optimized DAG network is constructed and the training scheme is continuously changed to train the network, so that the extracted The features of the multi-feature fusion and complex local or global features are composed of local features to restore the object; finally, the normal and lesion classification is performed by the softmax classifier. The present invention uses the DIARETDB1 data set and the data of Dalian Third People's Hospital to evaluate the performance of the classification model, and the results show that the classification accuracy rates of the present invention are 98.7% and 98.5% for the DIARETDB1 data set images and hospital data, respectively.

Description of drawings

FIG. 1 is a diagram showing the result of visual extraction of retinal hemorrhage spots according to an embodiment of the present invention.

FIG. 2 is a graph showing the result of extracting the characteristics of fundus neovascularization according to an embodiment of the present invention.

FIG. 3 is a diagram showing a result of extracting retinal varices features according to an embodiment of the present invention.

FIG. 4 is a structural diagram of a DAG network optimized according to an embodiment of the present invention.

FIG. 5 is an overall flow chart of an embodiment of the present invention.

Detailed ways

A method for classifying and modeling diabetic retinopathy based on a multi-feature DAG network of the present invention, as shown in Figure 5, is performed according to the following steps:

Step 1: Take the DIARETDB1 data set and the Dalian Third People's Hospital data set (hospital data for short), divide them into training set and test set, and preprocess each diabetic retina image in the training set to obtain a characteristic image training set

Step 1.1 Obtain the characteristic image of retinal hemorrhage

Extract the green channel image in the RGB color mode of the retina image, grayscale the green channel image and divide it into several sub-blocks, count the cumulative distribution histogram of each sub-block, and set a finite threshold T _c in the histogram:

T _c =max(1,T _d ×h×w/S)

where T _d is the iterative adaptive soft threshold, S is the total pixels of the image, h and w are the length and width of the image;

Compare the gray value in the histogram with the set finite threshold T _c , and distribute the gray value area in the histogram that exceeds the finite threshold T _c evenly under the histogram and ensure that the total area of the histogram remains unchanged. The linear interpolation method is used to optimize the transition of each sub-block to make the retinal hemorrhage feature prominent, that is, to obtain the retinal hemorrhage feature image, as shown in Figure 1;

Step 2.2 Obtain the characteristic image of fundus neovascularization

First, each pixel in the retinal image is convolved with 8 masks M _q , _q =1, 2, . Response, taking the maximum value G of the maximum response as the output of the edge magnitude image:

G ₌ max{|M0 _| ,|M1|,| _M2 |,| _M3 |,| _M4 |,| _M5 |,| _M6 |,| _M7 |}

Finally, the image is binarized according to the adaptive soft threshold to make the features of new blood vessels stand out, that is, the feature image of fundus new blood vessels is obtained, as shown in Figure 2;

Step 3.3 Obtain retinal varices feature images

Divide the retinal image into N sub-blocks, and iteratively calculate the clustering center C _v of retinal blood vessels in the retinopathy image and the corresponding degree of membership D _pv :

In the formula, N is the total number of sub-blocks, C is the number of clusters of clusters, x _p , p=1, 2,...., N represents the p-th sub-block, ||* represents the metric of any distance, m∈ [1,∞) belongs to a weighting index; C _k represents the cluster center of the kth class; when the membership degree satisfies the following iteration termination conditions, the iteration is stopped, and the local optimal value J _m is calculated to make the retinal varicose feature prominent, Obtain a characteristic image of retinal varicose vessels, as shown in Figure 3;

where l is the number of iteration steps, and ε is the error threshold;

Step 3.4 uses the acquired retinal feature image set as the feature image training set;

Step 2: Input the feature images in the feature image training set to the DAG network for training

Step 2.1 Establish an optimized DAG network. As shown in Figure 4, the optimized DAG network consists of a trunk, two branches, an add layer, a pooling layer avpool and a fully connected layer Full connect (fc). Divided into five groups, each group is composed of convolution layer Conv, normalization layer BN and activation function layer relu, the five groups are convolution layer Conv1-5, normalization layer BN1-5 and activation function layer relu1 -5 composition, the two branches are convolutional layers skipConv-1 and skipConv-2 respectively, and one trunk and two branches are connected to the add layer at the same time;

Step 2.2 Input the image of the feature image training set to the DAG network, realize multi-feature fusion and learning, and output the multi-feature fusion result F _i ^add :

F _i ^add =(X _i +Y _i +Z _i )*K=X _i *K+Y _i *K+Z _i *K

In the formula, K represents the convolution kernel of the convolution layer, * represents the convolution, X _i represents the retinal hemorrhage spot feature, Y _i represents the fundus neovascularization feature; Z _i represents the retinal varicose feature;

The training parameters and training parameter values are shown in Table 1.

Table 1

Step 3: Send the multi-feature fusion result F _i ^add to the softmax classifier, and calculate the predicted probability of being normal or diseased according to the following formula, so as to achieve effective classification of diabetic retinopathy and normal;

In the formula, R is the prediction probability, i=1,2,...,M represents the i-th image, M is the total number of diabetic retinal images, and e is a parameter;

When Y∈(0,0.5), the image is judged as normal, and when Y∈[0.5,1), the image is judged as lesion.

experiment:

Input the test set images in the DIARETDB1 data set and the Dalian Third People's Hospital data set (referred to as the hospital data set) into the model established by the embodiment of the present invention, and the model identifies and classifies the retinal images of the data set into normal fundus images. and lesion fundus images. For the retinal image binary classification problem, the evaluation indicators are the classification accuracy (Accurary), precision (Precision), recall (Recall), specificity (Specificity) and F1-score obtained after the data test set image enters the model. Evaluate the performance of the model to make the model more stable and reliable. The relevant formula is as follows:

where TP represents the number of correctly classified positive samples; TN represents the number of correctly classified negative samples; FP represents the number of falsely labeled positive samples among negative samples; FN represents the number of falsely labeled negative samples among positive samples.

In order to prove the importance of extracting features, the present invention uses hospital data to compare experimental results with its own algorithms that do not extract features and only extract one or two of the features. The comparison results are shown in Table 2.

Table 2

At the same time, in order to verify the performance of the model of the present invention, the same data set DIARETDB1 was selected to compare the performance of the traditional model and the model of the present invention, and the comparison results are shown in Table 3.

table 3

Note: '--' means missing value.

Claims (1)

1. a method for classifying and modeling diabetic retinopathy based on multi-feature DAG network, is characterized in that carrying out according to the following steps: Step 1: Preprocess each retinal image in the training set to obtain a training set of feature images Step 1.1 Obtain the characteristic image of retinal hemorrhage Extract the green channel image in the RGB color mode of the retina image, grayscale the green channel image and divide it into several sub-blocks, count the cumulative distribution histogram of each sub-block, and set a finite threshold T _c in the histogram: T _c =max(1,T _d ×h×w/S) where T _d is the iterative adaptive soft threshold, S is the total pixels of the image, h and w are the length and width of the image; Compare the gray value in the histogram with the set finite threshold T _c , distribute the gray value area in the histogram that exceeds the finite threshold T _c evenly under the histogram and ensure that the total area of the histogram remains unchanged, and finally Use the method of linear interpolation to optimize the processing to make the retinal hemorrhage feature prominent, that is, to obtain the retinal hemorrhage feature image; Step 2.2 Obtain the characteristic image of fundus neovascularization First, each pixel in the retinal image is convolved with 8 masks M _q , _q =1, 2, . Response, taking the maximum value G of the maximum response as the output of the edge magnitude image: G ₌ max{|M0 _| ,|M1|,| _M2 |,| _M3 |,| _M4 |,| _M5 |,| _M6 |,| _M7 |} Finally, the image is binarized according to the adaptive soft threshold to make the features of new blood vessels stand out, that is, the feature image of fundus new blood vessels is obtained; Step 3.3 Obtain retinal varices feature images Divide the retinal image into N sub-blocks, and iteratively calculate the clustering center C _v of retinal blood vessels in the retinopathy image and the corresponding degree of membership D _pv :

In the formula, N is the total number of sub-blocks, C is the number of clusters of clusters, x _p , p=1, 2,...., N represents the p-th sub-block, ||*|| represents the measure of any distance, m∈[1,∞) belongs to a weighting index; C _k represents the cluster center of the k-th class; when the membership degree satisfies the following iteration termination conditions, the iteration is stopped, and the local optimal value J _m is calculated to make the retinal varicose characteristics Prominent to obtain a characteristic image of retinal varicose vessels;

where l is the number of iteration steps, and ε is the error threshold;

Step 3.4 uses the acquired retinal feature image set as the feature image training set; Step 2: Input the feature images in the feature image training set to the DAG network for training Step 2.1 Establish an optimized DAG network. The optimized DAG network consists of a trunk, two branches, an add layer, a pooling layer avpool, and a full connection layer Full connect. The trunk is divided into five groups, each of which is It consists of a convolutional layer Conv, a normalization layer BN and an activation function layer relu. The two branches are both convolutional layers skipConv, and one trunk and two branches are linked with the add layer at the same time; Step 2.2 Input the image of the feature image training set to the optimized DAG network, realize multi-feature fusion and learning, and output the multi-feature fusion result F _i ^add : F _i ^add =(X _i +Y _i +Z _i )*K=X _i *K+Y _i *K+Z _i *K In the formula, K represents the convolution kernel of the convolution layer, * represents the convolution, X _i represents the retinal hemorrhage spot feature, Y _i represents the fundus neovascularization feature; Z _i represents the retinal varicose feature; Step 3: Send the multi-feature fusion result F _i ^add to the softmax classifier, and calculate the predicted probability of being normal or diseased according to the following formula, so as to realize the effective classification of diabetic retinopathy and normal;

In the formula, R is the prediction probability, i=1,2,...,M represents the ith image, M is the total number of retinal images, and e is a parameter; When Y∈(0,0.5), the image is judged as normal, and when Y∈[0.5,1), the image is judged as lesion.

Images