CN114241187A - Muscle disease diagnosis system, device and medium based on ultrasonic bimodal images - Google Patents
Muscle disease diagnosis system, device and medium based on ultrasonic bimodal images Download PDFInfo
- Publication number
- CN114241187A CN114241187A CN202111413100.3A CN202111413100A CN114241187A CN 114241187 A CN114241187 A CN 114241187A CN 202111413100 A CN202111413100 A CN 202111413100A CN 114241187 A CN114241187 A CN 114241187A
- Authority
- CN
- China
- Prior art keywords
- roi
- muscle disease
- image
- disease diagnosis
- muscle
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 208000029578 Muscle disease Diseases 0.000 title claims abstract description 83
- 238000003745 diagnosis Methods 0.000 title claims abstract description 79
- 230000002902 bimodal effect Effects 0.000 title claims abstract description 39
- 238000007781 pre-processing Methods 0.000 claims abstract description 22
- 238000012545 processing Methods 0.000 claims abstract description 11
- 230000004927 fusion Effects 0.000 claims abstract description 7
- 238000003384 imaging method Methods 0.000 claims description 30
- 238000012549 training Methods 0.000 claims description 30
- 210000003205 muscle Anatomy 0.000 claims description 26
- 238000002604 ultrasonography Methods 0.000 claims description 25
- 238000004590 computer program Methods 0.000 claims description 10
- 238000010606 normalization Methods 0.000 claims description 10
- 238000011176 pooling Methods 0.000 claims description 10
- 210000000544 articulatio talocruralis Anatomy 0.000 claims description 8
- 238000004422 calculation algorithm Methods 0.000 claims description 8
- 230000006870 function Effects 0.000 claims description 7
- 238000009432 framing Methods 0.000 claims description 4
- 230000003189 isokinetic effect Effects 0.000 claims description 2
- 238000004519 manufacturing process Methods 0.000 claims 1
- 238000000034 method Methods 0.000 abstract description 13
- 210000002027 skeletal muscle Anatomy 0.000 description 11
- 238000010586 diagram Methods 0.000 description 10
- 201000000585 muscular atrophy Diseases 0.000 description 9
- 230000008569 process Effects 0.000 description 7
- 206010028289 Muscle atrophy Diseases 0.000 description 6
- 230000020763 muscle atrophy Effects 0.000 description 6
- 210000002569 neuron Anatomy 0.000 description 6
- 238000013528 artificial neural network Methods 0.000 description 5
- 238000012360 testing method Methods 0.000 description 5
- 239000002131 composite material Substances 0.000 description 4
- 230000035945 sensitivity Effects 0.000 description 4
- 210000003484 anatomy Anatomy 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 3
- 238000013527 convolutional neural network Methods 0.000 description 3
- 238000013135 deep learning Methods 0.000 description 3
- 230000004913 activation Effects 0.000 description 2
- 210000003423 ankle Anatomy 0.000 description 2
- 238000013473 artificial intelligence Methods 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 238000002591 computed tomography Methods 0.000 description 2
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 2
- 230000009977 dual effect Effects 0.000 description 2
- 210000002683 foot Anatomy 0.000 description 2
- 230000001575 pathological effect Effects 0.000 description 2
- 230000009467 reduction Effects 0.000 description 2
- 238000012216 screening Methods 0.000 description 2
- 238000012795 verification Methods 0.000 description 2
- 206010003694 Atrophy Diseases 0.000 description 1
- 208000029549 Muscle injury Diseases 0.000 description 1
- 238000005481 NMR spectroscopy Methods 0.000 description 1
- 206010002026 amyotrophic lateral sclerosis Diseases 0.000 description 1
- 230000037444 atrophy Effects 0.000 description 1
- 238000012512 characterization method Methods 0.000 description 1
- 238000007635 classification algorithm Methods 0.000 description 1
- 238000003759 clinical diagnosis Methods 0.000 description 1
- 230000006378 damage Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000013399 early diagnosis Methods 0.000 description 1
- 238000002592 echocardiography Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 230000003902 lesion Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 201000006938 muscular dystrophy Diseases 0.000 description 1
- 210000001087 myotubule Anatomy 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 210000003699 striated muscle Anatomy 0.000 description 1
Images
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B8/00—Diagnosis using ultrasonic, sonic or infrasonic waves
- A61B8/08—Detecting organic movements or changes, e.g. tumours, cysts, swellings
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B8/00—Diagnosis using ultrasonic, sonic or infrasonic waves
- A61B8/48—Diagnostic techniques
- A61B8/485—Diagnostic techniques involving measuring strain or elastic properties
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B8/00—Diagnosis using ultrasonic, sonic or infrasonic waves
- A61B8/52—Devices using data or image processing specially adapted for diagnosis using ultrasonic, sonic or infrasonic waves
- A61B8/5215—Devices using data or image processing specially adapted for diagnosis using ultrasonic, sonic or infrasonic waves involving processing of medical diagnostic data
- A61B8/5223—Devices using data or image processing specially adapted for diagnosis using ultrasonic, sonic or infrasonic waves involving processing of medical diagnostic data for extracting a diagnostic or physiological parameter from medical diagnostic data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Health & Medical Sciences (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- General Health & Medical Sciences (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Molecular Biology (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Artificial Intelligence (AREA)
- Radiology & Medical Imaging (AREA)
- Surgery (AREA)
- Veterinary Medicine (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Public Health (AREA)
- Animal Behavior & Ethology (AREA)
- Medical Informatics (AREA)
- Heart & Thoracic Surgery (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Pathology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Software Systems (AREA)
- Mathematical Physics (AREA)
- Evolutionary Biology (AREA)
- Computing Systems (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computational Linguistics (AREA)
- Physiology (AREA)
- Ultra Sonic Daignosis Equipment (AREA)
Abstract
The invention is suitable for the technical field of biological signal processing, and provides a muscle disease diagnosis system based on ultrasonic bimodal images, an electronic device and a storage medium, wherein the method comprises the following steps: the muscle disease diagnosis system based on the ultrasonic bimodal image comprises a data preprocessing module and a muscle disease diagnosis module, wherein the data preprocessing module is used for preprocessing an ultrasonic bimodal image sequence of a subject to obtain a plurality of size-normalized ROI image pairs, each ROI image pair comprises an ROI gray image and an ROI elastic image, the muscle disease diagnosis module is used for performing channel fusion on each ROI gray image and each ROI elastic image, inputting the fused ROI gray image and each ROI elastic image into a pre-trained muscle disease diagnosis model, and outputting a muscle disease diagnosis result, so that the full-automatic diagnosis of muscle diseases is realized, and the muscle diseases are diagnosed by two diagnosis modes based on an ultrasonic gray mode and an elastic mode, so that the diagnosis accuracy is improved.
Description
Technical Field
The invention belongs to the technical field of biological signal processing, and particularly relates to a muscle disease diagnosis system based on ultrasonic bimodal images, electronic equipment and a storage medium.
Background
Muscle atrophy refers to the reduction of muscle volume caused by striated muscle dystrophy, thinning and even loss of muscle fibers, and the like. Current Imaging modalities for clinically identifying muscle atrophy are primarily nuclear Magnetic Resonance (MRI), Computed Tomography (CT), and Ultrasound (US). The good characterization of muscle structure by ultrasound is helpful in understanding the status of muscle atrophy or injury, and is currently the most widespread method for quantitatively evaluating the characteristics of muscle structures. Gray scale ultrasound (B-mode) imaging can detect changes in the pathological structural features of muscles and can also characterize the condition of muscles by measuring echoes and inhomogeneities obtained as gray scale differences. Just because muscles have a specific texture, the use of grayscale ultrasound to observe ultrasound images of muscles enables the detection of muscle lesions. However, when the anatomical structure of the muscle is often shown to be changed on the grayscale ultrasound image, the atrophy degree of the muscle is already very serious, and a single ultrasound grayscale mode has a problem of insufficient sensitivity, and cannot provide a basis for early diagnosis.
In recent years, with the rapid development of artificial intelligence and deep learning, in the medical field, artificial intelligence can rapidly identify pictures and images and extract important information from the pictures and images, so as to help doctors to rapidly acquire information required for diagnosis. Because the traditional machine learning method needs to manually extract the features and carry out screening design, human subjective factors are inevitably mixed in the whole process of feature extraction and dimension reduction screening, and the extracted features can not comprehensively represent the information of the image.
Aiming at the defects of the existing clinical diagnosis technology and the lack of wide application of deep learning in the field of muscular atrophy, the patent utilizes the unique biomechanical characteristics of skeletal muscles, acquires a real-time ultrasonic bimodal image sequence during passive stretching, and adopts a convolutional neural network algorithm to provide a full-automatic identification and classification algorithm of muscular atrophy under an ultrasonic multimodal image.
Disclosure of Invention
The invention aims to provide a muscle disease diagnosis system based on ultrasonic bimodal images, an electronic device and a storage medium, and aims to solve the problem that the prior art cannot realize full-automatic muscle disease diagnosis.
In one aspect, the present invention provides a muscle disease diagnosis system based on ultrasound bimodal imaging, the muscle disease diagnosis system comprising:
the data preprocessing module is used for preprocessing an ultrasonic bimodal image sequence of a subject to obtain a plurality of size-normalized ROI image pairs, and each ROI image pair comprises an ROI grayscale image and an ROI elastic image;
and the muscle disease diagnosis module is used for inputting the ROI grayscale images and the ROI elastic images of each group into a muscle disease diagnosis model trained in advance after channel fusion is carried out on the ROI grayscale images and the ROI elastic images, and outputting a muscle disease diagnosis result.
Preferably, the data preprocessing module further comprises a frame processing module, a cropping module, and a normalization module, wherein,
the frame processing module is used for performing framing and equidistant frame taking operations on the ultrasonic bimodal image sequence to obtain a plurality of groups of original image pairs;
the cutting module is used for cutting ROI areas of each frame of image in a plurality of groups of original image pairs to obtain a plurality of original ROI image pairs;
the normalization module is used for unifying the original multiple groups of ROI image pairs to a fixed size to obtain the multiple groups of ROI image pairs with normalized sizes.
Preferably, the ultrasound bimodal image sequence is an elastic image sequence and a gray image sequence of a muscle tissue, which are acquired synchronously through a shear wave elastic imaging modality and a gray imaging modality of an ultrasound system when the ankle joint of the subject performs a constant-speed passive motion.
Preferably, the ankle joint of the subject performs the uniform passive motion from 40 degrees of plantarflexion to 40 degrees of dorsiflexion under the drive of a constant-velocity muscle strength training system, and the subject does not provide any main force or resistance by himself.
Preferably, the muscle disease diagnosis system further comprises a data acquisition module, a data set making module and a model training module, wherein,
the data acquisition module is used for acquiring an ultrasonic bimodal image sequence of a subject;
the dataset generation module to generate a base dataset based on the normalized ROI grayscale and ROI elasticity images of a plurality of subjects and corresponding label data;
the model training module is used for training the muscle disease diagnosis model by using the basic data set to obtain the trained muscle disease diagnosis model.
Preferably, the muscle disease diagnosis model is used for diagnosing a muscle atrophy disease, and includes four convolution layers, a full-link layer and a softmax layer which are connected in sequence, wherein the first two convolution layers each include two sub-convolution layers and one maximum pooling layer, the second two convolution layers each include three sub-convolution layers and one maximum pooling layer, the convolution kernel size of each sub-convolution layer is 3 × 3, the convolution kernel size of each maximum pooling layer is 2 × 2, and the number of nodes of the full-link layer is 512.
Preferably, the input of each network layer except the input layer in the muscle disease diagnosis model is a feature obtained by normalizing the output of the previous layer by using a BN algorithm.
Preferably, the fully connected layer of the muscle disease diagnostic model is followed by a dropout layer.
In another aspect, the present invention also provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method when executing the computer program.
In another aspect, the present invention also provides a computer-readable storage medium storing a computer program which, when executed by a processor, implements the steps of the method as described above.
The muscle disease diagnosis system based on the ultrasonic bimodal image comprises a data preprocessing module and a muscle disease diagnosis module, wherein the data preprocessing module is used for preprocessing an ultrasonic bimodal image sequence of a subject to obtain a plurality of size-normalized ROI image pairs, each ROI image pair comprises an ROI gray image and an ROI elastic image, the muscle disease diagnosis module is used for performing channel fusion on each ROI gray image and each ROI elastic image, inputting the fused ROI gray image and each ROI elastic image into a pre-trained muscle disease diagnosis model, and outputting a muscle disease diagnosis result, so that the full-automatic diagnosis of muscle diseases is realized, and the muscle diseases are diagnosed based on two diagnosis modes of an ultrasonic gray mode and an elastic mode, so that the diagnosis accuracy is improved.
Drawings
FIG. 1A is a schematic structural diagram of a muscle disease diagnosis system based on ultrasound bimodal imaging according to an embodiment of the present invention;
FIG. 1B is a flowchart illustrating a pre-processing of an elastic image sequence according to an embodiment of the present invention;
fig. 1C is a diagram illustrating a comparison of neural network structures with or without dropout according to an embodiment of the present invention;
fig. 2 is a schematic structural diagram of an electronic device according to a third embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
The following detailed description of specific implementations of the present invention is provided in conjunction with specific embodiments:
the first embodiment is as follows:
fig. 1A is a schematic structural diagram of a muscle disease diagnosis system based on ultrasound bimodal imaging according to an embodiment of the present invention, which only shows the relevant parts of the embodiment of the present invention for convenience of description, and the details are as follows:
in the embodiment of the present invention, the muscle disease diagnosis system 1 based on the ultrasound bimodal image includes a data preprocessing module 11 and a muscle disease diagnosis module 12, the data preprocessing module is configured to preprocess an ultrasound bimodal image sequence of a subject to obtain a plurality of sets of ROI (Region of Interest) image pairs with normalized size, each set of ROI image pairs includes a ROI grayscale image and a ROI elasticity image, and the muscle disease diagnosis module is configured to perform channel fusion on each set of ROI grayscale image and ROI elasticity image, and input the result into a muscle disease diagnosis model trained in advance, and output a muscle disease diagnosis result.
In the embodiment of the invention, the anatomical structure of the muscle can be imaged based on the gray-scale ultrasonic mode, the structural parameters such as the volume, the thickness and the cross-sectional area of the muscle can be obtained, and the shear wave elastic imaging mode can noninvasively and quantitatively obtain the hardness value of the muscle tissue, so that the muscle diseases are diagnosed by an ultrasonic bimodal image sequence and deep learning. Wherein the ultrasound bimodal image sequence comprises a sequence of grayscale images of skeletal muscle acquired under an ultrasound grayscale imaging modality (B-mode imaging modality), and a sequence of elasticity images of skeletal muscle acquired under an ultrasound shear wave elasticity imaging modality. Wherein each elastic image in the sequence of elastic images is displayed superimposed on the gray scale image.
Preferably, the ultrasonic bimodal image sequence is an elastic image sequence and a gray level image sequence of muscle tissues acquired synchronously through a shear wave elastic imaging mode and a gray level imaging mode of an ultrasonic system when the ankle joint of the subject performs a constant-speed passive motion, so that the ultrasonic bimodal image sequence when the skeletal muscle stretches is acquired by using unique biomechanical characteristics of the skeletal muscle, and a more comprehensive basis is provided for diagnosis of subsequent muscle diseases. Further preferably, the ankle joint of the subject performs a uniform passive motion from plantarflexion of 40 ° to dorsiflexion of 40 ° under the drive of the constant muscle strength training system, and the subject does not autonomously provide any main force or resistance force to complete the acquisition of the ultrasonic bimodal image sequence through the uniform passive motion.
In the specific implementation, the examinee can adopt a supine position, the right foot of the examinee is fixed on a pedal of the constant-speed muscle strength training system, the constant-speed muscle strength training system drives the ankle of the examinee to perform constant-speed passive motion from 40 degrees of plantarflexion to 40 degrees of dorsiflexion, the detected skeletal muscle is pulled to cause passive stretching, the structure of the skeletal muscle is continuously imaged under a gray imaging mode of the ultrasonic imaging system, and meanwhile, an elastic image sequence of the skeletal muscle is synchronously acquired under a shear wave elastic imaging mode of the ultrasonic imaging system. In the whole process of constant-speed passive rotation of the ankle joint, a subject is required to completely relax the muscles of the lower leg and not provide any main force or resistance autonomously.
Preferably, the data preprocessing module further includes a frame processing module, a clipping module and a normalization module, wherein the frame processing module is configured to perform framing and equidistant frame capturing operations on the ultrasound bimodal image sequence to obtain a plurality of sets of original image pairs, the clipping module is configured to clip an ROI region of each frame of image in the plurality of sets of original image pairs to obtain a plurality of original sets of ROI image pairs, and the normalization module is configured to unify the plurality of original sets of ROI image pairs to a fixed size to obtain a plurality of sets of ROI image pairs with normalized size, so as to implement preprocessing of the ultrasound bimodal image sequence.
In the specific implementation, an ultrasound bimodal image sequence, that is, a grayscale image sequence and an elastic image sequence, are framed, and N frames are extracted at equal intervals, for example, N is 10, and based on a shear wave elastic imaging modality, an original elastic image is displayed by being superimposed on the grayscale image, so that each set of obtained original image pairs includes an original grayscale image and an original elastic grayscale composite image displayed by being superimposed on the original grayscale image and the original elastic image, then, an ROI region of the original grayscale image and the original elastic grayscale composite image is cut out, an original ROI grayscale image and an ROI elastic grayscale composite image are obtained, and then, the ROI elastic grayscale composite image is subtracted from the grayscale image, so that the original ROI elastic image and the ROI grayscale image, that is, the original ROI image pair are obtained. FIG. 1B is a diagram illustrating an exemplary flow of pre-processing an elastic image sequence.
Preferably, the muscle disease diagnosis system further comprises a data acquisition module, a data set making module and a model training module, wherein the data acquisition module is used for acquiring an ultrasonic bimodal image sequence of the subjects, the data set generation module is used for generating a basic data set based on normalized ROI gray-scale images and ROI elasticity images of a plurality of subjects and corresponding label data, and the model training module is used for training the muscle disease diagnosis model by using the basic data set to obtain the trained muscle disease diagnosis model so as to realize the training of the muscle disease diagnosis model. In a specific implementation, the basic data set can be divided into a training set, a verification set and a test set, wherein the model trained by the training set needs to use the verification set to determine the network structure or parameters for controlling the complexity of the model, and the test set is used for checking the performance of the model.
Preferably, the muscle disease diagnosis model is used for diagnosing muscle atrophy diseases, and comprises four convolution layers, a full-link layer and a softmax layer which are connected in sequence, wherein the first two convolution layers each comprise two sub-convolution layers and a maximum pooling layer, the last two convolution layers each comprise three sub-convolution layers and one maximum pooling layer, the convolution kernel size of each sub-convolution layer is 3 × 3, the convolution kernel size of each maximum pooling layer is 2 × 2, and the number of nodes of the full-link layer is 512, so that the muscle disease diagnosis model is simplified, and the parameter number and the calculation amount are reduced.
Because the input data is artificially normalized by each sample, the essence of the training network is an updating parameter, and the updating of the weight of the previous layer of network can cause the distribution of the output data to change, the distribution of the input data of each layer of the rest network layers except the input layer is changed, thereby greatly reducing the training speed of the network. Preferably, the inputs of each network layer except the input layer in the muscle disease diagnosis model are characteristic features obtained by normalizing the output of the previous layer by using a BN (Batch Normalization) algorithm, so as to solve the problem that the distribution of the hidden layer data changes in the network training process and accelerate the model training speed.
In the concrete implementation, before each layer of the network is input, the output data distribution of the previous layer can be normalized by using a BN algorithm and then can enter the next layer of the network, so that the problem that the data distribution of the hidden layer is changed in the process of training the network can be solved. However, when the output data of the previous layer is normalized, the characteristics learned by the network of the previous layer cannot be damaged, so that learnable parameters gamma and beta are introduced, the average value and the variance of all neurons of the characteristic diagram corresponding to all samples are calculated, and then the normalization processing is performed on the neurons of the whole characteristic diagram. The method comprises the following specific steps:
1): introducing initial values of parameters beta, gamma and epsilon to obtain data B ═ x before inputting the activation function1,x2,…xn}; step two: calculating the mean value mu of the inputBSum variance
2): inputting the input data intoLine normalization processing is carried out to obtain normalized output data
Where ε is an arbitrary value close to 0.
3): training parameters β, γ:
wherein,
the partial derivatives are calculated, and l is the loss value.
5): output data y after batch standardization operationi：
After the BN algorithm is introduced, higher learning rate is allowed, and training is accelerated. And by introducing learnable reconstruction parameters beta and gamma, the network can learn and restore the characteristic distribution to be learned by the original network.
The full-connection layer maps the distributed feature description learned by operations such as convolution, pooling and activation to a sample mark space, so that the influence of feature positions on classification is greatly reduced, and the full-connection layer plays a role of a classifier in the whole convolutional neural network. But its main disadvantage is that it contains a large number of parameters and requires complex calculations during the training process. Preferably, a dropout layer is introduced after a full connection layer of the muscle disease diagnosis model, so that nodes and connections are eliminated by using a dropout technology, and the generalization of the model is enhanced so that the model is not too dependent on some local features. The main function of Dropout is to let some neurons stop working randomly, so as to eliminate joint adaptability among neuron nodes and enhance generalization capability of the network. Because the clinical medical data set is small, a large number of parameters are difficult to train when the deep convolutional neural network is trained, and the over-fitting problem that the training set has good performance but the classification effect on the test set is poor is easily caused. The Dropout mechanism discards some neurons randomly according to a certain probability so as to effectively prevent overfitting of the network, enhance sparsity of the network and reduce network parameters. Fig. 1C is a comparison diagram of a neural network structure with or without a dropout layer, and in fig. 1C, (a) the diagram is not added with dropout, and (b) the diagram sets dropout to 0.5, that is, a certain neuron stops working with a probability of 0.5.
The muscle disease diagnosis model enhances the generalization ability of the model after introducing a BN algorithm and a Dropout layer, and the network parameters of the muscle disease diagnosis model are shown in the following table 1:
TABLE 1
It should be noted that the muscle disease diagnosis model can also be used for diagnosis of other muscle diseases, such as amyotrophic lateral sclerosis, muscular dystrophy, etc. In diagnosing other muscle diseases, the number of layers of the neural network and the size of the convolution kernel may be altered based on the muscle disease to be diagnosed.
The muscle disease diagnosis system based on the ultrasonic bimodal image comprises a data preprocessing module and a muscle disease diagnosis module, wherein the data preprocessing module is used for preprocessing an ultrasonic bimodal image sequence of a subject to obtain a plurality of ROI image pairs with normalized sizes, each ROI image pair comprises an ROI gray image and an ROI elastic image, the muscle disease diagnosis module is used for performing channel fusion on each ROI gray image and each ROI elastic image, inputting the fused ROI gray image and the image into a pre-trained muscle disease diagnosis model, and outputting a muscle disease diagnosis result, so that the full-automatic diagnosis of muscle diseases is realized, and the muscle diseases are diagnosed by the scheme based on two diagnosis modes of an ultrasonic gray mode and an elastic mode, so that the diagnosis accuracy is improved.
Example two:
this example further illustrates the muscle disease diagnosis system in the first example with reference to the experimental examples.
(1) The basic data set:
clinical data set: all the testees adopt a supine position, the right foot of each tester is fixed on a pedal of the isokinetic muscle strength training system, the ankle is driven to do uniform-speed passive motion from 40 degrees of plantarflexion to 40 degrees of dorsiflexion, and the tested skeletal muscle is drawn to cause passive stretching. In the whole process of constant-speed passive rotation of the ankle joint, a subject is required to completely relax the muscles of the lower leg and not provide any main force or resistance autonomously.
Acquisition of a bimodal image sequence: the method comprises the steps of imaging a skeletal muscle structure by utilizing a gray level mode of an ultrasonic imaging system, displaying structural form change of the muscle during passive stretching, acquiring an elastic image sequence of the skeletal muscle by utilizing a shear wave elastic imaging mode of the ultrasonic imaging system, and acquiring different elastic image sequences of the muscle tissue in the passive stretching process, wherein an imaging area is 13mm x 7mm, and the sequence duration is 28-32 s.
Data preprocessing: using opencv to complete framing operation on the acquired bimodal image sequence, uniformly and equidistantly extracting 9 frames to obtain a gray image, subtracting a gray ultrasonic imaging region from an imaging region of a shear wave elastic mode to obtain an original elastic image, intercepting an ROI region of each frame of image to obtain an ROI image pair, carrying out normalization processing on each group of ROI image pairs, and normalizing the processed image size to 224 x 224.
(2) Building a model:
a muscle disease diagnosis model is constructed according to the description in the first embodiment, the model introduces a BN algorithm and a Dropout layer, the experimental example names the muscle disease diagnosis model MAVGG-BN, and the detailed parameters are shown in the table 1.
(3) And (3) model evaluation:
and (3) respectively training and classifying the gray ultrasonic data, the elastic data and the bimodal data fused with the gray ultrasonic data and the elastic data by using MAVGG-BN, and comparing the performances of the gray ultrasonic data, the elastic data and the bimodal data. Specifically, the image is initialized to 224 × 224 size, input to MAVGG-BN for training, the first layer convolutional layer size is 3 × 3, and the stride size is 1. The batch (batch) size is set to 16 and the number of iterations (epoch) is 50 rounds. The experimental parameters were set as shown in table 2.
TABLE 2
Comparing the diagnosis result of the pathological examination of muscle atrophy with the diagnosis result given by a doctor according to the result predicted by the model, and classifying the test result into a True class (TP), a False Negative class (FP), a True Negative class (TN) and a False Negative class (FN). From these four cases, accuracy, sensitivity, F1-score, was calculated to measure the model's prediction. The calculation formula is as follows:
the results of the classification of tests on muscular atrophy after training of a gray-scale ultrasound modality (B-mode), a shear wave elasticity imaging modality (SWE) and a fusion dual modality (B-mode + SWE) on the magvg-BN neural network are shown in table 3, where Acc represents accuracy, Sen represents sensitivity, Spe represents specificity and Pre represents accuracy. According to the experimental result, the accuracy rate of the corresponding model classification in the dual mode is 95.56%, the accuracy rate is obviously better than that in the gray ultrasonic mode, the accuracy rate is respectively 9.45% higher than that in the gray mode and 2.78% higher than that in the elastic mode, and the index values of the sensitivity and the F1 score are also higher than those in the elastic mode and the gray ultrasonic mode.
TABLE 3
Therefore, the problem of low single-mode diagnosis performance is solved by taking the fused data of the anatomical structure and the elastic information of the gray ultrasonic imaging as the input of the neural network.
Example three:
fig. 2 shows a structure of an electronic device according to a third embodiment of the present invention, and for convenience of description, only a part related to the third embodiment of the present invention is shown.
The electronic device 2 of an embodiment of the invention comprises a processor 20, a memory 21 and a computer program 22 stored in the memory 21 and executable on the processor 20. The processor 20, when executing the computer program 22, implements the functions of the modules in the system embodiments described above, for example, the functions of the units 11 to 12 shown in fig. 1A.
Example four:
in an embodiment of the present invention, a computer-readable storage medium is provided, which stores a computer program that, when executed by a processor, implements functions of units in the above-described apparatus embodiments, for example, the functions of the units 11 to 12 shown in fig. 1A.
The computer readable storage medium of the embodiments of the present invention may include any entity or device capable of carrying computer program code, a recording medium, such as a ROM/RAM, a magnetic disk, an optical disk, a flash memory, or the like.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents and improvements made within the spirit and principle of the present invention are intended to be included within the scope of the present invention.
Claims (10)
1. A muscle disease diagnostic system based on ultrasound bimodal imaging, the muscle disease diagnostic system comprising:
the data preprocessing module is used for preprocessing an ultrasonic bimodal image sequence of a subject to obtain a plurality of size-normalized ROI image pairs, and each ROI image pair comprises an ROI grayscale image and an ROI elastic image;
and the muscle disease diagnosis module is used for inputting the ROI grayscale images and the ROI elastic images of each group into a muscle disease diagnosis model trained in advance after channel fusion is carried out on the ROI grayscale images and the ROI elastic images, and outputting a muscle disease diagnosis result.
2. The muscle disease diagnostic system of claim 1, wherein the data pre-processing module further comprises a frame processing module, a cropping module, and a normalization module, wherein,
the frame processing module is used for performing framing and equidistant frame taking operations on the ultrasonic bimodal image sequence to obtain a plurality of groups of original image pairs;
the cutting module is used for cutting ROI areas of each frame of image in a plurality of groups of original image pairs to obtain a plurality of original ROI image pairs;
the normalization module is used for unifying the original multiple groups of ROI image pairs to a fixed size to obtain the multiple groups of ROI image pairs with normalized sizes.
3. The muscle disease diagnostic system of claim 1, wherein the ultrasound bimodal image sequence is an elastic image sequence and a gray scale image sequence of muscle tissue acquired simultaneously by a shear wave elasticity imaging modality and a gray scale imaging modality of an ultrasound system while the ankle joint of the subject is in constant velocity passive motion.
4. A muscle disease diagnostic system as claimed in claim 3, wherein the ankle joint of the subject performs the uniform passive movement from plantarflexion 40 ° to dorsiflexion 40 ° under the driving of the isokinetic muscle strength training system, and the subject does not provide any main force or resistance by himself.
5. The muscle disease diagnostic system of claim 1, further comprising a data acquisition module, a data set production module, and a model training module, wherein,
the data acquisition module is used for acquiring an ultrasonic bimodal image sequence of a subject;
the dataset generation module to generate a base dataset based on the normalized ROI grayscale and ROI elasticity images of a plurality of subjects and corresponding label data;
the model training module is used for training the muscle disease diagnosis model by using the basic data set to obtain the trained muscle disease diagnosis model.
6. The muscle disease diagnosis system according to claim 1, wherein the muscle disease diagnosis model includes four convolutional layers, a fully-connected layer, and a softmax layer, which are connected in this order, wherein the first two convolutional layers each include two sub-convolutional layers and one maximum pooling layer, the second two convolutional layers each include three sub-convolutional layers and one maximum pooling layer, the convolutional kernel size of each sub-convolutional layer is 3 x 3, the convolutional kernel size of each maximum pooling layer is 2 x 2, and the number of nodes of the fully-connected layer is 512.
7. The muscle disease diagnosis system according to claim 6, wherein the input of each network layer other than the input layer in the muscle disease diagnosis model is a feature obtained by normalizing the output of the previous layer using the BN algorithm.
8. The muscle disease diagnosis system according to claim 6, wherein a dropout layer is introduced after the fully connected layer of the muscle disease diagnosis model.
9. An electronic device, characterized in that the electronic device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that the processor implements the functions of the muscle disease diagnostic system according to any one of claims 1 to 8 when executing the computer program.
10. A computer-readable storage medium, in which a computer program is stored, which, when being executed by a processor, carries out the functions of a muscle disease diagnosis system according to any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111413100.3A CN114241187A (en) | 2021-11-25 | 2021-11-25 | Muscle disease diagnosis system, device and medium based on ultrasonic bimodal images |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111413100.3A CN114241187A (en) | 2021-11-25 | 2021-11-25 | Muscle disease diagnosis system, device and medium based on ultrasonic bimodal images |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114241187A true CN114241187A (en) | 2022-03-25 |
Family
ID=80751183
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111413100.3A Pending CN114241187A (en) | 2021-11-25 | 2021-11-25 | Muscle disease diagnosis system, device and medium based on ultrasonic bimodal images |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114241187A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115177291A (en) * | 2022-08-01 | 2022-10-14 | 首都医科大学附属北京朝阳医院 | Method and device for identifying acquired anergy in intensive care unit |
| WO2024051016A1 (en) * | 2022-09-08 | 2024-03-14 | 深圳大学 | Ultrasound muscle image analysis method for small model animal |
| CN117911775A (en) * | 2024-01-22 | 2024-04-19 | 重庆大学 | Automatic intelligent diagnosis method for double-mode fusion interpretable breast cancer |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108961215A (en) * | 2018-06-05 | 2018-12-07 | 上海大学 | Parkinson's disease assistant diagnosis system and method based on Multimodal medical image |
| CN109035160A (en) * | 2018-06-29 | 2018-12-18 | 哈尔滨商业大学 | The fusion method of medical image and the image detecting method learnt based on fusion medical image |
| CN112288733A (en) * | 2020-11-06 | 2021-01-29 | 深圳先进技术研究院 | Muscle ultrasonic image detection method, system, terminal and storage medium |
-
2021
- 2021-11-25 CN CN202111413100.3A patent/CN114241187A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108961215A (en) * | 2018-06-05 | 2018-12-07 | 上海大学 | Parkinson's disease assistant diagnosis system and method based on Multimodal medical image |
| CN109035160A (en) * | 2018-06-29 | 2018-12-18 | 哈尔滨商业大学 | The fusion method of medical image and the image detecting method learnt based on fusion medical image |
| CN112288733A (en) * | 2020-11-06 | 2021-01-29 | 深圳先进技术研究院 | Muscle ultrasonic image detection method, system, terminal and storage medium |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115177291A (en) * | 2022-08-01 | 2022-10-14 | 首都医科大学附属北京朝阳医院 | Method and device for identifying acquired anergy in intensive care unit |
| CN115177291B (en) * | 2022-08-01 | 2023-11-10 | 首都医科大学附属北京朝阳医院 | Method and device for identifying myasthenia gravis acuminata in intensive care unit |
| WO2024051016A1 (en) * | 2022-09-08 | 2024-03-14 | 深圳大学 | Ultrasound muscle image analysis method for small model animal |
| CN117911775A (en) * | 2024-01-22 | 2024-04-19 | 重庆大学 | Automatic intelligent diagnosis method for double-mode fusion interpretable breast cancer |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114241187A (en) | Muscle disease diagnosis system, device and medium based on ultrasonic bimodal images | |
| CN112070119A (en) | Ultrasonic tangent plane image quality control method and device and computer equipment | |
| Militante et al. | Pneumonia detection through adaptive deep learning models of convolutional neural networks | |
| WO2018155765A1 (en) | Method and device analyzing plaque from computed tomography image | |
| Tobias et al. | CNN-based deep learning model for chest X-ray health classification using tensorflow | |
| Balasooriya et al. | Intelligent brain hemorrhage diagnosis using artificial neural networks | |
| WO2022095254A1 (en) | Muscle ultrasonic image detection method and system, terminal, and storage medium | |
| CN114694236A (en) | Eyeball motion segmentation positioning method based on cyclic residual convolution neural network | |
| CN111462082A (en) | Focus picture recognition device, method and equipment and readable storage medium | |
| CN113782184A (en) | Cerebral apoplexy auxiliary evaluation system based on facial key point and feature pre-learning | |
| CN114398979A (en) | Ultrasonic image thyroid nodule classification method based on feature decoupling | |
| CN113397485A (en) | Scoliosis screening method based on deep learning | |
| CN113313714B (en) | Coronary OCT (optical coherence tomography) image lesion plaque segmentation method based on improved U-Net network | |
| CN110827275B (en) | Liver nuclear magnetic artery image quality grading method based on raspberry pie and deep learning | |
| CN116433560A (en) | Muscle disease identification method, device and equipment based on ultrasonic bimodal images | |
| CN116597950A (en) | Medical image layering method | |
| CN109480842B (en) | System and apparatus for diagnosing functional dyspepsia | |
| CN114743664A (en) | Gait analysis and determination learning-based spinal cervical spondylosis auxiliary diagnosis system | |
| CN114041758A (en) | Radial artery palpation positioning method and device, electronic equipment and storage medium | |
| IT202100029405A1 (en) | A contrast enhancement medium for diagnostic imaging methods and systems | |
| CN114549558B (en) | Breast tumor segmentation method based on multimodal feature fusion Vnet | |
| Gunasekara et al. | A feasibility study for deep learning based automated brain tumor segmentation using magnetic resonance images | |
| CN117036302B (en) | Method and system for determining calcification degree of aortic valve | |
| CN118512173B (en) | Deep learning-based children hearing detection method and system | |
| Islami et al. | Customized Convolutional Neural Network for Glaucoma Detection in Retinal Fundus Images |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |