CN113139485A - Bone marrow cell classification and identification method, device and system based on deep learning - Google Patents
Bone marrow cell classification and identification method, device and system based on deep learning Download PDFInfo
- Publication number
- CN113139485A CN113139485A CN202110474397.8A CN202110474397A CN113139485A CN 113139485 A CN113139485 A CN 113139485A CN 202110474397 A CN202110474397 A CN 202110474397A CN 113139485 A CN113139485 A CN 113139485A
- Authority
- CN
- China
- Prior art keywords
- bone marrow
- power lens
- image
- smear
- low
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 210000002798 bone marrow cell Anatomy 0.000 title claims abstract description 71
- 238000000034 method Methods 0.000 title claims abstract description 46
- 238000013135 deep learning Methods 0.000 title claims abstract description 30
- 210000001185 bone marrow Anatomy 0.000 claims abstract description 51
- 238000013145 classification model Methods 0.000 claims abstract description 30
- 230000000007 visual effect Effects 0.000 claims abstract description 29
- 238000003384 imaging method Methods 0.000 claims abstract description 28
- 208000014951 hematologic disease Diseases 0.000 claims abstract description 26
- 238000001514 detection method Methods 0.000 claims abstract description 17
- 210000004027 cell Anatomy 0.000 claims description 29
- 238000006073 displacement reaction Methods 0.000 claims description 29
- 238000012549 training Methods 0.000 claims description 22
- 208000018706 hematopoietic system disease Diseases 0.000 claims description 13
- 208000019838 Blood disease Diseases 0.000 claims description 11
- 238000006243 chemical reaction Methods 0.000 claims description 4
- 238000004590 computer program Methods 0.000 claims description 4
- 238000010186 staining Methods 0.000 claims description 3
- 238000012545 processing Methods 0.000 abstract description 3
- 238000011206 morphological examination Methods 0.000 description 5
- 238000003745 diagnosis Methods 0.000 description 4
- 210000004698 lymphocyte Anatomy 0.000 description 4
- 210000001616 monocyte Anatomy 0.000 description 4
- 238000010586 diagram Methods 0.000 description 3
- 210000003743 erythrocyte Anatomy 0.000 description 3
- 210000003714 granulocyte Anatomy 0.000 description 3
- 238000012360 testing method Methods 0.000 description 3
- 208000014697 Acute lymphocytic leukaemia Diseases 0.000 description 2
- 208000031261 Acute myeloid leukaemia Diseases 0.000 description 2
- 208000033776 Myeloid Acute Leukemia Diseases 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 2
- 238000013523 data management Methods 0.000 description 2
- 238000004043 dyeing Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 210000000265 leukocyte Anatomy 0.000 description 2
- 230000000877 morphologic effect Effects 0.000 description 2
- 238000012795 verification Methods 0.000 description 2
- 206010000830 Acute leukaemia Diseases 0.000 description 1
- 208000024893 Acute lymphoblastic leukemia Diseases 0.000 description 1
- 206010002065 Anaemia megaloblastic Diseases 0.000 description 1
- 208000010839 B-cell chronic lymphocytic leukemia Diseases 0.000 description 1
- 208000032791 BCR-ABL1 positive chronic myelogenous leukemia Diseases 0.000 description 1
- 208000010833 Chronic myeloid leukaemia Diseases 0.000 description 1
- 208000031422 Lymphocytic Chronic B-Cell Leukemia Diseases 0.000 description 1
- 208000000682 Megaloblastic Anemia Diseases 0.000 description 1
- 208000033761 Myelogenous Chronic BCR-ABL Positive Leukemia Diseases 0.000 description 1
- 206010035226 Plasma cell myeloma Diseases 0.000 description 1
- 208000006664 Precursor Cell Lymphoblastic Leukemia-Lymphoma Diseases 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 208000003464 asthenopia Diseases 0.000 description 1
- 210000003651 basophil Anatomy 0.000 description 1
- 210000000601 blood cell Anatomy 0.000 description 1
- 208000032852 chronic lymphocytic leukemia Diseases 0.000 description 1
- 238000002790 cross-validation Methods 0.000 description 1
- 230000003583 cytomorphological effect Effects 0.000 description 1
- 230000004069 differentiation Effects 0.000 description 1
- 210000003979 eosinophil Anatomy 0.000 description 1
- 230000002489 hematologic effect Effects 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 210000003593 megakaryocyte Anatomy 0.000 description 1
- 231100001016 megaloblastic anemia Toxicity 0.000 description 1
- 238000001000 micrograph Methods 0.000 description 1
- 238000000386 microscopy Methods 0.000 description 1
- 201000000050 myeloid neoplasm Diseases 0.000 description 1
- 210000005170 neoplastic cell Anatomy 0.000 description 1
- 210000003924 normoblast Anatomy 0.000 description 1
- 239000002245 particle Substances 0.000 description 1
- 210000005259 peripheral blood Anatomy 0.000 description 1
- 239000011886 peripheral blood Substances 0.000 description 1
- 210000004180 plasmocyte Anatomy 0.000 description 1
- 238000004321 preservation Methods 0.000 description 1
- 210000004765 promyelocyte Anatomy 0.000 description 1
- 238000011160 research Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/60—Type of objects
- G06V20/69—Microscopic objects, e.g. biological cells or cellular parts
- G06V20/693—Acquisition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/60—Type of objects
- G06V20/69—Microscopic objects, e.g. biological cells or cellular parts
- G06V20/695—Preprocessing, e.g. image segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/60—Type of objects
- G06V20/69—Microscopic objects, e.g. biological cells or cellular parts
- G06V20/698—Matching; Classification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2411—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on the proximity to a decision surface, e.g. support vector machines
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A90/00—Technologies having an indirect contribution to adaptation to climate change
- Y02A90/10—Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Artificial Intelligence (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Investigating Or Analysing Biological Materials (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a bone marrow cell classification and identification method, device and system based on deep learning, and belongs to the field of medical image processing. The method comprises the following steps: (1) acquiring a low-power lens full-section image of the bone marrow smear by using a low-power lens; (2) cutting the low-power lens full-section image of the bone marrow smear into a plurality of low-power lens minimaps with the high-power lens imaging visual field size; (3) classifying the miniascape obtained in the step (2) by using a trained image classification model to obtain a miniascape with a good visual field; (4) scanning and imaging a small image of the low-power lens with a good visual field by using the high-power lens to obtain a corresponding high-power lens image; (5) classifying and counting bone marrow cells in the high power lens image obtained in the step (4) by using a trained target detection model; (6) and (5) predicting the blood diseases according to the classification and counting results of the bone marrow cells. Can reliably and automatically realize the identification and classification of bone marrow cells and can improve the accuracy of the prediction of blood diseases.
Description
Technical Field
The invention relates to a bone marrow cell classification and identification method, device and system based on deep learning, and belongs to the technical field of medical image processing.
Background
Microscopic examination and classification of blood cells has been an important basis for hematological diagnosis. Morphological examination of leukocytes in peripheral blood and bone marrow samples is the initial step in the diagnosis of blood diseases (e.g., acute myeloid leukemia, acute lymphoid leukemia, etc.), and among them, the FAB method, which is the most commonly used method for classifying acute leukemia, strongly depends on the morphology of cells.
Morphological analysis of white blood cells in bone marrow is critical for the diagnosis of hematological disorders, but to date, in practice, morphological examination of cells in bone marrow smears has relied on manual microscopy. The manual microscopic examination requires a certain professional knowledge and experience of an observer, requires a large amount of manpower and time, is tedious and time-consuming, and easily causes eye fatigue and human errors after long-time observation; and the selection of classification parts and the difficult selection and selection of cells are different along with the difference of observers, so that the subjectivity of the manual microscopic examination result is high, the standardization is difficult to realize, and the accuracy of classification counting is interfered by the factors, so that the reliability and the stability of the microscopic examination result are reduced.
In summary, the morphological examination of bone marrow cells by means of manual microscopic examination is complex and time-consuming, and the microscopic examination result has strong subjectivity and low reliability, so that the accuracy of blood disease prediction according to the result of the manual microscopic examination based on morphology is low. Therefore, the research of a reliable and automatic morphological examination method for morphological examination of bone marrow cells is of great significance for improving the overall level of diagnosis of blood diseases.
Disclosure of Invention
The invention aims to provide a bone marrow cell classification and identification method, device and system based on deep learning, which can reliably and automatically realize the identification and classification of bone marrow cells and can improve the accuracy of blood disease prediction.
In order to achieve the above object, the present invention provides a bone marrow cell classification and identification method based on deep learning, which comprises the following steps:
(1) acquiring a low-power lens full-section image of the bone marrow smear by using a low-power lens;
(2) cutting the low-power lens full-section image of the bone marrow smear into a plurality of low-power lens minimaps with the high-power lens imaging visual field size;
(3) classifying the low-power lens small image obtained in the step (2) by using a trained image classification model to obtain a low-power lens small image with a good visual field, wherein the low-power lens small image with the good visual field is a low-power lens small image with uniform cell distribution, good staining and less degraded cells; the input variable of the image classification model is a low power lens small image, and the output variable is whether a good visual field exists or not;
(4) scanning and imaging a small image of the low-power lens with a good visual field by using the high-power lens to obtain a corresponding high-power lens image;
(5) classifying and counting bone marrow cells in the high power lens image obtained in the step (4) by using a trained target detection model; the input variable of the target detection model is a high power lens image, and the output variable is the type and the number of cells contained in the high power lens image;
(6) and (5) predicting the blood diseases according to the classification and counting results of the bone marrow cells.
The method has the beneficial effects that: the imaging of the bone marrow smear and the classification and counting of the bone marrow cells can be automatically realized, and the blood disease prediction can be carried out according to the classification and counting results of the bone marrow cells, so that the method has the following beneficial effects: (1) the imaging of the bone marrow smear is automated, so that the reading of the smear by a plurality of people and the preservation of the case are facilitated; (2) the identification and classification of bone marrow cells are automatically realized, the detection efficiency is high, and the reproducibility is good; (3) the classification and counting results of the bone marrow cells are accurate and reliable, and the accuracy of blood disease prediction can be improved; (4) has the characteristic of autonomous learning.
Further, in the above method, the training sample for training the image classification model is a plurality of labeled hyposcope minigrams obtained by cropping the hyposcope full-section images of several cases of bone marrow smears of different blood diseases into a high-power scope imaging field size, the label of the hyposcope minigram is good field or poor field, and the label of each hyposcope minigram is labeled by an expert.
Further, in the above method, the training sample used for training the target detection model is a plurality of high power mirror images with labels, the high power mirror images are obtained by performing high power mirror scanning on a small low power mirror image with a good field of view in the training sample of the image classification model, the labels of the high power mirror images are cell types and positions in the high power mirror images, and the label of each high power mirror image is labeled by an expert.
Further, in the above method, the trained image classification model is a ResNet-50 classification model.
Further, in the above method, the trained target detection model is a fast R-CNN model.
The invention also provides a bone marrow cell classification and identification device based on deep learning, which comprises:
the smear imaging device comprises an objective group, a light source and a camera, wherein the objective group comprises a biological objective and an electric objective conversion table, and the light source comprises an LED light source and a condenser;
the CCD control and data acquisition system is in control connection with the camera;
the smear loading device consists of a smear fixture and a driving motor, wherein the smear fixture is used for accommodating a dyed bone marrow smear, and the driving motor is used for controlling the smear fixture to enter and exit the smear imaging device;
the electric displacement table comprises an XYZ-axis electric displacement device, the XY axis is used for moving the bone marrow smear so as to image the whole bone marrow smear, and the Z axis is used for automatically focusing the bone marrow smear;
the multi-axis displacement control system is in control connection with the driving motor, the objective lens group and the electric displacement table;
and the computer is in control connection with the CCD control and data acquisition system and the multi-axis displacement control system and is used for realizing the bone marrow cell classification and identification method based on deep learning.
The device has the advantages that: the bone marrow cell classification and identification method based on deep learning can be realized, the identification and classification of bone marrow cells can be reliably and automatically realized, and the accuracy of blood disease prediction can be improved.
Further, in the above device, the device further comprises a big data center for storing the bone marrow smear image, the bone marrow cell classification and counting result and the blood disease prediction result.
The invention also provides a bone marrow cell classification and identification system based on deep learning, which comprises a bone marrow smear image acquisition device, a memory and a processor, wherein the processor is used for executing a computer program stored in the memory to realize the bone marrow cell classification and identification method based on deep learning.
The system has the advantages that: the bone marrow cell classification and identification method based on deep learning can be realized, the identification and classification of bone marrow cells can be reliably and automatically realized, and the accuracy of blood disease prediction can be improved.
Drawings
FIG. 1 is a schematic diagram of a bone marrow cell classification and identification system according to an embodiment of the present invention;
FIG. 2 is a flow chart of a method for classifying and identifying bone marrow cells in an embodiment of the system of the present invention;
FIG. 3 is a schematic diagram illustrating image classification and target detection in a bone marrow cell classification and identification method according to an embodiment of the present invention;
FIG. 4 is a schematic diagram illustrating a bone marrow cell classification in an embodiment of the system of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments.
The embodiment of the system is as follows:
as shown in fig. 1, the bone marrow cell classification and identification system based on deep learning (hereinafter referred to as bone marrow cell classification and identification system) of this embodiment includes: smear imaging device, CCD control and data acquisition system, smear loading device, electric displacement table, multi-axis displacement control system, computer, big data center.
The smear imaging device is mainly used for acquiring clear marrow smear images for follow-up smear reading, and comprises an objective group, a light source and a camera (the three are positioned on the same straight line), wherein the objective group comprises a plurality of biological objectives (including but not limited to a 10-time objective and a 100-time objective) and an electric objective conversion table (for automatically converting imaging multiples); the light source comprises an LED light source and a condenser lens, and uniform illumination of the whole field of view is guaranteed; the camera is a high-resolution camera and is used for acquiring clear marrow smear images;
the CCD control and data acquisition system is connected with the camera and is used for controlling the camera to acquire images;
the smear loading device is used for loading dyed bone marrow smears and consists of a smear fixture and a driving motor, wherein the smear fixture can contain 1-8 dyed bone marrow smears, and the driving motor is used for controlling the smear fixture to go in and out of the whole detection device;
the electric displacement platform comprises an XYZ-axis electric displacement device, the XY axis is used for moving the bone marrow smear so as to image the whole bone marrow smear, and the Z axis is used for automatically focusing the bone marrow smear;
the multi-axis displacement control system is connected with the driving motor, the objective lens group and the electric displacement table and is used for controlling the displacement of each part;
the computer is connected with the CCD control and data acquisition system and the multi-axis displacement control system, is used for controlling the CCD control and data acquisition system and the multi-axis displacement control system, and is used for realizing a bone marrow cell classification and identification method (hereinafter referred to as a bone marrow cell classification and identification method) based on deep learning shown in FIG. 2, the method obtains bone marrow cell classification and counting results by processing the acquired bone marrow smear images, and carries out blood disease prediction according to the bone marrow cell classification and counting results;
the big data center realizes the storage of bone marrow smear images, bone marrow cell classification and counting results and blood disease prediction results. The big data center comprises a big data management server and a big database, the system can be connected with the big data management server and the big database through wired transmission or wireless transmission, and a user can upload scanned bone marrow picture information to the big database, so that the function of deep learning intelligent software is improved, and the generalization capability of the algorithm is improved. In addition, the system can be connected with different hospital systems for more experts to analyze the cell information.
The stained bone marrow smear is imaged, analyzed and counted by using the system, and the computer is used as a core for controlling, analyzing and storing data, so that the cells in the image obtained by the smear imaging device are identified, segmented and classified, and the number of the cells, and the spatial position, the contour and the morphological characteristics of each cell are obtained.
In practical applications, the bone marrow cell classifying and identifying system only needs to include the image capturing device, the memory and the processor, as long as the image capturing device has the functions of low power mirror scanning imaging and high power mirror scanning imaging, the memory stores a computer program capable of implementing the bone marrow cell classifying and identifying method, and the processor can implement the bone marrow cell classifying and identifying method by executing the computer program stored in the memory, and is not limited to the specific implementation manner given in the embodiment. In addition, the embodiment provides an image acquisition device comprising a smear imaging device, a CCD control and data acquisition system, a smear loading device, an electric displacement table and a multi-axis displacement control system, which is only a specific implementation manner of the image acquisition device.
Fig. 2 is a flowchart of a method for classifying and identifying bone marrow cells based on a bone marrow cell classification and identification system, which is specifically as follows:
(1) placing the dyed bone marrow smear into a smear loading device, shooting the bone marrow smear by a smear imaging device to record the overall information of the smear, wherein the overall information comprises the specific information of the patient of the bone marrow smear, the sample number and the smear overview;
(2) the method comprises the following steps of (1) recording the whole information of a bone marrow smear, then sending the bone marrow smear into an electric displacement table, controlling an objective lens group to be switched to a low power lens by a computer through a multi-shaft displacement control system, and firstly scanning the whole smear by the low power lens to obtain a low power lens full-section image of the bone marrow smear;
(3) automatically cutting the low-power lens full-section image of the bone marrow smear into a plurality of low-power lens small images with the high-power lens imaging view field size by the computer, namely cutting the low-power lens full-section image of the bone marrow smear into a plurality of low-power lens small images, wherein the view field of the low-power lens small images is exactly one view field presented under the high-power lens;
(4) classifying the low-power lens small images obtained in the step (3) by using a trained image classification model by the computer, and selecting the low-power lens small images with good visual fields (see the upper half part of the figure 3), wherein the low-power lens small images with good visual fields mainly refer to images with uniform cell distribution, good dyeing and less degraded cells;
in this embodiment, the trained image classification model is a ResNet-50 classification model, which can better balance time and accuracy. In practical applications, other types of image classification models, such as an SVM classification model or a VGG classification model, may also be used as needed.
In this embodiment, before classifying the low power mirror minimap obtained in step (3) by using the trained ResNet-50 classification model, the low power mirror minimap obtained in step (3) is scaled to a preset size, and the preset size is 224 × 224 for the ResNet-50 classification model; and then sending the minim image which is scaled to the preset size into a trained ResNet-50 classification model to obtain the minim image with a good visual field.
The training process of the ResNet-50 classification model is as follows:
in this example, the samples used to train the ResNet-50 classification model were from low power full-section images of about 300 bone marrow smears from different hospitals covering 6 types of blood diseases including acute myelogenous leukemia, acute lymphatic leukemia, chronic myelogenous leukemia, chronic lymphatic leukemia, myeloma, and megaloblastic anemia.
Cutting all low-power lens full-section images of all bone marrow pictures into a plurality of low-power lens small images with preset sizes, selecting a small image with a good visual field and a small image with a poor visual field by experts according to the cell distribution, dyeing and the number of degraded cells in the images, labeling each low-power lens small image, taking the labeled low-power lens small images as training samples to train a ResNet-50 classification model, selecting 20% of samples from the training samples as test sets in the training process, and randomly dividing the rest samples into the training sets and the verification sets according to a K-fold cross-validation method. The training set is used as network parameters for training a model and establishing the model, the verification set is used for evaluating the performance of the model in the training process and determining the structure of the network model, and the test set is used for finally selecting the optimal network model parameters. After the trained ResNet-50 classification model is obtained, the input low power lens small image can be automatically distinguished into a good visual field or a poor visual field by using the trained ResNet-50 classification model.
(5) The computer controls the objective lens group to be switched to the high-power lens through the multi-axis displacement system, and the high-power lens is used for scanning and imaging the small image of the low-power lens with good visual field to obtain an image of the high-power lens;
(6) and (3) classifying and counting the bone marrow cells in the high-power microscope image obtained in the step (5) by using the trained target detection model by using the computer (see the lower half part of the figure 3), predicting blood diseases according to the bone marrow cell classification and counting results, and uploading the results to a big data center for storage.
In this embodiment, the trained target detection model is a Fast R-CNN model, and as other embodiments, in practical applications, other types of target detection models, such as a YOLO model or a Fast R-CNN model, may also be selected as needed.
The training process of the Faster R-CNN model is as follows:
in this embodiment, a miniscope image with a good visual field for training a ResNet-50 classification model is scanned by a high power lens to obtain a corresponding high power lens image, then an expert judges cell types according to various characteristics of the size, particles, color, nucleus and the like of cells in the high power lens image by combining with cytomorphological standards and clinical practice, and marks the cell types and positions in all the high power lens images, so that each high power lens image is labeled, and the labeled high power lens images are used as training samples of a fast R-CNN model. The training sample comprises the characteristics of the position, the type and the number of cells contained in the high-power lens visual field, when the Faster R-CNN model is trained, the input variable is a high-power lens image, and the output variable is the type and the number of the cells contained in the high-power lens image.
Bone marrow cells can be classified into 17 classes (see FIG. 4) using the trained Faster R-CNN model, including 7 classes, respectively: granulocytes, monocytes, lymphocytes, erythrocytes, plasma cells, megakaryocytes, neoplastic cells. Among them, granulocytes are classified into: primitive granulocytes, promyelocytes, mesogranulocytes, metagranulocytes, baculocytes, subtenocytes, basophils, eosinophils; monocytes are divided into: primary and naive monocytes, mature monocytes; lymphocytes are classified into: primary and naive lymphocytes, mature lymphocytes; the red blood cells are divided into: primary and naive erythrocytes, mid-late erythroblasts.
In this example, when a hematological disease is predicted based on the results of bone marrow cell classification and counting, differentiation is performed based on a hematological disease classification standard according to the ratio of the cell types.
Through tests, the average accuracy of cell classification realized by the bone marrow cell classification and identification system of the embodiment on more than 300 cases reaches more than 90%, and the prediction accuracy of blood diseases reaches more than 97%.
The method comprises the following steps:
as shown in fig. 2, the method for classifying and identifying bone marrow cells based on deep learning (hereinafter referred to as bone marrow cell classification and identification method) of the present embodiment includes the following steps:
(1) acquiring a low-power lens full-section image of the bone marrow smear by using a low-power lens;
(2) cutting the low-power lens full-section image of the bone marrow smear into a plurality of low-power lens minimaps with the high-power lens imaging visual field size;
(3) classifying the low-power lens small image obtained in the step (2) by using a trained image classification model to obtain a low-power lens small image with a good visual field, wherein the low-power lens small image with the good visual field is a low-power lens small image with uniform cell distribution, good staining and less degraded cells; the input variable of the image classification model is a minimap of a low power lens, and the output variable is whether a good visual field exists or not;
(4) scanning and imaging a small image of the low-power lens with a good visual field by using the high-power lens to obtain a corresponding high-power lens image;
(5) classifying and counting bone marrow cells in the high power lens image obtained in the step (4) by using a trained target detection model; the input variable of the target detection model is a high power lens image, and the output variable is the type and the number of cells contained in the high power lens image;
(6) and (5) predicting the blood diseases according to the classification and counting results of the bone marrow cells.
The specific implementation of the bone marrow cell classification and identification method based on deep learning is shown in the system examples, and is not described herein again.
The embodiment of the device is as follows:
as shown in fig. 1, the bone marrow cell classification and identification device based on deep learning of the present embodiment includes:
the smear imaging device comprises an objective group, a light source and a camera, wherein the objective group comprises a biological objective and an electric objective conversion table, and the light source comprises an LED light source and a condenser;
the CCD control and data acquisition system is in control connection with the camera;
the smear loading device consists of a smear fixture and a driving motor, wherein the smear fixture is used for accommodating a dyed bone marrow smear, and the driving motor is used for controlling the smear fixture to enter and exit the smear imaging device;
the electric displacement platform comprises an XYZ-axis electric displacement device, the XY axis is used for moving the bone marrow smear so as to image the whole bone marrow smear, and the Z axis is used for automatically focusing the bone marrow smear;
the multi-axis displacement control system is connected with the driving motor, the objective lens group and the electric displacement table in a control mode;
the computer is in control connection with the CCD control and data acquisition system and the multi-axis displacement control system and is used for realizing the bone marrow cell classification and identification method based on deep learning as shown in figure 2;
and the big data center is used for storing the bone marrow smear image, the bone marrow cell classification and counting result and the blood disease prediction result.
The specific implementation of the bone marrow cell classifying and identifying device based on deep learning is referred to the bone marrow cell classifying and identifying system in the system embodiment, which is not described herein again.
Claims (8)
1. A bone marrow cell classification and identification method based on deep learning is characterized by comprising the following steps:
(1) acquiring a low-power lens full-section image of the bone marrow smear by using a low-power lens;
(2) cutting the low-power lens full-section image of the bone marrow smear into a plurality of low-power lens minimaps with the high-power lens imaging visual field size;
(3) classifying the low-power lens small image obtained in the step (2) by using a trained image classification model to obtain a low-power lens small image with a good visual field, wherein the low-power lens small image with the good visual field is a low-power lens small image with uniform cell distribution, good staining and less degraded cells; the input variable of the image classification model is a low power lens small image, and the output variable is whether a good visual field exists or not;
(4) scanning and imaging a small image of the low-power lens with a good visual field by using the high-power lens to obtain a corresponding high-power lens image;
(5) classifying and counting bone marrow cells in the high power lens image obtained in the step (4) by using a trained target detection model; the input variable of the target detection model is a high power lens image, and the output variable is the type and the number of cells contained in the high power lens image;
(6) and (5) predicting the blood diseases according to the classification and counting results of the bone marrow cells.
2. The bone marrow cell classification and identification method based on deep learning of claim 1, characterized in that the training samples for training the image classification model are several labeled hyposcope minigrams obtained by cropping hyposcope full-section images of several bone marrow smears of different blood diseases into a high-power scope imaging field of view size, the label of the hyposcope minigram is good field of view or poor field of view, and the label of each hyposcope minigram is labeled by an expert.
3. The bone marrow cell classification and identification method based on deep learning of claim 2, characterized in that the training samples for training the target detection model are several high power lens images with labels, the high power lens images are obtained by performing high power lens scanning on a small low power lens image with a good visual field in the training samples of the image classification model, the labels of the high power lens images are cell types and positions in the high power lens images, and the label of each high power lens image is labeled by an expert.
4. The deep learning-based bone marrow cell classification and identification method according to any one of claims 1-3, wherein the trained image classification model is ResNet-50 classification model.
5. The deep learning based bone marrow cell classification and identification method according to any one of claims 1-3, wherein the trained target detection model is fast R-CNN model.
6. A bone marrow cell classification and identification device based on deep learning is characterized by comprising:
the smear imaging device comprises an objective group, a light source and a camera, wherein the objective group comprises a biological objective and an electric objective conversion table, and the light source comprises an LED light source and a condenser;
the CCD control and data acquisition system is in control connection with the camera;
the smear loading device consists of a smear fixture and a driving motor, wherein the smear fixture is used for accommodating a dyed bone marrow smear, and the driving motor is used for controlling the smear fixture to enter and exit the smear imaging device;
the electric displacement table comprises an XYZ-axis electric displacement device, the XY axis is used for moving the bone marrow smear so as to image the whole bone marrow smear, and the Z axis is used for automatically focusing the bone marrow smear;
the multi-axis displacement control system is in control connection with the driving motor, the objective lens group and the electric displacement table;
a computer, which is connected with the CCD control and data acquisition system and the multi-axis displacement control system and is used for realizing the bone marrow cell classification and identification method based on deep learning of any one of claims 1 to 5.
7. The deep learning based bone marrow cell classification and identification device according to claim 6, characterized in that the device further comprises a big data center for storing bone marrow smear images, bone marrow cell classification and counting results and blood disease prediction results.
8. A deep learning based bone marrow cell differential identification system, characterized in that the system comprises a bone marrow smear image collecting device, a memory and a processor, wherein the processor is used for executing a computer program stored in the memory to realize the deep learning based bone marrow cell differential identification method of any one of claims 1-5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110474397.8A CN113139485B (en) | 2021-04-29 | 2021-04-29 | Bone marrow cell classification and identification method, device and system based on deep learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110474397.8A CN113139485B (en) | 2021-04-29 | 2021-04-29 | Bone marrow cell classification and identification method, device and system based on deep learning |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113139485A true CN113139485A (en) | 2021-07-20 |
| CN113139485B CN113139485B (en) | 2023-11-17 |
Family
ID=76816436
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110474397.8A Active CN113139485B (en) | 2021-04-29 | 2021-04-29 | Bone marrow cell classification and identification method, device and system based on deep learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113139485B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114187480A (en) * | 2022-01-05 | 2022-03-15 | 北京瀚远医药科技有限公司 | Bone marrow image cell image detection and classification method and system based on deep learning |
| CN115578598A (en) * | 2022-10-26 | 2023-01-06 | 北京大学第三医院（北京大学第三临床医学院） | Bone marrow cell identification method and system based on convolutional neural network |
| CN118628557A (en) * | 2024-07-18 | 2024-09-10 | 深圳市生强科技有限公司 | Method and system for rapidly positioning living cells and observing growth cycle and application of method and system |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1553165A (en) * | 2003-12-19 | 2004-12-08 | 武汉大学 | Automatic multispectral cell smear and analyzing instrument and method used for cervical cell analysis |
| CN105223110A (en) * | 2014-06-27 | 2016-01-06 | 苏州惠生电子科技有限公司 | A kind of microscope locating and tracking formation method, device and urinal system |
| CN109815974A (en) * | 2018-12-10 | 2019-05-28 | 清影医疗科技（深圳）有限公司 | A kind of cell pathology slide classification method, system, equipment, storage medium |
| CN110765855A (en) * | 2019-09-12 | 2020-02-07 | 杭州迪英加科技有限公司 | Pathological image processing method and system |
| CN111289510A (en) * | 2018-12-07 | 2020-06-16 | 苏州迈瑞科技有限公司 | In-vitro diagnostic apparatus, image switching method, and readable storage medium |
| CN112507969A (en) * | 2020-12-24 | 2021-03-16 | 重庆德方信息技术有限公司 | Automatic positioning method for microscopic image cells |
| KR20210045091A (en) * | 2019-10-16 | 2021-04-26 | (주)유아이엠디 | Bone marrow imaging method using bone marrow interpretation |
-
2021
- 2021-04-29 CN CN202110474397.8A patent/CN113139485B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1553165A (en) * | 2003-12-19 | 2004-12-08 | 武汉大学 | Automatic multispectral cell smear and analyzing instrument and method used for cervical cell analysis |
| CN105223110A (en) * | 2014-06-27 | 2016-01-06 | 苏州惠生电子科技有限公司 | A kind of microscope locating and tracking formation method, device and urinal system |
| CN111289510A (en) * | 2018-12-07 | 2020-06-16 | 苏州迈瑞科技有限公司 | In-vitro diagnostic apparatus, image switching method, and readable storage medium |
| CN109815974A (en) * | 2018-12-10 | 2019-05-28 | 清影医疗科技（深圳）有限公司 | A kind of cell pathology slide classification method, system, equipment, storage medium |
| CN110765855A (en) * | 2019-09-12 | 2020-02-07 | 杭州迪英加科技有限公司 | Pathological image processing method and system |
| KR20210045091A (en) * | 2019-10-16 | 2021-04-26 | (주)유아이엠디 | Bone marrow imaging method using bone marrow interpretation |
| CN112507969A (en) * | 2020-12-24 | 2021-03-16 | 重庆德方信息技术有限公司 | Automatic positioning method for microscopic image cells |
Non-Patent Citations (2)
| Title |
|---|
| WEI-CHE HUANG ET AL.: "Automatic HCC Detection Using Convolutional Network with Multi-Magnification Input Images", 2019 IEEE INTERNATIONAL CONFERENCE ON ARTIFICIAL INTELLIGENCE CIRCUITS AND SYSTEMS (AICAS), pages 194 - 198 * |
| 柴政: "宫颈细胞图像拼接和融合技术算法研究及应用", 中国优秀硕士学位论文全文数据库医药卫生科技辑, vol. 2020, no. 02, pages 068 - 29 * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114187480A (en) * | 2022-01-05 | 2022-03-15 | 北京瀚远医药科技有限公司 | Bone marrow image cell image detection and classification method and system based on deep learning |
| CN115578598A (en) * | 2022-10-26 | 2023-01-06 | 北京大学第三医院（北京大学第三临床医学院） | Bone marrow cell identification method and system based on convolutional neural network |
| CN115578598B (en) * | 2022-10-26 | 2023-09-05 | 北京大学第三医院（北京大学第三临床医学院） | Bone marrow cell identification method and system based on convolutional neural network |
| CN118628557A (en) * | 2024-07-18 | 2024-09-10 | 深圳市生强科技有限公司 | Method and system for rapidly positioning living cells and observing growth cycle and application of method and system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113139485B (en) | 2023-11-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113139485B (en) | Bone marrow cell classification and identification method, device and system based on deep learning | |
| JP6453298B2 (en) | System and method for observing and analyzing cytological specimens | |
| US6148096A (en) | Specimen preview and inspection system | |
| CN101685060B (en) | Sample imaging apparatus | |
| US6430309B1 (en) | Specimen preview and inspection system | |
| US20070031043A1 (en) | System for and method of intelligently directed segmentation analysis for automated microscope systems | |
| US10324022B2 (en) | Analysis accuracy improvement in automated testing apparatus | |
| CN111504885B (en) | Analysis method of full-automatic blood smear morphological analysis device based on machine vision | |
| CN112825622B (en) | Sample image capturing method and sample image capturing apparatus | |
| JP3916395B2 (en) | Test system with sample pretreatment function | |
| CN106706643B (en) | A kind of liver cancer comparison slice detection method | |
| CN111062346A (en) | Automatic leukocyte positioning detection and classification recognition system and method | |
| WO2005121863A1 (en) | Automated diagnosis of malaria and other infections | |
| KR20010017092A (en) | Method for counting and analyzing morphology of blood cell automatically | |
| CN112001315A (en) | Bone marrow cell classification and identification method based on transfer learning and image texture features | |
| CN111656247B (en) | Cell image processing system, cell image processing method, automatic film reading device and storage medium | |
| CN112924452A (en) | Blood examination auxiliary system | |
| CN113192080A (en) | Portable cell imaging and intelligent analysis system based on edge calculation | |
| JP2007504478A (en) | A system for classifying slides using a scatter plot distribution | |
| US20200074628A1 (en) | Image processing apparatus, imaging system, image processing method and computer readable recoding medium | |
| WO2021134474A1 (en) | Sample analysis system and method | |
| CN114252385B (en) | Portable leukocyte subclass detection device and method | |
| CN115629072A (en) | Bone marrow smear image analysis and diagnosis method and pathological section scanner device | |
| WO2023031622A1 (en) | System and method for identifying and counting biological species | |
| CN114518362A (en) | Sperm quality analysis device, system, method and readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |