Nothing Special   »   [go: up one dir, main page]

CN111240004A - System and method for automatically identifying two insects by microscope - Google Patents

System and method for automatically identifying two insects by microscope Download PDF

Info

Publication number
CN111240004A
CN111240004A CN202010165944.XA CN202010165944A CN111240004A CN 111240004 A CN111240004 A CN 111240004A CN 202010165944 A CN202010165944 A CN 202010165944A CN 111240004 A CN111240004 A CN 111240004A
Authority
CN
China
Prior art keywords
microscope
picture
program
software
computer
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202010165944.XA
Other languages
Chinese (zh)
Inventor
陈辉
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
HKY TECHNOLOGY Co.,Ltd.
Original Assignee
Beijing Ailuke Technology Testing Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Ailuke Technology Testing Co ltd filed Critical Beijing Ailuke Technology Testing Co ltd
Priority to CN202010165944.XA priority Critical patent/CN111240004A/en
Publication of CN111240004A publication Critical patent/CN111240004A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B21/00Microscopes
    • G02B21/0004Microscopes specially adapted for specific applications
    • G02B21/002Scanning microscopes
    • G02B21/0024Confocal scanning microscopes (CSOMs) or confocal "macroscopes"; Accessories which are not restricted to use with CSOMs, e.g. sample holders
    • G02B21/0052Optical details of the image generation
    • G02B21/0076Optical details of the image generation arrangements using fluorescence or luminescence
    • CCHEMISTRY; METALLURGY
    • C12BIOCHEMISTRY; BEER; SPIRITS; WINE; VINEGAR; MICROBIOLOGY; ENZYMOLOGY; MUTATION OR GENETIC ENGINEERING
    • C12QMEASURING OR TESTING PROCESSES INVOLVING ENZYMES, NUCLEIC ACIDS OR MICROORGANISMS; COMPOSITIONS OR TEST PAPERS THEREFOR; PROCESSES OF PREPARING SUCH COMPOSITIONS; CONDITION-RESPONSIVE CONTROL IN MICROBIOLOGICAL OR ENZYMOLOGICAL PROCESSES
    • C12Q1/00Measuring or testing processes involving enzymes, nucleic acids or microorganisms; Compositions therefor; Processes of preparing such compositions
    • C12Q1/02Measuring or testing processes involving enzymes, nucleic acids or microorganisms; Compositions therefor; Processes of preparing such compositions involving viable microorganisms
    • C12Q1/04Determining presence or kind of microorganism; Use of selective media for testing antibiotics or bacteriocides; Compositions containing a chemical indicator therefor
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01NINVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/62Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light
    • G01N21/63Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light optically excited
    • G01N21/64Fluorescence; Phosphorescence
    • G01N21/645Specially adapted constructive features of fluorimeters
    • G01N21/6456Spatial resolved fluorescence measurements; Imaging
    • G01N21/6458Fluorescence microscopy
    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B21/00Microscopes
    • G02B21/36Microscopes arranged for photographic purposes or projection purposes or digital imaging or video purposes including associated control and data processing arrangements
    • G02B21/361Optical details, e.g. image relay to the camera or image sensor
    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B21/00Microscopes
    • G02B21/36Microscopes arranged for photographic purposes or projection purposes or digital imaging or video purposes including associated control and data processing arrangements
    • G02B21/365Control or image processing arrangements for digital or video microscopes
    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B21/00Microscopes
    • G02B21/36Microscopes arranged for photographic purposes or projection purposes or digital imaging or video purposes including associated control and data processing arrangements
    • G02B21/365Control or image processing arrangements for digital or video microscopes
    • G02B21/367Control or image processing arrangements for digital or video microscopes providing an output produced by processing a plurality of individual source images, e.g. image tiling, montage, composite images, depth sectioning, image comparison

Landscapes

  • Physics & Mathematics (AREA)
  • Chemical & Material Sciences (AREA)
  • Engineering & Computer Science (AREA)
  • Analytical Chemistry (AREA)
  • Multimedia (AREA)
  • General Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Optics & Photonics (AREA)
  • Organic Chemistry (AREA)
  • Proteomics, Peptides & Aminoacids (AREA)
  • Biochemistry (AREA)
  • General Health & Medical Sciences (AREA)
  • Immunology (AREA)
  • Zoology (AREA)
  • Wood Science & Technology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Pathology (AREA)
  • Biotechnology (AREA)
  • Microbiology (AREA)
  • Molecular Biology (AREA)
  • Biophysics (AREA)
  • Toxicology (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • General Engineering & Computer Science (AREA)
  • Genetics & Genomics (AREA)
  • Investigating, Analyzing Materials By Fluorescence Or Luminescence (AREA)

Abstract

The invention discloses an automatic amphibian recognition system and method for a microscope, the automatic amphibian recognition system for the microscope comprises a fluorescence microscope and a CCD camera, the fluorescence microscope is provided with a fluorescence power supply, the control system comprises a motion control card and a motion controller, the data processing system is a computer, a control program and a recognition program run on the computer, the fluorescence microscope is provided with the CCD camera, the CCD camera is connected with the computer, the computer and the motion controller are both connected with the motion control card, the motion control card is connected with the fluorescence microscope, and the control program is used for collecting data of the CCD camera. Has the advantages that: the detection efficiency is greatly improved, the automatic identification system does not need professional operation, the detection threshold is greatly reduced, the machine identification accuracy can be continuously improved through continuous training and upgrading of the identification algorithm, the problem of large human error is thoroughly solved, and the two-insect detection is more concise and efficient.

Description

System and method for automatically identifying two insects by microscope
Technical Field
The invention relates to the field of giardia and cryptosporidium identification, in particular to a microscope system and a microscope method for automatically identifying two worms.
Background
Giardia and cryptosporidium (abbreviated as 'two worms') are pathogenic microorganisms widely existing in nature, and the detection methods of the two worms in China comprise an immunomagnetic separation fluorescent antibody method in the national standard 'domestic drinking water standard inspection method' (GB5750-2006) and a filter membrane concentration/density gradient separation fluorescent antibody method and a filter capsule concentration/density gradient separation fluorescent antibody method in the industry standard 'town water quality standard inspection method' (CJT 141-2008). The three methods all comprise a process of identifying the two insects by a fluorescent microscope, the process completely depends on manual identification at present, the requirement on the professional level of an identifier is high, and a detector can accurately identify the two insects only after long-term training. In the identification process, the detection personnel need to stare at the microscope for a long time, so that eye fatigue is easily caused, identification errors are caused, the accuracy is low, in addition, manual identification cannot be continuously operated for a long time, and the effective identification time causes the reduction of the whole detection working efficiency.
An effective solution to the problems in the related art has not been proposed yet.
Disclosure of Invention
Aiming at the problems in the related art, the invention provides a system and a method for automatically identifying two insects by a microscope, so as to overcome the technical problems in the prior related art.
The technical scheme of the invention is realized as follows:
according to one aspect of the invention, a microscope automatic identification two-insect system is provided.
The microscope automatic two-insect recognition system comprises an imaging system, a control system and a data processing system, the imaging system includes a fluorescence microscope configured with a fluorescence power source and a CCD camera, the control system comprises a motion control card and a motion controller, the data processing system is a computer, the computer runs a control program and an identification program, the fluorescence microscope is provided with a CCD camera, the CCD camera is connected with the computer, the computer and the motion controller are both connected with the motion control card, the motion control card is connected with a fluorescence microscope, the control program is used for collecting the CCD camera data, the control program is used for controlling the movement of the objective lens and the objective table of the fluorescence microscope, and the identification program can identify the authenticity of the two insects and acquire position coordinates based on artificial intelligence image processing.
Further, the CCD camera is connected with the computer through a USB data line.
Furthermore, the motion control card is connected with the computer through a USB data line.
Furthermore, the objective lens and the objective table of the fluorescence microscope are both connected with the motion control card.
According to another aspect of the present invention, a method for a microscope to automatically identify a two-insect system is provided.
The microscope automatic identification system for the two insects comprises the following steps:
opening a fluorescence microscope switch, a fluorescence power switch and a motion control card switch, and connecting a motion control card USB line and a CCD data USB line to a computer;
setting a port number com of a USB line of the motion control card, wherein a specific checking path is as follows: this computer-property-device manager-port;
the software end clicks the 'connection microscope', when the connection is successful, the interface pops up a 'connection success' word, a 'confirmation' dialog box is clicked to disappear, and then a picture storage path is set;
observing through an ocular lens, moving the position of a sample through a motion controller, generally defining the upper left corner point of a circumscribed rectangle of a circle drawn by the sample as a starting point, clicking a 'set starting point' button of a software interface after moving in place, then observing through the ocular lens, moving the sample to the lower right corner position of the rectangle, clicking a 'set end point' button of the software interface, switching a microscope from ocular lens observation to CCD observation after finishing setting, and then clicking a 'start scanning' button;
before scanning is started, software can detect the multiple of the objective lens used at present, if the multiple is not 20 times of the objective lens, the objective lens is automatically switched into the 20 times of the objective lens, snake-shaped scanning is started from a starting point position defined by a user, the CCD shoots at the same time, the shot pictures are stored in a path folder designated by the user, and at the moment, the user only needs to wait, and all working software can be automatically completed;
after scanning under the 20-time objective lens is finished, the software can identify the scanned picture, circle out suspicious pictures and coordinates, generate a suspicious picture parameter list in a suspicious object list on the left upper side of the software, then the software enables the microscope to automatically switch the 40-time objective lens through a command, the objective table moves to a first suspicious position, focusing and photographing are carried out again, and the like is carried out until photographing of the Nth coordinate is finished;
after the 40-time objective lens is photographed, the software can recognize the photographed picture again, the two-insect result and the coordinate list are in the microorganism list on the left lower side of the software to generate a microorganism parameter list, and all work is automatically completed;
the user can double click certain parameter information in the two lists through the mouse to enable the microscope objective table to quickly move to the coordinate position, and the user can verify the result through the eyepiece.
Further, the image recognition program is developed based on the python language, and the software used includes python, pytorech, scimit-spare, numpy, and opencv-python.
Further, after the 20 times objective lens finishes scanning, the software can identify the scanned picture, the background of the picture display picture collected by the fluorescence microscope is usually black, and the bug is usually green, according to the characteristic, the program takes the green channel value in the RGB three-color channel as the judgment basis, and the suspected region selection algorithm is formulated as follows:
in image preprocessing, in order to increase the execution speed of the program, the input picture is first subjected to size compression, and the picture is compressed to 1/4 of the original picture by default, and the ratio can be adjusted by modifying the "rate" parameter in the program. Then, since the background color of part of the picture is not black, in order to avoid the influence caused by the background color, the picture is subjected to mean filtering. Finally, the program only focuses on the green channel value, so all other color channel values are set to 0.
Scanning the picture pixel by pixel, respectively scanning the picture from the horizontal direction and the vertical direction by adopting a loop nesting mode, and adjusting the loop step length by modifying a step parameter in a program to increase the scanning speed, wherein the default is 2. The larger the number, the shorter the scanning time, but since the smaller the size of the bug, it is easy to cause omission, and therefore it needs to be selected properly.
Judging whether the green channel numerical value of the pixel point meets the condition or not, setting a threshold value [35, 50] for the green channel numerical value through experiments, if the green channel numerical value of the pixel point is in the threshold value range, storing the coordinate of the point into a bug _ cord list, and if not, carrying out no treatment on the point. The threshold range can be adjusted by modifying a 'threshold' parameter in a program, and the larger the range is, the more suspected areas are found, which brings burden to the subsequent identification work; the smaller the range, the greater the probability of missed detection.
The clustering of the coordinates of the pixel points in the list is performed by DBSCAN, the clustering of the DBSCAN is a clustering algorithm based on density, adjacent and close points in low-latitude data can be aggregated into one class, and the class number and the clustering center do not need to be specified in advance, so that the method is very suitable for determining a suspected area. The pixel coordinates stored in the bug _ code list are clustered, and each category represents a suspected area.
Further, the coordinate point determination specifically includes: and clustering to obtain a plurality of categories, averaging the coordinate values of the pixel points in each category to obtain the central coordinate of the suspected area, and selecting a square box as the suspected area by taking the coordinate as the center, wherein the default size of the square box is 60 pixels by 60 pixels.
After the shooting under the 40-time objective lens is completed, the software can recognize the shot picture again, the recognition program reads the high-magnification picture, the two insects are recognized by adopting an image classification algorithm based on a convolutional neural network, the convolutional neural network comprises 1 input layer, 4 hidden layers and 1 output layer, the output is a one-dimensional array which comprises three data, and the three data respectively represent the probability that the recognized objects are giardia, cryptosporidium and non-amphibian. Before the application of the program, a designed neural network is trained, thousands of pictures of two worms with artificial labels are used as a training data set, and the recognition precision is ensured to reach more than 95%. After training is finished, the high-magnification picture information is input into a neural network to obtain accurate prediction. And after the recognition program finishes the content recognition of the high-magnification photo, feeding back the recognition result to the control program.
The invention has the beneficial effects that: the automatic identification speed of present solitary two worm slides is about 60 minutes/piece, and is equivalent with skilled detection personnel naked eye identification speed, but automatic identification system can continuous automatic work, does not have the tired problem that needs the rest of eye, consequently detection efficiency improves greatly, and automatic identification system does not need professional's operation, greatly reduced detects the threshold, through the continuous training and the upgrading of recognition algorithm, machine identification rate of accuracy can constantly improve, thoroughly solved the big problem of human error, two worm detects and will become more succinct high-efficient.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings needed in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings without creative efforts.
FIG. 1 is a schematic diagram of the general architecture of an automatic two-insect recognition system for a microscope in accordance with the present invention;
FIG. 2 is an isometric view of a fluorescent power supply;
FIG. 3 is an isometric view of the fluorescence microscope according to FIG. 1;
FIG. 4 is an isometric view of the motion controller of FIG. 1;
FIG. 5 is a flow chart of a method for using a microscope system for automatically identifying two insects, in accordance with an embodiment of the present invention;
FIG. 6 is a table of software and development kit version information used by an automatic two-bug identification system for a microscope according to an embodiment of the invention;
FIG. 7 is a flowchart of a suspected area selection algorithm in an automatic microscope identification system according to an embodiment of the present invention;
fig. 8 is a parameter table of a convolutional neural network structure in a microscope automatic recognition two-insect system according to an embodiment of the invention.
In the figure:
1. a fluorescence microscope; 2. a CCD camera; 3. a computer; 4. a motion control card; 5. a motion controller; 6. a fluorescent power supply; 7. an object stage; 8. an eyepiece.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments that can be derived by one of ordinary skill in the art from the embodiments given herein are intended to be within the scope of the present invention.
According to an embodiment of the invention, a microscope automatic identification two-insect system is provided.
As shown in fig. 1-4, the microscope system for automatically identifying two insects comprises an imaging system, a control system and a data processing system, wherein the imaging system comprises a fluorescence microscope (1) and a CCD camera (2), the fluorescence microscope (1) is provided with a fluorescence power supply (6), the control system comprises a motion control card (4) and a motion controller (5), the data processing system is a computer (3), the computer (3) runs a control program and an identification program, the fluorescence microscope (1) is provided with the CCD camera (2), the CCD camera (2) is connected with the computer (3), the computer (3) and the motion controller (5) are both connected with the motion control card (4), the motion control card (4) is connected with the fluorescence microscope (1), and the control program is used for collecting data of the CCD camera (2), the control program is used for controlling the movement of an objective lens and an objective table (7) of the fluorescence microscope (1), and the identification program can identify the authenticity of the two insects and acquire position coordinates based on artificial intelligence image processing.
Wherein, the CCD camera (2) is connected with the computer (3) through a USB data line.
The motion control card (4) is connected with the computer (3) through a USB data line.
The objective lens and the objective table (7) of the fluorescence microscope (1) are both connected with the motion control card (4).
For the convenience of understanding the above technical solution of the present invention, the following detailed description is made on the flow of the above solution of the present invention with reference to the accompanying drawings, and specifically is as follows:
according to the embodiment of the invention, a method for automatically identifying two insect systems by a microscope is also provided.
As shown in fig. 5-8, in actual use, the microscope automatically recognizes the use of the two-worm system, comprising the steps of:
step S101, turning on a fluorescent microscope switch, a fluorescent power switch and a motion control card switch, and connecting a motion control card USB line and a CCD data USB line to a computer;
step S103, setting a com port number of the USB line of the motion control card, wherein the specific checking path is as follows: this computer-property-device manager-port;
step S105, clicking a 'connection microscope' on the software end, popping up a 'connection success' word on the interface after the connection is successful, clicking a 'confirmation' dialog box to disappear, and then setting a picture storage path;
step S107, observing through an ocular lens, moving the position of a sample through a motion controller, generally defining the upper left corner point of a circumscribed rectangle of a circle drawn by the sample as a starting point, clicking a 'set starting point' button of a software interface after moving in place, then observing through the ocular lens, moving the sample to the position of the lower right corner point of the rectangle, clicking a 'set end point' button of the software interface, switching a microscope from ocular lens observation to CCD observation after finishing setting, and then clicking a 'start scanning' button;
step S109, before scanning, software detects the multiple of the objective lens used at present, if the multiple is not 20 times, the objective lens is automatically switched to be 20 times, then serpentine scanning is started from the position of a starting point defined by a user, the CCD takes pictures at the same time, the taken pictures are stored in a path folder designated by the user, and at the moment, the user only needs to wait, and all the working software can be automatically completed;
step S111, after the 20-time objective lens is scanned, software can identify the scanned picture, circle out suspicious pictures and coordinates, generate a suspicious picture parameter list in a suspicious object list on the left upper side of the software, then the software enables the microscope to automatically switch the 40-time objective lens through a command, the objective table moves to a first suspicious position, focusing and photographing are carried out again, and the like are carried out until the photographing of the Nth coordinate is completed;
step S113, after the shooting under the 40-time objective lens is finished, the software can recognize the shot picture again, the two-insect result and the coordinate list are in the microorganism list on the left lower side of the software to generate a microorganism parameter list, and all work is finished automatically;
step S115, the user can double click certain parameter information in the two lists through the mouse to enable the microscope objective table to rapidly move to the coordinate position, and the user can verify the result through the eyepiece.
In one embodiment, the image recognition program is developed based on the python language, and the software used includes python, pytorech, scimit-spare, numpy, and opencv-python, see FIG. 6 in particular.
In an embodiment, after the 20 times objective lens finishes scanning, the software identifies the scanned picture, the background of the picture display picture collected by the fluorescence microscope is usually black, and the bug is usually green, according to the characteristic, the program uses the green channel value in the RGB three-color channel as the judgment basis, and makes the suspected region selection algorithm as shown in fig. 7, which is as follows:
in image preprocessing, in order to increase the execution speed of the program, the input picture is first subjected to size compression, and the picture is compressed to 1/4 of the original picture by default, and the ratio can be adjusted by modifying the "rate" parameter in the program. Then, since the background color of part of the picture is not black, in order to avoid the influence caused by the background color, the picture is subjected to mean filtering. Finally, the program only focuses on the green channel value, so all other color channel values are set to 0.
Scanning the picture pixel by pixel, respectively scanning the picture from the horizontal direction and the vertical direction by adopting a loop nesting mode, and adjusting the loop step length by modifying a step parameter in a program to increase the scanning speed, wherein the default is 2. The larger the number, the shorter the scanning time, but since the smaller the size of the bug, it is easy to cause omission, and therefore it needs to be selected properly.
Judging whether the green channel numerical value of the pixel point meets the condition or not, setting a threshold value [35, 50] for the green channel numerical value through experiments, if the green channel numerical value of the pixel point is in the threshold value range, storing the coordinate of the point into a bug _ cord list, and if not, carrying out no treatment on the point. The threshold range can be adjusted by modifying a 'threshold' parameter in a program, and the larger the range is, the more suspected areas are found, which brings burden to the subsequent identification work; the smaller the range, the greater the probability of missed detection.
The clustering of the coordinates of the pixel points in the list is performed by DBSCAN, the clustering of the DBSCAN is a clustering algorithm based on density, adjacent and close points in low-latitude data can be aggregated into one class, and the class number and the clustering center do not need to be specified in advance, so that the method is very suitable for determining a suspected area. The pixel coordinates stored in the bug _ code list are clustered, and each category represents a suspected area.
In one embodiment, the coordinate point determination is specifically: and clustering to obtain a plurality of categories, averaging the coordinate values of the pixel points in each category to obtain the central coordinate of the suspected area, and selecting a square box as the suspected area by taking the coordinate as the center, wherein the default size of the square box is 60 pixels by 60 pixels.
After the shooting under the 40-time objective lens is completed, the software can recognize the shot picture again, the recognition program reads the high-magnification picture, the two insects are recognized by adopting an image classification algorithm based on a convolutional neural network, the convolutional neural network comprises 1 input layer, 4 hidden layers and 1 output layer, the output is a one-dimensional array which comprises three data, and the three data respectively represent the probability that the recognized objects are giardia, cryptosporidium and non-amphibian. Before the application of the program, a designed neural network is trained, thousands of pictures of two worms with artificial labels are used as a training data set, and the recognition precision is ensured to reach more than 95%. After training is finished, the high-magnification picture information is input into a neural network to obtain accurate prediction. And after the recognition program finishes the content recognition of the high-magnification photo, feeding back the recognition result to the control program.
For the convenience of understanding the technical solutions of the present invention, the following detailed description will be made on the working principle or the operation mode of the present invention in the practical process.
In practical application, a sample slide to be identified is placed on a fluorescence microscope objective table, a motion controller controls the fluorescence microscope objective table to move in three axes through a motion control card, a diagonal line of a rectangular area of the sample slide to be identified is taken as an identification starting point and an identification end point, a control program on a computer is connected with the motion control card in a serial port communication mode, the control program obtains coordinate point data of the identification starting point and the end point through the motion control card and checks whether a fluorescence microscope objective is in low magnification, if not, the identification starting point and the end point are automatically switched to the low magnification, then the fluorescence microscope objective table moves in a mode from an X-axis starting point to an X-axis end point and from a Y-axis starting point to a Y-axis end point, the step distance is the minimum pixel distance each time, the fluorescence microscope objective table moves up and down to complete automatic focusing, and, and when the fluorescence microscope objective table moves to the scanning end point, the control program finishes moving and stops photographing.
The identification program reads the low-magnification picture, and in order to improve the identification speed, the picture is firstly subjected to size compression because the background of the picture in the real object is black in a FITC mode of a fluorescence microscope, two insects are usually displayed as green after fluorescence dyeing is excited, according to the characteristic, the picture is subjected to filtering processing to shield other mixed colors except the green, the green channel numerical value in an RGB three-color channel is taken as a judgment basis, the picture is scanned pixel by pixel from the horizontal direction and the vertical direction respectively, a green channel numerical value threshold value is set, pixel point coordinates meeting conditions are stored into a list through comparison, all pixel points in the list are subjected to density-based DBSCAN clustering, and each category represents a suspicious region. And averaging the coordinate values of the pixel points in each category of the plurality of categories obtained after clustering to obtain a central coordinate point of the suspicious region, and selecting a square frame as the suspicious region by taking the coordinate point as the center. And the recognition program recognizes the complete picture and feeds back all the coordinate points of the suspicious region to the control program.
The control program controls the microscope to switch the objective lens to high magnification through the motion control card, controls the fluorescence microscope objective table to automatically move the coordinate points of each suspicious region in sequence, controls the CCD camera to photograph and store the coordinate points of each suspicious region until all the coordinate points of all the suspicious regions are photographed.
The recognition program reads the high-magnification picture, and the two worms are recognized by adopting an image classification algorithm based on a convolutional neural network, wherein the convolutional neural network comprises 1 input layer, 4 hidden layers and 1 output layer, and the output is a one-dimensional array which comprises three data which respectively represent the probability that the recognized objects are giardia, cryptosporidium and non-two worms. Before the application of the program, a designed neural network is trained, thousands of pictures of two worms with artificial labels are used as a training data set, and the recognition precision is ensured to reach more than 95%. After training is finished, the high-magnification picture information is input into a neural network to obtain accurate prediction. And after the recognition program finishes the content recognition of the high-magnification photo, feeding back the recognition result to the control program.
The control program arranges the data to give an automatic identification result report, simultaneously the result supports manual recheck, besides supporting manual checking of the original record photo of the result, the control program controls the fluorescence microscope objective table to automatically move to a result coordinate point through the motion control card, automatically switches to high magnification through controlling the fluorescence microscope objective lens, and realizes the purpose of manual naked eye recheck through switching a DAPI mode and a DIC mode.
The automatic identification control software is compiled through Visual C + +2015, is controlled through communication between a USB (universal serial bus) line and a hardware microscope, and is connected with the CCD camera through the USB line;
the control motion principle is as follows: the software is connected with the microscope control platform by setting a 'com' port. The position of the sample in the visual field is manually adjusted through an ocular lens, then the software end clicks a set starting point and an equipment end point, and the software records the coordinates of the starting point and the end point set by a user. Clicking 'start scanning' again, the software will self-check the objective lens multiple, automatically switch 20 times objective lens for use (if it is 20 times, then directly scan), at this time, the software sends a command to the platform, starts scanning from the starting position, meanwhile, the software calls the CCD to take a picture, the taken picture is stored in a file folder with a specified path, after all the pictures are taken, the image recognition module carries out image recognition, the image recognition module finds out suspected positions in the picture and records the coordinates in a list form, after scanning is finished, the software sends a command to switch the 40-time objective lens, starts sending the command to move to each point in the suspected list for re-photographing, photographs are taken and stored in a folder with a specified path, after all photographs are taken, the image recognition module will perform image recognition and will record the coordinates of the two worms (giardia or cryptosporidium) in a list. An operator can click the list coordinate moving platform to a corresponding position through a mouse to perform manual verification;
the principle of image recognition: the image recognition module is developed by adopting python3.6 language, and simultaneously, a pitorch, scimit-spare, numpy and opencv-python development toolkit is used. The background of the picture taken by the microscope is usually black, while the bug is usually green. According to the characteristics, the program takes the value of a green channel in an RGB three-color channel as a judgment basis to formulate a suspected area selection algorithm. For the confirmation of the coordinate points of the suspected area, DBSCAN clustering is carried out on the coordinates of the pixel points in the list, a plurality of categories are obtained after clustering, the coordinate values of the pixel points in each category are averaged to obtain the central coordinate of the suspected area, a square frame is selected as the suspected area by taking the coordinate as the center, and the default size of the square frame is 60 x 60 (unit: pixel);
for the pictures shot under the 40-time objective lens, the image recognition module classifies the bugs through an image classification algorithm based on a convolutional neural network;
how to determine giardia or cryptozoa: the program classifies the bugs using an image classification algorithm based on a convolutional neural network. The convolutional neural network comprises 1 input layer, 4 hidden layers and 1 output layer, specific parameters of a network structure are shown in fig. 8, the convolutional neural network inputs a picture with the size of 60 × 60 pixels, outputs a group of one-dimensional arrays and comprises three data which respectively represent the probability of belonging to three categories (other, giardia and hidden insects). The designed network is first trained. 1598 small insect pictures with artificial labels are collected in the early stage to serve as a training set, a cross entropy loss function is adopted to train the network, the optimization algorithm is Adam, the learning rate is 0.001, the batch size is 5, and 150 cycles of iteration are performed in total. And selecting 100 small insect pictures with artificial labels as a test set to verify the training effect of the network. After 150 iterations, the classification accuracy of the network can reach 95%, and the trained network parameters are stored.
Then, according to the coordinate center of the suspected area, adjusting a microscope lens, and taking an image enlarged by 400 times by taking the suspected area as the center. And selecting an area which possibly contains the bugs in the center from the pictures magnified by 400 times as an object for image classification.
And finally, compressing the selected picture into 60 × 60 pixels, and inputting the pixels into a trained network to obtain three predicted values. The maximum value among the three predicted values represents the category to which the picture belongs, for example, the output value is [0.1,0.8,0.1], the probability of belonging to giardia is 0.8, and the picture is the giardia.
The invention provides an automatic two-insect recognition system of a microscope, which utilizes the strong calculation and control capability of a computer, adopts automatic motion control to replace manual operation, and adopts artificial intelligent image processing to replace artificial visual recognition. The problem that a tester is required to have high professional level and cannot manually observe a microscope for a long time is thoroughly solved, the two-worm recognition threshold is reduced, recognition errors are reduced, the recognition accuracy is improved, the recognition efficiency is improved through continuous automatic recognition, and the overall detection working efficiency is obviously improved.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like that fall within the spirit and principle of the present invention are intended to be included therein.

Claims (8)

1. The microscope automatic amphibian recognition system is characterized by comprising an imaging system, a control system and a data processing system, wherein the imaging system comprises a fluorescence microscope (1) and a CCD (charge coupled device) camera (2), the fluorescence microscope (1) is provided with a fluorescence power supply (6), the control system comprises a motion control card (4) and a motion controller (5), the data processing system is a computer (3), the computer (3) runs a control program and an identification program, the fluorescence microscope (1) is provided with the CCD camera (2), the CCD camera (2) is connected with the computer (3), the computer (3) and the motion controller (5) are both connected with the motion control card (4), the motion control card (4) is connected with the fluorescence microscope (1), the control program is used for collecting data of the CCD camera (2), the control program is used for controlling the movement of an objective lens and an objective table (7) of the fluorescence microscope (1), and the identification program can identify the authenticity of the two insects and acquire position coordinates based on artificial intelligence image processing.
2. The microscope automatic two-worm recognition system according to claim 1, characterized in that the CCD camera (2) is connected with the computer (3) through a USB data line.
3. The microscope system for automatically identifying two insects according to claim 2, wherein the motion control card (4) is connected to the computer (3) through a USB data line.
4. The system for microscope automatic identification of two worms according to claim 3 characterised by the fact that the objective and the stage (7) of the fluorescence microscope (1) are both connected to a motion control card (4).
5. A method for microscope automatic identification of two-worm system, characterized in that for use of the microscope automatic identification of two-worm system of claim 4, comprising the steps of:
opening a fluorescence microscope switch, a fluorescence power switch and a motion control card switch, and connecting a motion control card USB line and a CCD data USB line to a computer;
setting a port number com of a USB line of the motion control card, wherein a specific checking path is as follows: this computer-property-device manager-port;
the software end clicks the 'connection microscope', when the connection is successful, the interface pops up a 'connection success' word, a 'confirmation' dialog box is clicked to disappear, and then a picture storage path is set;
observing through an ocular lens, moving the position of a sample through a motion controller, generally defining the upper left corner point of a circumscribed rectangle of a circle drawn by the sample as a starting point, clicking a 'set starting point' button of a software interface after moving in place, then observing through the ocular lens, moving the sample to the lower right corner position of the rectangle, clicking a 'set end point' button of the software interface, switching a microscope from ocular lens observation to CCD observation after finishing setting, and then clicking a 'start scanning' button;
before scanning is started, software can detect the multiple of the objective lens used at present, if the multiple is not 20 times of the objective lens, the objective lens is automatically switched into the 20 times of the objective lens, snake-shaped scanning is started from a starting point position defined by a user, the CCD shoots at the same time, the shot pictures are stored in a path folder designated by the user, and at the moment, the user only needs to wait, and all working software can be automatically completed;
after scanning under the 20-time objective lens is finished, the software can identify the scanned picture, circle out suspicious pictures and coordinates, generate a suspicious picture parameter list in a suspicious object list on the left upper side of the software, then the software enables the microscope to automatically switch the 40-time objective lens through a command, the objective table moves to a first suspicious position, focusing and photographing are carried out again, and the like is carried out until photographing of the Nth coordinate is finished;
after the 40-time objective lens is photographed, the software can recognize the photographed picture again, the two-insect result and the coordinate list are in the microorganism list on the left lower side of the software to generate a microorganism parameter list, and all work is automatically completed;
the user can double click certain parameter information in the two lists through the mouse to enable the microscope objective table to quickly move to the coordinate position, and the user can verify the result through the eyepiece.
6. The method for microscope to automatically identify two-insect system according to claim 5, wherein the image recognition program is developed based on python language, and the software used includes python, pyroch, scimit-spare, numpy and opencv-python.
7. The method for automatically identifying two worms by a microscope as claimed in claim 5, wherein the background of the picture display picture collected by the fluorescence microscope is usually black, and the bug is usually green, and according to the characteristic, the program takes the value of the green channel in the RGB three-color channel as the judgment basis, and the suspected area selection algorithm is formulated as follows:
in image preprocessing, in order to increase the execution speed of the program, the input picture is first subjected to size compression, and the picture is compressed to 1/4 of the original picture by default, and the ratio can be adjusted by modifying the "rate" parameter in the program. Then, since the background color of part of the picture is not black, in order to avoid the influence caused by the background color, the picture is subjected to mean filtering. Finally, the program only focuses on the green channel value, so all other color channel values are set to 0.
Scanning the picture pixel by pixel, respectively scanning the picture from the horizontal direction and the vertical direction by adopting a loop nesting mode, and adjusting the loop step length by modifying a step parameter in a program to increase the scanning speed, wherein the default is 2. The larger the number, the shorter the scanning time, but since the smaller the size of the bug, it is easy to cause omission, and therefore it needs to be selected properly.
Judging whether the green channel numerical value of the pixel point meets the condition or not, setting a threshold value [35, 50] for the green channel numerical value through experiments, if the green channel numerical value of the pixel point is in the threshold value range, storing the coordinate of the point into a bug _ cord list, and if not, carrying out no treatment on the point. The threshold range can be adjusted by modifying a 'threshold' parameter in a program, and the larger the range is, the more suspected areas are found, which brings burden to the subsequent identification work; the smaller the range, the greater the probability of missed detection.
The clustering of the coordinates of the pixel points in the list is performed by DBSCAN, the clustering of the DBSCAN is a clustering algorithm based on density, adjacent and close points in low-latitude data can be aggregated into one class, and the class number and the clustering center do not need to be specified in advance, so that the method is very suitable for determining a suspected area. The pixel coordinates stored in the bug _ code list are clustered, and each category represents a suspected area.
8. The method for microscope to automatically identify two-insect system according to claim 5, wherein the coordinate point determination is specifically: and clustering to obtain a plurality of categories, averaging the coordinate values of the pixel points in each category to obtain the central coordinate of the suspected area, and selecting a square box as the suspected area by taking the coordinate as the center, wherein the default size of the square box is 60 pixels by 60 pixels.
CN202010165944.XA 2020-03-11 2020-03-11 System and method for automatically identifying two insects by microscope Pending CN111240004A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010165944.XA CN111240004A (en) 2020-03-11 2020-03-11 System and method for automatically identifying two insects by microscope

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010165944.XA CN111240004A (en) 2020-03-11 2020-03-11 System and method for automatically identifying two insects by microscope

Publications (1)

Publication Number Publication Date
CN111240004A true CN111240004A (en) 2020-06-05

Family

ID=70866333

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010165944.XA Pending CN111240004A (en) 2020-03-11 2020-03-11 System and method for automatically identifying two insects by microscope

Country Status (1)

Country Link
CN (1) CN111240004A (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112098383A (en) * 2020-09-21 2020-12-18 郑州中普医疗器械有限公司 Method for identifying microorganism bacteria and electronic device
CN112150446A (en) * 2020-09-27 2020-12-29 郑州中普医疗器械有限公司 Microscope examination and film reading scanning method, computer equipment and system
CN114859539A (en) * 2022-05-24 2022-08-05 宾盛科技(武汉)有限公司 Fluorescent intelligent identification sensor and multi-channel fluorescent scanning system
CN114973223A (en) * 2022-03-03 2022-08-30 上海精密计量测试研究所 Component quantity identification and mixed batch detection system based on image identification

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101799438A (en) * 2010-03-10 2010-08-11 广州市番禺区胆囊病研究所 Method for detecting clonorchis sinensis eggs in gallstone
CN106841072A (en) * 2017-03-30 2017-06-13 莱森光学(深圳)有限公司 A kind of method differentiated for algae with algae proliferation non-destructive monitoring situation
CN110672608A (en) * 2019-10-15 2020-01-10 南京泰立瑞信息科技有限公司 Dynamic planning method and system for full-slice scanning path

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101799438A (en) * 2010-03-10 2010-08-11 广州市番禺区胆囊病研究所 Method for detecting clonorchis sinensis eggs in gallstone
CN106841072A (en) * 2017-03-30 2017-06-13 莱森光学(深圳)有限公司 A kind of method differentiated for algae with algae proliferation non-destructive monitoring situation
CN110672608A (en) * 2019-10-15 2020-01-10 南京泰立瑞信息科技有限公司 Dynamic planning method and system for full-slice scanning path

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112098383A (en) * 2020-09-21 2020-12-18 郑州中普医疗器械有限公司 Method for identifying microorganism bacteria and electronic device
CN112150446A (en) * 2020-09-27 2020-12-29 郑州中普医疗器械有限公司 Microscope examination and film reading scanning method, computer equipment and system
CN114973223A (en) * 2022-03-03 2022-08-30 上海精密计量测试研究所 Component quantity identification and mixed batch detection system based on image identification
CN114859539A (en) * 2022-05-24 2022-08-05 宾盛科技(武汉)有限公司 Fluorescent intelligent identification sensor and multi-channel fluorescent scanning system
CN114859539B (en) * 2022-05-24 2024-07-02 宾盛科技(武汉)有限公司 Fluorescent intelligent identification sensor and multichannel fluorescent scanning system

Similar Documents

Publication Publication Date Title
CN111240004A (en) System and method for automatically identifying two insects by microscope
US8116543B2 (en) System for and method of intelligently directed segmentation analysis for automated microscope systems
CN109559275B (en) Microscopic image stitching method of urine analyzer
CN110793946A (en) Fungus sample microscopic imaging and intelligent identification system and method
JPH0484884A (en) Image recognizing system and image recognizing control system
US12032150B2 (en) Method and system for identifying objects in a blood sample
CN113139485B (en) Bone marrow cell classification and identification method, device and system based on deep learning
CN113631982A (en) Automated micro-scanning system and method
CN111079831A (en) Intelligent optical detection sample characteristic and flaw automatic marking method and device
US20200074628A1 (en) Image processing apparatus, imaging system, image processing method and computer readable recoding medium
CN209803001U (en) Chromosome scanning imaging system
CN111797706A (en) Image-based parasite egg shape recognition system and method
WO2021134508A1 (en) Cell image analysis apparatus and sample analysis method
CN115079393B (en) Chromosome karyotype analysis microscopic shooting device and method based on 10-fold objective lens
WO2023031622A1 (en) System and method for identifying and counting biological species
CN114829944A (en) Sample analysis system and method
CN117455791A (en) Lightweight network-based rapid focusing method for scanning pathological images
CN110739051A (en) Method for establishing eosinophilic granulocyte proportion model by using nasal polyp pathological picture
CN100483432C (en) Genital-infection inteeligent identification system and method
CN118279702B (en) Tear overflow detection model construction method, tear overflow detection method, device and equipment
JPH03123860A (en) Chromosome inspecting device
CN110298905A (en) It is a kind of to be sliced the method and apparatus for generating digital slices based on biological sample
CN116230214B (en) HCC and VETC auxiliary diagnosis device and equipment
RU2793054C1 (en) Method for digitizing a biological preparation of bone marrow using neural networks
US20230291992A1 (en) Method and system for event-based imaging

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
TA01 Transfer of patent application right
TA01 Transfer of patent application right

Effective date of registration: 20200904

Address after: 100076 No. 10, golden street, Xihongmen Town, Beijing, Daxing District

Applicant after: HKY TECHNOLOGY Co.,Ltd.

Address before: 5 / F (05) 501-7, building 75, yard 6, Jiuwen Road, Chaoyang District, Beijing 100020

Applicant before: BEIJING AILUKE TECHNOLOGY TESTING Co.,Ltd.

RJ01 Rejection of invention patent application after publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20200605