CN115131293A - Traditional Chinese medicine origin identification method based on LIBS spectrum and image fusion - Google Patents

Abstract

The invention provides a traditional Chinese medicine decoction piece producing area identification method based on LIBS spectrum and image feature fusion, which comprises the following steps: feature extraction is carried out on the LIBS spectrum based on an RC value of a PLS-DA model, feature extraction is carried out on the LIBS spectrum and an LIBS image through a convolutional neural network, feature layer fusion and decision layer fusion are carried out on the extracted spectral features and the image features, the fused features are placed into a classifier to be classified, and traditional Chinese medicine origin place identification results based on map fusion are obtained. The invention combines LIBS and a multivariate information fusion technology, and proves that the method has great potential in the field of traditional Chinese medicine identification through classification of the producing areas of the white paeony root and the dahurian angelica root.

Description

Traditional Chinese medicine production area identification method based on LIBS spectrum and image fusion

Technical Field

The invention relates to the field of Chinese medicinal material origin place identification, in particular to a Chinese medicinal material origin place classification method based on LIBS spectrum and image fusion.

Background

Modern researches show that the traditional Chinese medicinal materials have the effects of resisting inflammation, relieving pain, protecting liver, resisting oxidation and the like, are widely applied to clinic and have wide development prospects. The chemical components of the traditional Chinese medicinal materials are complex and various, and the main chemical substances of the traditional Chinese medicinal materials have differences in the aspects of exerting the drug effect and treating diseases. The quality of herbs is closely related to the origin. The content of each component in the traditional Chinese medicine directly influences the exertion of the pharmacological action, and the content of each component is influenced by the producing area.

The traditional method for identifying the origin of the traditional Chinese medicinal material is more, but due to the influences of factors such as complex sample preparation process, large sample loss, poor identification efficiency, low automation degree and the like, people have not realized rapid detection in the practice of identifying the origin of the traditional Chinese medicinal material. Laser-Induced Breakdown Spectroscopy (LIBS) technology focuses ultrashort pulse Laser on the surface of a sample to form plasma, and then analyzes the plasma emission spectrum to determine the material composition and content of the sample. LIBS has the advantages of simple sample preparation, small sample loss, high detection efficiency, capability of realizing on-line detection and the like, and is widely applied to the fields of clinical medicine, archaeology, industry and the like at present.

The literature "Distinguish Fritillaria cirrhosa and non-Fritillaria cirrhosa using-induced branched spectroscopy" by Wei et al suggests that LIBS is combined with learning vector quantization to Distinguish Bulbus Fritillariae Cirrhosae from non-Bulbus Fritillariae Cirrhosae, and the feasibility of LIBS in classification of Chinese medicinal materials is proved through experiments.

Liu et al, Geogaphic evaluation of mental morphology by LIBS coupled with multi-variant analysis, uses LIBS in combination with principal component analysis and least squares support vector machine to classify and identify mints, and classifies herbs by machine learning methods.

The method illustrates the application prospect and great potential of the LIBS in the field of Chinese medicinal material classification, but the research only focuses on the LIBS spectrum, does not relate to an LIBS image, and is high in identification difficulty and low in precision.

Disclosure of Invention

The invention aims to provide a method based on LIBS spectrum and image fusion, which realizes the rapid identification of the producing area of traditional Chinese medicine decoction pieces and improves the accuracy of the identifying of the producing area.

In order to achieve the purpose, the invention provides the following technical scheme:

a traditional Chinese medicine decoction piece production place identification method based on LIBS spectrum and image fusion comprises the following steps:

step 1, performing feature screening on LIBS spectral data by using an RC value based on a partial least square discriminant analysis method PLS-DA (partial least squares) to obtain screened spectral features;

step 2, performing feature extraction on the LIBS spectral data and the LIBS image data by using a convolutional neural network to obtain spectral features and image features after feature extraction;

step 3, fusing the spectral features and the image features on a feature level and a decision level;

step 4, placing the fused features into different classifiers for classification;

and 5, obtaining a classification result based on the fusion characteristics.

Further, the step 1 of extracting LIBS spectral features by using the PLS-DA-based RC values specifically includes: inputting spectral data with the size of (m multiplied by n), and marking as X; manually adding a response value Y to the spectral data, wherein Y is an (m multiplied by 1) matrix; establishing PLS-DA classification model to obtain corresponding different wavelengths (lambda) ₁ ，λ ₂ ，λ ₃ ……λ _m ) RC value (w) of ₁ .w ₂ ,w ₃ ……w _m ) And screening out the characteristic wavelength of the LIBS spectrum by comparing the RC values.

Further, selecting a residual network (ResNet18, ResNet50) in the convolutional neural network as a network for extracting the spectrum and the image features in the step 2; after the input data is subjected to convolution, batch normalization, activation and pooling, the features of the last full-connection layer are selected as the extracted features.

Further, the LIBS image data in the step 2 is an overall image of the herbal pieces prepared for decoction after laser dotting, the image is obtained by shooting through an automatic focusing program by a camera in the LIBS system, and the automatic focusing program adopts a hill climbing method as a search function.

Further, the specific steps of obtaining the image data are as follows: 1. obtaining the image definition value of the current position and recording the value as Ten ₀ (ii) a 2. The X-Y-Z sample stage for placing the sample moves by a given step length s along the designated Z direction ₀ Obtaining the current image definition Ten ₁ If Ten ₁ >Ten ₀ Then move the same step length s in the original direction ₀ Otherwise, multiplying the step length by the convergence coefficient p to obtain the step length s after convergence ₁ ＝p×s ₀ (ii) a 3. Comparing the converged step lengths s ₁ Given the magnitude of the threshold t, if s ₁ <t is the focusing is completed, if s ₁ >t is the step length s after the sample stage moves reversely and converges ₁ And repeating the steps 2-3 until the focusing is completed.

Further, the image definition Ten value is calculated by using a Tenengrad function, and the Tenengrad function extracts gradient values in the horizontal direction and the vertical direction by using a Sobel operator; the specific process is as follows: 1. let sober convolution kernel be G _x ,G _y Then the gradient of image I at point (x, y) is

Define the Ten value of the image as

n is the total number of pixels in the image.

Further, the extracted features are respectively normalized in the step 3 to obtain the size (m) ₁ X 1) spectral data and size (m) ₂ X 1), and the two are joined to obtain image data having a size of ((m) ₁ +m ₂ ) X 1), and the spliced features are used as final fusion features and are respectively input into different classifiers for classification.

Further, the decision layer in step 3 is fused by inputting different modal information into trained classifiers respectively to output scores or decisions for fusion

Further, the decision layer fusion is specifically as follows: the spectrum and the image characteristics are put into respective classifiers to be trained, and the probability of producing a place of each type output by the spectrum classifier is obtained as (P) ₁ ,P ₂ ,P ₃ ) The probability of producing a place of each class output by the image classifier is (K) ₁ ,K ₂ ,K ₃ ) Calculating the arithmetic mean value thereof

And taking the category with the highest arithmetic mean value as the final decision of the origin of the traditional Chinese medicine decoction pieces.

Has the advantages that: according to the invention, LIBS spectrum information and LIBS image information are subjected to data fusion for the first time, a model is established based on the fusion data and is used for identifying the producing area of the traditional Chinese medicine decoction pieces, and the information with different sources is synthesized, so that the multi-angle analysis of the same sample is realized, the LIBS classification precision is further improved, and the effective improvement of the identification precision rate of the traditional Chinese medicine producing area is realized.

Drawings

In order to better explain the technical scheme, the specific implementation steps and the experimental results, the figures required in the implementation process are briefly introduced below.

FIG. 1 is a detailed flow chart of LIBS map fusion-based identification of the origin of Chinese herbal pieces.

Fig. 2 is a graph illustrating an example of the average spectrum of a sample of white peony in a data set.

FIG. 3 is a graph illustrating an example of an average spectrum of a sample of Angelica dahurica in a data set.

Fig. 4 is a LIBS image of herbal pieces-white peony samples in a data set.

Fig. 5 is a LIBS image of a herbal piece angelica dahurica sample in a data set.

Fig. 6 is a schematic diagram of the fusion process of the spectral features extracted by ResNet18 and the image features extracted by ResNet50 at the feature level.

Fig. 7 is a schematic diagram of a fusion process of spectral information and image information decision layers.

Detailed Description

Embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

The invention aims to provide a traditional Chinese medicine decoction piece producing area identification method based on LIBS spectrum and image fusion.

In order to make the aforementioned objects, features and advantages of the present invention more comprehensible, the present invention is further described in detail with reference to the accompanying drawings and examples.

As shown in the figure, the traditional Chinese medicine producing area identification method based on LIBS spectrum and image fusion comprises the following specific steps:

step 1, LIBS spectral data and image data are obtained, an LIBS instrument detects the surface of a traditional Chinese medicine sample according to a preset 4 x 4 dot matrix, 16 positions are detected, each point is hit nine times, the surface of the sample is cleaned in the first four times, and the spectrum obtained in the last five times of hits is used for specific research. The experiment is carried out under the normal pressure condition, parameters of the LIBS instrument need to be adjusted before the LIBS instrument is used, wherein the laser pulse energy is 100mJ, the pulse frequency is 1Hz, the acquisition delay time is 2 mus, and the integration time is 10 mus. The collection mode can further collect more sample surface information and reduce the influence of other factors. The acquired LIBS spectral data need to be preprocessed, the preprocessing method comprises area normalization and spectral averaging, the area normalization is to divide the intensity of a spectrum by the integral area of the spectrum, the spectral averaging is to take the spectral averaging of the last five times as the spectrum of one point location, and then the spectral averaging of 16 point locations is taken as the final spectrum of one sample. Spectral preprocessing can reduce signal noise and improve model performance. The obtained LIBS image is a full image of a sample subjected to laser ablation, and the image format is a gray scale image. The image after ablation contains more information than the image before ablation, and the volume, area, depth and the like of the ablation pits have certain influence on the result of the identification of the production place. The Chinese medicinal materials used for the experiment are radix paeoniae alba and radix angelicae dahuricae, and the two samples come from three different production places, which are respectively marked as 0, 1 and 2.

And 2, performing feature screening on the LIBS spectrum by using a regression coefficient value RC based on PLS-DA, wherein the RC value is used for measuring the relation between a variable and a response value, and the variable with the larger RC value contributes to the model with the larger RC value. Firstly, a PLS-DA classification model is established for spectral data, parameters are adjusted to obtain an optimal model, and different wavelengths (lambda) are obtained ₁ ，λ ₂ ，λ ₃ ……λ _m ) Corresponding RC value (w) ₁ .w ₂ ,w ₃ ……w _m ). Comparing the RC values corresponding to different wavelengths, selecting 111 features of 37 characteristic wavelengths and the peak intensities of two adjacent wavelengths in front and at back from the RC values, and selecting 90 features of 30 characteristic wavelengths and the peak intensities of two adjacent wavelengths in front and at back from the RC values, wherein the spectral characteristic wavelengths are shown in tables 1 and 2:

table 1: and (4) selecting the spectral characteristic wavelength of the white paeony root according to PLS-DA.

Note: i represents the atomic line of the element; II represents the primary ion spectrum of the element; -representing a characteristic line without corresponding elements.

Table 2: and (4) selecting the characteristic wavelength of the spectrum of the angelica dahurica according to PLS-DA.

Note: i represents the atomic line of the element; II represents the primary ion spectrum of the element; -representing a characteristic line without corresponding elements.

And 3, extracting spectral features by using ResNet18, wherein the initial learning rate of the model is 0.0001, the optimizer is an SGDM optimizer, the iteration times are 500, and the size of the small batch is 50. And inputting the spectral data into a ResNet18 network, extracting high-dimensional features by using convolution operation, and taking 256 features output by the last layer of full-connection layer as spectral fusion features.

And 4, extracting image features by using ResNet50, wherein the initial learning rate of the model is 0.0001, the optimizer is an Adam optimizer, the iteration times are 300 times, and the size of the small batch is 8. And extracting high-dimensional information of the image through ResNet50, and taking 512 features output by the last layer of fully-connected layer as image fusion features.

And 5, fusing the spectral features extracted by the PLS-DA and ResNet18 and the image features extracted by ResNet50 on a feature layer level, firstly normalizing the two types of features respectively to obtain white peony root spectral data with the sizes of (111 × 1) and (256 × 1), radix angelicae dahuricae spectral data with the sizes of (90 × 1) and (256 × 1), and white peony root and radix angelicae dahuricae image data with the sizes of (512 × 1), and respectively splicing the spectra and the image features of the two types of Chinese medicinal materials to obtain white peony root fusion features with the sizes of (623 × 1) and (768 × 1) and radix angelicae dahuricae fusion features with the sizes of (602 × 1) and (768 × 1). And inputting the fusion features into different classifiers to obtain a result based on feature-layer fusion modeling, wherein the classifiers used in the method comprise Support Vector Machines (SVM), PLS-DA and ResNet 18.

And 6, respectively putting the extracted spectral features and the extracted image features into different classifiers for modeling, wherein the classifiers for modeling the spectral features comprise PLS-DA, SVM and ResNet18, and the classifier for modeling the image features is ResNet 50. The decision layer fusion of white peony samples PLS-DA and ResNet50 is here illustrated as an example: the probabilities of obtaining three different production places (0, 1, 2) output by the spectrum classifier PLS-DA are respectively (P) ₁ ,P ₂ ,P ₃ ) Three different types of output from the image classifier ResNet50The probabilities of producing the areas (0, 1, 2) are respectively (K) ₁ ,K ₂ ,K ₃ ) Calculating the arithmetic mean value of the probability of different producing areas (0, 1, 2) of the three types of the spectrum classifier and the image classifier

And taking the class with the highest arithmetic mean value as a final result of decision layer fusion PLS-DA-ResNet50 radix paeoniae alba decoction piece production place attribution. Such as: if it is

And is

The final classification result is 0, if

The final result is randomly chosen between category 0 and category 1.

And 7, obtaining the recognition results of the radix paeoniae alba and the radix angelicae dahuricae production places based on feature fusion, wherein compared with a single feature, the model established based on the fusion feature has higher recognition accuracy. Specific results are shown in tables 3 and 4:

table 3: and identifying the production area of the white paeony root.

Note: IMG: image feature RC of the complete sample image after ablation: spectral characteristics ResNet18 screened according to the PLS-DA model RC value: spectral signature selected according to the ResNet18 model ResNe 50: and (4) image features screened according to ResNet 50.

Table 4: and (5) identifying the origin and the place of production of the angelica dahurica.

Note: IMG: image characteristics RC of the complete sample image after ablation: spectral characteristics ResNet18 screened according to the PLS-DA model RC value: spectral signature selected according to the ResNet18 model ResNe 50: and (4) image features screened according to ResNet 50.

Claims (9)

1. A traditional Chinese medicine decoction piece production place identification method based on LIBS spectrum and image fusion is characterized by comprising the following steps:

step 1, carrying out feature screening on LIBS spectral data by using an RC value based on a partial least square discriminant analysis method PLS-DA (partial least squares-DA) to obtain screened spectral features;

step 2, performing feature extraction on the LIBS spectral data and the LIBS image data by using a convolutional neural network to obtain spectral features and image features after feature extraction;

step 3, fusing the spectral features and the image features on a feature level and a decision level;

step 4, placing the fused features into different classifiers for classification;

and 5, obtaining a classification result based on the fusion characteristics.

2. The method for identifying the origin of decoction pieces of Chinese herbal medicine based on LIBS spectrum and image fusion as claimed in claim 1, wherein:

in the step 1, the extraction of LIBS spectral features by using the RC values based on PLS-DA specifically comprises the following steps: inputting spectral data with the size of (m multiplied by n), and recording the spectral data as X; manually adding a response value Y to the spectral data, wherein Y is an (m multiplied by 1) matrix; establishing PLS-DA classification model to obtain corresponding different wavelengths (lambda) ₁ ，λ ₂ ，λ ₃ ……λ _m ) RC value (w) of ₁ .w ₂ ,w ₃ ……w _m ) Screening LIBS by comparing RC valueCharacteristic wavelength of the spectrum.

3. The method for identifying the origin of decoction pieces of Chinese herbal medicine based on LIBS spectrum and image fusion as claimed in claim 1, wherein:

selecting residual networks (ResNet18 and ResNet50) in the convolutional neural network as networks for extracting the spectrum and the image characteristics in the step 2; after the input data is subjected to convolution, batch normalization, activation and pooling, the features of the last full-connection layer are selected as the extracted features.

4. The method for identifying origin of herbal pieces prepared for decoction based on LIBS spectroscopy and image fusion as claimed in claim 1, wherein:

the LIBS image data in the step 2 is an integral image of the traditional Chinese medicine decoction pieces after laser dotting, the image is obtained by shooting through an automatic focusing program by a camera in the LIBS system, and the automatic focusing program adopts a hill climbing method as a search function.

5. The method for identifying the origin of decoction pieces of Chinese herbal medicine based on LIBS spectrum and image fusion as claimed in claim 1, wherein:

the specific steps for obtaining the image data are as follows: 1. obtaining the image definition value of the current position and recording the value as Ten ₀ (ii) a 2. The X-Y-Z sample stage for placing the sample moves by a given step length s along the designated Z direction ₀ Obtaining the definition Ten of the current image ₁ If Ten is Ten ₁ ＞Ten ₀ Then move the same step length s in the original direction ₀ Otherwise, multiplying the step length by the convergence coefficient p to obtain the step length s after convergence ₁ ＝p×s ₀ (ii) a 3. Comparing the converged step lengths s ₁ Given the magnitude of the threshold t, if s ₁ If t, the focusing is completed, if s ₁ If t is greater than t, the step length s after the sample stage moves reversely and converges ₁ And repeating the step 2-3 until focusing is completed.

6. The method for identifying the origin of decoction pieces of Chinese herbal medicine based on LIBS spectrum and image fusion as claimed in claim 5, wherein:

the image definition Ten value is calculated by using a Tenengrad function, and the Tenengrad function extracts gradient values in the horizontal direction and the vertical direction by using a Sobel operator; the specific process is as follows: 1. let sober convolution kernel be G _x ，G _y Then the gradient of image I at point (x, y) is

Define the Ten value of the image as

n is the total number of pixels in the image.

7. The method for identifying the origin of decoction pieces of Chinese herbal medicine based on LIBS spectrum and image fusion as claimed in claim 1, wherein:

respectively normalizing the extracted features in the step 3 to obtain the size (m) ₁ X 1) spectral data and size (m) ₂ X 1), and the two are joined to obtain image data having a size of ((m) ₁ +m ₂ ) X 1), and the spliced features are used as final fusion features and are respectively input into different classifiers for classification.

8. The method for identifying the origin of decoction pieces of Chinese herbal medicine based on LIBS spectrum and image fusion as claimed in claim 1, wherein:

and the decision layer fusion in the step 3 is to input different modal information into the trained classifier respectively so as to output scores or decisions for fusion.

9. The method for identifying the origin of decoction pieces of Chinese herbal medicine based on LIBS spectrum and image fusion as claimed in claim 1, wherein:

the decision layer fusion is specifically as follows: the spectrum and the image characteristics are put into respective classifiers to be trained, and the probability of producing a place of each type output by the spectrum classifier is obtained as (P) ₁ ，P ₂ ，P ₃ ) The probability of producing a place of each class output by the image classifier is (K) ₁ ，K ₂ ，K ₃ ) Calculating the arithmetic mean value thereof

And taking the category with the highest arithmetic mean value as the final decision of the origin of the traditional Chinese medicine decoction pieces.

Images