CN114266337A - Intelligent tobacco leaf grading model based on residual error network and grading method using model - Google Patents

Abstract

The invention discloses an intelligent tobacco leaf grading model based on a residual error network and a grading method using the intelligent tobacco leaf grading model. And intelligently grading the tobacco leaves by building a custom training model based on a convolution residual error neural network. And an image enhancement function is added into the preprocessing module, so that the polymorphism of the tobacco leaf sample is expanded, and the compatibility of the model is improved. Regularization is added into a convolutional neural network module to solve the phenomenon of overfitting in the network. The hole convolution is added in the user-defined network model, and under the condition that the number of network parameters is not increased by the hole convolution, the receptive field area is increased, and the accuracy of the tobacco leaf grading model is effectively improved.

Description

Intelligent tobacco leaf grading model based on residual error network and grading method using model

Technical Field

The invention belongs to the technical field of tobacco leaf grading, and particularly relates to a tobacco leaf intelligent grading model based on a residual error network and a grading method using the model.

Background

Tobacco leaves are important raw materials in the production of tobacco products, the tobacco leaves with different qualities are distinguished, and the cigarette industry can blend cigarettes with various styles according to the quality characteristics of various levels and keep the quality of the cigarette products stable. The tobacco leaves with different qualities have different use values and also have different economic values. The quality of various tobacco leaves can be fully embodied only through accurate grading. The currently adopted tobacco leaf grading method is manual grading, mainly depends on the experience and sensory judgment of tobacco leaf grading workers, is time-consuming and labor-consuming and has great subjectivity; the grading accuracy rate depends on the experience and environment of a grader, so that intelligent grading of tobacco leaves is imperative.

In recent years, research on intelligent grading of tobacco leaves mainly focuses on grading methods based on image features. And extracting image characteristics such as color, texture, geometry and the like related to the artificial classification factors by using the tobacco leaf image, and performing classification identification on the tobacco leaves by adopting a certain classification method. However, for the tobacco leaves of adjacent grades, the image cross characteristics are obvious, the grade standards are difficult to unify and standardize, the problems of low efficiency and unstable grading of tobacco leaf grading identification are caused, and the large-scale production of tobacco leaf grading is not facilitated.

Disclosure of Invention

The invention aims to provide an intelligent tobacco leaf grading model based on a residual error network, and a grading method using the model, and aims to provide a stable grading model and an efficient grading method, which can effectively predict tobacco leaves in a grading manner in real time and improve the identification efficiency of tobacco leaf grading.

A tobacco leaf intelligent grading model based on a residual error network is characterized in that a tobacco leaf image is coded, then a tobacco leaf grade mapping digital code is mapped, the code is converted into images and labels lists, then a sample and label table is created according to the information of the two lists, and then the data of the table is processed according to the following steps of 8: 1: 1, dividing the ratio into a training set, a testing set and a verification set, and continuously performing iterative training on the training set and the verification set by using a deep learning residual error network until an optimal tobacco leaf grade model is trained.

The method comprises the following specific steps:

step 1: tobacco sample treatment

1.1, dividing a tobacco leaf sample into 42 folders according to 42 grades, renaming the tobacco leaf sample by using a grade name, and storing the corresponding tobacco leaves under the folders of the corresponding grades according to the grades;

step 1.2, coding the tobacco leaf 42 grade into a number of [0,41], and forming a coding table by using the mapping relation between the tobacco leaf grade and the number;

step 1.3, after determining the coding table in step 1.2, obtaining a storage path of each tobacco sample and a label number corresponding to the storage path according to an actual storage mode of tobacco, and respectively representing the storage path and the label number as two list objects of images and labels, wherein the image list object stores a path character string of each sample, the labels list stores category numbers of the samples, the two list objects have the same length, and the corresponding element positions are associated with each other;

step 1.4 store images and labels of step 1.3 into a sample and label table CSV;

step 2: tobacco leaf image preprocessing

Step 2.1, reading the picture of the path corresponding to the image list, and converting the picture into a tensor form;

step 2.3, performing data enhancement operation on the read and converted image;

step 2.4, normalizing the image after data enhancement;

and 2.5, carrying out tensor conversion on the tobacco leaf grade label, and carrying out unique hot coding, wherein the tensor conversion is carried out as follows:

grade 1: 10000000000000000000000000000000000000000(41 pieces 0)

Grade 2: 01000000000000000000000000000000000000000(41 pieces 0)

Grade 3: 00100000000000000000000000000000000000000(41 pieces 0)

And so on for a total of 42 levels;

and step 3: data set partitioning

Step 3.1 loads a data set, selects the data in the CSV described in step 1.4, and performs the following steps according to 8: 1: 1, dividing the ratio into a training set, a test set and a verification set; the training set is used for training parameters of the model, the testing set is used for testing generalization ability of the model, and the verification set is used for feeding back selection of hyper-parameters of the model, wherein the selection is expressed by adjusting learning rate, attenuation coefficient, training times, topological structure of the network, overfitting, underfitting and the like. And continuously adjusting the hyper-parameter setting of the model according to the performance result of the verification set, and improving the accuracy and generalization capability of the model.

Step 3.2, scattering the training set in the step 3.1, and preventing a computer from memorizing position information and easily generating an overfitting phenomenon; the scattering tool is used for scattering the sequence among the data, so that the data are prevented from being generated according to a fixed sequence during each training, the model is enabled to memorize label information, and an overfitting phenomenon is easy to generate;

and 4, step 4: building a custom network model, wherein the model is a custom ResNet18 neural network model based on a residual error network;

step 4.1: building a residual error unit, wherein the structural schematic diagram of the residual error unit is shown in an attached diagram 2, and a residual error unit module consists of 2 parts, wherein F (x) represents the result of input x after 2 convolutional layer calculations, Identity is equivalent to a line, x is directly transmitted without being processed, and finally F (x) + x is calculated by addition, so that F (x) + x can be directly added, the shape of x is ensured to be the same as that of F (x), and when x is inconsistent with F (x), lower convolution operation is required to be performed to convert the x into the same shape; a Skip Connection is added between input and output, and a Connection hop is added in the network, so that the situation that the gradient of the network disappears and the network is not updated and optimized is prevented;

step 4.2, building a residual error module, as shown in an appendix 2, inputting x which needs to pass through 2 convolution layers, inputting x which passes through a convolution product with the size of 3 x 3 and a BN layer, performing Relu calculation, entering a second layer, adding x after passing through a convolution product with the size of 3 x 3 and BN, and finally performing Relu calculation;

4.3 building residual stacking modules, wherein a residual network consists of stacking modules with different numbers, and building ResNet with different layers by adjusting the stacking number and different image characteristic channels;

and 5: training a tobacco leaf grading model; the method specifically comprises the following steps:

step 5.1, assembling the model, and setting an optimizer, a loss function and a monitoring index of the model; the optimizer is an Adam self-adaptive optimizer, the monitoring index is an accuracy index, the loss function is a cross entropy loss function, and the cross entropy loss function formula is as follows:

where N is the total number of training data, y_iIs a predicted value of the network, t_iIs a target value for the network;

adding an additional constraint term on the loss function, namely adding L2 regularization in the network, wherein the regularization adopts an L2 norm as the constraint term of the loss function, and the L2 regularization term is formulated as:

wherein E₀For the cross entropy loss function, λ is the coefficient of the regularization term, and w represents all weight parameters and offsets. After adding the regular termNonlinear characteristics are reduced in the network, and the complexity of the network is reduced, so that better generalization capability is obtained;

performing loss calculation on the predicted result by using a cross entropy loss function to obtain a real tobacco leaf result and a loss value of the predicted result; the larger the error of the network is, the faster the learning speed of the network is, and the convergence speed of the network can be quickly accelerated.

The Adam self-adaptive optimizer is used for gradient descent, 1 batch of data is extracted from a training set for gradient calculation once through each training of the Adam optimizer, the gradient is calculated and the data moves along the reverse direction of the gradient, so that the error between a predicted value and a true value of the network is reduced, and the appropriate weight and deviation are found to enable the loss value of the network to be smaller and smaller;

the accuracy is used as a monitoring index of the model, and after each training is finished, the test set data is tested once to obtain the training accuracy of the model so as to evaluate the performance of the model;

step 5.2, training the model, and after the model is assembled, sending the data set to be trained and the test set into a network for training;

step 5.3, setting a visual callback function, monitoring the training progress and the training result curve of the network through the Web end in the network training process, and visually checking the performance of the evaluation model;

step 5.4, setting a user-defined learning rate function, wherein the learning rate is gradually reduced along with the increase of the training times Epochs;

step 5.5, setting a training early-stop function, monitoring the training accuracy of the model, and if a plurality of continuous Epochs are unchanged, considering that the model is optimal, and stopping training;

in the step 5, an output value of a network is obtained through forward propagation calculation, a network error is calculated through a loss function, then, updating iteration is carried out on the gradient, a model with the optimal performance is trained, model assembly is carried out through specifying hyper-parameters such as an optimizer and the loss function in the model, and the model can be automatically calculated and updated;

the tobacco leaf grading model further comprises a step 6, which is specifically as follows: and (4) storing the model, namely storing the trained model structure and parameters, and loading the model for use without retraining subsequently.

Further comprising step 7, specifically: and (5) evaluating the model, loading the stored model, verifying by using a verification set, and checking the accuracy.

In step 2.3, the data enhancement includes one or more of horizontal flipping, vertical flipping, random rotation, and scaling.

In step 2.4, the normalization standard processing process is as follows: firstly, the size of image pixels is normalized to [0,1], the image pixels are mapped to the distribution around 0, the optimization of a network is facilitated, the standardization operation is carried out by using a standardization function, and the standardization function formula is as follows:

x_o＝(x_i-mean)/std

wherein mean is [0.485,0.456,0.406 ]]Mean values of R, G, B channels, std ═ 0.229,0.224,0.225, respectively]Respectively representing the variance, x, of the R, G, B channels_iFor the input tensor of the tobacco leaf image, x_oThe normalized tobacco leaf image tensor is obtained.

An intelligent tobacco leaf grading method based on a residual error network comprises the following steps:

A. acquiring an image of the tobacco leaves to be detected and storing the image on a terminal computer;

B. b, reading the tobacco leaf image in the step A: the grading system detects that a tobacco leaf image exists under the folder to be detected, and starts to read the image;

C. tobacco leaf image preprocessing: the tobacco leaf image is required to be normalized and standardized, the image pixels are mapped to a [0,1] interval, and the standardization operation is carried out by using a standardization function, wherein the standardization formula is as follows:

x_o＝(x_i-mean)/std

wherein mean is [0.485,0.456,0.406 ]]Mean values of R, G, B channels, std ═ 0.229,0.224,0.225, respectively]Respectively representing the variance, x, of the R, G, B channels_iFor the input tensor of the tobacco leaf image, x_oThe normalized tobacco leaf image tensor is obtained;

D. loading a model: loading the trained model into the system through a path of the specified model;

E. model prediction: firstly, calculating an input image by using a model to obtain probability values of all levels of the image, taking an index with the maximum probability as an index result of model prediction, and then inquiring a level corresponding to the index from a level coding table to obtain the prediction level of the tobacco leaf.

Compared with the prior art, the invention has the following beneficial effects:

1. the tobacco leaf grading model is based on the tobacco leaf grading model to automatically grade the tobacco leaves, so that the phenomenon of low tobacco leaf grading accuracy caused by manual grading by a tobacco leaf grading worker is prevented, an image enhancement technology is added in the training process, the polymorphism of a tobacco leaf sample is expanded, the compatibility of the model is improved, the model can effectively adapt to multi-posture tobacco leaf images, and the generalization capability is stronger; regularization is added in the convolutional neural network, so that an overfitting phenomenon in convolutional neural network training can be effectively inhibited;

2. and self-defining a network model based on the residual error network structure so as to obtain a tobacco leaf grading network model with stable training and superior performance. By using the model, the tobacco leaf image can be read in real time on a computer terminal, and grading prediction can be rapidly carried out, so that the online grading efficiency of tobacco leaves can be improved;

3. adding an image enhancement function into the preprocessing module, performing preprocessing such as random rotation, shearing, scaling and the like, expanding polymorphism of the tobacco sample, and improving compatibility of the model;

4. the hole convolution is added in the user-defined network model, and under the condition that the number of network parameters is not increased by the hole convolution, the receptive field area is increased, and the accuracy of the tobacco leaf grading model is effectively improved.

Drawings

FIG. 1 is a flow chart of a tobacco leaf grading model;

FIG. 2 residual module;

FIG. 3VGG16 training accuracy curves;

fig. 4VGG19 training accuracy curves.

FIG. 5ResNet18 training accuracy curves;

FIG. 6ResNet34 training accuracy curves;

FIG. 7ResNet18 adds a L2 regularized training accuracy curve;

FIG. 8ResNet18 adds a training accuracy curve for L2 regularization and hole convolution;

FIG. 9 tobacco leaf test samples;

FIG. 10 is a graph of the first five probabilities of tobacco leaf grade prediction.

Detailed Description

The invention is further illustrated by the following specific examples.

Example 1 tobacco grading model training

A tobacco leaf intelligent grading model based on a residual error network is characterized in that a tobacco leaf image is coded, then a tobacco leaf grade mapping digital code is mapped, the code is converted into images and labels lists, then a sample and label table is created according to the information of the two lists, and then the data of the table is processed according to the following steps of 8: 1: 1, dividing the ratio into a training set, a testing set and a verification set, and continuously performing iterative training on the training set and the verification set by using a deep learning residual error network until an optimal tobacco leaf grade model is trained.

The method comprises the following specific steps:

step 1: tobacco sample treatment

1.1, dividing a tobacco leaf sample into 42 folders according to 42 grades, renaming the tobacco leaf sample by using a grade name, and storing the corresponding tobacco leaves under the folders of the corresponding grades according to the grades;

step 1.2, coding the tobacco leaf 42 grade into a number of [0,41], and forming a coding table by using the mapping relation between the tobacco leaf grade and the number;

step 1.3, after determining the coding table in step 1.2, obtaining a storage path of each tobacco sample and a label number corresponding to the storage path according to an actual storage mode of tobacco, and respectively representing the storage path and the label number as two list objects of images and labels, wherein the image list object stores a path character string of each sample, the labels list stores category numbers of the samples, the two list objects have the same length, and the corresponding element positions are associated with each other;

step 1.4 store images and labels of step 1.3 into a sample and label table CSV;

step 2: tobacco leaf image preprocessing

Step 2.1, reading the picture of the path corresponding to the image list, and converting the picture into a tensor form;

step 2.3, performing data enhancement operation on the read and converted image; the data enhancement comprises one or more than two of horizontal turning, vertical turning, random rotation and scaling;

step 2.4, normalizing the image after data enhancement; the normalized standard processing procedure is as follows: firstly, the size of image pixels is normalized to [0,1], the image pixels are mapped to the distribution around 0, the optimization of a network is facilitated, the standardization operation is carried out by using a standardization function, and the standardization function formula is as follows:

x_o＝(x_i-mean)/std

wherein mean is [0.485,0.456,0.406 ]]Mean values of R, G, B channels, std ═ 0.229,0.224,0.225, respectively]Respectively representing the variance, x, of the R, G, B channels_iFor the input tensor of the tobacco leaf image, x_oThe normalized tobacco leaf image tensor is obtained;

and 2.5, carrying out tensor conversion on the tobacco leaf grade label, and carrying out unique hot coding, wherein the tensor conversion is carried out as follows:

grade 1: 10000000000000000000000000000000000000000(41 pieces 0)

Grade 2: 01000000000000000000000000000000000000000(41 pieces 0)

Grade 3: 00100000000000000000000000000000000000000(41 pieces 0)

And so on for a total of 42 levels;

and step 3: data set partitioning

Step 3.1 loads a data set, selects the data in the CSV described in step 1.4, and performs the following steps according to 8: 1: 1, dividing the ratio into a training set, a test set and a verification set; the training set is used for training parameters of the model, the testing set is used for testing generalization ability of the model, and the verification set is used for feeding back selection of hyper-parameters of the model, wherein the selection is expressed by adjusting learning rate, attenuation coefficient, training times, topological structure of the network, overfitting, underfitting and the like. And continuously adjusting the hyper-parameter setting of the model according to the performance result of the verification set, and improving the accuracy and generalization capability of the model.

Step 3.2, scattering the training set in the step 3.1, and preventing a computer from memorizing position information and easily generating an overfitting phenomenon; the scattering tool is used for scattering the sequence among the data, so that the data are prevented from being generated according to a fixed sequence during each training, the model is enabled to memorize label information, and an overfitting phenomenon is easy to generate;

and 4, step 4: building a custom network model, wherein the model is a custom ResNet18 neural network model based on a residual error network;

step 4.1: building a residual error unit, wherein the structural schematic diagram of the residual error unit is shown in an attached diagram 2, and a residual error unit module consists of 2 parts, wherein F (x) represents the result of input x after 2 convolutional layer calculations, Identity is equivalent to a line, x is directly transmitted without being processed, and finally F (x) + x is calculated by addition, so that F (x) + x can be directly added, the shape of x is ensured to be the same as that of F (x), and when x is inconsistent with F (x), lower convolution operation is required to be performed to convert the x into the same shape; a Skip Connection is added between input and output, and a Connection hop is added in the network, so that the situation that the gradient of the network disappears and the network is not updated and optimized is prevented;

step 4.2, building a residual error module, as shown in an appendix 2, inputting x which needs to pass through 2 convolution layers, inputting x which passes through a convolution product with the size of 3 x 3 and a BN layer, performing Relu calculation, entering a second layer, adding x after passing through a convolution product with the size of 3 x 3 and BN, and finally performing Relu calculation;

4.3 building residual stacking modules, wherein a residual network consists of stacking modules with different numbers, and building ResNet with different layers by adjusting the stacking number and different image characteristic channels;

and 5: training a tobacco leaf grading model; the method specifically comprises the following steps:

step 5.1, assembling the model, and setting an optimizer, a loss function and a monitoring index of the model; the optimizer is an Adam self-adaptive optimizer, the monitoring index is an accuracy index, the loss function is a cross entropy loss function, and the cross entropy loss function formula is as follows:

where N is the total number of training data, y_iIs a predicted value of the network, t_iIs a target value for the network;

adding an additional constraint term on the loss function, namely adding L2 regularization in the network, wherein the regularization adopts an L2 norm as the constraint term of the loss function, and the L2 regularization term is formulated as:

wherein E₀For the cross entropy loss function, λ is the coefficient of the regularization term, and w represents all weight parameters and offsets. After the regular term is added, the nonlinear characteristics are reduced in the network, the complexity of the network is reduced, and therefore better generalization capability is obtained;

performing loss calculation on the predicted result by using a cross entropy loss function to obtain a real tobacco leaf result and a loss value of the predicted result; the larger the error of the network is, the faster the learning speed of the network is, and the convergence speed of the network can be quickly accelerated.

The Adam self-adaptive optimizer is used for gradient descent, 1 batch of data is extracted from a training set for gradient calculation once through each training of the Adam optimizer, the gradient is calculated and the data moves along the reverse direction of the gradient, so that the error between a predicted value and a true value of the network is reduced, and the appropriate weight and deviation are found to enable the loss value of the network to be smaller and smaller;

the accuracy is used as a monitoring index of the model, and after each training is finished, the test set data is tested once to obtain the training accuracy of the model so as to evaluate the performance of the model;

step 5.2, training the model, and after the model is assembled, sending the data set to be trained and the test set into a network for training;

step 5.3, setting a visual callback function, monitoring the training progress and the training result curve of the network through the Web end in the network training process, and visually checking the performance of the evaluation model;

step 5.4, setting a user-defined learning rate function, wherein the learning rate is gradually reduced along with the increase of the training times Epochs;

step 5.5, setting a training early-stop function, monitoring the training accuracy of the model, and if a plurality of continuous Epochs are unchanged, considering that the model is optimal, and stopping training;

in the step 5, an output value of a network is obtained through forward propagation calculation, a network error is calculated through a loss function, then, updating iteration is carried out on the gradient, a model with the optimal performance is trained, model assembly is carried out through specifying hyper-parameters such as an optimizer and the loss function in the model, and the model can be automatically calculated and updated;

step 6: and (4) storing the model, namely storing the trained model structure and parameters, and loading the model for use without retraining subsequently.

And 7, evaluating the model, loading the stored model, verifying by using the verification set and checking the accuracy.

Example 2

The grading method using the intelligent tobacco leaf grading model based on the residual error network in the embodiment 1 comprises the following steps:

A. acquiring an image of the tobacco leaves to be detected and storing the image on a terminal computer;

B. b, reading the tobacco leaf image in the step A: the grading system detects that a tobacco leaf image exists under the folder to be detected, and starts to read the image;

C. tobacco leaf image preprocessing: firstly, converting an image format into a tensor form, and performing image shape conversion, wherein the size of the image is consistent with that of a model; and then normalizing the tobacco leaf image, firstly mapping image pixels to a [0,1] interval, and performing normalization operation by using a normalization function, wherein the normalization formula is as follows:

x_o＝(x_i-mean)/std

wherein mean is [0.485,0.456,0.406 ]]Each represents RG, B channel mean, std ═ 0.229,0.224,0.225]Respectively representing the variance, x, of the R, G, B channels_iFor the input tensor of the tobacco leaf image, x_oThe normalized tobacco leaf image tensor is obtained;

D. loading a model: the trained model is loaded into the system through the path of the specified model, and the model is directly loaded to train the stored model parameters and structure without being trained and directly loaded for use;

E. model prediction: taking the tobacco leaf image to be verified as input, and predicting to obtain a prediction result of each grade; converting the prediction result into probability; acquiring 5 grade probabilities with the maximum probability and corresponding indexes, wherein the one with the maximum probability is a predicted grade result; mapping to five levels with the maximum probability through a coding table; the prediction results are displayed using visualization, as shown in appendix fig. 9, which is an image of tobacco leaves undergoing verification, and fig. 10, which is five levels with the highest prediction probability.

Example 3 validation set prediction

Step 1: loading a model, and loading the trained model parameters and structure;

step 2: traversing the verification set image folder, and preprocessing the image;

step 2.1: loading an image, and carrying out scaling treatment to keep the image consistent with the input size of the model;

step 2.2: establishing two empty lists, namely test _ Image and test _ Lable, which are respectively used for storing tobacco leaf Image data and the original grade of the tobacco leaf sample;

and step 3: carrying out normalization standardization processing on the tobacco leaf data listed in the step 2.2, wherein the tobacco leaf data are consistent with the model;

and 4, step 4: model prediction, namely predicting the data normalized in the step 3 as input to obtain all verification results;

and 5: traversing the result obtained in the step 4, and converting the result into probability;

step 6: finding the index with the maximum probability, and finding the corresponding grade through the coding table, namely the prediction grade of the tobacco leaves;

and 7: and comparing the predicted grade with the original grade, if the predicted grade is consistent with the original grade, correctly predicting, and counting and adding 1.

And 8: obtaining the accuracy of the verification set; accuracy is the predicted correct number/verified lumped number.

Comparative experiment of effects

To more clearly illustrate the effectiveness of the models in the embodiments of the present invention, experiments are performed below with different models, with or without L2 regularization, respectively. It should be noted that, in order to be able to check the stable convergence of the network, the early-stop function is not specified in the experiment, and all the early-stop functions are accuracy graphs obtained by training 100 Epochs, but the early-stop function, ResNet18, is adopted in the classification of the final model.

Experimental example 1: comparing different models, and performing model training by using network model commonly used in the prior art

The model is trained by using network structures of VGG16, VGG19, ResNet18 and ResNet34 respectively, and the training results are shown in appendix FIGS. 3, 4, 5 and 6 respectively, wherein train _ accuracy is the accuracy of the training set, and test _ accuracy is the accuracy of the test set. Comparing the four models, the accuracy of ResNet18 and ResNet34 is obviously better than that of VGG16 and VGG19, and the accuracy of ResNet18 and ResNet34 are similar, but the ResNet18 parameters are less and the convergence is faster, so the ResNet18 network structure is adopted.

Experimental example 2: l2 regularization

In experimental example 1, ResNet18 shows an overfitting phenomenon, and L2 regularization is added to the custom network, and the training result is shown in fig. 7. As can be seen from fig. 7, the L2 regularization can suppress the overfitting phenomenon to some extent, and the accuracy stably converges to 80%.

Experimental example 3: res18 hole convolution training accuracy test

On the basis of experimental example 2, hole convolution is added in the network layer, and the training result is shown in fig. 8, and the accuracy is stably converged to 88%. It should be noted that, in order to make the same comparison with the above experimental structure, the early-stop function is not used in the present experiment, but the early-stop function is added in the final model, and the training is started and ended when the 46 th epoch is trained.

Claims (7)

1. The intelligent tobacco leaf grading model based on the residual error network is characterized in that a tobacco leaf image is coded, then the tobacco leaf grade is mapped to digital codes, the codes are converted into images and labels lists, then a sample and label table is created according to the information of the two lists, and then the data of the table are processed according to the following steps of 8: 1: 1, dividing the ratio into a training set, a testing set and a verification set, and continuously performing iterative training on the training set and the verification set by using a deep learning residual error network until an optimal tobacco leaf grade model is trained.

2. The intelligent tobacco leaf grading model based on the residual error network according to claim 1, which is obtained by the following specific steps:

step 1: tobacco sample treatment

1.1, dividing a tobacco leaf sample into 42 folders according to 42 grades, renaming the tobacco leaf sample by using a grade name, and storing the corresponding tobacco leaves under the folders of the corresponding grades according to the grades;

step 1.2, coding the tobacco leaf 42 grade into a number of [0,41], and forming a coding table by using the mapping relation between the tobacco leaf grade and the number;

step 1.3, after determining the coding table in step 1.2, obtaining a storage path of each tobacco sample and a label number corresponding to the storage path according to an actual storage mode of tobacco, and respectively representing the storage path and the label number as two list objects of images and labels, wherein the image list object stores a path character string of each sample, the labels list stores category numbers of the samples, the two list objects have the same length, and the corresponding element positions are associated with each other;

step 1.4 store images and labels of step 1.3 into a sample and label table CSV;

step 2: tobacco leaf image preprocessing

Step 2.1, reading the picture of the path corresponding to the image list, and converting the picture into a tensor form;

step 2.3, performing data enhancement operation on the read and converted image;

step 2.4, normalizing the image after data enhancement;

and 2.5, carrying out tensor conversion on the tobacco leaf grade label, and carrying out unique hot coding, wherein the tensor conversion is carried out as follows:

grade 1: 10000000000000000000000000000000000000000(41 pieces 0)

Grade 2: 01000000000000000000000000000000000000000(41 pieces 0)

Grade 3: 00100000000000000000000000000000000000000(41 pieces 0)

And so on for a total of 42 levels;

and step 3: data set partitioning

Step 3.1 loads a data set, selects the data in the CSV described in step 1.4, and performs the following steps according to 8: 1: 1, dividing the ratio into a training set, a test set and a verification set;

step 3.2, scattering the training set in the step 3.1;

and 4, step 4: custom network model building

Step 4.1: building a residual error unit, wherein a residual error unit module consists of 2 parts, wherein F (x) represents the result of input x after 2 convolutional layer calculations, Identity is equivalent to a line, x is directly transmitted without processing, and finally F (x) + x is calculated in an adding way;

step 4.2, building a residual error module, inputting x, wherein the x needs to pass through 2 convolution layers, inputting x, entering a second layer after passing through a convolution sum with the size of 3 x 3 and a BN layer, performing Relu calculation, adding x after passing through a convolution sum with the size of 3 x 3 and BN, and finally performing Relu calculation;

4.3 building residual stacking modules, wherein a residual network consists of stacking modules with different numbers, and building ResNet with different layers by adjusting the stacking number and different image characteristic channels;

and 5: training a tobacco leaf grading model; the method specifically comprises the following steps:

step 5.1, assembling the model, and setting an optimizer, a loss function and a monitoring index of the model;

the optimizer is an Adam self-adaptive optimizer, the monitoring index is an accuracy index, the loss function is a cross entropy loss function, and the cross entropy loss function formula is as follows:

where N is the total number of training data, y_iIs a predicted value of the network, t_iIs a target value for the network;

adding an additional constraint term on the loss function, namely adding L2 regularization in the network, wherein the regularization adopts an L2 norm as the constraint term of the loss function, and the L2 regularization term is formulated as:

wherein E₀For the cross entropy loss function, lambda is the coefficient of a regular term, and w represents all weight parameters and offset;

step 5.2, training the model, and after the model is assembled, sending the data set to be trained and the test set into a network for training;

step 5.3, setting a visual callback function, monitoring the training progress and the training result curve of the network through the Web end in the network training process, and visually checking the performance of the evaluation model;

step 5.4, setting a user-defined learning rate function, wherein the learning rate is gradually reduced along with the increase of the training times Epochs;

and 5.5, setting a training early-stop function, monitoring the training accuracy of the model, and if a plurality of continuous Epochs are unchanged, considering that the model is optimal, and stopping training.

3. The intelligent tobacco leaf grading model based on the residual error network according to claim 2, further comprising a step 6, specifically: and (4) storing the model, namely storing the trained model structure and parameters, and loading the model for use without retraining subsequently.

4. The intelligent tobacco leaf grading model based on residual error network according to claim 3, further comprising a step 7, specifically: and (5) evaluating the model, loading the stored model, verifying by using a verification set, and checking the accuracy.

5. The intelligent tobacco leaf grading model based on residual error network according to any of the claims 2-4, characterized in that in step 2.3, the data enhancement comprises one or more of horizontal flipping, vertical flipping, random rotation, and scaling.

6. The intelligent tobacco leaf grading model based on the residual error network according to any one of claims 2-4, wherein in step 2.4, the normalization standard processing procedure is as follows: firstly, the size of image pixels is normalized to [0,1], the image pixels are mapped to the distribution around 0, the optimization of a network is facilitated, the standardization operation is carried out by using a standardization function, and the standardization function formula is as follows:

x_o＝(x_i-mean)/std

wherein mean is [0.485,0.456,0.406 ]]Mean values of R, G, B channels, std ═ 0.229,0.224,0.225, respectively]Respectively representing the variance, x, of the R, G, B channels_iFor the input tensor of the tobacco leaf image, x_oThe normalized tobacco leaf image tensor is obtained.

7. The method for tobacco intelligent grading model grading by using residual error network of claim 1 or 2, characterized by comprising the following steps:

A. acquiring an image of the tobacco leaves to be detected and storing the image on a terminal computer;

B. b, reading the tobacco leaf image in the step A: the grading system detects that a tobacco leaf image exists under the folder to be detected, and starts to read the image;

C. tobacco leaf image preprocessing: the tobacco leaf image is required to be normalized and standardized, the image pixels are mapped to a [0,1] interval, and the standardization operation is carried out by using a standardization function, wherein the standardization formula is as follows:

x_o＝(x_i-mean)/std

wherein mean is [0.485,0.456,0.406 ]]Mean values of R, G, B channels, std ═ 0.229,0.224,0.225, respectively]Respectively represent R,G. Variance of B channel, x_iFor the input tensor of the tobacco leaf image, x_oThe normalized tobacco leaf image tensor is obtained;

D. loading a model: loading the trained model into the system through a path of the specified model;

E. model prediction: firstly, calculating an input image by using a model to obtain probability values of all levels of the image, taking an index with the maximum probability as an index result of model prediction, and then inquiring a level corresponding to the index from a level coding table to obtain the prediction level of the tobacco leaf.

Images