CN104408469A - Firework identification method and firework identification system based on deep learning of image - Google Patents

Abstract

The invention discloses a firework recognition method and system based on image deep learning, comprising: step 1, collecting an unlabeled sample image set and a labeled sample image set; step 2, obtaining an unlabeled training data set and a labeled training data set ; Step 3, whitening preprocessing of training data; Step 4, based on unlabeled training data after whitening preprocessing, use unsupervised learning to construct a deep neural network based on sparse self-encoding, and extract basic image features of unlabeled training data set; step 5, convolving basic image features and pooling image data; step 6, training Softmax classifier based on the labeled training data set after convolution and pooling; step 7, convolving and pooling the waiting The recognition image is input into the trained Softmax classifier to obtain the recognition result. The invention can effectively improve the visual recognition rate of fireworks and similar targets, and can realize higher-precision automatic recognition of fireworks.

Description

Fireworks recognition method and system based on image deep learning

technical field

The invention belongs to the technical field of fire intelligent monitoring and pyrotechnic target automatic recognition based on digital images, and in particular relates to a pyrotechnic recognition method and system based on image deep learning.

Background technique

The intelligent monitoring of pyrotechnics based on digital images is a classic problem related to image processing, computer vision, artificial intelligence, machine learning and many other fields. At present, there are some literatures on automatic recognition of pyrotechnic objects. The recognition process can generally be divided into target segmentation, feature Extraction, comprehensive judgment and other stages.

Phase 1, target segmentation.

Pyrotechnic automatic target segmentation can be roughly divided into threshold segmentation, edge detection segmentation, region characteristic segmentation, feature space clustering segmentation and other methods. Threshold segmentation methods mainly include histogram threshold, maximum inter-class variance (Otsu) threshold, two-dimensional maximum entropy value, fuzzy threshold, co-occurrence matrix threshold, etc.; edge detection segmentation methods mainly include Sobel operator, Canny operator, Laplacan operator, Roberts operator, Prewitt operator, Susan operator, active contour model, watershed algorithm, level set method, etc.; regional feature segmentation methods mainly include area growth, area separation and merging, mathematical morphology, etc.; feature space clustering segmentation methods mainly Including K-means, fuzzy C-means, Mean-Shift, etc. Specifically, the acquisition of pyrotechnic targets is often achieved by color segmentation, such as the color range of fire and the gray range of smoke, and commonly used color models include RGB, HSI, YCbCr, etc.

The second stage is feature extraction.

The visual features of pyrotechnic targets mainly include features such as color, shape, texture, and spatial relationship. Color features are not affected by image rotation and translation changes, and further normalization is not affected by scale changes. Commonly used color features include color histograms, color sets, color moments, color aggregation vectors, and color correlation maps. Shape features include contour features and region features. Contour features are mainly aimed at object boundaries, while region features are related to the entire object area. Commonly used shape features include boundary chain codes, Fourier descriptors, geometric shape parameters, and shape invariant moments. and wavelet relative moments, etc. Texture features have strong resistance to noise, but will be affected by related factors such as resolution, directionality, and prior assumptions. Commonly used texture analysis methods include statistical analysis, geometric analysis, and spectrum analysis. Spatial relationship refers to the mutual position or orientation relationship between multiple targets, which can enhance the ability to distinguish image content descriptions, but it is sensitive to target rotation and scale changes, and it is often not enough to use only spatial relationship information in practical applications. The expression of the above-mentioned pyrotechnic features often requires the help of certain mathematical tools, such as Laplace operator, Fourier transform, gray level co-occurrence matrix, hidden Markov model, LBP operator, discrete wavelet analysis, etc.

Stage three, comprehensive judgment.

The comprehensive judgment of pyrotechnic targets is based on the extracted multiple features to give the conclusion of whether there is a fire, that is, the design and use of the pattern recognition classifier. The pyrotechnic image features commonly used in comprehensive judgment include brightness value, color distribution value, texture parameter, centroid, area, average density, circularity, curvature, eccentricity, number of sharp corners, fractal code, transmittance, etc. Pattern classification includes supervised and unsupervised two types, which can be carried out independently or jointly at the three levels of information layer, feature layer and decision-making layer. The pattern classification for pyrotechnics is mainly implemented at the feature layer, and common methods include voting method, least mean square fusion, Bayes classifier, fuzzy logic, artificial neural network, support vector machine, etc.

The above method has verified its effectiveness in occasions such as building fire monitoring, but in natural scenes, sometimes there are objects similar to fireworks, such as red flowers, red leaves, and red flags similar to fire, and fog, clouds, and haze similar to smoke wait. The objective existence of these objects leads to low accuracy of pyrotechnic recognition, and high rate of false positives and false positives. Therefore, there is an urgent need for a more accurate pyrotechnic target recognition method in the field of fire intelligent monitoring. In recent years, deep learning technology in the field of machine learning has been gradually applied to image processing and pattern recognition. Deep learning can achieve complex function approximation by learning a deep nonlinear network structure, representing the distributed representation of input data, and showing a powerful The ability to learn the essential characteristics of the data set from a small number of samples. So far, there has been no research combining deep learning with pyrotechnic recognition.

Contents of the invention

Aiming at the deficiencies in the prior art, the present invention combines deep learning with pyrotechnic recognition to provide a pyrotechnic recognition method and system based on image deep learning to achieve higher-precision automatic pyrotechnic recognition.

In order to solve the problems of the technologies described above, the present invention adopts the following technical solutions:

A firework recognition method based on image deep learning, including steps:

Step 1, collect a sample image set, including (1) an unlabeled sample image set composed of unclassified and labeled target images and target similar images and (2) a labeled sample image composed of classified and labeled target images and target similar images set;

Step 2, randomly obtain unit image blocks from the unlabeled sample image set and the labeled sample image set, respectively, to form an unlabeled training data set and a labeled training data set;

Step 3, performing whitening preprocessing on the training data in the unlabeled training data set and the labeled training data set, wherein the training data is the color value matrix of the RGB three-color channel corresponding to the unit image block;

Step 4, based on the unlabeled training data after whitening preprocessing, use unsupervised learning to construct a deep neural network based on sparse self-encoding, and extract the basic image feature set of the unlabeled training data;

Step 5, convolving the basic image features of the unlabeled training data and pooling the image data, the image data including the labeled training data and the image to be recognized;

Step 6, training the Softmax classifier based on the labeled training data set after convolution and pooling;

Step 7, input the image to be recognized after convolution and pooling into the trained Softmax classifier to obtain the recognition result.

The whitening preprocessing described in the above step 3 is ZCA whitening preprocessing or PCA whitening preprocessing.

Above-mentioned step 4 further comprises sub-steps:

4.1 Construct a deep neural network, including a single input layer, multiple hidden layers and a single output layer;

4.2 Use the unlabeled training data after whitening preprocessing as the input and output of the deep neural network, and perform unsupervised learning by training the deep neural network based on sparse self-encoding;

4.3 Extract the basic image feature set of unlabeled training data based on the trained deep neural network.

Unsupervised learning by training a deep neural network based on sparse autoencoder described in sub-step 4.2, specifically:

4.2.1 Obtain the weighted sum of neuron input values and neuron output values;

4.2.2 Set the target cost function with sparsity restriction;

4.2.3 Set the gradient descent direction of the weight coefficient vector and bias item vector of the deep neural network, that is, the iteration rule;

4.2.4 Use the LBFGS parameter training algorithm to iteratively solve the weight coefficient vector and bias item vector according to the set iteration rules.

Above-mentioned step 5 further comprises sub-steps:

5.1 Convolve the basic image features of the unlabeled training data with each color channel of the image data to obtain a convolution image;

5.2 Using the statistical characteristics of local areas in natural images, the feature dimensionality reduction of convolutional images is realized through mean pooling.

Above-mentioned step 6 further comprises sub-steps:

6.1 Use the labeled training data set after convolution and pooling as the training sample;

6.2 Construct a Softmax classifier regression model;

6.3 Set the gradient of the parameters of the regression model, that is, the iteration rule;

6.4 Use the LBFGS parameter training algorithm to iteratively solve the model parameter θ according to the set iteration rules.

The system corresponding to the above-mentioned pyrotechnic recognition method based on image deep learning includes:

The sample image collection module is used to collect sample image sets, including (1) an unlabeled sample image set composed of unclassified and marked target images and target similar object images and (2) an unlabeled sample image set composed of classified and marked target images and target similar object images A set of labeled sample images;

The training data acquisition module is used to randomly obtain unit image blocks from the unlabeled sample image set and the labeled sample image set respectively to form an unlabeled training data set and a labeled training data set;

The whitening preprocessing module is used to carry out whitening preprocessing to the training data in the unlabeled training data set and the training data set with labels, and the described training data is the color value matrix of the RGB three-color channel corresponding to the unit image block;

The unsupervised learning module is used to construct a deep neural network based on sparse self-encoding by using unsupervised learning based on the unlabeled training data after whitening preprocessing, and extract the basic image feature set of the unlabeled training data;

Convolution and pooling modules are used to convolve the basic image features of unlabeled training data with pooled image data, and the image data includes labeled training data and images to be identified;

The classifier training module is used to train the Softmax classifier based on the labeled training data set after convolution and pooling;

The identification module is used to input the image to be identified after convolution and pooling into the trained Softmax classifier to obtain the identification result.

Compared with the prior art, the present invention has the following advantages and positive effects:

(1) Deep learning achieves complex function approximation by learning deep nonlinear network structure, characterizes the distributed representation of input data, and has a strong ability to learn the essential characteristics of data from large sample sets, so the classification accuracy of sparse self-encoded deep neural network than traditional neural networks.

(2) The ZCA technology used is used for data dimension reduction, and the whitening technology is used to reduce the correlation between input image pixels, which is conducive to improving the speed of unsupervised learning.

(3) The convolution technology adopted helps to reduce the parameters that need to be trained by the neural network and simplifies the feature extraction process. The pooling technology helps to use local area statistical features to achieve feature dimensionality reduction and prevent overfitting.

(4) The Softmax classifier used is an extension of the binary classification method, which can solve multi-classification problems and is beneficial to the recognition of fireworks and more types of similar objects.

Description of drawings

Figure 1 shows the sample image and the basic image feature set learned by the deep neural network.

Detailed ways

The technical solution of the present invention can be implemented by those skilled in the art by means of computer software, and the present invention will be further described with specific embodiments below.

Concrete steps of the present invention are as follows:

Step 1. Obtain a sample image set composed of an unlabeled sample image set and a labeled sample image set, and obtain an unlabeled training data set and a labeled training data set based on the sample image set.

This step further includes the following sub-steps:

Step 1.1, collecting unlabeled sample images to form an unlabeled sample image set.

The unlabeled sample image set includes unlabeled target images and target similar images. Targets are fire and smoke, and target similar objects refer to objects similar to fire and smoke. For example, safflower, red leaves, red flags, etc. are similar to fire; fog, cloud, haze, etc. are similar to smoke. In this sub-step, a large number of images of fire, red flowers, red leaves, and red flags are used to form the first type of unlabeled sample image set, and a large number of images of smoke, fog, cloud, and haze are used to form the second type of unlabeled sample image set.

Step 1.2, collecting labeled sample images to form a labeled sample image set.

The labeled sample image set includes classified and marked target images and target similar images, such as the first type of labeled sample image set composed of classified and marked images of fire, safflower, red leaves, and red flags, classified and marked smoke, The second type of labeled sample image set consists of images of fog, cloud, and haze.

In step 1.3, an unlabeled training dataset and a labeled training dataset are obtained based on the sample image set.

Randomly obtain unit image blocks of fixed size (such as 8 pixels×8 pixels) from unlabeled sample images as an unlabeled training data set; randomly obtain fixed size (such as 8 pixels×8 pixels) from labeled sample images Unit image patches, as a labeled training dataset.

Step 2, whitening preprocessing of the training data in the unlabeled training data set and the labeled training data set, the training data is the unit image block obtained in step 1.3, that is, the color value matrix of the RGB three-color channel corresponding to the unit image block.

In this specific embodiment, ZCA whitening preprocessing is carried out to the training data, including the following sub-steps in turn:

Step 2.1, zero-meanization of training data.

Subtract the average value of each dimension to get x ⁱ , and normalize the training data to the range of [0,1]. Let m be the number of training data, and the covariance matrix Σ of the training data can be obtained:

$\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}<span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; m</span>}}\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; m</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}<span\; class="notranslate">\; [</span>{\mathrm{<span\; class="notranslate">\; x</span>}}^{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; \&Center\; Dot;</span>{<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}^{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; T</span>}}<span\; class="notranslate">\; ]</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>$

Step 2.2, calculate the vector basis of the training data after zero-meanization in the new dimension.

Perform singular value decomposition on the covariance matrix Σ to obtain the eigenvalue diagonal matrix S and the n-dimensional eigenvector U=[u ₁ u ₂ ...u _n ], where u ₁ is the main eigenvector of Σ, and u ₂ is the secondary eigenvector , u _n is the lowest eigenvector, and these eigenvectors constitute a set of vector bases under the new dimension coordinates.

Step 2.3, obtain the training data under the new dimension.

Dimensional transformation of the training data to obtain new data x ^r = U ^T x ⁱ , obviously the dimensions in x ^r are independent of each other, and then divide x ^r by the standard deviation The variance of each dimension is obtained to be 1, so as to satisfy the two necessary conditions that the mean value of whitening is close to 0 and the variance is equal. Let ε be the ZCA whitening parameter. In this specific implementation, ε is set to 10 ^-5 , and the final ZCA whitening result is

${\mathrm{<span\; class="notranslate">\; x</span>}}^{\mathrm{<span\; class="notranslate">\; t</span>}}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; u</span>}<span\; class="notranslate">\; \&Center\; Dot;</span>\frac{{\mathrm{<span\; class="notranslate">\; x</span>}}^{\mathrm{<span\; class="notranslate">\; r</span>}}}{\sqrt{\mathrm{<span\; class="notranslate">\; S</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; \&epsiv;</span>}}}<span\; class="notranslate">\; \&Center\; Dot;</span>{\mathrm{<span\; class="notranslate">\; u</span>}}^{\mathrm{<span\; class="notranslate">\; T</span>}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; )</span>$

The training data preprocessing in the present invention is not limited to ZCA whitening, and other conventional whitening techniques such as PCA whitening may also be used.

Step 3, conduct unsupervised learning on the unlabeled training data, and build a deep neural network based on sparse autoencoder.

This step further includes the following sub-steps:

Step 3.1, constructing a deep neural network, including an input layer, a hidden layer and an output layer, the input layer and the output layer are both single layers, and the hidden layer is multi-layered, and the unlabeled training data is used as the input and output of the deep neural network.

In step 3.2, unsupervised learning is performed based on training the deep neural network, that is, the weight coefficient vector and the bias item vector of the deep neural network are obtained.

Step 3.2.1, obtain the weighted sum of neuron input values:

set up Represents the jth neuron in the first layer of the connected neural network and the weight coefficient of the i-th neuron in layer l+1, Represents the bias item of the i-th neuron in layer l+1, S _l represents the total number of neurons in layer l, Represents the weighted sum of the input values of the i-th neuron in the l+1th layer, then:

${\mathrm{<span\; class="notranslate">\; z</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}^{\mathrm{<span\; class="notranslate">\; l</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{{\mathrm{<span\; class="notranslate">\; S</span>}}_{\mathrm{<span\; class="notranslate">\; l</span>}}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; w</span>}}_{\mathrm{<span\; class="notranslate">\; ij</span>}}^{\mathrm{<span\; class="notranslate">\; l</span>}}{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}^{\mathrm{<span\; class="notranslate">\; l</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; b</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}^{\mathrm{<span\; class="notranslate">\; l</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 3</span>}<span\; class="notranslate">\; )</span>$

Step 3.2.2, get neuron output value:

Known neuron activation function is Indicates the output value of the i-th neuron in the l-th layer of the neural network, namely Let the unlabeled training data x ^t , that is, the input sample of the self-encoded deep neural network and the output result y ^t be equal, that is, y ^t = x ^t , let M be the number of unlabeled training data x ^t , and t be the number of unlabeled training data, Then 1≤t≤M; let Indicates the output value of the j-th neuron in the l-th layer when the input sample is x ^t , then the average output value of the j-th neuron in the hidden layer for:

Step 3.2.3, define the objective cost function of the deep neural network:

Adding sparsity constraints to deep neural networks, that is, ρ is a sparsity parameter, and the sparsity parameter is a positive number close to 0, and generally takes a value between 0 and 0.05. In this specific implementation, ρ=0.035. That is to say, to make the average output value of the jth neuron in the hidden layer Close to ρ, in order to achieve the sparsity limit, define the cost objective function J(w,b):

The cost objective function consists of the sum of three parts, the first part is the mean square error term, the second part is the regularization term, and the third part is the penalty term, which is used to penalize those and ρ are significantly different to achieve sparsity constraints on neural networks. Among them, N is the number of self-encoded deep neural network layers; λ is the regularization coefficient, and in this specific implementation, λ=0.003; h _w,b ( ^xi ) is the output value of the neural network output layer corresponding to the input sample x ^t ; β is the coefficient for controlling the sparsity limitation penalty term, in this specific implementation, β=5; w and b are the weight coefficient vector and bias term vector of the deep neural network respectively. yes The relative entropy between ρ and ρ, which is used to measure the difference between two distributions, as a convex function, the relative entropy is calculated as:

Step 3.2.4, solve the objective cost function:

For the weight coefficient vector and bias term vector of the deep neural network, define their gradient descent direction:

$\left\{\begin{array}{c}{<span\; class="notranslate">\; \&dtri;</span>\mathrm{<span\; class="notranslate">\; w</span>}}^{\mathrm{<span\; class="notranslate">\; l</span>}}<span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; m</span>}}<span\; class="notranslate">\; \&CenterDot;</span>{\mathrm{<span\; class="notranslate">\; \&sigma;</span>}}^{\mathrm{<span\; class="notranslate">\; l</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; \&CenterDot;</span>{<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; a</span>}}^{\mathrm{<span\; class="notranslate">\; l</span>}}<span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; T</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; \&lambda;w</span>}}^{\mathrm{<span\; class="notranslate">\; l</span>}}\\ {<span\; class="notranslate">\; \&dtri;</span>\mathrm{<span\; class="notranslate">\; b</span>}}^{\mathrm{<span\; class="notranslate">\; l</span>}}<span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; m</span>}}\underset{\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; m</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; \&sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}}^{\mathrm{<span\; class="notranslate">\; l</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}}\end{array}\right.<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 7</span>}<span\; class="notranslate">\; )</span>$

In formula (7), Indicates the gradient descent direction of the l-th layer weight coefficient vector, Indicates the gradient descent direction of the bias item vector of the lth layer; w ^l indicates the weight coefficient vector of the lth layer; a ^l is the output vector of the lth layer of the neural network, is the residual value corresponding to the input sample x ^t at layer l+1, and σ ^l+1 is the residual vector of this layer.

Formula (7) determines the iteration rules of w and b. In this specific implementation, the LBFGS parameter training algorithm is used to iteratively solve w and b. The current weight coefficient vector w and bias term vector b when the iteration converges or reaches the maximum number of iterations, That is, the weight coefficient vector w and the bias item vector b of the trained sparse self-encoded deep neural network. The iteration convergence standard and the maximum number of iterations are preset according to actual needs. The weight coefficient vector w and the bias item vector b are obtained, that is, the training of the sparse self-encoding deep neural network is completed.

Step 3.3, based on the trained deep neural network, the basic image feature set expressing the unlabeled training data is extracted.

The basic image feature set refers to the set of basic image features that can constitute a complex image. See Figure 1. The upper left is one of the sample images used for training (1), and the right is the basic image feature set of all sample images learned by deep neural networks. (3), and the combination of basic image features can express any unit image block in the sample image (2).

Step 4, using the basic image feature set to convolve and pool the image data, the image data includes labeled training data and image data to be recognized.

This step further includes sub-steps:

In step 4.1, the basic image features are respectively convolved with each color channel of each image data, that is, the image pixels within the scope of the convolution template are averaged and the average value is used as the target value, and the convolution of the three color channels The results are added up to get the convolved image.

Step 4.2, using the statistical characteristics of the local area in the natural image, realize the feature dimension reduction of the convolution image through mean pooling, that is, divide the convolution image into regions, calculate the average value of pixels in each region, and use the average value of pixels in each region to represent the region.

Statistical characteristics of local regions are inherent characteristics of natural images, that is, the statistical characteristics of a part of natural images are similar to other parts. For example, a region of a landscape image has similarities to other regions. This means that the features learned in one part of the image can also be applied to another part, and the mean pooling is the specific implementation method.

Step 5, train the Softmax classifier based on the labeled training data set.

This step further includes sub-steps:

Step 5.1, construct the training sample set of Softmax classifier.

The labeled training data after convolution and pooling form a training sample set {(x ¹ ,y ¹ ),(x ² ,y ² ),...,(x ^K ,y ^K )}, K is labeled The number of training samples, ^xi represents the i-th training sample, that is, the labeled training data after convolution and pooling, and y ⁱ is the classification label corresponding to the training sample ^xi . Suppose the Softmax classifier is used to solve the k classification problem, then y ⁱ ∈ {1,2,...,k}.

Step 5.2, construct the Softmax classifier regression model.

Let θ be the model parameter, which is to be estimated; $h_{\mathrm{\&theta;}}\left(x^i\right)=\left[\begin{array}{c}p({\mathrm{the\; y}}^i=1|x^i;\mathrm{\&theta;})\\ p({\mathrm{the\; y}}^i=2|x^i;\mathrm{\&theta;})\\ .\\ .\\ .\\ p({\mathrm{the\; y}}^i=k|x^i;\mathrm{\&theta;})\end{array}\right]$ is the evaluation function of the model parameter θ, where The cost function J(θ) of the evaluation function h _θ ( ^xi ) is:

$\mathrm{<span\; class="notranslate">\; J</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; \&theta;</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span><span\; class="notranslate">\; -</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; K</span>}}<span\; class="notranslate">\; [</span>\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; K</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\underset{\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; k</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; the\; y</span>}}^{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span>\mathrm{<span\; class="notranslate">\; log</span>}{\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; \&theta;</span>}}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}^{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ]</span><span\; class="notranslate">\; ]</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 8</span>}<span\; class="notranslate">\; )</span>$

Among them, f(y ⁱ =j) is an indicator function, the value is 0 or 1, if the i-th training sample x ⁱ label is category j, then the function f(y ⁱ =j)=1, otherwise, the function f( y ⁱ =j)=0.

Step 5.3, define the gradient of the model parameter θ

${<span\; class="notranslate">\; \&dtri;</span>}_{\mathrm{<span\; class="notranslate">\; \&theta;j</span>}}\mathrm{<span\; class="notranslate">\; J</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; \&theta;</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span><span\; class="notranslate">\; -</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; K</span>}}\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; K</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}<span\; class="notranslate">\; [</span>{\mathrm{<span\; class="notranslate">\; x</span>}}^{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; the\; y</span>}}^{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; p</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; the\; y</span>}}^{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; x</span>}}^{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; ;</span>\mathrm{<span\; class="notranslate">\; \&theta;</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; )</span><span\; class="notranslate">\; ]</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 9</span>}<span\; class="notranslate">\; )</span>$

Formula (9) gives the iteration rule for the model parameter θ. In this specific implementation, the LBFGS parameter training algorithm is used to iteratively solve the model parameter θ, and iterative calculation is performed based on the iteration rule of formula (9). When the iteration converges or reaches the maximum number of iterations The current regression model parameter θ is the optimal solution of the regression model parameter θ of the Softmax classifier, and the regression model parameter θ is obtained, that is, the training of the Softmax classifier is completed.

After completing steps 1 to 5, the image to be recognized is convolved and pooled using the basic image feature set learned by the sparse self-encoded deep neural network, and the convolved and pooled image to be recognized is input into the trained Softmax classifier , the classification result can be obtained, and it can be judged that the image to be recognized is an image of fire, red flower, red leaf or red flag, or an image of smoke, fog, cloud or haze.

The specific embodiments described herein are merely illustrative of the spirit of the invention. Those skilled in the art to which the present invention belongs can make various modifications or supplements to the described specific embodiments or adopt similar methods to replace them, but they will not deviate from the spirit of the present invention or go beyond the definition of the appended claims range.

Claims (7)

1. The fireworks recognition method based on image deep learning, is characterized in that, comprises steps: Step 1, collect a sample image set, including (1) an unlabeled sample image set composed of unclassified and labeled target images and target similar images and (2) a labeled sample image composed of classified and labeled target images and target similar images set; Step 2, randomly obtain unit image blocks from the unlabeled sample image set and the labeled sample image set, respectively, to form an unlabeled training data set and a labeled training data set; Step 3, performing whitening preprocessing on the training data in the unlabeled training data set and the labeled training data set, wherein the training data is the color value matrix of the RGB three-color channel corresponding to the unit image block; Step 4, based on the unlabeled training data after whitening preprocessing, use unsupervised learning to construct a deep neural network based on sparse self-encoding, and extract the basic image feature set of the unlabeled training data; Step 5, convolving the basic image features of the unlabeled training data and pooling the image data, the image data including the labeled training data and the image to be recognized; Step 6, training the Softmax classifier based on the labeled training data set after convolution and pooling; Step 7, input the image to be recognized after convolution and pooling into the trained Softmax classifier to obtain the recognition result. 2. the fireworks recognition method based on image deep learning as claimed in claim 1, is characterized in that: The whitening preprocessing described in step 3 is ZCA whitening preprocessing or PCA whitening preprocessing. 3. the fireworks recognition method based on image deep learning as claimed in claim 1, is characterized in that: Step 4 further includes sub-steps: 4.1 Construct a deep neural network, including a single input layer, multiple hidden layers and a single output layer; 4.2 Use the unlabeled training data after whitening preprocessing as the input and output of the deep neural network, and perform unsupervised learning by training the deep neural network based on sparse self-encoding; 4.3 Extract the basic image feature set of unlabeled training data based on the trained deep neural network. 4. the pyrotechnic identification method based on image deep learning as claimed in claim 3, is characterized in that: Unsupervised learning by training a deep neural network based on sparse autoencoder described in sub-step 4.2, specifically: 4.2.1 Obtain the weighted sum of neuron input values and neuron output values; 4.2.2 Set the target cost function with sparsity restriction; 4.2.3 Set the gradient descent direction of the weight coefficient vector and bias item vector of the deep neural network, that is, the iteration rule; 4.2.4 Use the LBFGS parameter training algorithm to iteratively solve the weight coefficient vector and bias item vector according to the set iteration rules. 5. the pyrotechnic identification method based on image deep learning as claimed in claim 1, is characterized in that: Step 5 further includes sub-steps: 5.1 Convolve the basic image features of the unlabeled training data with each color channel of the image data to obtain a convolution image; 5.2 Using the statistical characteristics of local areas in natural images, the feature dimensionality reduction of convolutional images is realized through mean pooling. 6. the pyrotechnic identification method based on image deep learning as claimed in claim 1, is characterized in that: Step 6 further includes sub-steps: 6.1 Use the labeled training data set after convolution and pooling as the training sample; 6.2 Construct a Softmax classifier regression model; 6.3 Set the gradient of the regression model parameters, that is, the iteration rule; 6.4 Use the LBFGS parameter training algorithm to iteratively solve the model parameters according to the set iteration rules . 7. The firework recognition system based on image depth learning, is characterized in that, comprises: The sample image collection module is used to collect sample image sets, including (1) an unlabeled sample image set composed of unclassified and marked target images and target similar object images and (2) an unlabeled sample image set composed of classified and marked target images and target similar object images A set of labeled sample images; The training data acquisition module is used to randomly obtain unit image blocks from the unlabeled sample image set and the labeled sample image set respectively to form an unlabeled training data set and a labeled training data set; The whitening preprocessing module is used to carry out whitening preprocessing to the training data in the unlabeled training data set and the training data set with labels, and the described training data is the color value matrix of the RGB three-color channel corresponding to the unit image block; The unsupervised learning module is used to construct a deep neural network based on sparse self-encoding by using unsupervised learning based on the unlabeled training data after whitening preprocessing, and extract the basic image feature set of the unlabeled training data; Convolution and pooling modules are used to convolve the basic image features of unlabeled training data with pooled image data, and the image data includes labeled training data and images to be identified; The classifier training module is used to train the Softmax classifier based on the labeled training data set after convolution and pooling; The identification module is used to input the image to be identified after convolution and pooling into the trained Softmax classifier to obtain the identification result.