CN110427967A - The zero sample image classification method based on embedded feature selecting semanteme self-encoding encoder - Google Patents

Abstract

The invention discloses a zero-sample image classification method based on an embedded feature selection semantic autoencoder. The present invention uses embedded feature selection to optimize the objective function of the semantic autoencoder, so that the obtained mapping matrix is as sparse as possible, so as to achieve the purpose of selecting the underlying features that match the semantic attributes; the obtained sparse mapping is used in the test phase The matrix maps the underlying features to semantic attributes, which can automatically suppress the feature dimensions that play a negative role, enhance the feature dimensions that play a positive role, achieve the purpose of feature matching selection, and improve the accuracy of zero-shot image classification.

Description

A Zero-Shot Image Classification Method Based on Embedded Feature Selection Semantic Autoencoders

technical field

The invention belongs to the field of pattern recognition, and in particular relates to a zero-sample image classification method.

Background technique

For the field of pattern recognition, zero-shot learning has always been a hot research topic. Due to the lack of human-labeled samples, the labeled categories cannot cover all object classes, so the application scenario of the zero-shot problem is more in line with the actual situation than the multi-classification problem. At present, the methods of zero-shot image classification mainly include: attribute-based methods; text-based methods; category-based similarity methods; methods combining different intermediate layers. At present, zero-sample learning is mainly using the most effective "attribute-based method".

Attribute-based zero-shot learning is to use attributes as an intermediate layer to transfer knowledge from visible classes to invisible classes. Because semantic attributes are a set of vectors that describe the characteristics of category objects, such as "fur", "tail", "four-legged" and other descriptive words, the attributes are shared by visible and invisible classes Therefore, it can play the role of zero-shot learning intermediate layer. At present, attribute learning includes two aspects: binary attribute learning and relative attribute learning. ”; while the attribute value of the relative attribute is a continuous value, indicating the relative strength value of the attribute. The concept and setting of relative attributes are more in line with human cognition and actual conditions, and the corresponding classification effect, that is, the accuracy is better than that of binary attributes under the same circumstances. Therefore, for zero-sample classification, the model method of relative attributes is better.

For the specific implementation of attribute-based zero-shot learning, there are currently two models mainly through the direct attribute prediction model (DAP) and the indirect attribute prediction model (IAP). The category labels in DAP are directly predicted by the attribute classifier, while the category labels in IAP are predicted indirectly. The main difference between DAP and IAP models is the difference in the learned classifiers. IAP needs to learn multi-class classifiers, and its test samples can only be assigned to unseen classes. However, DAP only needs to learn a set of attribute classifiers, and its test samples can be predicted as seen and unseen classes without restriction.

The difficulty of zero-shot learning is that there is no intersection between the test sample category and the training sample category. The classification results of traditional methods are often biased towards the training category label, which leads to the strong bias problem of zero-shot learning. The semantic self-encoder uses the semantic attributes as the middle layer, and the underlying features of the image as the input, so that the output can reconstruct the input features, which means that the encoded data can be restored to the original data as much as possible under the original encoding rules, which is in To a certain extent, the problem of strong bias is alleviated. However, due to the existence of global and local features in the image, and noise problems in some scenarios, not all dimensions in the underlying features of the image play a positive role in the learning of a certain attribute. Those interfering features Dimensionality affects the accuracy of attribute learning, resulting in zero-shot image classification performance.

Contents of the invention

In order to solve the technical problems mentioned above in the background technology, the present invention proposes a zero-shot image classification method based on embedded feature selection semantic autoencoder.

In order to realize above-mentioned technical purpose, technical scheme of the present invention is:

A zero-shot image classification method based on embedded feature selection semantic autoencoder, including the following steps:

(1) Fuse the matching selection of features with the training of semantic autoencoder to obtain the objective function of embedded feature selection semantic autoencoder:

Among them, W is the mapping matrix, superscript T means transpose, X is the underlying feature of the sample, S is the relative attribute vector, λ is the weight parameter of the encoding part, is the regularization parameter;

(2) Using the proximal gradient descent method to optimize the objective function obtained in step (1), and obtain the optimized embedded feature selection semantic autoencoder model;

(3) In the training phase of zero-shot image classification, the underlying features of the training samples and the corresponding relative attribute vectors are input into the optimized embedded feature selection semantic autoencoder model to obtain a sparse mapping matrix W;

(4) In the testing phase, input the underlying features of the test sample and the sparse mapping matrix obtained in step (3), and predict the relative attribute vector of the test sample;

(5) Assign category labels according to the relative attribute vector obtained in step (4).

Further, in step (2), the iterative equation of each step is as follows:

In the above formula, W _k is the kth iteration value of W, so that f′(W _k ) is the first-order derivative of f(W) at step k, L=-SX ^T +SS ^T W ₀ +λW ₀ XX ^T -λSX ^T , W ₀ is a matrix of all 1s with the same dimension as W .

Further, in step (3), at first, use folded cross-validation to randomly select the sample category in the training stage in the image set, then the remaining categories in the image set are used as the test set; then the underlying features of the training sample category and the corresponding The embedded feature selection semantic autoencoder optimized with respect to attribute vector input is trained to obtain a sparse mapping matrix.

Further, the relative attribute vectors of the training samples and the relative attribute vectors of the test samples are both statistically in the form of Gaussian distribution.

Further, in step (5), the mean and variance of the relative attribute vectors of the training samples and the mean and variance of the relative attribute vectors of the test samples are respectively calculated, and then classify the category labels through the maximum posterior probability.

The beneficial effect brought by adopting the above-mentioned technical scheme:

The present invention aims at the problem that the semantic autoencoder cannot actively select the underlying features for learning when performing zero-sample classification, and combines embedded feature selection, that is, adding an L1 norm regularization item in the objective function, and proposes an embedded feature selection semantic autoencoder. The encoder is optimized using the proximal gradient descent method, which improves the accuracy of relative attribute learning and the accuracy of the final zero-sample classification, and improves the overall performance. The present invention can be used in zero-sample image classification scenarios involving domain drift and strong bias, and can also be used in image zero-sample classification with noise background.

Description of drawings

Fig. 1 is the overall flowchart of the present invention;

Fig. 2 is a structural diagram of an embedded feature selection semantic autoencoder in the present invention.

Detailed ways

The technical solutions of the present invention will be described in detail below in conjunction with the accompanying drawings.

As shown in Figure 1, the present invention designs a zero-sample image classification method based on embedded feature selection semantic autoencoder, the steps are as follows:

Step 1: Add an L1 norm regularization item to the objective function of the semantic autoencoder, integrate the matching selection of features with the training of the semantic autoencoder, and form an embedded feature selection semantic autoencoder, whose objective The function is as follows:

Among them, W is the mapping matrix, superscript T means transpose, X is the underlying feature of the sample, S is the relative attribute vector, λ is the weight parameter of the encoding part, is a regularization parameter.

The semantic autoencoder consists of three layers: input layer, intermediate layer, and output layer. The input layer is the underlying feature of the image; the middle layer is the semantic attribute layer, that is, the semantic attribute is used as the middle layer; the output layer is obtained by decoding the semantic attribute layer. Make the output and input as identical as possible, which solves the problem of strong bias in zero-shot learning to a certain extent.

As shown in Figure 2, an L1 norm regularization term is added to the structure of the original semantic autoencoder to constrain the training process of the mapping matrix W, and a sparse mapping matrix W is obtained. Because the mathematical meaning of the L1 norm regular term is to calculate the sum of the absolute values of all elements in the matrix, so finding its minimum value can make the number of non-zero elements in the matrix as small as possible, and a sparse mapping matrix W can be obtained .

Step 2: Use the proximal gradient descent method to optimize the objective function obtained in step 1.

For the first two terms of the objective function, let Then its first-order derivative f′(W) satisfies the L-Lipschitz condition, then:

(f'(W ₂ )-f'(W ₁ ))≤L(W ₂ -W ₁ )

The above formula also satisfies:

(f'(W ₂ )-f'(W ₁ ))=(f"(W)(W ₂ -W ₁ ))≤L(W ₂ -W ₁ )

Therefore, it can be seen from the above that the value of L is the maximum value of the second derivative of f(W), and because its second derivative is:

f″(W)＝-SX ^T +SS ^T W+λWXX ^T -λSX ^T

And because the values of the elements of the W matrix are all values between 0 and 1, when W takes a matrix of all 1s, L can take the maximum value. Let W ₀ be a matrix of all 1s with the same dimension as W in the above formula, so the value of L is:

L＝-SX ^T +SS ^T W ₀ +λW ₀ XX ^T -λSX ^T

Around W _k , f(W) can be approximated by the second-order Taylor expansion as:

Among them, const is a constant that has nothing to do with W, <,> means the inner product, obviously the minimum value is obtained in the following W _k+1 :

Therefore, the iterations for each step can be obtained:

make Available:

After simplification and arrangement, we can get:

where the superscript i denotes the i-th component of W _k+1 and Z.

Step 3: In the training stage of zero-shot image classification, input the underlying features of the training samples and the corresponding relative attribute vectors into the model formed by the optimization results of Step 2, and obtain a sparse mapping matrix W.

First, use folded cross-validation to randomly select the sample category in the training stage in the image set, and then use the remaining categories in the image set as the test set; then input the underlying features of the training sample category and the corresponding relative attribute vectors into the optimized embedded features The semantic autoencoder is selected for training to obtain a sparse mapping matrix.

Step 4: In the test phase, input the underlying features of the test sample and the sparse mapping matrix obtained in step 3, and predict the relative attribute vector of the test sample.

Step 5: Assign category labels according to the relative attribute vector obtained in step 4.

Statistically calculate the relative attribute vector with the label prior as a Gaussian distribution, and calculate its mean and variance; calculate the predicted attribute vector value of the test sample as a Gaussian distribution, and calculate its mean and variance; according to the obtained mean and variance Maximum a posterior probability assigns class labels.

The embodiment is only to illustrate the technical idea of the present invention, and can not limit the scope of protection of the present invention with this. All technical ideas proposed in the present invention, any changes made on the basis of technical solutions, all fall within the scope of protection of the present invention .

Claims (5)

1. The zero-shot image classification method based on embedded feature selection semantic autoencoder, is characterized in that, comprises the following steps: (1) Fuse the matching selection of features with the training of semantic autoencoder to obtain the objective function of embedded feature selection semantic autoencoder: Among them, W is the mapping matrix, superscript T means transpose, X is the underlying feature of the sample, S is the relative attribute vector, λ is the weight parameter of the encoding part, is the regularization parameter; (2) Using the proximal gradient descent method to optimize the objective function obtained in step (1), and obtain the optimized embedded feature selection semantic autoencoder model; (3) In the training phase of zero-shot image classification, the underlying features of the training samples and the corresponding relative attribute vectors are input into the optimized embedded feature selection semantic autoencoder model to obtain a sparse mapping matrix W; (4) In the testing phase, input the underlying features of the test sample and the sparse mapping matrix obtained in step (3), and predict the relative attribute vector of the test sample; (5) Assign category labels according to the relative attribute vector obtained in step (4). 2. according to the described zero sample image classification method based on embedded feature selection semantic self-encoder of claim 1, it is characterized in that, in step (2), the iterative equation of each step is as follows: In the above formula, W _k is the kth iteration value of W, so that is the first-order derivative of f(W) at step k, L=-SX ^T +SS ^T W ₀ +λW ₀ XX ^T -λSX ^T , W ₀ is a matrix of all 1s with the same dimension as W. 3. the zero-shot image classification method based on embedded feature selection semantic autoencoder according to claim 1, is characterized in that, in step (3), at first utilize folding cross-validation to randomly select the sample of training stage in image set category, the remaining categories in the image set are used as the test set; then the underlying features of the training sample category and the corresponding relative attribute vectors are input into the optimized embedded feature selection semantic autoencoder for training, and a sparse mapping matrix is obtained. 4. The zero-shot image classification method based on an embedded feature selection semantic autoencoder according to claim 1, wherein the relative attribute vectors of the training samples and the relative attribute vectors of the test samples are all counted as a form of Gaussian distribution. 5. the zero-shot image classification method based on embedded feature selection semantic autoencoder according to claim 4, is characterized in that, in step (5), calculate the mean value and the variance of the relative attribute vector of training sample and test sample respectively The mean and variance of the relative attribute vector of , and then classify the category labels through the maximum posterior probability.