CN114841257B - A small sample target detection method based on self-supervised contrast constraints - Google Patents

Abstract

The present invention provides a small sample target detection method based on self-supervised contrast constraints. The method includes: modeling the small-sample target detection problem into a mathematical optimization problem based on self-supervised learning, building a small-sample target detection model that is sensitive to data disturbances; designing an optimized objective function for the small-sample target detection model; based on the optimized objective function Use the deep learning update process to train the small sample target detection model to obtain the trained small sample target detection model, and use the trained small sample target detection model to perform target detection on the small sample to be detected. This invention is based on a two-stage learning process, uses transfer learning to learn domain knowledge, and fine-tunes the model on a small sample data set. Experimental results prove that the present invention has achieved good performance on the PASCAL-VOC public data set, can effectively improve the performance of the model on small sample target detection problems, and has strong practical application significance.

Description

A small sample target detection method based on self-supervised contrast constraints

Technical field

The invention relates to the technical field of target detection, and in particular to a small sample target detection method based on self-supervised contrast constraints.

Background technique

In recent years, with the development of deep convolutional neural networks, target detection has made significant progress. However, existing target detection methods rely heavily on a large amount of annotated data. When annotated data becomes scarce, deep neural networks may There will be serious overfitting and failure to generalize problems. However, in reality, there are many object categories with scarce examples or data where object bounding box annotations are difficult to obtain, such as medical data. Few-shot learning aims to train the model by providing fewer examples. Most existing few-shot learning works Focus on the image classification problem, and only a few focus on the small sample target detection problem. Since object detection requires not only predicting the category but also locating the object, this makes it much more difficult than the small sample classification task. Specifically, the small sample target detection method based on self-supervised contrast constraints maximizes the difference between objects of different categories and minimizes the difference between objects of the same category when available training data is small, so that the object Category prediction and positioning achieve the best results, which is a mathematical optimization problem based on self-supervised learning.

In order to enhance the prediction and positioning effect of small sample object categories, reasonable detection methods need to be designed. Currently, methods based on two-stage fine-tuning have shown great advantages in improving small sample target detection. The two major stages based on two-stage fine-tuning are: the first stage: training base classes on large-scale data; the second stage: freezing all Base class training parameters, using small amounts of new data to fine-tune the classifier and bounding box regressor. However, this small-sample target detection method still has some problems. After the model is fine-tuned on new data, the target objects are often mislabeled as other confusing categories.

Contents of the invention

Embodiments of the present invention provide a small-sample target detection method based on self-supervised contrast constraints to achieve effective target detection of small samples.

In order to achieve the above object, the present invention adopts the following technical solutions.

A small-sample target detection method based on self-supervised contrast constraints, including:

Model the small-sample target detection problem as a mathematical optimization problem based on self-supervised learning, and build an input-oriented small-sample target detection model that is sensitive to data disturbances;

Design the optimized objective function of the small sample target detection model;

Based on the optimized objective function, the deep learning update process is used to train the small sample target detection model to obtain the trained small sample target detection model, and the trained small sample target detection model is used to perform target detection on the small sample to be detected.

Preferably, the small-sample target detection problem is modeled as a mathematical optimization problem based on self-supervised learning, and building an input-oriented small-sample target detection model that is sensitive to data disturbances includes:

(1) In the first stage, the data set D _train is constructed, which contains all training data of the basic class;

(2) In the second stage, a basic data set D _base is constructed. The category information of the data set D _base is the same as the data set D _train . The number of training data of each category is the same as the number of small sample target data set D _novel ;

(3) In the second stage, a small sample target data set D _novel is constructed, in which the category information is different from the first stage data set D _train and the second stage basic data set D _base . The number of samples of each type of training data is different from that in the second stage. The basic data set D _base is the same;

(4) Use contrast loss for feature consistency constraints, and propose a contrast loss based on the prediction distribution to constrain the prediction distribution of the sample by constructing a positive and negative sample pair, where a represents the sample pair, a _p represents the positive sample pair, a _n represents the negative sample pair, y _a represents the label of the sample pair, S, S ⁺ , S ^- represents the sample features used to construct positive and negative sample pairs, S represents the features corresponding to the benchmark sample, S ⁺ represents the sample features that are in the same category as the benchmark sample and have the largest IoU value, S ^- represents the features that are consistent with the benchmark sample Sample characteristics of different sample categories, that is, a _p = {S, S ⁺ }, a _n = {S, S ^- }.

Preferably, the optimization objective function of designing a small sample target detection model includes:

The optimization objective function for setting the small sample target detection model includes the base class training network optimization objective function L _base =L _rpn +L _cls +L _reg and the fine-tuning network optimization objective function L _{fine_tune} =L _rpn +L _cls +L _reg +L _contrastive +L _{contrastive-JS} , the fine-tuning network adds a contrastive optimization objective function based on the base training network;

1: L _rpn is the area extraction network loss function, and the calculation method is as shown in formula (1):

The loss function of the region extraction network is divided into two parts: the classification loss function L _{rpn_cls} and the bounding box regression loss function L _{rpn_reg} . L _{rpn_cls} is used for network training in which the classification anchor boxes are positive and negative samples. Its complete description is shown in formula (2), L _{rpn_reg} is used for bounding box regression network training. The complete formula description is as shown in formula (3), where N _{rpn_cls} represents the batch size of training samples in the region extraction network, and N _{rpn_reg} represents the number of anchor boxes generated by the region extraction network. Represents the true classification probability corresponding to the i-th anchor box,/>

L _{rpn_cls} uses cross entropy to calculate the loss of whether the anchor box contains the target. It is a two-class loss. p _i represents the predicted classification probability of the i-th anchor box. Represents the true classification probability corresponding to the i-th anchor box. This function is used to determine whether the extracted image area contains an object;

The general expression of is shown in formula (4), t _i ={t _x , t _y , t _w , t _h } represents the bounding box prediction regression parameter of the i-th anchor box,/> Indicates the regression parameters of the ground truth box corresponding to the i-th anchor box, t _i ,/> The calculation process is shown in formula (5) and formula (6).

t _x = (xx _anchor )/w _anchor , t _y = (yy _anchor )/h _anchor

t _w =log(w/w _anchor ), t _h =log(h/h _anchor ) (5)

x, y represent the coordinates of the center point of the predicted bounding box, w, h represent the width and height of the predicted bounding box, x _anchor , y _anchor represent the center point coordinates of the current anchor box, w _anchor , h _anchor represent the width and height of the current anchor box .

x ^* , y ^* represent the center point coordinates of the real bounding box of the object in the image, w ^* , h ^* represent the width and height of the real bounding box of the object in the image;

2: The calculation formula of the classification loss function L _cls is as follows:

Cross entropy is used as the classification loss function in the target detection network, where s _i represents the i-th detection frame, p _i represents the predicted classification probability of the i-th detection frame, Represents the true classification value of the i-th detection frame. This function provides a basis for the classification behavior of the network. Through this function, it is judged whether the network is accurate in classifying the object category in the detection area, and the model is updated by calculating the loss value for inaccurate objects;

3: The calculation formula of the bounding box regression loss function L _reg is as follows:

t _i and Respectively represent the predicted value and true value of the parameterized coordinates of the bounding box of the i-th detection box,/> is the smoothing loss, through which the position information of the detection area is further adjusted;

4: The calculation formula of the contrastive loss function L _contrastive is as follows:

Construct s, S ⁺ , S ^- sample features, construct a positive sample pair a _p = {S, S ⁺ }, a negative sample pair a _n = {S, S ^- }, D _a represents a positive sample pair a _p or a negative sample pair The Euclidean distance between a _n , y _a represents the labeling of a by the sample, That is, when the current sample pair is a positive sample pair a _p , the model will minimize the distance between the updated sample and the positive sample;/> m represents the upper bound of the distance between the sample pairs. When the distance between the sample and the negative sample is greater than m, the loss value is equal to 0 and the model is not updated; otherwise, the model will be updated until the distance between the negative sample pair reaches m;

5: The calculation formula of Contrastive-JS loss function L _{contrastive-JS} is as follows:

Among them, p _a is the predicted distribution of sample pair a, and y _a is the label of the current sample pair. p _a [i] represents the i-th predicted distribution in the sample pair, m′ represents the upper bound of the sample pair distance, which has the same meaning as m in formula (9).

Preferably, the small sample target detection model is trained using a deep learning update process based on the optimization objective function to obtain a trained small sample target detection model, including:

The small sample target detection model is trained using an optimized objective function through a two-stage deep learning model update process. The two-stage deep learning process consists of two stages: data training and small sample data fine-tuning. In the first stage, the training sample is used to train the entire detection model. The framework is trained to obtain the model parameters of the model based on basic samples; in the second stage, the model parameters of the first stage are first used to initialize the network parameters, and the parameters of the feature extraction module are fixed. Then, the small sample data set is used to evaluate the model. The parameters are fine-tuned, and in the second stage, a consistency strategy based on self-supervised learning is introduced to constrain the feature expression and distribution expression of the sample. Finally, the training of the small-sample target detection model is completed, and the trained small-sample target detection model is obtained.

Preferably, the small sample target detection model is trained using a deep learning update process based on the optimization objective function to obtain a trained small sample target detection model, including:

Step 3-1: Use the PASCAL VOC data set to generate data sets D _train , D _base and D _noval . There are 20 categories in the PASCAL VOC data set. Divide its 15 categories into basic categories and 5 new categories. Use the basic categories All instances construct D _train , and K=1, 2, 3, 5, and 10 instances are randomly sampled from the new category and the basic category as the D _base and D _noval of K-shot;

Step 3-2: Create a base training network with Faster-RCNN as the basic framework, select ResNet101 and feature pyramid as the feature extraction network, initialize model parameters, set hyperparameters, the standard batch size is 16, create a standard SGD optimizer, momentum is 0.9, and the weight decays to 1e-4;

Step 3-3: Build the D _train data loader and perform data enhancement on the original input;

Step 3-4: Train the base class training network, calculate the output value of each base class training sample, calculate the loss L _base , and use the gradient descent algorithm to update the network parameters;

Step 3-5: If the model converges or reaches the required number of training steps, end the base class training network training process and save the model parameters; otherwise, return to step 4-2;

Step 3-6: Build D _base and D _noval data loaders, create a fine-tuned network model, initialize the network using the model parameters obtained from the base class training network, and create an optimizer;

Step 3-7: Train the fine-tuning network. Based on the base class training network, obtain the candidate box feature map generated by the pooling operation of the training sample in the area of interest, traverse the candidate box feature map list, and provide each candidate box feature The image matches a candidate frame feature map of the same category as a positive example, matches a candidate frame feature map of a different category as a negative example, selects two positive examples and the current sample to form 2 positive sample pairs, selects two negative examples and the current sample Form two negative sample pairs, calculate L _contrastive for the obtained candidate box feature maps of the positive sample pair and negative sample pair; obtain the category probability distribution generated by the training sample after the classification operation, and compare the category probabilities of the positive sample pair and negative sample pair Distribution calculation L _contrastive , calculate the output value of each training sample, calculate the loss L _{fine_tune} , and use the gradient descent algorithm to update the network parameters;

Step 3-8: Use AP50 for new prediction (nAP50) as the model performance evaluation index on the PASCAL VOC 2007 test set, and observe the model convergence. If the model converges or reaches the required number of training steps, the fine-tuning network training process ends; Otherwise, return to steps 3-7.

It can be seen from the technical solutions provided by the above embodiments of the present invention that the method proposed by the present invention is based on a two-stage learning process, uses transfer learning to learn domain knowledge, and performs model fine-tuning on a small sample data set. Experimental results prove that the method proposed by the present invention has achieved good performance on the PASCAL-VOC data set public data set, can effectively improve the performance of the model on small sample target detection problems, and has strong practical application significance.

Additional aspects and advantages of the invention will be set forth in part in the description which follows, and will be obvious from the description, or may be learned by practice of the invention.

Description of the drawings

In order to explain the technical solutions of the embodiments of the present invention more clearly, the drawings needed to be used in the description of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention. Those of ordinary skill in the art can also obtain other drawings based on these drawings without exerting creative efforts.

Figure 1 is a schematic diagram of a base class training network provided by an embodiment of the present invention.

Figure 2 is a schematic diagram of a fine-tuning network provided by an embodiment of the present invention.

Detailed ways

Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements with the same or similar functions. The embodiments described below with reference to the drawings are exemplary and are only used to explain the present invention and cannot be construed as limitations of the present invention.

Those skilled in the art will understand that, unless expressly stated otherwise, the singular forms "a", "an", "the" and "the" used herein may also include the plural form. It should be further understood that the word "comprising" used in the description of the present invention refers to the presence of stated features, integers, steps, operations, elements and/or components, but does not exclude the presence or addition of one or more other features, Integers, steps, operations, elements, components and/or groups thereof. It will be understood that when we refer to an element being "connected" or "coupled" to another element, it can be directly connected or coupled to the other element or intervening elements may also be present. Additionally, "connected" or "coupled" as used herein may include wireless connections or couplings. As used herein, the term "and/or" includes any and all combinations of one or more of the associated listed items.

It will be understood by one of ordinary skill in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. It should also be understood that terms such as those defined in general dictionaries are to be understood to have meanings consistent with their meaning in the context of the prior art, and are not to be taken in an idealized or overly formal sense unless defined as herein. explain.

In order to facilitate understanding of the embodiments of the present invention, several specific embodiments will be further explained below with reference to the accompanying drawings, and each embodiment does not constitute a limitation to the embodiments of the present invention.

Self-supervised learning is a type of unsupervised learning method. Self-supervised learning mainly uses auxiliary tasks to mine its own supervisory information from large-scale unsupervised data, and trains the network by constructing supervisory information to learn a universal feature. Expressions are used for downstream tasks. The self-supervised learning method based on contrastive constraints mainly learns how to construct representations of similar and dissimilar things, so that the distance between samples and positive samples is much greater than the distance between samples and negative samples, that is, by constructing positive samples and negative samples Sample, measure the distance between positive and negative samples to achieve self-supervised learning.

A small sample target detection method based on self-supervised contrast constraints provided by the embodiment of the present invention includes the following steps:

Step S1: Based on the characteristics of the small-sample target detection problem, model the small-sample target detection problem as a mathematical optimization problem based on self-supervised learning, and build an input-oriented small-sample target detection model that is sensitive to data disturbances.

Step S2: Set the optimization objective function of the small sample target detection model.

Step S3: Use the deep learning update process to train the small sample target detection model based on the optimized objective function to obtain the trained small sample target detection model, and use the trained small sample target detection model to perform target detection on the small sample to be detected.

Specifically, the above step S1 includes:

The above small sample target detection model is specifically expressed as follows:

(1) In the first stage, the data set D _train is constructed, which contains all training data of the base class. The core of this stage is to provide initialization parameters for the second stage model training, and at the same time, the model extraction module is trained for feature extraction in the second stage. In order to obtain a better model extraction module, the adequacy of data needs to be ensured in the first stage, so D _train contains all category information of the base class and relatively complete data information;

(2) In the second stage, a basic data set D _base is constructed. The category information of the data set D _base is the same as the data set D _train . The number of training data for each category is the same as the number of small sample target data sets D _novel . The second stage is the fine-tuning stage. The main goal of this stage is to obtain a model that performs well on a small sample data set. After completing the first phase of training using D _train , in order to balance the target data set and the basic data set, it is necessary to use certain basic data for training in the second phase, namely D _base . At this time, the category information of D _base is the same as that of D _train , but the number of image data is the same as the final small sample data set. In summary, D _base and D _train come from the same source and are auxiliary data constructed to achieve good performance of the model on small sample target data;

(3) In the second stage, a small sample target data set D _novel is constructed, in which the category information is different from the first stage training set and the second stage basic data set, and the number of samples of each type of training data is different from the second stage basic data set D _base is the same. D _novel is the core data for evaluating the model on small sample target detection problems. In order to reflect the characteristics of small sample learning, the number of samples in the data set is very limited, and the number under different evaluation indicators is different;

(4) The biggest feature of the small sample problem is insufficient training data. In order to make full use of the limited data, the present invention proposes to use contrast loss to constrain feature consistency, and proposes contrast loss based on the prediction distribution to perform prediction distribution of the sample. Consistency constraints, by constructing pairs of positive and negative samples, enable the model to fully learn the consistency features of similar samples during the training process, and effectively distinguish the features between samples of different types. Among them, a represents a sample pair, a _p represents a positive sample pair, a _n represents a negative sample pair, y _a represents the label of the sample pair, S, S ⁺ , S ^- represents the sample features used to construct positive and negative sample pairs, S represents the features corresponding to the benchmark sample, S ⁺ represents the sample features that are in the same category as the benchmark sample and have the largest IoU value, S ^- represents the features that are consistent with the benchmark sample Sample characteristics of different sample categories, that is, a _p = {S, S ⁺ }, a _n = {S, S ^- }.

The specific constraint forms of the above feature consistency constraints are shown in the following formulas (9) and (10).

Specifically, the above step S2 includes:

Set the optimization objective function of the small sample target detection model. The present invention adopts a two-stage network training process. First, the base class training network is trained on a large number of base class data sets, and then fine-tuned on a balanced data set. Therefore, the optimization of the small sample target detection model set by the present invention The objective function can be divided into the base class training network optimization objective function L _base =L _rpn +L _cls +L _reg and the fine-tuning network optimization objective function L _{fine_tune} =L _rpn +L _cls +L _reg +L _contrastwe +L _{contrastwe-JS} . The goal of the fine-tuning network is to maximize the differences between objects of different categories and minimize the differences between objects of the same category when there is less available training data. Specifically, the fine-tuning network is added on the basis of the base class training network Compare and optimize the objective function.

(1)Region extraction network loss function

The function of the region extraction network is to filter out anchor frames that may have targets. Specifically, the region extraction network implements two major functions: 1) Determine whether the anchor frame is an object or background, and use NMS (non maximum suppression, non-maximum suppression) to determine whether the anchor frame is an object or a background. value suppression) method, filter out a specified number of anchor boxes, set the IoU threshold, and consider that the anchor box with IoU greater than the upper limit of the given threshold contains the target, which is a positive sample, and the anchor box with IoU less than the lower limit of the given threshold. is the background, which is a negative sample. Other anchor boxes do not participate in training; 2) Coordinate correction and regression problems, that is, finding the mapping relationship between the anchor box and the ground truth box, can be achieved through translation and scaling. When the anchor box and the ground truth box are relatively close, the prediction boundary is considered The transformation between the box and the ground truth box is a linear transformation, and a linear regression model can be used to fine-tune the bounding box parameter coordinates. After obtaining the correction parameters of each anchor box, the precise anchor box parameter coordinates can be calculated. The complete description of the region extraction network loss function is shown in formula (1).

The loss function of the region extraction network is divided into two parts: the classification loss function L _{rpn_cls} and the bounding box regression loss function L _{rpn_reg} . L _{rpn_cls} is used for network training in which the classification anchor boxes are positive and negative samples. Its complete description is shown in formula (2), L _{rpn_reg} is used for bounding box regression network training, and the complete formula description is shown in formula (3). Among them, N _{rpn_cls} represents the batch size of training samples in the region extraction network, and N _{rpn_reg} represents the number of anchor boxes generated by the region extraction network. Represents the true classification probability corresponding to the i-th anchor box,/> λ is the weight balance parameter.

L _{rpn_cls} uses cross entropy to calculate the loss of whether the anchor box contains the target. It is a two-class loss. p _i represents the predicted classification probability of the i-th anchor box. Indicates the true classification probability corresponding to the i-th anchor box. This function is used to determine whether the extracted image area contains objects. It is mainly used to distinguish whether the current area is the foreground or the background.

The purpose of L _{rpn_reg} is to adjust the position information of the nominated area in the region extraction network to guide the region extraction network to extract more accurate object positions. It uses Calculate the difference between the predicted bounding box and the real bounding box, The general expression of is shown in formula (4), t _i ={t _x , t _y , t _w , t _h } represents the bounding box prediction regression parameter of the i-th anchor box,/> Indicates the regression parameters of the ground truth box corresponding to the i-th anchor box, t _i ,/> The calculation process is shown in formula (5) and formula (6).

t _x = (xx _anchor )/w _anchor , t _y = (yy _anchor )/h _anchor

t _w =log(w/w _anchor ), t _h =log(h/h _anchor ) (5)

x, y represent the coordinates of the center point of the predicted bounding box, w, h represent the width and height of the predicted bounding box, x _anchor , y _anchor represent the center point coordinates of the current anchor box, w _anchor , h _anchor represent the width and height of the current anchor box .

x ^* , y ^* represent the center point coordinates of the real bounding box of the object in the image, w ^* , h ^* represent the width and height of the real bounding box of the object in the image.

(2)Classification loss function

Cross entropy is used as the classification loss function in the target detection network, where s _i represents the i-th detection frame, p _i represents the predicted classification probability of the i-th detection frame, Represents the true classification value of the i-th detection frame. This function provides a basis for the classification behavior of the network. Through this function, it can be judged whether the network has accurately classified the object category in the detection area, and the model can be updated by calculating the loss value for inaccurate objects.

(3) Bounding box regression loss function

The bounding box regression loss function used in the target detection network is the same as formula (3), where t _i and Represents the predicted value and true value of the parameterized coordinates of the bounding box of the i-th detection box respectively. /> is the smoothing loss. This function can further adjust the position information of the detection area.

(4)Contrast loss function

Since the detection frame can be regarded as a perturbation variant of the real target value, the contrast loss function reduces the distance between the positive sample pair a _p and expands the distance between the negative sample pair a _n by constructing the detection frame positive sample pair a _p and the negative sample pair a _n distance. By controlling the feature expression of sample pairs in the model training process, the feature expressions of objects of the same category in the model are closer, and the feature expression differences of objects of different categories are more obvious, so as to achieve the purpose of better learning the feature expression of detection frames.

Construct s, S ⁺ , S ^- sample features, construct a positive sample pair a _p = {S, S ⁺ }, a negative sample pair a _n = {S, S ^- }, D _a represents a positive sample pair a _p or a negative sample pair The Euclidean distance between a _n , y _a represents the labeling of a by the sample, That is, when the current sample pair is a positive sample pair a _p , the model will minimize the distance between the updated sample and the positive sample;/> m represents the upper bound of the distance between the sample pairs. When the distance between the sample and the negative sample is greater than m, the loss value is equal to 0 and the model is not updated; otherwise, the model will be updated until the distance between the negative sample pair reaches m.

(5)Contrastive-JS loss function

In order to expand the role of contrastive constraints, in addition to using the contrastive loss function to constrain the feature learning process, the present invention also proposes a Contrastive-JS loss function to provide guidance for the prediction distribution, thereby constraining the prediction distribution generated by the classifier. , making the model more sensitive to the category information of the object, its specific form is shown in formula (10).

Among them, p _a is the predicted distribution of sample pair a, and y _a is the label of the current sample pair. p _a [i] represents the i-th predicted distribution in the sample pair. m′ represents the upper bound of the sample pair distance, which has the same meaning as m in formula (9).

Specifically, the above step S3 includes:

Aiming at the problem of small sample target detection, the present invention constructs a two-stage deep learning model update process and uses the optimization objective function to train the small sample target detection model. The two-stage learning process consists of two stages: sufficient data training and small sample data fine-tuning. Among them, the first stage uses sufficient training samples to train the entire detection framework to obtain the model parameters of the model on the basic samples; the second stage first uses the model parameters of the first stage to initialize the parameters of the network, and sets the parameters of the feature extraction module. The parameters are fixed, and the small sample data set is used to fine-tune the model parameters. In addition, in the second stage, the consistency strategy based on self-supervised learning proposed by the present invention is introduced to constrain the characteristic expression and distribution expression of the sample, and finally completes the model training. .

The specific process is as follows:

Step 3-1: Use the PASCAL VOC data set to generate data sets D _train , D _base and D _noval . There are 20 categories in the PASCAL VOC data set, and its 15 categories are divided into basic categories and 5 new categories. Construct D _train with all instances of the basic category, and randomly sample K = 1, 2, 3, 5, 10 instances from the new category and the basic category as D _base and D _noval of K-shot. When dividing on PASCAL VOC, three different random partitioning methods are used, called Split1, Split2 and Split3.

Step 3-2: Create a base training network with Faster-RCNN as the basic framework, select ResNet101 and feature pyramid as the feature extraction network, initialize model parameters, set hyperparameters, and the standard batch size is 16. Create a standard SGD optimizer with a momentum of 0.9 and a weight decay of 1e-4.

Step 3-3: Build the D _train data loader and perform data augmentation on the original input.

Step 3-4: Train the base class training network, calculate the output value of each base class training sample, calculate the loss L _base , and use the gradient descent algorithm to update the network parameters.

Step 3-5: If the model converges or reaches the required number of training steps, end the base class training network training process and save the model parameters; otherwise, return to step 4-2.

Step 3-6: Build D _base and D _noval data loaders, create a fine-tuned network model, initialize the network using the model parameters obtained from the base class training network, and create an optimizer.

Step 3-7: Train the fine-tuning network. Based on the base class training network, obtain the candidate box feature map generated by the pooling operation of the training sample in the area of interest, traverse the candidate box feature map list, and provide each candidate box feature The image matches a candidate frame feature map of the same category as a positive example, matches a candidate frame feature map of a different category as a negative example, selects two positive examples and the current sample to form 2 positive sample pairs, selects two negative examples and the current sample Form two negative sample pairs, calculate L _contrastive for the obtained candidate box feature maps of the positive sample pair and negative sample pair; obtain the category probability distribution generated by the training sample after the classification operation, and compare the category probabilities of the positive sample pair and negative sample pair Distribution calculation L _contrastive , calculate the output value of each training sample, calculate the loss L _{fine_tune} , and use the gradient descent algorithm to update the network parameters.

Step 3-8: Use AP50 as the model performance evaluation index on the PASCAL VOC 2007 test set to evaluate the performance of the model on new categories and observe the model convergence. If the model converges or reaches the required number of training steps, it will end. Fine-tune the network training process; otherwise, return to steps 3-7.

Experimental results

The comparison results between the method designed in this invention and previous small sample target detection algorithms are shown in Table 1. As shown in the table, the present invention has achieved the highest accuracy in different base class and new class setting methods, that is, different data set segmentation methods. When the number of new class instances is 1, compared with the better small sample target The detection accuracy of the detection algorithm has increased by up to 7.0%.

Table 1 Comparative experimental results of the present invention under different data divisions

In summary, the small-sample target detection method based on self-supervised contrast constraints provided by the present invention uses self-supervised contrast constraints to enhance the small-sample target detection effect, and is different from the traditional algorithm that indirectly adjusts the region generation network and feature pyramid parameters through a fully connected layer. In contrast, the present invention directly affects feature extraction. Specifically, the present invention directly limits the parameter updates of the region generation network and feature pyramid, without introducing new parameters into the network and without adding additional calculations.

The present invention uses self-supervised contrast constraints to enhance the target detection network, which provides the research value of self-supervised learning in small sample target detection. Compared with traditional algorithms, the present invention uses contrast loss to enhance the classification and positioning capabilities of small sample targets, which can stably Improve the target detection ability of categories under different number of instances.

Those of ordinary skill in the art can understand that the accompanying drawing is only a schematic diagram of an embodiment, and the modules or processes in the accompanying drawing are not necessarily necessary for implementing the present invention.

From the above description of the embodiments, those skilled in the art can clearly understand that the present invention can be implemented by means of software plus a necessary general hardware platform. Based on this understanding, the technical solution of the present invention can be embodied in the form of a software product in essence or that contributes to the existing technology. The computer software product can be stored in a storage medium, such as ROM/RAM, disk , optical disk, etc., including a number of instructions to cause a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in various embodiments or certain parts of the embodiments of the present invention.

Each embodiment in this specification is described in a progressive manner. The same and similar parts between the various embodiments can be referred to each other. Each embodiment focuses on its differences from other embodiments. In particular, the device or system embodiments are described simply because they are basically similar to the method embodiments. For relevant details, please refer to the partial description of the method embodiments. The device and system embodiments described above are only illustrative, in which the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, It can be located in one place, or it can be distributed over multiple network elements. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Persons of ordinary skill in the art can understand and implement the method without any creative effort.

The above are only preferred specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any person familiar with the technical field can easily think of changes or modifications within the technical scope disclosed in the present invention. All substitutions are within the scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

Claims (3)

1. The small sample target detection method based on self-supervision comparison constraint is characterized by comprising the following steps of:

modeling a small sample target detection problem into a mathematical optimization problem based on self-supervision learning, and constructing a small sample target detection model which is sensitive to data disturbance and is input-oriented, wherein the method specifically comprises the following steps of:

(1) First phase construction data set D _train All training data of the base class are contained therein;

(2) Second phase building of basic dataset D _base Data set D _base Category information and data set D of (a) _train The same quantity of each type of training data as the small sample target data set D _novel The number of (3) is the same;

(3) Second stage construction of Small sample target dataset D _novel Wherein the category information is associated with the first stage data set D _train Second stage basic data set D _base Different, the number of samples of each type of training data is equal to the second stage basic data set D _base The same;

(4) Feature consistency constraint is carried out by using a contrast loss function, the consistency constraint is carried out on the prediction distribution of the sample by the contrast loss function based on the prediction distribution, and a positive and negative sample pair a= { a is constructed _p ，a _n Sum of corresponding labelsFeature set { S, S ⁺ ，S ^- Wherein a represents a sample pair, a _p Represents a positive sample pair, a _n Representing negative sample pairs, y _a A label representing a sample pair->S，S ⁺ ，S ^- Representing sample characteristics for constructing positive and negative sample pairs, S representing characteristics corresponding to a reference sample, S ⁺ Represents the sample characteristics of the same class as the reference sample and the maximum IoU value, S ^- Representing sample characteristics different from the reference sample class, i.e. a _p ＝{S，S ⁺ }，a _n ＝{S，S ^- }；

Designing an optimized objective function of a small sample objective detection model;

training the small sample target detection model by using a deep learning updating process based on an optimized target function to obtain a trained small sample target detection model, which specifically comprises the following steps: training a small sample target detection model by using an optimized objective function through a two-stage deep learning model updating process, wherein the two-stage deep learning process consists of two stages of data training and small sample data fine tuning, and the first stage uses a training sample to train the whole detection frame to obtain model parameters of the model on a basic sample; the second stage firstly uses the model parameters of the first stage to initialize the parameters of the network, fixes the parameters of the feature extraction module, then uses the small sample data set to fine tune the model parameters, introduces the consistency strategy based on self-supervision learning to restrain the feature expression and the distribution expression of the sample in the second stage, and finally completes the training of the small sample target detection model to obtain a trained small sample target detection model;

and carrying out target detection on the small sample to be detected by using the trained small sample target detection model.

2. The method of claim 1, wherein the designing the optimized objective function of the small sample objective detection model comprises:

the optimization objective function for setting the small sample target detection model comprises a basic class training network optimization objective function L _base ＝L _rpn +L _cls +L _reg Trimming network optimization objective function L _{fine_tune} ＝L _rpn +L _cls +L _reg +L _contrastive +L _{contrastive-JS} The fine tuning network increases a contrast optimization objective function on the basis of the basic class training network;

1：L _rpn for the area extraction network loss function, the calculation method is as shown in formula (1):

the loss function of the area extraction network is divided into a classification loss function L _{rpn_cls} And bounding box regression loss function L _{rpn_reg} Two parts, L _{rpn_cls} Network training for classifying positive and negative samples of anchor frames, wherein the complete description of the network training is shown in a formula (2), L _{rpn_reg} For bounding box regression network training, the complete formula description is shown as formula (3), where N _{rpn_cls} Representing the batch size, N, of training samples in a regional extraction network _{rpn_reg} Representing the number of anchor boxes generated by the area extraction network,representing the true classification probability corresponding to the ith anchor box,/->λ is the weight balance parameter:

L _{rpn_cls} using cross entropy to calculate whether the loss of object is contained in anchor frame is a two-class loss, p _i Representing the prediction classification probability of the ith anchor box,representing the true classification probability corresponding to the ith anchor frame, wherein the function is used for judging whether the extracted image area contains an object or not;

the generalized representation of (c) is shown in equation (4), t _i ＝{t _x ，t _y ，t _w ，t _h The boundary box predictive regression parameters of the ith anchor box, +.>Regression parameters, t, representing the truth box corresponding to the ith anchor box _i ，/>The calculation process is shown in a formula (5) and a formula (6);

t _x ＝(x-x _anchor )/w _anchor ，t _y ＝(y-y _anchor )/h _anchor

t _w ＝log(w/w _anchor )，t _h ＝log(h/h _anchor ) (5)

x, y represents the coordinates of the center point of the prediction boundary box, w, h represents the width and height of the prediction boundary box, and x _anchor ，y _anchor Represents the coordinates of the central point, w, of the current anchor frame _anchor ，h _anchor Representing the width and height of the current anchor frame;

x ^* ，y ^* representing the coordinates, w, of the center point of the true bounding box of an object in an image ^* ，h ^* Representing the width and height of the real bounding box of the object in the image;

2: classification loss function L _cls The calculation formula of (2) is as follows:

target inspectionCross entropy is used as a class-loss function in a test network, where s _i Represents the ith detection frame, p _i Representing the predictive classification probability of the ith detection box,the classification truth value of the ith detection frame is represented, the function provides basis for classification behavior of the network, whether the classification of the object class of the detection area is accurate or not is judged through the function, and model updating is carried out on an inaccurate object through calculation of a loss value;

3: bounding box regression loss function L _reg The calculation formula of (2) is as follows:

t _i andpredicted and actual values of the bounding box parameterized coordinates representing the ith detection box, respectively,/->Is a smooth loss, and the position information of the detection area is further adjusted through the function;

4: contrast loss function L _contrastive The calculation formula of (2) is as follows:

construction of S, S ⁺ ，S ^- Sample characteristics, constructing positive sample pair a _p ＝{S，S ⁺ Negative sample pair a _n ＝{S，S ^- }，D _a Representing positive sample pair a _p Or negative sample pair a _n Euclidean distance between, y _a Representing the labeling of a pair of samples a,i.e. the current sample pair is positive sample pair a _p When the model will update the distance between the sample and the positive sample with a minimum; />m represents the upper boundary of the sample pair distance, and when the distance between the sample and the negative sample is greater than m, the loss value is equal to 0, and the model is not updated; otherwise, updating the model until the distance between the negative sample pair reaches m;

5: contrastive-JS loss function L _{contrastive-JS} The calculation formula of (2) is as follows:

wherein p is _a Is the predictive distribution of a sample pair, y _a Is the label of the current sample pair, p _a [i]Representing the i-th prediction distribution in the sample pair,m' represents the upper bound of the sample pair distance, and is the same meaning as m in formula (9).

3. The method of claim 1, wherein training the small sample target detection model based on the optimized objective function using a deep learning update process to obtain a trained small sample target detection model comprises:

step 3-1: generating dataset D using PASCAL VOC dataset _train ，D _base D (D) _noval There are 20 categories in the pasal VOC dataset, and 15 categories are divided into a base category and 5 new categories, D is built with all instances of the base category _train Randomly sampling K=1, 2, 3, 5, 10 instances from the new class and the basic class as D of the K-shot _base And D _noval ；

Step 3-2: establishing a base class training network taking a fast-RCNN as a basic framework, selecting ResNet101 and a feature pyramid as a feature extraction network, initializing model parameters, setting super parameters, establishing a standard SGD optimizer with a standard batch size of 16, and setting the momentum of 0.9 and the weight attenuation of 1e-4;

step 3-3: construction D _train The data loader is used for carrying out data enhancement on the original input;

step 3-4: training the basic class training network, calculating the output value of each basic class training sample, and calculating the loss L _base Updating network parameters using a gradient descent algorithm;

step 3-5: if the model converges or reaches the required training step number, ending the basic class training network training process and storing the model parameters; otherwise, returning to the step 4-2;

step 3-6: construction D _base D (D) _noval The data loader creates a fine tuning network model, uses model parameters obtained by a base class training network to initialize the network, and creates an optimizer;

step 3-7: training a fine-tuning network, obtaining candidate frame feature images generated by training samples after the pooling operation of the region of interest on the basis of a basic training network, traversing a candidate frame feature image list, matching one candidate frame feature image with the same category for each candidate frame feature image as a positive example, matching one candidate frame feature image with different categories as a negative example, selecting two positive examples and a current sample to form 2 positive sample pairs, selecting two negative examples and the current sample to form 2 negative sample pairs, and calculating L for the obtained candidate frame feature images of the positive sample pairs and the negative sample pairs _contrastive The method comprises the steps of carrying out a first treatment on the surface of the Obtaining class probability distribution generated after classification operation of training sample, calculating L for class probability distribution of positive sample pair and negative sample pair _contrastive Calculating the output value of each training sample and calculating the loss L _{fine_tune} Updating network parameters using a gradient descent algorithm;

step 3-8: using the AP50 as a model performance evaluation index on the PASCAL VOC 2007 test set, observing model convergence conditions, and ending the fine tuning network training process if the model converges or reaches the required training step number; otherwise, go back to step 3-7.