CN114841257B - Small sample target detection method based on self-supervision comparison constraint - Google Patents

Abstract

The invention provides a small sample target detection method based on self-supervision contrast constraint. The method comprises the following steps: modeling a small sample target detection problem into a mathematical optimization problem based on self-supervision learning, and constructing a small sample target detection model sensitive to data disturbance; designing an optimized objective function of a small sample objective detection model; training the small sample target detection model by using a deep learning updating process based on the optimized target function to obtain a trained small sample target detection model, and carrying out target detection on the small sample to be detected by using the trained small sample target detection model. The invention is based on a two-stage learning process, uses transfer learning to learn domain knowledge, and performs model fine tuning on a small sample data set. Experimental results prove that the invention obtains good performance on the PASCAL-VOC public data set, can effectively improve the performance of the model on the small sample target detection problem, and has stronger practical application significance.

Description

Small sample target detection method based on self-supervision comparison constraint

Technical Field

The invention relates to the technical field of target detection, in particular to a small sample target detection method based on self-supervision comparison constraint.

Background

In recent years, with the development of deep convolutional neural networks, object detection has made remarkable progress, however, existing object detection methods are severely dependent on a large amount of annotated data, and when the annotated data becomes scarce, the deep neural network may have serious problems of over fitting and inability to generalize. However, in reality, there are many kinds of object classes with rare examples or data which are difficult to obtain by object bounding box annotation like medical data, and small sample learning aims at that by providing fewer example training models, most of the existing small sample learning works are focused on image classification problems, and only a few are focused on small sample target detection problems. Since target detection requires not only prediction of class, but also localization of the target, this makes it much more difficult than the task of small sample classification. Specifically, the small sample target detection method based on self-supervision comparison constraint is to maximize the difference between objects of different categories under the condition that available training data is less, minimize the difference between objects of the same category, and enable the category prediction and positioning of the objects to achieve the best effect, which is a mathematical optimization problem based on self-supervision learning.

In order to enhance the small sample object class prediction and localization effect, a reasonably designed detection method is required. The current method based on two-stage fine tuning shows great advantages in improving the detection of a small sample target, and the two stages based on the two-stage fine tuning are as follows: the first stage: training a base class on the large-scale data; and a second stage: all base class training parameters are frozen and the classifier and bounding box regressor are trimmed using a small amount of new data. However, there are still some problems with this small sample target detection method, and after the model is trimmed on the new data, the target object is often mislabeled as other confusing category.

Disclosure of Invention

The embodiment of the invention provides a small sample target detection method based on self-supervision comparison constraint, so as to effectively carry out target detection on a small sample.

In order to achieve the above purpose, the present invention adopts the following technical scheme.

A small sample target detection method based on self-supervision contrast constraint comprises the following steps:

modeling a small sample target detection problem into a mathematical optimization problem based on self-supervision learning, and constructing a small sample target detection model which is sensitive to data disturbance and is input-oriented;

designing an optimized objective function of a small sample objective detection model;

training the small sample target detection model by using a deep learning updating process based on the optimized target function to obtain a trained small sample target detection model, and carrying out target detection on the small sample to be detected by using the trained small sample target detection model.

Preferably, the modeling the small sample target detection problem as a mathematical optimization problem based on self-supervision learning, and constructing the small sample target detection model sensitive to data disturbance for input includes:

(1) First phase construction data set D _train All training data of the base class are contained therein;

(2) Second phase building of basic dataset D _base Data set D _base Category information and data set D of (a) _train The same quantity of each type of training data as the small sample target data set D _novel The number of (3) is the same;

(3) Second stage construction of Small sample target dataset D _novel Wherein the category information is associated with the first stage data set D _train Second stage basic data set D _base Different, the number of samples of each type of training data is equal to the second stage basic data set D _base The same;

(4) Feature consistency constraint is performed by using comparison loss, the prediction distribution of samples is subjected to consistency constraint by providing comparison loss based on the prediction distribution, and positive and negative sample pairs are constructedWherein a represents a sample pair, a _p Represents a positive sample pair, a _n Representing negative sample pairs, y _a A label representing a pair of samples,S，S ⁺ ，S ^- representing sample characteristics for constructing positive and negative sample pairs, S representing characteristics corresponding to a reference sample, S ⁺ Represents the sample characteristics of the same class as the reference sample and the maximum IoU value, S ^- Representing sample characteristics different from the reference sample class, i.e. a _p ＝{S，S ⁺ }，a _n ＝{S，S ^- }。

Preferably, the optimizing objective function for designing the small sample target detection model includes:

the optimization objective function for setting the small sample target detection model comprises a basic class training network optimization objective function L _base ＝L _rpn +L _cls +L _reg Trimming network optimization objective function L _{fine_tune} ＝L _rpn +L _cls +L _reg +L _contrastive +L _{contrastive-JS} The fine tuning network increases a contrast optimization objective function on the basis of the basic class training network;

1：L _rpn for the area extraction network loss function, the calculation method is as shown in formula (1):

the loss function of the area extraction network is divided into a classification loss function L _{rpn_cls} And bounding box regression loss function L _{rpn_reg} Two parts, L _{rpn_cls} Network training for classifying positive and negative samples of anchor frames, wherein the complete description of the network training is shown in a formula (2), L _{rpn_reg} For bounding box regression network training, the complete formula description is shown as formula (3), where N _{rpn_cls} Representing the batch size, N, of training samples in a regional extraction network _{rpn_reg} Representing the number of anchor boxes generated by the area extraction network,representing the true classification probability corresponding to the ith anchor box,/->

L _{rpn_cls} Using cross entropy to calculate whether the loss of object is contained in anchor frame is a two-class loss, p _i Representing the prediction classification probability of the ith anchor box,representing the true classification probability corresponding to the ith anchor frame, wherein the function is used for judging whether the extracted image area contains an object or not;

the generalized representation of (c) is shown in equation (4), t _i ＝{t _x ，t _y ，t _w ，t _h The boundary box predictive regression parameters of the ith anchor box, +.>Regression parameters, t, representing the truth box corresponding to the ith anchor box _i ，/>The calculation process is shown in the formula (5) and the formula (6).

t _x ＝(x-x _anchor )/w _anchor ，t _y ＝(y-y _anchor )/h _anchor

t _w ＝log(w/w _anchor )，t _h ＝log(h/h _anchor ) (5)

x, y represents the coordinates of the center point of the prediction boundary box, w, h represents the width and height of the prediction boundary box, and x _anchor ，y _anchor Represents the coordinates of the central point, w, of the current anchor frame _anchor ，h _anchor Representing the width and height of the current anchor frame.

x ^* ，y ^* Representing the coordinates, w, of the center point of the true bounding box of an object in an image ^* ，h ^* Representing the width and height of the real bounding box of the object in the image;

2: classification loss function L _cls The calculation formula of (2) is as follows:

cross entropy is used as a class loss function in an object detection network, where s _i Represents the ith detection frame, p _i Representing the predictive classification probability of the ith detection box,the classification truth value of the ith detection frame is represented, the function provides basis for classification behavior of the network, whether the classification of the object class of the detection area is accurate or not is judged through the function, and model updating is carried out on an inaccurate object through calculation of a loss value;

3: edge(s)Bounding box regression loss function L _reg The calculation formula of (2) is as follows:

t _i andpredicted and actual values of the bounding box parameterized coordinates representing the ith detection box, respectively,/->Is a smooth loss, and the position information of the detection area is further adjusted through the function;

4: contrast loss function L _contrastive The calculation formula of (2) is as follows:

construction of S, S ⁺ ，S ^- Sample characteristics, constructing positive sample pair a _p ＝{S，S ⁺ Negative sample pair a _n ＝{S，S ^- }，D _a Representing positive sample pair a _p Or negative sample pair a _n Euclidean distance between, y _a Representing the labeling of a pair of samples a,i.e. the current sample pair is positive sample pair a _p When the model will update the distance between the sample and the positive sample with a minimum; />m represents the upper boundary of the sample pair distance, and when the distance between the sample and the negative sample is greater than m, the loss value is equal to 0, and the model is not updated; otherwise, updating the model until the distance between the negative sample pair reaches m;

5: contrastive-JS loss function L _{contrastive-JS} The calculation formula of (2) is as follows:

wherein p is _a Is the predictive distribution of a sample pair, y _a Is the label of the current sample pair. P is p _a [i]Representing the i-th prediction distribution in the sample pair,m' represents the upper bound of the sample pair distance, and is the same meaning as m in formula (9).

Preferably, the training the small sample target detection model based on the optimized objective function by using a deep learning update process to obtain a trained small sample target detection model includes:

training a small sample target detection model by using an optimized objective function through a two-stage deep learning model updating process, wherein the two-stage deep learning process consists of two stages of data training and small sample data fine tuning, and the first stage uses a training sample to train the whole detection frame to obtain model parameters of the model on a basic sample; the second stage firstly uses the model parameters of the first stage to initialize the parameters of the network, fixes the parameters of the feature extraction module, then uses the small sample data set to fine tune the model parameters, introduces the consistency strategy based on self-supervision learning to restrain the feature expression and the distribution expression of the sample in the second stage, and finally completes the training of the small sample target detection model to obtain the trained small sample target detection model.

Preferably, the training the small sample target detection model based on the optimized objective function by using a deep learning update process to obtain a trained small sample target detection model includes:

step 3-1: generating dataset D using PASCAL VOC dataset _train ，D _base D (D) _noval The PASCAL VOC data set has 20 categories, 15 categories are divided into basic categories and 5 new categories, and all examples of the basic categories are usedConstruction D _train Randomly sampling K=1, 2, 3, 5, 10 instances from the new class and the basic class as D of the K-shot _base And D _noval ；

Step 3-2: establishing a base class training network taking a fast-RCNN as a basic framework, selecting ResNet101 and a feature pyramid as a feature extraction network, initializing model parameters, setting super parameters, establishing a standard SGD optimizer with a standard batch size of 16, and setting the momentum of 0.9 and the weight attenuation of 1e-4;

step 3-3: construction D _train The data loader is used for carrying out data enhancement on the original input;

step 3-4: training the basic class training network, calculating the output value of each basic class training sample, and calculating the loss L _base Updating network parameters using a gradient descent algorithm;

step 3-5: if the model converges or reaches the required training step number, ending the basic class training network training process and storing the model parameters; otherwise, returning to the step 4-2;

step 3-6: construction D _base D (D) _noval The data loader creates a fine tuning network model, uses model parameters obtained by a base class training network to initialize the network, and creates an optimizer;

step 3-7: training a fine-tuning network, obtaining candidate frame feature images generated by training samples after the pooling operation of the region of interest on the basis of a basic training network, traversing a candidate frame feature image list, matching one candidate frame feature image with the same category for each candidate frame feature image as a positive example, matching one candidate frame feature image with different categories as a negative example, selecting two positive examples and a current sample to form 2 positive sample pairs, selecting two negative examples and the current sample to form 2 negative sample pairs, and calculating L for the obtained candidate frame feature images of the positive sample pairs and the negative sample pairs _contrastive The method comprises the steps of carrying out a first treatment on the surface of the Obtaining class probability distribution generated after classification operation of training sample, calculating L for class probability distribution of positive sample pair and negative sample pair _contrastive Calculating the output value of each training sample and calculating the loss L _{fine_tune} Updating network parameters using a gradient descent algorithm;

step 3-8: using an AP50 for new prediction (nAP) on the PASCAL VOC 2007 test set as a model performance evaluation index, observing model convergence condition, and ending the fine tuning network training process if the model converges or reaches the required training steps; otherwise, go back to step 3-7.

According to the technical scheme provided by the embodiment of the invention, the method provided by the invention is based on a two-stage learning process, uses transfer learning to learn domain knowledge, and performs model fine tuning on a small sample data set. Experimental results prove that the method provided by the invention has good performance on the PASCAL-VOC data set disclosure data set, can effectively improve the performance of the model on the small sample target detection problem, and has stronger practical application significance.

Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

Fig. 1 is a schematic diagram of a basic training network according to an embodiment of the present invention.

Fig. 2 is a schematic diagram of a fine-tuning network according to an embodiment of the present invention.

Detailed Description

Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals refer to the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the drawings are exemplary only for explaining the present invention and are not to be construed as limiting the present invention.

As used herein, the singular forms "a", "an", "the" and "the" are intended to include the plural forms as well, unless expressly stated otherwise, as understood by those skilled in the art. It will be further understood that the terms "comprises" and/or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. It will be understood that when an element is referred to as being "connected" or "coupled" to another element, it can be directly connected or coupled to the other element or intervening elements may also be present. Further, "connected" or "coupled" as used herein may include wirelessly connected or coupled. The term "and/or" as used herein includes any and all combinations of one or more of the associated listed items.

It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the prior art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.

For the purpose of facilitating an understanding of the embodiments of the invention, reference will now be made to the drawings of several specific embodiments illustrated in the drawings and in no way should be taken to limit the embodiments of the invention.

The self-supervision learning is a kind of non-supervision learning method, and mainly utilizes auxiliary tasks to mine self-supervision information from large-scale non-supervision data, and trains a network by constructing the supervision information, so that a general feature expression is learned for downstream tasks. The self-supervision learning method based on contrast constraint mainly learns how to construct characterization for similar things and dissimilar things, so that the distance between a sample and a positive sample is far greater than the distance between the sample and a negative sample, namely the self-supervision learning is realized by constructing the positive sample and the negative sample and measuring the distance between the positive sample and the negative sample.

The small sample target detection method based on self-supervision comparison constraint provided by the embodiment of the invention comprises the following steps:

step S1: aiming at the characteristics of the small sample target detection problem, the small sample target detection problem is modeled into a mathematical optimization problem based on self-supervision learning, and a small sample target detection model sensitive to data disturbance for input is constructed.

Step S2: setting an optimized objective function of a small sample target detection model.

Step S3: training the small sample target detection model by using a deep learning updating process based on the optimized target function to obtain a trained small sample target detection model, and carrying out target detection on the small sample to be detected by using the trained small sample target detection model.

Specifically, the step S1 includes:

the small sample target detection model is specifically expressed as follows:

(1) First phase construction data set D _train All training data of the base class is contained therein. The core of the stage is to provide initialization parameters for the model training of the second stage, and simultaneously train to obtain a model extraction module for feature extraction of the second stage. To obtain a better model extraction module, the sufficiency of the data needs to be ensured in the first stage, thus D _train All category information and more complete data information of the base class are contained;

(2) Second phase building of basic dataset D _base Data set D _base Category information and data set D of (a) _train The same quantity of each type of training data as the small sample target data set D _novel The number of (3) is the same. The second stage is a fine tuning stage, the main goal of which is to get a model that performs well on small sample datasets. Using D _train After the first stage training is completed, in order to balance the target data set and the basic data set, certain basic data is required to be used for training in the second stage, namely D _base . At this time, D _base Category information of (D) _train The same number of image data as the final small sample data set. Summarizing, D _base And D _train The sources are the same, and the auxiliary data is constructed for realizing that the model performs well on the small sample target data;

(3) Second stage construction of Small sample target dataset D _novel Wherein, the category information is different from the first stage training set and the second stage basic data set, and the sample number of each category training data is different from the second stage basic data set D _base The same applies. D (D) _novel Core data of the model on the small sample target detection problem is evaluated. In order to embody the learning characteristics of small samples, the number of samples in a data set is very limited, and the number of samples under different evaluation indexes is different;

(4) The method has the characteristics that the problem of small samples is the largest, training data is insufficient, in order to fully utilize limited data, the method provides feature consistency constraint by using comparison loss, and provides consistency constraint on the prediction distribution of samples based on the comparison loss of the prediction distribution, and the model can fully learn consistency features in similar samples and effectively distinguish the features among different types of samples in the training process by constructing positive and negative sample pairs. Wherein a represents a sample pair, a _p Represents a positive sample pair, a _n Representing negative sample pairs, y _a A label representing a pair of samples,S，S ⁺ ，S ^- representing sample characteristics for constructing positive and negative sample pairs, S representing characteristics corresponding to a reference sample, S ⁺ Represents the sample characteristics of the same class as the reference sample and the maximum IoU value, S ^- Representing sample characteristics different from the reference sample class, i.e. a _p ＝{S，S ⁺ }，a _n ＝{S，S ^- }。

The specific constraint forms of the feature consistency constraints described above are shown in the following formulas (9) and (10).

Specifically, the step S2 includes:

setting an optimized objective function of a small sample target detection model. The invention adopts a two-stage network training process, firstly, a base class training network is trained on a large number of base class data sets, and then fine adjustment is carried out on one balance data set, so that the optimization objective function of the small sample target detection model set by the invention can be divided into a base class training network optimization objective function L _base ＝L _rpn +L _cls +L _reg Trimming network optimization objective function L _{fine_tune} ＝L _rpn +L _cls +L _reg +L _contrastwe +L _{contrastwe-JS} . The goal of the fine-tuning network is to maximize the differences between different classes of objects with less available training data, minimize the differences between the same class of objects, and specifically, the fine-tuning network adds a contrast-optimized objective function on the basis of the base class training network.

(1) Area extraction network loss function

The function of the area extraction network is to screen out anchor frames that may have targets, specifically, the area extraction network realizes two functions: 1) Whether the anchor frames are objects or backgrounds is judged, a designated number of anchor frames are screened out through an NMS (non maximum suppression, non-maximum suppression) method, ioU threshold values are set, the anchor frames with IoU larger than the upper limit of the given threshold value are considered to contain targets, namely positive samples, and the anchor frames with IoU smaller than the lower limit of the given threshold value are considered to be backgrounds, namely negative samples. Other anchor frames do not participate in training; 2) Coordinate correction, regression problem, namely, finding the mapping relation between the anchor frame and the truth frame, can be realized through translation and scaling, when the anchor frame and the truth frame are relatively close, the transformation between the prediction boundary frame and the truth frame is considered as linear transformation, and the parameter coordinates of the boundary frame can be finely adjusted by using a linear regression model. After the correction parameters of each anchor frame are obtained, accurate anchor frame parameter coordinates can be calculated. The complete description of the area extraction network loss function is shown in formula (1).

The loss function of the area extraction network is divided into a classification loss function L _{rpn_cls} And bounding box regression loss function L _{rpn_reg} Two parts, L _{rpn_cls} Network training for classifying positive and negative samples of anchor frames, wherein the complete description of the network training is shown in a formula (2), L _{rpn_reg} For bounding box regression network training, the complete formula description is shown in formula (3). Wherein N is _{rpn_cls} Representing the batch size, N, of training samples in a regional extraction network _{rpn_reg} Representing the number of anchor boxes generated by the area extraction network.Representing the true classification probability corresponding to the ith anchor box,/->λ is the weight balance parameter.

L _{rpn_cls} Using cross entropy to calculate whether the loss of object is contained in anchor frame is a two-class loss, p _i Representing the prediction classification probability of the ith anchor box,and representing the true classification probability corresponding to the ith anchor frame. The function is used for judging whether the extracted image area contains objects or not, and is mainly used for distinguishing whether the current area is a foreground or a background.

L _{rpn_reg} The object is to adjust the position information of the nominated region in the region extraction network to guide the region extraction network to extract more accurate object position, which usesCalculate the gap between the prediction bounding box and the real bounding box,the generalized representation of (c) is shown in equation (4), t _i ＝{t _x ，t _y ，t _w ，t _h The boundary box predictive regression parameters of the ith anchor box, +.>Regression parameters, t, representing the truth box corresponding to the ith anchor box _i ，/>The calculation process is shown in the formula (5) and the formula (6).

t _x ＝(x-x _anchor )/w _anchor ，t _y ＝(y-y _anchor )/h _anchor

t _w ＝log(w/w _anchor )，t _h ＝log(h/h _anchor ) (5)

x, y represents the coordinates of the center point of the prediction boundary box, w, h represents the width and height of the prediction boundary box, and x _anchor ，y _anchor Represents the coordinates of the central point, w, of the current anchor frame _anchor ，h _anchor Representing the width and height of the current anchor frame.

x ^* ，y ^* Representing the coordinates, w, of the center point of the true bounding box of an object in an image ^* ，h ^* Representing the width and height of the real bounding box of the object in the image.

(2) Classification loss function

Cross entropy is used as a class loss function in an object detection network, where s _i Represents the ith detection frame, p _i Representing the predictive classification probability of the ith detection box,the classification truth value of the ith detection frame is represented, the function provides basis for classification behavior of the network, whether the classification of the network to the object class of the detection area is accurate or not can be judged through the function, and model updating is carried out on an inaccurate object through calculation of a loss value.

(3) Bounding box regression loss function

The bounding box regression loss function used in the target detection network is the same as equation (3), where t _i Andrespectively representing the predicted value and the actual value of the boundary frame parameterized coordinates of the ith detection frame. />Is a smoothing loss. The position information of the detection area can be further adjusted by this function.

(4) Contrast loss function

Since the detection box can be regarded as a disturbance variant of the true target value, the contrast loss function is obtained by constructing the positive sample pair a of the detection box _p And negative sample pair a _n Narrowing the positive sample pair a _p Enlarging the distance of the negative sample pair a _n Is a distance of (3). The characteristic expression of the same class of objects in the model is more similar by controlling the characteristic expression of the sample pairs in the model training process, and the characteristic expression difference of different classes of objects is more obvious, so that the purpose of better learning the characteristic expression of the detection frame is achieved.

Construction of S, S ⁺ ，S ^- Sample characteristics, constructing positive sample pair a _p ＝{S，S ⁺ Negative sample pair a _n ＝{S，S ^- }，D _a Representing positive sample pair a _p Or negative sample pair a _n Euclidean distance between, y _a Representing the labeling of a pair of samples a,i.e. the current sample pair is positive sample pair a _p When the model will update the distance between the sample and the positive sample with a minimum; />m represents the upper boundary of the sample pair distance, and when the distance between the sample and the negative sample is greater than m, the loss value is equal to 0, and the model is not updated; otherwise the model will be updated until the distance of the negative sample pair reaches m.

(5) Contrastive-JS loss function

In order to expand the effect of contrast constraint, the invention provides a contrast-JS loss function to provide guidance for prediction distribution besides using the contrast loss function to constrain the feature learning process, so that the prediction distribution generated by a classifier is subjected to consistency constraint, the model is more sensitive to the class information of an object, and the specific form is shown in a formula (10).

Wherein p is _a Is the predictive score of a sample pair aCloth, y _a Is the label of the current sample pair. P is p _a [i]Representing the i-th prediction distribution in the sample pair.m' represents the upper bound of the sample pair distance, and is the same meaning as m in formula (9).

Specifically, the step S3 includes:

aiming at the problem of small sample target detection, the invention constructs a two-stage deep learning model updating process, and trains a small sample target detection model by using an optimized target function. The two-stage learning process consists of two stages, full data training and small sample data fine tuning. The first stage uses sufficient training samples to train the whole detection frame to obtain model parameters of the model on basic samples; in the second stage, the parameters of the network are initialized by using the model parameters in the first stage, the parameters of the feature extraction module are fixed, the model parameters are finely adjusted by using the small sample data set, in addition, the self-supervision learning-based consistency strategy provided by the invention is introduced in the second stage to restrain the feature expression and the distribution expression of the sample, and finally the model training is completed.

The specific process is as follows:

step 3-1: generating dataset D using PASCAL VOC dataset _train ，D _base D (D) _noval There are 20 categories in the pasal VOC dataset, and their 15 categories are divided into a base category and 5 new categories. Building D with all instances of the basic class _train Randomly sampling K=1, 2, 3, 5, 10 instances from the new class and the basic class as D of the K-shot _base And D _noval . When partitioning on a pasal VOC, three different random partitioning schemes are used, called Split1, split2, and Split3.

Step 3-2: a base class training network based on a Faster-RCNN basic framework is created, resNet101 and a feature pyramid are selected as feature extraction networks, model parameters are initialized, super parameters are set, and the standard batch size is 16. A standard SGD optimizer was created with a momentum of 0.9 and a weight decay of 1e-4.

Step 3-3: construction D _train And a data loader, and performs data enhancement on the original input.

Step 3-4: training the basic class training network, calculating the output value of each basic class training sample, and calculating the loss L _base Network parameters are updated using a gradient descent algorithm.

Step 3-5: if the model converges or reaches the required training step number, ending the basic class training network training process and storing the model parameters; otherwise, go back to step 4-2.

Step 3-6: construction D _base D (D) _noval And the data loader creates a fine tuning network model, initializes the network by using model parameters obtained by the base class training network, and creates an optimizer.

Step 3-7: training a fine-tuning network, obtaining candidate frame feature images generated by training samples after the pooling operation of the region of interest on the basis of a basic training network, traversing a candidate frame feature image list, matching one candidate frame feature image with the same category for each candidate frame feature image as a positive example, matching one candidate frame feature image with different categories as a negative example, selecting two positive examples and a current sample to form 2 positive sample pairs, selecting two negative examples and the current sample to form 2 negative sample pairs, and calculating L for the obtained candidate frame feature images of the positive sample pairs and the negative sample pairs _contrastive The method comprises the steps of carrying out a first treatment on the surface of the Obtaining class probability distribution generated after classification operation of training sample, calculating L for class probability distribution of positive sample pair and negative sample pair _contrastive Calculating the output value of each training sample and calculating the loss L _{fine_tune} Network parameters are updated using a gradient descent algorithm.

Step 3-8: using an AP50 as a model performance evaluation index on the PASCAL VOC 2007 test set for evaluating the performance of the model on a new category, observing the convergence condition of the model, and ending the fine tuning network training process if the model converges or reaches the required training step number; otherwise, go back to step 3-7.

Experimental results

The comparison result of the method designed by the invention and the previous small sample target detection algorithm is shown in table 1. As shown in the table, the method and the device have the highest accuracy in different base classes and new class setting modes, namely different data set segmentation modes, and the highest detection accuracy is improved by 7.0% compared with a better small sample target detection algorithm when the number of new class examples is 1.

TABLE 1 comparative experimental results of the invention under different data partitions

In summary, the method for detecting the small sample target based on the self-supervision comparison constraint provided by the invention uses the self-supervision comparison constraint to enhance the detection effect of the small sample target, and compared with the traditional algorithm for generating the network and the characteristic pyramid parameters through the fully-connected interlayer adjustment region, the method directly influences the characteristic extraction, specifically, the method directly limits the parameter update of the region generation network and the characteristic pyramid, does not introduce new parameters into the network, and does not increase additional calculation amount.

The invention uses the self-supervision contrast constraint to enhance the target detection network, provides the research value of self-supervision learning in the detection of the small sample targets, and compared with the traditional algorithm, the invention uses the contrast loss to enhance the classification and positioning capability of the small sample targets, and can stably improve the target detection capability of the classes under different numbers of examples.

Those of ordinary skill in the art will appreciate that: the drawing is a schematic diagram of one embodiment and the modules or flows in the drawing are not necessarily required to practice the invention.

From the above description of embodiments, it will be apparent to those skilled in the art that the present invention may be implemented in software plus a necessary general hardware platform. Based on such understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art in the form of a software product, which may be stored in a storage medium, such as a ROM/RAM, a magnetic disk, an optical disk, etc., including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method described in the embodiments or some parts of the embodiments of the present invention.

In this specification, each embodiment is described in a progressive manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments. In particular, for apparatus or system embodiments, since they are substantially similar to method embodiments, the description is relatively simple, with reference to the description of method embodiments in part. The apparatus and system embodiments described above are merely illustrative, wherein the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art will understand and implement the present invention without undue burden.

The present invention is not limited to the above-mentioned embodiments, and any changes or substitutions that can be easily understood by those skilled in the art within the technical scope of the present invention are intended to be included in the scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

Claims (3)

1. The small sample target detection method based on self-supervision comparison constraint is characterized by comprising the following steps of:

modeling a small sample target detection problem into a mathematical optimization problem based on self-supervision learning, and constructing a small sample target detection model which is sensitive to data disturbance and is input-oriented, wherein the method specifically comprises the following steps of:

(1) First phase construction data set D _train All training data of the base class are contained therein;

(2) Second phase building of basic dataset D _base Data set D _base Category information and data set D of (a) _train The same quantity of each type of training data as the small sample target data set D _novel The number of (3) is the same;

(3) Second stage construction of Small sample target dataset D _novel Wherein the category information is associated with the first stage data set D _train Second stage basic data set D _base Different, the number of samples of each type of training data is equal to the second stage basic data set D _base The same;

(4) Feature consistency constraint is carried out by using a contrast loss function, the consistency constraint is carried out on the prediction distribution of the sample by the contrast loss function based on the prediction distribution, and a positive and negative sample pair a= { a is constructed _p ，a _n Sum of corresponding labelsFeature set { S, S ⁺ ，S ^- Wherein a represents a sample pair, a _p Represents a positive sample pair, a _n Representing negative sample pairs, y _a A label representing a sample pair->S，S ⁺ ，S ^- Representing sample characteristics for constructing positive and negative sample pairs, S representing characteristics corresponding to a reference sample, S ⁺ Represents the sample characteristics of the same class as the reference sample and the maximum IoU value, S ^- Representing sample characteristics different from the reference sample class, i.e. a _p ＝{S，S ⁺ }，a _n ＝{S，S ^- }；

Designing an optimized objective function of a small sample objective detection model;

training the small sample target detection model by using a deep learning updating process based on an optimized target function to obtain a trained small sample target detection model, which specifically comprises the following steps: training a small sample target detection model by using an optimized objective function through a two-stage deep learning model updating process, wherein the two-stage deep learning process consists of two stages of data training and small sample data fine tuning, and the first stage uses a training sample to train the whole detection frame to obtain model parameters of the model on a basic sample; the second stage firstly uses the model parameters of the first stage to initialize the parameters of the network, fixes the parameters of the feature extraction module, then uses the small sample data set to fine tune the model parameters, introduces the consistency strategy based on self-supervision learning to restrain the feature expression and the distribution expression of the sample in the second stage, and finally completes the training of the small sample target detection model to obtain a trained small sample target detection model;

and carrying out target detection on the small sample to be detected by using the trained small sample target detection model.

2. The method of claim 1, wherein the designing the optimized objective function of the small sample objective detection model comprises:

the optimization objective function for setting the small sample target detection model comprises a basic class training network optimization objective function L _base ＝L _rpn +L _cls +L _reg Trimming network optimization objective function L _{fine_tune} ＝L _rpn +L _cls +L _reg +L _contrastive +L _{contrastive-JS} The fine tuning network increases a contrast optimization objective function on the basis of the basic class training network;

1：L _rpn for the area extraction network loss function, the calculation method is as shown in formula (1):

the loss function of the area extraction network is divided into a classification loss function L _{rpn_cls} And bounding box regression loss function L _{rpn_reg} Two parts, L _{rpn_cls} Network training for classifying positive and negative samples of anchor frames, wherein the complete description of the network training is shown in a formula (2), L _{rpn_reg} For bounding box regression network training, the complete formula description is shown as formula (3), where N _{rpn_cls} Representing the batch size, N, of training samples in a regional extraction network _{rpn_reg} Representing the number of anchor boxes generated by the area extraction network,representing the true classification probability corresponding to the ith anchor box,/->λ is the weight balance parameter:

L _{rpn_cls} using cross entropy to calculate whether the loss of object is contained in anchor frame is a two-class loss, p _i Representing the prediction classification probability of the ith anchor box,representing the true classification probability corresponding to the ith anchor frame, wherein the function is used for judging whether the extracted image area contains an object or not;

the generalized representation of (c) is shown in equation (4), t _i ＝{t _x ，t _y ，t _w ，t _h The boundary box predictive regression parameters of the ith anchor box, +.>Regression parameters, t, representing the truth box corresponding to the ith anchor box _i ，/>The calculation process is shown in a formula (5) and a formula (6);

t _x ＝(x-x _anchor )/w _anchor ，t _y ＝(y-y _anchor )/h _anchor

t _w ＝log(w/w _anchor )，t _h ＝log(h/h _anchor ) (5)

x, y represents the coordinates of the center point of the prediction boundary box, w, h represents the width and height of the prediction boundary box, and x _anchor ，y _anchor Represents the coordinates of the central point, w, of the current anchor frame _anchor ，h _anchor Representing the width and height of the current anchor frame;

x ^* ，y ^* representing the coordinates, w, of the center point of the true bounding box of an object in an image ^* ，h ^* Representing the width and height of the real bounding box of the object in the image;

2: classification loss function L _cls The calculation formula of (2) is as follows:

target inspectionCross entropy is used as a class-loss function in a test network, where s _i Represents the ith detection frame, p _i Representing the predictive classification probability of the ith detection box,the classification truth value of the ith detection frame is represented, the function provides basis for classification behavior of the network, whether the classification of the object class of the detection area is accurate or not is judged through the function, and model updating is carried out on an inaccurate object through calculation of a loss value;

3: bounding box regression loss function L _reg The calculation formula of (2) is as follows:

t _i andpredicted and actual values of the bounding box parameterized coordinates representing the ith detection box, respectively,/->Is a smooth loss, and the position information of the detection area is further adjusted through the function;

4: contrast loss function L _contrastive The calculation formula of (2) is as follows:

construction of S, S ⁺ ，S ^- Sample characteristics, constructing positive sample pair a _p ＝{S，S ⁺ Negative sample pair a _n ＝{S，S ^- }，D _a Representing positive sample pair a _p Or negative sample pair a _n Euclidean distance between, y _a Representing the labeling of a pair of samples a,i.e. the current sample pair is positive sample pair a _p When the model will update the distance between the sample and the positive sample with a minimum; />m represents the upper boundary of the sample pair distance, and when the distance between the sample and the negative sample is greater than m, the loss value is equal to 0, and the model is not updated; otherwise, updating the model until the distance between the negative sample pair reaches m;

5: contrastive-JS loss function L _{contrastive-JS} The calculation formula of (2) is as follows:

wherein p is _a Is the predictive distribution of a sample pair, y _a Is the label of the current sample pair, p _a [i]Representing the i-th prediction distribution in the sample pair,m' represents the upper bound of the sample pair distance, and is the same meaning as m in formula (9).

3. The method of claim 1, wherein training the small sample target detection model based on the optimized objective function using a deep learning update process to obtain a trained small sample target detection model comprises:

step 3-1: generating dataset D using PASCAL VOC dataset _train ，D _base D (D) _noval There are 20 categories in the pasal VOC dataset, and 15 categories are divided into a base category and 5 new categories, D is built with all instances of the base category _train Randomly sampling K=1, 2, 3, 5, 10 instances from the new class and the basic class as D of the K-shot _base And D _noval ；

Step 3-2: establishing a base class training network taking a fast-RCNN as a basic framework, selecting ResNet101 and a feature pyramid as a feature extraction network, initializing model parameters, setting super parameters, establishing a standard SGD optimizer with a standard batch size of 16, and setting the momentum of 0.9 and the weight attenuation of 1e-4;

step 3-3: construction D _train The data loader is used for carrying out data enhancement on the original input;

step 3-4: training the basic class training network, calculating the output value of each basic class training sample, and calculating the loss L _base Updating network parameters using a gradient descent algorithm;

step 3-5: if the model converges or reaches the required training step number, ending the basic class training network training process and storing the model parameters; otherwise, returning to the step 4-2;

step 3-6: construction D _base D (D) _noval The data loader creates a fine tuning network model, uses model parameters obtained by a base class training network to initialize the network, and creates an optimizer;

step 3-7: training a fine-tuning network, obtaining candidate frame feature images generated by training samples after the pooling operation of the region of interest on the basis of a basic training network, traversing a candidate frame feature image list, matching one candidate frame feature image with the same category for each candidate frame feature image as a positive example, matching one candidate frame feature image with different categories as a negative example, selecting two positive examples and a current sample to form 2 positive sample pairs, selecting two negative examples and the current sample to form 2 negative sample pairs, and calculating L for the obtained candidate frame feature images of the positive sample pairs and the negative sample pairs _contrastive The method comprises the steps of carrying out a first treatment on the surface of the Obtaining class probability distribution generated after classification operation of training sample, calculating L for class probability distribution of positive sample pair and negative sample pair _contrastive Calculating the output value of each training sample and calculating the loss L _{fine_tune} Updating network parameters using a gradient descent algorithm;

step 3-8: using the AP50 as a model performance evaluation index on the PASCAL VOC 2007 test set, observing model convergence conditions, and ending the fine tuning network training process if the model converges or reaches the required training step number; otherwise, go back to step 3-7.