CN112347957A - Pedestrian re-identification method and device, computer equipment and storage medium - Google Patents
Pedestrian re-identification method and device, computer equipment and storage medium Download PDFInfo
- Publication number
- CN112347957A CN112347957A CN202011264154.3A CN202011264154A CN112347957A CN 112347957 A CN112347957 A CN 112347957A CN 202011264154 A CN202011264154 A CN 202011264154A CN 112347957 A CN112347957 A CN 112347957A
- Authority
- CN
- China
- Prior art keywords
- feature
- pedestrian
- features
- library
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 62
- 238000000605 extraction Methods 0.000 claims abstract description 72
- 238000012549 training Methods 0.000 claims abstract description 33
- 238000013528 artificial neural network Methods 0.000 claims abstract description 11
- 230000006870 function Effects 0.000 claims description 57
- 239000013598 vector Substances 0.000 claims description 23
- 230000008569 process Effects 0.000 claims description 19
- 239000011159 matrix material Substances 0.000 claims description 9
- 238000004590 computer program Methods 0.000 claims description 8
- 238000004422 calculation algorithm Methods 0.000 claims description 7
- 230000003044 adaptive effect Effects 0.000 claims description 5
- 238000010276 construction Methods 0.000 claims description 5
- 238000013507 mapping Methods 0.000 claims description 4
- 239000002131 composite material Substances 0.000 claims description 2
- 238000005457 optimization Methods 0.000 description 6
- 238000012545 processing Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 5
- 230000006978 adaptation Effects 0.000 description 3
- 239000000463 material Substances 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 125000004122 cyclic group Chemical group 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000001815 facial effect Effects 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 230000001105 regulatory effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/103—Static body considered as a whole, e.g. static pedestrian or occupant recognition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Biomedical Technology (AREA)
- Multimedia (AREA)
- Health & Medical Sciences (AREA)
- Human Computer Interaction (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a pedestrian re-identification method, which comprises the following steps: acquiring reference images of a plurality of reference pedestrians, and extracting corresponding reference features from the reference images by using a feature extraction model; wherein the feature extraction model is obtained by utilizing neural network training; storing the reference features into a pedestrian feature library, and updating the reference features stored in the pedestrian feature library based on a preset frequency; acquiring an image to be recognized containing a pedestrian to be recognized, and extracting corresponding features to be recognized from the image to be recognized by using the feature extraction model; comparing the feature to be recognized with candidate features stored in the pedestrian feature library to determine a target feature which is different from the image to be recognized by less than a first threshold value from the candidate features; and identifying the pedestrian to be identified according to the category label corresponding to the target feature.
Description
Technical Field
The present invention relates to the field of image recognition technologies, and in particular, to a pedestrian re-recognition method, apparatus, computer device, and storage medium.
Background
The pedestrian re-identification is essentially the identification of the identity of the pedestrian to be identified, and the accuracy of the algorithm identification directly influences the final performance of the whole system. In the pedestrian re-identification method at the present stage, generally, images of pedestrians to be identified are collected in advance to establish a galery (pedestrian feature library), and the identity of each pedestrian in the galery is marked in advance. In the identification process, firstly, the characteristic F of the pedestrian to be identified is extracted, then the characteristic F is compared with each pedestrian characteristic in the galery, and the label of the pedestrian closest to the characteristic F (the confidence coefficient meets the set threshold condition) in the galery is the identification result of the system. The disadvantage of this method is that, since the appearance characteristics such as the clothing style of the pedestrian are changed continuously with time, the stored appearance characteristics of the corresponding pedestrian in the galery may have a large difference from the actual appearance characteristics of the pedestrian to be identified, which may result in inaccurate characteristic comparison and significant reduction in pedestrian re-identification capability.
Disclosure of Invention
The present invention is directed to provide a technical solution capable of accurately and reliably re-identifying a pedestrian, so as to solve the above-mentioned problems in the prior art.
In order to achieve the above object, the present invention provides a pedestrian re-identification method, comprising the steps of:
acquiring reference images of a plurality of reference pedestrians, and extracting corresponding reference features from the reference images by using a feature extraction model; wherein the feature extraction model is obtained by utilizing neural network training;
storing the reference features into a pedestrian feature library, and updating the reference features stored in the pedestrian feature library based on a preset frequency;
acquiring an image to be recognized containing a pedestrian to be recognized, and extracting corresponding features to be recognized from the image to be recognized by using the feature extraction model;
comparing the feature to be recognized with candidate features stored in the pedestrian feature library to determine a target feature which is different from the image to be recognized by less than a first threshold value from the candidate features;
and identifying the pedestrian to be identified according to the category label corresponding to the target feature.
According to the pedestrian re-identification method provided by the invention, the step of updating the reference features stored in the pedestrian feature library based on the preset frequency comprises the following steps:
acquiring an updated image containing a reference pedestrian;
extracting corresponding updated features from the updated image by using the feature extraction model;
judging whether reference features are stored in the pedestrian feature library or not;
when the pedestrian feature library does not store reference features, storing the updated features into the pedestrian feature library as new reference features, and adding new category labels to the updated features;
calculating a first distance between the updated feature and one of the stored reference features in the case that the reference features are already stored in the pedestrian feature library;
judging whether the first distance is smaller than a first threshold value;
if the first distance is smaller than a first threshold value, storing the updated feature into the pedestrian feature library as a new reference feature, and adding an existing category label which is the same as one of the reference features to the updated feature;
and if the first distance is not smaller than a first threshold value, storing the updated feature into the pedestrian feature library as a new reference feature, and adding a new category label for the updated feature.
According to the pedestrian re-identification method provided by the invention, the training process of the feature extraction model comprises the following steps:
constructing a first loss function based on an improved softmax function, constructing a second loss function based on a triplet function, and constructing a third loss function based on an Angular softmax loss;
and determining a comprehensive loss function of the feature extraction model according to the first loss function, the second loss function and the third loss function.
The pedestrian re-identification method provided by the invention is characterized in that the first loss function L1 is as follows:
in the above formula, yiRepresenting a category label corresponding to the feature to be identified; c represents a category label corresponding to the reference feature in the feature library; i (y)iC) stands for yiIf c is true, 1 is returned, otherwise, 0 is returned; w represents a parameter matrix of a full connection layer contained in the feature extraction model; b is a parameter to be optimized;
the second loss function L2 is:
in the formula, a represents any reference image in the training set, p represents a reference image with the same type label as a, and n represents a reference image with a different type label from a; d (a, p) represents the Euclidean distance between a and p, d (a, n) represents the Euclidean distance between a and n, m is a parameter to be optimized, [ d (a, p) -d (a, n) + m,0] represents taking the larger value of d (a, p) -d (a, n) + m and 0;
the third loss function L3 is:
in the above formula, i and j represent different class labels;
the composite loss function L is:
L＝αL1+βL2+γL3
wherein α, β, γ represent weight coefficients, respectively.
According to the pedestrian re-identification method provided by the invention, the training process of the feature extraction model further comprises the following steps:
training the feature extraction model based on a domain adaptive algorithm to adapt the feature extraction model to the span from a life scene to a construction scene.
According to the pedestrian re-identification method provided by the invention, before the training of the feature extraction model, the method further comprises the following steps:
acquiring a training image sample;
performing data enhancement processing on the training image samples to increase the number of samples; the data enhancement processing comprises any one or more of turning, rotating, scaling, random clipping, shifting and noise adding.
According to the pedestrian re-identification method provided by the invention, the step of identifying the pedestrian to be identified according to the class label corresponding to the target feature comprises the following steps:
searching a pedestrian identity corresponding to the category label according to a preset mapping relation;
and identifying the pedestrian to be identified based on the pedestrian identity.
In order to achieve the above object, the present invention also provides a pedestrian re-identification apparatus, comprising:
the system comprises a reference feature module, a feature extraction module and a feature extraction module, wherein the reference feature module is suitable for acquiring reference images of a plurality of reference pedestrians and extracting corresponding reference features from the reference images by using a feature extraction model; wherein the feature extraction model is obtained by utilizing neural network training;
the characteristic library module is suitable for storing the reference characteristics to a pedestrian characteristic library and updating the reference characteristics stored in the pedestrian characteristic library based on a preset frequency;
the characteristic extraction module is suitable for acquiring an image to be identified containing a pedestrian to be identified, and extracting corresponding characteristics to be identified from the image to be identified by using the characteristic extraction model;
the comparison module is used for comparing the feature to be recognized with candidate features stored in the pedestrian feature library so as to determine a target feature which is different from the image to be recognized by less than a first threshold value from the candidate features;
and the identification module is suitable for identifying the pedestrian to be identified according to the category label corresponding to the target feature.
To achieve the above object, the present invention further provides a computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the steps of the above method when executing the computer program.
To achieve the above object, the present invention also provides a computer-readable storage medium having stored thereon a computer program which, when being executed by a processor, carries out the steps of the above method.
The pedestrian re-identification method, the pedestrian re-identification device, the computer equipment and the computer readable storage medium provided by the invention can be used for regularly updating the pedestrian feature library based on the preset frequency, so that the matching degree between the features stored in the feature library and the actual features of pedestrians can be improved, and the accuracy of pedestrian re-identification is improved. Furthermore, the feature extraction model of the invention utilizes a softmax function, a triplet function and an Angular softmax function to jointly construct a loss function for parameter optimization, and can significantly improve the accuracy of pedestrian feature extraction. Aiming at the problems of small scene difference, small training data amount and the like, the invention adopts a public data set, self-owned data, data enhancement and Domain Adaptation (field self-Adaptation) method to carry out iterative optimization on a feature extraction model, thereby greatly improving the accuracy and cross-Domain generalization performance of a pedestrian re-identification model.
Drawings
FIG. 1 is a flowchart of a pedestrian re-identification method according to a first embodiment of the present invention;
FIG. 2 is a schematic diagram illustrating a logic algorithm in a pedestrian re-identification method according to a first embodiment of the present invention;
FIG. 3 is a schematic flow chart illustrating updating of reference features stored in the pedestrian feature library according to a first embodiment of the present invention;
FIG. 4 is a schematic diagram illustrating a training process of a feature extraction model according to a first embodiment of the present invention;
fig. 5 is a schematic diagram showing program modules of a pedestrian re-identification apparatus according to a first embodiment of the present invention;
fig. 6 is a schematic diagram illustrating a hardware structure of a pedestrian re-identification apparatus according to a first embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Example one
Referring to fig. 1, the present embodiment provides a pedestrian re-identification method, including the following steps:
s100, acquiring reference images of a plurality of reference pedestrians, and extracting corresponding reference features from the reference images by using a feature extraction model; wherein the feature extraction model is obtained by utilizing neural network training.
The reference pedestrian may be a candidate to be identified, such as all workers in an office building, all people entering and exiting a construction site, and the like. The reference image may be obtained from a pre-taken photograph or a recorded video. The feature extraction model in this embodiment may be trained using a neural network based on a machine learning algorithm. The specific neural network may include any existing neural network, such as a convolutional neural network, a cyclic neural network, a boltzmann machine network, and the like, which is not limited in this embodiment. The trained feature extraction model may output an N-dimensional feature vector corresponding to the reference pedestrian with respect to the input image including the reference pedestrian. The N-dimensional feature vector generally reflects skin color features, posture features, facial features, and the like related to the reference pedestrian, and in short, the image features of the reference pedestrian can be accurately described through the N-dimensional feature vector.
And S200, storing the reference characteristics into a pedestrian characteristic library, and updating the reference characteristics stored in the pedestrian characteristic library based on a preset frequency.
The pedestrian feature library is used for storing a large number of feature vector samples to serve as a comparison basis for pedestrian re-identification. It is understood that the larger the amount of samples stored in the pedestrian feature library is, the higher the pedestrian recognition capability is, and the amount of calculation is increased accordingly. Therefore, the number of feature vector samples stored in the pedestrian feature library needs to be controlled within a reasonable range.
It can be understood that the appearance features of the clothing and the like of the reference pedestrian at different time points are different, and therefore, the reference features stored in the pedestrian feature library are updated regularly in the embodiment, so that a plurality of different feature vectors of the same reference pedestrian are covered as much as possible. In the updating of the present embodiment, a new reference image corresponding to the reference pedestrian is periodically acquired, and after feature recognition is performed based on the new reference image, the new feature vector is stored according to the category label, so as to provide the latest comparison feature vector for pedestrian re-recognition. The category labels represent different pedestrians to which one feature vector belongs, and usually have the same category label based on a plurality of feature vectors obtained by the same reference pedestrian. For example, for the same reference pedestrian A, outputting a first reference feature through a feature extraction model based on a first reference image acquired at a first moment; and outputting a second reference feature through the feature extraction model based on a second reference image acquired at a second moment. In this way, a category label a may be added to both the first reference feature and the second reference feature, indicating that the first reference feature and the second reference feature belong to the same pedestrian a. The specific form of the category label may include numbers, letters, symbols, or a combination thereof, which is not limited by the embodiment.
S300, acquiring an image to be recognized containing a pedestrian to be recognized, and extracting corresponding features to be recognized from the image to be recognized by using the feature extraction model.
When a pedestrian in a certain area needs to be identified to confirm the identity of the pedestrian, an image to be identified of the pedestrian can be shot by a video camera, a camera and the like, the image to be identified is input into the feature extraction model to output the corresponding feature to be identified, and the feature to be identified can be an N-dimensional vector.
And S400, comparing the feature to be recognized with candidate features stored in the pedestrian feature library to determine a target feature which is different from the image to be recognized by less than a first threshold value from the candidate features.
Suppose that the feature to be recognized obtained by the feature extraction model is fqWherein f isqIs an array of 1 XN. In this step, f is required to beqAnd comparing the average features under all category labels stored in the pedestrian feature library, wherein the features under all category labels stored in the feature library can be represented by H, H is a matrix of N × M, and M is the total number of all category labels in the pedestrian feature library. The alignment can be expressed as dq: dq＝fq×H。
It can be understood that dqIs a 1 × M array, each element in the 1 × M array represents fqDistance from each average feature. Calculating dqMedian minimum value DminIf D isminLess than or equal to a preset first threshold value T, then DminThe class label corresponding to the corresponding feature is fqOtherwise f denotesqThe corresponding pedestrian is not in the galery, and the recognition result is empty.
It should be noted that each vector in the matrix H may be an average feature vector under a category label. As mentioned above, during the updating of the pedestrian feature library, a plurality of feature vectors may be stored under the same category label. Therefore, in this step, all feature vectors under the same class label are averaged to obtain an average feature vector, which is used as one of the vectors in the matrix H.
And S500, identifying the pedestrian to be identified according to the category label corresponding to the target feature.
The class labels can be used for distinguishing different pedestrians, and different class labels are added to feature vectors belonging to different pedestrians. On the basis, the mapping relation between the pedestrian identity identifications corresponding to the category labels can be established, the corresponding mapping relation is searched according to different category labels, and the identity of the pedestrian is finally determined.
Fig. 2 shows a schematic diagram of a logic algorithm in the pedestrian re-identification method according to the first embodiment of the invention. As shown in fig. 2, the pedestrian identification method of the embodiment may be divided into three processes, which respectively include an automatic library building process, a feature extraction process, and an identification process. In the automatic library building process, extracting characteristics of the obtained video frames or images and storing the characteristics such as a pedestrian characteristic library; in the characteristic extraction process, the pedestrian characteristics are extracted from the input image by using a trained characteristic extraction model; in the identification process, the characteristics of the pedestrian to be identified are compared with the characteristics stored in the pedestrian characteristic library, and the identity of the pedestrian is finally confirmed.
Fig. 3 shows a schematic flowchart of updating the reference feature stored in the pedestrian feature library according to a first embodiment of the present invention. As illustrated in fig. 3, step 200 includes:
and S210, acquiring an updated image containing the reference pedestrian. The updated image is a new image including the reference pedestrian, which is different from the reference image. The acquisition date of the updated image is generally later than that of the reference image, and therefore, the similarity is greater closer to the current time than that of the reference image. The updated image may be derived directly from a photograph or from a video frame. When the image is derived from a video frame, it is necessary to respectively screen each frame of image in the video frame, delete the background image not containing any pedestrian, and keep the image containing the pedestrian as the updated image.
And S220, extracting corresponding updated features from the updated image by using the feature extraction model. Specifically, the updated image is input into the feature extraction model to obtain an output updated feature. The update feature may be an N-dimensional vector.
And S230, judging whether the reference characteristics are stored in the pedestrian characteristic library.
This step is used for judging whether the present pedestrian characteristic storehouse is empty. The condition that the feature library is empty may be based on various reasons, such as that the library is first built or the original features stored in the library are formatted, etc., which are not listed in this embodiment. If the current pedestrian feature library does not store any reference feature data, the current pedestrian feature library is empty, and if not, the current pedestrian feature library is not empty.
And S240, storing the updated feature into the pedestrian feature library as a new reference feature and adding a new category label to the updated feature when the reference feature is not stored in the pedestrian feature library.
If the current pedestrian feature library is empty, there is no reference feature available for comparison, and naturally the category label corresponding to the updated feature obtained in step S220 will not be included. In this case, this step may store the updated features in the pedestrian feature library and add a category label to the updated features. The category label may be in the form of any one of a number, letter, symbol, or combination thereof. When the category labels are in the form of numbers, there may be an increasing relationship of numbers between different category labels, for example, each succeeding category label is increased by 1 over the preceding category label.
And S250, under the condition that the reference features are stored in the pedestrian feature library, calculating a first distance between the updated feature and one of the stored reference features.
If the reference feature is stored in the pedestrian feature library, the class tag corresponding to the reference feature is also stored, so that it is necessary to determine whether the updated feature and the stored reference feature belong to the same class tag in this step. The determination is based on a first distance, e.g., a euclidean distance, between the updated feature and the reference feature.
S260, judging whether the first distance is smaller than a first threshold value;
and S270, if the first distance is smaller than a first threshold value, storing the updated feature into the pedestrian feature library as a new reference feature, and adding an existing class label which is the same as the one reference feature to the updated feature.
A first threshold may be set for the first distance, and when the first distance is smaller than the first threshold, it indicates that the similarity between the updated feature and the reference feature is large, and the updated feature and the reference feature may belong to the same category label. Otherwise, it is indicated that the similarity between the updated feature and the reference feature is small, and a new category label needs to be established for the updated feature.
And S280, if the first distance is not smaller than a first threshold value, storing the updated feature into the pedestrian feature library as a new reference feature, and adding a new category label for the updated feature.
The new category labels may be numerically accumulated based on the category labels already stored in the pedestrian feature library. For example, the maximum value of the stored class labels in the pedestrian feature library is M, then the new class label may be M + 1. Of course, if the form of the category label is not a number but another form, the new category label may be generated based on another rule, which is not limited in this embodiment.
Through the steps, the updating of the pedestrian feature library can be rapidly realized, and more reference information is provided for feature comparison on the basis, so that the accuracy of pedestrian re-identification is improved.
In order to improve the accuracy of the feature extraction model, the embodiment trains and optimizes the feature extraction model through the synthetic loss function. As shown in fig. 4, the training process of the feature extraction model in this embodiment includes the following steps:
and S410, constructing a first loss function based on the improved softmax function, constructing a second loss function based on the triplet function, and constructing a third loss function based on the Angular softmax function.
In order to improve the recognition degree of the feature extraction model, so that the distance between the features of the same type is as close as possible and the distance between the features of different types is as far as possible, the embodiment optimizes the model parameters by simultaneously adopting softmax-loss and triplet-loss, and simultaneously takes the interval between the class centers as an auxiliary optimization target by taking the regularization thought in regulated face as reference, thereby achieving the purposes of small distance in the class and moderate distance outside the class. Wherein softmax loss is the first loss function in this embodiment, tripletloss is the second loss function in this embodiment, and Angular softmax loss is the third loss function in this embodiment.
Specifically, if the last fully connected layer of the feature extraction model is f (fullyconnect layer) and the corresponding parameter is W, then generally W is a C × N matrix, C is the number of classes included in the training set, and N is the dimension of the feature. In the training stage, when the training set picture I is subjected to network to obtain the characteristic fiThen, the first loss function L1 can be obtained by full concatenation and softmax-loss, as shown in the following formula:
in the above formula, yiRepresenting a category label corresponding to the feature to be identified; c represents a category label corresponding to the reference feature in the feature library; i (y)iC) stands for yiIf c is true, 1 is returned, otherwise, 0 is returned; w represents a parameter matrix of a full connection layer contained in the feature extraction model; and b is the parameter to be optimized.
Further, in the embodiment, m triples [ a, p, n ] are constructed by using the training set, a is any graph in the training set, p represents a graph of the same category as a, n represents a graph of a different category from a, and the value of m is determined according to the training requirement. The loss of the triplet in training calculated by the triplet-loss is the second loss function L2, as follows:
in the formula, a represents any reference image in the training set, p represents a reference image with the same type label as a, and n represents a reference image with a different type label from a; d (a, p) represents the Euclidean distance between a and p, d (a, n) represents the Euclidean distance between a and n, m is the parameter to be optimized, [ d (a, p) -d (a, n) + m,0] represents taking the larger value of d (a, p) -d (a, n) + m and 0.
Further, to increase the feature distances of the different classes, the spacing between class centers is taken as a third loss function L3, as follows:
in the above formula i and j represent different class labels.
And S420, determining a comprehensive loss function of the feature extraction model according to the first loss function, the second loss function and the third loss function.
In this step, the first loss function L1, the second loss function L2, and the third loss function L3 jointly constrain the optimization of the feature extraction model parameters, and a specific optimization target is a comprehensive loss function L, as follows:
L＝αL1+βL2+γL3
wherein α, β, γ represent weight coefficients, respectively.
Through the steps, the recognition degree of the feature extraction model to different features can be improved, and the features output by the model are more accurate.
In one example, the feature extraction model of the present invention is applied in a construction scenario. It can be understood that a large number of sample images are needed in the model training process, most of the sample images are from life scenes, and certain differences exist between the sample images and construction scenes of practical application. In order to improve the cross-Domain generalization capability of the feature extraction model, the embodiment adopts a Domain Adaptation (Domain Adaptation) algorithm to perform iterative optimization on the model in the training process of the feature extraction model, so as to ensure that the model still has higher accuracy when being applied to different scenes. The specific implementation method may include an existing sample adaptive method, a feature level adaptive method, a model level adaptive method, and the like, which is not limited in the present invention.
Further, in order to increase the number of samples and overcome the influence of factors such as scale, illumination, and inversion on the model result, the present embodiment further includes a step of performing data enhancement processing on the acquired training image samples before training the feature extraction model. The specific data enhancement processing means comprises any one or more of turning, rotating, scaling, random clipping, shifting and noise adding. Therefore, the identification accuracy of the feature extraction model can be further improved.
With continued reference to fig. 5, a pedestrian re-identification apparatus is shown, in the embodiment, the pedestrian re-identification apparatus 50 may include or be divided into one or more program modules, and the one or more program modules are stored in a storage medium and executed by one or more processors to implement the present invention and implement the above-mentioned pedestrian re-identification method. The program module referred to in the present invention means a series of computer program instruction segments capable of performing a specific function, and is more suitable than the program itself for describing the execution process of the pedestrian re-identification apparatus 50 in the storage medium. The following description will specifically describe the functions of the program modules of the present embodiment:
a reference feature module 51, adapted to obtain reference images of a plurality of reference pedestrians, and extract corresponding reference features from the reference images by using a feature extraction model; wherein the feature extraction model is obtained by utilizing neural network training;
the feature library module 52 is adapted to store the reference features into a pedestrian feature library, and update the reference features stored in the pedestrian feature library based on a preset frequency;
the feature extraction module 53 is adapted to obtain an image to be identified including a pedestrian to be identified, and extract a corresponding feature to be identified from the image to be identified by using the feature extraction model;
a comparison module 54, adapted to compare the feature to be recognized with candidate features stored in the pedestrian feature library, so as to determine, from the candidate features, a target feature that differs from the image to be recognized by less than a first threshold;
and the identification module 55 is adapted to identify the pedestrian to be identified according to the category label corresponding to the target feature.
The embodiment also provides a computer device, such as a smart phone, a tablet computer, a notebook computer, a desktop computer, a rack server, a blade server, a tower server or a rack server (including an independent server or a server cluster composed of a plurality of servers) capable of executing programs, and the like. The computer device 60 of the present embodiment includes at least, but is not limited to: a memory 61, a processor 62, which may be communicatively coupled to each other via a system bus, as shown in FIG. 6. It is noted that fig. 6 only shows a computer device 60 with components 61-62, but it is to be understood that not all shown components are required to be implemented, and that more or fewer components may be implemented instead.
In the present embodiment, the memory 61 (i.e., a readable storage medium) includes a flash memory, a hard disk, a multimedia card, a card-type memory (e.g., SD or DX memory, etc.), a Random Access Memory (RAM), a Static Random Access Memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, an optical disk, and the like. In some embodiments, the storage 61 may be an internal storage unit of the computer device 60, such as a hard disk or a memory of the computer device 60. In other embodiments, the memory 61 may also be an external storage device of the computer device 20, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), etc. provided on the computer device 60. Of course, the memory 61 may also include both internal and external storage devices of the computer device 60. In this embodiment, the memory 61 is generally used for storing an operating system and various application software installed in the computer device 60, such as the program code of the pedestrian re-identification apparatus 60 in the first embodiment. Further, the memory 61 may also be used to temporarily store various types of data that have been output or are to be output.
Processor 62 may be a Central Processing Unit (CPU), controller, microcontroller, microprocessor, or other data Processing chip in some embodiments. The processor 62 is typically used to control the overall operation of the computer device 60. In this embodiment, the processor 62 is configured to operate the program codes stored in the memory 61 or process data, for example, operate the pedestrian re-identification device 50, so as to implement the pedestrian re-identification method according to the first embodiment.
The present embodiment also provides a computer-readable storage medium, such as a flash memory, a hard disk, a multimedia card, a card-type memory (e.g., SD or DX memory, etc.), a Random Access Memory (RAM), a Static Random Access Memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, an optical disk, a server, an App application mall, etc., on which a computer program is stored, which when executed by a processor implements corresponding functions. The computer-readable storage medium of the present embodiment is used for storing the pedestrian re-identification apparatus 50, and when being executed by the processor, the pedestrian re-identification method of the first embodiment is implemented.
The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.
Any process or method descriptions in flow charts or otherwise described herein may be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing specific logical functions or steps of the process, and alternate implementations are included within the scope of the preferred embodiment of the present invention in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those reasonably skilled in the art of the present invention.
It will be understood by those skilled in the art that all or part of the steps carried by the method for implementing the above embodiments may be implemented by hardware related to instructions of a program, which may be stored in a computer readable medium, and when executed, the program includes one or a combination of the steps of the method embodiments.
In the description herein, references to the description of the term "one embodiment," "some embodiments," "an example," "a specific example" or "some examples" or the like are intended to mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the schematic representations of the terms used above do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner.
The above description is only a preferred embodiment of the present invention, and not intended to limit the scope of the present invention, and all modifications of equivalent structures and equivalent processes, which are made by using the contents of the present specification and the accompanying drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the present invention.
Claims (10)
1. A pedestrian re-identification method is characterized by comprising the following steps:
acquiring reference images of a plurality of reference pedestrians, and extracting corresponding reference features from the reference images by using a feature extraction model; wherein the feature extraction model is obtained by utilizing neural network training;
storing the reference features into a pedestrian feature library, and updating the reference features stored in the pedestrian feature library based on a preset frequency;
acquiring an image to be recognized containing a pedestrian to be recognized, and extracting corresponding features to be recognized from the image to be recognized by using the feature extraction model;
comparing the feature to be recognized with candidate features stored in the pedestrian feature library to determine a target feature which is different from the image to be recognized by less than a preset first threshold value from the candidate features;
and identifying the pedestrian to be identified according to the category label corresponding to the target feature.
2. The pedestrian re-identification method according to claim 1, wherein the step of updating the reference features stored in the pedestrian feature library based on a preset frequency includes:
acquiring an updated image containing a reference pedestrian;
extracting corresponding updated features from the updated image by using the feature extraction model;
judging whether reference features are stored in the pedestrian feature library or not;
when the pedestrian feature library does not store reference features, storing the updated features into the pedestrian feature library as new reference features, and adding new category labels to the updated features;
calculating a first distance between the updated feature and one of the stored reference features in the case that the reference features are already stored in the pedestrian feature library;
judging whether the first distance is smaller than a first threshold value;
if the first distance is smaller than a first threshold value, storing the updated feature into the pedestrian feature library as a new reference feature, and adding an existing category label which is the same as one of the reference features to the updated feature;
and if the first distance is not smaller than a first threshold value, storing the updated feature into the pedestrian feature library as a new reference feature, and adding a new category label for the updated feature.
3. The pedestrian re-identification method according to claim 1 or 2, wherein the training process of the feature extraction model includes:
constructing a first loss function based on the improved softmax function, constructing a second loss function based on the triplet function, and constructing a third loss function based on the Angular softmax function;
and determining a comprehensive loss function of the feature extraction model according to the first loss function, the second loss function and the third loss function.
4. The pedestrian re-identification method according to claim 3, wherein the first loss function L1 is:
in the above formula, yiRepresenting a category label corresponding to the feature to be identified; c represents a category label corresponding to the reference feature in the feature library; i (y)iC) stands for yiIf c is true, 1 is returned, otherwise, 0 is returned; w representing fully connected layers contained in the feature extraction modelA parameter matrix; b is a parameter to be optimized;
the second loss function L2 is:
in the formula, a represents any reference image in the training set, p represents a reference image with the same type label as a, and n represents a reference image with a different type label from a; d (a, p) represents the Euclidean distance between a and p, d (a, n) represents the Euclidean distance between a and n, m is a parameter to be optimized, [ d (a, p) -d (a, n) + m,0] represents taking the larger value of d (a, p) -d (a, n) + m and 0;
the third loss function L3 is:
in the above formula, i and j represent different class labels;
the composite loss function L is:
L＝αL1+βL2+γL3
wherein α, β, γ represent weight coefficients, respectively.
5. The pedestrian re-identification method according to claim 3, wherein the training process of the feature extraction model further comprises:
training the feature extraction model based on a domain adaptive algorithm to adapt the feature extraction model to the span from a life scene to a construction scene.
6. The pedestrian re-identification method according to claim 1, wherein the step of comparing the feature to be identified with candidate features stored in the pedestrian feature library to determine a target feature which is different from the image to be identified by less than a preset first threshold from the candidate features comprises:
multiplying the first vector formed by the features to be identified and the first matrix formed by the candidate features to obtain a second vector; each column vector in the first matrix corresponds to one candidate feature respectively;
judging whether the minimum value in the second vector is smaller than the preset first threshold value or not;
and if so, taking the candidate feature corresponding to the minimum value as a target feature.
7. The pedestrian re-identification method according to claim 1, wherein the step of identifying the pedestrian to be identified according to the class label corresponding to the target feature comprises:
searching a pedestrian identity corresponding to the category label according to a preset mapping relation;
and identifying the pedestrian to be identified based on the pedestrian identity.
8. A pedestrian re-recognition apparatus, comprising:
the system comprises a reference feature module, a feature extraction module and a feature extraction module, wherein the reference feature module is suitable for acquiring reference images of a plurality of reference pedestrians and extracting corresponding reference features from the reference images by using a feature extraction model; wherein the feature extraction model is obtained by utilizing neural network training;
the characteristic library module is suitable for storing the reference characteristics to a pedestrian characteristic library and updating the reference characteristics stored in the pedestrian characteristic library based on a preset frequency;
the characteristic extraction module is suitable for acquiring an image to be identified containing a pedestrian to be identified, and extracting corresponding characteristics to be identified from the image to be identified by using the characteristic extraction model;
the comparison module is used for comparing the feature to be recognized with candidate features stored in the pedestrian feature library so as to determine a target feature which is different from the image to be recognized by less than a first threshold value from the candidate features;
and the identification module is suitable for identifying the pedestrian to be identified according to the category label corresponding to the target feature.
9. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the steps of the method of any of claims 1 to 7 are implemented by the processor when executing the computer program.
10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011264154.3A CN112347957A (en) | 2020-11-12 | 2020-11-12 | Pedestrian re-identification method and device, computer equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011264154.3A CN112347957A (en) | 2020-11-12 | 2020-11-12 | Pedestrian re-identification method and device, computer equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112347957A true CN112347957A (en) | 2021-02-09 |
Family
ID=74362677
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011264154.3A Pending CN112347957A (en) | 2020-11-12 | 2020-11-12 | Pedestrian re-identification method and device, computer equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112347957A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113033410A (en) * | 2021-03-26 | 2021-06-25 | 中山大学 | Domain generalization pedestrian re-identification method, system and medium based on automatic data enhancement |
| CN113610028A (en) * | 2021-08-13 | 2021-11-05 | 广联达科技股份有限公司 | Pedestrian re-identification method and device, computer equipment and storage medium |
| CN114863475A (en) * | 2022-04-12 | 2022-08-05 | 芯峰科技(广州)有限公司 | Automatic updating method for pedestrian re-recognition feature library |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107657249A (en) * | 2017-10-26 | 2018-02-02 | 珠海习悦信息技术有限公司 | Method, apparatus, storage medium and the processor that Analysis On Multi-scale Features pedestrian identifies again |
| CN107844753A (en) * | 2017-10-20 | 2018-03-27 | 珠海习悦信息技术有限公司 | Pedestrian in video image recognition methods, device, storage medium and processor again |
| CN109740413A (en) * | 2018-11-14 | 2019-05-10 | 平安科技（深圳）有限公司 | Pedestrian recognition methods, device, computer equipment and computer storage medium again |
| WO2019128367A1 (en) * | 2017-12-26 | 2019-07-04 | 广州广电运通金融电子股份有限公司 | Face verification method and apparatus based on triplet loss, and computer device and storage medium |
| CN111476168A (en) * | 2020-04-08 | 2020-07-31 | 山东师范大学 | Cross-domain pedestrian re-identification method and system based on three stages |
| CN111611880A (en) * | 2020-04-30 | 2020-09-01 | 杭州电子科技大学 | Efficient pedestrian re-identification method based on unsupervised contrast learning of neural network |
| US20200285896A1 (en) * | 2019-03-09 | 2020-09-10 | Tongji University | Method for person re-identification based on deep model with multi-loss fusion training strategy |
| WO2020186914A1 (en) * | 2019-03-20 | 2020-09-24 | 北京沃东天骏信息技术有限公司 | Person re-identification method and apparatus, and storage medium |
-
2020
- 2020-11-12 CN CN202011264154.3A patent/CN112347957A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107844753A (en) * | 2017-10-20 | 2018-03-27 | 珠海习悦信息技术有限公司 | Pedestrian in video image recognition methods, device, storage medium and processor again |
| CN107657249A (en) * | 2017-10-26 | 2018-02-02 | 珠海习悦信息技术有限公司 | Method, apparatus, storage medium and the processor that Analysis On Multi-scale Features pedestrian identifies again |
| WO2019128367A1 (en) * | 2017-12-26 | 2019-07-04 | 广州广电运通金融电子股份有限公司 | Face verification method and apparatus based on triplet loss, and computer device and storage medium |
| CN109740413A (en) * | 2018-11-14 | 2019-05-10 | 平安科技（深圳）有限公司 | Pedestrian recognition methods, device, computer equipment and computer storage medium again |
| US20200285896A1 (en) * | 2019-03-09 | 2020-09-10 | Tongji University | Method for person re-identification based on deep model with multi-loss fusion training strategy |
| WO2020186914A1 (en) * | 2019-03-20 | 2020-09-24 | 北京沃东天骏信息技术有限公司 | Person re-identification method and apparatus, and storage medium |
| CN111476168A (en) * | 2020-04-08 | 2020-07-31 | 山东师范大学 | Cross-domain pedestrian re-identification method and system based on three stages |
| CN111611880A (en) * | 2020-04-30 | 2020-09-01 | 杭州电子科技大学 | Efficient pedestrian re-identification method based on unsupervised contrast learning of neural network |
Non-Patent Citations (1)
| Title |
|---|
| 樊琳;张惊雷;: "联合损失优化孪生网络的行人重识别", 计算机工程与科学, no. 02, 15 February 2020 (2020-02-15) * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113033410A (en) * | 2021-03-26 | 2021-06-25 | 中山大学 | Domain generalization pedestrian re-identification method, system and medium based on automatic data enhancement |
| CN113033410B (en) * | 2021-03-26 | 2023-06-06 | 中山大学 | Domain generalization pedestrian re-recognition method, system and medium based on automatic data enhancement |
| CN113610028A (en) * | 2021-08-13 | 2021-11-05 | 广联达科技股份有限公司 | Pedestrian re-identification method and device, computer equipment and storage medium |
| CN114863475A (en) * | 2022-04-12 | 2022-08-05 | 芯峰科技(广州)有限公司 | Automatic updating method for pedestrian re-recognition feature library |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112232241B (en) | Pedestrian re-identification method and device, electronic equipment and readable storage medium | |
| US20200160124A1 (en) | Fine-grained image recognition | |
| CN112347957A (en) | Pedestrian re-identification method and device, computer equipment and storage medium | |
| CN111339979B (en) | Image recognition method and image recognition device based on feature extraction | |
| CN112465840B (en) | Semantic segmentation model training method, semantic segmentation method and related device | |
| CN108564102A (en) | Image clustering evaluation of result method and apparatus | |
| CN113128478B (en) | Model training method, pedestrian analysis method, device, equipment and storage medium | |
| US20090228510A1 (en) | Generating congruous metadata for multimedia | |
| CN111291887A (en) | Neural network training method, image recognition method, device and electronic equipment | |
| CN113128536A (en) | Unsupervised learning method, system, computer device and readable storage medium | |
| CN113378852A (en) | Key point detection method and device, electronic equipment and storage medium | |
| CN112101156B (en) | Target identification method and device and electronic equipment | |
| CN113902944A (en) | Model training and scene recognition method, device, equipment and medium | |
| CN114676777A (en) | Self-supervision learning fine-grained image classification method based on twin network | |
| CN114170558B (en) | Method, system, apparatus, medium, and article for video processing | |
| CN114637877A (en) | Labeling method, electronic device and storage medium | |
| CN118154583A (en) | Battery screen printing quality detection method, battery screen printing quality detection device, battery screen printing model acquisition method, battery screen printing quality detection device and terminal equipment | |
| CN112446428B (en) | Image data processing method and device | |
| CN112241736A (en) | Text detection method and device | |
| CN113744280A (en) | Image processing method, apparatus, device and medium | |
| CN112132150B (en) | Text string recognition method and device and electronic equipment | |
| CN112989869B (en) | Optimization method, device, equipment and storage medium of face quality detection model | |
| CN113361511B (en) | Correction model establishing method, device, equipment and computer readable storage medium | |
| CN113378707A (en) | Object identification method and device | |
| CN118521472B (en) | Image processing method, device and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |