CN111582150A - A method, device and computer storage medium for face quality assessment - Google Patents

Abstract

The invention discloses a face quality assessment method, device and computer storage medium. The assessment method includes the following steps: acquiring a plurality of face images; using a convolutional neural network algorithm to perform feature extraction on the acquired face images as Feature training set; input the feature data in the feature training set into the image quality evaluation model, output the quality evaluation value, perform face recognition on the feature data whose quality evaluation value meets preset conditions, and perform face recognition on the feature data whose quality evaluation value satisfies the preset condition Conditional feature data for face recognition The loss function is modified on the basis of face recognition, so that the face recognition network has the function of face quality assessment at the same time. Face image quality tags for easy implementation.

Description

A method, device and computer storage medium for face quality assessment

technical field

The present invention relates to the technical field of computer vision, and in particular, to a method, a device and a computer storage medium for face quality assessment.

Background technique

The process of face recognition is to map the face image to a feature vector, and use the residual distance between the feature vectors to calculate the similarity. During the training process, the distance within the class is reduced and the distance between the classes is increased. For a certain face recognition network, inputting two similar pictures, even non-face pictures, will get two feature vectors with high similarity. For example, input two blurred face pictures, although they have high similarity, but people are not sure that they represent the same person, because people can judge that these blurry pictures are not suitable for people face recognition.

The factors that affect the performance of face recognition include illumination, clarity, profile, expression and occlusion, etc. These factors can be expressed by face quality. Face quality is to predict whether a face image is suitable for face recognition. For the training of face quality, training data is a big bottleneck. Since there are many factors that affect face quality, it is difficult for people to evaluate the degree of influence between different factors, so it is difficult to label the quality of face pictures. . Existing methods calculate the similarity based on the face recognition model. For example, select a picture with the best quality from each person's picture, and then calculate the difference between each person's other face pictures and the best quality picture according to the pre-trained face recognition model. The similarity between the two, and the similarity is used as the quality label, and the training of face quality is performed based on the above data. The similarity calculated based on the existing face recognition model does not necessarily represent the quality. For example, the two face pictures are both clear frontal faces, but due to the long time interval, their quality is high, but the similarity is not necessarily high. . This results in an error in the face quality label.

SUMMARY OF THE INVENTION

In view of the above technical problems, the present invention provides a method, device and computer storage medium for evaluating the quality of a face, which simultaneously estimates the quality during the face recognition process, and does not require a face image quality label during the training process, which is convenient for implementation.

An embodiment of the present invention provides a method for evaluating face quality, the method comprising the steps of: acquiring multiple face images; using a convolutional neural network algorithm to perform feature extraction on the acquired face images as a feature training set; The feature data in the feature training set is input into the image quality evaluation model, and the quality evaluation value is output, wherein the image quality evaluation model adopts a machine learning method, using the preset feature data of the sample image and the feature data in the feature training set. The characteristic data is obtained by training based on the loss function according to the Gaussian distribution and probability density; face recognition is performed on the characteristic data whose quality evaluation value satisfies the preset conditions.

Optionally, the image quality assessment model adopts a machine learning method, and according to the Gaussian distribution and probability density of the feature data, the steps of training based on the loss function include: if the identity ID of the feature data x _i in the training set is j, according to The Gaussian distribution obtains the probability density of the feature data _xi with the identity ID j; calculates the probability that the feature data _xi belongs to the identity ID j; obtains the loss function according to the probability density and the probability.

Optionally, the value of the loss function is used to represent the difference between the feature data of the sample image of the same person and the quality evaluation value of the feature data in the feature training set.

Optionally, the step of obtaining the loss function according to the probability density and the probability further includes: calculating a cosine value between the feature data of each of the sample images and the feature data in the corresponding feature training set.

The present invention also provides a face quality assessment device, the device includes: an acquisition unit for acquiring multiple face images; a feature extraction unit for using a convolutional neural network algorithm to characterize the acquired face images Extraction as a feature training set; the evaluation unit inputs the feature data in the feature training set into an image quality evaluation model, and outputs a quality evaluation value, wherein the image quality evaluation model adopts a machine learning method and uses a preset sample image The feature data and the feature data in the feature training set are obtained based on Gaussian distribution and probability density, based on loss function training; the recognition unit performs face recognition on the feature data whose quality evaluation value meets preset conditions.

Preferably, the device further comprises: a training unit for using a machine learning method, using the preset feature data of the sample image and the feature data in the feature training set, and training the result based on the loss function according to the Gaussian distribution and the probability density. Described image quality assessment model that comes out; Described training unit obtains described image quality assessment model based on following steps training: If the identity ID of characteristic data x _i in the training set is j, obtain the characteristic data that identity ID is j according to Gaussian distribution _xi probability density; calculate the probability that the feature data _xi belongs to the identity ID j; obtain a loss function according to the probability density and probability.

Preferably, the value of the loss function is used to represent the difference between the feature data of the sample image of the same person and the quality evaluation value of the feature data in the feature training set.

The present invention also provides a computer storage medium, when the computer program is executed by a processor, the steps of the method described in any one of the preceding claims are implemented.

In the technical solution provided by the embodiment of the present invention, a feature training set is extracted by using a convolutional neural network algorithm, the feature data in the feature training set is input into an image quality evaluation model, and a quality evaluation value is output, and the quality evaluation value satisfies a preset condition Therefore, compared with the prior art, the embodiment of the present invention modifies the loss function on the basis of face recognition, so that the face recognition network has the function of face quality assessment at the same time. The quality is evaluated at the same time in the training process, and the face image quality label is not required during the training process, which is convenient for implementation.

Description of drawings

1 is a schematic flowchart of an embodiment of a method for evaluating the quality of a human face according to the present invention;

2 is a schematic flowchart of another embodiment of a method for evaluating the quality of a human face according to the present invention;

FIG. 3 is a schematic structural diagram of an apparatus for evaluating the quality of a human face according to the present invention.

Detailed ways

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative efforts shall fall within the protection scope of the present invention.

The present invention provides a method for evaluating the quality of a human face, which runs on an electronic device, such as a terminal device or a server. Please refer to FIG. 1 . The method includes the following steps:

Step S10, acquiring multiple face images. In this embodiment, multiple face images may be directly input to the terminal device or server, and the face images to be evaluated may be face images collected in various environments, and may be visible light face images or non-visible light face images The image, which can be a clear face image or a face image with motion blur/defocus blur, can be a face image collected in the cooperating state or non-cooperating state of the subject, or a face image containing noise, and many more.

Step S20, using a convolutional neural network algorithm to extract features from the acquired face image as a feature training set. Input the face image to be evaluated into the convolutional neural network, extract the features of the face image through the computer, and use the extracted features as the feature training set.

Step S30, input the feature data in the feature training set into the image quality evaluation model, and output the quality evaluation value, wherein, the image quality evaluation model adopts a machine learning method, and uses the preset feature data of the sample image and the feature training method. The centralized feature data, according to the Gaussian distribution and probability density, is obtained by training based on the loss function.

In one of the embodiments of the present invention, please refer to FIG. 2 , the image quality evaluation model in step S30 adopts the machine learning method, according to the Gaussian distribution and probability density of the feature data, and the steps obtained by training based on the loss function Specifically include:

Step S31, if the identity ID of the feature data xi in the training set is _j , obtain the probability density of the feature data _xi whose identity ID is j according to the Gaussian distribution;

Step S32, calculate the probability that the feature data x _i belongs to the identity ID j;

Step S33, obtaining a loss function according to the probability density and probability.

Specifically, the feature data of the face image is regarded as a Gaussian difference, assuming that there are data of N people in the training set, y∈{1, 2,...N} represents the identity ID of each feature data, assuming that one of the input The feature data corresponding to the image is x _i , its corresponding identity ID is j, the corresponding feature obeys the Gaussian distribution N(f _i , σ ² I), and the vector of the sample image whose corresponding identity ID is j is w _j , Then the probability density of the feature data of the sample image whose identity ID is j is _xi is formula (1):

Here D is the feature dimension of the feature vector. In addition, it is assumed that the prior probability of each identity ID is equal, that is, a face image to be evaluated is randomly selected, and the probability of belonging to each identity ID is the same, then the feature data x _i belongs to the probability that the identity ID is j for:

则Simplify the above formula, constrain f _i and w _j to be unit vectors, define

but

The above formula can be expressed as:

The above formula _si can be regarded as the reliability of the image _xi , but the value of _si is not limited in the range, and it is not a good evaluation index for the quality of the face. Therefore, we assume that the maximum value of _si takes S, then,

s _i =S*d _i , d _i ∈ [0, 1]

So the above formula can be expressed as:

Here d _i represents the quality of the face, and the higher the quality of the face, the closer the value of d _i is to 1. f _i w _j =cos(θ _ij ),

θ _ij is the _{angle between the vectors fi and w j .} During training, refer to ArcFace to add Margin loss. The loss function is as follows:

The loss function in the above formula (6) introduces the quality evaluation value d _i on the basis of ArcFace. During training, the high-quality face is closer to the feature vector w _j of the corresponding person's sample image, so the angle between the vectors is When θ _ij is small, in order to reduce the loss value, the value of d _i will be increased.

During training, the similarity between the low-quality face and the feature w _j of the corresponding person is low, so the angle θ _ij between the vectors is large. In order to reduce the loss value, the value of d _i will be reduced.

Therefore, no matter the low quality caused by factors such as occlusion, blur, or profile, as long as it is difficult to recognize, when training face recognition, the angle between their features and the feature data of the same person's sample image is relatively large. A smaller quality evaluation value can be obtained.

During training, as the value of low-quality face d _i decreases, its training weight will also decrease; on the contrary, the weight of high-quality face is higher. Make each person's feature _wj automatically close to the high-quality face.

Step S40, performing face recognition on the feature data whose quality evaluation value satisfies a preset condition. The feature data that meets the preset conditions in the quality evaluation value output by the image quality evaluation model is input into the face recognition model, that is, the face recognition is performed in the convolutional neural network algorithm.

In the present invention, the image quality evaluation model is used to evaluate the quality of the face image, and the obtained quality evaluation result of the face image is obtained. The above image quality evaluation model is obtained by using the machine learning method, using the preset set of face images to be evaluated, that is, the set of feature training sets and the feature data of the corresponding sample images, based on loss function training, that is, the loss function. The value of is used to characterize the difference between the feature data of the sample image with the same person and the quality evaluation value of the feature data in the feature training set. Here, the sample image is typically a well-focused, high-resolution frontal visible light image under a uniform light source.

The above-mentioned image quality assessment model is based on a neural network model, and may also use logistic regression, hidden Markov model, or the like.

Specifically, according to embodiments of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the method illustrated in the flowchart.

Referring to FIG. 3 , in one embodiment of the present invention, an apparatus 100 for evaluating face quality is further provided. The apparatus includes: an acquisition unit 101 , a feature extraction unit 102 , an evaluation unit 103 , and an identification unit 104 , the acquisition unit 101 is used to acquire multiple face images; the feature extraction unit 102 uses the convolutional neural network algorithm to perform feature extraction on the acquired face images as a feature training set; the evaluation unit 103 uses the feature training set The feature data is input into the image quality evaluation model, and the quality evaluation value is output, wherein the image quality evaluation model adopts the machine learning method, using the preset feature data of the sample image and the feature data in the feature training set, according to the Gaussian distribution and probability density, which are obtained by training based on the loss function; the identifying unit 104 performs face recognition on the feature data whose evaluation of the quality evaluation value satisfies the preset condition.

In one embodiment of the present invention, the apparatus 100 for evaluating the quality of a human face further includes a training unit, configured to adopt a machine learning method, using the preset feature data of the sample image and the feature data in the feature training set, According to Gaussian distribution and probability density, the image quality evaluation model obtained based on loss function training; the training unit obtains the image quality evaluation model based on the following steps:

If the identity ID of the feature data xi in the training set is _j , obtain the probability density of the feature data _xi with the identity ID j according to the Gaussian distribution;

Calculate the probability that the feature data x _i belongs to the identity ID j;

A loss function is obtained from the probability density and probability.

In one of the embodiments of the present invention, a computer storage medium is also provided, and when the computer storage medium is executed by a processor, the steps of the method for evaluating the quality of a face described in any of the foregoing embodiments are implemented. Specifically, the computer-readable medium may be included in the apparatus described in the above embodiments; or may exist alone without being assembled into the apparatus.

The method and device for face quality assessment provided by the present invention modifies the loss function on the basis of face recognition, so that the face recognition network has the function of face quality assessment at the same time. There is no need for face image quality labels in the process, which is easy to implement.

The above embodiments are only used to illustrate the technical solutions of the present invention, but not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: The recorded technical solutions are modified, or some technical features thereof are equivalently replaced; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims (8)

1. a method of face quality assessment, is characterized in that, described method comprises the steps: Get multiple face images; Use the convolutional neural network algorithm to extract features from the acquired face images as a feature training set; Input the feature data in the feature training set into an image quality assessment model, and output a quality assessment value, wherein the image quality assessment model adopts a machine learning method, using the preset feature data of the sample image and the feature training set. The characteristic data of , is obtained by training the loss function according to the Gaussian distribution and probability density; Perform face recognition on the feature data whose quality evaluation value satisfies the preset condition. 2. the method for face quality assessment according to claim 1, is characterized in that, described image quality assessment model adopts machine learning method, according to the Gaussian distribution and probability density of characteristic data, the step that draws based on loss function training include: If the identity ID of the feature data xi in the training set is _j , obtain the probability density of the feature data _xi with the identity ID j according to the Gaussian distribution; Calculate the probability that the feature data x _i belongs to the identity ID j; A loss function is obtained from the probability density and probability. 3. The method for face quality assessment according to claim 2, wherein the value of the loss function is used to characterize the difference between the quality assessment value of the characteristic data of the sample image of the same person and the characteristic data in the characteristic training set. difference between. 4. The method for assessing human face quality according to claim 3, wherein the step of obtaining a loss function according to the probability density and probability further comprises: Calculate the cosine value between the feature data of each of the sample images and the feature data in the corresponding feature training set. 5. A device for face quality assessment, wherein the device comprises: an acquisition unit for acquiring multiple face images; The feature extraction unit uses the convolutional neural network algorithm to perform feature extraction on the acquired face image as a feature training set; The evaluation unit, which inputs the feature data in the feature training set into an image quality evaluation model, and outputs a quality evaluation value, wherein the image quality evaluation model adopts a machine learning method, using the preset feature data of the sample image and the The feature data in the feature training set is obtained from the loss function training according to the Gaussian distribution and probability density; A recognition unit, which performs face recognition on the feature data whose quality evaluation value satisfies a preset condition. 6. The device for human face quality assessment according to claim 5, wherein the device further comprises: A training unit for using a machine learning method, using the preset feature data of the sample image and the feature data in the feature training set, according to Gaussian distribution and probability density, and training the image quality evaluation model based on the loss function; The training unit obtains the image quality assessment model based on the following steps: If the identity ID of the feature data xi in the training set is _j , obtain the probability density of the feature data _xi with the identity ID j according to the Gaussian distribution; Calculate the probability that the feature data x _i belongs to the identity ID j; A loss function is obtained from the probability density and probability. 7. The device for evaluating the quality of human face according to claim 5, wherein the value of the loss function is used to characterize the difference between the quality evaluation value of the characteristic data of the sample image of the same person and the characteristic data in the characteristic training set. difference between. 8. A computer storage medium, wherein when the computer program is executed by a processor, the steps of the method according to any one of claims 1-4 are implemented.

Images