Nothing Special   »   [go: up one dir, main page]

CN111222446B - Face recognition method, face recognition device and mobile terminal - Google Patents

Face recognition method, face recognition device and mobile terminal Download PDF

Info

Publication number
CN111222446B
CN111222446B CN201911421186.7A CN201911421186A CN111222446B CN 111222446 B CN111222446 B CN 111222446B CN 201911421186 A CN201911421186 A CN 201911421186A CN 111222446 B CN111222446 B CN 111222446B
Authority
CN
China
Prior art keywords
face image
face
image
pixel
images
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201911421186.7A
Other languages
Chinese (zh)
Other versions
CN111222446A (en
Inventor
黄海东
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangdong Oppo Mobile Telecommunications Corp Ltd
Original Assignee
Guangdong Oppo Mobile Telecommunications Corp Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangdong Oppo Mobile Telecommunications Corp Ltd filed Critical Guangdong Oppo Mobile Telecommunications Corp Ltd
Priority to CN201911421186.7A priority Critical patent/CN111222446B/en
Publication of CN111222446A publication Critical patent/CN111222446A/en
Application granted granted Critical
Publication of CN111222446B publication Critical patent/CN111222446B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16Human faces, e.g. facial parts, sketches or expressions
    • G06V40/161Detection; Localisation; Normalisation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/045Combinations of networks
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00Geometric image transformations in the plane of the image
    • G06T3/40Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4007Scaling of whole images or parts thereof, e.g. expanding or contracting based on interpolation, e.g. bilinear interpolation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16Human faces, e.g. facial parts, sketches or expressions
    • G06V40/172Classification, e.g. identification

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Multimedia (AREA)
  • Human Computer Interaction (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Biomedical Technology (AREA)
  • Software Systems (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Mathematical Physics (AREA)
  • Evolutionary Computation (AREA)
  • Computational Linguistics (AREA)
  • Biophysics (AREA)
  • Artificial Intelligence (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Image Processing (AREA)
  • Image Analysis (AREA)

Abstract

The application is applicable to the technical field of face recognition, and provides a face recognition method, a face recognition device, a mobile terminal and a computer readable storage medium, wherein the method comprises the following steps: acquiring continuous N frames of initial images aiming at a target face; acquiring a face image of the target face in each frame of initial image in the N frames of initial images; sub-pixel interpolation is carried out on the N frames of face images, so that a first face image is obtained; inputting the first face image into a preset neural network to obtain a second face image; and recognizing the face in the second face image. The face recognition precision can be improved through the face recognition method and device.

Description

Face recognition method, face recognition device and mobile terminal
Technical Field
The application belongs to the technical field of face recognition, and particularly relates to a face recognition method, a face recognition device, a mobile terminal and a computer readable storage medium.
Background
Face recognition is a biological recognition technology for performing identity recognition based on facial feature information of a person, and an image containing the face is acquired by using an imaging device and detected in the image, so that the detected face is subjected to face recognition, and can also be called as image recognition and face recognition. However, when the face is small in the image or the imaging quality is poor, the face recognition accuracy is lowered.
Disclosure of Invention
The application provides a face recognition method, a face recognition device, a mobile terminal and a computer readable storage medium, so as to improve face recognition accuracy.
In a first aspect, an embodiment of the present application provides a face recognition method, where the face recognition method includes:
acquiring continuous N frames of initial images aiming at a target face, wherein N is an integer greater than 1;
acquiring face images of the target face in each frame of initial images in the N frames of initial images, wherein the face images of the target face in each frame of initial images refer to images of areas where the target face is located in each frame of initial images, the N frames of initial images correspond to the N frames of face images, and the N frames of face images have the same size;
sub-pixel interpolation is carried out on the N frames of face images to obtain a first face image, wherein the size of the first face image is larger than that of each frame of face image, and the resolution of the first face image is larger than that of each frame of face image;
inputting the first face image into a preset neural network to obtain a second face image, wherein the size of the second face image is larger than that of the first face image, and the resolution of the second face image is larger than that of the first face image;
And recognizing the face in the second face image.
In a second aspect, an embodiment of the present application provides a face recognition device, including:
the initial image acquisition module is used for acquiring continuous N frames of initial images aiming at a target face, wherein N is an integer greater than 1;
the face image acquisition module is used for acquiring the face image of the target face in each frame of initial image in the N frames of initial images, wherein the face image of the target face in each frame of initial image refers to an image of an area where the target face is located in each frame of initial image, the N frames of initial images correspond to the N frames of face images, and the N frames of face images are identical in size;
the sub-pixel interpolation module is used for carrying out sub-pixel interpolation on the N frames of face images to obtain a first face image, wherein the size of the first face image is larger than that of each frame of face image, and the resolution of the first face image is larger than that of each frame of face image;
the face image input module is used for inputting the first face image into a preset neural network to obtain a second face image, wherein the size of the second face image is larger than that of the first face image, and the resolution of the second face image is larger than that of the first face image;
And the face recognition module is used for recognizing the face in the second face image.
In a third aspect, an embodiment of the present application provides a mobile terminal, including a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor implements the steps of the face recognition method according to the first aspect described above when the processor executes the computer program.
In a fourth aspect, embodiments of the present application provide a computer readable storage medium storing a computer program, which when executed by a processor, implements the steps of the face recognition method according to the first aspect.
In a fifth aspect, embodiments of the present application provide a computer program product which, when run on a mobile terminal, causes the mobile terminal to perform the steps of the face recognition method as described in the first aspect above.
From the above, the method and the device acquire continuous N frames of initial images aiming at the target face, acquire face images of the target face in each frame of initial images, conduct sub-pixel interpolation on the N frames of face images to obtain a first face image with amplified size and resolution, input the first face image into a preset neural network, conduct image enhancement on the first face image, and further amplify the size and resolution of the first face image, namely, the method and the device can effectively improve the number of real details in the face image by combining multi-frame super-resolution with deep learning, and further improve face recognition accuracy.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are required for the embodiments or the description of the prior art will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a schematic implementation flow chart of a face recognition method according to an embodiment of the present application;
FIG. 2 is a diagram of an example subpixel interpolation;
fig. 3 is a schematic implementation flow chart of a face recognition method according to a second embodiment of the present application;
FIG. 4 is an exemplary diagram of a face image processing procedure;
fig. 5 is a schematic structural diagram of a face recognition device according to a third embodiment of the present application;
fig. 6 is a schematic structural diagram of a mobile terminal according to a fourth embodiment of the present application;
fig. 7 is a schematic structural diagram of a mobile terminal according to a fifth embodiment of the present application.
Detailed Description
In the following description, for purposes of explanation and not limitation, specific details are set forth, such as particular system configurations, techniques, etc. in order to provide a thorough understanding of the embodiments of the present application. It will be apparent, however, to one skilled in the art that the present application may be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present application with unnecessary detail.
It should be understood that the terms "comprises" and/or "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
It should also be understood that the term "and/or" as used in this specification and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes such combinations.
As used in this specification and the appended claims, the term "if" may be interpreted as "when..once" or "in response to a determination" or "in response to detection" depending on the context. Similarly, the phrase "if a determination" or "if a [ described condition or event ] is detected" may be interpreted in the context of meaning "upon determination" or "in response to determination" or "upon detection of a [ described condition or event ]" or "in response to detection of a [ described condition or event ]".
In addition, in the description of the present application and the appended claims, the terms "first," "second," "third," and the like are used merely to distinguish between descriptions and are not to be construed as indicating or implying relative importance.
Reference in the specification to "one embodiment" or "some embodiments" or the like means that a particular feature, structure, or characteristic described in connection with the embodiment is included in one or more embodiments of the application. Thus, appearances of the phrases "in one embodiment," "in some embodiments," "in other embodiments," and the like in the specification are not necessarily all referring to the same embodiment, but mean "one or more but not all embodiments" unless expressly specified otherwise. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless expressly specified otherwise.
In order to illustrate the technical solutions described in the present application, the following description is made by specific examples.
Referring to fig. 1, a schematic implementation flow chart of a face recognition method according to an embodiment of the present application, where the face recognition method is applied to a mobile terminal, as shown in the figure, the face recognition method may include the following steps:
step S101, acquiring continuous N frames of initial images for a target face.
Wherein N is an integer greater than 1.
Optionally, the value range of N is [3,16], and when the value range of N is [3,16], the operation amount increase degree and the face recognition precision improvement degree of the mobile terminal in the face recognition process are economical.
In the embodiment of the present application, the target face may refer to a face to be identified. When the camera device of the mobile terminal is used for photographing the target face, continuous N frames of images (namely N frames of initial images) captured by the camera device can be obtained.
Step S102, obtaining a face image of the target face in each frame of initial image in the N frames of initial images.
The face image of the target face in each frame of initial image refers to an image of an area where the target face is located in each frame of initial image, the N frames of initial images correspond to the N frames of face images, and the sizes of the N frames of face images are the same, for example, the sizes of the N frames of face images are w×h, W is the width of the face image, and H is the height of the face image.
In the embodiment of the present application, acquiring the face image of the target face in each frame of initial image may refer to cutting out the face image of the target face from each frame of initial image. Specifically, a center point of a target face in each frame of initial image is obtained, the center point is used as a center point of a face image, a face image with a preset size is cut out, and the face image comprises the whole target face.
Step S103, sub-pixel interpolation is carried out on the N frames of face images, and a first face image is obtained.
The size of the first face image is larger than that of each frame of face image, and the resolution of the first face image is larger than that of each frame of face image. Optionally, the user may set the size and resolution of the first face image according to the actual requirement.
In the embodiment of the application, sub-pixel interpolation is performed on N frames of face images, so that multi-frame super-division is realized, the number of real details of the face images can be increased, the detail definition of the face images is improved, noise is reduced, and a first face image with higher resolution is obtained.
Optionally, the sub-pixel interpolation is performed on the N frames of face images, and obtaining the first face image includes:
selecting a frame of face image from the N frames of face images as a reference face image;
performing bilinear interpolation on the reference face image to obtain a third face image, wherein the size of the third face image is the same as that of the first face image, and the resolution of the third face image is larger than that of the reference face image;
step a1, performing bilinear interpolation on an ith frame of face image in the remaining N-1 frame of face images to obtain a fourth face image, wherein the remaining N-1 frame of face image refers to face images except the reference face image in the N frame of face images, the size of the fourth face image is the same as that of the first face image, the resolution of the fourth face image is larger than that of the ith frame of face image, and i is an integer larger than zero and smaller than or equal to N-1;
Step a2, performing image matching on the third face image and the fourth face image, and obtaining a first pixel in the third face image and a sub-pixel matched with the first pixel in the fourth face image;
step a3, obtaining pixel values of four pixels adjacent to the sub-pixel in the fourth face image;
step a4, obtaining the pixel value of the sub-pixel in the fourth face image according to the pixel values of four pixels adjacent to the sub-pixel in the fourth face image;
step a5, adding and averaging the pixel value of the first pixel in the third face image and the pixel value of the sub-pixel in the fourth face image, and taking the average value as the pixel value of the first pixel in the third face image;
and repeatedly executing the steps a1, a2, a3, a4 and a5 until the rest N-1 frames of face images are traversed, and determining the processed third face image as the first face image, wherein the processed third face image is the third face image of which the pixel values of the first pixels are subjected to addition and averaging processing.
In the embodiment of the application, the definition of each frame of face image in the N frames of face images can be acquired first, the face image with the highest definition in the N frames of face images is used as the reference face image, the detail characteristics of the reference face image can be enhanced by carrying out bilinear interpolation on the reference face image, the size of the reference face image is enlarged, the resolution of the reference face image is improved, and the reference face image with the enlarged size and the improved resolution is the third face image.
For any frame of the remaining N-1 frame of face images, bilinear interpolation can be carried out on the frame of face images, the real details of the frame of face images are enhanced, the size of the frame of face images is enlarged, the resolution of the frame of face images is improved, and the frame of face images with enlarged size and improved resolution is the fourth face image.
And carrying out image matching on the third face image and the fourth face image, so that matching points which represent the same content in the third face image and the fourth face image can be obtained, according to the matching points which represent the same content in the third face image and the fourth face image, first pixels in the third face image and sub-pixels which are matched with the first pixels in the fourth face image can be obtained, and according to the positions of the sub-pixels in the fourth face image, four pixels adjacent to the sub-pixels in the fourth face image and pixel values of the four pixels can be obtained. It should be noted that, the first pixel in the third face image is obtained by performing image matching on the third face image and the fourth face image, and for different frame face images in the remaining N-1 frame face images, the fourth face image corresponding to the different frame face images may be different, and for different frame face images, the first pixel in the third face image may also be different, for example, for two different frame face images (for example, the first frame face image and the second frame face image) in the remaining N-1 frame face images, after performing image matching on the third face image and the fourth face image corresponding to the first frame face image, the coordinate of the first pixel in the third face image is obtained as (1, 1), and after performing image matching on the fourth face image corresponding to the third face image and the second frame face image, the coordinate of the first pixel in the third face image is obtained as (1, 2).
Optionally, the obtaining the pixel value of the sub-pixel in the fourth face image according to the pixel values of four pixels adjacent to the sub-pixel in the fourth face image includes:
acquiring offset of four pixels adjacent to the sub-pixel and the sub-pixel in the fourth face image;
and acquiring the pixel value of the sub-pixel in the fourth face image according to the offset of the four pixels adjacent to the sub-pixel in the fourth face image and the sub-pixel and the pixel value of the four pixels adjacent to the sub-pixel in the fourth face image.
FIG. 2 is a diagram of an example of subpixel interpolationIntermediate OP 1,1 、OP 1,2 、OP 2,1 、OP 2,2 Pixels in the third face image; OP (optical path) 1,1 IP for the first pixel in the third face image and the sub-pixel in the fourth face image 1,1 、IP 1,2 、IP 2,1 、IP 2,2 Four pixels adjacent to the sub-pixel in the fourth face image. The pixel values of the sub-pixels in the fourth face image may be calculated using the following formula:
Figure BDA0002352426800000081
IP in the above formula 1,1 、IP 1,2 、IP 2,1 、IP 2,2 Respectively pixels IP 1,1 Pixel value, pixel IP of (a) 1,2 Pixel value, pixel IP of (a) 2,1 Pixel value, pixel IP of (a) 2,2 The weight used in the formula is the offset.
Optionally, the performing image matching on the third face image and the fourth face image, and obtaining the first pixel in the third face image and the sub-pixel matched with the first pixel in the fourth face image includes:
And performing image matching on the third face image and the fourth face image to obtain a point matched with a pixel in the third face image in the fourth face image, if the coordinate of the point in the fourth face image is a decimal, determining the point as a sub-pixel of the fourth face image, wherein the pixel matched with the point in the third face image is a first pixel in the third face image.
Step S104, inputting the first face image into a preset neural network to obtain a second face image.
The size of the second face image is larger than that of the first face image, and the resolution of the second face image is larger than that of the first face image. Optionally, the user may set the size and resolution of the second face image according to the actual requirement.
The preset neural network may refer to a preset neural network for improving the resolution of the first face image, including but not limited to, deep rearrangement of the images into a spatial data block depth-to-space layer. After multi-frame superdivision, a deep learning superdivision network (namely a preset neural network) is added, and the deep learning superdivision model can remove noise and blur and improve the resolution on the basis of ensuring the unchanged face characteristics.
Step S105, recognizing a face in the second face image.
In this embodiment of the present application, the face in the second face image may be identified by a preset face detection algorithm, where the preset face detection algorithm may be a preset algorithm for face recognition, and the specific algorithm is not limited herein.
According to the method and the device, the continuous N-frame initial image aiming at the target face is firstly obtained, then the face image of the target face in each frame of initial image is obtained, sub-pixel interpolation is carried out on the N-frame face image, the first face image with amplified size and resolution is obtained, the first face image is input into the preset neural network, image enhancement is carried out on the first face image, the size and resolution of the first face image are further amplified, namely, the method and the device can effectively improve the quantity of real details in the face image by combining multi-frame super-resolution with deep learning, and further improve face recognition accuracy.
Referring to fig. 3, a schematic implementation flow chart of a face recognition method provided in a second embodiment of the present application, where the face recognition method is applied to a mobile terminal, as shown in the figure, the face recognition method may include the following steps:
Step S301, the light intensity of the current environment and/or the size of the target face in the picture are obtained.
The current environment may refer to a current photographing environment, and the light intensity of the current environment may be obtained through a light sensor in the mobile terminal. The picture may refer to a picture captured by the image pickup device.
Step S302, if the light intensity of the current environment is less than the intensity threshold and/or the size of the target face in the frame is less than the size threshold, acquiring initial images with the same exposure degree for N continuous frames of the target face.
In the embodiment of the application, if the light intensity of the current photographing environment is smaller than the intensity threshold, the light of the current photographing environment is poor, and imaging quality is affected, so that continuous N frames of initial images with the same exposure degree aiming at a target face can be obtained, the N frames of initial images with the same exposure degree are utilized to obtain a high-resolution face image, and further face recognition accuracy is improved; if the size of the target face in the image captured by the image capturing device is smaller than the size threshold, the fact that the target face in the image is smaller and the face recognition precision is affected is indicated, so that continuous N frames of initial images with the same exposure degree aiming at the target face can be obtained, the N frames of initial images with the same exposure degree are utilized to obtain high-resolution face images, and the face recognition precision is further improved; if the light intensity of the current photographing environment is smaller than the intensity threshold and the size of the target face in the picture captured by the image pickup device is smaller than the size threshold, the light of the current photographing environment is poorer, the target face in the picture is smaller, and the face recognition precision is affected, so that the initial images with the same exposure degree for the continuous N frames of the target face can be obtained, the face images with high resolution can be obtained by utilizing the initial images with the same exposure degree for the N frames, and the face recognition precision is further improved.
Step S303, obtaining face images of the target face in each frame of initial image in the N frames of initial images.
The step is the same as step S102, and the detailed description of step S102 will be omitted herein.
Optionally, the acquiring the face image of the target face in each frame of initial image includes:
selecting one frame of initial image from the N frames of initial images as a reference initial image;
acquiring the target face from the reference initial image;
and acquiring face images of the target face in each frame of initial image in the N frames of initial images according to the target face acquired in the reference initial image.
In the embodiment of the application, the definition of each frame of initial image in the N frames of initial images can be acquired, the initial image with the highest definition in the N frames of initial images is used as a reference initial image, the target face needing superdivision can be acquired from the reference initial image, and then the N frames of initial images are cut according to the target face determined from the reference initial image, so that the face image of the target face in the N frames of initial images is cut.
Step S304, sub-pixel interpolation is carried out on the N frames of face images, and a first face image is obtained.
The step is the same as step S103, and specific reference may be made to the description related to step S103, which is not repeated here.
Step S305, inputting the first face image to a preset neural network, and obtaining a second face image.
The step is the same as step S104, and the detailed description of step S104 will be omitted herein.
As shown in fig. 4, which is an exemplary face image processing procedure diagram, a continuous five-frame initial image for a target face is acquired first, then one frame is selected as a reference initial image, the target face is selected from the reference initial image, the five-frame initial image is cut according to the target face, the five-frame face image is cut, sub-pixel interpolation is performed on the five-frame face image, a first face image is obtained, and then the first face image is input into a preset neural network, and a second face image is obtained.
Step S306, recognizing a face in the second face image.
The step is the same as step S105, and specific reference may be made to the description related to step S105, which is not repeated here.
According to the embodiment of the application, when the face is smaller in a picture or the light of the current photographing environment is weaker, the quantity of real details in the face image can be effectively improved through combination of multi-frame super-resolution and deep learning, and then face recognition accuracy is improved.
Referring to fig. 5, a schematic structural diagram of a face recognition device according to a third embodiment of the present application is shown, for convenience of explanation, only a portion related to the embodiment of the present application is shown.
The face recognition device includes:
an initial image acquisition module 51, configured to acquire N continuous frames of initial images for a target face, where N is an integer greater than 1;
the face image obtaining module 52 is configured to obtain a face image of the target face in each of the N frame initial images, where the face image of the target face in each of the N frame initial images is an image of an area where the target face in each of the N frame initial images is located, the N frame initial images correspond to the N frame face images, and the N frame face images have the same size;
a subpixel interpolation module 53, configured to perform subpixel interpolation on the N frames of face images to obtain a first face image, where a size of the first face image is greater than a size of each frame of face image, and a resolution of the first face image is greater than a resolution of each frame of face image;
the face image input module 54 is configured to input the first face image to a preset neural network to obtain a second face image, where a size of the second face image is greater than a size of the first face image, and a resolution of the second face image is greater than a resolution of the first face image;
And the face recognition module 55 is used for recognizing the face in the second face image.
Optionally, the subpixel interpolation module 53 includes:
the face selecting unit is used for selecting one frame of face image from the N frames of face images to serve as a reference face image;
a first obtaining unit, configured to perform bilinear interpolation on the reference face image to obtain a third face image, where a size of the third face image is the same as a size of the first face image, and a resolution of the third face image is greater than a resolution of the reference face image;
the second obtaining unit is used for performing bilinear interpolation on an ith frame of face image in the remaining N-1 frame of face images to obtain a fourth face image, wherein the remaining N-1 frame of face images refer to face images except the reference face image in the N frame of face images, the size of the fourth face image is the same as that of the first face image, the resolution of the fourth face image is larger than that of the ith frame of face image, and i is an integer larger than zero and smaller than or equal to N-1;
the first acquisition unit is used for carrying out image matching on the third face image and the fourth face image, and acquiring a first pixel in the third face image and a sub-pixel matched with the first pixel in the fourth face image;
A second obtaining unit, configured to obtain pixel values of four pixels adjacent to the subpixel in the fourth face image;
a third obtaining unit, configured to obtain pixel values of the sub-pixels in the fourth face image according to pixel values of four pixels adjacent to the sub-pixels in the fourth face image;
a pixel determining unit, configured to add and average a pixel value of the first pixel in the third face image and a pixel value of the sub-pixel in the fourth face image, and use the average value as the pixel value of the first pixel in the third face image;
the image determining unit is configured to repeatedly execute the second obtaining unit, the first obtaining unit, the second obtaining unit, the third obtaining unit and the pixel determining unit until the remaining N-1 frame face images are traversed, and determine the processed third face image as the first face image, where the processed third face image is the third face image in which the pixel values of the first pixels are subjected to addition and averaging processing.
Optionally, the third obtaining unit is specifically configured to:
acquiring offset of four pixels adjacent to the sub-pixel and the sub-pixel in the fourth face image;
And acquiring the pixel value of the sub-pixel in the fourth face image according to the offset of the four pixels adjacent to the sub-pixel in the fourth face image and the sub-pixel and the pixel value of the four pixels adjacent to the sub-pixel in the fourth face image.
Optionally, the first obtaining unit is specifically configured to:
performing image matching on the third face image and the fourth face image to obtain points matched with pixels in the third face image in the fourth face image;
and if the coordinates of the point in the fourth face image are decimal, determining that the point is a sub-pixel of the fourth face image, and determining that the pixel matched with the point in the third face image is a first pixel in the third face image.
Optionally, the face recognition device further includes:
the parameter acquisition module is used for acquiring the light intensity of the current environment and/or the size of a target face in a picture;
the initial image acquisition module 51 is specifically configured to:
and if the light intensity of the current environment is smaller than an intensity threshold value and/or the size of the target face in the picture is smaller than a size threshold value, acquiring continuous N-frame initial images aiming at the target face.
Optionally, the exposure degrees of the N frame initial images are the same.
Optionally, the face image acquisition module 52 includes:
the image selecting unit is used for selecting one frame of initial image from the N frames of initial images as a reference initial image;
a face acquisition unit, configured to acquire the target face from the reference initial image;
the image acquisition unit is used for acquiring a face image of the target face in each initial image in the N frames of initial images according to the target face acquired in the reference initial image.
The face recognition device provided in the embodiment of the present application may be applied to the first and second embodiments of the foregoing method, and details refer to the description of the first and second embodiments of the foregoing method, which are not repeated herein.
Fig. 6 is a schematic structural diagram of a mobile terminal according to a fourth embodiment of the present application. The mobile terminal as shown may include: one or more processors 601 (only one shown in the figure); one or more input devices 602 (only one shown in the figure), one or more output devices 603 (only one shown in the figure), and a memory 604. The processor 601, input device 602, output device 603, and memory 604 are connected by a bus 605. The memory 604 is used for storing instructions and the processor 601 is used for executing the instructions stored by the memory 604. Wherein:
The processor 601 is configured to obtain continuous N frames of initial images for a target face, where N is an integer greater than 1; acquiring face images of the target face in each frame of initial images in the N frames of initial images, wherein the face images of the target face in each frame of initial images refer to images of areas where the target face is located in each frame of initial images, the N frames of initial images correspond to the N frames of face images, and the N frames of face images have the same size; sub-pixel interpolation is carried out on the N frames of face images to obtain a first face image, wherein the size of the first face image is larger than that of each frame of face image, and the resolution of the first face image is larger than that of each frame of face image; inputting the first face image into a preset neural network to obtain a second face image, wherein the size of the second face image is larger than that of the first face image, and the resolution of the second face image is larger than that of the first face image; and recognizing the face in the second face image.
Optionally, the processor 601 is specifically configured to:
selecting a frame of face image from the N frames of face images as a reference face image;
Performing bilinear interpolation on the reference face image to obtain a third face image, wherein the size of the third face image is the same as that of the first face image, and the resolution of the third face image is larger than that of the reference face image;
step a1, performing bilinear interpolation on an ith frame of face image in the remaining N-1 frame of face images to obtain a fourth face image, wherein the remaining N-1 frame of face image refers to face images except the reference face image in the N frame of face images, the size of the fourth face image is the same as that of the first face image, the resolution of the fourth face image is larger than that of the ith frame of face image, and i is an integer larger than zero and smaller than or equal to N-1;
step a2, performing image matching on the third face image and the fourth face image, and obtaining a first pixel in the third face image and a sub-pixel matched with the first pixel in the fourth face image;
step a3, obtaining pixel values of four pixels adjacent to the sub-pixel in the fourth face image;
step a4, obtaining the pixel value of the sub-pixel in the fourth face image according to the pixel values of four pixels adjacent to the sub-pixel in the fourth face image;
Step a5, adding and averaging the pixel value of the first pixel in the third face image and the pixel value of the sub-pixel in the fourth face image, and taking the average value as the pixel value of the first pixel in the third face image;
and repeatedly executing the steps a1, a2, a3, a4 and a5 until the rest N-1 frames of face images are traversed, and determining the processed third face image as the first face image, wherein the processed third face image is the third face image of which the pixel values of the first pixels are subjected to addition and averaging processing.
Optionally, the processor 601 is specifically configured to:
acquiring offset of four pixels adjacent to the sub-pixel and the sub-pixel in the fourth face image;
and acquiring the pixel value of the sub-pixel in the fourth face image according to the offset of the four pixels adjacent to the sub-pixel in the fourth face image and the sub-pixel and the pixel value of the four pixels adjacent to the sub-pixel in the fourth face image.
Optionally, the processor 601 is specifically configured to:
performing image matching on the third face image and the fourth face image to obtain points matched with pixels in the third face image in the fourth face image;
And if the coordinates of the point in the fourth face image are decimal, determining that the point is a sub-pixel of the fourth face image, and determining that the pixel matched with the point in the third face image is a first pixel in the third face image.
Optionally, before acquiring the N consecutive frames of initial images for the target face, the processor 601 is further configured to:
and acquiring the light intensity of the current environment and/or the size of the target face in the picture.
Optionally, the processor 601 is specifically configured to:
and if the light intensity of the current environment is smaller than an intensity threshold value and/or the size of the target face in the picture is smaller than a size threshold value, acquiring continuous N-frame initial images aiming at the target face.
Optionally, the exposure degrees of the N frame initial images are the same.
Optionally, the processor 601 is specifically configured to:
selecting one frame of initial image from the N frames of initial images as a reference initial image;
acquiring the target face from the reference initial image;
and acquiring a face image of the target face in each initial image in the N frames of initial images according to the target face acquired in the reference initial image.
It should be appreciated that in embodiments of the present application, the processor 601 may be a central processing unit (Central Processing Unit, CPU), which may also be other general purpose processors, digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), off-the-shelf programmable gate arrays (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The input device 602 may include a touch pad, a fingerprint sensor (for collecting fingerprint information of a user and direction information of a fingerprint), a microphone, a data receiving interface, and the like. The output device 603 may include a display (LCD, etc.), a speaker, a data transmission interface, etc.
The memory 604 may include read only memory and random access memory and provides instructions and data to the processor 601. A portion of memory 604 may also include non-volatile random access memory. For example, the memory 604 may also store information of device type.
In a specific implementation, the processor 601, the input device 602, the output device 603, and the memory 604 described in the embodiments of the present application may perform the implementation described in the embodiments of the face recognition method provided in the embodiments of the present application, and may also perform the implementation described in the face recognition device described in the third embodiment, which is not described herein again.
Fig. 7 is a schematic structural diagram of a mobile terminal according to a fifth embodiment of the present application. As shown in fig. 7, the mobile terminal 7 of this embodiment includes: one or more processors 70 (only one shown), a memory 71, and a computer program 72 stored in the memory 71 and executable on the at least one processor 70. The processor 70, when executing the computer program 72, implements the steps of the various face recognition method embodiments described above. The mobile terminal 7 may be a computing device such as a desktop computer, a notebook computer, a palm computer, a cloud server, etc. The mobile terminal may include, but is not limited to, a processor 70, a memory 71. It will be appreciated by those skilled in the art that fig. 7 is merely an example of the mobile terminal 7 and is not intended to limit the mobile terminal 7, and may include more or fewer components than shown, or may combine certain components, or different components, e.g., the mobile terminal may further include an input-output device, a network access device, a bus, etc.
The processor 70 may be a central processing unit, CPU, or other general purpose processor, digital signal processor, DSP, application specific integrated circuit, ASIC, off-the-shelf programmable gate array, FPGA, or other programmable logic device, discrete gate or transistor logic device, discrete hardware components, or the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The memory 71 may be an internal storage unit of the mobile terminal 7, such as a hard disk or a memory of the mobile terminal 7. The memory 71 may be an external storage device of the mobile terminal 7, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card) or the like, which are provided on the mobile terminal 7. Further, the memory 71 may also include both an internal storage unit and an external storage device of the mobile terminal 7. The memory 71 is used for storing the computer program as well as other programs and data required by the mobile terminal. The memory 71 may also be used for temporarily storing data that has been output or is to be output.
It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-described division of the functional units and modules is illustrated, and in practical application, the above-described functional distribution may be performed by different functional units and modules according to needs, i.e. the internal structure of the apparatus is divided into different functional units or modules to perform all or part of the above-described functions. The functional units and modules in the embodiment may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit, where the integrated units may be implemented in a form of hardware or a form of a software functional unit. In addition, specific names of the functional units and modules are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the present application. The specific working process of the units and modules in the above system may refer to the corresponding process in the foregoing method embodiment, which is not described herein again.
In the foregoing embodiments, the descriptions of the embodiments are emphasized, and in part, not described or illustrated in any particular embodiment, reference is made to the related descriptions of other embodiments.
Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
In the embodiments provided in the present application, it should be understood that the disclosed apparatus/mobile terminal and method may be implemented in other manners. For example, the apparatus/mobile terminal embodiments described above are merely illustrative, e.g., the division of the modules or units is merely a logical function division, and there may be additional divisions in actual implementation, e.g., multiple units or components may be combined or integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection via interfaces, devices or units, which may be in electrical, mechanical or other forms.
The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional unit in each embodiment of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.
The integrated modules/units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the present application may implement all or part of the flow of the method of the above embodiment, or may be implemented by a computer program to instruct related hardware, where the computer program may be stored in a computer readable storage medium, and when the computer program is executed by a processor, the computer program may implement the steps of each method embodiment described above. Wherein the computer program comprises computer program code which may be in source code form, object code form, executable file or some intermediate form etc. The computer readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a U disk, a removable hard disk, a magnetic disk, an optical disk, a computer Memory, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), an electrical carrier signal, a telecommunications signal, a software distribution medium, and so forth. It should be noted that the computer readable medium contains content that can be appropriately scaled according to the requirements of jurisdictions in which such content is subject to legislation and patent practice, such as in certain jurisdictions in which such content is subject to legislation and patent practice, the computer readable medium does not include electrical carrier signals and telecommunication signals.
The present application implementing all or part of the flow of the method of the above embodiments may also be implemented by a computer program product, which when run on a mobile terminal, causes the mobile terminal to implement the steps of the method embodiments described above.
The above embodiments are only for illustrating the technical solution of the present application, and are not limiting; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present application, and are intended to be included in the scope of the present application.

Claims (9)

1. A face recognition method, characterized in that the face recognition method comprises:
acquiring continuous N frames of initial images aiming at a target face, wherein N is an integer greater than 1;
acquiring face images of the target face in each frame of initial images in the N frames of initial images, wherein the face images of the target face in each frame of initial images refer to images of areas where the target face is located in each frame of initial images, the N frames of initial images correspond to the N frames of face images, and the N frames of face images have the same size;
Sub-pixel interpolation is carried out on the N frames of face images so as to realize multi-frame super division, and a frame of first face image is obtained, wherein the size of the first face image is larger than that of each frame of face image, and the resolution of the first face image is larger than that of each frame of face image;
inputting the first face image into a preset neural network to obtain a second face image, wherein the size of the second face image is larger than that of the first face image, and the resolution of the second face image is larger than that of the first face image;
identifying a face in the second face image;
the sub-pixel interpolation is performed on the N frames of face images to realize multi-frame super-division, and obtaining a frame of first face image includes:
selecting a frame of face image from the N frames of face images as a reference face image;
performing bilinear interpolation on the reference face image to obtain a third face image, wherein the size of the third face image is the same as that of the first face image, and the resolution of the third face image is larger than that of the reference face image;
step a1, performing bilinear interpolation on an ith frame of face image in the remaining N-1 frame of face images to obtain a fourth face image, wherein the remaining N-1 frame of face image refers to face images except the reference face image in the N frame of face images, the size of the fourth face image is the same as that of the first face image, the resolution of the fourth face image is larger than that of the ith frame of face image, and i is an integer larger than zero and smaller than or equal to N-1;
Step a2, performing image matching on the third face image and the fourth face image, and obtaining a first pixel in the third face image and a sub-pixel matched with the first pixel in the fourth face image;
step a3, obtaining pixel values of four pixels adjacent to the sub-pixel in the fourth face image;
step a4, obtaining the pixel value of the sub-pixel in the fourth face image according to the pixel values of four pixels adjacent to the sub-pixel in the fourth face image;
step a5, adding and averaging the pixel value of the first pixel in the third face image and the pixel value of the sub-pixel in the fourth face image, and taking the average value as the pixel value of the first pixel in the third face image;
and repeatedly executing the steps a1, a2, a3, a4 and a5 until the rest N-1 frames of face images are traversed, and determining the processed third face image as the first face image, wherein the processed third face image is the third face image of which the pixel values of the first pixels are subjected to addition and averaging processing.
2. The face recognition method of claim 1, wherein the obtaining the pixel values of the sub-pixels in the fourth face image according to the pixel values of four pixels adjacent to the sub-pixels in the fourth face image comprises:
Acquiring offset of four pixels adjacent to the sub-pixel and the sub-pixel in the fourth face image;
and acquiring the pixel value of the sub-pixel in the fourth face image according to the offset of the four pixels adjacent to the sub-pixel in the fourth face image and the sub-pixel and the pixel value of the four pixels adjacent to the sub-pixel in the fourth face image.
3. The face recognition method of claim 1, wherein the image matching the third face image with the fourth face image, and obtaining a first pixel in the third face image and a subpixel in the fourth face image that matches the first pixel, comprises:
performing image matching on the third face image and the fourth face image to obtain points matched with pixels in the third face image in the fourth face image;
and if the coordinates of the point in the fourth face image are decimal, determining that the point is a sub-pixel of the fourth face image, and determining that the pixel matched with the point in the third face image is a first pixel in the third face image.
4. The face recognition method of claim 1, further comprising, prior to acquiring the consecutive N frames of initial images for the target face:
Acquiring the light intensity of the current environment and/or the size of a target face in a picture;
correspondingly, the acquiring the continuous N frames of initial images aiming at the target face comprises the following steps:
and if the light intensity of the current environment is smaller than an intensity threshold value and/or the size of the target face in the picture is smaller than a size threshold value, acquiring continuous N-frame initial images aiming at the target face.
5. The face recognition method of claim 1, wherein the exposure of the N initial images is the same.
6. The face recognition method according to any one of claims 1 to 5, wherein the acquiring the face image of the target face in each of the N frames of initial images includes:
selecting one frame of initial image from the N frames of initial images as a reference initial image;
acquiring the target face from the reference initial image;
and acquiring a face image of the target face in each initial image in the N frames of initial images according to the target face acquired in the reference initial image.
7. A face recognition device, characterized in that the face recognition device comprises:
the initial image acquisition module is used for acquiring continuous N frames of initial images aiming at a target face, wherein N is an integer greater than 1;
The face image acquisition module is used for acquiring the face image of the target face in each frame of initial image in the N frames of initial images, wherein the face image of the target face in each frame of initial image refers to an image of an area where the target face is located in each frame of initial image, the N frames of initial images correspond to the N frames of face images, and the N frames of face images are identical in size;
the sub-pixel interpolation module is used for carrying out sub-pixel interpolation on the N frames of face images so as to realize multi-frame super division and obtain a frame of first face image, wherein the size of the first face image is larger than that of each frame of face image, and the resolution of the first face image is larger than that of each frame of face image;
the face image input module is used for inputting the first face image into a preset neural network to obtain a second face image, wherein the size of the second face image is larger than that of the first face image, and the resolution of the second face image is larger than that of the first face image;
the face recognition module is used for recognizing the face in the second face image;
the subpixel interpolation module includes:
The face selecting unit is used for selecting one frame of face image from the N frames of face images to serve as a reference face image;
a first obtaining unit, configured to perform bilinear interpolation on the reference face image to obtain a third face image, where a size of the third face image is the same as a size of the first face image, and a resolution of the third face image is greater than a resolution of the reference face image;
the second obtaining unit is used for performing bilinear interpolation on an ith frame of face image in the remaining N-1 frame of face images to obtain a fourth face image, wherein the remaining N-1 frame of face images refer to face images except the reference face image in the N frame of face images, the size of the fourth face image is the same as that of the first face image, the resolution of the fourth face image is larger than that of the ith frame of face image, and i is an integer larger than zero and smaller than or equal to N-1;
the first acquisition unit is used for carrying out image matching on the third face image and the fourth face image, and acquiring a first pixel in the third face image and a sub-pixel matched with the first pixel in the fourth face image;
A second obtaining unit, configured to obtain pixel values of four pixels adjacent to the subpixel in the fourth face image;
a third obtaining unit, configured to obtain pixel values of the sub-pixels in the fourth face image according to pixel values of four pixels adjacent to the sub-pixels in the fourth face image;
a pixel determining unit, configured to add and average a pixel value of the first pixel in the third face image and a pixel value of the sub-pixel in the fourth face image, and use the average value as the pixel value of the first pixel in the third face image;
and the image determining unit is used for repeatedly executing the second obtaining unit, the first obtaining unit, the second obtaining unit, the third obtaining unit and the pixel determining unit until the residual N-1 frame face images are traversed, and determining the processed third face image as the first face image, wherein the processed third face image is the third face image with the pixel value of the first pixel subjected to addition and averaging processing.
8. A mobile terminal comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that the processor implements the steps of the face recognition method according to any one of claims 1 to 6 when the computer program is executed.
9. A computer readable storage medium storing a computer program, characterized in that the computer program when executed by a processor implements the steps of the face recognition method according to any one of claims 1 to 6.
CN201911421186.7A 2019-12-31 2019-12-31 Face recognition method, face recognition device and mobile terminal Active CN111222446B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201911421186.7A CN111222446B (en) 2019-12-31 2019-12-31 Face recognition method, face recognition device and mobile terminal

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201911421186.7A CN111222446B (en) 2019-12-31 2019-12-31 Face recognition method, face recognition device and mobile terminal

Publications (2)

Publication Number Publication Date
CN111222446A CN111222446A (en) 2020-06-02
CN111222446B true CN111222446B (en) 2023-05-16

Family

ID=70808321

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201911421186.7A Active CN111222446B (en) 2019-12-31 2019-12-31 Face recognition method, face recognition device and mobile terminal

Country Status (1)

Country Link
CN (1) CN111222446B (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112509144B (en) * 2020-12-09 2024-08-27 深圳云天励飞技术股份有限公司 Face image processing method and device, electronic equipment and storage medium
CN115908119B (en) * 2023-01-05 2023-06-06 广州佰锐网络科技有限公司 Face image beautifying processing method and system based on artificial intelligence

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102902966A (en) * 2012-10-12 2013-01-30 大连理工大学 Super-resolution face recognition method based on deep belief networks
CN107481188A (en) * 2017-06-23 2017-12-15 珠海经济特区远宏科技有限公司 A kind of image super-resolution reconstructing method
CN107895345B (en) * 2017-11-29 2020-05-26 浙江大华技术股份有限公司 Method and device for improving resolution of face image
CN110008811A (en) * 2019-01-21 2019-07-12 北京工业职业技术学院 Face identification system and method
CN110246084B (en) * 2019-05-16 2023-03-31 五邑大学 Super-resolution image reconstruction method, system and device thereof, and storage medium

Also Published As

Publication number Publication date
CN111222446A (en) 2020-06-02

Similar Documents

Publication Publication Date Title
CN109064428B (en) Image denoising processing method, terminal device and computer readable storage medium
CN109005368B (en) High dynamic range image generation method, mobile terminal and storage medium
CN111079764B (en) Low-illumination license plate image recognition method and device based on deep learning
CN110335216B (en) Image processing method, image processing apparatus, terminal device, and readable storage medium
CN111833285B (en) Image processing method, image processing device and terminal equipment
CN109214996B (en) Image processing method and device
CN109948439B (en) Living body detection method, living body detection system and terminal equipment
CN111444555B (en) Temperature measurement information display method and device and terminal equipment
CN109286758B (en) High dynamic range image generation method, mobile terminal and storage medium
CN111667504B (en) Face tracking method, device and equipment
CN111368587B (en) Scene detection method, device, terminal equipment and computer readable storage medium
CN112214773B (en) Image processing method and device based on privacy protection and electronic equipment
CN111222446B (en) Face recognition method, face recognition device and mobile terminal
CN111429371A (en) Image processing method and device and terminal equipment
CN109064504A (en) Image processing method, device and computer storage medium
CN117496560B (en) Fingerprint line identification method and device based on multidimensional vector
CN111488779A (en) Video image super-resolution reconstruction method, device, server and storage medium
CN111340722B (en) Image processing method, processing device, terminal equipment and readable storage medium
CN112146834B (en) Method and device for measuring structural vibration displacement
CN114140481A (en) Edge detection method and device based on infrared image
CN111062279B (en) Photo processing method and photo processing device
CN111489289B (en) Image processing method, image processing device and terminal equipment
CN108629219B (en) Method and device for identifying one-dimensional code
CN113239738B (en) Image blurring detection method and blurring detection device
CN111986144B (en) Image blurring judging method, device, terminal equipment and medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant