Nothing Special   »   [go: up one dir, main page]

WO2022257766A1 - 图像处理方法、装置、设备及介质 - Google Patents

图像处理方法、装置、设备及介质 Download PDF

Info

Publication number
WO2022257766A1
WO2022257766A1 PCT/CN2022/094925 CN2022094925W WO2022257766A1 WO 2022257766 A1 WO2022257766 A1 WO 2022257766A1 CN 2022094925 W CN2022094925 W CN 2022094925W WO 2022257766 A1 WO2022257766 A1 WO 2022257766A1
Authority
WO
WIPO (PCT)
Prior art keywords
information
target
face
image
sample
Prior art date
Application number
PCT/CN2022/094925
Other languages
English (en)
French (fr)
Inventor
张小伟
刘更代
Original Assignee
百果园技术(新加坡)有限公司
张小伟
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 百果园技术(新加坡)有限公司, 张小伟 filed Critical 百果园技术(新加坡)有限公司
Publication of WO2022257766A1 publication Critical patent/WO2022257766A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
    • G06F18/2135—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods based on approximation criteria, e.g. principal component analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2411—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on the proximity to a decision surface, e.g. support vector machines
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/243—Classification techniques relating to the number of classes
    • G06F18/24323—Tree-organised classifiers
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T15/00—3D [Three Dimensional] image rendering
    • G06T15/005—General purpose rendering architectures

Definitions

  • the present application relates to the field of image processing, for example, to an image processing method, device, equipment and medium.
  • emoticon images such as emoji, etc.
  • emoticon images can be understood as a group of graphical abstract human facial expressions, which are usually used to highlight specific emotions based on instant text communication. Therefore, when people use instant messaging applications (such as WeChat, WhatsApp, Internet Instant Messaging Office (Instant Messaging Office, IMO), etc.), they often use emoticons for text communication.
  • instant messaging applications such as WeChat, WhatsApp, Internet Instant Messaging Office (Instant Messaging Office, IMO), etc.
  • the present application provides an image processing method, device, equipment, and medium to solve the problem that it is impossible to generate personalized facial expression images of target users.
  • the application provides an image processing method, including:
  • the target image contains a human face
  • the expression image of the target user is rendered; wherein, the 3D model of the target face is based on the target expression information, the The second feature information and the preset basic three-dimensional face model are determined.
  • the application provides an image processing device, including:
  • An acquisition unit configured to acquire target facial expression information and a target image of a target user
  • the processing unit is configured to, if it is determined that the target image contains a human face, then determine the first characteristic information of the preset characteristic parts on the human face in the target image and the second characteristic information of the human face;
  • the rendering unit is configured to render an expression image of the target user according to the 3D model of the target face and the stored material image corresponding to the first feature information; wherein, the 3D model of the target face is based on the determined by the target expression information, the second feature information, and a preset basic three-dimensional face model.
  • the present application provides an electronic device, which includes at least a processor and a memory, and the processor is configured to implement the above image processing method when executing a computer program stored in the memory.
  • the present application provides a computer-readable storage medium, which stores a computer program, and implements the above-mentioned image processing method when the computer program is executed by a processor.
  • FIG. 1 is a schematic flow diagram of an image processing method provided in an embodiment of the present application
  • FIG. 2 is a schematic diagram of a process for determining feature information of preset feature parts on a human face in a target image provided by an embodiment of the present application;
  • FIG. 3 is a schematic diagram of a three-dimensional vertex located in a skin area of a human face in a target image provided by an embodiment of the present application;
  • FIG. 4 is a schematic flowchart of a training method for a geometric feature extraction model provided in an embodiment of the present application
  • FIG. 5 is a schematic flow diagram of an image processing method provided by an embodiment of the present application.
  • FIG. 6 is a schematic diagram of an image processing scene provided by an embodiment of the present application.
  • FIG. 7 is a schematic diagram of a process for determining the second feature information of a face in a target image provided by an embodiment of the present application.
  • FIG. 8 is a schematic structural diagram of an image processing device provided by an embodiment of the present application.
  • FIG. 9 is a schematic structural diagram of an electronic device provided by an embodiment of the present application.
  • embodiments of the present application provide an image processing method, device, equipment, and medium.
  • Fig. 1 is a schematic flow chart of an image processing method provided by an embodiment of the present application, the method comprising:
  • S101 Obtain target expression information and a target image of a target user.
  • the image processing method provided in the embodiment of the present application is applied to an electronic device, and the electronic device may be a smart device such as a mobile terminal or a server.
  • the target image of the target user and target expression information can be input to the smart device.
  • the smart device After receiving the target image and target expression information, the smart device generates an image generation request according to the target image and target expression information and sends it to the electronic device for image processing.
  • the electronic device performing image processing may acquire the target user's target image and target facial expression information carried in the image generation request.
  • the image processing method provided by the embodiment of the present application is used to perform corresponding processing to generate the target user's expression image.
  • the target image is an image including the face of the target user
  • the target expression information is used to indicate the required expression in the generated expression image.
  • the target image may be any image selected by the target user on the display interface of the smart device, or may be an image collected by a camera of the smart device.
  • the target expression information may be determined according to an expression selected by the target user from a plurality of preset expressions displayed on the display interface of the smart device. For example, the target user selects a smiling expression from a plurality of preset expressions displayed on the display interface of the smart device, and then determines the target expression information corresponding to the smiling expression according to the correspondence between the preset expressions and expression information .
  • the target facial expression information may also be extracted from at least one image selected by the target user on the display interface of the smart device, or may be extracted from at least one image collected by the camera of the smart device.
  • the at least one image used to extract target expression information may be the same as or different from the target image.
  • the target user may input the target image and target expression information of the target user to the smart device, which may be through trigger operation input, such as click, double-click, slide and other trigger operations, or Through voice input, for example, input voice information "select X picture and W expression". It can also be input by manipulating a control device (eg, remote controller, mouse). During the specific implementation process, it can be flexibly set according to actual needs, which is not limited here.
  • trigger operation input such as click, double-click, slide and other trigger operations
  • voice input for example, input voice information "select X picture and W expression”. It can also be input by manipulating a control device (eg, remote controller, mouse).
  • a control device eg, remote controller, mouse
  • the target image acquired by the electronic device for image processing may not contain a human face, making it impossible to personalize the expression image of the target user based on the face in the target image. Therefore, in the embodiment of the present application, after the target image is acquired, it may be determined whether the target image contains a human face. When it is determined that the target image contains a human face, it means that the expression image of the target user can be generated, and then the subsequent steps can be performed, that is, determining the first feature information of the preset feature parts on the human face in the target image and the second feature of the human face information.
  • LBP Local Binary Patterns
  • the target image can be determined according to the key points on the human face in the target image
  • the target image is updated according to the sub-image containing all the pixels corresponding to the face.
  • the first feature information of preset feature parts on the face and the second feature information of the face are determined in the sub-image.
  • the process of obtaining the key point information on the face of the target image belongs to the related technology, which is not limited here.
  • the characteristic parts on the human face include at least one of the following: hair, eyebrows, eyes, mouth, facial decorations, pupils, and the like.
  • the first characteristic information of the preset characteristic parts on the human face may include hair information on the human face, such as hairstyle, hair color, eyebrow shape, eyebrow color, beard shape, beard color, etc.
  • Facial features information such as the shape information of the facial features, the ratio information of the facial features and the face, pupil color information, etc., the shape information of the face, such as face shape, etc., the decoration information on the face, such as the shape information of glasses, the color of glasses Information, card issuance shape information, ear jewelry shape information, etc., skin information of the face, such as skin color, distribution information of moles, skin texture information, etc., identity information of the face, such as age, gender, etc.
  • Facial features information such as the shape information of the facial features, the ratio information of the facial features and the face, pupil color information, etc.
  • the shape information of the face such as face shape, etc.
  • the decoration information on the face such as the shape information of glasses, the color of glasses Information, card issuance shape information, ear jewelry shape information, etc.
  • skin information of the face such as skin color, distribution information of moles, skin texture information, etc.
  • identity information of the face such as age, gender, etc.
  • the above are several examples of the first characteristic information of the preset characteristic parts on the human face, and the first characteristic information also includes other types of information, which are not limited in this embodiment of the present application.
  • the first feature information includes one or more of the following:
  • Gender information hairstyle information, facial decoration information, hair color information, eye shape information, eyebrow shape information, pupil color information, beard type information.
  • gender information includes male and female.
  • the face decoration information takes glasses information as an example, and the glasses information may include no glasses, rimless glasses, half rim glasses, round rim glasses, square rim glasses, and sunglasses.
  • Eyebrow shape information may include arched eyebrows, S-shaped eyebrows, crescent eyebrows, straight eyebrows, character-splayed eyebrows and saber eyebrows; pupil color information may include brown, black, blue, gray, green, amber and other colors, wherein , the colors that do not belong to the first 6 colors in the pupil color information will be determined as other colors.
  • the hairstyle information may generally include female hairstyle information as well as male hairstyle information.
  • the female hairstyle information may include ultra-short hair without bangs, ultra-short hair with bangs, medium-short hair without bangs, medium-short hair with bangs, long hair with shoulders without bangs, long hair with shoulders with bangs, single-shoulder long hair, double braids, Single braid, ball head, Nezha head, other types of ultra-short hair, other types of short and medium hair, other types of long hair, and other types of braids.
  • the information on these female hairstyles is based on the length of the hair, and subdivided into whether there are bangs and Whether to braid.
  • the hairstyles can be classified into other types of ultra-short hair, other types of short and medium hair, other types of long hair, and other types of braids according to the length of the hair in the hairstyle .
  • the male hairstyle information may include bald head, short hair/very short hair, short hair without bangs, short hair with bangs, medium hair without bangs, medium hair with bangs, long hair and other types. Hairstyles that cannot be classified into the top 7 male hairstyles in the male hairstyle information will be classified into other types.
  • Beard type information may include no beard, stubble, upper lip beard, goatee, ring beard, full cheek beard, and extra long beard.
  • the first feature information of the preset feature parts on the face of the target image can be extracted through a feature extraction algorithm, or the first feature information of the preset feature parts on the face of the target image can be obtained through a pre-trained model. It can be flexibly set according to requirements, and is not limited here.
  • the second characteristic information of the human face may include at least one of the following: geometric shape information of the human face, texture color information, and expression information.
  • the target expression information is used to indicate the required expression in the generated expression image, it is not necessary to obtain the expression information of the face in the target image, and the second feature information of the face includes Geometry information and texture color information.
  • the method also includes:
  • a prompt message for re-uploading the target image can be generated and controlled.
  • the smart device of the target user outputs the prompt information.
  • the prompt information output by the smart device can be in audio format, such as a voice broadcast prompt message "The current image cannot generate an emoticon image", or the corresponding prompt information in text form can be displayed on the display interface, such as displaying prompt information on the display interface "The current image cannot generate an emoticon image", pop-up prompts, etc. It is also possible to combine the two ways of outputting prompt information, such as simultaneously broadcasting prompt information in audio format and displaying prompt information in text format on a display interface. It can be flexibly set according to actual requirements, and is not limited here.
  • S103 Render an expression image of the target user according to the 3D model of the target face and the stored material image corresponding to the first feature information; wherein, the 3D model of the target face is based on the target expression information , the second feature information and a preset basic three-dimensional face model.
  • the basic three-dimensional face model is set in advance. After obtaining the second feature information of the face in the target image, the preconfigured basic 3D face model can be adjusted according to the second feature information and target expression information to obtain the target 3D face model of the target user .
  • the process of adjusting the pre-configured basic three-dimensional face model according to the second feature information and the target expression information belongs to related technologies, and will not be introduced here.
  • the material image corresponding to the first characteristic information can be quickly determined according to the material image corresponding to each characteristic information of the preset characteristic part saved in advance.
  • the material image corresponding to the first characteristic information is rendered to the corresponding area in the target face three-dimensional model, thereby acquiring the expression image of the target user.
  • the target expression information may include expression information corresponding to faces in multiple consecutive expression images, or may only include expression information of a face in one expression image.
  • the target expression information includes expression information corresponding to faces in multiple expression images
  • the material image corresponding to the first feature information is rendered to the corresponding area in the target face three-dimensional model
  • multiple emoticon images of the target user can be acquired, thereby generating an emoticon animation of the target user.
  • GIF Graphics Interchange Format
  • the target expression information only includes the expression information of a human face in an expression image
  • the material image corresponding to the first feature information is rendered to the corresponding area in the target human face three-dimensional model , an emoticon image of the target user can be obtained.
  • the rendered emoticon image of the target user may be three-dimensional or two-dimensional.
  • it can be flexibly set according to actual needs, which is not limited here.
  • the first feature information of the preset feature parts on the face of the target image and the second feature information of the human face can be automatically determined , Reduce manual control and workload.
  • the material image corresponding to the first feature information is stored in advance, the material image that fits the image of the target user can be accurately determined later, and the material image is rendered to the image according to the target expression information, the second feature information and the target face 3D model determined by the pre-set basic 3D face model, so as to render the target user's expression image, realize personalized customization of the target user's expression image, and do not need to manually draw the target user's expression according to the target user's target image images, reducing labor costs.
  • the determining the first feature information of the preset feature parts on the face of the target image includes: determining the first feature information based on the target image through a pre-trained feature extraction model.
  • a feature extraction model is pre-trained, so that the preset features of the face in the target image can be extracted through the pre-trained feature extraction model The first feature information of the part.
  • the quantity of the feature extraction model is at least one, and any feature extraction model is used to extract at least one kind of feature information, and the feature information extracted by each feature extraction model is different.
  • feature extraction model A is only used to extract the eye shape information of the eyes on the human face and the pupil color information of the pupils on the human face in the target image
  • feature extraction model B is only used to extract the eyebrow shape information of the eyebrows on the human face in the target image
  • the feature extraction model C is used to extract the hairstyle information and hair color information of the hair on the face of the target image.
  • the pre-trained feature extraction model can be a classification network, for example, a convolutional neural network (Convolutional Neural Networks, CNN) classification model with MobileNet as a feature extraction layer, or a traditional feature extraction plus classifier Networks (such as Support Vector Machine (SVM), Random Forest), etc.
  • CNN convolutional Neural Networks
  • SVM Support Vector Machine
  • Random Forest Random Forest
  • the target image can be simultaneously input to each pre-trained feature extraction model.
  • the input target image is processed to determine the first feature information of the preset feature parts on the face of the target image.
  • feature extraction models corresponding to gender information, hair color information, eye shape information, eyebrow shape information, and pupil color information are pre-trained. Simultaneously input the target image into the feature extraction models corresponding to gender information, hair color information, eye shape information, eyebrow shape information, and pupil color information. Through the feature extraction models corresponding to gender information, hair color information, eye shape information, eyebrow shape information, and pupil color information respectively, based on the input target image, the gender information, hair color information, and eye shape of the face in the target image are obtained. information, eyebrow shape information, and pupil color information.
  • a feature extraction model corresponding to the hair color information may be used to determine the hair region of the face in the target image. For example, the mask of the area where the hair is located. Then the color of each pixel in the area is counted, so as to accurately determine the hair color information. For example, hair color information is determined according to the color corresponding to the most pixels in the region.
  • the key points on the pupil of the face in the target image may be determined through a feature extraction model corresponding to the pupil color information. Then the colors of the pixels corresponding to the multiple key points are counted, so as to accurately determine the pupil color information.
  • the feature extraction models corresponding to the various types of feature information can be determined according to the correlation between the various types of feature information.
  • the feature extraction model corresponding to the feature information that will not be affected by other feature information is used as the feature extraction model with the earlier execution order, and according to the processing results of the feature extraction model with the earlier execution order, it is determined to be executed after the feature extraction model feature extraction model.
  • the target image can be simultaneously input into each feature extraction model that is executed earlier in order for processing.
  • the processing result of the feature extraction model whose execution order is earlier determine the preset feature extraction model executed after the feature extraction model in the case of the processing result. And the target image is input into the feature extraction model in the execution sequence for processing.
  • FIG. 2 is a schematic diagram of a process of determining feature information of preset feature parts on a human face in a target image provided by an embodiment of the present application.
  • the feature extraction models corresponding to the gender information, hair color information, facial decoration information, eyebrow shape information, and pupil color information are preset as the feature extraction models that are executed first. Input the target image into the feature extraction model corresponding to gender information, hair color information, facial decoration information, eyebrow shape information and pupil color information respectively.
  • the feature extraction models corresponding to gender information, hair color information, facial decoration information, eyebrow shape information and pupil color information the gender information, hair color information, facial decoration information, eyebrow type information and pupil color information.
  • the target image is input into the feature extraction model corresponding to the pre-trained beard type information and male hairstyle information for processing, and the beard type and the beard type of the beard on the face in the target image are determined. Hairstyles information for men. If the gender information is female, the target image is input into the feature extraction model corresponding to the pre-trained female hairstyle information for processing, and the female hairstyle information of the hair on the face of the target image is determined.
  • the first sample set contains sample images (denoted as the first sample set).
  • a sample image contains a human face.
  • Each first sample image is marked, and the feature information (denoted as the third feature information) of the preset feature parts on the face of each first sample image is determined.
  • the third characteristic information can be represented by numbers, letters, character strings, etc., or in other forms, as long as the characteristic information that uniquely identifies the characteristic parts of the face in the first sample image is guaranteed.
  • the original feature extraction model is trained based on the acquired first sample image and its corresponding third feature information.
  • the electronic device for training the feature extraction model may be the same as or different from the electronic device for image processing.
  • any first sample image is acquired.
  • the fourth feature information of the preset feature parts on the human face in the first sample image is determined through the original feature extraction model.
  • a loss value of the first sample image is determined according to the fourth feature information and the third feature information of the first sample image. Based on the loss value of the first sample image, the original feature extraction model is trained to adjust the parameter values of the parameters in the original feature extraction model.
  • the above-mentioned steps are performed for each first sample image until a convergence condition is reached, then it is determined that the feature extraction model training is completed.
  • Satisfying the preset convergence condition can be that the sum of the loss values of each first sample image determined for the current iteration is less than the preset convergence threshold, or the number of iterations for training the original feature extraction model reaches the set maximum number of iterations, etc. . It can be set flexibly in specific implementation, and is not limited here.
  • the first sample image can be divided into training samples and test samples, and the original feature extraction model is trained based on the training samples first, and then the above-mentioned The reliability of the trained feature extraction model is verified.
  • the second feature information includes the texture color information
  • determine the feature information of the face includes: obtaining the 3D feature information of the human face based on the target image through a pre-trained 3D feature extraction model; based on the face geometric information and texture information contained in the 3D feature information, for The basic three-dimensional face model is adjusted to determine the three-dimensional vertices corresponding to the human face and the first texture color information of the three-dimensional vertices; according to the three-dimensional vertices and the first texture color information, determine the human face Texture color information for the face.
  • the face of the target user in the target image generally contains different levels of illumination. That is to say, the texture color of the face in the target image is mainly composed of two parts, one is the color of the face texture, and the other is the ambient light.
  • the texture color information of the face in the target image can be obtained to facilitate the subsequent generation of a three-dimensional expression of the target user based on the texture color information The skin tone of the people in the image.
  • the basic three-dimensional face model can be a Blendshape three-dimensional model, and the Blendshape three-dimensional model is formed based on three-dimensional feature information of a human face, and the three-dimensional feature information is mainly composed of three principal component analysis (Principal Components Analysis, PCA )constitute.
  • the three principal component analyzes respectively include: 1.
  • the principal component analysis of the geometric deformation caused by the corresponding expression which can be composed of multiple (for example, 64) expression Blendshape coefficients; 3.
  • the texture principal component analysis corresponding to the face the principal component analysis There may be multiple (eg, 79) texture coefficients. Therefore, by adjusting the preset Blendshape 3D model through the 3D feature information of the target face, a 3D face model similar to the face in the target image can be completely reconstructed.
  • the three-dimensional feature information of the human face in the target image is obtained.
  • the three-dimensional feature information includes 97-dimensional ID Blendshape coefficients, 64-dimensional expression Blendshape coefficients, and 79-dimensional texture coefficients.
  • the preset Blendshape three-dimensional model is processed. Adjust, so as to completely reconstruct a 3D face model similar to the face in the target image.
  • a 3D feature extraction model is pre-trained. Input the acquired target image into the pre-trained 3D feature extraction model. Through the pre-trained 3D feature extraction model, the input target image is processed, and the 3D feature information of the face in the target image can be obtained.
  • the pre-trained three-dimensional feature extraction model may be a convolutional neural network, such as a convolutional neural network using MobileNet as a feature extraction layer.
  • the expression of the person in the expression image to be generated is determined according to the target expression information, it is not necessary to perform a principal component analysis on the pre-set basic 3D face model based on the principal component analysis of the geometric deformation caused by the corresponding expression in the 3D feature information. Adjustment. Therefore, after obtaining the 3D feature information of the face, the preset basic 3D face model can be adjusted based on the face geometry information and texture information contained in the 3D feature information, so that the target image can be determined The 3D vertices corresponding to the face and the first texture color information of the 3D vertices.
  • the process of adjusting the pre-set basic 3D model of the face belongs to the related technology and will not be repeated here.
  • the first texture color information may be directly determined as the texture color information of the face, or the statistics of each For the number of 3D vertices corresponding to the first texture color information, the first texture color information corresponding to more 3D vertices is determined as the texture color information of the human face. It is also possible to calculate the mean value of the first texture color information of each three-dimensional vertex, and determine the mean value as the texture color information of the human face, and the like.
  • flexible settings can be made according to actual needs, and details will not be described here.
  • the skin color of the person in the generated emoticon image is the same as the skin color of the face in the target image, so that the person in the emoticon image fits the image of the target user more closely, and the emoticon image is more suitable for the personalization of the target user , improving the user experience.
  • determining the texture color information of the human face includes: determining the target three-dimensional vertex according to the pixel points corresponding to the human face in the target image and the three-dimensional vertex; wherein, the target The three-dimensional vertex is the three-dimensional vertex corresponding to the pixel point corresponding to the skin of the human face in the target image; according to the first texture color information of the target three-dimensional vertex, determine the second texture color information of the target three-dimensional vertex ; Determine the texture color information of the human face according to the second texture color information of the target 3D vertex.
  • the acquired first texture color information will also include the 3D vertices corresponding to the non-skin areas on the face.
  • the texture color information of the corresponding three-dimensional vertices affects the accuracy of the determined texture color information of the human face. Therefore, after the three-dimensional vertex corresponding to the face in the target image is obtained, the pixel point corresponding to the skin of the face in the target image is determined according to the pixel point corresponding to the face in the target image (denoted as the target pixel point).
  • FIG. 3 is a schematic diagram of a three-dimensional vertex located in a skin area of a human face in a target image provided by an embodiment of the present application.
  • the white points located in the skin area of the human face in FIG. 3 are three-dimensional vertices located in the skin area of the human face in the target image, that is, the target three-dimensional vertices.
  • the second texture color information is determined according to the first texture color information of the target three-dimensional vertex. Then, according to the second texture color information of the target three-dimensional vertices, the texture color information of the human face in the target image is determined.
  • the second texture color information may be determined according to the first texture color information of the target 3D vertex in the following manner:
  • Method 1 Determine the first texture color information of any target 3D vertex as the second texture color information.
  • Method 2 Determine the first texture color information of the specified target three-dimensional vertex as the second texture color information.
  • Mode 3 Process the first texture color information of each target 3D vertex through a preset mathematical function, and determine the processed texture color information as the second texture color information.
  • the second texture color information may be directly determined as the texture color information of the face, or the statistics of each The second texture color information corresponds to the number of target three-dimensional vertices, and the second texture color information corresponding to more target three-dimensional vertices is determined as the texture color information of the face. It is also possible to calculate the mean value of the second texture color information of each target 3D vertex, and determine the mean value as the texture color information of the human face, etc.
  • flexible settings can be made according to actual needs, and details will not be described here.
  • the determination of the face The feature information includes: acquiring the three-dimensional feature information of the human face based on the target image through a pre-trained geometric feature extraction model; and determining the geometric information of the human face according to the three-dimensional feature information.
  • the faces of the target user in the target image generally contain different degrees of expressions. That is to say, the 3D geometric shape of the face in the target image is mainly composed of two parts, one is the first 3D geometric shape of the face without any expression, and the other is the first 3D geometric shape added on the basis of the first 3D geometric shape
  • the second three-dimensional geometric shape obtained after the geometric deformation brought by the expression is taken into account.
  • the face geometric information of the face in the target image can be obtained, so as to facilitate subsequent generation of a 3D image of the target user based on the face geometric information 3D model of the face of the person in the facial expression image.
  • the pre-trained geometric feature extraction model based on the target image, the three-dimensional feature information of the face in the target image can be obtained. Then, according to the three-dimensional feature information, the face geometric information of the face in the target image is determined.
  • this principal component analysis can be made up of multiple (for example, 97) identity identification (ID) Blendshape coefficients; 2, corresponding
  • ID identity identification
  • this principal component analysis can be made up of multiple (for example, 64) expression Blendshape coefficients; (eg, 79) texture coefficients. Therefore, the principal component analysis of the geometric shape of the face corresponding to the expressionless face in the three-dimensional feature information can be determined as the face geometric information of the face in the target image, that is, multiple ID Blendshape coefficients can be determined as the face of the face in the target image geometric information.
  • Blendshape coefficients are determined as the face geometry information of the face in the target image. Subsequently, according to the obtained facial geometric information, the 3D face model of the target user's expressionless face can be accurately reconstructed to ensure that the face shape of the person in the expression image is consistent with the face shape of the target user.
  • the geometric feature extraction model is obtained in the following manner:
  • any sample image contained in the sample set wherein, the sample image contains a sample face; through the original geometric feature extraction model, obtain the three-dimensional feature information of the sample face in the sample image; based on the three-dimensional Feature information, adjusting the basic three-dimensional face model to determine the sample three-dimensional vertices corresponding to the sample faces and the third texture color information of the sample three-dimensional vertices; according to the third texture of the sample three-dimensional vertices
  • the color information and the pixel value of the pixel point corresponding to the sample three-dimensional vertex corresponding to the sample face in the sample image are used to train the original geometric feature extraction model.
  • the second sample set contains sample images (denoted as the second sample set).
  • sample images denoted as the second sample set.
  • the second sample image contains a human face (denoted as a sample human face).
  • the first sample image and the second sample image may be completely or partially identical, or completely different.
  • the electronic device for training the geometric feature extraction model may be the same as or different from the electronic device for image processing.
  • the three-dimensional feature information is mainly used to adjust the preset basic three-dimensional face model.
  • the 3D vertices of the face in the adjusted basic 3D face model and the texture color information of the 3D vertices can reflect the accuracy of the geometric feature extraction model to a certain extent.
  • the basic 3D face model can be adjusted based on the 3D feature information output by the original geometric feature extraction model to determine the sample face in the second sample image
  • the corresponding 3D vertex (denoted as sample 3D vertex) and the texture color information of the sample 3D vertex (denoted as third texture color information).
  • the sample 3D vertex corresponding to the sample face in each sample image in the current iteration and the third texture color information of the sample 3D vertex are used to train the original geometric feature extraction model.
  • any second sample image is acquired.
  • the three-dimensional feature information of the sample face in the second sample image is determined through the original geometric feature extraction model.
  • the basic 3D face model is adjusted to determine the sample 3D vertices corresponding to the sample face and the third texture color information of the sample 3D vertices.
  • the loss value of the second sample image is determined according to the third texture color information of each sample 3D vertex and the pixel value of the pixel point corresponding to the sample 3D vertex on the sample face in the sample image.
  • the original geometric feature extraction model is trained to adjust parameter values of parameters in the original geometric feature extraction model.
  • the method further includes: according to the sample The pixel points corresponding to the sample face in the image and the sample three-dimensional vertices determine the target sample three-dimensional vertices; wherein, the target sample three-dimensional vertices are pixel points corresponding to the skin of the sample face in the sample image The corresponding 3D vertex; according to the third texture color information of the 3D vertex of the target sample, determine the fourth texture color information of the 3D vertex of the target sample; according to the 3D vertex of the target sample and the third texture color information of the 3D vertex of the target sample Four texture color information, updating the sample 3D vertex and the third texture color information of
  • the obtained third texture color information will also include the sample face
  • the texture color information of the sample three-dimensional vertices corresponding to the upper non-skin area affects the accuracy of the determined texture color information of the sample face. Therefore, after the sample three-dimensional vertex corresponding to the sample face in the sample image is obtained based on the above-mentioned embodiment, the pixel point corresponding to the skin of the sample face in the sample image is determined according to the pixel point corresponding to the sample face in the sample image (record is the sample pixel).
  • the sample three-dimensional vertices corresponding to the sample pixel points are determined.
  • the target sample 3D vertex may be understood as a sample 3D vertex located in the skin area of the sample face in the sample image.
  • the fourth texture color information is determined. The determined sample 3D vertex and the third texture color information of the sample 3D vertex are updated according to the target sample 3D vertex and the fourth texture color information.
  • the sample 3D vertices other than the target sample 3D vertices are deleted, only the target sample 3D vertices are kept, and the fourth texture color information of the target sample 3D vertices is determined as the sample image The texture color information of the sample face.
  • the fourth texture color information may be determined according to the third texture color information of the three-dimensional vertices of the target sample in the following manner:
  • Method 1 Determine the third texture color information of any target sample 3D vertex as the fourth texture color information.
  • Method 2 Determine the third texture color information of the specified three-dimensional vertex of the target sample as the fourth texture color information.
  • Mode 3 Process the third texture color information of the three-dimensional vertices of each target sample through a preset mathematical function, and determine the processed texture color information as the fourth texture color information.
  • the average value of the third texture color information of the three-dimensional vertices of each target sample is determined through a preset mathematical function, and the average value is determined as the fourth texture color information.
  • Satisfying the preset convergence condition can be that the sum of the loss values of each second sample image determined for the current iteration is less than the preset convergence threshold, or the number of iterations for training the original geometric feature extraction model reaches the set maximum number of iterations, etc. . It can be set flexibly in specific implementation, and is not limited here.
  • the second sample image can be divided into a training sample and a test sample, and the original geometric feature extraction model is trained based on the training sample, and then the original geometric feature extraction model is trained based on the test sample. The reliability of the above-mentioned trained geometric feature extraction model is verified.
  • FIG. 4 is a schematic flow diagram of a training method for a geometric feature extraction model provided by the embodiment of the present application. The method includes:
  • S401 Acquire any second sample image in the second sample set.
  • S406 Determine an average value of the third texture color information of the three-dimensional vertices of the target sample as fourth texture color information.
  • S407 According to the target sample 3D vertex and the fourth texture color information, update the sample 3D vertex determined in S403 and the third texture color information of the sample 3D vertex.
  • the offline method is generally adopted, and the original geometric feature extraction model is trained in advance through the training device based on the second sample image in the second sample set, so as to obtain the trained geometric feature extraction model .
  • the trained geometric feature extraction model can be stored in an image processing electronic device, so as to facilitate the generation of an expression image of the target user.
  • FIG. 5 is a schematic flow chart of an image processing method provided by the embodiment of the present application. The method includes:
  • S501 Obtain target expression information and a target image of a target user.
  • S502 Determine whether the target image contains a human face, if it is determined that the target image contains a human face, execute S503, and if it is determined that the target image does not contain a human face, execute S508.
  • the process of performing face detection on the target image includes: using a pre-trained face detection model to determine whether the target image contains a human face. If it is determined that the target image contains a human face, execute S503; if it is determined that the target image does not contain a human face, execute S508.
  • key points on the face of the target image can also be determined.
  • the pixel points corresponding to the human face in the target image can be determined according to the key points on the human face in the target image.
  • the target image is updated according to the sub-image containing all the pixels corresponding to the face.
  • S503 Determine the first characteristic information of the preset characteristic parts on the human face in the target image.
  • One or more of the following first feature information gender information, hairstyle information, hair color information, eye shape information, eyebrow shape information, pupil color information, beard type information.
  • FIG. 6 is a schematic diagram of an image processing scene provided by an embodiment of the present application.
  • the first feature information of the preset feature parts on the face of the target image can be determined by the first recognition module.
  • the target image is processed by the first identification module to determine the first feature information of the preset feature parts on the face of the target image. For example, gender information, hairstyle information, hair color information, eye shape information, eyebrow shape information, pupil color information, and beard type information as shown in FIG. 6 .
  • a feature extraction model may be pre-stored in the first recognition module. Through the feature extraction model, the first feature information of the preset feature parts on the human face in the target image can be obtained.
  • the feature extraction models used to extract different types of first feature information may be the same or different.
  • S504 Determine second feature information of the face in the target image.
  • the second feature information includes one or more of the following: face geometry information, texture color information, expression information.
  • the second feature information of the face in the target image may be determined by the second identification module.
  • the target image may also be processed by the second recognition module to determine the second feature information of the face in the target image. For example, face geometry information and texture color information as shown in FIG. 6 .
  • the process of determining the texture color information of the face in the target image is as shown in FIG.
  • the CNN network shown in 7 based on the target image, obtains the three-dimensional feature information of the face in the target image. Then, based on the facial geometric information and texture information contained in the obtained 3D feature information, the basic 3D model of the face (the Blendshape 3D model shown in Figure 7) is adjusted to determine the 3D vertices corresponding to the face in the target image and the first texture color information of the 3D vertices.
  • the target 3D vertices corresponding to the pixel points corresponding to the skin of the face in the target image are determined.
  • the second texture color information is determined according to the average value of the first texture color information of each target three-dimensional vertex. Then, according to the second texture color information of the target three-dimensional vertices, the texture color information of the human face in the target image is determined.
  • the process of determining the face geometric information of the face in the target image includes: using a pre-trained geometric feature extraction model based on the target image, obtaining The three-dimensional feature information of the face in the target image; according to the three-dimensional feature information, the geometric information of the face is determined.
  • the execution order of S503 and S504 is not limited, that is, S503 and S504 can be executed at the same time, or S503 can be executed first and then S504 can be executed, or S504 can be executed first and then S503 can be executed.
  • S505 Determine a material image corresponding to the first characteristic information.
  • a material library is preset, and the material image corresponding to each kind of characteristic information of a preset characteristic part is stored in the material library.
  • the material image corresponding to the first characteristic information may be determined from the material library.
  • the corresponding relationship between each characteristic information and the material image is stored in the material library, and the material image corresponding to the first characteristic information can be determined subsequently according to the stored corresponding relationship between each characteristic information and the material image.
  • S506 Determine a target three-dimensional face model according to the target expression information, the second feature information, and a preset basic three-dimensional face model.
  • S507 Render an expression image of the target user according to the 3D model of the target face and the material image corresponding to the first feature information.
  • the rendered emoticon image may be a dynamic emoticon image or a static emoticon image.
  • the electronic device for image processing is a server
  • the generated facial expression image can be sent to the target user's smart device, so that the target user can use the facial expression image.
  • the target user may use the emoticon image in an instant messaging scenario (such as IMO).
  • the target user can also use the emoticon image in a live video scene (such as Live).
  • the identity information of the target user, the material image corresponding to the determined first characteristic information, and the second characteristic information can be stored correspondingly.
  • Subsequent target users may not need to upload the target image again, but only need to upload the target expression information.
  • the electronic device for image processing can directly determine the 3D model of the target face according to the target expression information selected by the target user, the second characteristic information and the preset basic 3D face model. And according to the 3D model of the target face and the material image corresponding to the saved first feature information, an expression image of the target user is rendered.
  • Embodiment 6 is a diagrammatic representation of Embodiment 6
  • FIG. 8 is a schematic structural diagram of an image processing device provided in the embodiment of the present application.
  • the device includes:
  • the acquisition unit 81 is configured to acquire the target expression information and the target image of the target user; the processing unit 82 is configured to determine the first preset characteristic part of the human face in the target image if it is determined that the target image contains a human face. A characteristic information and the second characteristic information of the human face; the rendering unit 83 is configured to render the expression of the target user according to the three-dimensional model of the target human face and the material image corresponding to the stored first characteristic information image; wherein, the 3D model of the target face is determined according to the target expression information, the second feature information, and a preset basic 3D model of the face.
  • the first feature information of the preset feature parts on the face of the target image and the second feature information of the human face can be automatically determined , Reduce manual control and workload.
  • the material image corresponding to the first feature information is stored in advance, the material image that fits the image of the target user can be accurately determined later, and the material image is rendered to the image according to the target expression information, the second feature information and the target face 3D model determined by the pre-set basic 3D face model, so as to render the target user's expression image, realize personalized customization of the target user's expression image, and do not need to manually draw the target user's expression according to the target user's target image images, reducing labor costs.
  • the image processing device provided in the embodiment of the present application can execute the image processing method provided in any embodiment of the present application, and has corresponding functional modules and effects for executing the method.
  • Embodiment 7 is a diagrammatic representation of Embodiment 7:
  • the electronic device includes: a processor 91, a communication interface 92, a memory 93, and a communication bus 94, wherein the processor 91, the communication interface 92, and the memory 93 pass through The communication bus 94 completes the mutual communication;
  • a computer program is stored in the memory 93, and when the program is executed by the processor 91, the processor 91 is executed to: obtain target facial expression information and a target image of the target user; if it is determined that the target image contains If there is a face, then determine the first feature information of the preset feature parts on the face in the target image and the second feature information of the face; according to the 3D model of the target face and the stored first feature information Corresponding to the material image, an expression image of the target user is rendered; wherein, the three-dimensional model of the target face is determined according to the target expression information, the second feature information, and a preset basic three-dimensional face model.
  • the implementation of the above-mentioned electronic device can refer to the implementation of the method, and the repetition will not be repeated.
  • the communication interface 92 is provided for communication between the above-mentioned electronic device and other devices.
  • the memory 93 may include a random access memory (Random Access Memory, RAM), and may also include a non-volatile memory (Non-Volatile Memory, NVM).
  • RAM Random Access Memory
  • NVM Non-Volatile Memory
  • the first feature information of the preset feature parts on the face of the target image and the second feature information of the human face can be automatically determined , Reduce manual control and workload.
  • the material image corresponding to the first feature information is stored in advance, the material image that fits the image of the target user can be accurately determined later, and the material image is rendered to the image according to the target expression information, the second feature information and the target face 3D model determined by the pre-set basic 3D face model, so as to render the target user's expression image, realize personalized customization of the target user's expression image, and do not need to manually draw the target user's expression according to the target user's target image images, reducing labor costs.
  • Embodiment 8 is a diagrammatic representation of Embodiment 8
  • the embodiments of the present application also provide a computer-readable storage medium
  • the computer-readable storage medium stores a computer program executable by a processor, when the program runs on the processor
  • the processor realizes when executing: acquiring the target expression information and the target image of the target user; if it is determined that the target image contains a human face, then determine the preset feature position of the human face in the target image The first characteristic information and the second characteristic information of the human face; according to the 3D model of the target human face and the material image corresponding to the saved first characteristic information, render the expression image of the target user; wherein, the The 3D model of the target face is determined according to the target expression information, the second feature information and a preset basic 3D model of the face.
  • the first feature information of the preset feature parts on the face of the target image and the second feature information of the human face can be automatically determined , Reduce manual control and workload.
  • the material image corresponding to the first feature information is stored in advance, the material image that fits the image of the target user can be accurately determined later, and the material image is rendered to the image according to the target expression information, the second feature information and the target face 3D model determined by the pre-set basic 3D face model, so as to render the target user's expression image, realize personalized customization of the target user's expression image, and do not need to manually draw the target user's expression according to the target user's target image images, reducing labor costs.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Computational Linguistics (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Computer Graphics (AREA)
  • Image Analysis (AREA)
  • Processing Or Creating Images (AREA)

Abstract

本文公开了一种图像处理方法、装置、设备及介质。该图像处理方法包括：获取目标表情信息及目标用户的目标图像；在确定所述目标图像中包含有人脸的情况下，确定所述目标图像中人脸上预设特征部位的第一特征信息以及所述人脸的第二特征信息；根据目标人脸三维模型、以及保存的所述第一特征信息所对应的素材图像，渲染出所述目标用户的表情图像；其中，所述目标人脸三维模型是根据所述目标表情信息、所述第二特征信息以及预先设置的基础人脸三维模型确定的。

Description

图像处理方法、装置、设备及介质
本申请要求在2021年06月10日提交中国专利局、申请号为202110646132.1的中国专利申请的优先权，该申请的全部内容通过引用结合在本申请中。
技术领域
本申请涉及图像处理领域，例如涉及一种图像处理方法、装置、设备及介质。
背景技术
随着社交网络和社交软件的兴起，表情图像(例如emoji等)变得日益流行。其中，表情图像可以理解为是一组图形化的抽象的人脸表情，通常用以在即时文字通信的基础上突出表达特定的情感。因此，人们在使用即时通信应用(比如微信，WhatsApp，互联网即时通讯办公室(Instant Messaging Office，IMO)等)时，经常会采用表情图像进行文字通信。
比较常见的表情图像一般都是开发者预先制作好并发布，所有的用户可以下载并使用这些预先制作好的表情图像。因此，这些表情图像的内容和样式一般都是固定的，且基本上是以静态彩色图标的形式出现，所有用户均使用相同的表情图像，不够个性化，无法体现用户的个人属性和特征。
发明内容
本申请提供了一种图像处理方法、装置、设备及介质，用以解决无法个性化生成目标用户的表情图像的问题。
本申请提供了一种图像处理方法，包括：
获取目标表情信息及目标用户的目标图像；
若确定所述目标图像中包含有人脸，则确定所述目标图像中人脸上预设特征部位的第一特征信息以及所述人脸的第二特征信息；
根据目标人脸三维模型、以及保存的所述第一特征信息所对应的素材图像，渲染出所述目标用户的表情图像；其中，所述目标人脸三维模型是根据所述目标表情信息、所述第二特征信息以及预先设置的基础人脸三维模型确定的。
本申请提供了一种图像处理装置，包括：
获取单元，设置为获取目标表情信息及目标用户的目标图像；
处理单元，设置为若确定所述目标图像中包含有人脸，则确定所述目标图像中人脸上预设特征部位的第一特征信息以及所述人脸的第二特征信息；
渲染单元，设置为根据目标人脸三维模型、以及保存的所述第一特征信息所对应的素材图像，渲染出所述目标用户的表情图像；其中，所述目标人脸三维模型是根据所述目标表情信息、所述第二特征信息以及预先设置的基础人脸三维模型确定的。
本申请提供了一种电子设备，所述电子设备至少包括处理器和存储器，所述处理器设置为执行存储器中存储的计算机程序时实现上述的图像处理方法。
本申请提供了一种计算机可读存储介质，其存储有计算机程序，所述计算机程序被处理器执行时实现上述的图像处理方法。
附图说明
图1为本申请实施例提供的一种图像处理方法的流程示意图；
图2为本申请实施例提供的一种确定目标图像中人脸上预设特征部位的特征信息的过程示意图；
图3为本申请实施例提供的一种位于目标图像中人脸的皮肤区域的三维顶点的示意图；
图4为本申请实施例提供的一种几何特征提取模型的训练方法的流程示意图；
图5为本申请实施例提供的一种图像处理方法的流程示意图；
图6为本申请实施例提供的一种图像处理的场景示意图；
图7为本申请实施例提供的一种确定目标图像中人脸的第二特征信息的过程示意图；
图8为本申请实施例提供的一种图像处理装置的结构示意图；
图9为本申请实施例提供的一种电子设备结构示意图。
具体实施方式
下面将结合附图对本申请进行描述，所描述的实施例仅仅是本申请一部分实施例。
为了快速生成目标用户的个性化表情图像，本申请实施例提供了一种图像处理方法、装置、设备及介质。
实施例1：
图1为本申请实施例提供的一种图像处理方法的流程示意图，该方法包括：
S101：获取目标表情信息及目标用户的目标图像。
本申请实施例提供的图像处理方法应用于电子设备，该电子设备可以是如移动终端等智能设备，也可以是服务器。
当目标用户需要生成表情图像时，可以向智能设备输入该目标用户的目标图像以及目标表情信息(比如微笑，愤怒，惊讶等)。智能设备接收到该目标图像以及目标表情信息后，根据该目标图像以及目标表情信息，生成图像生成请求并发送至进行图像处理的电子设备。进行图像处理的电子设备接收到生成目标用户的表情图像的图像生成请求后，可以获取该图像生成请求中携带的目标用户的目标图像以及目标表情信息。然后基于该目标图像以及目标表情信息，采用本申请实施例提供的图像处理的方法，进行相应的处理，以生成该目标用户的表情图像。其中，目标图像为包含目标用户的人脸的图像，目标表情信息用于指示生成的表情图像中所需具有的表情。
在一种示例中，目标图像可以是目标用户在智能设备的显示界面上选择的任一张图像，也可以通过智能设备的摄像头采集到的图像。
在一种示例中，目标表情信息可以是根据目标用户从智能设备的显示界面上所显示的预设的多个表情中选择的表情所确定的。例如，目标用户从智能设备的显示界面上所显示的预设的多个表情中，选择了一个微笑表情，然后根据预设的表情与表情信息的对应关系，确定该微笑表情对应的目标表情信息。目标表情信息也可以从目标用户在智能设备的显示界面上选择的至少一张图像中提取到的，还可以是从通过智能设备的摄像头采集到的至少一张图像中提取到的。
用于提取目标表情信息的至少一张图像可以与目标图像相同也可以不相同。
作为一种可能的实施方式，目标用户向智能设备输入该目标用户的目标图像以及目标表情信息的方式有很多，可以是通过触发操作输入，比如，单击、双击、滑动等触发操作，也可以通过语音输入，比如，输入语音信息“选择X号图片和W表情”。还可以通过操作控制设备(例如，遥控器，鼠标)输入。具体实施过程中，可以根据实际需求进行灵活设置，在此不作限定。
S102：若确定所述目标图像中包含有人脸，则确定所述目标图像中人脸上预设特征部位的第一特征信息以及所述人脸的第二特征信息。
由于可能存在目标用户误操作等情况，导致进行图像处理的电子设备获取 到的目标图像中可能不存在人脸，导致无法根据目标图像中的人脸，个性化生成目标用户的表情图像。因此，在本申请实施例中，当获取到目标图像后，可以确定该目标图像中是否包含有人脸。当确定目标图像中包含有人脸时，说明可以生成目标用户的表情图像，则可以进行后续的步骤，即确定目标图像中人脸上预设特征部位的第一特征信息以及人脸的第二特征信息。
可以通过特征提取算法，例如，局部二值模式(Local Binary Patterns，LBP)算法等，确定目标图像中是否包含有人脸，也可以通过预先训练的人脸检测模型，确定目标图像中是否包含有人脸。
可选的，为了方便后续生成目标用户的表情图像，减少生成表情图像所需的计算量，在确定目标图像中包含有人脸后，可以根据该目标图像中人脸上的关键点，确定目标图像中人脸对应的像素点。根据包含该人脸对应的所有像素点的子图像，对目标图像进行更新。在该子图像中确定人脸上预设特征部位的第一特征信息以及人脸的第二特征信息。其中，获取目标图像中人脸上的关键点信息的过程属于相关技术，在此不做限定。
在一种示例中，人脸上的特征部位包括以下的至少一项：头发、眉毛、眼睛、嘴巴、脸部装饰物、瞳孔等。基于此，人脸上的预设特征部位的第一特征信息可以包括，人脸上的毛发信息，比如，发型、发色、眉型、眉色、胡子形状、胡子颜色等，人脸上的五官信息，比如，五官的形状信息、五官与人脸的比例信息、瞳孔颜色信息等，人脸的形状信息，比如，脸型等，人脸上的装饰物信息，比如，眼镜形状信息、眼镜颜色信息、发卡形状信息、耳饰品形状信息等，人脸的皮肤信息，比如，肤色，痣的分布信息、皮肤的纹理信息等，人脸的身份信息，比如，年龄、性别等。
上述是对人脸上的预设特征部位的第一特征信息的几种举例，该第一特征信息还包括其它种类的信息，本申请实施例不作限定。
在一种示例中，所述第一特征信息包括以下的一项或多项：
性别信息、发型信息、脸部装饰物信息、头发颜色信息、眼型信息、眉型信息、瞳孔颜色信息、胡须类型信息。
比如，性别信息包括男性和女性。
脸部装饰物信息以眼镜信息为例，该眼镜信息可以包括不戴眼镜、无边框眼镜、半边框眼镜、圆形边框眼镜、方形边框眼镜以及墨镜。
眉型信息可以包括拱形眉、S形眉、新月眉、平直眉、八字眉以及大刀眉；瞳孔颜色信息可以包括棕色、黑色、蓝色、灰色、绿色、琥珀色以及其他颜色，其中，不属于瞳孔颜色信息中前6种颜色的颜色会被确定为其他颜色。
发型信息可以大体包括女性发型信息以及男性发型信息。该女性发型信息可以包括超短发无刘海、超短发有刘海、中短发无刘海、中短发有刘海、披双肩长发无刘海、披双肩长发有刘海、批单肩长发、双麻花辫、单麻花辫、丸子头、哪吒头、超短发其它类型、中短发其它类型、长发其它类型以及辫子其它类型，这些女性发型信息以头发的长短为大类，并细分了有无刘海以及是否扎辫子。对于无法归类到女性发型信息中前11种女性发型的发型可以按照该发型中头发的长短，将该发型归类到超短发其它类型、中短发其它类型、长发其它类型以及辫子其它类型中。该男性发型信息可以包括光头、寸头/超短发、短发无刘海、短发有刘海、中发无刘海、中发有刘海、长发以及其它类型。对于无法归类到男性发型信息中前7种男性发型的发型会被归类为其它类型中。
胡须类型信息可以包括无胡须、胡茬、上嘴唇胡须、山羊胡、环形胡须、全脸颊胡须以及超长胡须。
可以通过特征提取算法提取目标图像中人脸上预设特征部位的第一特征信息，也可以通过预先训练的模型获取目标图像中人脸上预设特征部位的第一特征信息。可以根据需求进行灵活设置，在此不作限定。
在一种示例中，人脸的第二特征信息可以包括以下的至少一种：人脸几何形状信息、纹理颜色信息以及表情信息。
可选的，由于目标表情信息用于指示生成的表情图像中所需具有的表情的，因此，并不需要获取目标图像中人脸的表情信息了，该人脸的第二特征信息包括人脸几何形状信息和纹理颜色信息。
在一种可能的实施方式中，所述方法还包括：
若确定所述目标图像中未包含有人脸，则输出重新上传目标图像的提示信息。
当基于本实施例中的方法，确定目标图像中未包含有人脸时，说明无法根据该目标图像生成目标用户的表情图像，则为了提高用户体验，可以生成重新上传目标图像的提示信息，并控制目标用户的智能设备输出该提示信息。
智能设备输出的提示信息可以是音频格式的提示信息，比如语音播报提示信息“当前图像无法生成表情图像”，也可以在显示界面上显示文本形式对应的提示信息，比如在显示界面上显示提示信息“当前图像无法生成表情图像”、弹框提示等方式。也可以将两种输出提示信息的方式结合，比如同时播报音频格式的提示信息以及在显示界面上显示文本格式的提示信息等。可以根据实际要求灵活设置，在此不做限定。
S103：根据目标人脸三维模型、以及保存的所述第一特征信息所对应的素 材图像，渲染出所述目标用户的表情图像；其中，所述目标人脸三维模型是根据所述目标表情信息、所述第二特征信息以及预先设置的基础人脸三维模型确定的。
为了方便生成目标用户三维的表情图像，预先设置好基础人脸三维模型。当获取到目标图像中人脸的第二特征信息后，可以根据该第二特征信息以及目标表情信息，对该预先配置的基础人脸三维模型进行调整，以获取目标用户的目标人脸三维模型。其中，根据该第二特征信息以及目标表情信息，对该预先配置的基础人脸三维模型进行调整的过程属于相关技术，在此不做介绍。
在一种可能的实施方式中，为了方便快速生成目标用户的表情图像，并使得该表情图像中的人物与目标用户的形象贴近，还预先保存有预设特征部位的每种特征信息所对应的素材图像。比如，以人脸上的特征部位为眼睛，该眼睛的特征信息为眼型为例，预先保存有每种眼型所对应的素材图像。后续在生成目标用户的表情图像时，可以快速根据预先保存的预设特征部位的每种特征信息所对应的素材图像，确定第一特征信息所对应的素材图像。将第一特征信息所对应的素材图像渲染到目标人脸三维模型中对应的区域，从而获取到目标用户的表情图像。将第一特征信息所对应的素材图像贴到目标人脸三维模型中对应的区域。比如，将丸子头所对应的素材图像贴到目标人脸三维模型中头发的对应区域。
将第一特征信息所对应的素材图像渲染到目标人脸三维模型中对应的区域的过程，本申请不做描述。
目标表情信息可以包括连续多张表情图像中人脸分别对应的表情信息，也可以只包括一张表情图像中人脸的表情信息。
在一种可能的实施例中，若目标表情信息包括多张表情图像中人脸分别对应的表情信息，则将第一特征信息所对应的素材图像渲染到目标人脸三维模型中对应的区域后，可以获取到目标用户的多张表情图像，从而生成目标用户的表情动画。生成目标用户的动态表情图像。比如，生成目标图像的图像交换格式(Graphics Interchange Format，GIF)的表情图像。
在另一种可能的实施例中，若目标表情信息只包括一张表情图像中人脸的表情信息，则将第一特征信息所对应的素材图像渲染到目标人脸三维模型中对应的区域后，可以获取到目标用户的一张表情图像。生成目标用户的静态表情图像。
在一种示例中，渲染出的目标用户的表情图像可以三维的，也可以是二维的。具体实施过程中，可以根据实际需求，进行灵活设置，在此不做限定。
采用上述的方法，在获取到了目标用户的包含有人脸的目标图像以及目标表情信息后，可以自动确定该目标图像中人脸上预设特征部位的第一特征信息以及人脸的第二特征信息，减少人工的控制和工作量。并且由于预先保存有第一特征信息对应的素材图像，使得后续可以准确地确定出与目标用户的形象贴合的素材图像，并将该素材图像渲染到根据目标表情信息、所述第二特征信息以及预先设置的基础人脸三维模型确定的目标人脸三维模型中，从而渲染出目标用户的表情图像，实现个性化定制目标用户的表情图像，无需人工根据目标用户的目标图像绘制目标用户的表情图像，减少了人工成本。
实施例2：
为了快速且准确地确定目标图像中人脸的特征信息，在上述实施例的基础上，在本实施例中，所述确定所述目标图像中人脸上预设特征部位的第一特征信息，包括：通过预先训练的特征提取模型，基于所述目标图像，确定所述第一特征信息。
为了快速且准确地确定目标图像中人脸的特征信息，在本申请实施例中，预先训练有特征提取模型，以通过预先训练的特征提取模型，可以提取到目标图像中人脸上预设特征部位的第一特征信息。其中，该特征提取模型的数量为至少一个，任一特征提取模型用于提取至少一种特征信息，每个特征提取模型所提取到的特征信息是不同的。比如，特征提取模型A只用于提取目标图像中人脸上眼睛的眼型信息以及人脸上瞳孔的瞳孔颜色信息，特征提取模型B只用于提取目标图像中人脸上眉毛的眉型信息，特征提取模型C用于提取目标图像中人脸上头发的发型信息以及头发颜色信息等。当基于上述实施例获取到目标图像后，可以通过预先训练的每个特征提取模型，基于目标图像，确定目标图像中人脸上预设特征部位的第一特征信息。
在一种示例中，预先训练的特征提取模型可以是分类网络，比如，以MobileNet作为特征提取层的卷积神经网络(Convolutional Neural Networks，CNN)分类模型，也可以是传统的特征提取加分类器的网络(比如支持向量机(Support Vector Machine，SVM)，随机森林)等。
在一种示例中，可以将目标图像同时输入到预先训练的每个特征提取模型。通过预先训练的每个特征提取模型，对输入的目标图像进行处理，确定目标图像中人脸上预设特征部位的第一特征信息。
比如，预先训练有性别信息、头发颜色信息、眼型信息、眉型信息以及瞳孔颜色信息分别对应的特征提取模型。将目标图像同时输入到性别信息、头发颜色信息、眼型信息、眉型信息以及瞳孔颜色信息分别对应的特征提取模型。分别通过性别信息、头发颜色信息、眼型信息、眉型信息以及瞳孔颜色信息分 别对应的特征提取模型，基于输入的目标图像，获取到目标图像中人脸的性别信息、头发颜色信息、眼型信息、眉型信息以及瞳孔颜色信息。
可选的，为了准确地确定头发颜色信息，可以通过头发颜色信息对应的特征提取模型，确定目标图像中人脸的头发所在区域。比如，头发所在区域的掩膜(mask)。然后对该区域中每个像素点的颜色进行统计，从而准确地确定头发颜色信息。比如，根据该区域中对应最多像素点的颜色，确定头发颜色信息。
可选的，为了准确地确定通孔颜色信息，可以通过瞳孔颜色信息对应的特征提取模型，确定目标图像中人脸的瞳孔上的关键点。然后对多个关键点分别对应的像素点的颜色进行统计，从而准确地确定瞳孔颜色信息。
在另一种可能的方式中，由于部分种类的特征信息之间存在着关联性，比如，女性的人脸一般不具有胡须类型信息，男性的人脸的发型与女性的人脸的发型不同等。因此，可以根据多种特征信息之间存在的关联性，确定多种特征信息分别对应的特征提取模型。比如，将不会受其他特征信息影响的特征信息所对应的特征提取模型作为执行顺序在前的特征提取模型，根据执行顺序在前的特征提取模型的处理结果，确定在该特征提取模型之后执行的特征提取模型。后续获取到目标图像后，可以将该目标图像同时输入到执行顺序在前的每个特征提取模型中进行处理。根据执行顺序在前的特征提取模型的处理结果，确定预先设置的在该种处理结果的情况下，在该特征提取模型之后执行的特征提取模型。并将目标图像输入到该执行顺序在后的特征提取模型中进行处理。
比如，图2为本申请实施例提供的一种确定目标图像中人脸上预设特征部位的特征信息的过程示意图。预先设置了性别信息、头发颜色信息、脸部装饰物信息、眉型信息以及瞳孔颜色信息分别对应的特征提取模型为执行顺序在前的特征提取模型。将目标图像输入到性别信息、头发颜色信息、脸部装饰物信息、眉型信息以及瞳孔颜色信息分别对应的特征提取模型。通过性别信息、头发颜色信息、脸部装饰物信息、眉型信息以及瞳孔颜色信息分别对应的特征提取模型，分别获取目标图像中人脸的性别信息、头发颜色信息、脸部装饰物信息、眉型信息以及瞳孔颜色信息。其中，若性别信息为男，则将目标图像输入到预先训练的胡须类型信息、以及男性发型信息分别对应的特征提取模型中进行处理，确定目标图像中人脸上胡须的胡须类型和人脸上头发的男性发型信息。若性别信息为女，则将目标图像输入到预先训练的女性发型信息对应的特征提取模型中进行处理，确定目标图像中人脸上头发的女性发型信息。
为了获取到特征提取模型，在本申请实施例中，需要预先收集用于训练特征提取模型的样本集(记为第一样本集)，该第一样本集中包含有样本图像(记为第一样本图像)，该第一样本图像中包含有人脸。对每张第一样本图像进行 标注，确定每张第一样本图像中人脸上预设特征部位的特征信息(记为第三特征信息)。该第三特征信息可以用数字、字母、字符串等进行表示，也可以用其它形式进行表示，只需保证唯一标识第一样本图像中人脸上特征部位的特征信息即可。后续基于获取到的第一样本图像及其对应的第三特征信息，对原始特征提取模型进行训练。
该进行特征提取模型训练的电子设备可以与进行图像处理的电子设备相同，也可以不同。
具体实施过程中，获取任一第一样本图像。通过原始特征提取模型，确定该第一样本图像中人脸上预设特征部位的第四特征信息。根据该第四特征信息以及该第一样本图像的第三特征信息，确定该第一样本图像的损失值。基于该第一样本图像的损失值，对原始特征提取模型进行训练，以调整原始特征提取模型中的参数的参数值。
由于第一样本集中存在多个第一样本图像，针对每个第一样本图像，均执行上述的步骤，直至达到收敛条件，则确定特征提取模型训练完成。
满足预设的收敛条件可以为当前迭代所确定的每个第一样本图像的损失值的和小于预设的收敛阈值，或对原始特征提取模型进行训练的迭代次数达到设置的最大迭代次数等。具体实施中可以灵活进行设置，在此不做限定。
在一种可能的实施方式中，在进行特征提取模型训练时，可以把第一样本图像分为训练样本和测试样本，先基于训练样本对原始特征提取模型进行训练，再基于测试样本对上述已训练的特征提取模型的可靠程度进行验证。
实施例3：
为了快速且准确地确定目标图像中人脸的特征信息，在上述实施例的基础上，在本申请实施例中，若所述第二特征信息包括所述纹理颜色信息，确定所述人脸的第二特征信息，包括：通过预先训练的三维特征提取模型，基于所述目标图像，获取所述人脸的三维特征信息；基于所述三维特征信息中包含的人脸几何信息以及纹理信息，对所述基础人脸三维模型进行调整，以确定所述人脸对应的三维顶点及所述三维顶点的第一纹理颜色信息；根据所述三维顶点以及所述第一纹理颜色信息，确定所述人脸的纹理颜色信息。
在实际应用过程中，目标用户的目标图像中的人脸一般会包含有不同明暗程度的光照。也就是说，目标图像中人脸的纹理颜色主要由两部分构成，一是人脸纹理的颜色，二是环境光照。为了使生成的表情图像中人物的肤色与目标图像中人脸的肤色保持一致，可以获取目标图像中人脸的纹理颜色信息，以方便后续根据该纹理颜色信息，生成该目标用户的三维的表情图像中人物的肤色。
在一种示例中，该基础人脸三维模型可以为Blendshape三维模型，该Blendshape三维模型是基于人脸的三维特征信息构成的，该三维特征信息主要由三个主成分分析(Principal Components Analysis，PCA)构成。该三个主成分分析分别包括：1、对应无表情的人脸的几何形状的主成分分析，该主成分分析可以由多个(例如，97个)身份标识(Identifier，ID)Blendshape系数组成；2、对应表情所带来的几何形状形变的主成分分析，该主成分分析可以由多个(例如，64个)表情Blendshape系数组成；3、对应人脸的纹理主成分分析，该主成分分析可以由多个(例如，79个)纹理系数。因此，通过目标人脸的三维特征信息，对预先设置的Blendshape三维模型进行调整，即可完整地重构出与目标图像中人脸相似的人脸三维模型。例如，获取到目标图像中人脸的三维特征信息，该三维特征信息包括97维ID Blendshape系数、64维表情Blendshape系数以及79维纹理系数，通过该三维特征信息，对预先设置的Blendshape三维模型进行调整，从而完整地重构出与目标图像中人脸相似的人脸三维模型。
为了快速且准确地获取人脸的三维特征信息，在本申请实施例中，预先训练有三维特征提取模型。将获取到的目标图像输入到预先训练的三维特征提取模型中。通过预先训练的三维特征提取模型，对输入的目标图像进行处理，可以获取目标图像中人脸的三维特征信息。
该预先训练的三维特征提取模型可以是卷积神经网络，比如以MobileNet作为特征提取层的卷积神经网络。
由于需要生成的表情图像中人物的表情是根据目标表情信息确定的，因此，无需根据三维特征信息中，对应表情所带来的几何形状形变的主成分分析对预先设置的基础人脸三维模型进行调整。因此，当获取到人脸的三维特征信息后，可以基于该三维特征信息中包含的人脸几何信息以及纹理信息，对该预先设置好的基础人脸三维模型进行调整，从而可以确定目标图像中人脸对应的三维顶点及三维顶点的第一纹理颜色信息。
基于该三维特征信息中包含的人脸几何信息以及纹理信息，对预先设置好的基础人脸三维模型进行调整的过程，属于相关技术，在此不做赘述。
在一种可能的实施方式中，在确定目标图像中人脸对应的三维顶点的第一纹理颜色信息后，可以直接将第一纹理颜色信息，确定为人脸的纹理颜色信息，也可以统计每种第一纹理颜色信息所对应的三维顶点的数量，将对应三维顶点较多的第一纹理颜色信息确定为人脸的纹理颜色信息。还可以计算每个三维顶点的第一纹理颜色信息的均值，将该均值确定为人脸的纹理颜色信息等。实施过程中，可以根据实际需求进行灵活设置，在此不做赘述。
采用上述的方法，可以保证生成的表情图像中人物的肤色与目标图像中人 脸的肤色相同，使得表情图像中的人物与目标用户的形象更加贴合，该表情图像更加满足目标用户的个性化，提高了用户体验。
在一种可能的实施方式中，为了提高获取到的纹理颜色信息的准确性，避免受到人脸上非皮肤区域的纹理颜色的影响，在本申请实施例中，所述根据所述三维顶点以及所述第一纹理颜色信息，确定所述人脸的纹理颜色信息，包括：根据所述目标图像中所述人脸对应的像素点以及所述三维顶点，确定目标三维顶点；其中，所述目标三维顶点为在所述目标图像中所述人脸的皮肤对应的像素点所对应的三维顶点；根据所述目标三维顶点的第一纹理颜色信息，确定所述目标三维顶点的第二纹理颜色信息；根据所述目标三维顶点的第二纹理颜色信息，确定所述人脸的纹理颜色信息。
由于获取到的目标图像中人脸对应的三维顶点中，可能包括人脸上非皮肤区域所对应的三维顶点，导致获取到的第一纹理颜色信息中，也会包含人脸上非皮肤区域所对应的三维顶点的纹理颜色信息，从而影响确定的人脸的纹理颜色信息的准确度。因此，当获取到目标图像中人脸对应的三维顶点后，根据目标图像中人脸对应的像素点，确定目标图像中人脸的皮肤所对应的像素点(记为目标像素点)。然后根据目标图像中像素点与三维顶点的对应关系，确定目标像素点所对应的三维顶点(记为目标三维顶点)。该目标三维顶点可以理解为位于目标图像中人脸的皮肤区域的三维顶点。图3为本申请实施例提供的一种位于目标图像中人脸的皮肤区域的三维顶点的示意图。如图3所示，位于图3中人脸上皮肤区域的白色的点为位于目标图像中人脸的皮肤区域的三维顶点，即目标三维顶点。
然后根据目标三维顶点的第一纹理颜色信息，确定第二纹理颜色信息。然后根据目标三维顶点的第二纹理颜色信息，确定目标图像中人脸的纹理颜色信息。
在一种可能的实施方式中，可以采用如下方式根据目标三维顶点的第一纹理颜色信息，确定第二纹理颜色信息：
方式一、将任一目标三维顶点的第一纹理颜色信息，确定为第二纹理颜色信息。
方式二、将指定的目标三维顶点的第一纹理颜色信息，确定为第二纹理颜色信息。
方式三、通过预设的数学函数，对每个目标三维顶点的第一纹理颜色信息进行处理，确定处理后的纹理颜色信息为第二纹理颜色信息。
比如，通过预设的数学函数，确定每个目标三维顶点的第一纹理颜色信息 的均值，将该均值确定为第二纹理颜色信息。
在一种可能的实施方式中，在确定目标图像中人脸对应的目标三维顶点的第二纹理颜色信息后，可以直接将第二纹理颜色信息，确定为人脸的纹理颜色信息，也可以统计每种第二纹理颜色信息所对应的目标三维顶点的数量，将对应目标三维顶点较多的第二纹理颜色信息确定为人脸的纹理颜色信息。还可以计算每个目标三维顶点的第二纹理颜色信息的均值，将该均值确定为人脸的纹理颜色信息等。实施过程中，可以根据实际需求进行灵活设置，在此不做赘述。
通过上述的方法，可以有效避免目标图像中人脸上非皮肤区域所对应的三维顶点的第一纹理颜色信息，比如，眼睛、眉毛、胡须等区域，对人脸上皮肤区域的纹理颜色的影响，保证了生成的表情图像中人物的肤色与目标图像中人脸的肤色相同。
实施例4：
为了快速且准确地确定目标图像中人脸的特征信息，在上述实施例的基础上，在本实施例中，若所述第二特征信息包括人脸几何信息，所述确定所述人脸的特征信息，包括：通过预先训练的几何特征提取模型，基于所述目标图像，获取所述人脸的三维特征信息；根据所述三维特征信息，确定所述人脸几何信息。
在实际应用过程中，目标用户的目标图像中的人脸一般会包含有不同程度的表情。也就是说，目标图像中人脸的三维几何形状主要由两部分构成，一是该人脸在没有任何表情的情况下的第一三维几何形状，二是在第一三维几何形状的基础上添加了表情所带来的几何形变后，所获取到的第二三维几何形状。为了使生成的表情图像中人物的脸型与目标图像中人脸的脸型保持一致，可以获取目标图像中人脸的人脸几何信息，以方便后续根据该人脸几何信息，生成该目标用户的三维的表情图像中人物的人脸三维模型。
在一种示例中，为了获取到人脸几何信息，需要预先训练几何特征提取模型。通过该预先训练的几何特征提取模型，基于目标图像，可以获取到目标图像中人脸的三维特征信息。然后根据该三维特征信息，确定目标图像中人脸的人脸几何信息。
该几何特征提取模型的结构和功能，已在上述实施例中进行描述，重复之处不做赘述。
由于三维特征信息中主要包括：1、对应无表情的人脸的几何形状的主成分分析，该主成分分析可以由多个(例如，97个)身份标识(ID)Blendshape系数组成；2、对应表情所带来的几何形状形变的主成分分析，该主成分分析可以 由多个(例如，64个)表情Blendshape系数组成；3、对应人脸的纹理主成分分析，该主成分分析可以由多个(例如，79个)纹理系数。因此，可以将三维特征信息中对应无表情的人脸的几何形状的主成分分析确定为目标图像中人脸的人脸几何信息，即将多个ID Blendshape系数确定为目标图像中人脸的人脸几何信息。比如，将97个ID Blendshape系数确定为目标图像中人脸的人脸几何信息。后续根据获取到的人脸几何信息，可以准确地重建出目标用户的无表情人脸的人脸三维模型，保证表情图像中人物的脸型与目标用户的人脸的脸型一致。
在一种可能的实施方式中，所述几何特征提取模型通过如下方式获取：
获取样本集中包含的任一样本图像；其中，所述样本图像中包含有样本人脸；通过原始几何特征提取模型，获取所述样本图像中所述样本人脸的三维特征信息；基于所述三维特征信息，对所述基础人脸三维模型进行调整，以确定所述样本人脸所对应的样本三维顶点以及所述样本三维顶点的第三纹理颜色信息；根据所述样本三维顶点的第三纹理颜色信息、以及所述样本人脸所对应的样本三维顶点在所述样本图像中对应的像素点的像素值，对所述原始几何特征提取模型进行训练。
为了方便训练几何特征提取模型，在本申请实施例中，需要预先收集用于训练几何特征提取模型的样本集(记为第二样本集)，该第二样本集中包含有样本图像(记为第二样本图像)，该第二样本图像中包含有人脸(记为样本人脸)。其中，第一样本图像与第二样本图像可以完全或部分相同，也可以完全不同。后续基于获取到的第二样本图像，对原始几何特征提取模型进行训练。
该进行几何特征提取模型训练的电子设备可以与进行图像处理的电子设备相同，也可以不同。
由于几何特征提取模型的输出为三维特征信息，而该三维特征信息主要是用于对预先设置的基础人脸三维模型进行调整的。而该调整后的基础人脸三维模型中人脸的三维顶点以及三维顶点的纹理颜色信息，可以一定程度上反映出该几何特征提取模型的精确度。基于此，在对原始几何特征提取模型进行训练的过程中，可以基于原始几何特征提取模型所输出的三维特征信息，对基础人脸三维模型进行调整，以确定第二样本图像中的样本人脸所对应的三维顶点(记为样本三维顶点)以及样本三维顶点的纹理颜色信息(记为第三纹理颜色信息)。后续每个样本图像中的样本人脸在当前迭代所对应的样本三维顶点、以及样本三维顶点的第三纹理颜色信息，对原始几何特征提取模型进行训练。
具体实施过程中，获取任一第二样本图像。通过原始几何特征提取模型，确定该第二样本图像中样本人脸的三维特征信息。基于该样本人脸的三维特征信息，对基础人脸三维模型进行调整，以确定该样本人脸所对应的样本三维顶 点以及样本三维顶点的第三纹理颜色信息。然后根据每个样本三维顶点的第三纹理颜色信息、以及该样本人脸上的样本三维顶点在样本图像中对应的像素点的像素值，确定该第二样本图像的损失值。基于该第二样本图像的损失值，对原始几何特征提取模型进行训练，以调整原始几何特征提取模型中的参数的参数值。
在一种可能的实施方式中，所述确定所述样本人脸所对应的样本三维顶点以及所述样本三维顶点的第三纹理颜色信息之后，所述根据所述样本三维顶点的第三纹理颜色信息、以及所述样本人脸所对应的样本三维顶点在所述样本图像中对应的像素点的像素值，对所述原始几何特征提取模型进行训练之前，所述方法还包括：根据所述样本图像中所述样本人脸对应的像素点以及所述样本三维顶点，确定目标样本三维顶点；其中，所述目标样本三维顶点为在所述样本图像中所述样本人脸的皮肤对应的像素点所对应的三维顶点；根据所述目标样本三维顶点的第三纹理颜色信息，确定所述目标样本三维顶点的第四纹理颜色信息；根据所述目标样本三维顶点以及所述目标样本三维顶点的第四纹理颜色信息，对所述样本三维顶点以及所述样本三维顶点的第三纹理颜色信息进行更新。
由于获取到的样本图像中样本人脸对应的样本三维顶点中，可能包括样本人脸上非皮肤区域所对应的样本三维顶点，导致获取到的第三纹理颜色信息中，也会包含样本人脸上非皮肤区域所对应的样本三维顶点的纹理颜色信息，从而影响确定的样本人脸的纹理颜色信息的准确度。因此，当基于上述实施例获取到样本图像中样本人脸对应的样本三维顶点后，根据样本图像中样本人脸对应的像素点，确定样本图像中样本人脸的皮肤所对应的像素点(记为样本像素点)。然后根据样本图像中像素点与样本三维顶点的对应关系，确定样本像素点所对应的样本三维顶点(记为目标样本三维顶点)。该目标样本三维顶点可以理解为位于样本图像中样本人脸的皮肤区域的样本三维顶点。然后根据目标样本三维顶点的第三纹理颜色信息，确定第四纹理颜色信息。根据目标样本三维顶点以及第四纹理颜色信息，对确定的样本三维顶点以及样本三维顶点的第三纹理颜色信息进行更新。将样本图像中样本人脸对应的样本三维顶点中，除目标样本三维顶点之外的样本三维顶点删除，只保留目标样本三维顶点，并确定目标样本三维顶点的第四纹理颜色信息为样本图像中样本人脸的纹理颜色信息。
在一种可能的实施方式中，可以采用如下方式根据目标样本三维顶点的第三纹理颜色信息，确定第四纹理颜色信息：
方式一、将任一目标样本三维顶点的第三纹理颜色信息，确定为第四纹理颜色信息。
方式二、将指定的目标样本三维顶点的第三纹理颜色信息，确定为第四纹理颜色信息。
方式三、通过预设的数学函数，对每个目标样本三维顶点的第三纹理颜色信息进行处理，确定处理后的纹理颜色信息为第四纹理颜色信息。
比如，通过预设的数学函数，确定每个目标样本三维顶点的第三纹理颜色信息的均值，将该均值确定为第四纹理颜色信息。
由于第二样本集中存在多个第二样本图像，针对每个第二样本图像，均执行上述的步骤，直至达到收敛条件，则确定几何特征提取模型训练完成。
满足预设的收敛条件可以为当前迭代所确定的每个第二样本图像的损失值的和小于预设的收敛阈值，或对原始几何特征提取模型进行训练的迭代次数达到设置的最大迭代次数等。具体实施中可以灵活进行设置，在此不做限定。
在一种可能的实施方式中，在进行几何特征提取模型训练时，可以把第二样本图像分为训练样本和测试样本，先基于训练样本对原始几何特征提取模型进行训练，再基于测试样本对上述已训练的几何特征提取模型的可靠程度进行验证。
下面通过具体的实施例对本申请实施例提供的对几何特征提取模型的训练过程进行介绍，图4为本申请实施例提供的一种几何特征提取模型的训练方法的流程示意图，该方法包括：
S401：获取第二样本集中的任一第二样本图像。
S402：通过原始几何特征提取模型，获取该第二样本图像中样本人脸的三维特征信息。
S403：基于该样本人脸的三维特征信息，对Blendshape三维模型进行调整，以确定该样本人脸对应的样本三维顶点以及样本三维顶点的第三纹理颜色信息。
S404：根据样本图像中样本人脸对应的像素点，确定样本图像中样本人脸的皮肤所对应的样本像素点。
S405：根据样本图像中像素点与样本三维顶点的对应关系，确定样本像素点所对应的目标样本三维顶点。
S406：将目标样本三维顶点的第三纹理颜色信息的均值确定为第四纹理颜色信息。
S407：根据目标样本三维顶点以及第四纹理颜色信息，对S403确定的样本三维顶点以及样本三维顶点的第三纹理颜色信息进行更新。
S408：根据每个样本三维顶点的第三纹理颜色信息、以及该样本人脸上的样本三维顶点在样本图像中对应的像素点的像素值，对原始几何特征提取模型进行训练，以调整原始几何特征提取模型中的参数的参数值。
在进行几何特征提取模型训练的过程中，一般采用离线的方式，预先通过训练设备基于第二样本集中的第二样本图像，对原始几何特征提取模型进行训练，以获得训练完成的几何特征提取模型。后续可以将该训练完成的几何特征提取模型保存在图像处理的电子设备中，以方便生成目标用户的表情图像。
实施例5：
下面通过实施例对本申请实施例提供的图像处理方法进行说明，图5为本申请实施例提供的一种图像处理方法的流程示意图，该方法包括：
S501：获取目标表情信息及目标用户的目标图像。
S502：判断目标图像中是否包含有人脸，若确定目标图像包含有人脸，则执行S503，若确定目标图像不包含有人脸，执行S508。
对目标图像进行人脸检测的过程，包括：通过预先训练的人脸检测模型，确定目标图像中是否包含有人脸。若确定目标图像中包含有人脸，则执行S503；若确定目标图像中不包含有人脸，则执行S508。
可选的，通过预先训练的人脸检测模型，还可以确定目标图像中人脸上的关键点。在确定目标图像中包含有人脸后，可以根据该目标图像中人脸上的关键点，确定目标图像中人脸对应的像素点。根据包含该人脸对应的所有像素点的子图像，对目标图像进行更新。
S503：确定目标图像中人脸上预设特征部位的第一特征信息。
第一特征信息以下的一项或多项：性别信息、发型信息、头发颜色信息、眼型信息、眉型信息、瞳孔颜色信息、胡须类型信息。
图6为本申请实施例提供的一种图像处理的场景示意图。如图6所示，可以通过第一识别模块确定目标图像中人脸上预设特征部位的第一特征信息。当获取到目标用户的目标图像后，通过该第一识别模块，对该目标图像进行处理，确定目标图像中人脸上预设特征部位的第一特征信息。比如，如图6所示的性别信息、发型信息、头发颜色信息、眼型信息、眉型信息、瞳孔颜色信息以及胡须类型信息。
该第一识别模块中可以预先保存有特征提取模型。通过该特征提取模型，可以获取到目标图像中人脸上预设特征部位的第一特征信息。
可选的，提取不同种类的第一特征信息所采用的特征提取模型可以相同也 可以不同。
S504：确定目标图像中人脸的第二特征信息。
第二特征信息包括以下的一项或多项：人脸几何信息、纹理颜色信息、表情信息。
如图6所示，可以通过第二识别模块确定目标图像中人脸的第二特征信息。当获取到目标用户的目标图像后，也可以通过该第二识别模块，对该目标图像进行处理，确定目标图像中人脸的第二特征信息。比如，如图6所示的人脸几何信息和纹理颜色信息。
在一种可能的实施方式中，若第二特征信息包括纹理颜色信息，确定目标图像中人脸的纹理颜色信息的过程如图7所示，包括：通过预先训练的三维特征提取模型(如图7所示的CNN网络)，基于目标图像，获取目标图像中人脸的三维特征信息。然后基于获取到的三维特征信息中包含的人脸几何信息以及纹理信息，对基础人脸三维模型(如图7所示的Blendshape三维模型)进行调整，从而确定目标图像中人脸对应的三维顶点及三维顶点的第一纹理颜色信息。根据目标图像中人脸对应的像素点以及获取到的三维顶点，确定目标图像中人脸的皮肤对应的像素点所对应的目标三维顶点。根据每个目标三维顶点的第一纹理颜色信息的均值，确定第二纹理颜色信息。然后根据目标三维顶点的第二纹理颜色信息，确定目标图像中人脸的纹理颜色信息。
在一种可能的实施方式中，若第二特征信息包括人脸几何信息，确定目标图像中人脸的人脸几何信息的过程，包括：通过预先训练的几何特征提取模型，基于目标图像，获取目标图像中人脸的三维特征信息；根据该三维特征信息，确定该人脸几何信息。
S503和S504的执行顺序不做限定，即S503和S504可以同时执行，也可以先执行S503再执行S504，还可以先执行S504再执行S503。
S505：确定第一特征信息所对应的素材图像。
如图6所示，预先设置有素材库，该素材库中保存有预设特征部位的每种特征信息所对应的素材图像。当基于S503，获取到目标图像中人脸上预设特征部位的第一特征信息后，可以从该素材库中，确定该第一特征信息所对应的素材图像。素材库中保存有每种特征信息与素材图像的对应关系，后续根据保存的每种特征信息与素材图像的对应关系，即可确定该第一特征信息对应的素材图像。
S506：根据所述目标表情信息、所述第二特征信息以及预先设置的基础人脸三维模型，确定目标人脸三维模型。
S507：根据目标人脸三维模型以及第一特征信息所对应的素材图像，渲染出目标用户的表情图像。
该渲染出的表情图像可以是动态的表情图像，也可以是静态的表情图像。
若进行图像处理的电子设备为服务器，当获取到目标用户的表情图像后，可以将生成的表情图像发送至目标用户的智能设备上，以便目标用户使用该表情图像。例如，目标用户可以在即时通信场景(比如IMO)下，使用该表情图像。同时，目标用户也可以用于视频直播场景(比如Live)下，使用该表情图像等。
在一种可能的实施方式中，考虑到目标用户的形象一般不会发生太大的变化，可以将目标用户的身份信息、确定的第一特征信息所对应的素材图像、第二特征信息对应保存。后续目标用户可以不需要再次上传目标图像，只需上传目标表情信息即可。进行图像处理的电子设备可以直接根据目标用户选择的目标表情信息、第二特征信息以及预先设置的基础人脸三维模型确定目标人脸三维模型。并根据该目标人脸三维模型以及保存的第一特征信息所对应的素材图像，渲染出目标用户的表情图像。
S508：输出重新上传目标图像的提示信息。
实施例6：
本申请实施例还提供了一种图像处理装置，图8为本申请实施例提供的一种图像处理装置的结构示意图，该装置包括：
获取单元81，设置为获取目标表情信息及目标用户的目标图像；处理单元82，设置为若确定所述目标图像中包含有人脸，则确定所述目标图像中人脸上预设特征部位的第一特征信息以及所述人脸的第二特征信息；渲染单元83，设置为根据目标人脸三维模型、以及保存的所述第一特征信息所对应的素材图像，渲染出所述目标用户的表情图像；其中，所述目标人脸三维模型是根据所述目标表情信息、所述第二特征信息以及预先设置的基础人脸三维模型确定的。
采用上述的方法，在获取到了目标用户的包含有人脸的目标图像以及目标表情信息后，可以自动确定该目标图像中人脸上预设特征部位的第一特征信息以及人脸的第二特征信息，减少人工的控制和工作量。并且由于预先保存有第一特征信息对应的素材图像，使得后续可以准确地确定出与目标用户的形象贴合的素材图像，并将该素材图像渲染到根据目标表情信息、所述第二特征信息以及预先设置的基础人脸三维模型确定的目标人脸三维模型中，从而渲染出目标用户的表情图像，实现个性化定制目标用户的表情图像，无需人工根据目标用户的目标图像绘制目标用户的表情图像，减少了人工成本。
本申请实施例所提供的图像处理装置可执行本申请任意实施例所提供的图像处理方法，具备执行方法相应的功能模块和效果。
实施例7：
图9为本申请实施例提供的一种电子设备结构示意图，该电子设备，包括：处理器91、通信接口92、存储器93和通信总线94，其中，处理器91，通信接口92，存储器93通过通信总线94完成相互间的通信；
所述存储器93中存储有计算机程序，当所述程序被所述处理器91执行时，使得所述处理器91执行：获取目标表情信息及目标用户的目标图像；若确定所述目标图像中包含有人脸，则确定所述目标图像中人脸上预设特征部位的第一特征信息以及所述人脸的第二特征信息；根据目标人脸三维模型、以及保存的所述第一特征信息所对应的素材图像，渲染出所述目标用户的表情图像；其中，所述目标人脸三维模型是根据所述目标表情信息、所述第二特征信息以及预先设置的基础人脸三维模型确定的。
由于上述电子设备解决问题的原理与图像处理方法相似，因此上述电子设备的实施可以参见方法的实施，重复之处不再赘述。
通信接口92设置为上述电子设备与其他设备之间的通信。
存储器93可以包括随机存取存储器(Random Access Memory，RAM)，也可以包括非易失性存储器(Non-Volatile Memory，NVM)。
采用上述的方法，在获取到了目标用户的包含有人脸的目标图像以及目标表情信息后，可以自动确定该目标图像中人脸上预设特征部位的第一特征信息以及人脸的第二特征信息，减少人工的控制和工作量。并且由于预先保存有第一特征信息对应的素材图像，使得后续可以准确地确定出与目标用户的形象贴合的素材图像，并将该素材图像渲染到根据目标表情信息、所述第二特征信息以及预先设置的基础人脸三维模型确定的目标人脸三维模型中，从而渲染出目标用户的表情图像，实现个性化定制目标用户的表情图像，无需人工根据目标用户的目标图像绘制目标用户的表情图像，减少了人工成本。
实施例8：
在上述实施例的基础上，本申请实施例还提供了一种计算机可读存储介质，所述计算机可读存储介质内存储有可由处理器执行的计算机程序，当所述程序在所述处理器上运行时，使得所述处理器执行时实现：获取目标表情信息及目标用户的目标图像；若确定所述目标图像中包含有人脸，则确定所述目标图像中人脸上预设特征部位的第一特征信息以及所述人脸的第二特征信息；根据目 标人脸三维模型、以及保存的所述第一特征信息所对应的素材图像，渲染出所述目标用户的表情图像；其中，所述目标人脸三维模型是根据所述目标表情信息、所述第二特征信息以及预先设置的基础人脸三维模型确定的。
由于计算机可读存储介质解决问题的原理与上述实施例中的图像处理方法相似，因此具体实施可以参见图像处理方法的实施。
采用上述的方法，在获取到了目标用户的包含有人脸的目标图像以及目标表情信息后，可以自动确定该目标图像中人脸上预设特征部位的第一特征信息以及人脸的第二特征信息，减少人工的控制和工作量。并且由于预先保存有第一特征信息对应的素材图像，使得后续可以准确地确定出与目标用户的形象贴合的素材图像，并将该素材图像渲染到根据目标表情信息、所述第二特征信息以及预先设置的基础人脸三维模型确定的目标人脸三维模型中，从而渲染出目标用户的表情图像，实现个性化定制目标用户的表情图像，无需人工根据目标用户的目标图像绘制目标用户的表情图像，减少了人工成本。

Claims (13)

  1. 一种图像处理方法，包括：
    获取目标表情信息及目标用户的目标图像；
    在确定所述目标图像中包含有人脸的情况下，确定所述目标图像中人脸上预设特征部位的第一特征信息以及所述人脸的第二特征信息；
    根据目标人脸三维模型、以及保存的所述第一特征信息所对应的素材图像，渲染出所述目标用户的表情图像；其中，所述目标人脸三维模型是根据所述目标表情信息、所述第二特征信息以及预先设置的基础人脸三维模型确定的。
  2. 根据权利要求1所述的方法，还包括：
    在确定所述目标图像中未包含有人脸的情况下，输出重新上传目标图像的提示信息。
  3. 根据权利要求1所述的方法，其中，所述第一特征信息包括以下的至少一项：
    性别信息、发型信息、脸部装饰物信息、头发颜色信息、眼型信息、眉型信息、瞳孔颜色信息、胡须类型信息。
  4. 根据权利要求3所述的方法，其中，所述确定所述目标图像中人脸上预设特征部位的第一特征信息，包括：
    通过预先训练的特征提取模型，基于所述目标图像，确定所述第一特征信息。
  5. 根据权利要求1所述的方法，其中，所述第二特征信息包括以下的至少一项：
    人脸几何信息、纹理颜色信息、表情信息。
  6. 根据权利要求5所述的方法，其中，在所述第二特征信息包括所述纹理颜色信息的情况下，确定所述人脸的第二特征信息，包括：
    通过预先训练的三维特征提取模型，基于所述目标图像，获取所述人脸的三维特征信息；
    基于所述三维特征信息中包含的人脸几何信息以及纹理信息，对所述基础人脸三维模型进行调整，以确定所述人脸对应的三维顶点及所述三维顶点的第一纹理颜色信息；
    根据所述三维顶点以及所述第一纹理颜色信息，确定所述人脸的纹理颜色信息。
  7. 根据权利要求6所述的方法，其中，所述根据所述三维顶点以及所述第一纹理颜色信息，确定所述人脸的纹理颜色信息，包括：
    根据所述目标图像中所述人脸对应的像素点以及所述三维顶点，确定目标三维顶点；其中，所述目标三维顶点为在所述目标图像中所述人脸的皮肤对应的像素点所对应的三维顶点；
    根据所述目标三维顶点的第一纹理颜色信息，确定所述目标三维顶点的第二纹理颜色信息；
    根据所述目标三维顶点的第二纹理颜色信息，确定所述人脸的纹理颜色信息。
  8. 根据权利要求5所述的方法，其中，在所述第二特征信息包括人脸几何信息的情况下，所述确定所述人脸的特征信息，包括：
    通过预先训练的几何特征提取模型，基于所述目标图像，获取所述人脸的三维特征信息；
    根据所述三维特征信息，确定所述人脸几何信息。
  9. 根据权利要求8所述的方法，其中，所述几何特征提取模型通过如下方式获取：
    获取样本集中包含的一样本图像；其中，所述样本图像中包含有样本人脸；
    通过原始几何特征提取模型，获取所述样本图像中所述样本人脸的三维特征信息；
    基于所述三维特征信息，对所述基础人脸三维模型进行调整，以确定所述样本人脸所对应的样本三维顶点以及所述样本三维顶点的第三纹理颜色信息；
    根据所述样本三维顶点的第三纹理颜色信息、以及所述样本人脸所对应的样本三维顶点在所述样本图像中对应的像素点的像素值，对所述原始几何特征提取模型进行训练。
  10. 根据权利要求9所述的方法，在所述确定所述样本人脸所对应的样本三维顶点以及所述样本三维顶点的第三纹理颜色信息之后，所述根据所述样本三维顶点的第三纹理颜色信息、以及所述样本人脸所对应的样本三维顶点在所述样本图像中对应的像素点的像素值，对所述原始几何特征提取模型进行训练之前，还包括：
    根据所述样本图像中所述样本人脸对应的像素点以及所述样本三维顶点，确定目标样本三维顶点；其中，所述目标样本三维顶点为在所述样本图像中所 述样本人脸的皮肤对应的像素点所对应的三维顶点；
    根据所述目标样本三维顶点的第三纹理颜色信息，确定所述目标样本三维顶点的第四纹理颜色信息；
    根据所述目标样本三维顶点以及所述目标样本三维顶点的第四纹理颜色信息，对所述样本三维顶点以及所述样本三维顶点的第三纹理颜色信息进行更新。
  11. 一种图像处理装置，包括：
    获取单元，设置为获取目标表情信息及目标用户的目标图像；
    处理单元，设置为在确定所述目标图像中包含有人脸的情况下，确定所述目标图像中人脸上预设特征部位的第一特征信息以及所述人脸的第二特征信息；
    渲染单元，设置为根据目标人脸三维模型、以及保存的所述第一特征信息所对应的素材图像，渲染出所述目标用户的表情图像；其中，所述目标人脸三维模型是根据所述目标表情信息、所述第二特征信息以及预先设置的基础人脸三维模型确定的。
  12. 一种电子设备，包括处理器和存储器，所述处理器设置为执行所述存储器中存储的计算机程序时实现如权利要求1-10中任一项所述图像处理方法。
  13. 一种计算机可读存储介质，存储有计算机程序，所述计算机程序被处理器执行时实现如权利要求1-10中任一项所述图像处理方法。
PCT/CN2022/094925 2021-06-10 2022-05-25 图像处理方法、装置、设备及介质 WO2022257766A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110646132.1 2021-06-10
CN202110646132.1A CN113361419A (zh) 2021-06-10 2021-06-10 一种图像处理方法、装置、设备及介质

Publications (1)

Publication Number Publication Date
WO2022257766A1 true WO2022257766A1 (zh) 2022-12-15

Family

ID=77533545

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/094925 WO2022257766A1 (zh) 2021-06-10 2022-05-25 图像处理方法、装置、设备及介质

Country Status (2)

Country Link
CN (1) CN113361419A (zh)
WO (1) WO2022257766A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116110099A (zh) * 2023-01-19 2023-05-12 北京百度网讯科技有限公司 头像生成的方法和头像更换的方法

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113361419A (zh) * 2021-06-10 2021-09-07 百果园技术(新加坡)有限公司 一种图像处理方法、装置、设备及介质
CN113658313B (zh) * 2021-09-09 2024-05-17 北京达佳互联信息技术有限公司 人脸模型的渲染方法、装置及电子设备

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104346824A (zh) * 2013-08-09 2015-02-11 汉王科技股份有限公司 基于单张人脸图像自动合成三维表情的方法及装置
CN108573527A (zh) * 2018-04-18 2018-09-25 腾讯科技（深圳）有限公司 一种表情图片生成方法及其设备、存储介质
CN110096925A (zh) * 2018-01-30 2019-08-06 普天信息技术有限公司 人脸表情图像的增强方法、获取方法和装置
US20200051303A1 (en) * 2018-08-13 2020-02-13 Pinscreen, Inc. Real-time avatars using dynamic textures
CN113361419A (zh) * 2021-06-10 2021-09-07 百果园技术(新加坡)有限公司 一种图像处理方法、装置、设备及介质

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10210648B2 (en) * 2017-05-16 2019-02-19 Apple Inc. Emojicon puppeting
CN109118569B (zh) * 2018-08-16 2023-03-10 Oppo广东移动通信有限公司 基于三维模型的渲染方法和装置

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104346824A (zh) * 2013-08-09 2015-02-11 汉王科技股份有限公司 基于单张人脸图像自动合成三维表情的方法及装置
CN110096925A (zh) * 2018-01-30 2019-08-06 普天信息技术有限公司 人脸表情图像的增强方法、获取方法和装置
CN108573527A (zh) * 2018-04-18 2018-09-25 腾讯科技（深圳）有限公司 一种表情图片生成方法及其设备、存储介质
US20200051303A1 (en) * 2018-08-13 2020-02-13 Pinscreen, Inc. Real-time avatars using dynamic textures
CN113361419A (zh) * 2021-06-10 2021-09-07 百果园技术(新加坡)有限公司 一种图像处理方法、装置、设备及介质

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116110099A (zh) * 2023-01-19 2023-05-12 北京百度网讯科技有限公司 头像生成的方法和头像更换的方法

Also Published As

Publication number Publication date
CN113361419A (zh) 2021-09-07

Similar Documents

Publication Publication Date Title
KR102241153B1 (ko) 2차원 이미지로부터 3차원 아바타를 생성하는 방법, 장치 및 시스템
US11861936B2 (en) Face reenactment
WO2022257766A1 (zh) 图像处理方法、装置、设备及介质
US10853987B2 (en) Generating cartoon images from photos
CN111432267B (zh) 视频调整方法、装置、电子设备及存储介质
US12002160B2 (en) Avatar generation method, apparatus and device, and medium
CN107341435A (zh) 视频图像的处理方法、装置和终端设备
JP2024500896A (ja) ３ｄ頭部変形モデルを生成するための方法、システム及び方法
US11461946B2 (en) Digital makeup artist
WO2023138345A1 (zh) 虚拟形象生成方法和系统
WO2019142127A1 (en) Method and system of creating multiple expression emoticons
CN116547721A (zh) 用于分析用户的头部的头发区域的图像的像素数据以生成一个或多个用户特定推荐的数字成像和学习系统及方法
JP2024503794A (ja) ２次元（２ｄ）顔画像から色を抽出するための方法、システム及びコンピュータプログラム
US11961169B2 (en) Digital makeup artist
KR20230118191A (ko) 디지털 메이크업 아티스트
US11983819B2 (en) Methods and systems for deforming a 3D body model based on a 2D image of an adorned subject
CN113287118B (zh) 用于面部再现的系统和方法
CN118470218A (zh) 一种个性化定制出图方法及系统
CN117994395A (zh) 数字人面部资产生成方法、装置、电子设备及存储介质
CN118587331A (zh) 生成虚拟人物形象的方法及电子设备
CN114201631A (zh) 照片发布方法、装置、计算机设备和存储介质
CN112614044A (zh) 一种手绘头像动漫化方法、系统、电子设备及存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22819364

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 22819364

Country of ref document: EP

Kind code of ref document: A1