CN106600668A - Animation generation method used for carrying out interaction with virtual role, apparatus and electronic equipment - Google Patents

Abstract

The invention discloses an animation generation method used for carrying out interaction with a virtual role, an apparatus and electronic equipment. The method comprises the following steps of establishing a depth Q-value network model and training the depth Q-value network model; mapping a real role in a real space and a hand-held weapon to a virtual space, wherein the virtual space includes the real role, a geometry entity corresponding to the hand-held weapon and a geometry entity corresponding to a virtual role which is interacted with the real role; taking the real role, a current state of the hand-held weapon, and a current state of the virtual role in a current frame as input of the depth Q-value network model, and according to the current state, determining a withstanding action of the virtual role of a next frame by the depth Q-value network model, and then generating a current frame animation. In the method, a role animation of the virtual role can be automatically generated in real time; and the animation can carry out real-time feedback based on real person motion capturing data so as to withstand an attack action from a real person.

Description

Animation generation method and device for interaction with virtual character and electronic equipment

Technical Field

The invention relates to the fields of virtual reality, artificial intelligence and the like, in particular to an animation generation method and device for interacting with a virtual character and electronic equipment.

Background

With the continuous development of virtual reality technology, virtual character animation is often applied to many important fields such as animation, movie and television, games, and the like. Users can play in the virtual world through a variety of virtual reality hardware devices. This has to create a problem as to how a real person interacts with the virtual world, and in particular with the virtual character, in the virtual environment.

Today, artificial intelligence technology develops at a high speed, people have human-like intelligence and carry out human-computer interaction with real people, and the human-computer interaction intelligent human-computer interaction. Similarly, in the field of virtual reality, the role animation is generated in real time by the virtual role in the process of interacting with the real person, and the method has important significance.

In general, there are several methods for generating animation of a character. The first is motion repositioning, i.e., mapping directly to a new virtual character from animation or motion capture data of an existing other character, which cannot change based on different spatial environments during the interaction. The second is animation synthesis, in which many basic animations are synthesized into a new animation through interpolation transformation, the method depends on a large number of animation sets, and the animation sets need a large number of animation modeling by early art designing. And the third method is inverse dynamics calculation, namely, the skeleton structure of the virtual role is used as a chain dynamics system, the target space coordinates of the end points are known, and the space displacement rotation of each joint point is inversely calculated.

Disclosure of Invention

Technical problem to be solved

The invention provides an animation generation method and device for interacting with a virtual character and electronic equipment. The technical scheme uses a brand-new interactive animation real-time generation method, and can automatically generate the role animation of the virtual role in real time. The animation can make real-time feedback based on the motion of a real person, resist the attack from the real person, simulate the stress response of the real person to the maximum extent and simulate the action of a human-like person. The virtual character is able to learn game strategy from previous interactions. And the method is not limited to any predefined animation, but interacts with the human player in real-time in an online manner.

(II) technical scheme

In order to solve the technical problem, the invention provides an animation generation method, an animation generation device and electronic equipment for interaction between a real person and a virtual character.

According to an aspect of the present invention, there is provided an animation generation method for interacting with a virtual character, including the steps of:

establishing a depth Q-value network model, and training the depth Q-value network model;

mapping the real character and the handheld weapon thereof in the real space to the virtual space; the virtual space comprises a real character, a geometric entity corresponding to a handheld weapon of the real character and a geometric entity corresponding to a virtual character interacting with the real character;

and taking the current states of the real character and the handheld weapon thereof and the virtual character in the current frame as the input of the depth Q-value network model, and determining the blocking action of the next frame of the virtual character by the depth Q-value network model according to the current states so as to generate the current frame animation.

Wherein the method further comprises:

before generating the current frame animation, the limb actions of the virtual character are calibrated to be fitted into human-like actions.

Wherein the mapping of the real character and its handheld weapon in real space to virtual space comprises:

establishing a geometric entity in a virtual environment, wherein the geometric entity comprises a geometric entity corresponding to a real character in the real environment and a geometric entity corresponding to a virtual character interacting with the real character in the virtual environment;

acquiring a current image of a real role in a real environment by using an image acquisition device;

obtaining the spatial position of each joint of the real character according to the current image, and taking the hand joint of the weapon held by the real character as the spatial position of a weapon holding end;

acquiring a depth map of a weapon held by the real character according to the current image, and acquiring a rotation vector of the weapon according to the depth map of the held weapon;

and mapping the spatial posture of the weapon held by the real character into the virtual environment according to the spatial position of the weapon holding end and the rotation vector of the weapon.

Wherein the Q-value function of the depth Q-value net model is represented as follows:

Q(s，a)＝E_π(R_t|s_t，a_t，π)

R_tis the true value of the future return, E_πIs an expectation under strategy π, E_π(R_t|s_t，a_tπ) is represented at s_t，a_tR under the condition of pi_tExpressed as follows:

wherein, γ^t′-tFor the reward of environmental feedback obtained at a future time T' -T, gamma is more than 0 and less than 1, T is the current time, T is the termination time, r_t′Indicating the environmental reward at time t'.

Wherein the environmental reward includes three conditions:

when the handheld weapon of the real character collides with the weapon of the virtual character, the virtual character successfully withstands, and the environment reward is positive feedback;

when the handheld weapon of the real character collides with the virtual character or the weapon of the virtual character collides with the virtual character, the virtual character fails to resist, and the environment reward is negative feedback;

when no collision is detected, the environmental reward is 0.

Wherein the training of the deep Q-value network model comprises:

randomly selecting an action sample in the action set according to a preset probability;

initializing parameters of a depth Q-value network model;

in the ith iteration, the target Q value is rewarded r by the environment of the current frame_tAnd the Q value of the next frame state is multiplied by the breakage rate to jointly determine;

calculating a target error function value by using a least square method by using the target Q value;

and updating the parameters of the depth Q-value network model by using a random gradient descent method, and performing i +1 times of iteration until an iteration end condition is reached.

Wherein the current state comprises a pose of the real character handheld weapon and a rotation value of a joint of the virtual character, and the pose of the real character handheld weapon comprises a spatial displacement coordinate and a rotation value of the real character handheld weapon.

Before generating the current frame animation, calibrating the limb action of the virtual character to fit the limb action into a human-like action, including:

establishing a dynamic model of the resisting joint;

the rotation angular velocity of the virtual character withholding action output by the Q-value network model is corrected by using the dynamic model, and the rotation angular velocity is expressed as follows:

τ_intfor the moment inside the joint in the dynamic model,is the angle of the target retaining joint, q is the angle of the current retaining joint, k_p、k_dAngle gain and angular velocity gain, respectively; angular acceleration of joint rotationCalculated by the following formula:

where M (q) is the bone mass matrix, τ_extFor the outer part of the joint in the kinetic modelThe moment of force is generated by the motor,is a centrifugal force; the angular velocity q of the current frame t is calculated by the following formula^(t)Rotation angular velocity of the virtual character's movement of blocking output by the modified Q-value network model:

wherein q is^(t-1)、Respectively representing the angle, the angular velocity and the angular acceleration of the previous frame;

and predicting the corresponding actions of other joints of the virtual character by using the obtained withholding actions of the withholding joints.

According to a second aspect of the present invention, there is provided an animation generation apparatus for interacting with a virtual character, comprising:

the model establishing and training module is used for establishing a deep Q-value network model and training the deep Q-value network model;

the mapping module is used for mapping the real role in the real space and the handheld weapon thereof to the virtual space; the virtual space comprises a real character, a geometric entity corresponding to a handheld weapon of the real character and a geometric entity corresponding to a virtual character interacting with the real character;

and the animation generation module is used for taking the current state of the real character and the handheld weapon thereof and the current state of the virtual character in the current frame as the input of the depth Q-value network model, and determining the blocking action of the next frame of the virtual character by the depth Q-value network model according to the current state so as to generate the current frame animation.

According to a third aspect of the present invention, there is provided an electronic device comprising:

a memory for storing executable instructions; and

a processor to execute executable instructions stored in the memory to perform the following operations:

establishing a depth Q-value network model, and training the depth Q-value network model;

mapping the real character and the handheld weapon thereof in the real space to the virtual space; the virtual space comprises a real character, a geometric entity corresponding to a handheld weapon of the real character and a geometric entity corresponding to a virtual character interacting with the real character;

and taking the current states of the real character and the handheld weapon thereof and the virtual character in the current frame as the input of the depth Q-value network model, and determining the blocking action of the next frame of the virtual character by the depth Q-value network model according to the current states so as to generate the current frame animation.

The animation-based frame-by-frame generation is obtained through the continuous interaction process of the intelligent agent and the environment. When the agent obtains the current state given by the environment, the agent makes a decision based on the current state and needs to select an action. After the action interacts with the environment, the environment feeds back to the state of the next frame of the agent. The invention can automatically generate the character animation of the virtual character in real time, the animation can make real-time feedback based on the real person motion capture data, resists the attack action from the real person, and the generated virtual character animation can simulate the stress response of the real person to the maximum extent and interact with the real person player in real time by the humanoid action.

Drawings

FIG. 1 is a flow chart of an animation generation method for interacting with a virtual character according to the present invention.

FIG. 2 is a diagram of a neural network model architecture based on a deep Q-value function in the present invention.

Detailed Description

In order that the objects, technical solutions and advantages of the present invention will become more apparent, the present invention will be further described in detail with reference to the accompanying drawings in conjunction with the following specific embodiments.

As shown in fig. 1, the method is a flowchart of an animation generation method interacting with a virtual character, and the method is divided into three stages, namely, a mapping step, a decision step and a calibration step, wherein the model establishment and training in the decision step can be established and trained in advance before the animation generation, that is, the model establishment and training can be performed before the mapping step, and the method specifically includes the following steps:

and step S1, entity mapping, wherein the objects in the real space are mapped into the virtual environment in the form of geometric entities and are collided and interacted with the objects in the virtual environment. Wherein, the step S1 can be realized by the following steps:

step 101, establishing a geometric entity. And establishing predetermined geometric entities in the virtual environment, and skinning each established geometric entity by using a physical agent for later collision detection. The predetermined geometric entities include virtual characters, virtual character weapons, real player weapons. In an embodiment, the virtual environment is established based on a rendering engine.

And 102, tracking by a camera to obtain the spatial position of the main joint of the human body, and taking the spatial position of the hand joint as the spatial position of one end of the handheld weapon. A human player stands in front of an image capture device such as RGB-D, binocular camera, etc., and swings a weapon in the hand, and the camera captures the motion of objects including the human and its weapon. In this embodiment, a depth point cloud (i.e. information included in a depth map, specifically, a depth value corresponding to each pixel in an image) is obtained by a camera, and a distance from a spatial object corresponding to each pixel in the image to a camera, i.e. a depth point, is obtainedDepth value of each pixel in the cloud. The depth map is used to calculate real-time spatial position and rotation values of objects in the image at each frame. The spatial positions of the main joints of the human body (i.e. the spatial positions of the joints of the human body in the environment of the human body) can be obtained by a development tool set of RGB-D cameras, such as kinect. For example, in an online weaponry game, note (x)_h，y_h，z_h) The space coordinate of the hand joint point is used for replacing the space coordinate of one end of the hand-held weapon with the position of the hand joint point.

And 103, extracting the spatial attitude. Due to the problem of motion blur of the end points, morphological means are used in this embodiment to solve for the spatial position of the other end of the weapon. In the depth map, the threshold value d is first taken in this embodiment₀Selecting depth value capable of distinguishing foreground and background in image as threshold value by artificial method, removing image background, i.e. making pixel value greater than threshold value d from depth image₀The value of the pixel point of (a) is set to 0. Then only the first target depth map remains. The first target depth map includes a human body and appendages of the human body, such as weapons, etc. Then, in the first target depth map, since the weapon portion is thinner than the real person, the first target depth map is first subjected to the opening operation, i.e., the erosion operation is first performed, and then the dilation operation is performed, so that the second target depth map including only the human body is obtained. In this way, a portion of the weapon can be removed from the image, and a difference between the first target depth map and the second target depth map is made to obtain a third target depth map, i.e., a depth map of the weapon in the depth map. In this step, the motion blurred points in the weapon section are all deleted during the opening operation. In this embodiment, a point with a non-zero pixel value included in the third target depth map is taken as the spatial point set (X, Y, Z), where (X, Y) is the point coordinate in the third target depth map and Z is the corresponding pixel value (X, Y), i.e., the depth value. Fitting the space point set into a straight line by using a least square method to obtain a rotation vector of the weapon

And 104, mapping the spatial attitude. In the present embodiment, the spatial displacement coordinates (x) of the real hand-held weapon obtained in step 102 and step 103_h，y_h，z_h) And rotation valueTo the geometrical entity of the corresponding real weapon in the virtual environment, which weapon is then mapped as a geometrical entity into the virtual space.

Step S2, decision of state to action. In the step, the resisting action of the next frame of the virtual character is determined according to the current states of the real-person handheld weapon and the virtual character. This step S2 may be implemented by:

step 201, establishing a network model deep Q-value network (DQN). And generating a withstanding animation of the virtual character through the network model of the DQN. The animation generation task may be viewed as a finite Markov decision process. Character animation can be written as a decision sequence against an attack. The neural network is used to select the action of each frame.

The current state involved in the current frame animation generation is composed of the pose of the current real weapon and the rotation value of the joint related to the virtual character, which are obtained in step 103, and is denoted by s_i. The current pose of the real weapon comprises spatial displacement coordinates and rotation values of the real weapon. The motion decision of the current frame is determined from the current state, i.e. a_t＝π(s_t) Where pi represents a policy, which is a mapping from the current state to the current action. Current action a_tIs based on a q-value function q (s, a). The deep Q-value network is then an approximator of the action-value function Q (s, a).

The virtual character as an agent constantly interacts with the environment during animation generation. The frame-by-frame generation of the animation is achieved through interaction of the agent with the environment. When the agent obtains the current state s_iThen, the agent makes a decision based on the current state and selects action a_tAs animation of the current frame and fed back to the virtual framePseudo-environmental action a_t. The environment is fed back to the state s of the next frame of the intelligent agent_t+1And an environmental reward r_t。

In the task in this embodiment, the set of actions includes various rotations of the resistant joint. The resisting action involves two joints: shoulder and elbow joints. Before all operations, the validity detection is carried out, namely whether the current generated action is within the threshold value of the human motion rationality. Because of the euler angle, which is a phenomenon of gimbal deadlock, a joint may lose one degree of freedom, and thus each joint operates two degrees of freedom. Movement i.e. rotation of each joint about a single axis of rotationIn each decision process, it can choose to rotateOr 0 deg., there are 3 rotation angles per joint. In order to avoid dead locking of the universal joints and unsmooth rotating animation caused by the combination of the rotating actions, each joint only selects two degrees of freedom, so that the total number of the joints is 2, each joint has 2 axial directions, and the total number of the rotating shafts is 4. The animation collection will have a total of 3⁴The operation is 81. Wherein,not a constant value, will be set in the calibration step.

The observation of a single frame of the environment is not sufficient to understand the overall motion situation. In the present exemplary embodiment, therefore, a sequence is observed as the current state s_t＝x₁，x₂，...，x_t. Wherein x_tThat is, the current frame state of the environment of the t-th frame, in this embodiment, the current state of the attacker and the current state of the blocker need to be considered at the same time. State x_tThe virtual character right shoulder rotation value, the virtual character right elbow rotation value, the player weapon spatial displacement and the player weapon rotation value. To normalize the input, in this exampleIn the embodiment, the latest four frames are taken as the current state s_t＝x_t-3，x_t-2，x_t-1，x_tSo in the first 3 frames the avatar does not produce any action, so the state is a matrix of 3 × 4 × 4 ═ 48.

Prize r_tThere are 3 cases. When the real person weapon collides with the virtual character weapon, the virtual character is considered to successfully resist, and positive feedback can be generated; when a real weapon collides with the virtual character or the weapon of the virtual character collides with the virtual character, the virtual character is considered to fail to resist, and negative feedback is generated. The other times when no collision is detected are all considered to be a feedback of 0.

For future reporting R_tIs the feedback value r of all frames following the current frame_tThe feedback obtained after multiplying by the weighted sum of the impairment ratios, i.e. n frames, is the feedback impairment γ obtained for the current frameⁿWhere 0 < γ < 1, T is the current time, T is the end time, which in this embodiment represents the time at which a collision is detected, γ^t′-tThe breakage rate at the current time for feedback obtained at each time in the future. This also means that the approach in this embodiment tends to select a path that can complete the withstand task as quickly as possible.

Q(s，a)＝E_π(R_t|s_t，a_t，π)

R_tIs the true value of the future return, and the future return needs to be fitted and estimated in training. The Q-value function is the expectation of future returns, E_πIs an expectation under strategy π, E_π(R_t|s_t，a_tπ) is represented at s_t，a_tR under the condition of pi_tThe expectation is that. Let Q^*(s, a) is the optimal future expected return, but due to uncertainty in future strategies, it is desirable to do soTo determine Q^*Is not practical. Thus, the deep neural network is used in the present embodiment as a nonlinear equation approximator to estimate the Q-value function. Q (s, a; theta) is approximated as the optimum Q value. θ is a parameter of the deep Q-value network. When θ is solved in this embodiment, the Q-value of each state-action pair can be estimated in this embodiment.

Q(s，a；θ)≈Q^*(s，a)

As shown in fig. 2, the structure of the network model. The state vector of the latest frames is used as the input of the network model, the input layer is 48 nodes, and the q value of each action in the action set corresponding to the current state is output through weighting operation. The dimensionality output by the network model is the same as the number of actions in the action set, and the output layer is 81 nodes.

Step S3 is part of the network model training. This step is used to train the established network model. This step S3 may be implemented by:

step 202, greedy search. During training, a greedy exploration strategy is used. I.e. the probability with which an action is randomly selected in the set of actions, instead of selecting the action with the largest q value. This is an exploration of more path possibilities during the optimization process, thus avoiding falling into a local optimum.

In the above equation, the current time action a is given_tI.e. selecting the strategy that maximizes the Q value, argmaxQ(s), with a probability p of 1-_tAnd a) selecting an action in the action set A at random according to the probability.

The values in the method in this embodiment are not constant but gradually decrease as the number of iterations increases. After L iterations, the value is fixed to a constant value. I.e., iteration number 0 → L, probability: 1 → σ. This means that during the initial training process the agent goes more to explore possible paths, and as the number of iterations increases, the probability of exploring new paths decreases as the training converges.

In the formula, the value of the calculated probability changes with the increase of the iteration number i, L represents a certain iteration number set artificially, and after the iteration of the L times, the value is stabilized to a small positive number sigma and does not change, and the sigma is a constant set artificially.

Exploration is only applied in the process of model training. During model usage, the strategy only follows a_t＝argmaxQ(s_t，a)。

Step 203, q-value iteration. Initial action-value function Q₀(s，a；θ₀) To theta₀Initializing theta₀U (-0.5, 0.5). The difference between the output Q value and the target Q value is introduced by training the network.

In the ith iteration, the Q value of the target is fed back r by the current frame_tAnd the Q value of the next time state is multiplied by the breakage rate.

The target value y of the Q function is obtained by calculation according to the formula, and the target error function is calculated by using a least square method.

Updating parameter θ using a random gradient descent method_iThereby reducing the error function. Is the error function pair theta_iIs derived by

Parameter theta_iUpdated by multiplying the learned rate α by the derivative of the error function (theta)_i) Less than a certain threshold, i.e. Q_i≈Q_i-₁Then the Q-network model is considered to converge to the optimal action-value function Q^*。

To solve the problem that the space of the state-action pairs is too large, a method using empirical playback may be selected. In the training process, the s of each frame is divided into_t，a_t，r_t，s_t+1And storing the tuples into a playback memory as a tuple, and extracting a certain batch of tuples from the playback memory to update the network parameters in the training of each frame. This may smooth the distribution of the training data.

Step 204, use of a network model. The real-time action decision based on the current state, and the current frame animation generation takes the animation generation as a decision step. That is, when the current state of each frame is used as input, the virtual character is used as an agent, and in the interaction process with the environment, based on the decision made by the state, an animation is selected as the current frame real-time response animation in the animation set. The decision of current frame animation generation is based on the fact that more positive feedback can be obtained in the future by interaction. This approach is an animation of the generation of successive decisions until the termination of the interaction is reached, i.e. a physical collision is detected.

In step S4, limb correction is a calibration step.

Step 301, bone dynamic fitting. Resist and keep outThe dynamic simulation of the joint type, the virtual character only has the right shoulder joint and the right elbow joint to complete the attack resistance, so the two joints are called the resistance joint and are also the task related joint in the invention. The skeleton structure of the virtual character belongs to a chain rigid body dynamic system, and because the generated animation is uniform angular velocity, the dynamic model is used for correcting the angular velocity mentioned in the step 201To conform to the human kinetic model.

The joint moment of the kinetic model of the chain joint is tau_int，τ_intInternal moment, rotation angle from target jointAnd angular velocityAnd (4) obtaining. WhereinIs the target joint angle and q is the current joint angle. k is a radical of_p、k_dAngle gain and angular velocity gain, respectively. The internal and external moments of the limb correspond to a force balance. External moment τ_extIncluding gravity, etc.

Where M (q) is a matrix of bone mass,is a centrifugal force, and the angular acceleration of the joint rotation can be calculated

Wherein q is^(t)Is the angular velocity of the current frame t, which can be calculated from the previous frame, q^(t-1)、 Respectively representing the angle, angular velocity and angular acceleration of the previous frame. Obtained q^(t)For replacing action setsThe value is obtained.

Step 302, inter-joint prediction. In the above steps, the motion animation of the right shoulder joint and the right elbow joint of the barrier joint is obtained, but in the present embodiment, the animation of all other joints of the virtual character is obtained by using the method of inter-joint prediction. Other joints are not related to human beings and are mainly used for generating tiny swings to keep the body balance and make the whole-body animation more realistic.

In real human motion, the joints are related to each other. The law of motion between the joints can be derived by analysis of the motion capture data. For example, when the right arm moves, the spine rotates accordingly. While the left arm swings in the opposite direction based on the body balance.

During the exercise, in order to keep the body balanced, the projection of the center of mass of the human body should be located within the contact range of the foot and the ground. The central point of each skeleton is used as the skeleton position, and the skeleton mass is set in direct proportion to the skeleton length, so as to calculate the mass center of human body.

The above-mentioned embodiments are intended to illustrate the objects, technical solutions and advantages of the present invention in further detail, and it should be understood that the above-mentioned embodiments are only exemplary embodiments of the present invention and are not intended to limit the present invention, and any modifications, equivalents, improvements and the like made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims (10)

1. An animation generation method for interacting with a virtual character is characterized by comprising the following steps:

establishing a depth Q-value network model, and training the depth Q-value network model;

mapping the real character and the handheld weapon thereof in the real space to the virtual space; the virtual space comprises a real character, a geometric entity corresponding to a handheld weapon of the real character and a geometric entity corresponding to a virtual character interacting with the real character;

and taking the current states of the real character and the handheld weapon thereof and the virtual character in the current frame as the input of the depth Q-value network model, and determining the blocking action of the next frame of the virtual character by the depth Q-value network model according to the current states so as to generate the current frame animation.

2. The method of claim 1, wherein the method further comprises:

before generating the current frame animation, the limb actions of the virtual character are calibrated to be fitted into human-like actions.

3. The method of claim 1 or 2, wherein mapping the real character in real space and its handheld weapon to virtual space comprises:

establishing a geometric entity in a virtual environment, wherein the geometric entity comprises a geometric entity corresponding to a real character in the real environment and a geometric entity corresponding to a virtual character interacting with the real character in the virtual environment;

acquiring a current image of a real role in a real environment by using an image acquisition device;

obtaining the spatial position of each joint of the real character according to the current image, and taking the hand joint of the weapon held by the real character as the spatial position of a weapon holding end;

acquiring a depth map of a weapon held by the real character according to the current image, and acquiring a rotation vector of the weapon according to the depth map of the held weapon;

and mapping the spatial posture of the weapon held by the real character into the virtual environment according to the spatial position of the weapon holding end and the rotation vector of the weapon.

4. The method of claim 1 or 2, wherein the Q-value function of the depth Q-value net model is represented as follows:

Q(s，a)＝E_π(R_t|s_t，a_t，π)

R_tis the true value of the future return, E_πIs an expectation under strategy π, E_π(R_t|s_t，a_tπ) is represented at s_t，a_tR under the condition of pi_tExpressed as follows:

$R_t=\underset{t^{\&prime;}=t}{\overset{T}{\&Sigma;}}{\mathrm{\&gamma;}}^{t^{\&prime;}-t}{r}_{t^{\&prime;}}$

wherein, γ^t′-tFor the environmental reward obtained at the future time T' -T, gamma is more than 0 and less than 1, T is the current time, T is the termination time, r_t′Indicating the environmental reward at time t'.

5. The method of claim 4, wherein the environmental feedback includes three conditions:

when the handheld weapon of the real character collides with the weapon of the virtual character, the virtual character successfully withstands, and the environment reward is positive feedback;

when the handheld weapon of the real character collides with the virtual character or the weapon of the virtual character collides with the virtual character, the virtual character fails to resist, and the environment reward is negative feedback;

when no collision is detected, the environmental reward is 0.

6. The method of claim 1 or 2, wherein said training the deep Q-value network model comprises:

randomly selecting an action sample in the action set according to a preset probability;

initializing parameters of a depth Q-value network model;

in the ith iteration, the target Q value is rewarded and fed back r by the environment of the current frame_tAnd the Q value of the next frame state is multiplied by the breakage rate to jointly determine;

calculating a target error function value by using a least square method by using the target Q value;

and updating the parameters of the depth Q-value network model by using a random gradient descent method, and performing i +1 times of iteration until an iteration end condition is reached.

7. The method of claim 1, wherein the current state comprises a pose of the real character hand-held weapon and a rotation value of a joint of the virtual character, the pose of the real character hand-held weapon comprising spatial displacement coordinates and rotation values of the real character hand-held weapon.

8. The method of claim 2, wherein calibrating the limb movements of the virtual character to fit human-like movements prior to generating the current frame animation comprises:

establishing a dynamic model of the resisting joint;

the rotation angular velocity of the virtual character withholding action output by the Q-value network model is corrected by using the dynamic model, and the rotation angular velocity is expressed as follows:

${\mathrm{\&tau;}}_{\mathrm{int}}=k_p(\tilde{q}-q)-k_d\stackrel{\&CenterDot;}{q}$

τ_intfor the moment inside the joint in the dynamic model,is the angle of the target retaining joint, q is the angle of the current retaining joint, k_p、k_dAngle gain and angular velocity gain, respectively; angular acceleration of joint rotationCalculated by the following formula:

$M\left(q\right)\stackrel{\&CenterDot;\&CenterDot;}{q}+C(q,\stackrel{\&CenterDot;}{q})={\mathrm{\&tau;}}_{\mathrm{int}}+{\mathrm{\&tau;}}_{ext}$

where M (q) is the bone mass matrix, τ_extFor the external moment of the joint in the dynamic model,is a centrifugal force; the angular velocity q of the current frame t is calculated by the following formula^(t)Rotation angular velocity of the virtual character's movement of blocking output by the modified Q-value network model:

$\begin{array}{c}{q}^{\left(t\right)}=q^{(t-1)}+{\stackrel{\&CenterDot;}{q}}^{\left(t\right)}\mathrm{\&Delta;}t\\ =q^{(t-1)}+{\stackrel{\&CenterDot;}{q}}^{(t-1)}\mathrm{\&Delta;}t+{\stackrel{\&CenterDot;\&CenterDot;}{q}}^{(t-1)}{\mathrm{\&Delta;t}}^2\end{array}$

wherein q is^(t-1)、Respectively representing the angle, the angular velocity and the angular acceleration of the previous frame;

and predicting the corresponding actions of other joints of the virtual character by using the obtained withholding actions of the withholding joints.

9. An animation generation device for interacting with a virtual character, comprising:

the model establishing and training module is used for establishing a deep Q-value network model and training the deep Q-value network model;

the mapping module is used for mapping the real role in the real space and the handheld weapon thereof to the virtual space; the virtual space comprises a real character, a geometric entity corresponding to a handheld weapon of the real character and a geometric entity corresponding to a virtual character interacting with the real character;

and the animation generation module is used for taking the current state of the real character and the handheld weapon thereof and the current state of the virtual character in the current frame as the input of the depth Q-value network model, and determining the blocking action of the next frame of the virtual character by the depth Q-value network model according to the current state so as to generate the current frame animation.

10. An electronic device, comprising:

a memory for storing executable instructions; and

a processor to execute executable instructions stored in the memory to perform the following operations:

establishing a depth Q-value network model, and training the depth Q-value network model;

mapping the real character and the handheld weapon thereof in the real space to the virtual space; the virtual space comprises a real character, a geometric entity corresponding to a handheld weapon of the real character and a geometric entity corresponding to a virtual character interacting with the real character;

and taking the current states of the real character and the handheld weapon thereof and the virtual character in the current frame as the input of the depth Q-value network model, and determining the blocking action of the next frame of the virtual character by the depth Q-value network model according to the current states so as to generate the current frame animation.