TWI659335B - Graphic processing method and device, virtual reality system, computer storage medium - Google Patents

Abstract

The present application provides a graphics processing method and device, a virtual reality system, and a computer storage medium. The method includes: obtaining position information of an observer; determining a target object in a virtual reality VR picture to be displayed according to the position information; obtaining At least two images corresponding to the target object stored in advance, the at least two images are images respectively taken from different shooting positions; according to the position information and the corresponding shooting of the at least two images Position, using the at least two images to generate a target image, the target image being an image of the target object corresponding to the position of the observer; and displaying the VR picture, and displaying the VR picture in the VR picture Rendering the target image.

Description

   Graphic processing method and device, virtual reality system and computer storage medium   

This application relates to the field of graphics processing, and in particular, to a graphics processing method and device, a virtual reality system, and a computer storage medium.

A mainstream technology for generating a virtual reality (VR) scene is three-dimensional (3D) modeling technology. 3D modeling technology generates VR scenes based on 3D models. In some VR game products, VR scenes are mainly completed using 3D modeling technology combined with real-time rendering technology. Users use VR head-mounted display devices, such as VR glasses or VR helmets, as viewing media, integrate into VR scenes, and interact with characters or other objects in VR scenes, so as to get a real sense of space. The most common example is the roller coaster VR scene.

An embodiment of the present application provides a graphic processing method, including: a graphic processing method applied to a computing device, including: obtaining position information of an observer; and determining a target object in a virtual reality VR picture to be displayed according to the position information Obtaining at least two images corresponding to the target object stored in advance, the at least two images being images respectively taken from different shooting positions; corresponding to the at least two images according to the position information Using the at least two images to generate a target image, where the target image is an image of the target object corresponding to the position of the observer; displaying the VR picture, and displaying the VR image in the VR The target image is rendered in a picture.

An embodiment of the present application provides a graphics processing apparatus, which includes a processor and a storage, where the storage stores computer-readable instructions, which can cause the processor to perform: obtaining position information of an observer; and according to the position The message determines a target object in the virtual reality VR picture to be displayed; acquires at least two images corresponding to the target object stored in advance, and the at least two images are images respectively taken from different shooting positions; Generating a target image using the at least two images according to the position information and a shooting position corresponding to the at least two images, where the target image is a target object corresponding to the position of the observer An image; displaying the VR picture, and rendering the target image in the VR picture.

An embodiment of the present application provides a graphic processing method adapted to a computing device, including: collecting current pose information of an observer; obtaining position information of the observer according to the pose information; and determining a to-be-displayed based on the position information. A target object in a virtual reality VR picture; obtaining at least two images corresponding to the target object stored in advance, the at least two images are images respectively taken from different shooting positions; according to the position information A shooting position corresponding to the at least two images, and using the at least two images to generate a target image, where the target image is an image of the target object corresponding to the position of the observer; Describe the VR picture, and render the target image in the VR picture.

An embodiment of the present application provides a virtual reality VR system, including a gesture collection device, a processing device, and a display device: the gesture collection device is configured to: collect current pose information of an observer; the processing device is configured to: Posture information to obtain the position information of the observer; determine the target object in the virtual reality VR picture to be displayed according to the position information; obtain at least two images corresponding to the target object stored in advance, the at least The two images are images respectively taken from different shooting positions; and according to the position information and the shooting positions corresponding to the at least two images, a target image is generated by using the at least two images, and the target The image is an image of the target object corresponding to the position of the observer; the display device is configured to display the VR picture, and render the target image in the VR picture.

An embodiment of the present application provides a computer storage medium having instructions stored thereon, and when the instructions are run on a computer, the computer is caused to execute the method described in the embodiment of the present application.

An embodiment of the present application provides a computer storage medium having instructions stored thereon, and when the instructions are run on a computer, the computer is caused to execute the method described in the embodiment of the present application.

The embodiment of the present application provides a computer program product including instructions. When a computer runs the instructions of the computer program product, the computer executes the method described in the embodiment of the present application.

The embodiment of the present application provides a computer program product including instructions. When a computer runs the instructions of the computer program product, the computer executes the method described in the embodiment of the present application.

30‧‧‧VR system

32‧‧‧ posture collection device

34‧‧‧Processing device

36‧‧‧display device

42‧‧‧ Character

43‧‧‧ objects

44‧‧‧ objects

46‧‧‧ objects

52‧‧‧ objects

54‧‧‧ objects

82‧‧‧First texture

84‧‧‧Second Texture

86‧‧‧Third texture

88‧‧‧ Fourth texture

101‧‧‧VR headset equipment

102‧‧‧ Computing Equipment

103‧‧‧Camera

201 ~ 205‧‧‧ steps

900‧‧‧ technical equipment

901‧‧‧ processor

902‧‧‧Storage

903‧‧‧I / O interface

904‧‧‧display interface

905‧‧‧Network communication interface

906‧‧‧Bus

907‧‧‧operating system

908‧‧‧I / O module

909‧‧‧Communication Module

900A‧‧‧Graphics Processing Device

900B‧‧‧ processor

910‧‧‧Get Module

920‧‧‧Computing Module

930‧‧‧ rendering module

1000‧‧‧VR helmet

1010‧‧‧Head Tracker

1011‧‧‧Angle Sensor

1012‧‧‧Signal Processor

1013‧‧‧Data Transmitter

1014‧‧‧Display

1020‧‧‧CPU

1030‧‧‧GPU

1040‧‧‧ Display

1110‧‧‧VR Glasses

1112‧‧‧Angle Sensor

1114‧‧‧Signal Processor

1116‧‧‧Data Transmitter

1118‧‧‧Display

1120‧‧‧Host

C1‧‧‧ shooting location

C2‧‧‧ shooting location

C3‧‧‧ shooting location

Cview‧‧‧ average position

L41‧‧‧ Object

L80‧‧‧Left eye picture

LE‧‧‧left eye position

R45‧‧‧ Object

R80‧‧‧Right eye picture

RE‧‧‧Right eye position

S310 ~ S370‧‧‧step

FIG. 1 is a schematic diagram of a VR system according to an embodiment of the present application.

FIG. 2 is a schematic flowchart of a graphics processing method according to an embodiment of the present application.

FIG. 3 is a schematic flowchart of a graphics processing method according to an embodiment of the present application.

FIG. 4 is a schematic diagram of a scene to be presented according to an embodiment of the present application.

FIG. 5 is a schematic diagram of a scene taken in advance according to an embodiment of the present application.

FIG. 6 is a schematic diagram of videos obtained at different shooting positions according to an embodiment of the present application.

FIG. 7 is a schematic diagram of determining a target video according to an embodiment of the present application.

FIG. 8 is a schematic diagram of presenting a target video according to an embodiment of the present application.

FIG. 9A is a schematic structural diagram of a computing device where a graphics processing apparatus according to an embodiment of the present application is located.

FIG. 9B is a schematic block diagram of a processor according to an embodiment of the present application.

FIG. 10 is a schematic diagram of a virtual reality system according to an embodiment of the present application.

FIG. 11 is a schematic diagram of a virtual reality system according to another embodiment of the present application.

The technical solutions in this application will be described below with reference to the drawings.

Embodiments of the present application provide a graphics processing method, device, and VR system.

It should be understood that the methods and devices of the embodiments of the present application are applied in the field of VR scenes, for example, in the field of VR games, and also in other interactive scenes, such as interactive VR movies and interactive VR concerts. The embodiments of this application do not limit this.

Before describing the graphics processing method in the embodiments of the present application in detail, the real-time rendering technology involved in the embodiments of the present application is first introduced. The essence of real-time rendering technology is real-time calculation and output of graphics data, and its biggest characteristic is real-time. Currently, a processor in a personal computer (PC), a workstation, a game console, a mobile device, or a VR system performs operations at a speed of at least 24 frames per second. In other words, the rendering of a picture must be at least 1/24 second. In actual 3D games, the frame rate per second is even higher. Because of the real-time nature of real-time rendering, it is possible to achieve continuous playback of 3D games, as well as to enable users in 3D games to interact with characters or other objects in the game scene.

The real-time rendering involved in the embodiments of the present application may be implemented through a Central Processing Unit (CPU) or a Graphics Processing Unit (GPU), which is not limited in the embodiments of the present application. Specifically, the GPU is a type of processor specifically used to implement image computing work, which may exist in a graphics card, which is also called a display core, a vision processor, or a display chip. Is a kind of VR glasses

FIG. 1 is a schematic diagram of a VR system according to an embodiment of the present application. As shown in FIG. 1, the system includes a VR headset device 101 and a computing device 102.

The VR headset device 101 may be VR glasses or a VR helmet, and may include an angle sensor 1011, a signal processor 1012, a data transmitter 1013, and a display 1014. The angle sensor 1011 can collect attitude information of the observer.

The computing device 102 may be a smart terminal device such as a personal computer (PC), a notebook computer, or a smart mobile terminal device such as a smartphone, a PAD, or a tablet computer. The computing device 102 may include a CPU and a GPU for calculating and rendering an observation screen, and The observation screen is sent to the display 1014 for display. The signal processor 1012 and the data transmitter 1013 are mainly used for communication between the VR headset device 101 and the computing device 102.

In some examples, the VR system according to the embodiment of the present application may further include: a camera 103, configured to shoot videos of objects in the VR scene captured from multiple different shooting positions.

Based on the system shown in Figure 1, an embodiment of the present application proposes a graphics processing method. FIG. 2 is a flowchart of a graphics processing method 200 provided by an embodiment of the present application, and the method is executed by a computing device 102 in a VR system. As shown in Figure 2, the method includes the following steps:

Step 201: Obtain the position information of the observer.

In some examples, left-eye position information, right-eye position information, left-eye orientation information, and right-eye orientation information of the observer are obtained. The left-eye position information, the right-eye position information, the left-eye direction information, and the right-eye direction information are determined according to the collected current posture information of the user, and the posture information includes a head posture At least one of a message, a limb posture message, a trunk posture message, a muscle electrical stimulation message, an eye tracking message, a skin perception message, a motion perception message, and a brain signal message.

Step 202: Determine a target object in the virtual reality VR picture to be displayed according to the position information.

In some examples, the target object may be a character. Among them, the character is an object that is expected to be improved in its authenticity, that is, the target object.

For example, each scene or multiple scenes may have a target object list. When generating a VR scene, the target object in the target scene is found according to the target object list. As another example, in the game design of VR scenes, it is stipulated that the characters in the close range (scene within a certain range from the user) are the target objects, the objects other than the characters in the close range are not the target objects, and the distant view (the scene outside the user's range) All objects at) are not target objects, and so on. The determination of the target object in the scene may be performed by the processing device 34, for example, may be determined by a CPU in the processing device 34, which is not limited in the embodiment of the present application.

Step 203: Obtain at least two images corresponding to the target object that are stored in advance, and the at least two images are images respectively taken from different shooting positions.

In some examples, a video frame corresponding to the time information in each video is determined as the image according to the time information of the VR picture to be displayed from a plurality of videos taken in advance, where the VR picture The time information can be the current time of the VR screen.

Step 204: Generate a target image by using the at least two images according to the position information and a shooting position corresponding to the at least two images, where the target image is the position corresponding to the position of the observer. The image of the target object.

In some examples, the target image is rendered onto a first preset texture in the VR picture, wherein the first preset texture is based on a billboard patch technology.

In some examples, a first object in the VR picture is determined according to the location information; a three-dimensional model corresponding to the first object is determined from a three-dimensional model library; and the three-dimensional model is rendered to the VR picture On the second preset texture, wherein the VR picture includes the target object and the first object other than the target object, the second preset texture may be a background of the VR picture.

In some examples, the plurality of videos are videos that include only the target object after transparent processing of the original videos of the plurality of videos, and the target object may be a person.

In some examples, in the step of determining the target image, the left-eye position information and the right-eye position information are averaged to obtain an average position; according to the average position, from the pre-captured At least two videos are selected from a plurality of videos, and the multiple videos are obtained from different shooting positions; a video frame is selected from each of the at least two videos as the image; according to the A spatial position relationship between an average position and a shooting position of the at least two videos, and the target image is obtained by performing an operation on the image.

Specifically, after the average position is obtained, at least one video is selected from left and right sides of the average position, and one video frame corresponding to the time information is selected from the selected at least one video as the image. The time information may be the current time information of the VR picture, and the target image is obtained by performing an interpolation operation on the image according to a spatial position relationship between the average position and the shooting positions of the at least two videos.

In some examples, in the step of determining the target image, the left-eye position information and the right-eye position information are averaged to obtain an average position; according to the average position, from the pre-captured A target video is selected from a plurality of videos, wherein the distance between the shooting position of the target video and the average position is the smallest of the spatial distance between the shooting position of the plurality of pre-shot videos and the average position; One video frame is selected from the target video, and the video frame is used as the target image.

Specifically, after obtaining the average position, a video whose shooting position is closest to the average position is selected from the plurality of pre-shot videos, and a video frame corresponding to the time information is selected from the target videos as the foregoing. The target image, wherein the time information may be the current time information of the VR picture.

Step 205: Display the VR picture, and render the target image in the VR picture.

In some examples, the left-eye picture is determined according to the left-eye position information and the left-eye orientation information; the right-eye picture is determined according to the right-eye position information and the right-eye orientation information; The left eye is oriented to the message and the target image, and the left eye frame is rendered in real time, and the target image is rendered in the left eye frame; according to the right eye orientation message and the target image, it is rendered in real time. The right-eye picture, and rendering the target image in the right-eye picture.

The technical solution of the embodiment of the present application can determine the target object in the virtual reality VR picture to be displayed according to the position information of the observer; obtain at least two images corresponding to the target object stored in advance, and the at least two The images are images respectively taken from different shooting positions; according to the position information and the shooting positions corresponding to the at least two images, a target image is generated by using the at least two images, and the target image An image of the target object corresponding to the position of the observer; displaying the VR picture, and rendering the target image in the VR picture. This VR picture can truly show the real scene, and on the basis of maintaining the interactivity of the entire VR scene, it can provide users with a real sense of presence, which can improve the user experience.

FIG. 3 is a schematic flowchart of a graphics processing method 300 according to an embodiment of the present application. The method 300 is performed by the VR system 30. The VR system 30 may include a gesture collection device 32, a processing device 34, and a display device 36. The method 300 may include the following steps.

S310. Collect the current posture information of the user. It should be understood that S310 may be performed by the gesture collection device 32.

S320: Obtain left-eye position information, right-eye position information, left-eye orientation information, and right-eye orientation information of the user according to the posture information.

S330. Determine a target three-dimensional model from the three-dimensional model library according to the left-eye position information and the right-eye position information.

S340: Determine a target video according to the left-eye position information, the right-eye position information, and a plurality of videos shot in advance, where the plurality of videos are videos shot from different shooting positions, respectively.

S350: Render the left-eye image in real time according to the left-eye orientation information, the target three-dimensional model, and the target video.

S360: Render the right-eye picture in real time according to the right-eye orientation information, the target three-dimensional model, and the target video.

It should be understood that S320 to S360 may be executed by the processing device 34.

S370: Display a left-eye picture and a right-eye picture, where a VR scene is formed when the left-eye picture and the right-eye picture are displayed, and the VR scene includes an image of a target three-dimensional model and an image of a target video.

It should be understood that S370 may be performed by the display device 36.

The graphic processing method in the embodiment of the present application collects the pose information of the user to determine the positions of the left and right eyes of the user, determines the target three-dimensional model based on the position information of the left and right eyes of the user, and determines the target video based on a plurality of pre-shot videos, and renders them in real time. The rendering technology renders the left-eye image and the right-eye image separately to display the VR scene. The VR scene includes an image of the target three-dimensional model and an image of the target video. The target video can truly show the actual scene and maintain the entire scene. Based on the interactivity of VR scenes, it provides users with a real sense of presence, which can improve the user experience.

It should be understood that, generally speaking, the VR system 30 includes a VR headset device, and the display device 36 may be integrated into the VR headset device. The processing device 34 and / or the attitude collection device 32 in the embodiment of the present application may be integrated into a VR headset device, or may be deployed separately from the VR headset device. The above VR headset device may be a VR headset display device. , Such as VR glasses or VR helmets. The attitude collection device 32, the processing device 34, and the display device 36 may communicate through wired or wireless communication, which is not limited in the embodiment of the present application.

Each step of the graphics processing method 300 and the components of the VR system 30 are described in detail below.

In the embodiment of the present application, in S310, the gesture collection device 32 collects the current gesture information of the user.

The gesture collection device 32 may include a VR head-mounted display device, such as a sensor in VR glasses or a VR helmet. The sensor may include a light-sensitive sensor, such as an infrared sensor, a camera lens, etc .; the sensor may also include a force-sensitive sensor, such as a gyroscope; the sensor may also include a magnetic sensor, such as a brain-computer port; etc .; the sensor may also include a sound-sensitive sensor, etc. In the embodiment of the present application, the specific type of the sensor is not limited. The sensor in the VR head-mounted display device can collect at least one of a user's current head posture information, eye tracking information, skin perception information, muscle electrical stimulation information, and brain signal information. Then, the processing device 34 can determine the left-eye position information, the right-eye position information, the left-eye orientation information, and the right-eye orientation information of the user based on these messages.

In a specific example, in a VR scene, the user's perspective refers to the azimuth of the user's eye direction in the virtual space, including the position and orientation of the human eye. In virtual space, the user's perspective can change as the user's head poses change in real space. In a specific case, the change of the user's perspective in the virtual space is the same speed and direction as the change of the user's head posture in the real space. The user's perspective includes a left-eye perspective and a right-eye perspective, that is, the user's left-eye position, right-eye position, left-eye orientation, and right-eye orientation.

In this example, the sensor on the VR headset device worn by the user can sense movements such as head rotation, movement, and posture changes during the process of using the VR headset device, and perform calculations on various motions to obtain Relevant head posture information (such as the speed and angle of movement), the processing device 34 can determine the user's left-eye position information, right-eye position information, left-eye orientation information, and right-eye orientation information based on the obtained head posture information. .

The posture collection device 32 may further include a positioner, a control handle, a body-sensing glove, a body-sensing clothing, and other motion-sensing devices such as a treadmill, etc., for collecting posture information of the user, and then the processing device 34 processes to obtain the position of the left eye of the user Message, right eye position message, left eye direction message, and right eye direction message. Among them, the posture collection device 32 can collect user's limb posture information, trunk posture information, electrical muscle stimulation information, skin perception information, and motion perception information through a control handle, somatosensory gloves, somatosensory clothes, and a treadmill.

In a specific example, one or more locators may be provided on the VR headset device to monitor the user's head position (which may include height), orientation, and the like. At this time, the user may be provided with a positioning system in the real space where the VR headset device is worn, and the positioning system may perform positioning communication with one or more locators on the VR headset device worn by the user to determine that the user is in this reality. Specific position (including height) and orientation information in space. then. The above-mentioned posture information may be converted by the processing device 34 into information such as the relevant position (which may include the height) and the orientation of the user's head in the virtual space. That is, the processing device 34 obtains left-eye position information, right-eye position information, left-eye orientation information, and right-eye orientation information of the user.

It should be understood that the left-eye position information and the right-eye position information of the embodiment of the present application may be represented by coordinate values in the coordinate system; the left-eye orientation message and the right-eye orientation message may be represented by a vector in the coordinate system, However, this embodiment of the present application does not limit this.

It should also be understood that after the gesture collection device 32 collects the gesture information, it needs to send the gesture information to the processing device 34 through wired communication or wireless communication, which is not described in detail herein.

It should also be understood that the embodiments of the present application may also collect the posture information of the user in other ways, and obtain and / or represent the left-eye position information, the right-eye position information, the left-eye orientation information, and the right-eye orientation information through other methods. The application embodiment does not limit the specific manner.

In the design of a VR scene, for example, in a game design of a VR scene, a position is designed to correspond to an object group. In a specific example, the objects corresponding to the left eye position LE and the right eye position RE of the user are respectively shown in FIG. 4. The position of the left eye of the user corresponds to the object L41, the object 43, the object 44, the object 46, and the person 42, and the position of the right eye of the user corresponds to the object R45, object 43, object 44, object 46, and person 42. Among them, the character 42 is an object that is expected to be improved in its authenticity, and is a target object.

Specifically, determining which object in the object group corresponding to the left eye position or the right eye position of the user is the target object may be based on the design of the VR scene. For example, each scene or multiple scenes may have a target object list. When generating a VR scene, the target object in the target scene is found according to the target object list. As another example, in the game design of VR scenes, it is stipulated that the characters in the close range (scene within a certain range from the user) are the target objects, the objects other than the characters in the close range are not the target objects, and the distant view (the scene outside the user's range) All objects at) are not target objects, and so on. The determination of the target object in the scene may be performed by the processing device 34, for example, may be determined by a CPU in the processing device 34, which is not limited in the embodiment of the present application.

It should be understood that for VR scenes, objects other than the target object may be generated through 3D modeling in advance and stored in a 3D model library. Specifically, the 3D models of the object L41, the object 43, the object 44, the object R45, and the object 46 shown in FIG. 4 are all stored in a 3D model library. After the processing device 34 (for example, the CPU in the processing device 34) obtains the left-eye position information and the right-eye position information, the target three-dimensional model is determined from the 3D model library, that is, the object L41, the object 43, the object 44, the object R45, and the object 46 3D model for subsequent renderings. Of course, the target three-dimensional model may also be determined through other methods, which is not limited in the embodiment of the present application.

For the target object in the VR scene, such as the person 42 in the VR scene shown in FIG. 4, it is generated based on a plurality of videos shot in advance. The plurality of videos are videos including target objects, which are shot from different shooting positions, respectively.

Specifically, assuming that the target object is a person 42, in the embodiment of the present application, multiple videos about the person 42 are pre-shot from multiple shooting positions. FIG. 5 is a schematic diagram of a scene shot in advance. As shown in FIG. 5, the scene to be shot includes a person 42, an object 52, and an object 54. The scene to be shot is as close as possible to the situation of the VR scene finally displayed to increase the realism. For the scene to be shot, multiple shooting devices can be placed in the horizontal direction, and shooting is performed from the shooting position C1, the shooting position C2, and the shooting position C3, and the original videos of the people at different shooting positions can be obtained as shown in FIG.

It should be understood that when shooting a video in advance, shooting can be performed on a circle with a certain radius from the target object. The more and more densely selected shooting positions on this circle, the greater the probability that the same or similar left or right eye position of the user will be selected from it. The final selected or calculated target video is placed in the VR scene. The higher the authenticity.

Furthermore, in addition to the shooting position when shooting a pre-shooting video, in addition to a straight line or a circle with a certain radius from the target object, the shooting position can also form a plane or curved surface or even different positions in three-dimensional space, thereby achieving 360 degrees Panorama.

In the embodiment of the present application, the plurality of videos may be videos including only the target object after the original video is transparently processed. Specifically, the person 42 can be separated from the objects 52 and 54 constituting the background from the three videos respectively captured from the three shooting positions, and three videos including only the person 42 can be obtained. The three videos are videos that have been filmed at the same time for the same length of time.

Optionally, in the embodiment of the present application, the transparent processing may be processing based on an alpha (alpha) transparent technology. Specifically, if the 3D environment of the VR scene allows pixels to have a set of alpha values, the alpha value is used to record the transparency of the pixels, so that objects can have different degrees of transparency. In the embodiment of the present application, the target object person 42 in the original video may be processed as opaque, and the object 52 and the object 54 constituting the background are processed as transparent.

In a specific solution, S340 determines a target video according to the left-eye position information, the right-eye position information, and a plurality of videos taken in advance, which may include: comparing the left-eye position information and the right-eye position information. An average is obtained to obtain an average position; the target video is selected from the plurality of videos according to the average position, and a distance between a shooting position of the target video and the average position is the plurality of videos Of all the shooting positions, the closest to the average position.

It should be understood that in the embodiments of the present application, the left-eye position, the right-eye position, and the shooting position may be uniformly represented as coordinates of a virtual space in a VR scene, for example, coordinates in a three-axis coordinate system of x-axis, y-axis, and z-axis Ball coordinates. The left eye position, the right eye position, and the shooting position may also be expressed in other forms, which are not limited in the embodiment of the present application.

In this solution, the left eye position information and the right eye position information are averaged to obtain the average position. For example, taking the three-axis coordinate system as an example, the left eye position is (x1, y1, z1), and the right eye position is (x2, y2, z2), then the average position is ((x1 + x2) / 2, (y1 + y2) / 2, (z1 + z2) / 2). The video with the closest shooting position to the average position is selected from the multiple videos as the target video.

In the case where the multiple shooting positions are multiple positions on a circle with a certain radius from the target object, the shooting position of the target video is closest to the average position, which can be understood as the shooting position (xt, yt, zt) and the average position of the target video. The distance of ((x1 + x2) / 2, (y1 + y2) / 2, (z1 + z2) / 2) must be less than a preset threshold, that is, the distance between the shooting position of the target video and the average position is sufficiently small.

When multiple shooting positions are not on a circle with a certain radius from the target object, the shooting position of the target video is closest to the average position. It can be understood that the line segment formed by the average position and the target object and the shooting position of the target video and the target object constitute The included angle between the line segments is the smallest of the included angles between the average position and the line segment formed by the target object and all the shooting positions and the line segment formed by the target object.

In another specific solution, S340 determines the target video according to the left-eye position information, the right-eye position information, and multiple videos taken in advance, which may include: comparing the left-eye position information and the right-eye position The information is averaged to obtain an average position; at least two videos are selected from the plurality of videos according to the average position; a video frame corresponding to each of the at least two videos at a corresponding moment is extracted; Performing an interpolation operation on the at least two video frames according to the average position and the shooting positions of the at least two videos to obtain the current target video.

In this solution, at least one left and right shooting position of the average position of the left and right eyes of the user may be selected, and a video shot at least one left and right shooting position may be selected from a plurality of videos as a reference for calculating a target video. Intercepting video frames corresponding to at least two videos at the same time to obtain a target video.

In the case where the multiple shooting positions are multiple positions on a circle with a certain radius from the target object, selecting at least two videos from multiple videos may be selecting and averaging positions ((x1 + x2) / 2, (y1 + y2) / 2, (z1 + z2) / 2) at least two videos with the smallest distance. At least one of the shooting positions of at least two videos is distributed to the left of the average position, and at least one is distributed to the right of the average position.

In the case where the multiple shooting positions are not on a circle with a certain radius from the target object, selecting at least two videos from the multiple videos may be a line segment formed by the average position and the target object and the shooting positions of the at least two videos and the target object The included angle between the line segments is the smallest of the included angles between the average position and the line segment formed by the target object and all the shooting positions and the line segment formed by the target object. At least one of the shooting positions of at least two videos is distributed to the left of the average position, and at least one is distributed to the right of the average position.

It should be understood that in the embodiments of the present application, a video that can be used as a reference may also be selected according to other criteria, which is not limited in the embodiments of the present application.

It should also be understood that, in the embodiments of the present application, videos captured at different shooting positions represent different observation positions when observing a target object (for example, a person 42). In other words, the video frames corresponding to the three videos shown in Figure 6 at the same physical moment are images when viewed at different observation positions. The three shooting angles can correspond to three shooting positions C1, C2, and C3, respectively.

It should be understood that, in the embodiment of the present application, in addition to capturing multiple videos in advance, multiple groups of photos (or multiple groups of images) of the target object may be captured in advance from multiple shooting positions. According to the relationship between the position of the left eye and the position of the right eye (or average position) and multiple shooting positions, find at least two images corresponding to at least two shooting positions from a plurality of groups of images, and perform interpolation on the at least two images To get the target image. The specific interpolation algorithm will be described in detail below.

FIG. 7 is a schematic diagram of determining a target video according to an embodiment of the present application. According to the average position, at least two videos are selected from the multiple videos, and the video frame corresponding to each of the at least two videos is extracted at the corresponding moment. According to the average position and the shooting position of the at least two videos, at least two Each video frame is interpolated, and the specific process of obtaining the target video can be shown in FIG. 7.

When the user observes the VR scene, the observation position may change. For example, when the user faces the VR scene, the observation position may move in the left-right direction. The three shooting positions are C1, C2, and C3. C1, C2, and C3 can be represented by coordinate values of a three-dimensional rectangular coordinate system, can also be represented by coordinate values of a spherical coordinate system, and can also be represented by other methods, which are not limited in the embodiment of the present application. According to the left eye position information and the right eye position information of the user, the average position Cview when the user observes can be determined. As shown in Figure 7, the average position Cview is between C1 and C2. When determining the target video, because the average position Cview is between C1 and C2, the video shot in advance at the shooting positions C1 and C2 is selected as a reference. When generating the video frames (images) of the target video, simultaneously take out the video frames I1 and I2 corresponding to the videos corresponding to C1 and C2 at the same time, and then interpolate the two video frames I1 and I2, for example, linear interpolation . Among them, the weight of interpolation depends on the distance between the average position Cview and C1 and C2. The output video frame of the target video is Iout = I1 * (1- (C1-Cview / C1-C2)) + I2 * (1- (C2-Cview / C1-C2)).

It should be understood that only the case where the user's observation position moves in the left-right direction has been discussed above. If the user's observation position moves back and forth, because it is in a 3D scene in VR, the characters seen by the observer will naturally appear near large and small. The effect, although the angle of physical display should also be changed, but this change has little effect, and the general user will not care or observe. In addition, in a general scene, the user only moves back and forth, left and right, and rarely moves in a large range in the up and down direction. Therefore, for the target video determined according to the method of the embodiment of the present application, the user feels very distorted small.

It should be understood that, in the embodiment of the present application, a target object is used as an example for description. Of course, the target object may also be an animal that requires authenticity, or even a building or a plant, which is not limited in the embodiments of the present application.

Optionally, in the embodiment of the present application, S350 rendering the left-eye picture in real time according to the left-eye orientation information, the target three-dimensional model, and the target video may include: according to the left-eye orientation information, the target is The three-dimensional model is rendered onto a first texture; the target video is rendered onto a second texture according to the left-eye orientation information, where the first texture may be a background of the left-eye picture, and the second The texture is based on billboard patch technology; S360 renders the right-eye picture in real time according to the right-eye orientation message, the target three-dimensional model, and the target video, which may include: according to the right-eye orientation message, the target is three-dimensionally The model is rendered onto a third texture; the target video is rendered onto a fourth texture according to the right-eye orientation information, where the third texture may be the background of the right-eye picture and the fourth texture It is based on billboard face sheet technology.

Hereinafter, the process of rendering the left-eye picture and the right-eye picture in the embodiment of the present application will be described in detail with reference to FIG. 8. As described above, the processing device 34 (for example, the CPU therein) has determined the target three-dimensional model in S330, and the target video has been determined in S340. The processing device 34 (for example, the GPU therein) determines the left-eye picture that should be presented according to the left-eye orientation message; and determines the right-eye picture that should be presented according to the right-eye orientation message. For example, in the scene shown in FIG. 4, according to the left-eye orientation message (facing the person 42), it is determined that the object L41, the object 43, the object 44 and the person 42 are presented in the left-eye picture; according to the right-eye orientation message (facing the person 42) , Determine that the object 43, the object 44, the object R45, and the person 42 are present in the right-eye frame.

The processing device 34 (for example, the GPU therein) renders the target three-dimensional model object L41, object 43, and object 44 onto the first texture 82 of the left-eye picture L800, and renders the target video to the second texture of the left-eye picture L800. 84; rendering the target three-dimensional model object 43, object 44, and object R45 onto the third texture 86 of the right-eye picture R800, and rendering the target video onto the fourth texture 88 of the right-eye picture R800.

Specifically, for the left-eye picture and the right-eye picture, a billboard patch can be set at the position of a target object on the screen, and a target video can be presented on the billboard patch. Billboard technology is a method for rapid rendering in the field of computer graphics. In the case of high real-time requirements similar to 3D games, the use of billboard technology can greatly speed up the drawing and thus improve the smoothness of the 3D game screen. Billboard technology is to use 2D to represent an object in a 3D scene, so that the object always faces the user.

Specifically, the billboard face sheet may have an oblique angle in the left eye picture, and specific parameters of the oblique angle may be calculated according to the left eye position information; the billboard face sheet may have an oblique angle in the right eye picture, and the specific parameters of the oblique angle may be based on Right eye position information to calculate.

In fact, since the VR scene is rendered in real time, at any time, it can be considered that the aforementioned video frame obtained through interpolation is presented at the position of the target object. In a continuous period of time during which the scene changes, it can be equivalent to playing a video on a billboard patch.

As shown in Figure 8, a billboard patch is set at the position corresponding to the target object. Each frame of the video is drawn as a texture to the texture of the billboard patch, and each frame of the video will always face users.

It should be understood that when rendering the left-eye picture and the right-eye picture, a depth buffer technology and a billboard technology may be combined. Depth buffer technology helps target objects form occlusion relationships and size-ratio relationships with other objects at near and far distances. In the embodiment of the present application, other techniques may be used for rendering the target video, which is not limited in the embodiment of the present application.

It should also be understood that the embodiment of the present application further provides a graphics processing method, which includes steps S320 to S360, and the method is executed by a processor.

It should also be understood that, in various embodiments of the present invention, the size of the sequence numbers of the above processes does not mean the order of execution, and the execution order of each process should be determined by its function and internal logic, and should not deal with the embodiments of this application. The implementation process constitutes any limitation.

The graphics processing method according to the embodiment of the present application is described in detail above with reference to FIGS. 1 to 8. The device, the processor, and the VR system according to the embodiments of the present application will be described in detail below with reference to FIGS. 9A, 9B, and 10.

FIG. 9A is a schematic structural diagram of a computing device used for a graphics processing method in an embodiment of the present application. As shown in FIG. 9A, the computing device 900 includes a processor 901, a non-volatile computer-readable storage 902, an I / O interface 903, a display interface 904, and a network communication interface 905. These components communicate via a bus 906. In some embodiments of the present application, the memory 902 stores a plurality of program modules: an operating system 907, an I / O module 908, a communication module 909, and an image processing device 900A. The processor 901 may read the computer-readable instructions corresponding to the image processing apparatus 900A in the storage 902 to implement the solution provided by the embodiment of the present application.

In the embodiment of the present application, the I / O interface 903 may be connected to an input / output device. The I / O interface 903 sends input data received from the input device to the I / O module 908 for processing, and sends data output by the I / O module 908 to the output device.

The network communication interface 905 may send data received from the communication bus 906 to the communication module 909, and send data received from the communication module 909 through the bus 906.

In some examples, the computer-readable instructions corresponding to the image processing device 900A stored in the storage 902 may cause the processor 901 to execute: obtain the position information of the observer; and determine the to-be-displayed information according to the position information. A target object in a virtual reality VR picture; obtaining at least two images corresponding to the target object stored in advance, the at least two images are images respectively taken from different shooting positions; according to the position information A shooting position corresponding to the at least two images, and using the at least two images to generate a target image, where the target image is an image of the target object corresponding to the position of the observer; Describe the VR picture, and render the target image in the VR picture.

In some examples, the instruction may cause the processor 901 to determine, according to the time information of the VR picture to be displayed, a video frame corresponding to the time information in each video from a plurality of videos captured in advance as The image.

In some examples, the instructions may cause the processor 901 to render the target image onto a first preset texture in the VR picture, where the first preset texture is based on a billboard Pasta technology.

In some examples, the instructions may cause the processor 901 to obtain left-eye position information, right-eye position information, left-eye orientation information, and right-eye orientation information of the observer; wherein the VR picture Including a left-eye image and a right-eye image; determining the left-eye image according to the left-eye position information and the left-eye orientation information; determining the right-eye image based on the right-eye position information and the right-eye orientation information ; Rendering the left-eye picture in real time according to the left-eye orientation message and the target image, and rendering the target image in the left-eye picture; according to the right-eye orientation message and the target image Image, the right eye picture is rendered in real time, and the target image is rendered in the right eye picture.

In some examples, the instructions may cause the processor 901 to: determine a first object in the VR picture according to the position information; and determine a target three-dimensional model corresponding to the first object from a three-dimensional model library ; Rendering the three-dimensional model onto a second preset texture of the VR picture. In some examples, the instruction may cause the processor 901 to: average the left-eye position information and the right-eye position information to obtain an average position; according to the average position, from the pre-photographed At least two videos are selected from a plurality of videos, and the multiple videos are obtained from different shooting positions; a video frame is selected from each of the at least two videos as the image; according to the A spatial position relationship between an average position and a shooting position of the at least two videos, and the target image is obtained by performing an operation on the image.

In some examples, the instruction may cause the processor 901 to: average the left-eye position information and the right-eye position information to obtain an average position; according to the average position, from the pre-photographed A target video is selected from a plurality of videos, wherein the distance between the shooting position of the target video and the average position is the smallest of the spatial distance from the shooting position of the plurality of pre-shot videos and the average position; Selecting a video frame from the target video, and using the video frame as the target image.

In some examples, the plurality of videos are videos including only the target object after the original videos of the plurality of videos are transparently processed, and the target object is a person.

In some examples, the left-eye position information, the right-eye position information, the left-eye orientation information, and the right-eye orientation information are determined according to the collected current posture information of the user.

In some examples, the gesture information includes at least one of a head gesture message, a limb gesture message, a trunk gesture message, an electrical muscle stimulation message, an eye tracking message, a skin-aware message, a motion-aware message, and a brain signal message.

FIG. 9B is a schematic block diagram of a processor 900BB according to an embodiment of the present application. The processor 900B may correspond to the processing device 34 described above. As shown in FIG. 9, the processor 900B may include an obtaining module 910, a computing module 920, and a rendering module 930.

The obtaining module 910 is configured to obtain left-eye position information, right-eye position information, left-eye orientation information, and right-eye orientation information of a user.

The calculation module 920 is configured to determine a target three-dimensional model from a three-dimensional model library according to the left-eye position information and the right-eye position information obtained by the acquisition module; The eye position information, the right eye position information, and a plurality of videos shot in advance determine a target video, where the plurality of videos are videos shot from different shooting positions, respectively.

The rendering module 930 is configured to render the left-eye picture in real time according to the left-eye orientation information, the target three-dimensional model, and the target video; the rendering module 930 is further configured to use the right-eye orientation information, the target three-dimensional model, and the target The target video renders the right eye picture in real time; wherein the left eye picture and the right eye picture form a VR scene when displayed on a virtual reality VR display, and the VR scene includes an image of the target three-dimensional model and An image of the target video.

The graphics processing device according to the embodiment of the present application determines a target three-dimensional model according to the position information of the left and right eyes of the user, and determines a target video according to a plurality of videos shot in advance, and renders the left-eye picture and the right-eye picture by rendering technology in real time. , Thereby displaying a VR scene, where the VR scene includes an image of a target three-dimensional model and an image of a target video, and the target video can truly show the actual scene, while providing the user with real reality while maintaining the interactivity of the entire VR scene Presence, which can enhance the user experience.

Optionally, as an embodiment, the rendering module 930 may be specifically configured to: render the target three-dimensional model to the first texture according to the left-eye orientation information; and render the target three-dimensional model according to the left-eye orientation information. The target video is rendered onto a second texture, wherein the second texture is based on a billboard patch technology; the target three-dimensional model is rendered onto a third texture according to the right eye orientation information; The right-eye-oriented information is used to render the target video onto a fourth texture, where the fourth texture is based on a billboard patch technology.

Optionally, as an embodiment, the calculating module 920 determines the target video according to the left-eye position information, the right-eye position information, and a plurality of videos taken in advance, which may include: regarding the left-eye position Average the information and the right eye position information to obtain an average position; select at least two videos from the plurality of videos according to the average position; and place each of the at least two videos at a corresponding time The corresponding video frames are extracted; according to the average position and the shooting positions of the at least two videos, an interpolation operation is performed on the at least two video frames to obtain the current target video.

Optionally, as an embodiment, the calculating module 920 determines the target video according to the left-eye position information, the right-eye position information, and a plurality of videos taken in advance, which may include: regarding the left-eye position Average the information and the right eye position information to obtain an average position; and select the target video from the plurality of videos according to the average position, wherein a shooting position of the target video and the average position The distance of is the closest to the average position among all the shooting positions of the plurality of videos.

Optionally, as an embodiment, the plurality of videos are videos that include only the target object after being transparently processed on the original video.

Optionally, as an embodiment, the target object is a person.

Optionally, as an embodiment, the left-eye position information, the right-eye position information, the left-eye orientation information, and the right-eye orientation information obtained by the acquisition module 910 are based on the collected information. Determined by the user's current posture message.

Optionally, as an embodiment, the posture information includes at least one of head posture information, limb posture information, trunk posture information, muscle electrical stimulation information, eye tracking information, skin perception information, motion perception information, and brain signal information. One.

It should be understood that the processor 900B may be a CPU or a GPU. The processor 900B may also include the functions of the CPU and the GPU. For example, the functions of the acquisition module 910 and the calculation module 920 (S320 to S340) are executed by the CPU, and the functions of the rendering module 930 (S350 and S360) are performed by The execution of the GPU is not limited in the embodiment of the present application.

FIG. 10 is a schematic diagram of a VR system according to an embodiment of the present application. FIG. 10 shows a VR helmet 1000. The VR helmet 1000 may include a head tracker 1010, a CPU 1020, a GPU 1030, and a display 1040. Among them, the head tracker 1010 corresponds to the attitude collecting device, the CPU 1020 and the GPU 1030 correspond to the processing device, and the display 1040 corresponds to the display device. More details.

It should be understood that the head tracker 1010, the CPU 1020, the GPU 1030, and the display 1040 shown in FIG. 10 are integrated in the VR helmet 1000. There may be other posture collection devices outside the VR helmet 1000, which collect the posture information of the user and send it to the CPU 1020 for processing, which is not limited in this embodiment of the present application.

FIG. 11 is a schematic diagram of another VR system according to an embodiment of the present application. FIG. 11 shows a VR system composed of VR glasses 1110 and a host 1120. The VR glasses 1110 may include an angle sensor 1112, a signal processor 1114, a data transmitter 1116, and a display 1118. Among them, the angle sensor 1112 corresponds to a gesture collection device, the host 1120 includes a CPU and a GPU to calculate and render a picture corresponding to a processing device, and a display 1118 corresponds to a display device. The angle sensor 1112 collects the posture information of the user, and sends the posture information to the host 1120 for processing. The host 1120 calculates and renders the left-eye frame and the right-eye frame, and sends the left-eye frame and the right-eye frame to the display 1118 for display. The signal processor 1114 and the data transmitter 1116 are mainly used for communication between the VR glasses 1110 and the host 1120.

There may be other posture collection devices outside the VR glasses 1110, which collect the posture information of the user and send it to the host 1120 for processing, which is not limited in this embodiment of the present application.

The virtual reality system of the embodiment of the present application collects posture information of the user to determine the positions of the left and right eyes of the user, determines the target three-dimensional model according to the position information of the left and right eyes of the user, and determines the target video according to multiple videos taken in advance. Rendering methods Rendering technology renders left-eye and right-eye images separately to display VR scenes. The VR scene includes the image of the target three-dimensional model and the image of the target video. The target video can truly show the actual scene and maintain Based on the interactivity of the entire VR scene, it provides users with a real sense of presence, which can improve the user experience.

An embodiment of the present application further provides a computer-readable storage medium having instructions stored thereon, and when the instructions are run on a computer, the computer is caused to execute the graphics processing method of the foregoing method embodiment. Specifically, the computer may be the aforementioned VR system or a processor.

The embodiment of the present application further provides a computer program product including instructions, characterized in that, when a computer runs the finger of the computer program product, the computer executes the graphics processing method of the above method embodiment. Specifically, the computer program product can run in a VR system or a processor.

In the above embodiments, it may be implemented in whole or in part through software, hardware, firmware, or any combination thereof. When implemented in software, it may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, the processes or functions according to the embodiments of the present application are generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be from a website site, computer, server, or The data center transmits to another website site, computer, server or data center through wired (such as coaxial cable, optical fiber, Digital Subscriber Line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) . The computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, a data center, and the like that includes one or more available medium integration. The usable medium may be a magnetic medium (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (for example, a Digital Video Disc (DVD)), or a semiconductor medium (for example, a solid state disk (Solid State Disk, SSD)) and so on.

It should be understood that the first, second, and various digital numbers referred to herein are only for the convenience of description and are not intended to limit the scope of the present application.

It should be understood that the term "and / or" in this document is only an association relationship describing an associated object, which means that there can be three kinds of relationships, for example, A and / or B can mean: A exists alone, A and B exist simultaneously, alone There are three cases of B. In addition, the character "/" in this article generally indicates that the related objects are an "or" relationship.

Those having ordinary knowledge in the technical field may realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices, and units described above can refer to the corresponding processes in the foregoing method embodiments, and are not repeated here.

In the several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods may be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. In actual implementation, there may be another division manner. For example, multiple units or components may be combined or may be combined. Integration into another system, or some features can be ignored or not implemented. In addition, the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some ports, devices or units, which may be electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, which may be located in one place, or may be distributed on multiple network units. . Some or all of the units may be selected according to actual needs to achieve the objective of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each of the units may exist separately physically, or two or more units may be integrated into one unit.

The above description is only the specific implementation of this application, but the scope of protection of this application is not limited to this. Those skilled in the art can easily think of changes or replacements within the technical scope disclosed in this application. Covered within the scope of this application. Therefore, the protection scope of this application shall be subject to the protection scope of the patent application scope.

Claims (34)

A graphics processing method applied to a computing device includes: obtaining position information of an observer; determining a target object in a virtual reality (VR) picture to be displayed according to the position information; and obtaining the target stored in advance At least two images corresponding to the object, the at least two images are images respectively taken from different shooting positions; according to the position information and the shooting positions corresponding to the at least two images, using the at least Two images generate a target image, the target image being an image of the target object corresponding to the position of the observer; and displaying the VR picture, and rendering the target picture in the VR picture Image, wherein obtaining the position information of the observer includes: obtaining left-eye position information, right-eye position information, left-eye orientation information, and right-eye orientation information of the observer; wherein the VR picture includes the left eye A picture and a right-eye picture, and displaying the VR picture, and rendering the target image in the VR picture include: determining the left-eye position information and the left-eye direction information according to The left-eye picture; determining the right-eye picture according to the right-eye position information and the right-eye orientation message; rendering the left-eye in real time according to the left-eye orientation message and the target image An eye picture, and rendering the target image in the left eye picture; and rendering the right eye picture in real time according to the right eye orientation information and the target image, and rendering in the right eye picture The target image, wherein generating the target image by using the at least two images according to the shooting information corresponding to the position information and the at least two images includes: The right eye position information is averaged to obtain an average position; a target video is selected from the plurality of pre-captured videos according to the average position, wherein the shooting position of the target video and the average position are The distance is the smallest of the spatial distance between the shooting position of the plurality of pre-shot videos and the average position; and selecting a video frame from the target video, and using the video frame as the target image. The method according to item 1 of the scope of patent application, wherein obtaining at least two images corresponding to the target object stored in advance includes: from a plurality of videos shot in advance according to the time information of the VR picture to be displayed Determining at least two video frames corresponding to the time information as the at least two images. The method according to item 1 of the scope of patent application, further comprising: rendering the target image onto a first preset texture in the VR picture, wherein the first preset texture is based on a billboard (billboard ) Patch technology. The method according to item 1 of the scope of patent application, further comprising determining a first object in the VR frame according to the position information; determining a three-dimensional model corresponding to the first object from a three-dimensional model library; and The three-dimensional model is rendered onto a second preset texture of the VR picture. The method according to item 2 of the scope of patent application, wherein the plurality of videos are videos including only the target object after the original videos of the plurality of videos are transparently processed. The method according to item 5 of the scope of patent application, wherein the target object is a person. The method according to item 1 of the scope of patent application, wherein the left-eye position information, the right-eye position information, the left-eye orientation information, and the right-eye orientation information are based on the collected current posture information of the user definite. The method according to item 7 of the scope of patent application, wherein the posture information includes head posture information, limb posture information, trunk posture information, muscle electrical stimulation information, eye tracking information, skin perception information, motion perception information, and brain signals At least one of the messages. A graphics processing device includes a processor and a storage, and the storage stores computer-readable instructions that can cause the processor to perform: obtaining an observer's location information; and determining a virtual to be displayed according to the location information A target object in a virtual reality (VR) picture; acquiring at least two images corresponding to the target object stored in advance, the at least two images are images taken from different shooting positions respectively; according to A shooting position corresponding to the position information and the at least two images, and using the at least two images to generate a target image, where the target image is a map of the target object corresponding to the position of the observer And displaying the VR picture and rendering the target image in the VR picture, wherein the instruction may cause the processor to: obtain left-eye position information, right-eye position information of the observer, A left-eye orientation message and a right-eye orientation message; wherein the VR picture includes a left-eye picture and a right-eye picture; and determining the left-eye according to the left-eye position message and the left-eye orientation message Determine the right-eye picture according to the right-eye position information and the right-eye orientation information; render the left-eye picture in real time according to the left-eye orientation information and the target image, and Rendering the target image in the left-eye frame; and rendering the right-eye frame in real-time according to the right-eye orientation message and the target image; and rendering the target image in the right-eye frame. For example, the instructions may cause the processor to: average the left-eye position information and the right-eye position information to obtain an average position; and based on the average position, from the plurality of videos shot in advance Select a target video, wherein the distance between the shooting position of the target video and the average position is the smallest of the spatial distance from the shooting position of the plurality of pre-shot videos and the average position; and A video frame is selected from the target video, and the video frame is used as the target image. The device according to item 9 of the scope of patent application, wherein the instruction may cause the processor to determine the description in each video from a plurality of pre-shot videos according to the time information of the VR picture to be displayed. At least two video frames corresponding to the time information are used as the at least two images. The device according to item 9 of the scope of patent application, wherein the instruction may cause the processor to: render the target image onto a first preset texture in the VR picture, wherein the first The preset texture is based on billboard patch technology. The device according to item 11 of the scope of patent application, wherein the instructions enable the processor to: determine a first object in the VR frame according to the position information; and determine the first object from a three-dimensional model library. A target three-dimensional model corresponding to an object; and rendering the three-dimensional model onto a second preset texture of the VR picture. The device according to item 10 of the scope of patent application, wherein the plurality of videos are videos including only the target object after transparent processing of the original videos of the plurality of videos. The device according to item 13 of the patent application scope, wherein the target object is a person. The device according to item 9 of the scope of patent application, wherein the left-eye position information, the right-eye position information, the left-eye orientation information, and the right-eye orientation information are based on the collected current user information The gesture message is ok. The device according to item 15 of the scope of patent application, wherein the posture information includes head posture information, limb posture information, trunk posture information, electrical muscle stimulation information, eye tracking information, skin perception information, motion perception information, and brain signals At least one of the messages. The device according to any one of claims 9 to 16, wherein the processor includes at least one of a Central Processing Unit (CPU) and a Graphics Processing Init (GPU). A graphics processing method adapted to a computing device includes: collecting current posture information of an observer; obtaining position information of the observer according to the posture information; and determining a virtual reality to be displayed according to the position information. (VR) a target object in the picture; acquiring at least two images corresponding to the target object stored in advance, the at least two images are images respectively taken from different shooting positions; according to the position information and A shooting position corresponding to the at least two images, using the at least two images to generate a target image, the target image being an image of the target object corresponding to the position of the observer; The VR picture, and rendering the target image in the VR picture, wherein the obtaining the position information of the observer includes: obtaining the left-eye position information, the right-eye position information, and the left-eye orientation information of the observer And the right eye orientation message; wherein the VR picture includes a left eye picture and a right eye picture, the displaying the VR picture, and rendering the target image in the VR picture, Including: determining the left-eye picture according to the left-eye position information and the left-eye orientation information; determining the right-eye picture according to the right-eye position information and the right-eye orientation information; and Towards the message and the target image, rendering the left-eye picture in real time, and rendering the target image in the left-eye picture; according to the right-eye orientation message and the target image, Rendering the right-eye picture in real time and rendering the target image in the right-eye picture, wherein according to the position information and a shooting position corresponding to the at least two images, using the at least two The image generating a target image includes: averaging the left-eye position information and the right-eye position information to obtain an average position; and selecting a target from the plurality of pre-shot videos according to the average position. A video, wherein the distance between the shooting position of the target video and the average position is the smallest of the spatial distances between the shooting position of the plurality of previously shot videos and the average position; and from the target view Select a video frame in the frequency, and use the video frame as the target image. The method according to item 18 of the scope of patent application, wherein according to the time information of the VR picture to be displayed, a video frame corresponding to the time information in each video is determined from the multiple videos taken in advance as the picture image. The method according to item 18 of the patent application scope, further comprising: rendering the target image onto a first preset texture in the VR picture, wherein the first preset texture is based on a billboard ( billboard) patch technology. The method of claim 18, further comprising: determining a first object in the VR frame based on the position information; determining a three-dimensional model corresponding to the first object from a three-dimensional model library; and Rendering the three-dimensional model onto a second preset texture of the VR picture. The method according to item 19 of the scope of patent application, wherein the plurality of videos are videos including only the target object after transparent processing of the original videos of the plurality of videos. The method according to item 22 of the scope of patent application, wherein the target object is a person. The method according to item 18 of the scope of patent application, wherein said collecting current observer posture information includes collecting current observer posture information, limb posture information, trunk posture information, electrical muscle stimulation information, At least one of an eye-tracking message, a skin-aware message, a motion-aware message, and a brain signal message. A virtual reality (VR) system includes a gesture collection device, a processing device, and a display device. The gesture collection device is configured to: collect current pose information of an observer; the processing device is configured to: Information to obtain the position information of the observer; determine the target object in the VR picture to be displayed according to the position information; obtain at least two images corresponding to the target object stored in advance, the at least two images Are images taken from different shooting positions respectively; and a target image is generated by using the at least two images according to the position information and the shooting positions corresponding to the at least two images, where the target images are An image of the target object corresponding to the position of the observer; the display device is configured to display the VR image and render the target image in the VR image, wherein the processing device obtains The observer's left-eye position information, right-eye position information, left-eye orientation information, and right-eye orientation information; wherein the VR picture includes a left-eye picture and a right-eye picture, where The processing device determines the left-eye picture according to the left-eye position information and the left-eye orientation information; determines the right-eye picture according to the right-eye position information and the right-eye orientation information; The left-eye orientation message and the target image, rendering the left-eye picture in real time, and rendering the target image in the left-eye picture; and according to the right-eye orientation message and the target Image, rendering the right-eye picture in real time, and rendering the target image in the right-eye picture, wherein the processing device uses the position information and a shooting position corresponding to the at least two images to use Generating a target image by the at least two images includes: averaging the left-eye position information and the right-eye position information to obtain an average position; and based on the average position, from a plurality of previously photographed A target video is selected from the videos, and the distance between the shooting position of the target video and the average position is the smallest of the spatial distances between the shooting positions of the plurality of pre-shot videos and the average position. And selecting a video frame from the target video, and the video frame as the target image. The VR system as described in claim 25, wherein the processing device determines a video corresponding to the time information in each video from a plurality of pre-shot videos according to the time information of the VR picture to be displayed. A frame is used as the image. The VR system as described in claim 25, wherein the processing device renders the target image onto a first preset texture in the VR picture, wherein the first preset texture is based on Billboard patch technology. The VR system according to item 25 of the scope of patent application, wherein the processing device is further configured to: determine a first object in the VR frame according to the position information; and determine the first object from a three-dimensional model library. A three-dimensional model corresponding to an object; and rendering the three-dimensional model onto a second preset texture of the VR picture. The VR system according to item 25 of the scope of patent application, wherein the plurality of videos are videos including only the target object after transparent processing of the original videos of the plurality of videos. The VR system according to item 29 of the application, wherein the target object is a person. The VR system according to item 25 of the scope of patent application, wherein the posture collecting device is specifically configured to collect the current head posture information, limb posture information, trunk posture information, electrical muscle stimulation information, and eye tracking information of the user. , Skin-aware messages, motion-aware messages, and brain signals. The VR system according to item 25 of the scope of patent application, wherein the processing device includes at least one of a Central Processing Unit (CPU) and a Graphics Processing Init (GPU). A computer storage medium stores instructions thereon. When the instructions are run on a computer, the computer is caused to execute the method according to any one of claims 1 to 8. A computer storage medium stores instructions thereon, and when the instructions are run on a computer, the computer is caused to execute the method according to any one of claims 18 to 24 of the scope of patent application.