JP7290852B2 - MULTI-CONTENT DRAWING SYSTEM AND MULTI-CONTENT DRAWING METHOD - Google Patents

Description

The present invention relates to a multi-content drawing system and a multi-content drawing method.

2. Description of the Related Art In recent years, a large number of services that reproduce virtual worlds during computer games and chats have appeared. Such services allow users to upload their own models, making them feel as if they have visited the world.

For example, Patent Document 1 discloses game processing means for performing game processing including movement of a plurality of characters in a virtual world, and identification means for identifying one character as a specific character from a plurality of characters based on predetermined conditions. and a determining means for determining whether or not there is commonality in the movement of a plurality of non-specific characters excluding a specific character among the plurality of characters, and if it is determined that there is no commonality by the determining means, the specified determination means for determining a character as a priority display character, and determining at least one non-specific character as a priority display character if it is determined that there is commonality; and at least the priority display character among a plurality of characters. A system is disclosed that includes adjustment means for adjusting drawing-related parameters such that the .DELTA.

Content structures that allow users to upload their own models are not widely implemented. In such a system, users must download other users' avatars in advance in order for users to enter the same server, room, or other space where they can see each other at the same time. Assuming such a structure entails problems such as an increase in the amount of data transfer and rendering costs in user devices.

That is, the more participants there are, the more data each user must download from the server. Also, if some users upload a fairly large model, other users will have to download it as well, consuming a lot of network bandwidth. And since the rendering of this model is done by each client, users of low-spec devices have the problem that the processing load of the device changes greatly depending on the uncontrollable conditions such as other people entering the display area. .

JP 2015-71012 A

In this way, in virtual space services where users can freely upload models, the current situation is that the number of connections at one time is limited to about 20 to 30 people.

SUMMARY OF THE INVENTION The present invention has been made in view of the above problems, and an object of the present invention is to provide a multi-content rendering system and a multi-content rendering method that render a large number of various types of content existing in a specific area with an optimum load. .

One aspect of the present invention is a multi-content rendering system that renders a large number of various types of content existing in a specific area to which a plurality of client terminals are connected , and generates a rendering result of at least the specific area containing the content. An area rendering server and a content rendering server that perform calculations related to content rendering are provided. Information on content including registered characters is sent to the content rendering server from each client terminal, and the area rendering server receives information on position and viewpoint. It is characterized by transmitting information to a content rendering server and receiving information as a picture regarding content rendering from the content rendering server.

According to one aspect of the present invention, the content server only performs calculations on registered models, and the region rendering server receives only graphical information regarding the rendering of the content without requiring calculations performed by the content server. It is possible to render a large number of various types of content existing in a specific area with an optimum load.

At this time, in one aspect of the present invention, the content may include a 3D model.

Even information that requires heavy data processing such as a 3D model can be rendered with an optimum load because, according to one aspect of the present invention, information is exchanged as an image regarding rendering.

Further, in one aspect of the present invention, the image information related to rendering may be a G-buffer based on delayed shading.

In this way, even if the content has a large drawing load, it is only the time of calculation in the content rendering server that becomes heavy. load will not be affected.

Further, according to one aspect of the present invention, the image information related to rendering may include rendering information based on forward rendering.

Image information regarding rendering is not limited to deferred shading, and may be based on forward rendering.

Further, in one aspect of the present invention, a specific area may be rendered by processing of a plurality of area rendering servers.

By dividing and managing by a plurality of area rendering servers, it is possible to reduce the load even when rendering a large area.

Another aspect of the present invention is a multi-content rendering method for rendering a large number of various types of content present in a specific area to which a plurality of client terminals are connected . A calculation step of performing calculations related to drawing content including a character ; a drawing condition acquisition step of obtaining information about the position and viewpoint of content in the drawing means; and a drawing step of generating a drawing result of a specific area containing the content from information about the rendering as an image.

According to another aspect of the present invention, the computing means only performs operations related to content rendering, and the rendering means does not require computations related to content rendering, and receives only image information related to content rendering, so that a specific It is possible to render a large number of various types of contents present in the area with an optimum load.

Then, in another aspect of the invention, the content may include a 3D model.

Even information that requires heavy data processing such as a 3D model can be rendered with an optimum load because, according to one aspect of the present invention, information is exchanged as an image regarding rendering.

Also, in another aspect of the present invention, the image information related to rendering may be a G-buffer based on deferred shading.

In this way, even if the content has a large drawing load, it is only the time of calculation in the content rendering server that becomes heavy. load will not be affected.

Further, according to one aspect of the present invention, the image information related to rendering may include rendering information based on forward rendering.

Image information regarding rendering is not limited to deferred shading, and may be based on forward rendering.

In another aspect of the present invention, a specific area is divided by a plurality of drawing means, and the drawing means for generating the drawing results acquires the drawing results of the area managed by the drawing means from the adjacent drawing means. You may do so.

By dividing and managing by a plurality of drawing means, it is possible to reduce the load even when drawing a large area.

Another aspect of the present invention is a program for causing a computer to execute the multi-content drawing method.

As described above, according to the present invention, it is possible to render a large number of various types of content existing in a specific area with an optimum load.

1 is a schematic diagram showing the configuration of a conventional drawing system; FIG. 1 is a schematic diagram showing the configuration of a multi-content rendering system according to one embodiment of the present invention; FIG. FIG. 4 is a schematic diagram showing the configuration of a multi-content drawing system according to another embodiment of the present invention; 1 is a flow diagram showing an overview of a multiple content rendering method according to an embodiment of the present invention; FIG. FIG. 4 is a schematic diagram showing a rendering method when a specific area is divided by a plurality of rendering means;

Preferred embodiments of the present invention will be described in detail below. It should be noted that the present embodiment described below does not unduly limit the content of the present invention described in the claims, and all of the configurations described in the present embodiment are essential as means for solving the present invention. not necessarily.

As an example of application of the multi-content rendering system according to an embodiment of the present invention, users register and operate customized models (participant characters, etc.) in virtual content such as games and virtual worlds. A situation is assumed in which an image corresponding to the viewpoint of each user (model) is drawn on the client terminal. FIG. 1 is a schematic diagram showing the configuration of a conventional rendering system, and FIG. 2 is a schematic diagram showing the configuration of a multi-content rendering system according to one embodiment of the present invention.

In the conventional drawing system 100, as shown in FIG. 1, the user registers (uploads) the model M from the client terminal 101 to the rendering server 102, and then requests the rendering result from the rendering server 102. FIG. Also, the user transmits input data such as movement of the character M from the client terminal 101 to the rendering server 102 .

As in the drawing system 100 shown in FIG. 1, the conventional method of downloading the drawing information of each of the models M, M1, and M2 based on the viewpoint from the model M of a certain user to the drawing device requires network There was a problem of wasting the bandwidth of the device or increasing the calculation load of the device when model information is downloaded and drawn on the device.

In addition, in order to deal with cases such as when other 3D models M1 and M2 enter the field of view S of the user's model M, the rendering server 102 must download the models M1 and M2 from the user in advance. A lot of bandwidth was wasted. In order to solve such problems, it is necessary to avoid a state in which the model data must be suddenly downloaded.

The present invention was invented to solve such problems, and one aspect of the present invention is, as shown in FIG. comprising at least an area rendering server 12 for generating a rendering result of a specific area R and a content rendering server 13 for performing calculations related to rendering of the content, the area rendering server 12: It is characterized by transmitting information on position and viewpoint to the content rendering server 13 and receiving information as an image on content rendering from the content rendering server 13 .

In the present invention, with such a configuration, after the model M is stored (uploaded) in the content rendering server 13A, it is not necessary to transfer it to another server, and only the rendering results on the spot are obtained as needed. is good, thus solving the bandwidth problem due to the model M download.

Even if the model drawing in the content rendering server 13 is heavy, it is only the calculation of the model that becomes heavy, and does not affect the overall drawing processing load. In the case of a user's character (avatar) with a heavy model, only the content server is heavy. A high-performance computer can be allocated only to the content server according to the user's demand, or the content can simply be placed in a heavy state. In addition, depending on the embodiment of the present invention, it is possible to avoid placing the load on the user who sees the content by strengthening the content rendering server 13A where the user raises his/her own model M, and put the load on the uploader of the content. .

In addition, since what is delivered to the user side is not the model data but the rendering result of the content, the rendering result can be obtained without letting the delivery destination know the information of the content. Therefore, it is possible to protect the content without requiring the user to download the 3D model of the content. In addition, since there is no need to download the 3D model to the area rendering server 12, the model display can be used in the service without passing the 3D model to the service provider when the user himself distributes the model data. can.

FIG. 3 is a schematic diagram showing the configuration of a multi-content drawing system 20 according to another embodiment of the invention. In the multiple content rendering system 20 according to one embodiment of the present invention, as shown in FIG. 3, a specific area R can be divided and managed by a plurality of area rendering servers 22 (22A to 22P). . As a result, the load can be reduced even when drawing a large area. Details will be described later.

Next, configurations of the area rendering server 12 and the content rendering server 13, which are components of the multiple content rendering system according to one embodiment of the present invention, will be described.

The area rendering server 12 and the content rendering server 13 each include one or more CPUs (Central Processing Units), ROMs (Read Only Memories), RAMs (Random Access Memories), communication units, etc., which should be provided as servers. A GPU (Graphics Processing Unit) may be provided. In addition, an input unit such as a keyboard and a touch panel, an output unit such as a display monitor, and the like may be provided as necessary. The number and performance of each component may be selected according to the situation, depending on the scale of the specific area R to be rendered, the type and number of contents, the computational load, and the like. Also, the area rendering server 12 and the content rendering server 13 each have a program for realizing a multiple content rendering method, which will be described later.

The CPUs of the area rendering server 12 and the content rendering server 13 have a function of controlling the operation of each component provided in each server according to data received via mutual communication units and various programs stored in the ROM. . In the present embodiment, the CPU controls operations for drawing multi-content according to a program for executing the multi-content drawing method. The CPU also has a function of appropriately storing necessary data and the like in a RAM (random access memory) or the like when executing these various types of processing. A GPU (Graphics Processing Unit) is a computing device specialized for image processing, and is preferably provided in each server in order to execute rendering processing and the like.

Memories such as ROM and RAM have the function of storing information necessary for controlling the operation of each component of the multiple content rendering system 10 . In this embodiment, the memory stores a program for executing the multiple content rendering method executed by the multiple content rendering system 10 between the region rendering server 12 and the content rendering server 13 .

The communication unit has a function of transmitting and receiving data among the area rendering server 12 , the content rendering server 13 and the client terminal 11 . The communication unit communicates with other servers and terminals through wired or wireless communication lines. When there are multiple area rendering servers 12, each area rendering server 12 can communicate with at least other adjacent area rendering servers, and each area rendering server 12 can communicate with each of the one or more content rendering servers 13. can communicate with

Next, processing performed in each of the client terminal (user device) 11, area rendering server 12, and content rendering server 13 will be described.

A client terminal (user device) 11 is a terminal used by a user when using the multiple content drawing system 10 according to one embodiment of the present invention. For example, it refers to an information terminal device equipped with an arithmetic device capable of various arithmetic processing, such as a general-purpose computer, a tablet terminal, a smart phone, and a mobile information terminal. The client terminal (user device) 11 obtains the area rendering server 12 to which the request is sent from the position information of the character (avatar) M registered by the client terminal (user device) and requests the drawing result. Also, the client terminal (user device) 11 operates scene information such as the position of the avatar M synchronized on the server as necessary. The area rendering server 12 may manage the inside of the area and issue a request to change it, or there may be another server that manages the state of the scene.

Region-rendering server 12 creates the context in which specific rendering results can be obtained as needed. If there are multiple region rendering servers 12, the region rendering servers 12 establish connections with the region rendering servers that manage adjacent regions. When connected from the client terminal (user device) 11, the area rendering server 12 acquires a rendering result from an area rendering server that manages adjacent areas that can be included in the viewpoint of the character (avatar) M. Each region rendering obtains a drawing result using the camera parameters passed at the same time when there is a request from another region rendering server.

When there are a plurality of area rendering servers 22 (22A to 22P) as shown in FIG. 3, the drawing results obtained from the adjacent area rendering servers are pasted on the joint surface with the surrounding area rendering servers. This makes it possible to render planes outside the region as if they were drawn. The joint surface at this time may have a shape in which four-sided cubes are arranged on a plane, or a shape in which hexagonal columns are arranged. Any shape that sufficiently fills the space and is bounded by planes may be used.

In addition, the area rendering server 12 requests all the content rendering servers 13 (13A to 13C) that manage the content in the area to update information as an image regarding rendering. Image information related to rendering includes depth, normal, light accumulation buffer, mask image for display area judgment, and material information such as albedo, roughness, metallic, etc., but G-buffer based on delayed shading can be used. G-buffer is a buffer that represents the normal line, depth, etc. of the entire screen. The region rendering server, for example, combines the G-buffers updated by the content rendering server into one in the region and synthesizes them as one planar G-buffer. At this time, if necessary, a shadow map is created by requesting a G-buffer with only depth from a specific light viewpoint, and actual lighting is performed using the plane, light information, shadow map, etc., and the lighting result Generate an image that reflects At this time, it is possible to use a distorted projection matrix that matches the viewpoint seen by the actual user.

In the multi-content rendering system according to an embodiment of the present invention, the image information regarding rendering sent from the content rendering server to the region rendering server is not limited to only deferred shading, but also forward rendering. It may be information as an image based on (Forward Rendering). Forward rendering is a method in which light information is sent and lighting is performed by the content rendering server, and mask information or depth information necessary for synthesizing the final rendering result and transparency etc. is sent to the rendering image. to the region rendering server as information as

The content rendering server 13 pre-computes a minimum area surrounding the content or an area of suitable size for image compression to determine the appropriate surface size for rendering the content. This area calculation can be anything that is sufficient to calculate the required drawing screen size given a certain viewpoint. For example, it can be based on AABB (Axis Aligned Bounding Box) or calculation of a sphere surrounding a simple model. The content rendering server 13 calculates a projection matrix such that this area can be viewed well from the user's camera, for example.

When receiving a request from the area rendering server 12, the content rendering server 13 sends information as an image regarding rendering as a result in response to this request. As described above, image information related to rendering is, for example, G-buffer. At this time, if the data is compressed and transferred, the data transfer amount can be further reduced. The content rendering server generates a G-buffer from the existing model using the projection matrix and transfers it to the area rendering server 12 .

Next, a multi-content rendering method according to an embodiment of the present invention will be described with reference to the drawings. FIG. 4 is a flow diagram outlining a multi-content rendering method according to an embodiment of the present invention. One aspect of the present invention is a multi-content rendering method for rendering a large number of various types of content existing in a specific area, comprising: a computing step S1 of performing computation related to rendering of content using a computing means; Rendering condition acquisition step S2 for acquiring information about the position and viewpoint of , rendering information acquisition step S3 for transmitting information from the rendering means to the calculation means and receiving information as an image related to rendering from the calculation means, and as an image related to rendering and a drawing step S4 of generating a drawing result of a specific area containing the content from the information of.

The calculation step S1 is a step of performing calculations related to content drawing using a calculation means. Computing means include, for example, a content rendering server as described above. Also, the content is a model (avatar) set by the user in a virtual space such as a game or chat, and these models include 3D models. In the calculation means, for example, a minimum area surrounding the content or an area of a size suitable for image compression is calculated in order to determine the size of the surface suitable for rendering the content as described above, and this area is used by the user, for example. Calculate a projection matrix that looks good when viewed from the camera.

The drawing condition acquisition step S2 is a step of acquiring information about the position and viewpoint of the content in the drawing means. The rendering means is basically a configuration provided separately from the computing means, and includes, for example, the area rendering server as described above. In virtual spaces such as games and chats, users usually operate models (avatars) registered by themselves to change viewpoints and move locations within the screen. Information at this time is sent from the client terminal (user device) operated by the user to the drawing means.

The rendering information acquisition step S3 transmits information from the drawing means to the calculation means, and receives information as an image regarding rendering from the calculation means. At this time, if another user's model (avatar) is present in the field of view of the target user's model (avatar), information regarding the rendering of the other model as an image is also received. For image information related to rendering, for example, G-buffer based on delayed shading can be used as described above. In this way, the rendering means does not directly handle data related to the content, but receives only image information related to rendering via the computing means. can be prevented. Therefore, in the drawing method according to one embodiment of the present invention, it is possible to draw a large number of models (avatars) of each user at the same time even when more users participate.

The drawing step S4 is a step of generating a drawing result of a specific area containing content from information as an image regarding rendering. For example, image information (for example, G-buffer) regarding rendering is acquired from the calculation means in accordance with the position of the model (avatar), the viewpoint, the arrangement included in the field of view, and the scene information regarding other models (avatars). , G-buffer information and scene information to draw rendering results from the user's viewpoint.

In addition, in the multi-content drawing method according to one embodiment of the present invention, as shown in FIG. 3, for example, a specific area is divided by a plurality of drawing means (area rendering servers), and the drawing means for generating a drawing result is: The drawing result of the area managed by the drawing means may be acquired from the adjacent drawing means.

FIG. 5 is a schematic diagram showing a rendering method when a specific area R is divided by a plurality of drawing means. Suppose that a model M of a user exists in a specific region R, and an image of S within the field of view of the model M is required for rendering at some point. At that time, the area that can be imaged on the plane includes the area r1, and the resulting image is inquired of the node that manages the area r1. Since the resulting image at this time depends on the areas of r2 and r3, this node inquires the node of the adjacent area r4 to send the resulting image. This is basically recursive. In this way, the resulting image can be obtained by combining the images based on the information returned by the multiple nodes.

EXAMPLES The present invention will be described in more detail below using examples, but the present invention is not limited to the following examples.

<Example 1. A virtual reality environment in which users upload their own 3D avatar>
A multi-content rendering system and multi-content rendering method according to an embodiment of the present invention excels at displaying a wide variety of models that cannot be predicted in advance. Therefore, for example, an environment where each user can upload their own designed model, an environment where each user's movement can be observed by other users at the same time, and a new user can suddenly be placed in the space. It is suitable for representation in a virtual space such as a dynamic environment. Realistic examples include game elements such as MMORPG, SNS space in 3D space, and the like, in which each user operates independently at the same time.

Under such circumstances, for example, a user uploads his model (avatar) to the content rendering server when registering for the service. This content rendering server may be shared with other users, or if a heavy model is expected, a dedicated content rendering server may be assigned. As a result, the user himself/herself can be made to bear the penalty for the load imposed on the entire network.

The user then accesses the region rendering server for the region to which his avatar belongs. At this time, if the number of area rendering servers increases dynamically, there may be a server for the purpose of solving area rendering servers that manages a list of area rendering servers. If the area is statically determined in advance, it is sufficient that only these servers exist. Also, the connection may be established by simple socket communication, for example. Compression technology used for streaming distribution can be applied as needed.

The region rendering server requests the rendering results as seen from the avatar's position from the possible adjacent region rendering nodes. It can be generalized as the same situation as when the user is simply connected. At this time, a normal rendering result is obtained by using a Perspective Offcenter projection matrix.

Then, the scene information is acquired. If composed solely of avatars, the region rendering node can retrieve the scene information from the list of clients connected to it. In addition, when not only avatars but also pre-placed objects are mixed, a separate scene graph management node is prepared, and the poses of the object list on the scene are solved sequentially from these.

After that, the region rendering server acquires the G-buffer from the content rendering server in accordance with the scene information, performs lighting from the G-buffer information and the scene information, and draws the rendering result from the user's viewpoint.

<Example 2. Simultaneous display under conditions where the shape of the model can dynamically change at the same time>
In the multi-contents rendering system and the multi-contents rendering method according to one embodiment of the present invention, since the part that renders from the shape generation of the model itself and the part that renders the entire model are independent, the shape generation cost for simulation is It is also possible to simulate data that is so high that it is not practical to transfer it to the outside at a realistic speed. For example, there is a simulation that can acquire the G-buffer drawing result of a certain partial area from a given camera viewpoint. Depending on the purpose of the simulation, it can be done in real time, and even in situations where it is not necessary, if the simulation results of a specific area can be obtained at a realistic speed, a large-scale space can be drawn sequentially.

First, a simulation is performed on the content rendering server. The above simulation is performed by the content rendering server, and results are returned in response to requests from the region rendering server. In actual operation, the content rendering server may be set up by the person who executes the simulation. Such a content rendering server may be operated as a large-scale grid computing system by allocating free areas to nodes participating in the simulation.

The region rendering server then retrieves the results from the content rendering server. The Region Rendering Server performs the necessary processing globally. Global processing here is lighting agnostic. For example, in the case of a nebula simulation, values related to interaction between the content rendering servers are calculated from the simulation results of each partial area returned by the content rendering servers.

Although one embodiment of the present invention has been described in detail as above, those skilled in the art will easily understand that many modifications are possible without substantially departing from the novel matters and effects of the present invention. You can. Accordingly, all such modifications are intended to be included within the scope of the present invention.

For example, a term described at least once in the specification or drawings together with a different, broader or synonymous term can be replaced with the different term anywhere in the specification or drawings. Also, the configuration and operation of the multi-content drawing system are not limited to those described in the embodiment of the present invention, and various modifications are possible.

10, 20 multiple content rendering system, 11, 21 client terminal, 12, 22 (22A to 22P) area rendering server, 13 (13A to 13C), 23 (23A to 23C) content rendering server, 100 rendering system, 101 client terminal , 102 rendering server, M, M1, M2 model, R specific region, S within field of view, S1 calculation process, S2 drawing condition acquisition process, S3 rendering information acquisition process, S4 drawing process

Claims (11)

A multi-content rendering system that renders a large number of various types of content existing in a specific area to which a plurality of client terminals are connected ,
at least,
a region rendering server that generates a rendering result of the specific region containing the content;
A content rendering server that performs calculations related to drawing the content,
Information about contents including characters registered from each of the client terminals is transmitted to the content rendering server,
The multi-content rendering system, wherein the area rendering server transmits information on position and viewpoint to the content rendering server, and receives image information on content rendering from the content rendering server. 3. The multi-content rendering system of claim 1, wherein the content includes a 3D model. 3. The multi-contents rendering system according to claim 1, wherein the image information related to rendering is a G-buffer based on delayed shading. 3. The multi-contents rendering system according to claim 1, wherein said information as an image relating to rendering includes rendering information based on forward rendering. 5. The multi-content rendering system according to claim 1, wherein the specific area is rendered by processing of a plurality of area rendering servers. A multi-content rendering method for rendering a large number of various types of content existing in a specific area to which a plurality of client terminals are connected ,
a calculation step of performing calculations relating to drawing of the content including the character registered from each of the client terminals using a calculation means;
a drawing condition obtaining step of obtaining information about the position and viewpoint of the content in the drawing means;
a rendering information acquisition step of transmitting the information from the drawing means to the calculation means and receiving information as an image related to rendering from the calculation means;
A multi-contents drawing method, comprising: a drawing step of generating a drawing result of the specific area containing the content from information as an image regarding the rendering. 7. The method of claim 6, wherein said content includes a 3D model. 8. The multi-contents rendering method according to claim 6, wherein the image information related to rendering is a G-buffer based on delayed shading. 8. The multi-contents rendering method according to claim 6, wherein the image information regarding rendering includes rendering information based on forward rendering. 3. The drawing means for dividing the specific area by a plurality of the drawing means and generating the drawing result acquires the drawing result of the area managed by the drawing means from the adjacent drawing means. The multiple content rendering method according to any one of claims 6 to 9. A program for causing a computer to execute the multi-content drawing method according to any one of claims 6 to 10.

Images