JP2021086146A - Content control system, content control method, and content control program - Google Patents

Abstract

To effectively transmit to a user what has happened in a virtual space.SOLUTION: A content control system according to one embodiment is provided with at least one processor. The at least one processor specifies a motion of a target virtual object in a virtual space showing a class scene, determines a language expression corresponding to the specified motion with reference to a storage unit that stores a language expression rule, and outputs expression data corresponding to the determined language expression.SELECTED DRAWING: Figure 3

Description

One aspect of the disclosure relates to content control systems, content control methods, and content control programs.

Computer systems that allow users to experience the virtual world are applied in various fields. For example, Patent Document 1 describes a learning system in which a lecture is given between a device on the instructor side and a device on the student side. This learning system includes instructor-side equipment that includes instructor software that uses virtual reality technology to draw a virtual space depicted in three dimensions, and student-side equipment that includes student software that uses virtual reality technology. It is provided with a network means for transmitting and receiving lecture signals for drawing a virtual space between the device on the instructor side and the device on the student side.

Japanese Unexamined Patent Publication No. 2009-145883

A method for effectively communicating events in the virtual space to the user is desired.

The content control system according to one aspect of the present disclosure includes at least one processor. At least one processor identifies the movement of the target virtual object in the virtual space indicating the scene of the lesson, refers to the storage unit that stores the linguistic expression rule, and determines the linguistic expression corresponding to the specified movement. , Outputs the expression data corresponding to the determined language expression.

In such an aspect, expression data based on the linguistic expression corresponding to the operation of the target virtual object is output. By such a series of processes not described in Patent Document 1, an event in the virtual space can be effectively transmitted to the user of the terminal.

According to one aspect of the present disclosure, events in the virtual space can be effectively communicated to the user.

It is a figure which shows an example of application of the content distribution system (content control system) which concerns on embodiment. It is a figure which shows an example of the hardware configuration which concerns on the content distribution system which concerns on embodiment. It is a figure which shows an example of the functional structure which concerns on the content distribution system which concerns on embodiment. It is a flowchart which shows an example of the operation of the content distribution system which concerns on embodiment. It is a figure which shows an example of a virtual representation object. It is a figure which shows another example of a virtual representation object. It is a figure which shows still another example of a virtual representation object.

Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In the description of the drawings, the same or equivalent elements are designated by the same reference numerals, and duplicate description will be omitted.

[System overview]
The content control system according to the embodiment is a computer system that controls content distributed to users. Content is information provided by a computer or computer system that is human recognizable. Electronic data that indicates content is called content data. The representation format of the content is not limited, and for example, the content may be represented by an image (for example, a photograph, a video, etc.), a document, audio, music, or a combination of any two or more elements thereof. The purpose and usage of the content is not limited, for example, the content may be used for various purposes such as entertainment, news, education, medical care, games, chat, commerce, lectures, seminars, and training. Content control refers to the process performed to provide content to a user. Content control may include at least one of the generation, editing, storage, and distribution of content data, or may include processing other than these.

The content control system provides the content to the viewer by transmitting the content data to the viewer terminal. In one example, the content is provided by the distributor. A distributor is a person who wants to convey information to a viewer, that is, a sender of content. A viewer is a person who wants to obtain the information, that is, a user of the content.

In this embodiment, the content is expressed using at least an image. An image showing content is called a "content image". A content image is an image in which a person can visually recognize some information. The content image may be a moving image (video) or a still image.

The content image may reflect the real world or a virtual space. The virtual space is a virtual two-dimensional or three-dimensional space represented by an image displayed on a computer. When the content image includes a virtual space, it can be said that the content image is an image showing a landscape seen from a virtual camera set in the virtual space. The virtual camera is set in the virtual space so as to correspond to the line of sight of the user who sees the content image. Virtual space is represented by at least one virtual object. A virtual object is an object that does not actually exist in the real world and is represented only on a computer system.

The content image may show a person who is a performer, or may show an avatar instead of the performer. The distributor may or may not appear on the content image as a performer. At least a part of the plurality of viewers may appear in the content image as a performer (participant). The distributor or participant's avatar may appear on the content image. Content images may include both the real world and virtual space or virtual objects. The inclusion of virtual space or virtual objects in the content image allows the viewer to experience augmented reality (AR), virtual reality (VR), or mixed reality (MR). Can be done.

An avatar is a user's alter ego represented by a computer. Avatar is a type of virtual object. The avatar is represented by two-dimensional or three-dimensional computer graphics (CG) using image material independent of the original image, not the person who was photographed (ie, not the user itself shown in the original image). To. The expression method of the avatar is not limited. For example, the avatar may be represented using an animation material, or may be represented as close to the real thing based on a live-action image. The avatar may be freely selected by the user of the content distribution system (eg, distributor or viewer).

In one example, the content control system may deliver the content to the viewer. Distribution refers to the process of transmitting information to users via a communication network or broadcasting network. In the present disclosure, distribution is a concept that may include broadcasting. In the present disclosure, a content control system having a function of distributing content is also referred to as a content distribution system.

The method of delivering the content is not limited. For example, the content distribution system may distribute live content. In this case, the content distribution system generates content data by processing the real-time video provided from the distributor terminal, and transmits the content data to the viewer terminal in real time. It can be said that this is one aspect of live Internet broadcasting. Alternatively, the content distribution system may distribute images shot and generated in the past. The content distribution system may be used for a time shift in which the content can be viewed in a given period after real-time distribution. Alternatively, the content distribution system may be used for on-demand distribution in which the content can be viewed at any time.

In the present disclosure, the expression "transmitting" data or information from a first computer to a second computer means transmission for finally delivering the data or information to the second computer. It should be noted that this expression also includes the case where another computer or communication device relays data or information in the transmission.

As described above, the purpose and usage scene of the content are not limited. In the present embodiment, educational content is shown as an example of the content, and the content control system controls the educational content data. The educational content is content used to give a lesson to a student, and may be used, for example, for a teacher to give a lesson to a student. A teacher is a person who teaches schoolwork, arts, etc., and a student is a person who receives the teaching. Teachers can be distributors. Students are an example of a viewer. The teacher may be a person with a teacher's license or a person without a teacher's license. Class means that a teacher teaches students academics, arts, and so on. The age and affiliation of each teacher and student is not limited, and therefore the purpose and use of educational content is not limited. For example, educational content may be used in various schools such as nursery schools, kindergartens, elementary schools, junior high schools, high schools, universities, graduate schools, vocational schools, preparatory schools, online schools, etc. You may. In this regard, educational content can be used for a variety of purposes such as early childhood education, compulsory education, higher education, and lifelong learning.

[System configuration]
FIG. 1 is a diagram showing an example of application of the content distribution system (content control system) 1 according to the embodiment. In the present embodiment, the content distribution system 1 includes a server 10. The server 10 is a computer that generates and distributes content data. In one example, the server 10 connects at least one student terminal 20, a teacher terminal 30, an original video database 40, and a dictionary database 50 via a communication network N. FIG. 1 shows two student terminals 20 and one teacher terminal 30, but the number of each terminal is not limited in any way. The configuration of the communication network N is not limited. For example, the communication network N may be configured to include the Internet or may be configured to include an intranet.

The student terminal 20 is a computer used by a student, and is an example of a viewer terminal (a computer used by a viewer). In one example, the student terminal 20 has a function of accessing the content distribution system 1 to receive and display content data, and a function of transmitting student motion data to the content distribution system 1. Motion data refers to electronic data that indicates the movement of an object. Student motion data indicates the movement of a particular part of the student's body (eg, joints) by the position and angle of the body. The method of acquiring motion data is not limited. The motion data may be obtained by analyzing the image taken by the camera. Alternatively, the motion data may be obtained by a device for motion capture, such as a body strap, data glove, VR controller (hand controller), or the like. The type and configuration of the student terminal 20 is not limited. For example, the student terminal 20 may be a mobile terminal such as a high-performance mobile phone (smartphone), a tablet terminal, a wearable terminal (for example, a head-mounted display (HMD), a smart glass, etc.), a laptop personal computer, or a mobile phone. Alternatively, the student terminal 20 may be a stationary terminal such as a desktop personal computer.

The teacher terminal 30 is a computer used by a teacher, and is an example of a distributor terminal (a computer used by a distributor). Typically, the teacher terminal 30 is located in a remote location for the student terminal 20. In one example, the teacher terminal 30 has a function of capturing a video and a function of accessing the content distribution system 1 and transmitting electronic data (video data) indicating the video. The teacher terminal 30 may have a function of receiving and displaying a video or content. Similar to the student terminal 20, the teacher terminal 30 may have a function of transmitting the teacher's motion data to the content distribution system 1. The type and configuration of the teacher terminal 30 are not limited. For example, the teacher terminal 30 may be a photographing system having a function of photographing, recording, and transmitting an image. Alternatively, the teacher terminal 30 may be a mobile terminal such as a high-performance mobile phone (smartphone), a tablet terminal, a wearable terminal (for example, a head-mounted display (HMD), a smart glass, etc.), a laptop personal computer, or a mobile phone. Alternatively, the teacher terminal 30 may be a stationary terminal such as a desktop personal computer.

The classroom manager or student operates the student terminal 20 to log in to the content distribution system 1, whereby the student can view the educational content. The teacher operates the teacher terminal 30 to log in to the content distribution system 1, which enables distribution or recording of his / her lesson. In this embodiment, it is assumed that the user of the content distribution system 1 has already logged in.

The original video database 40 is a non-temporary storage device that stores the original video data. The original video data is electronic data indicating the original video used for generating the educational content data, and therefore can be said to be a material for generating the educational content. The original image may be a live-action image or may include a virtual space. Corresponding to the fact that the content of the original video is not limited, the data structure of the original video data is also not limited. For example, when a live-action video is used, the original video data includes video data taken by a camera. When a virtual space is used, the original video data includes spatial data that defines the virtual space and model data that defines the specifications of the virtual object, and includes scenario data for defining the progress of the story in the virtual space. Further may be included. The original video data is stored in the original video database 40 in advance by an arbitrary computer such as a server 10, a teacher terminal 30, or another computer. It can be said that the original video database 40 is a library that stores original video shot or generated in the past (that is, video that is not real-time).

The dictionary database 50 is a non-temporary storage device that stores dictionary data. In one example, each record in the dictionary data has a record ID, which is an identifier for identifying individual records, a viewer ID, which is an identifier for uniquely identifying a student (viewer), and an image specified by the student. Includes (still image or video).

The installation location of the original video database 40 and the dictionary database 50 is not limited. For example, at least one of the original video database 40 and the dictionary database 50 may be provided in a computer system different from the content distribution system 1, or may be a component of the content distribution system 1.

FIG. 2 is a diagram showing an example of a hardware configuration related to the content distribution system 1. FIG. 2 shows a server computer 100 that functions as a server 10 and a terminal computer 200 that functions as a student terminal 20 or a teacher terminal 30.

As an example, the server computer 100 includes a processor 101, a main storage unit 102, an auxiliary storage unit 103, and a communication unit 104 as hardware components.

The processor 101 is an arithmetic unit that executes an operating system and an application program. Examples of the processor include a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit), but the type of the processor 101 is not limited thereto. For example, the processor 101 may be a combination of a sensor and a dedicated circuit. The dedicated circuit may be a programmable circuit such as FPGA (Field-Programmable Gate Array), or may be another type of circuit.

The main storage unit 102 is a device that stores a program for realizing the server 10, a calculation result output from the processor 101, and the like. The main storage unit 102 is composed of, for example, at least one of a ROM (Read Only Memory) and a RAM (Random Access Memory).

The auxiliary storage unit 103 is a device capable of storing a larger amount of data than the main storage unit 102 in general. The auxiliary storage unit 103 is composed of a non-volatile storage medium such as a hard disk or a flash memory. The auxiliary storage unit 103 stores the server program P1 for making the server computer 100 function as the server 10 and various data. For example, the auxiliary storage unit 103 may store data relating to at least one of a virtual object such as an avatar and a virtual space. In this embodiment, the content control program is implemented as the server program P1.

The communication unit 104 is a device that executes data communication with another computer via the communication network N. The communication unit 104 is composed of, for example, a network card or a wireless communication module.

Each functional element of the server 10 is realized by reading the server program P1 on the processor 101 or the main storage unit 102 and causing the processor 101 to execute the program. The server program P1 includes a code for realizing each functional element of the server 10. The processor 101 operates the communication unit 104 according to the server program P1 to read and write data in the main storage unit 102 or the auxiliary storage unit 103. By such processing, each functional element of the server 10 is realized.

The server 10 may be composed of one or more computers. When a plurality of computers are used, one server 10 is logically configured by connecting these computers to each other via a communication network.

As an example, the terminal computer 200 includes a processor 201, a main storage unit 202, an auxiliary storage unit 203, a communication unit 204, an input interface 205, an output interface 206, and an imaging unit 207 as hardware components.

Processor 201 is an arithmetic unit that executes operating systems and application programs. The processor 201 can be, for example, a CPU or GPU, but the type of processor 201 is not limited to these.

The main storage unit 202 is a device that stores a program for realizing the student terminal 20 or the teacher terminal 30, a calculation result output from the processor 201, and the like. The main storage unit 202 is composed of, for example, at least one of ROM and RAM.

The auxiliary storage unit 203 is a device capable of storing a larger amount of data than the main storage unit 202 in general. The auxiliary storage unit 203 is composed of a non-volatile storage medium such as a hard disk or a flash memory. The auxiliary storage unit 203 stores the client program P2 for making the terminal computer 200 function as the student terminal 20 or the teacher terminal 30 and various data. For example, the auxiliary storage unit 203 may store data relating to at least one of a virtual object such as an avatar and a virtual space.

The communication unit 204 is a device that executes data communication with another computer via the communication network N. The communication unit 204 is composed of, for example, a network card or a wireless communication module.

The input interface 205 is a device that receives data based on a user's operation or operation. For example, the input interface 205 is composed of at least one of a keyboard, operation buttons, a pointing device, a microphone, a sensor, and a camera. The keyboard and operation buttons may be displayed on the touch panel. Corresponding to the fact that the type of the input interface 205 is not limited, the data to be input is not limited. For example, the input interface 205 may accept data input or selected by a keyboard, operating buttons, or pointing device. Alternatively, the input interface 205 may accept voice data input by the microphone. Alternatively, the input interface 205 may accept image data (eg, video data or still image data) captured by the camera.

The output interface 206 is a device that outputs data processed by the terminal computer 200. For example, the output interface 206 is composed of at least one of a monitor, a touch panel, an HMD and a speaker. Display devices such as monitors, touch panels, and HMDs display the processed data on the screen. The speaker outputs the voice indicated by the processed voice data.

The imaging unit 207 is a device that captures an image of the real world, and is specifically a camera. The imaging unit 207 may capture a moving image (video) or a still image (photograph). When shooting a moving image, the imaging unit 207 processes the video signal based on a given frame rate to acquire a series of frame images arranged in time series as a moving image. The imaging unit 207 can also function as an input interface 205.

Each functional element of the student terminal 20 or the teacher terminal 30 is realized by loading the corresponding client program P2 into the processor 201 or the main storage unit 202 and causing the processor 201 to execute the program. The client program P2 includes a code for realizing each functional element of the student terminal 20 or the teacher terminal 30. The processor 201 operates the communication unit 204, the input interface 205, the output interface 206, or the imaging unit 207 according to the client program P2, and reads and writes data in the main storage unit 202 or the auxiliary storage unit 203. By this process, each functional element of the student terminal 20 or the teacher terminal 30 is realized.

At least one of the server program P1 and the client program P2 may be provided after being non-temporarily recorded on a tangible recording medium such as a CD-ROM, a DVD-ROM, or a semiconductor memory. Alternatively, at least one of these programs may be provided via a communication network as a data signal superimposed on a carrier wave. These programs may be provided separately or together.

FIG. 3 is a diagram showing an example of a functional configuration related to the content distribution system 1. The server 10 includes a content management unit 11, a motion identification unit 12, a language expression determination unit 13, an object setting unit 14, an object transmission unit 15, and a dictionary management unit 16 as functional elements.

The content management unit 11 is a functional element that manages the generation and output of educational content, and includes a motion identification unit 12, a language expression determination unit 13, an object setting unit 14, and an object transmission unit 15. The motion specifying unit 12 is a functional element that specifies the movement of a virtual object in the virtual space indicating the scene of the lesson. In this embodiment, it is assumed that this virtual space is displayed at least on the student terminal 20. The language expression determination unit 13 is a functional element that determines the language expression corresponding to the specified movement. Linguistic expression means to objectively express meaning in natural language. In one example, determining the linguistic expression that corresponds to a movement is the linguistic representation of that movement or the events that result from that movement. The object setting unit 14 is a functional element that sets a virtual expression object corresponding to the language expression. A virtual representation object is a virtual object used to visualize a linguistic representation and constitutes at least a part of educational content data. The virtual expression object is an example of expression data corresponding to a linguistic expression. The object transmission unit 15 is a functional element that transmits the virtual object to the student terminal 20. This transmission is an example of a process of outputting expression data corresponding to a linguistic expression to a terminal displaying a virtual space. The dictionary management unit 16 is a functional element that manages dictionary data in response to a request from the student terminal 20.

The student terminal 20 includes a motion transmission unit 21, a display control unit 22, and a dictionary control unit 23 as functional elements. The motion transmission unit 21 is a functional element that transmits student motion data to the server 10. The display control unit 22 is a functional element that receives and processes educational content data and displays the educational content on the display device. The dictionary control unit 23 is a functional element that executes processing related to saving or searching dictionary data.

[System operation]
The operation of the content distribution system 1 (more specifically, the operation of the server 10) will be described, and the content control method (or content distribution method) according to the present embodiment will be described. FIG. 4 is a flowchart showing an example of the operation of the content distribution system 1 as a processing flow S1. In the processing flow S1, the content management unit 11 has already started to read the original video data requested from the student terminal 20 from the original video database 40 and provide the original video data as educational content data to the student terminal 20. It is assumed that.

In step S11, the content management unit 11 receives the student motion data from the student terminal 20. In the student terminal 20, the motion transmission unit 21 transmits motion data indicating the real-time movement of the student viewing the original video to the server 10, and the content management unit 11 receives the motion data. The data structure of motion data is not limited. For example, the motion data may indicate the movement of the student in a specific time width, or may indicate the posture of the student at a specific moment. The motion data may represent the movement or posture of any part of the student's body (eg, hands only, whole body, etc.).

In step S12, the motion specifying unit 12 specifies the movement of the virtual object based on the motion data. In the present disclosure, a virtual object whose motion is specified by the motion specifying unit 12 is also referred to as a “target virtual object”. Typically, the target virtual object is a virtual object that is moved by motion data. For example, if the motion data indicates the movement of the student's hand, the target virtual object may be a virtual hand. If the motion data shows the movement of the student's whole body, the target virtual object may be the student's avatar. In one example, the motion specifying unit 12 specifies the movement of the target virtual object based on the history of changes in the three-dimensional coordinates of a plurality of joints included in the motion data and the combination of adjacent joints (that is, bones). The movement of the target virtual object shows the real-time movement of the student.

In step S13, the linguistic expression determination unit 13 determines the linguistic expression corresponding to the specified movement based on the linguistic expression rule. A linguistic expression rule is a rule for deriving a linguistic expression from the movement of at least one virtual object. In one example, the language expression rule is pre-stored in the auxiliary storage unit 103. The implementation method of the language expression rule is not limited, and may be expressed by data such as a correspondence table or an algorithm such as a trained model.

The "linguistic representation corresponding to the specified movement" may indicate the movement of the target virtual object, or may relate to at least one related virtual object that changes based on the movement of the target virtual object. The associated virtual object may represent any object, eg, a person (avatar), any man-made object (eg, movable property, real estate), any natural object (eg, animal, plant), or any terrain (eg, mountain). , River, ground). The change of the related virtual object is not limited and may be, for example, a change in position or posture (ie, movement) or a change in state (eg, change in color). The "linguistic expression corresponding to the specified movement" may indicate a combination of the movement of the target object and the change of the related virtual object.

If the language expression cannot be determined in step S13, that is, if the language expression corresponding to the specified movement cannot be derived, the processing flow S1 ends at this point, and in this case, the content management unit 11 virtualizes the virtual expression. The original video data is transmitted to the student terminal 20 without adding an object. If the language expression is determined in step S13, the process proceeds to step S14.

In step S14, the object setting unit 14 sets a virtual expression object corresponding to the determined language expression. As long as the linguistic representation can be visualized, the format of the virtual representation object is not limited. For example, the object setting unit 14 may set a virtual expression object that expresses the language expression as it is in characters, and in this case, arbitrary visual effects such as decoration and animation effects may be applied to the characters. Alternatively, the object setting unit 14 may set a virtual expression object that indicates the language expression as an image (still image or moving image) without using characters. It can be said that this is a process of setting a typical image showing a scene similar to the movement of the specified virtual object.

The display time of the virtual representation object may be set by any policy. For example, the object setting unit 14 may set a given time limit. Alternatively, the object setting unit 14 may control the virtual representation object so that the virtual representation object is displayed until at least one of the movement of the target virtual object and the change of the related virtual object is completed.

In step S15, the object setting unit 14 generates educational content data including the set virtual expression object. In one example, the original video data indicates a virtual space, and in this case, the object setting unit 14 arranges the virtual expression object in the virtual space. "Place an object (such as a virtual representation object)" means to place an object in a fixed position, and is a concept that includes changing the position of the object.

There are no restrictions on the method and data structure of educational content data. For example, the content management unit 11 may generate educational content data including virtual space data indicating the positions, dimensions, and movements (postures) of the virtual space and individual objects including the virtual representation object. Alternatively, the content management unit 11 may generate educational content data by executing rendering based on the set virtual space. In this case, the educational content data indicates the content image itself including the virtual representation object.

In step S16, the object transmission unit 15 transmits educational content data including the virtual expression object to the student terminal 20. In the student terminal 20, the display control unit 22 receives and processes the educational content data, and displays the educational content on the display device. As a result, the virtual representation object appears in the virtual space displayed on the student terminal 20. When rendering is not executed on the server 10, the display control unit 22 displays the content image by executing rendering based on the educational content data. When the educational content data indicates the content image itself, the display control unit 22 displays the content image as it is. The student terminal 20 outputs sound from the speaker in accordance with the display of the content image.

The object transmission unit 15 may store the educational content data in a given database in addition to or instead of transmitting the educational content data to the student terminal 20. For example, the object transmission unit 15 may store the educational content data as the original video data in the original video database 40.

The processing flow S1 can be executed a plurality of times in one delivery to a certain student terminal 20. In this case, while the educational content is displayed, various virtual representation objects are displayed at the timing of the movement in response to the student's real-time movement. The processing flow S1 may be executed only for a part of the motion data. For example, the content distribution system 1 may execute the processing flow S1 only for the motion data received during the time width corresponding to a specific scene (for example, a scene where an exercise is performed) in the educational content. ..

An example of displaying a virtual representation object will be described with reference to FIGS. 5 to 7. 5 to 7 are diagrams showing an example of a virtual representation object.

In the example of FIG. 5, the change of the situation in the virtual space including the three virtual objects of the platform 301, the ball 302, and the virtual hand 311 is shown. In this example, the content distribution system 1 displays an English preposition indicating the position of the ball 302 with respect to the platform 301 by a virtual expression object.

The virtual hand 311 is a target virtual object that moves based on the actual movement of the student's hand. The pedestal 301 and the ball 302 can be treated as related virtual objects. When the student operates the virtual hand 311 to place the ball 302 on the table 301, the virtual representation object 321 is displayed in response to the operation. The virtual expression object 321 is expression data indicating the character itself of the preposition "on", which is a linguistic expression determined in response to the ball 302 being placed "on" the table 301. After that, when the student operates the virtual hand 311 to place the ball 302 near the table 301, the virtual representation object 322 is displayed in response to the operation. The virtual expression object 322 is expression data indicating the character itself of the preposition "by", which is a linguistic expression determined corresponding to the position of the ball 302 "near" the platform 301. It can be said that the virtual representation objects 321 and 322 are all related to the related virtual object.

In one example, as in the example of FIG. 5, by displaying a linguistic expression corresponding to the movement of the target virtual object based on the movement of the student, the learner is generally not easy to understand the subtleties of the foreign language vocabulary. Can understand the difference. For example, when the ball 302 comes into contact with the table 301 regardless of the direction, a virtual expression object indicating the English word “ON” may be displayed. When the virtual hand 311 lifts the ball 302 and positions the ball 302 directly above the base 301, the English word "over" may be displayed. When the virtual hand 311 positions the ball 302 substantially above the platform 301, the English word "above" may be displayed. This display of linguistic expressions is unique and not available in traditional education (eg, foreign language education), such as books, real classroom lessons, and television educational programs where teachers teach students unilaterally. It works.

In the example of FIG. 6, a virtual space 400 including a plurality of avatars corresponding to a plurality of students is shown. In this example, the content distribution system 1 displays English words indicating the movement of the avatar by a virtual expression object.

The virtual space 400 includes a student's avatar 401 and a ball 402. Avatar 401 is a target virtual object that moves based on the student's actual movements. The ball 402 can be treated as an associated virtual object. When the student operates the avatar 401 and throws the ball 402, the virtual representation object 411 is displayed in response to the operation. The virtual expression object 411 is expression data indicating the character itself of the verb "throw", which is a linguistic expression determined in response to the avatar 401 throwing the ball 402. It can be said that the virtual representation object 411 indicates the movement of the target virtual object.

The virtual space 400 further includes another student's avatar 421. Avatar 421 is a target virtual object that moves based on the student's actual movement. When the student jumps, the avatar 421 also jumps, and the character itself of the verb "jump", which is a linguistic expression determined in response to the movement, is displayed as the virtual expression object 431. It can be said that the virtual representation object 431 also indicates the movement of the target virtual object. Like the virtual space 400, the content distribution system 1 may display a virtual expression object corresponding to a linguistic expression caused by the movement of an avatar of a person other than the viewer on the terminal of the viewer.

In the example of FIG. 7, a virtual space 500 including an avatar corresponding to a certain student is shown. In this example, the content distribution system 1 displays an English sentence indicating the behavior of the avatar by a virtual expression object.

The virtual space 500 includes a wall 501, a student avatar 502, and a brush 503 possessed by the avatar 502. The avatar 502 is a target virtual object that moves based on the student's actual movement. Walls 501 and brushes 503 can be treated as related virtual objects. When the student operates the avatar 502 and paints the wall 501 red with the brush 503, the virtual representation object 511 is displayed in response to the operation. The virtual expression object 511 is expression data indicating the characters themselves of the English sentence "You are painting a wall in red.", Which is a linguistic expression determined in response to the action of painting the wall 501 in red. The virtual representation object 511 can be said to indicate the movement of the target virtual object, and can also be said to indicate the combination of the movement of the target virtual object and the change of the related virtual object.

As another example, as a scenario-based question sentence, the English sentence "You are painting a wall in red." Is displayed first, and the task "You should do this action." Is displayed by the teacher in the educational content. Was shown to the students. Then, the student must hold the brush 503 in the virtual space of the educational content, select the red paint from a plurality of colors of paint, apply the brush 503 to the red paint, and apply the brush 503 to the wall 501. .. If the student performs the action correctly, the content distribution system 1 may display a virtual expression object indicating the linguistic expression "correct answer" based on the action. If the student does not do the action correctly (for example, when approaching a paint of another color), the content distribution system 1 is based on the action and shows a virtual expression "Is it really that color?" The representation object may be displayed as a hint.

In one example, the educational content including the scenes shown in FIGS. 5 to 7 is the content of the language class. By applying the content distribution system 1 to language lessons, it is possible to convey various matters such as vocabulary usage and grammar to students using virtual expression objects without preparing supplementary materials for explanations about the language. Can be done. By looking at the virtual representation object, students can intuitively understand various matters related to language.

As described above, the purpose and usage of the content are not limited, and therefore, the matters specifically indicated by the virtual representation object are not limited. The virtual representation object may show notation in any language other than English. Alternatively, the virtual representation object may indicate a matter for the purpose of learning other than language, and may indicate, for example, a note of a given operation in a virtual skill class. The virtual representation object may be used for purposes other than learning, for example, to support information transmission or communication.

A student who views the educational content provided to the student terminal 20 by the processing flow S1 can register an image of the scene in which the virtual expression object is reflected as dictionary data, and further refers to the dictionary data later. Can be done. In the present embodiment, the dictionary control unit 23 of the student terminal 20 and the dictionary management unit 16 of the server 10 cooperate to register and refer to (search) dictionary data.

The student operates the student terminal 20 to specify an image (still image or video of a given time width) of the scene including the virtual representation object. In response to the operation, the dictionary control unit 23 records the designated image and transmits a registration request including the image and the viewer ID to the server 10. In the server 10, the dictionary management unit 16 receives the registration request. The dictionary management unit 16 generates a new record ID, and generates a record of dictionary data by associating the viewer ID and the image included in the registration request with the record ID. Then, the dictionary management unit 16 stores the record in the dictionary database 50.

At any subsequent timing, the student operates the student terminal 20 to specify a list of dictionary data or a specific record of one or more of the dictionary data. In response to the operation, the dictionary control unit 23 transmits a search request including at least the viewer ID to the server 10. In the server 10, the dictionary management unit 16 reads at least one record corresponding to the search request from the dictionary database 50, and transmits the record as a search result to the student terminal 20. In the student terminal 20, the dictionary control unit 23 displays the search result on the display device, whereby the student can re-reference the scene in which the virtual expression object is displayed (for example, the scene shown in FIGS. 5 to 7). Can be done.

As described above, the content distribution system 1 can be applied to various types of lessons. In one example, the content distribution system 1 can be used for real-time distance learning. The real-time distance lesson is a mode in which a lesson conducted by a teacher in real time is delivered in real time to one or more student terminals 20 via a teacher terminal 30 and a server 10. In this case, since one or more students are viewing common content (that is, a common virtual space) at the same time, it is difficult for all students to move a specific virtual object at the same time. Therefore, in this case, it is conceivable that the teacher or selected student moves the virtual object or operates in the virtual space.

In another example, the content distribution system 1 can be used for time-shifted distance learning. In time-shift distance learning, pre-photographed or generated educational content (that is, educational content pre-stored in a given database) is delivered to the student terminal 20 in response to a request from each student terminal 20. It is an aspect to be done. In this case, the real-time first student action is stored as virtual object information by adding or overwriting it to the educational content data and stored in a given database. In addition, a virtual representation object corresponding to the linguistic representation based on the behavior is also saved by being added or overwritten to the educational content data. After that, when the second student views the educational content, the second student can visually recognize the movement of the first student and the corresponding linguistic expression (virtual expression object). The action of the second student and the virtual expression object corresponding to the linguistic expression based on this action can also be added or overwritten in the educational content data. By such time-shift distance learning, it is possible to give each student a pseudo impression as if different students who are separated in space and time are taking the same lesson together at the same time.

[effect]
As described above, the content control system according to one aspect of the present disclosure includes at least one processor. At least one processor identifies the movement of the target virtual object in the virtual space indicating the scene of the lesson, refers to the storage unit that stores the linguistic expression rule, and determines the linguistic expression corresponding to the specified movement. , Outputs the expression data corresponding to the determined language expression.

The content control method according to one aspect of the present disclosure is executed by a content control system including at least one processor. The content control method determines the language expression corresponding to the specified movement by referring to the step of specifying the movement of the target virtual object in the virtual space showing the scene of the lesson and the storage unit that stores the language expression rule. The step of outputting the expression data corresponding to the determined linguistic expression is included.

The content control program according to one aspect of the present disclosure refers to the step of specifying the movement of the target virtual object in the virtual space indicating the scene of the lesson and the storage unit for storing the language expression rule, and the specified movement. The computer is made to execute the step of determining the linguistic expression corresponding to the above and the step of outputting the expression data corresponding to the determined linguistic expression.

In such an aspect, expression data based on the linguistic expression corresponding to the operation of the target virtual object is output. With this representation data, events in the virtual space can be effectively communicated to the user of the terminal.

In the content control system according to the other aspect, at least one processor receives motion data indicating the real-time movement of the student watching the class from the terminal, and identifies the movement of the target virtual object based on the motion data. May be good. By this process, the event corresponding to the student's real-time movement can be effectively communicated to the student in real time.

In the content control system according to the other aspect, at least one processor acquires the original video data indicating the original video shot or generated in the past from the database, and educates using the acquired original video data and the expression data. Content data for education may be generated and the generated educational content data may be output. Since existing video is converted into educational content data using expression data instead of real-time video, a huge amount of past video can be used or reused more effectively.

In the content control system according to the other aspect, at least one processor may determine a linguistic representation indicating the movement of the target virtual object. In this case, the movement of the target virtual object can be effectively transmitted to the user.

In the content control system according to the other aspect, at least one processor may determine the linguistic representation associated with the related virtual object that changes based on the movement of the target virtual object. In this case, changes in related virtual objects can be effectively communicated to the user.

In the content control system according to the other aspect, at least one processor may determine a linguistic representation indicating a combination of the movement of the target virtual object and the change of the related virtual object. In this case, it is possible to effectively convey to the user an event based on the combination of the target virtual object and the related virtual object.

In the content control system according to the other aspect, the representation data may indicate a virtual representation object displayed on the terminal. In this case, the events in the virtual space can be visually transmitted to the user of the terminal.

In the content control system according to the other aspect, the virtual representation object may contain characters. By expressing the events in the virtual space with characters, the events can be conveyed to the user in an easy-to-understand manner.

In the content control system according to the other aspect, at least one processor acquires a registration request including an image including a scene showing a virtual representation object displayed on the terminal and a viewer ID of a student watching the class. , The dictionary data including the viewer ID and the image is stored in the dictionary database, and in response to the search request from the terminal, the dictionary data corresponding to the search request is read from the dictionary database and the dictionary data is output to the terminal. May be good. In this case, the scene in which the virtual representation object is projected can be saved and searched, so that the user can look back at the virtual representation object once viewed.

[Modification example]
The above description has been made in detail based on the embodiments of the present disclosure. However, the present disclosure is not limited to the above embodiment. The present disclosure can be modified in various ways without departing from its gist.

In the above embodiment, the motion specifying unit 12 performs the movement of the target virtual object based on the motion data provided from the student terminal (viewer terminal) 20, that is, the motion data indicating the real-time movement of the student (viewer). Identify. However, the method for specifying the movement of the target virtual object is not limited to this, and in connection with this, the viewer terminal does not have to have a function corresponding to the motion transmission unit 21. For example, the motion specifying unit 12 may specify the movement of the target virtual object based on the motion data provided by the teacher terminal 30, that is, the motion data indicating the real-time movement of the teacher (distributor). Alternatively, the motion specifying unit 12 may specify the movement of the target virtual object displayed in the original image. For example, the motion specifying unit 12 may specify the movement of an arbitrary target virtual object recorded in advance by analyzing the original video or by referring to the scenario included in the original video data. The target virtual object selected from the original video may represent any tangible object, eg, a person (avatar), any man-made object (eg, animal, real estate), any natural object (eg, animal, plant), and any. It may be at least one of the terrains (eg, mountains, rivers, ground).

In the above embodiment, the representation data indicates a visible virtual representation object, but the structure of the representation data is not limited to this. Therefore, virtual representation objects are not required. For example, the expression data may be realized by audio data that expresses a linguistic expression by voice, and in this case, a viewer such as a student can hear the linguistic expression.

In the above embodiment, the server 10 includes the dictionary management unit 16, but this functional element is not essential. Therefore, the content control system does not have to have a function related to storage and reference of dictionary data. Correspondingly, the viewer terminal does not have to have a function corresponding to the dictionary control unit 23.

In the above embodiment, the content distribution system 1 is configured by using the server 10, but the content control system may be applied to direct distribution between user terminals that do not use the server 10. In this case, each functional element of the server 10 may be implemented on any user terminal, or may be implemented on either a distributor terminal or a viewer terminal, for example. Alternatively, the individual functional elements of the server 10 may be implemented separately in a plurality of user terminals, and may be implemented separately in, for example, a distributor terminal and a viewer terminal. In this regard, the content control program may be implemented as a client program. The content control system may be configured with or without a server. Since the viewer terminal such as the student terminal has the function of the server 10, it is possible to provide common educational contents to a large number of students while reducing the load on the server. In addition, since information about a distributor such as a student (for example, information indicating an operation) is not transmitted to the outside of the viewer terminal, it is possible to more reliably protect the confidentiality of the viewer information.

As described above, the content control system may control any type of content other than educational content. For example, the content control system may control arbitrary content to support arbitrary information transmission or communication between users.

In the present disclosure, the expression "at least one processor executes the first process, executes the second process, ... executes the nth process", or the expression corresponding thereto is the first. This is a concept including a case where the execution subject (that is, the processor) of n processes from the first process to the nth process changes in the middle. That is, this expression is a concept that includes both a case where all n processes are executed by the same processor and a case where the processor changes according to an arbitrary policy in the n processes.

The processing procedure of the method executed by at least one processor is not limited to the example in the above embodiment. For example, some of the steps (processes) described above may be omitted, or each step may be executed in a different order. Further, any two or more steps among the above-mentioned steps may be combined, or a part of the steps may be modified or deleted. Alternatively, other steps may be performed in addition to each of the above steps.

1 ... Content distribution system, 10 ... Server, 11 ... Content management unit, 12 ... Motion identification unit, 13 ... Language expression determination unit, 14 ... Object setting unit, 15 ... Object transmission unit, 16 ... Dictionary management department, 20 ... Student Terminal, 21 ... Motion transmission unit, 22 ... Display control unit, 23 ... Dictionary control unit, 30 ... Teacher terminal, 40 ... Original video database, 50 ... Dictionary database, 321, 322, 411, 431, 511 ... Virtual representation object, 400,500 ... virtual space, P1 ... server program, P2 ... client program.

Claims (11)

With at least one processor
The at least one processor
Identify the movement of the target virtual object in the virtual space that shows the scene of the lesson,
With reference to the storage unit that stores the linguistic expression rule, the linguistic expression corresponding to the specified movement is determined.
Outputs the expression data corresponding to the determined linguistic expression.
Content control system. The at least one processor
Motion data showing the real-time movement of the student watching the lesson is received from the terminal, and
Identify the movement of the target virtual object based on the motion data.
The content control system according to claim 1. The at least one processor
Obtain the original video data showing the original video shot or generated in the past from the database,
Educational content data is generated using the acquired original video data and the expression data.
Output the generated educational content data,
The content control system according to claim 1 or 2. The at least one processor determines the linguistic representation of the behavior of the target virtual object.
The content control system according to any one of claims 1 to 3. The at least one processor determines the linguistic representation associated with a related virtual object that changes based on the movement of the target virtual object.
The content control system according to any one of claims 1 to 4. The at least one processor determines the linguistic representation that indicates the combination of the movement of the target virtual object and the change of the associated virtual object.
The content control system according to claim 5. The representation data indicates a virtual representation object displayed on the terminal.
The content control system according to any one of claims 1 to 6. The virtual representation object contains characters,
The content control system according to claim 7. The at least one processor
Acquire a registration request including an image including a scene showing the virtual expression object displayed on the terminal and a viewer ID of a student who is watching the lesson.
The viewer ID and the dictionary data including the image are stored in the dictionary database, and the dictionary data is stored.
In response to a search request from the terminal, the dictionary data corresponding to the search request is read from the dictionary database and the dictionary data is output to the terminal.
The content control system according to claim 7 or 8. A content control method performed by a content control system with at least one processor.
Steps to identify the movement of the target virtual object in the virtual space that shows the scene of the lesson,
With reference to a storage unit that stores linguistic expression rules, a step of determining the linguistic expression corresponding to the specified movement, and
A content control method including a step of outputting expression data corresponding to the determined language expression. Steps to identify the movement of the target virtual object in the virtual space that shows the scene of the lesson,
With reference to a storage unit that stores linguistic expression rules, a step of determining the linguistic expression corresponding to the specified movement, and
A content control program that causes a computer to execute a step of outputting expression data corresponding to the determined language expression.

Images