JP7258472B2 - Program, Information Processing Apparatus, and Method - Google Patents

Description

The present invention relates to programs, information processing apparatuses, and methods.

Patent Documents 1 to 3 disclose examples of techniques for allowing users to view content in virtual space.

JP 2017-176728 JP 2018-007828 JP 2016-025633

Conventional technologies have room for making the amount of money users pay more appropriate for virtual experiences in virtual space.

One aspect of the present disclosure aims to make the amount paid by the user more appropriate for the virtual experience in the virtual space.

According to one aspect of the invention, there is provided a program executed by a computer having a processor to provide a virtual experience to a first user. The program instructs the processor to define a virtual space including a first region for providing a virtual experience to a first user; to cause a first avatar associated with the first user to enter the first region; Placing a second avatar associated with the second user in the first area; causing the second avatar to perform a performance in accordance with the movement of the second user; A step of executing processing relating to the first billing, and a step of executing processing relating to the second billing for the second billing amount according to the performance are executed.

According to one aspect of the present disclosure, it is possible to make the amount paid by the user more appropriate for the virtual experience in the virtual space.

1 is a schematic representation of the configuration of an HMD system according to an embodiment; FIG. It is a block diagram showing an example of the hardware constitutions of the computer according to an embodiment. FIG. 4 is a diagram conceptually representing a uvw viewing coordinate system set in an HMD according to an embodiment; FIG. 2 is a diagram conceptually representing one aspect of representing a virtual space according to an embodiment; FIG. 2 is a top view of a user's head wearing an HMD according to an embodiment; It is a figure showing the YZ cross section which looked at the field-of-view area from the X direction in virtual space. It is a figure showing the XZ cross section which looked at the field-of-view area from the Y direction in virtual space. FIG. 2 is a diagram representing a schematic configuration of a controller according to an embodiment; FIG. FIG. 4 illustrates an example of yaw, roll, and pitch directions defined for a user's right hand according to an embodiment; It is a block diagram showing an example of a hardware configuration of a server according to an embodiment. 1 is a block diagram representing a computer as a module configuration according to an embodiment; FIG. 4 is a sequence chart representing part of the processing performed in the HMD set according to one embodiment; FIG. 2 is a schematic diagram showing a situation in which each HMD provides a virtual space to a user in a network; It is a figure which shows the user's 5A view image in FIG. 12(A). FIG. 4 is a sequence diagram showing processing performed in the HMD system according to one embodiment; 3 is a block diagram showing the detailed configuration of the modules of the computer according to one embodiment; FIG. FIG. 4 illustrates virtual space and field of view images according to an embodiment; FIG. 4 illustrates virtual space and field of view images according to an embodiment; FIG. 4 is a sequence diagram showing an example of processing executed in the HMD system according to one embodiment; FIG. 4 is a diagram for explaining a method of obtaining dimension data according to an embodiment; FIG. FIG. 4 is a diagram showing an example data structure of location information according to an embodiment; FIG. 4 is a diagram showing an example data structure of dimension data according to an embodiment; 4 is a flow chart representing a process for obtaining dimensional data according to one embodiment. FIG. 4 illustrates an example rotation data structure according to one embodiment. 4 is a sequence chart representing part of the processing performed in the HMD set according to one embodiment; FIG. 4 shows a first virtual space and field of view image according to an embodiment; FIG. 4 shows a first virtual space and field of view image according to an embodiment; 4 is a sequence chart representing part of the processing performed in the HMD set according to one embodiment; FIG. 10 illustrates a second virtual space and field of view images according to an embodiment; FIG. 10 illustrates a second virtual space and field of view images according to an embodiment; FIG. 10 illustrates a second virtual space and field of view images according to an embodiment; FIG. 10 illustrates a second virtual space and field of view images according to an embodiment; FIG. 4 is a diagram representing an example of a user's posture according to an embodiment; FIG. 10 illustrates a second virtual space and field of view images according to an embodiment; FIG. 10 illustrates a second virtual space and field of view images according to an embodiment; FIG. 10 illustrates a second virtual space and field of view images according to an embodiment; FIG. 10 illustrates a second virtual space and field of view images according to an embodiment; FIG. 10 illustrates a second virtual space and field of view images according to an embodiment; FIG. 4 shows a first virtual space and field of view image according to an embodiment; FIG. 10 illustrates a second virtual space and field of view images according to an embodiment; FIG. 10 illustrates a second virtual space and field of view images according to an embodiment; FIG. 10 illustrates a second virtual space and field of view images according to an embodiment; FIG. 10 illustrates a second virtual space and field of view images according to an embodiment; FIG. 10 illustrates a second virtual space and live video according to an embodiment; FIG. 10 illustrates a second virtual space and field of view images according to an embodiment; FIG. 10 illustrates a second virtual space and field of view images according to an embodiment; 4 is a sequence chart representing part of the processing performed in the HMD set according to one embodiment; FIG. 10 illustrates a second virtual space and field of view images according to an embodiment; FIG. 10 illustrates a second virtual space and field of view images according to an embodiment; FIG. 10 illustrates a second virtual space and field of view images according to an embodiment; FIG. 10 illustrates a second virtual space and field of view images according to an embodiment; FIG. 10 illustrates a second virtual space and field of view images according to an embodiment; FIG. 10 illustrates a second virtual space and field of view images according to an embodiment; FIG. 10 illustrates a second virtual space and field of view images according to an embodiment; FIG. 10 illustrates a second virtual space and field of view images according to an embodiment; FIG. 10 illustrates a second virtual space and field of view images according to an embodiment; FIG. 10 illustrates a second virtual space and field of view images according to an embodiment; FIG. 10 illustrates a second virtual space and field of view images according to an embodiment; FIG. 10 illustrates a second virtual space and field of view images according to an embodiment; 4 is a sequence chart representing part of the processing performed in the HMD set according to one embodiment; FIG. 10 illustrates a second virtual space and field of view images according to an embodiment; FIG. 10 illustrates a second virtual space and field of view images according to an embodiment; FIG. 10 illustrates a second virtual space and field of view images according to an embodiment; FIG. 10 illustrates a second virtual space and field of view images according to an embodiment; FIG. 10 illustrates a second virtual space and field of view images according to an embodiment; FIG. 10 illustrates a second virtual space and field of view images according to an embodiment; FIG. 10 illustrates a second virtual space and field of view images according to an embodiment; FIG. 10 illustrates a second virtual space and field of view images according to an embodiment; 4 is a sequence chart representing part of the processing performed in the HMD set according to one embodiment; FIG. 13 illustrates a third virtual space and field of view image according to an embodiment; FIG. 13 illustrates a third virtual space and field of view image according to an embodiment; 4 is a sequence chart representing part of the processing performed in the HMD set according to one embodiment; FIG. 10 illustrates a second virtual space and field of view images according to an embodiment; FIG. 10 illustrates a second virtual space and field of view images according to an embodiment; FIG. 4 is a diagram representing an example of a user's posture according to an embodiment; FIG. 10 illustrates a second virtual space and field of view images according to an embodiment; FIG. 10 illustrates a second virtual space and field of view images according to an embodiment; FIG. 10 illustrates a second virtual space and field of view images according to an embodiment; FIG. 13 illustrates a third virtual space and field of view image according to an embodiment; 4 is a sequence chart showing an example of processing executed in the HMD system; FIG. 10 illustrates a second virtual space and field of view images according to an embodiment; FIG. 10 illustrates a second virtual space and field of view images according to an embodiment; FIG. 10 illustrates a second virtual space and field of view images according to an embodiment; FIG. 10 illustrates a second virtual space and field of view images according to an embodiment; FIG. 4 is a diagram representing an example of a user's posture according to an embodiment; FIG. 10 illustrates a second virtual space and field of view images according to an embodiment; FIG. 10 illustrates a second virtual space and field of view images according to an embodiment; 4 is a sequence chart showing an example of processing executed in the HMD system; FIG. 10 illustrates a second virtual space and field of view images according to an embodiment; FIG. 10 illustrates a second virtual space and field of view images according to an embodiment; FIG. 10 illustrates a second virtual space and field of view images according to an embodiment; FIG. 10 illustrates a second virtual space and field of view images according to an embodiment; FIG. 10 illustrates an example of data referenced by a processor to notify a user of at least one second performance; FIG. 10 illustrates a second virtual space and field of view images according to an embodiment; 4 is a sequence chart showing an example of processing executed in the HMD system; FIG. 10 illustrates a second virtual space and field of view images according to an embodiment; FIG. 10 illustrates a second virtual space and field of view images according to an embodiment; FIG. 10 illustrates a second virtual space and field of view images according to an embodiment; FIG. 10 illustrates a second virtual space and field of view images according to an embodiment; FIG. 10 illustrates a second virtual space and field of view images according to an embodiment; FIG. 10 illustrates a second virtual space and field of view images according to an embodiment; FIG. 10 illustrates a second virtual space and field of view images according to an embodiment; FIG. 10 illustrates a second virtual space and field of view images according to an embodiment; FIG. 10 illustrates a second virtual space and field of view images according to an embodiment; FIG. 4 is a diagram showing a first virtual space (first region) and view image according to an embodiment; FIG. 4 shows a first virtual space and field of view image according to an embodiment; 4 is a sequence chart representing part of the processing performed on the HMD set according to one embodiment. FIG. 10 illustrates a second virtual space and field of view images according to an embodiment; FIG. 4 shows a first virtual space and field of view image according to an embodiment; FIG. 10 illustrates a second virtual space and field of view images according to an embodiment; FIG. 10 illustrates a second virtual space and field of view images according to an embodiment; FIG. 10 illustrates a second virtual space and field of view images according to an embodiment; FIG. 10 illustrates a second virtual space and field of view images according to an embodiment; FIG. 10 illustrates a second virtual space and field of view images according to an embodiment; FIG. 10 illustrates a second virtual space and field of view images according to an embodiment; FIG. 10 illustrates a second virtual space and field of view images according to an embodiment; FIG. 10 illustrates a second virtual space and field of view images according to an embodiment;

[Embodiment 1]
Hereinafter, embodiments of this technical idea will be described in detail with reference to the drawings. In the following description, the same parts are given the same reference numerals. Their names and functions are also the same. Therefore, detailed description thereof will not be repeated. In one or more of the embodiments presented in this disclosure, elements of each embodiment may be combined with each other and the combined result shall also form part of the embodiments presented in this disclosure.

[Configuration of HMD system]
The configuration of an HMD (Head-Mounted Device) system 100 will be described with reference to FIG. FIG. 1 is a diagram showing a schematic configuration of an HMD system 100 according to this embodiment. The HMD system 100 is provided as a system for home use or as a system for business use.

HMD system 100 includes server 600 , HMD sets 110 A, 110 B, 110 C, and 110 D, external device 700 , and network 2 . Each of HMD sets 110A, 110B, 110C, and 110D is configured to communicate with server 600 and external device 700 via network 2 . Hereinafter, the HMD sets 110A, 110B, 110C, and 110D are collectively referred to as the HMD set 110 as well. The number of HMD sets 110 configuring the HMD system 100 is not limited to four, and may be three or less or five or more. HMD set 110 includes HMD 120 , computer 200 , HMD sensor 410 , display 430 and controller 300 . HMD 120 includes monitor 130 , gaze sensor 140 , first camera 150 , second camera 160 , microphone 170 , and speaker 180 . Controller 300 may include motion sensor 420 .

In one aspect, computer 200 is connectable to network 2 such as the Internet, and is capable of communicating with server 600 and other computers connected to network 2 . Other computers include, for example, computers of other HMD sets 110 and external devices 700 . In another aspect, HMD 120 may include sensor 190 instead of HMD sensor 410 .

The HMD 120 can be worn on the head of the user 5 and provide the user 5 with a virtual space during operation. More specifically, HMD 120 displays a right-eye image and a left-eye image on monitor 130 . When each eye of the user 5 views the respective image, the user 5 can perceive the image as a three-dimensional image based on the parallax of both eyes. The HMD 120 may include both a so-called head-mounted display having a monitor and a head-mounted device on which a smartphone or other terminal having a monitor can be attached.

The monitor 130 is implemented as, for example, a non-transmissive display device. In one aspect, the monitor 130 is arranged on the main body of the HMD 120 so as to be positioned in front of both eyes of the user 5 . Therefore, the user 5 can immerse himself in the virtual space by viewing the three-dimensional image displayed on the monitor 130 . In one aspect, the virtual space includes images of, for example, a background, objects that the user 5 can manipulate, and menus that the user 5 can select. In one aspect, the monitor 130 can be realized as a liquid crystal monitor or an organic EL (Electro Luminescence) monitor provided in a so-called smart phone or other information display terminal.

In another aspect, monitor 130 may be implemented as a transmissive display. In this case, the HMD 120 may be an open type, such as a glasses type, instead of a closed type that covers the eyes of the user 5 as shown in FIG. Transmissive monitor 130 may be temporarily configurable as a non-transmissive display by adjusting its transmittance. The monitor 130 may include a configuration for simultaneously displaying a portion of the image forming the virtual space and the real space. For example, the monitor 130 may display an image of the real space captured by a camera mounted on the HMD 120, or may make the real space visible by setting a partial transmittance high.

In one aspect, monitor 130 may include a sub-monitor for displaying images for the right eye and a sub-monitor for displaying images for the left eye. In another aspect, the monitor 130 may be configured to display the image for the right eye and the image for the left eye as one. In this case, monitor 130 includes a high speed shutter. The high speed shutter operates to alternately display the right eye image and the left eye image so that the image is perceived by only one eye.

In one aspect, the HMD 120 includes multiple light sources (not shown). Each light source is implemented by, for example, an LED (Light Emitting Diode) that emits infrared rays. HMD sensor 410 has a position tracking function for detecting movement of HMD 120 . More specifically, HMD sensor 410 reads a plurality of infrared rays emitted by HMD 120 and detects the position and tilt of HMD 120 in the physical space.

In another aspect, HMD sensor 410 may be implemented by a camera. In this case, the HMD sensor 410 can detect the position and tilt of the HMD 120 by executing image analysis processing using the image information of the HMD 120 output from the camera.

In another aspect, HMD 120 may include sensor 190 instead of HMD sensor 410 or in addition to HMD sensor 410 as a position detector. HMD 120 can detect the position and tilt of HMD 120 using sensor 190 . For example, if sensor 190 is an angular velocity sensor, a geomagnetic sensor, or an acceleration sensor, HMD 120 can detect its own position and tilt using any of these sensors instead of HMD sensor 410 . As an example, when the sensor 190 is an angular velocity sensor, the angular velocity sensor temporally detects angular velocities around three axes of the HMD 120 in real space. The HMD 120 calculates temporal changes in angles around the three axes of the HMD 120 based on the angular velocities, and further calculates the tilt of the HMD 120 based on the temporal changes in the angles.

Gaze sensor 140 detects the directions in which the user's 5 right and left eyes are directed. That is, the gaze sensor 140 detects the line of sight of the user 5 . Detection of the line-of-sight direction is achieved by, for example, a known eye-tracking function. Gaze sensor 140 is realized by a sensor having the eye tracking function. In one aspect, gaze sensor 140 preferably includes a right eye sensor and a left eye sensor. The gaze sensor 140 may be, for example, a sensor that irradiates the right and left eyes of the user 5 with infrared rays and receives reflected light from the cornea and iris of the irradiated light, thereby detecting the rotation angle of each eyeball. The gaze sensor 140 can detect the line of sight of the user 5 based on each detected rotation angle.

The first camera 150 photographs the lower part of the user's 5 face. More specifically, first camera 150 captures the nose, mouth, and the like of user 5 . The second camera 160 photographs the eyes, eyebrows, etc. of the user 5 . The housing of the HMD 120 on the side of the user 5 is defined as the inside of the HMD 120 , and the housing of the HMD 120 on the side opposite to the user 5 is defined as the outside of the HMD 120 . In one aspect, first camera 150 may be positioned outside HMD 120 and second camera 160 may be positioned inside HMD 120 . Images generated by first camera 150 and second camera 160 are input to computer 200 . In another aspect, the first camera 150 and the second camera 160 may be realized as one camera, and the face of the user 5 may be photographed with this one camera.

The microphone 170 converts the speech of the user 5 into an audio signal (electrical signal) and outputs it to the computer 200 . The speaker 180 converts the audio signal into audio and outputs it to the user 5 . In another aspect, HMD 120 may include earphones instead of speaker 180 .

The controller 300 is connected to the computer 200 by wire or wirelessly. The controller 300 accepts command input from the user 5 to the computer 200 . In one aspect, controller 300 is configured to be grippable by user 5 . In another aspect, controller 300 is configured to be attachable to part of user's 5 body or clothing. In yet another aspect, controller 300 may be configured to output at least one of vibration, sound, and light based on signals transmitted from computer 200 . In yet another aspect, controller 300 receives an operation from user 5 for controlling the position and movement of an object placed in the virtual space.

In one aspect, controller 300 includes multiple light sources. Each light source is implemented, for example, by an LED that emits infrared rays. The HMD sensor 410 has a position tracking function. In this case, the HMD sensor 410 reads multiple infrared rays emitted by the controller 300 and detects the position and tilt of the controller 300 in the physical space. In another aspect, HMD sensor 410 may be implemented by a camera. In this case, the HMD sensor 410 can detect the position and tilt of the controller 300 by executing image analysis processing using the image information of the controller 300 output from the camera.

Motion sensor 420 , in one aspect, is attached to the hand of user 5 to detect movement of the hand of user 5 . For example, the motion sensor 420 detects hand rotation speed, number of rotations, and the like. The detected signal is sent to computer 200 . Motion sensor 420 is provided in controller 300, for example. In one aspect, motion sensor 420 is provided, for example, in controller 300 configured to be grippable by user 5 . In another aspect, for safety in the real space, the controller 300 is attached to an object such as a glove that is attached to the hand of the user 5 so as not to fly off easily. In yet another aspect, sensors not worn by user 5 may detect movement of user's 5 hand. For example, a signal from a camera that takes an image of user 5 may be input to computer 200 as a signal representing the motion of user 5 . Motion sensor 420 and computer 200 are, for example, wirelessly connected to each other. In the case of wireless communication, the form of communication is not particularly limited, and for example, Bluetooth (registered trademark) or other known communication methods are used.

Display 430 displays an image similar to the image displayed on monitor 130 . This allows users other than the user 5 wearing the HMD 120 to view the same image as that of the user 5 . The image displayed on display 430 does not have to be a three-dimensional image, and may be an image for the right eye or an image for the left eye. Examples of the display 430 include a liquid crystal display and an organic EL monitor.

Server 600 may transmit programs to computer 200 . In another aspect, server 600 may communicate with other computers 200 to provide virtual reality to HMDs 120 used by other users. For example, when a plurality of users play a participatory game in an amusement facility, each computer 200 communicates a signal based on each user's action with the other computers 200 via the server 600 so that a plurality of users can participate in the same virtual space. users to enjoy common games. Each computer 200 may communicate signals based on each user's actions with other computers 200 without going through the server 600 .

The external device 700 may be any device as long as it can communicate with the computer 200 . The external device 700 may be, for example, a device capable of communicating with the computer 200 via the network 2, or may be a device capable of directly communicating with the computer 200 through short-range wireless communication or wired connection. Examples of the external device 700 include smart devices, PCs (Personal Computers), and peripheral devices of the computer 200, but are not limited to these.

[Computer hardware configuration]
A computer 200 according to the present embodiment will be described with reference to FIG. FIG. 2 is a block diagram showing an example of the hardware configuration of computer 200 according to this embodiment. The computer 200 includes a processor 210, a memory 220, a storage 230, an input/output interface 240, and a communication interface 250 as main components. Each component is connected to bus 260 respectively.

Processor 210 executes a series of instructions contained in a program stored in memory 220 or storage 230 based on a signal given to computer 200 or based on the establishment of a predetermined condition. In one aspect, processor 210 is implemented as a CPU (Central Processing Unit), GPU (Graphics Processing Unit), MPU (Micro Processor Unit), FPGA (Field-Programmable Gate Array), or other device.

Memory 220 temporarily stores programs and data. The program is loaded from storage 230, for example. Data includes data input to computer 200 and data generated by processor 210 . In one aspect, memory 220 is implemented as random access memory (RAM) or other volatile memory.

Storage 230 permanently holds programs and data. The storage 230 is implemented as, for example, a ROM (Read-Only Memory), hard disk device, flash memory, or other non-volatile storage device. The programs stored in the storage 230 include programs for providing a virtual space in the HMD system 100, simulation programs, game programs, user authentication programs, and programs for realizing communication with other computers 200. The data stored in the storage 230 includes data and objects for defining the virtual space.

In another aspect, storage 230 may be implemented as a removable storage device such as a memory card. In still another aspect, instead of storage 230 built into computer 200, a configuration using programs and data stored in an external storage device may be used. According to such a configuration, for example, in a scene where a plurality of HMD systems 100 are used, such as an amusement facility, it is possible to collectively update programs and data.

Input/output interface 240 communicates signals between HMD 120 , HMD sensor 410 , motion sensor 420 and display 430 . Monitor 130 , gaze sensor 140 , first camera 150 , second camera 160 , microphone 170 and speaker 180 included in HMD 120 can communicate with computer 200 via input/output interface 240 of HMD 120 . In one aspect, the input/output interface 240 is implemented using a USB (Universal Serial Bus), DVI (Digital Visual Interface), HDMI (registered trademark) (High-Definition Multimedia Interface), or other terminals. Input/output interface 240 is not limited to the one described above.

In some aspects, input/output interface 240 may also communicate with controller 300 . For example, input/output interface 240 receives signals output from controller 300 and motion sensor 420 . In another aspect, input/output interface 240 sends instructions output from processor 210 to controller 300 . The command instructs the controller 300 to vibrate, output sound, emit light, or the like. Upon receiving the command, the controller 300 performs any one of vibration, sound output, or light emission according to the command.

The communication interface 250 is connected to the network 2 and communicates with other computers connected to the network 2 (for example, the server 600). In one aspect, the communication interface 250 is implemented as, for example, a LAN (Local Area Network) or other wired communication interface, or a WiFi (Wireless Fidelity), Bluetooth (registered trademark), NFC (Near Field Communication) or other wireless communication interface. be done. Communication interface 250 is not limited to the one described above.

In one aspect, processor 210 accesses storage 230, loads one or more programs stored in storage 230 into memory 220, and executes the sequences of instructions contained in the programs. The one or more programs may include an operating system of computer 200, an application program for providing virtual space, game software executable in virtual space, and the like. Processor 210 sends a signal for providing virtual space to HMD 120 via input/output interface 240 . HMD 120 displays an image on monitor 130 based on the signal.

In the example shown in FIG. 2, computer 200 is provided outside HMD 120, but computer 200 may be built into HMD 120 in another aspect. As an example, a portable information communication terminal (for example, a smart phone) including monitor 130 may function as computer 200 .

The computer 200 may have a configuration that is commonly used by a plurality of HMDs 120 . According to such a configuration, for example, the same virtual space can be provided to a plurality of users, so each user can enjoy the same application as other users in the same virtual space.

In one embodiment, in the HMD system 100, a real coordinate system, which is a coordinate system in real space, is set in advance. The real coordinate system has three reference directions (axes) parallel to the vertical direction in the real space, the horizontal direction perpendicular to the vertical direction, and the front-rear direction perpendicular to both the vertical and horizontal directions. The horizontal direction, vertical direction (vertical direction), and front-rear direction in the real coordinate system are defined as x-axis, y-axis, and z-axis, respectively. More specifically, in the real coordinate system, the x-axis is parallel to the horizontal direction of the real space. The y-axis is parallel to the vertical direction in real space. The z-axis is parallel to the front-back direction of the physical space.

In one aspect, HMD sensor 410 includes an infrared sensor. The presence of HMD 120 is detected when the infrared sensor detects infrared rays emitted from each light source of HMD 120 . The HMD sensor 410 further detects the position and inclination (orientation) of the HMD 120 in the physical space according to the movement of the user 5 wearing the HMD 120, based on the value of each point (each coordinate value in the real coordinate system). do. More specifically, the HMD sensor 410 can detect temporal changes in the position and tilt of the HMD 120 using each value detected over time.

Each tilt of HMD 120 detected by HMD sensor 410 corresponds to each tilt of HMD 120 around three axes in the real coordinate system. The HMD sensor 410 sets the uvw visual field coordinate system to the HMD 120 based on the tilt of the HMD 120 in the real coordinate system. The uvw visual field coordinate system set in the HMD 120 corresponds to the viewpoint coordinate system when the user 5 wearing the HMD 120 sees an object in virtual space.

[uvw visual field coordinate system]
The uvw visual field coordinate system will be described with reference to FIG. FIG. 3 is a diagram conceptually representing the uvw viewing coordinate system set in the HMD 120 according to one embodiment. HMD sensor 410 detects the position and tilt of HMD 120 in the real coordinate system when HMD 120 is activated. Processor 210 sets the uvw viewing coordinate system to HMD 120 based on the detected values.

As shown in FIG. 3 , the HMD 120 sets a three-dimensional uvw visual field coordinate system centered (origin) on the head of the user 5 wearing the HMD 120 . More specifically, the HMD 120 rotates the horizontal direction, the vertical direction, and the front-rear direction (x-axis, y-axis, and z-axis) defining the real coordinate system by tilts around each axis of the HMD 120 within the real coordinate system. The three directions newly obtained by tilting around the axes are set as the pitch axis (u-axis), yaw-axis (v-axis), and roll-axis (w-axis) of the uvw visual field coordinate system in the HMD 120 .

In one aspect, when user 5 wearing HMD 120 is standing upright and looking straight ahead, processor 210 sets HMD 120 to a uvw viewing coordinate system that is parallel to the real coordinate system. In this case, the horizontal direction (x-axis), vertical direction (y-axis), and front-back direction (z-axis) in the real coordinate system are the pitch axis (u-axis) and yaw-axis (v-axis) of the uvw visual field coordinate system in the HMD 120. , and the roll axis (w-axis).

After the uvw visual field coordinate system is set on the HMD 120 , the HMD sensor 410 can detect the tilt of the HMD 120 in the set uvw visual field coordinate system based on the movement of the HMD 120 . In this case, the HMD sensor 410 detects the pitch angle (θu), yaw angle (θv), and roll angle (θw) of the HMD 120 in the uvw visual field coordinate system as the tilt of the HMD 120 . The pitch angle (θu) represents the tilt angle of the HMD 120 around the pitch axis in the uvw visual field coordinate system. The yaw angle (θv) represents the tilt angle of the HMD 120 around the yaw axis in the uvw visual field coordinate system. A roll angle (θw) represents the tilt angle of the HMD 120 around the roll axis in the uvw visual field coordinate system.

The HMD sensor 410 sets the uvw visual field coordinate system in the HMD 120 after the HMD 120 moves based on the detected inclination of the HMD 120 to the HMD 120 . The relationship between the HMD 120 and the uvw visual field coordinate system of the HMD 120 is always constant regardless of the position and tilt of the HMD 120 . When the position and tilt of the HMD 120 change, the position and tilt of the uvw visual field coordinate system of the HMD 120 in the real coordinate system change in conjunction with the change in the position and tilt.

In one aspect, HMD sensor 410 detects HMD 120 based on the infrared light intensity and the relative positional relationship between a plurality of points (for example, the distance between points) obtained based on the output from the infrared sensor. position in the physical space may be specified as a relative position with respect to the HMD sensor 410 . Processor 210 may determine the origin of the uvw visual field coordinate system of HMD 120 in physical space (real coordinate system) based on the specified relative position.

[Virtual space]
The virtual space will be further described with reference to FIG. FIG. 4 is a diagram conceptually showing one aspect of expressing virtual space 11 according to an embodiment. The virtual space 11 has a spherical structure covering the entire 360-degree direction of the center 12 . In FIG. 4, the celestial sphere in the upper half of the virtual space 11 is illustrated in order not to complicate the explanation. Each mesh is defined in the virtual space 11 . The position of each mesh is defined in advance as coordinate values in the XYZ coordinate system, which is the global coordinate system defined in the virtual space 11 . The computer 200 associates each partial image that constitutes the panorama image 13 (still image, moving image, etc.) that can be developed in the virtual space 11 with each corresponding mesh in the virtual space 11 .

In one aspect, the virtual space 11 defines an XYZ coordinate system with the center 12 as the origin. The XYZ coordinate system is parallel to the real coordinate system, for example. The horizontal direction, vertical direction (vertical direction), and front-rear direction in the XYZ coordinate system are defined as the X-axis, Y-axis, and Z-axis, respectively. Therefore, the X-axis (horizontal direction) of the XYZ coordinate system is parallel to the x-axis of the real coordinate system, the Y-axis (vertical direction) of the XYZ coordinate system is parallel to the y-axis of the real coordinate system, and the The Z-axis (front-rear direction) is parallel to the z-axis of the real coordinate system.

When the HMD 120 is activated, that is, in the initial state of the HMD 120 , the virtual camera 14 is arranged at the center 12 of the virtual space 11 . In one aspect, processor 210 displays an image captured by virtual camera 14 on monitor 130 of HMD 120 . The virtual camera 14 similarly moves in the virtual space 11 in conjunction with the movement of the HMD 120 in the real space. Thereby, changes in the position and tilt of the HMD 120 in the real space can be similarly reproduced in the virtual space 11 .

A uvw field-of-view coordinate system is defined in the virtual camera 14 as in the case of the HMD 120 . The uvw visual field coordinate system of the virtual camera 14 in the virtual space 11 is defined to interlock with the uvw visual field coordinate system of the HMD 120 in the real space (real coordinate system). Therefore, when the tilt of HMD 120 changes, the tilt of virtual camera 14 also changes accordingly. The virtual camera 14 can also move in the virtual space 11 in conjunction with the movement of the user 5 wearing the HMD 120 in the real space.

The processor 210 of the computer 200 defines the field of view area 15 in the virtual space 11 based on the position and tilt of the virtual camera 14 (reference line of sight 16). The field of view area 15 corresponds to an area of the virtual space 11 that is viewed by the user 5 wearing the HMD 120 . That is, the position of the virtual camera 14 can be said to be the viewpoint of the user 5 in the virtual space 11 .

The line of sight of the user 5 detected by the gaze sensor 140 is the direction in the viewpoint coordinate system when the user 5 visually recognizes an object. The uvw visual field coordinate system of the HMD 120 is equal to the viewpoint coordinate system when the user 5 visually recognizes the monitor 130 . The uvw visual field coordinate system of the virtual camera 14 is interlocked with the uvw visual field coordinate system of the HMD 120 . Therefore, the HMD system 100 according to one aspect can regard the line of sight of the user 5 detected by the gaze sensor 140 as the line of sight of the user 5 in the uvw field coordinate system of the virtual camera 14 .

[User's line of sight]
Determination of the line of sight of the user 5 will be described with reference to FIG. FIG. 5 is a top view of the head of user 5 wearing HMD 120 according to an embodiment.

In one aspect, gaze sensor 140 detects the line of sight of user 5's right and left eyes. In one aspect, when user 5 is looking near, gaze sensor 140 detects lines of sight R1 and L1. In another aspect, when user 5 is looking far away, gaze sensor 140 detects lines of sight R2 and L2. In this case, the angle between the lines of sight R2 and L2 with respect to the roll axis w is smaller than the angle between the lines of sight R1 and L1 with respect to the roll axis w. Gaze sensor 140 transmits the detection result to computer 200 .

When computer 200 receives detection values of lines of sight R1 and L1 from gaze sensor 140 as a detection result of lines of sight, computer 200 identifies point of gaze N1, which is the intersection of lines of sight R1 and L1, based on the detected values. On the other hand, when computer 200 receives detection values of lines of sight R2 and L2 from gaze sensor 140, computer 200 identifies the intersection of lines of sight R2 and L2 as the point of gaze. The computer 200 identifies the line of sight N0 of the user 5 based on the identified position of the gaze point N1. The computer 200 detects, for example, the extending direction of a straight line passing through the midpoint of the straight line connecting the right eye R and the left eye L of the user 5 and the gaze point N1 as the line of sight N0. The line of sight N0 is the direction in which the user 5 is actually looking with both eyes. The line of sight N<b>0 corresponds to the direction in which the user 5 actually looks toward the field of view area 15 .

In another aspect, HMD system 100 may include a television broadcast reception tuner. With such a configuration, the HMD system 100 can display television programs in the virtual space 11 .

In still another aspect, the HMD system 100 may have a communication circuit for connecting to the Internet or a calling function for connecting to a telephone line.

[Vision area]
The field of view area 15 will be described with reference to FIGS. 6 and 7. FIG. FIG. 6 is a diagram showing a YZ cross section of the visual field area 15 viewed from the X direction in the virtual space 11. As shown in FIG. FIG. 7 is a diagram showing an XZ cross section of the visual field area 15 viewed from the Y direction in the virtual space 11. As shown in FIG.

As shown in FIG. 6 , the field of view area 15 in the YZ cross section includes area 18 . A region 18 is defined by the position of the virtual camera 14 , the reference line of sight 16 and the YZ section of the virtual space 11 . The processor 210 defines an area 18 centered on the reference line of sight 16 in the virtual space and including the polar angle α.

As shown in FIG. 7 , the viewing area 15 in the XZ cross section includes area 19 . A region 19 is defined by the position of the virtual camera 14 , the reference line of sight 16 and the XZ section of the virtual space 11 . The processor 210 defines a range including the azimuth angle β around the reference line of sight 16 in the virtual space 11 as the region 19 . The polar angles α and β are determined according to the position of the virtual camera 14 and the tilt (orientation) of the virtual camera 14 .

In one aspect, HMD system 100 provides user 5 with a view in virtual space 11 by displaying view image 17 on monitor 130 based on a signal from computer 200 . The field-of-view image 17 is an image corresponding to a portion of the panoramic image 13 corresponding to the field-of-view area 15 . When the user 5 moves the HMD 120 worn on the head, the virtual camera 14 also moves in conjunction with the movement. As a result, the position of the viewing area 15 in the virtual space 11 changes. As a result, the field-of-view image 17 displayed on the monitor 130 is updated to an image of the panorama image 13 that is superimposed on the field-of-view area 15 in the direction the user 5 faces in the virtual space 11 . The user 5 can visually recognize a desired direction in the virtual space 11 .

Thus, the tilt of the virtual camera 14 corresponds to the line of sight (reference line of sight 16 ) of the user 5 in the virtual space 11 , and the position where the virtual camera 14 is arranged corresponds to the viewpoint of the user 5 in the virtual space 11 . Therefore, by changing the position or tilt of the virtual camera 14, the image displayed on the monitor 130 is updated and the field of view of the user 5 is moved.

While wearing the HMD 120, the user 5 can visually recognize only the panoramic image 13 developed in the virtual space 11 without visually recognizing the real world. Therefore, the HMD system 100 can give the user 5 a high sense of immersion in the virtual space 11 .

In one aspect, processor 210 can move virtual camera 14 in virtual space 11 in conjunction with movement of user 5 wearing HMD 120 in the physical space. In this case, processor 210 identifies an image area (viewing area 15 ) projected on monitor 130 of HMD 120 based on the position and tilt of virtual camera 14 in virtual space 11 .

In one aspect, virtual camera 14 may include two virtual cameras, a virtual camera for providing images for the right eye and a virtual camera for providing images for the left eye. Appropriate parallax is set for the two virtual cameras so that the user 5 can recognize the three-dimensional virtual space 11 . In another aspect, virtual camera 14 may be realized by one virtual camera. In this case, an image for the right eye and an image for the left eye may be generated from an image obtained by one virtual camera. In the present embodiment, the virtual camera 14 includes two virtual cameras, and the roll axis (w) generated by synthesizing the roll axes of the two virtual cameras is adapted to the roll axis (w) of the HMD 120. The technical idea according to the present disclosure is exemplified as configured as follows.

[controller]
An example of the controller 300 will be described with reference to FIG. FIG. 8 is a diagram showing a schematic configuration of controller 300 according to an embodiment.

As shown in FIG. 8, in one aspect, controller 300 may include a right controller 300R and a left controller (not shown). Right controller 300R is operated by user 5's right hand. The left controller is operated with the user's 5 left hand. In one aspect, right controller 300R and left controller are symmetrically configured as separate devices. Therefore, the user 5 can freely move the right hand holding the right controller 300R and the left hand holding the left controller. In another aspect, controller 300 may be an integrated controller that accepts two-handed operation. The right controller 300R will be described below.

Right controller 300R includes grip 310 , frame 320 , and top surface 330 . The grip 310 is configured to be gripped by the user's 5 right hand. For example, grip 310 may be held by user 5's right palm and three fingers (middle finger, ring finger, little finger).

Grip 310 includes buttons 340 and 350 and motion sensor 420 . Button 340 is arranged on the side surface of grip 310 and is operated by the middle finger of the right hand. Button 350 is arranged on the front surface of grip 310 and is operated by the index finger of the right hand. In one aspect, buttons 340, 350 are configured as trigger buttons. Motion sensor 420 is built into the housing of grip 310 . The grip 310 may not include the motion sensor 420 if the motion of the user 5 can be detected from around the user 5 by a camera or other device.

Frame 320 includes a plurality of infrared LEDs 360 arranged along its circumference. Infrared LED 360 emits infrared light in accordance with the progress of a program using controller 300 during execution of the program. Infrared rays emitted from infrared LED 360 can be used to detect the positions and orientations (inclination and orientation) of right controller 300R and left controller. Although the example shown in FIG. 8 shows infrared LEDs 360 arranged in two rows, the number of arrays is not limited to that shown in FIG. A single row or three or more row arrays may be used.

Top surface 330 includes buttons 370 and 380 and analog stick 390 . Buttons 370 and 380 are configured as push buttons. Buttons 370 and 380 accept operations by user 5's right thumb. In a certain aspect, the analog stick 390 accepts an operation in any direction of 360 degrees from the initial position (neutral position). The operation includes, for example, an operation for moving an object placed in the virtual space 11 .

In one aspect, right controller 300R and left controller include batteries for powering infrared LEDs 360 and other components. Batteries include, but are not limited to, rechargeable, button-type, dry cell-type, and the like. In another aspect, right controller 300R and left controller may be connected to a USB interface of computer 200, for example. In this case, right controller 300R and left controller do not require batteries.

As shown in states (A) and (B) of FIG. 8, for example, the yaw, roll, and pitch directions are defined for the user's 5 right hand. When the user 5 extends the thumb and index finger, the direction in which the thumb extends is the yaw direction, the direction in which the index finger extends is the roll direction, and the direction perpendicular to the plane defined by the yaw direction axis and the roll direction axis is the pitch direction. defined as

[Server hardware configuration]
Server 600 according to the present embodiment will be described with reference to FIG. FIG. 9 is a block diagram showing an example hardware configuration of server 600 according to an embodiment. The server 600 includes a processor 610, a memory 620, a storage 630, an input/output interface 640, and a communication interface 650 as main components. Each component is respectively connected to bus 660 .

Processor 610 executes a series of instructions contained in a program stored in memory 620 or storage 630 based on a signal provided to server 600 or based on the establishment of a predetermined condition. In one aspect, processor 610 is implemented as a CPU, GPU, MPU, FPGA, or other device.

Memory 620 temporarily stores programs and data. The program is loaded from storage 630, for example. Data includes data input to server 600 and data generated by processor 610 . In one aspect, memory 620 is implemented as RAM or other volatile memory.

Storage 630 permanently holds programs and data. The storage 630 is implemented as, for example, a ROM, hard disk device, flash memory, or other non-volatile memory device. The programs stored in the storage 630 may include a program for providing a virtual space in the HMD system 100, a simulation program, a game program, a user authentication program, and a program for realizing communication with the computer 200. The data stored in the storage 630 may include data and objects for defining the virtual space.

In another aspect, storage 630 may be implemented as a removable storage device such as a memory card. In still another aspect, instead of storage 630 built into server 600, a configuration using programs and data stored in an external storage device may be used. According to such a configuration, for example, in a scene where a plurality of HMD systems 100 are used, such as an amusement facility, it is possible to collectively update programs and data.

Input/output interface 640 communicates signals with input/output devices. In one aspect, input/output interface 640 is implemented using USB, DVI, HDMI, or other terminals. Input/output interface 640 is not limited to the one described above.

Communication interface 650 is connected to network 2 and communicates with computer 200 connected to network 2 . In one aspect, communication interface 650 is implemented as, for example, a LAN or other wired communication interface, or a WiFi, Bluetooth, NFC or other wireless communication interface. Communication interface 650 is not limited to those described above.

In one aspect, processor 610 accesses storage 630, loads one or more programs stored in storage 630 into memory 620, and executes the sequences of instructions contained in the programs. The one or more programs may include an operating system of server 600, an application program for providing virtual space, game software executable in virtual space, and the like. Processor 610 may send signals to computer 200 via input/output interface 640 to provide the virtual space.

[HMD control device]
A control device for the HMD 120 will be described with reference to FIG. In one embodiment, the controller is implemented by computer 200 having a well-known configuration. FIG. 10 is a block diagram representing a modular configuration of computer 200 according to one embodiment.

As shown in FIG. 10, computer 200 comprises control module 510 , rendering module 520 , memory module 530 and communication control module 540 . In one aspect, control module 510 and rendering module 520 are implemented by processor 210 . In another aspect, multiple processors 210 may act as control module 510 and rendering module 520 . Memory module 530 is implemented by memory 220 or storage 230 . Communication control module 540 is implemented by communication interface 250 .

A control module 510 controls the virtual space 11 provided to the user 5 . The control module 510 defines the virtual space 11 in the HMD system 100 using virtual space data representing the virtual space 11 . Virtual space data is stored, for example, in memory module 530 . The control module 510 may generate virtual space data or acquire virtual space data from the server 600 or the like.

The control module 510 places objects in the virtual space 11 using object data representing the objects. Object data is stored, for example, in memory module 530 . The control module 510 may generate object data or acquire object data from the server 600 or the like. The objects include, for example, an avatar object that is the alter ego of the user 5, a character object, an operation object such as a virtual hand operated by the controller 300, landscapes including forests, mountains, etc. arranged according to the progress of the story of the game, townscapes, and animals. etc.

The control module 510 places the avatar object of the user 5 of the other computer 200 connected via the network 2 in the virtual space 11 . In one aspect, control module 510 places user 5's avatar object in virtual space 11 . In one aspect, the control module 510 places an avatar object that resembles the user 5 in the virtual space 11 based on the image containing the user 5 . In another aspect, the control module 510 arranges in the virtual space 11 an avatar object that has been selected by the user 5 from among multiple types of avatar objects (for example, an object imitating an animal or a deformed human object). do.

Control module 510 identifies the tilt of HMD 120 based on the output of HMD sensor 410 . In another aspect, control module 510 identifies the tilt of HMD 120 based on the output of sensor 190, which functions as a motion sensor. The control module 510 detects the facial organs of the user 5 (eg, mouth, eyes, eyebrows) from the facial images of the user 5 generated by the first camera 150 and the second camera 160 . The control module 510 detects the motion (shape) of each detected organ.

The control module 510 detects the line of sight of the user 5 in the virtual space 11 based on the signal from the gaze sensor 140 . The control module 510 detects the viewpoint position (coordinate values in the XYZ coordinate system) where the detected line of sight of the user 5 and the celestial sphere of the virtual space 11 intersect. More specifically, the control module 510 detects the viewpoint position based on the line of sight of the user 5 defined by the uvw coordinate system and the position and tilt of the virtual camera 14 . Control module 510 transmits the detected viewpoint position to server 600 . In another aspect, the control module 510 may be configured to transmit line-of-sight information representing the line-of-sight of the user 5 to the server 600 . In this case, the viewpoint position can be calculated based on the line-of-sight information received by the server 600 .

The control module 510 reflects the movement of the HMD 120 detected by the HMD sensor 410 on the avatar object. For example, the control module 510 detects that the HMD 120 is tilted, and tilts and arranges the avatar object. The control module 510 reflects the motion of the detected facial features on the face of the avatar object placed in the virtual space 11 . The control module 510 receives line-of-sight information of the other user 5 from the server 600 and reflects the line-of-sight information of the avatar object of the other user 5 . In one aspect, control module 510 reflects movements of controller 300 on avatar objects and operation objects. In this case, controller 300 includes a motion sensor, an acceleration sensor, a plurality of light emitting elements (for example, infrared LEDs), or the like for detecting movement of controller 300 .

The control module 510 arranges in the virtual space 11 an operation object for receiving an operation by the user 5 in the virtual space 11 . The user 5 operates an object placed in the virtual space 11, for example, by operating the operation object. In one aspect, the manipulation object may include, for example, a hand object, which is a virtual hand corresponding to the user's 5 hand. In one aspect, the control module 510 moves the hand object in the virtual space 11 based on the output of the motion sensor 420 in conjunction with the hand movement of the user 5 in the real space. In one aspect, the manipulation object may correspond to a hand portion of the avatar object.

The control module 510 detects a collision when each of the objects placed in the virtual space 11 collides with another object. The control module 510 can detect, for example, the timing at which the collision area of a certain object touches the collision area of another object, and performs predetermined processing when the detection is made. The control module 510 can detect the timing when the objects are separated from the touching state, and performs predetermined processing when the detection is made. The control module 510 can detect that objects are touching. For example, when the operating object touches another object, the control module 510 detects that the operating object touches the other object, and performs predetermined processing.

In one aspect, control module 510 controls image display on monitor 130 of HMD 120 . For example, control module 510 places virtual camera 14 in virtual space 11 . The control module 510 controls the position of the virtual camera 14 in the virtual space 11 and the tilt (orientation) of the virtual camera 14 . The control module 510 defines the field of view area 15 according to the tilt of the head of the user 5 wearing the HMD 120 and the position of the virtual camera 14 . Rendering module 520 generates view image 17 displayed on monitor 130 based on determined view area 15 . The field-of-view image 17 generated by the rendering module 520 is output to the HMD 120 by the communication control module 540 .

When the control module 510 detects an utterance by the user 5 using the microphone 170 from the HMD 120, the control module 510 specifies the computer 200 to which the audio data corresponding to the utterance is to be transmitted. The audio data is sent to the computer 200 specified by control module 510 . When the control module 510 receives voice data from another user's computer 200 via the network 2 , the control module 510 outputs voice (utterance) corresponding to the voice data from the speaker 180 .

Memory module 530 holds data used by computer 200 to provide virtual space 11 to user 5 . In one aspect, memory module 530 holds spatial information, object information, and user information.

The spatial information holds one or more templates defined to provide the virtual space 11. FIG.

The object information includes a plurality of panoramic images 13 forming the virtual space 11 and object data for arranging objects in the virtual space 11 . Panorama image 13 may include still images and moving images. The panorama image 13 may include an image of unreal space and an image of real space. Images of unreal space include, for example, images generated by computer graphics.

User information holds a user ID that identifies the user 5 . The user ID can be, for example, an IP (Internet Protocol) address or a MAC (Media Access Control) address set in the computer 200 used by the user. In another aspect, the user ID can be set by the user. The user information includes programs and the like for causing the computer 200 to function as a control device for the HMD system 100 .

Data and programs stored in memory module 530 are input by user 5 of HMD 120 . Alternatively, processor 210 downloads a program or data from a computer (for example, server 600 ) operated by a content provider, and stores the downloaded program or data in memory module 530 .

Communication control module 540 can communicate with server 600 and other information communication devices via network 2 .

In one aspect, control module 510 and rendering module 520 may be implemented using, for example, Unity® provided by Unity Technologies. In another aspect, the control module 510 and the rendering module 520 can also be implemented as a combination of circuit elements that implement each process.

Processing in computer 200 is realized by hardware and software executed by processor 210 . Such software may be pre-stored on a hard disk or other memory module 530 . The software may be distributed as a program product stored in a CD-ROM or other computer-readable non-volatile data recording medium. Alternatively, the software may be provided as a downloadable program product by an information provider connected to the Internet or other network. Such software is read from a data recording medium by an optical disk drive or other data reading device, or downloaded from the server 600 or other computer via the communication control module 540, and then temporarily stored in the storage module. . The software is read from the storage module by processor 210 and stored in RAM in the form of executable programs. Processor 210 executes the program.

[Control Structure of HMD System]
The control structure of the HMD set 110 will be described with reference to FIG. FIG. 11 is a sequence chart representing part of the processing performed in HMD set 110 according to one embodiment.

As shown in FIG. 11, in step S1110, processor 210 of computer 200, as control module 510, specifies virtual space data and defines virtual space 11. As shown in FIG.

At step S1120, processor 210 initializes virtual camera . For example, the processor 210 places the virtual camera 14 at the predefined center 12 in the virtual space 11 in the work area of the memory, and directs the line of sight of the virtual camera 14 in the direction the user 5 is facing.

In step S1130, processor 210, as rendering module 520, generates view image data for displaying an initial view image. The generated visual field image data is output to HMD 120 by communication control module 540 .

In step S<b>1132 , monitor 130 of HMD 120 displays a field-of-view image based on the field-of-view image data received from computer 200 . The user 5 wearing the HMD 120 can recognize the virtual space 11 by viewing the visual field image.

In step S<b>1134 , HMD sensor 410 detects the position and tilt of HMD 120 based on a plurality of infrared rays emitted from HMD 120 . The detection result is output to the computer 200 as motion detection data.

In step S<b>1140 , processor 210 identifies the viewing direction of user 5 wearing HMD 120 based on the position and tilt included in the motion detection data of HMD 120 .

In step S1150, processor 210 executes the application program and places objects in virtual space 11 based on instructions included in the application program.

In step S<b>1160 , controller 300 detects the operation of user 5 based on the signal output from motion sensor 420 and outputs detection data representing the detected operation to computer 200 . In another aspect, manipulation of controller 300 by user 5 may be detected based on images from cameras placed around user 5 .

In step S<b>1170 , processor 210 detects an operation of controller 300 by user 5 based on the detection data acquired from controller 300 .

In step S<b>1180 , processor 210 generates view image data based on the operation of controller 300 by user 5 . The generated visual field image data is output to HMD 120 by communication control module 540 .

In step S<b>1190 , HMD 120 updates the field-of-view image based on the received field-of-view image data, and displays the updated field-of-view image on monitor 130 .

[Avatar object]
An avatar object according to this embodiment will be described with reference to FIGS. Hereafter, it is a figure explaining the avatar object of each user 5 of HMD set 110A, 110B. Hereinafter, the user of the HMD set 110A will be referred to as the user 5A, the user of the HMD set 110B as the user 5B, the user of the HMD set 110C as the user 5C, and the user of the HMD set 110D as the user 5D. A is attached to the reference sign of each component regarding the HMD set 110A, B is attached to the reference sign of each component regarding the HMD set 110B, C is attached to the reference sign of each component regarding the HMD set 110C, and the HMD set A D is attached to the reference number of each component related to 110D. For example, HMD 120A is included in HMD set 110A.

FIG. 12A is a schematic diagram showing a situation in which each HMD 120 provides the virtual space 11 to the user 5 in the network 2. FIG. Computers 200A-200D provide users 5A-5D with virtual spaces 11A-11D via HMDs 120A-120D, respectively. In the example shown in FIG. 12A, virtual space 11A and virtual space 11B are configured with the same data. In other words, the computers 200A and 200B share the same virtual space. An avatar object 6A of the user 5A and an avatar object 6B of the user 5B exist in the virtual space 11A and the virtual space 11B. Although the avatar object 6A in the virtual space 11A and the avatar object 6B in the virtual space 11B are each wearing the HMD 120, this is for the sake of clarity of explanation, and in reality these objects are not wearing the HMDs 120. not

In one aspect, processor 210A may position virtual camera 14A capturing view image 17A of user 5A at the eye position of avatar object 6A.

FIG. 12(B) is a diagram showing a field-of-view image 17A of the user 5A in FIG. 12(A). The field-of-view image 17A is an image displayed on the monitor 130A of the HMD 120A. This field-of-view image 17A is an image generated by the virtual camera 14A. An avatar object 6B of the user 5B is displayed in the field-of-view image 17A. Although not particularly illustrated, the avatar object 6A of the user 5A is similarly displayed in the visual field image of the user 5B.

In the state of FIG. 12B, the user 5A can communicate with the user 5B through dialogue through the virtual space 11A. More specifically, the voice of user 5A acquired by microphone 170A is transmitted to HMD 120B of user 5B via server 600 and output from speaker 180B provided in HMD 120B. The voice of user 5B is transmitted to HMD 120A of user 5A via server 600, and is output from speaker 180A provided in HMD 120A.

Actions of the user 5B (actions of the HMD 120B and actions of the controller 300B) are reflected in the avatar object 6B arranged in the virtual space 11A by the processor 210A. Thereby, the user 5A can recognize the action of the user 5B through the avatar object 6B.

FIG. 13 is a sequence chart showing part of the processing executed in HMD system 100 according to the present embodiment. Although the HMD set 110D is not shown in FIG. 13, the HMD set 110D operates similarly to the HMD sets 110A, 110B, and 110C. In the following description, A is attached to the reference sign of each component regarding the HMD set 110A, B is attached to the reference sign of each component regarding the HMD set 110B, and C is attached to the reference sign of each component regarding the HMD set 110C. , and D is added to the reference numeral of each component of the HMD set 110D.

In step S1310A, the processor 210A in the HMD set 110A acquires avatar information for determining the motion of the avatar object 6A in the virtual space 11A. This avatar information includes, for example, information about the avatar such as motion information, face tracking data, and voice data. The movement information includes information indicating temporal changes in the position and tilt of the HMD 120A, information indicating hand movements of the user 5A detected by the motion sensor 420A, and the like. The face tracking data includes data specifying the position and size of each part of the user 5A's face. The face tracking data includes data indicating the movement of each organ that constitutes the face of the user 5A and line-of-sight data. The audio data includes data indicating the audio of the user 5A acquired by the microphone 170A of the HMD 120A. The avatar information may include information specifying the avatar object 6A or the user 5A associated with the avatar object 6A, information specifying the virtual space 11A in which the avatar object 6A exists, and the like. A user ID is mentioned as information which specifies the avatar object 6A and the user 5A. Room ID is mentioned as information which specifies virtual space 11A in which avatar object 6A exists. Processor 210A transmits the avatar information obtained as described above to server 600 via network 2 .

In step S1310B, processor 210B in HMD set 110B acquires avatar information for determining the motion of avatar object 6B in virtual space 11B and transmits it to server 600, as in the process in step S1310A. Similarly, in step S1310C, processor 210C in HMD set 110C acquires avatar information for determining the motion of avatar object 6C in virtual space 11C and transmits it to server 600. FIG.

In step S1320, server 600 temporarily stores the player information received from each of HMD set 110A, HMD set 110B, and HMD set 110C. Server 600 integrates avatar information of all users (users 5A to 5C in this example) associated with common virtual space 11 based on user IDs, room IDs, and the like included in each avatar information. Then, the server 600 transmits the integrated avatar information to all users associated with the virtual space 11 at a predetermined timing. Accordingly, synchronization processing is executed. Such synchronization processing enables the HMD set 110A, the HMD set 110B, and the HMD set 110C to share each other's avatar information at approximately the same timing.

Subsequently, based on the avatar information transmitted from server 600 to each HMD set 110A-110C, each HMD set 110A-110C executes the process of steps S1330A-S1330C. The processing of step S1330A corresponds to the processing of step S1180 in FIG.

In step S1330A, the processor 210A in the HMD set 110A updates information on the avatar objects 6B and 6C of the other users 5B and 5C in the virtual space 11A. Specifically, the processor 210A updates the position, orientation, etc. of the avatar object 6B in the virtual space 11 based on the motion information included in the avatar information transmitted from the HMD set 110B. For example, processor 210A updates the information (such as position and orientation) of avatar object 6B contained in the object information stored in memory module 530. FIG. Similarly, the processor 210A updates the information (position, orientation, etc.) of the avatar object 6C in the virtual space 11 based on the motion information included in the avatar information transmitted from the HMD set 110C.

In step S1330B, processor 210B in HMD set 110B updates information on avatar objects 6A and 6C of users 5A and 5C in virtual space 11B, similar to the process in step S1330A. Similarly, in step S1330C, the processor 210C in the HMD set 110C updates information on the avatar objects 6A, 6B of the users 5A, 5B in the virtual space 11C.

[Detailed module configuration]
Details of the module configuration of the computer 200 will be described with reference to FIG. FIG. 14 is a block diagram showing the detailed configuration of the modules of computer 200 according to one embodiment. As shown in FIG. 14, the control module 510 includes a virtual object generation module 1421, a virtual camera control module 1422, a manipulated object control module 1423, an avatar object control module 1424, a motion detection module 1425, a collision detection module 1426, a virtual object control module 1421, a It has a module 1427 , a billing processing module 1428 , and a payment processing module 1429 .

A virtual object generation module 1421 generates various virtual objects in the virtual space 11 . In one aspect, the virtual objects may include, for example, landscapes including forests, mountains, etc., animals, etc. arranged according to the progress of the story of the game. In one aspect, the virtual object may include an avatar object, an operation object, a stage object, and a UI (User Interface) object.

The virtual camera control module 1422 controls behavior of the virtual camera 14 in the virtual space 11 . The virtual camera control module 1422 controls, for example, the arrangement position of the virtual camera 14 in the virtual space 11 and the orientation (inclination) of the virtual camera 14 .

The manipulation object control module 1423 controls manipulation objects for receiving manipulations by the user 5 in the virtual space 11 . The user 5 operates, for example, a virtual object placed in the virtual space 11 by operating the operation object. In one aspect, the operation object may include, for example, a hand object (virtual hand) corresponding to the hand of user 5 wearing HMD 120 . In one aspect, the manipulation object can correspond to a hand portion of an avatar object, which will be described later.

Avatar object control module 1424 reflects the movement of HMD 120 detected by HMD sensor 410 on the avatar object. For example, the avatar object control module 1424 detects that the HMD 120 is tilted and generates data for tilting and arranging the avatar object. In one aspect, the avatar object control module 1424 reflects movements of the controller 300 on the avatar object. In this case, controller 300 includes a motion sensor, an acceleration sensor, a plurality of light emitting elements (for example, infrared LEDs), or the like for detecting movement of controller 300 . The avatar object control module 1424 reflects the motion of facial organs detected by the motion detection module 1425 on the face of the avatar object placed in the virtual space 11 . That is, the avatar object control module 1424 reflects the motion of the user 5's face on the avatar object.

Motion detection module 1425 detects motion of user 5 . The motion detection module 1425 detects motion of the hand of the user 5 according to the output of the controller 300, for example. The motion detection module 1425 detects body motion of the user 5 according to the output of a motion sensor attached to the body of the user 5, for example. The motion detection module 1425 can also detect motion of the user's 5 facial features.

The collision detection module 1426 detects a collision when each virtual object placed in the virtual space 11 collides with another virtual object. Collision detection module 1426 may, for example, detect when one virtual object touches another virtual object. Collision detection module 1426 can detect when one virtual object is out of touch with another virtual object. Collision detection module 1426 can also detect when one virtual object is in contact with another virtual object. The collision detection module 1426 detects that the operating object touches another virtual object, for example, when the operating object touches another virtual object. Collision detection module 1426 performs predetermined processing based on these detection results.

The virtual object control module 1427 controls behavior of virtual objects other than avatar objects in the virtual space 11 . As an example, virtual object control module 1427 deforms the virtual object. As another example, the virtual object control module 1427 changes the placement position of the virtual object. As another example, virtual object control module 1427 moves virtual objects.

Billing processing module 1428 executes processing related to billing.

The payment processing module 1429 executes processing related to payment.

[Viewer's virtual space]
FIG. 15 is a diagram showing virtual space 11A and view image 1517A according to one embodiment. In FIG. 15A, at least avatar objects 6A to 6D, a virtual camera 14A, and a stage object 1532 are placed in a virtual space 11A for providing a virtual experience to a user 5A (first user). User 5A wears HMD 120A on the head. The user 5A holds the right controller 300RA with the right hand (first part) forming part of the right side of user 5A's body, and holds the left controller 300RA with the left hand (second part) forming part of the left side of user 5A's body. It holds the controller 300LA. Avatar object 6A (first avatar) includes virtual right hand 1531RA and virtual left hand 1531LA. The virtual right hand 1531RA is a kind of operation object, and can move in the virtual space 11A according to the movement of the right hand of the user 5A. The virtual left hand 1531LA is a kind of operation object, and can move in the virtual space 11A according to the movement of the left hand of the user 5A.

A virtual space 11A shown in FIG. 15A is constructed by playing live content on computer 200A. In virtual space 11A, avatar object 6B performs a performance as a live performer, and other avatar objects, including avatar object 6A, watch the performance as live viewers. In the virtual space 11A, avatar objects 6A-6D are individually associated with users 5A-5D, respectively.

The avatar object 6B (second avatar) is placed on the stage object 1532 in the virtual space 11A. The stage object 1532 has an appearance that imitates a stage in a real live venue. Avatar objects 6A, 6C, and 6D are all placed in front of stage object 1532 . In the virtual space 11A, the avatar object 6B performs a live performance by moving according to the movement of the user 5B (second user). In the virtual space 11A, the avatar object 6A watches the performance by the avatar object 6B. At this time, the avatar object 6C watches the performance performed by the avatar object 6B in the virtual space 11C provided to the user 5C. Similarly, in virtual space 11D provided to user 5D, avatar object 6D watches the performance performed by avatar object 6B. Therefore, it can be said that the user 5B is the performer and the users 5A, 5C and 5D are the viewers.

In FIG. 15A, the virtual camera 14A is arranged on the head of the avatar object 6A. The virtual camera 14A defines a viewing area 15A according to the position and orientation of the virtual camera 14A. The virtual camera 14A generates a field-of-view image 1517A corresponding to the field-of-view area 15A, and displays it on the HMD 120A as shown in FIG. 15(B). The user 5A visually recognizes a part of the virtual space from the viewpoint of the avatar object 6A by visually recognizing the view image 1517A. Thereby, the user 5A can obtain a virtual experience as if the user 5A himself were the avatar object 6A. View image 1517A includes avatar object 6B performing a performance. Therefore, the user 5A can view the performance by the avatar object 6B from the viewpoint of the avatar object 6A.

A plurality of different avatar objects 6B can also be arranged in the virtual space 11A. In one aspect, different users 5 are associated with the plurality of avatar objects 6B. In another aspect, the same user 5B is associated with multiple avatar objects 6B.

[Performer's virtual space]
FIG. 16 is a diagram showing virtual space 11B and view image 1617B according to one embodiment. In FIG. 16A, at least avatar objects 6A-6D, a virtual camera 14B, and a stage object 1532 are placed in a virtual space 11B for providing a virtual experience to a user 5B. User 5B wears HMD 120B on the head. The user 5B holds the right controller 300RB with the right hand (first part) forming part of the right side of the body of the user 5B, and holds the right controller 300RB with the left hand (second part) forming part of the left side of the body of the user 5B. It holds the controller 300LB.

HMD 120B includes sensor 190 that functions as a motion sensor. Right controller 300RB and left controller 300LB include motion sensors 420 . User 5B also wears motion sensors 1641-1643. A motion sensor 1641 is attached to the waist of the user 5B by a belt 1644. FIG. The motion sensor 1642 is attached to the instep of the right foot of the user 5B. The motion sensor 1643 is attached to the instep of the user 5B's left foot. Motion sensors 1641-1643 are wired or wirelessly connected to computer 200B.

In one aspect, a motion sensor worn by user 5B detects the arrival time and angle of a signal (eg, an infrared laser) emitted from a base station (not shown). A processor 210B (hereinafter simply processor 210B) of computer 200B detects the position of the motion sensor with respect to the base station based on the detection result of the motion sensor. Processor 210B may also normalize the positions of the motion sensors relative to the base station relative to a predetermined point (eg, the position of head-mounted sensor 190).

Avatar object 6B includes virtual right hand 1531RB and virtual left hand 1531LB. The virtual right hand 1531RB is a kind of operation object, and can move in the virtual space 11B according to the movement of the right hand of the user 5B. The virtual left hand 1531LA is a kind of operation object, and can move in the virtual space 11B according to the movement of the left hand of the user 5B.

A virtual space 11B shown in FIG. 16A is constructed by playing live content on a computer 200B. The virtual spaces 11A and 11B are synchronized with each other under the control of the server 600. FIG. The user 5B moves his/her own body to cause the avatar object 6B to perform the performance. The computer 200B detects the movement of the user 5B based on the outputs of various motion sensors attached to the user 5B. In the virtual space 11B, the avatar object 6B performs a performance reflecting the movement of the user 5B in the real space according to the specified movement of the user 5B. In virtual space 11B, avatar objects 6A, 6C, and 6D watch a performance by avatar object 6B. When the avatar object 6B performs a performance according to the movement of the user 5B in the virtual space 11B, the avatar object 6B also performs the same performance in the virtual spaces 11A, 11C, and 11D in synchronization therewith. In this way, user 5B has a role as a distributor who distributes a live performance by avatar object 6B to users 5A, 5C, and 5D, respectively.

In FIG. 16A, the virtual camera 14B is placed on the head of the avatar object 6B. The virtual camera 14B defines a viewing area 15B according to the position and orientation of the virtual camera 14B. The virtual camera 14B generates a field-of-view image 1617B corresponding to the field-of-view area 15B, and displays it on the HMD 120B as shown in FIG. 16(B). The user 5B visually recognizes a part of the virtual space from the viewpoint of the avatar object 6B by visually recognizing the view image 1617B. Thereby, the user 5B can obtain a virtual experience as if the user 5B himself were the avatar object 6B. View image 1617B includes avatar objects 6A, 6C, and 6D viewing the performance. Therefore, the user 5B can grasp the state of the avatar objects 6A, 6C, and 6D watching the performance by the avatar object 6B from the viewpoint of the avatar object 6B.

[Live streaming flow]
FIG. 17 is a sequence diagram showing an example of processing executed in the HMD system 100. As shown in FIG. A series of processes for distributing the live performance of the avatar object 6B performed in the virtual space 11B from the computer 200B to the computer 200A will be described below. Avatar object 6B is also distributed live to computers 200C and 200D based on a similar series of processing.

In step S1701, processor 210B detects the positions of user 5B's head, waist, hands, and feet from the motion sensors worn by user 5B. Hereinafter, the positions of the parts of the user 5B detected by each motion sensor are also referred to as "positional information". In step S1702, the processor 210B calculates the rotational directions of the joints of the user 5B based on the current position information of the user 5B and the pre-acquired dimension data of the user 5B. The dimension data is data representing the body dimension of the user 5B. Dimensional data and the direction of rotation will be described later. Detecting the current position information, calculating the direction of rotation, and detecting the movement of the user 5B are synonymous.

At step S1703, the processor 210B moves the avatar object 6B placed in the virtual space 11B based on the current position information and the rotation direction. Processor 210B moves the right upper arm of avatar object 6B, for example, based on the rotation direction of the right shoulder. The processor 210B further moves the position of the avatar object 6B in the virtual space 11B based on the current position information (for example, current waist position information). Thereby, the processor 210B reflects the movement of the user 5B in the real space on the avatar object 6B in the virtual space. In other words, the processor 210B causes the avatar object 6B to perform a performance according to the movements of the user 5B.

The processing for reflecting the movement of the user 5B on the avatar object 6B is not limited to the processing according to the above-described positional information and rotation direction. Processor 210B may, for example, move avatar object 6B in response to movement of user 5B without calculating the direction of rotation. For example, the processor 210B may control the position of each part object of the avatar object 6B corresponding to each part of the user 5B so as to correspond to the position of each part constituting the body of the user 5B.

In step S<b>1704 , processor 210</b>B generates motion information representing the motion of avatar object 6</b>B when performing the performance, and transmits avatar information of avatar object 6</b>B including this motion information to server 600 .

In step S1710, processor 210A of computer 200A (hereinafter simply processor 210A) transmits avatar information of avatar object 6A to the server. In step S1720, server 600 executes synchronization processing for synchronizing virtual spaces 11A and 11B. Specifically, server 600 transmits the avatar information of avatar object 6B received from computer 200B to computer 200A. Server 600 further transmits the avatar information of avatar object 6A received from computer 200A to computer 200B.

At step S<b>1705 , the processor 210</b>B receives the avatar information of the avatar object 6</b>A transmitted from the server 600 . At step S1706, the processor 210B controls the avatar object in the virtual space 11B based on the received avatar information. Thereby, the behavior of the avatar object 6A in the virtual space 11A is reflected in the avatar object 6A in the virtual space 11B. In other words, the behavior of avatar object 6A is synchronized in virtual spaces 11A and 11B. For example, when the avatar object 6A moves in the virtual space 11A, the avatar object 6A also moves in the virtual space 11B.

At step S1711, the processor 210A receives the avatar information of the avatar object 6B transmitted from the server 600. FIG. At step S1712, the processor 210A moves the avatar object 6B based on the motion information included in the received avatar information. Thereby, the processor 210B reflects the movement of the user 5B in the physical space on the avatar object 6B arranged in the virtual space 11A. In other words, the processor 210A causes the avatar object 6B to perform a performance according to the movement of the user 5B in the virtual space 11A. Thereby, the behavior of the avatar object 6B in the virtual space 11B is reflected in the avatar object 6B in the virtual space 11A. In other words, the behavior of avatar object 6B is synchronized in virtual spaces 11A and 11B. For example, when the avatar object 6B performs the first performance in the virtual space 11B, the avatar object 6B similarly performs the first performance in the virtual space 11A. In this way, the live performance of the avatar object 6B in the virtual space 11B is delivered to the virtual space 11A.

Although not shown, the processor 210B records the voice uttered by the user 5B using the microphone 170B. The processor 210B generates voice data representing the voice of the user 5B and transmits it to the server 600. FIG. The server 600 transmits the received voice data of the user 5B to the computer 200A by synchronous processing. The processor 210A outputs the voice represented by the received voice data of the user 5B to the speaker 180A. As a result of these series of processes, the user 5A can listen in real time to the voice uttered by the user 5B during the live performance.

[Acquisition of dimension data]
FIG. 18 is a diagram for explaining a method of acquiring dimension data. FIG. 18A shows a state in which the user 5B faces the front, spreads both hands horizontally, and stands up. Hereinafter, the state shown in FIG. 18A is also referred to as the first posture. FIG. 18B shows a state in which the user 5B faces the front, puts both hands on the sides of the thighs, and stands up. Hereinafter, the state shown in FIG. 18B is also referred to as the second posture.

In one aspect, processor 210B prompts user 5B to take a first pose and a second pose. As an example, processor 210B displays the characters in the first and second postures on monitor 130B and displays a message to the effect that they will take similar postures. As another example, the processor 210B may output a voice indicating that the first posture and the second posture are taken from the speaker 180B.

The processor 210B acquires position information of the head, waist, hands, and feet of the user 5B based on the outputs of the motion sensors worn by the user 5B in each of the two postures (the first posture and the second posture). . These position information can be obtained as positions in the real coordinate system (x, y, z) as shown in FIG.

The processor 210B calculates the dimension data of the user 5B from the position information corresponding to the two postures. In one embodiment, the processor 210B calculates the height, shoulder width, arm length, leg length, and head-to-shoulder height of the user 5B as dimension data, as shown in FIG. Processor 210B may calculate the distance between both hands in the second posture as the shoulder width. The processor 210B may calculate half the distance between the hands in the first posture minus the shoulder width as the arm length. The processor 210B may calculate the height as the distance from the height of the feet to the height of the head. Processor 210B may calculate the distance from the height of the legs to the height of the hips as the length of the legs. The processor 210B may calculate the height from the hand height to the head in the first posture as the height from the head to the shoulder.

FIG. 21 is a flowchart showing processing for acquiring dimension data. In step S2110, processor 210B arranges virtual camera 14B in virtual space 11B. Processor 210B further outputs field-of-view image 17B corresponding to the imaging range of virtual camera 14B to monitor 130B.

At step S2120, the processor 210B instructs the user 5B to take the first posture. For example, the processor 210B implements the process of step S2120 by arranging the object on which the instruction is written in the virtual space 11B. At step S2130, the processor 210B obtains position information corresponding to the first pose.

At step S2140, the processor 210B instructs the user 5B to take the second posture. At step S2150, the processor 210B obtains position information corresponding to the second pose.

In step S2160, processor 210B calculates dimension data of user 5B from the position information corresponding to the first posture and the position information corresponding to the second posture. Processor 210B stores the dimension data in storage 230B.

As described above, the user 5B can easily input his/her own dimensions to the computer 200B simply by taking two postures. In another aspect, user 5B may input his/her own dimensions into computer 200B using an input device such as a keyboard.

[Direction of joint rotation]
In one embodiment, the processor 210B estimates the rotational direction of the joints of the user 5B based on the outputs (positional information) of the six motion sensors attached to the user 5B and the dimensional data. As an example, the processor 210B estimates the position of the shoulder based on the head position information, the shoulder width, and the height from the head to the shoulder. Processor 210B estimates the elbow position from the shoulder position and hand position information. This estimation can be performed by known applications using Inverse Kinematics.

In one embodiment, the processor 210B acquires the joint tilts (rotational directions) of the neck (head), waist, both wrists, and both ankles of the user 5B from six motion sensors. In addition, the processor 210B estimates the rotational directions of the shoulder, elbow, hip, and knee joints based on inverse kinematics. As shown in FIG. 22, the processor 210B obtains or estimates the rotation direction of each joint in the uvw field coordinate system.

Note that when the rotation direction is calculated based on the position information and the dimension data, the processor 210B calculates the rotation direction when the user 5B is not facing the front (that is, when the head and waist are facing in different directions). The position of the shoulder, etc. cannot be estimated accurately. Therefore, in another embodiment, the computer 200B may further consider the inclination of the part of the user 5B detected by the motion sensor to estimate the rotational direction of the joint. For example, the computer 200B estimates the positions of the shoulders based on the position information of the head, the tilt of the head, the tilt of the waist, the shoulder width, and the height from the head to the shoulders. According to this configuration, the computer 200B can improve the accuracy of the rotational direction of the joint.

[Live venue selection processing flow]
FIG. 23 is a sequence chart representing part of the processing performed in HMD set 110A according to an embodiment. FIG. 24 is a diagram showing first virtual space 2411A and view image 2417A according to one embodiment. In this embodiment, it is assumed that the HMD set 110A executes a series of processes for selecting the virtual venue where the user 5A will watch the virtual live performance. However, the processing may be executed by the other HMD sets 110B, 110C, and 110D, or part or all of the processing may be executed by the server 600. FIG.

At step S2301, the processor 210A defines a first virtual space 2411A as shown in FIG. 24(A). This process corresponds to the process of step S1110 in FIG. Specifically, the processor 210A defines the first virtual space 2411A represented by the virtual space data by specifying the virtual space data. The first virtual space 2411A is a virtual space in which the avatar object 6A is arranged before the live performance by the avatar object 6B is started. The first virtual space 2411A is also a virtual space for allowing the user 5A to select a venue.

In step S2302, the processor 210A, as the virtual object generation module 1421, generates the virtual camera 14A and places it in the first virtual space 2411A. In step S2303, the processor 210A, as the virtual object generation module 1421, generates the avatar object 6A including the virtual right hand 1531RA and the virtual left hand 1531LA, and places it in the first virtual space 2411A. In step S2304, processor 210A, as virtual object generation module 1421, generates UI panel 2450 and pen object 2453, and places them in first virtual space 2411A.

UI panel 2450 is a type of UI object, and is used by user 5A to cause processor 210A to execute processing for selecting a venue. UI panel 2450 includes options 2451 and 2452 located in front of UI panel 2450 . Choices 2451 and 2452 include information describing the name of the venue that will be selected if that choice is selected by user 5A. Option 2451 is an item for selecting the first venue, and option 2452 is an item for selecting a second venue different from the first venue. Pen object 2453 is a type of virtual object, and is grasped and used by virtual right hand 1531RA or virtual left hand 1531LA to select option 2451 or 2452 on UI panel 2450 .

In step S2305, the processor 210A, as the virtual camera control module 1422, determines the position and tilt of the virtual camera 14A in the first virtual space 2411A according to the movement of the HMD 120A. More specifically, processor 210A adjusts the field of view, which is the field of view from virtual camera 14A in first virtual space 2411A, according to the posture of user 5A's head and the position of virtual camera 14A in first virtual space 2411A. 15A is controlled. This process corresponds to part of the process of step S1140 in FIG. Since the virtual camera 14A is arranged at the same position as the avatar object 6A, the position of the virtual camera 14A is synonymous with the position of the avatar object 6A. Furthermore, the field of view from the virtual camera 14A is synonymous with the field of view from the avatar object 6A.

In step S2306, processor 210A displays view image 17A on monitor 130A. Specifically, processor 210A generates visual field image 17A corresponding to visual field area 15A based on the movement of HMD 120A (that is, the position and tilt of virtual camera 14A) and the virtual space data defining first virtual space 2411A. Define Defining the field-of-view image 17A is synonymous with generating the field-of-view image 17A. Processor 210A further causes HMD 120A to display field-of-view image 17A by outputting field-of-view image 17A to monitor 130A of HMD 120A. This processing corresponds to the processing of steps S1180 and S1190 in FIG.

The processor 210A, for example, displays a view image 2417A corresponding to the first virtual space 2411A shown in FIG. 24(A) on the monitor 130A as shown in FIG. 24(B). User 5A recognizes that it is necessary to select option 2451 or option 2452 of UI panel 2450 by visually recognizing view image 2417A.

The processing of steps S2305 and S2306 described above (that is, updating of field-of-view image 17A in accordance with the movement of HMD 120A) is continuously and repeatedly performed while steps S2307 to S2310, which will be described later, are performed.

At step S2307, the processor 210A, as the motion detection module 1425, detects motion of the right hand of the user 5A based on the output of the right controller 300RA. In step S2308, the processor 210A, as the operating object control module 1423, moves the virtual right hand 1531RA in the first virtual space 2411A in accordance with the detected movement of the right hand of the user 5A. In one aspect, processor 210A moves virtual right hand 1531RA in first virtual space 2411A so as to bring virtual right hand 1531RA closer to pen object 2453 in response to movement of user 5A's right hand. After the virtual right hand 1531RA is sufficiently close to the pen object 2453, the processor 210A selects (grabs) the pen object 2453 with the virtual right hand 1531RA based on the movement of the right hand of the user 5A. The movement of the right hand of the user 5A includes, for example, the movement of pressing any button of the right controller 300RA. As a result, the pen object 2453 is gripped with the virtual right hand 1531RA.

FIG. 25 is a diagram showing first virtual space 2411A and view image 2517A according to one embodiment. As shown in FIG. 25A, after the virtual right hand 1531RA selects the pen object 2453, the processor 210A moves the tip of the pen object 2453 closer to the option 2451 based on the movement of the right hand of the user 5A. 1 Move the virtual right hand 1531RA and the pen object 2453 in the virtual space 2411A. Processor 210A, as collision detection module 1426, detects that the tip of pen object 2453 and option 2451 have collided when the tip of pen object 2453 and option 2451 are in the first positional relationship. The first positional relationship is, for example, that the distance between the tip of pen object 2453 and option 2451 is less than the first distance. Alternatively, the collision area defined at the tip of the pen object 2453 and the collision area set in the option 2451 at least partially collide. The processor 210A detects that the pen object 2453 has selected the option 2451 based on the collision between the tip of the pen object 2453 and the option 2451 . At step S2309, the processor 210A receives the selection of the first venue corresponding to the selected option 2451. The first venue is a virtual venue corresponding to the second virtual space 2711A, which is the environment in which the user 5A watches live performances.

The processor 210A, for example, displays a view image 2517A corresponding to the first virtual space 2411A shown in FIG. 25(A) on the monitor 130A as shown in FIG. 25(B). User 5A recognizes that he has selected option 2451 on UI panel 2450 (that is, he has selected the first venue) by visually recognizing view image 2517A.

At step S2310, the processor 210A, as the avatar object control module 1424, causes the avatar object 6A to leave the first virtual space 2411A. Specifically, the processor 210A finishes placing the avatar object 6A in the first virtual space 2411A. Thereafter, processor 210A continues definition of first virtual space 2411A. Further, the processor 210A leaves other virtual objects placed in the first virtual space 2411A even after the avatar object 6A is no longer placed. Processor 210A suspends provision of first virtual space 2411A to user 5A. Further, the processor 210A newly provides the user 5A with a second virtual space 2711A corresponding to the first venue by executing a series of processes for causing the avatar object 6A to view the performance by the avatar object 6B.

The venue selection method is not limited to the examples described above. The processor 210A arranges, for example, a spherical model that visualizes the contents of the venue in the first virtual space 2411A. Even when this spherical model is selected by virtual right hand 1531RA or virtual left hand 1531LA, processor 210A can newly provide user 5A with second virtual space 2711A corresponding to the venue indicated by the selected spherical model.

[Live Viewing Processing Flow]
FIG. 26 is a sequence chart representing part of the processing performed in HMD set 110A according to an embodiment. FIG. 27 is a diagram showing second virtual space 2711A and view image 2717A according to an embodiment. In this embodiment, it is assumed that a series of processes for starting live viewing are executed by the HMD set 110A. However, the processing may be executed by the other HMD sets 110B, 110C, and 110D, or part or all of the processing may be executed by the server 600. FIG.

At step S2601, the processor 210A defines a second virtual space 2711A as shown in FIG. 27(A). This process corresponds to the process of step S1110 in FIG. Specifically, processor 210A defines second virtual space 2711A represented by the virtual space data by specifying the virtual space data. The second virtual space 2711A is a virtual space different from the first virtual space 2411A, and is a virtual space in which the avatar object 6A watches the live performance of the avatar object 6B. In other words, the second virtual space 2711A is a virtual space in which the avatar object 6B performs.

In step S2602, processor 210A generates virtual camera 14A and places it in second virtual space 2711A. In step S2603, the processor 210A creates a stage object 1532 and places it in the second virtual space 2711A. At step S2604, processor 210A receives from server 600 each avatar information of avatar objects 6C and 6D. At step S2605, processor 210A arranges avatar objects 6C and 6D in second virtual space 2711A based on the received avatar information. The processor 210A also receives avatar information of other avatar objects 6 and places them in the second virtual space 2711A. A large number of other avatar objects 6 are arranged in the second virtual space 2711A, but only one of them is representatively illustrated in FIG. 27 for convenience of explanation. Henceforth, other avatar objects 6 will not be referred to unless particularly necessary.

At step S2606, the processor 210A, as the avatar object control module 1424, causes the avatar object 6A to enter the second virtual space 2711A. Specifically, the processor 210A places the avatar object 6A that has left the first virtual space 2411A in the second virtual space 2711A. In this way, allowing the avatar object 6A to enter the second virtual space 2711A is synonymous with placing the avatar object 6A in the second virtual space 2711A. Although not shown, the processor 210A generates avatar information of the avatar object 6A and transmits it to the server 600 at arbitrary timing.

At step S2607, the processor 210A, as the virtual object generation module 1421, generates a tag object 2733A and associates it with the avatar object 6A. Specifically, processor 210A places tag object 2733A in virtual left hand 1531LA of avatar object 6A. Tag object 2733A is a kind of virtual object. The tag object 2733A is a virtual object for managing billing information representing the billing amount to be billed to the user 5A who watches the live performance in the second virtual space 2711A.

The processor 210A sets the upper limit of the billing amount that can be billed to the user 5A during the live performance (hereinafter referred to as the upper limit billing amount). Processor 210A also registers, in tag object 2733A, maximum billing information representing the set maximum billing amount. Processor 210A can, for example, accept the input of the maximum billing amount by user 5A after placing tag object 2733A in virtual left hand 1531LA, and set the inputted maximum billing amount. For example, when the user 5B registers the user information in the HMD system 100, the processor 210A can preset the upper limit billing amount based on the credit card information or the like included in the user information. The processor 210A can also calculate the upper limit billing amount, for example, based on the user's past billing amount for live performances that the user 5A has viewed in the past. Processor 210A registers, in tag object 2733A, maximum billing information representing the set maximum billing amount.

In FIG. 27(A), the avatar object 6B is not arranged in the second virtual space 2711A. In other words, the live performance of avatar object 6B has not yet started in second virtual space 2711A. In step S2608, the processor 210A, as the virtual camera control module 1422, determines the position and tilt of the virtual camera 14A in the second virtual space 2711A according to the movement of the HMD 120A. Since this process is basically the same as the process in step S2305, detailed description will not be repeated. In step S2609, processor 210A displays view image 17A on monitor 130A. Since this process is basically the same as the process in step S2306, detailed description will not be repeated.

The processor 210A, for example, displays a view image 2717A corresponding to the second virtual space 2711A shown in FIG. 27(A) on the monitor 130A as shown in FIG. 27(B). User 5A recognizes that avatar object 6A has entered second virtual space 2711A by visually recognizing view image 2717A. The user 5A also recognizes that the live performance of the avatar object 6B has not yet started because the avatar object 6B is not placed on the stage object 1532 .

FIG. 28 is a diagram showing second virtual space 2711A and view image 2817A according to one embodiment. In step S2610, the processor 210A, as the billing processing module 1428, registers billing information representing the admission fee for the first venue corresponding to the second virtual space 2711A in the tag object 2733A. The admission fee for the first venue is the amount charged to the user 5A for viewing the live performance held in the second virtual space 2711A. The billing information registered here can also be said to be billing information according to the action of the avatar object 6A entering the second virtual space 2711A. Processor 210A visualizes billing information registered with tag object 2733A on tag object 2733A. Specifically, processor 210A displays the total billing amount represented by each piece of registered billing information (hereinafter, total billing amount) on the surface of tag object 2733A. In FIG. 28A, one billing information is registered in tag object 2733A. In this case, processor 210A displays the billing amount (500 yen) represented by the registered billing information as the total billing amount on tag object 2733A. Processor 210A further displays the maximum billing amount (30000 yen) represented by the maximum billing information on tag object 2733A.

After the billing information is registered, user 5A moves his/her left hand into his/her field of view as if checking the time on a wristwatch. Processor 210A detects the movement of user 5A's left hand based on the output of left controller 300LA. Processor 210A moves virtual left hand 1531LA into visual field area 15A as shown in FIG. 28(A) according to the movement of user 5A's left hand. The processor 210A, for example, displays a view image 2817A corresponding to the second virtual space 2711A shown in FIG. 28(A) on the monitor 130A as shown in FIG. 28(B). User 5A confirms the upper limit billing amount and total billing amount displayed in tag object 2733A by visually recognizing view image 2817A. In the example of FIG. 28B, the user recognizes that the maximum billing amount is 30000 yen and the current total billing amount is 500 yen. In other words, the user 5A understands that the entrance fee of 500 yen was charged to the user 5A as a result of the avatar object 6A entering the second virtual space 2711A.

[Start live]
FIG. 29 is a diagram showing second virtual space 2711B and view image 2917B according to one embodiment. After avatar object 6A enters second virtual space 2711A, processor 210B defines second virtual space 2711B shown in FIG. A second virtual space 2711B is a virtual space provided to the user 5B and synchronized with the second virtual space 2711A. The second virtual space 2711B is also a virtual space in which performances are performed by the avatar object 6B. Processor 210B arranges virtual camera 14B, avatar object 6B, and stage object 1532 in second virtual space 2711B. Processor 210B receives avatar information of avatar objects 6A, 6C, and 6D from server 600, and arranges avatar objects 6A, 6C, and 6D in second virtual space 2711B based on this avatar information.

The processor 210B, for example, displays a view image 2917B corresponding to the second virtual space 2711B shown in FIG. 29(A) on the monitor 130B as shown in FIG. 29(B). User 5B recognizes that live viewers avatar objects 6A, 6C, and 6D are in front of stage object 1532 by visually recognizing view image 2917B. After the avatar object 6B is placed on the stage object 1532, the live performance by the avatar object 6B is started. Thereafter, the user 5B makes the live performance of the avatar object 6B in the second virtual space 2711B progress by moving himself, emitting sounds, and operating the right controller 300RB and the left controller 300LB. The processor 210B transmits avatar information including movement information of the avatar object 6B to the server 600 in real time during the live performance.

FIG. 30 is a diagram showing second virtual space 2711A and view image 3017A according to an embodiment. After the second virtual space 2711B is defined, in step S2611 the processor 210A receives avatar information of the avatar object 6B from the server 600 in real time. In step S2612, processor 210A arranges avatar object 6B in second virtual space 2711A, as shown in FIG. 30A, based on the avatar information of avatar object 6B received first. Specifically, processor 210A places avatar object 6B on stage object 1532 .

The processor 210A, for example, displays a view image 3017A corresponding to the second virtual space 2711A shown in FIG. 30(A) on the monitor 130A as shown in FIG. 30(B). User 5A recognizes that avatar object 6B as a live performer has appeared on stage object 1532 by visually recognizing view image 3017A. The user 5B also recognizes that the live performance by the avatar object 6B is about to start.

FIG. 31 is a diagram showing an example of the posture of the user 5B according to an embodiment. After starting the live performance, the user 5B moves his or her body so as to take the posture shown in FIG. 31, for example. The posture shown in FIG. 31 is a posture corresponding to the first performance. The processor 210B causes the avatar object 6B to perform the first performance in the second virtual space 2711B according to the movement of the user 5B when taking the posture shown in FIG. Processor 210B transmits avatar information including movement information of avatar object 6B when avatar object 6B performs the first performance to server 600 in real time.

FIG. 32 is a diagram showing second virtual space 2711A and view image 3217A according to an embodiment. After the avatar object 6B performs the first performance in the second virtual space 2711B, the processor 210A receives avatar information of the avatar object 6B from the server 600 in real time in step S2613. At step S2614, the processor 210A causes the avatar object 6B to perform the first performance based on the motion information included in the received avatar information. Thereby, the processor 210A can cause the avatar object 6B to perform the first performance according to the movement of the user 5B shown in FIG.

In the example of FIG. 31, the avatar object 6B can move not only to the stage object 1532 but also to any position within the second virtual space 2711A to perform the first performance. Also, the avatar object 6A can view the first performance by the avatar object 6B at any position within the second virtual space 2711A. Therefore, it can be said that the area (first area) where the first performance is performed in the second virtual space 2711A is the entire second virtual space 2711A.

The processor 210A causes the avatar object 6A to view the first performance by the avatar object 6B in the second virtual space 2711A. The processor 210A, for example, displays a view image 3217A corresponding to the second virtual space 2711A shown in FIG. 32(A) on the monitor 130A as shown in FIG. 32(B). The user 5A recognizes that the avatar object 6B performed the first performance on the stage object 1532 by viewing the view image 3217A. Thereby, the user 5A can enjoy the performance by the avatar object 6B from the viewpoint of the avatar object 6A.

[Registration of billing information]
FIG. 33 is a diagram showing second virtual space 2711A and view image 3317A according to an embodiment. During the live, the processor 210A notifies the user 5A of a message guiding the user 5A to purchase the live video of the avatar object 6B taken live. Processor 210A, for example, displays the message in tag object 2733A. The message includes text guiding the user 5A to purchase the live video, and the billing amount required for the user 5B to purchase the live video. In the example of FIG. 33, the billing amount for purchasing the live video is 1000 yen.

In the second virtual space 2711A, a dedicated virtual camera (not shown) for shooting live video that can be purchased by the user 5A is arranged in the second virtual space 2711A. Processor 210A generates live video by capturing a portion of second virtual space 2711A with a dedicated virtual camera. Processor 210A can also shoot live video while moving a dedicated virtual camera within second virtual space 2711A.

After the message is displayed on tag object 2733A, user 5A moves his left hand into his field of view, as if checking the time on his wristwatch. The processor 210A moves the virtual left hand 1531LA into the visual field area 15A as shown in FIG. 33(A) according to the movement of the left hand of the user 5A. The processor 210A, for example, displays a view image 3317A corresponding to the second virtual space 2711A shown in FIG. 33(A) on the monitor 130A as shown in FIG. 33(B). User 5A confirms the message displayed on tag object 2733A by visually recognizing view image 3317A. Thereby, the user 5B recognizes that the live video can be purchased by paying a new charge of 1000 yen.

FIG. 34 is a diagram showing second virtual space 2711A and view image 3417A according to one embodiment. After confirming the message displayed on tag object 2733A, user 5A performs an operation to purchase the live video. User 5A moves his right hand so that the tip of virtual right hand 1531RA approaches tag object 2733A. The processor 210A moves the virtual right hand 1531RA in the second virtual space 2711A so that the tip of the virtual right hand 1531RA approaches the tag object 2733A according to the movement of the right hand of the user 5B.

Processor 210A, as collision detection module 1426, detects that the tip of virtual right hand 1531RA and tag object 2733A have collided when the tip of virtual right hand 1531RA and tag object 2733A are in the first positional relationship. The first positional relationship is, for example, that the distance between the tip of virtual right hand 1531RA and tag object 2733A is less than the first distance. Alternatively, the collision area defined at the tip of the virtual right hand 1531RA and the collision area set for the tag object 2733A at least partially collide. Processor 210A detects that tag object 2733A has been selected by virtual right hand 1531RA based on the collision between the tip of virtual right hand 1531RA and tag object 2733A. Processor 210A detects user 5A's operation for purchasing live video in response to selection of tag object 2733A displaying a message guiding purchase of live video.

The processor 210A, for example, displays a view image 3417A corresponding to the second virtual space 2711A shown in FIG. 34(A) on the monitor 130A as shown in FIG. 34(B). User 5A recognizes that tag object 2733A on which a message is displayed has been selected with virtual right hand 1531RA by visually recognizing view image 3417A. In the example of FIG. 34, the user 5A's operation for purchasing the live video is performed on the tag object 2733A, so the user 5A's sense of immersion in the second virtual space 2711A is not impaired.

FIG. 35 is a diagram showing second virtual space 2711A and view image 3517A according to an embodiment. In step S2615, the processor 210A, as the billing processing module 1428, registers billing information corresponding to the action of the avatar object 6B in the tag object 2733A. The action of the avatar object 6A referred to here means the action of the avatar object 6A that leads to charging the user 5B. In the example of FIG. 34, the action leading to billing for user 5B is avatar object 6A selecting tag object 2733A with virtual right hand 1531RA. Actions that lead to billing may also be actions that lead to billing in embodiments 2 to 8, which will be described later. Actions that lead to billing can also be actions of the avatar object 6A that lead to the purchase of virtual goods (for example, a virtual penlight) that can be used by the user 5A during a live performance. In the example of FIG. 34, processor 210A newly registers billing information representing the billing amount (1000 yen) required to purchase live video in tag object 2733A.

After the billing information is registered, processor 210A re-visualizes the billing information on tag object 2733A. Specifically, processor 210A clears the display of the message in tag object 2733A and displays the maximum charge amount and total charge amount in tag object 2733A. Specifically, processor 210A calculates the total (1,500 yen) of the two billing amounts (500 yen and 1,000 yen) represented by the two pieces of billing information registered in tag object 2733A, and stores the sum (1,500 yen) in tag object 2733A as the total billing amount. indicate. The processor 210A, for example, displays a view image 3517A corresponding to the second virtual space 2711A shown in FIG. 35(A) on the monitor 130A as shown in FIG. 35(B). User 5A recognizes that the current total billing amount has increased to 1500 yen by purchasing the live video by visually recognizing view image 3517A. Thus, the user 5A can confirm the total billing amount with a simple operation.

Processor 210A can also visualize the breakdown of not only the total billing amount on tag object 2733A. Processor 210A, for example, individually displays on tag object 2733A each billing amount represented by a plurality of pieces of billing information registered in tag object 2733A. The user 5A can confirm each billing amount by visually recognizing the tag object 2733A.

Processor 210A may also visualize billing information directly on view image 3517A without visualizing it on tag object 2733A. The processor 210A can display, on the monitor 130A, a notification UI such as a pop-up image superimposed on the view image 3517A, and display the total billing amount on the notification UI. Processor 210A may also display a pop-up image on monitor 130A in association with tag object 2733A on view image 3517A.

After registering the billing information corresponding to the live video purchase, every time the avatar object 6A takes a new action leading to billing in the second virtual space 2711A, the processor 210A stores the billing information corresponding to the action to the tag object 2733A. Register again. As the billing information registered in tag object 2733A increases, the total billing amount charged to user 5A further increases. User 5B can confirm the latest total billing amount by visually recognizing tag object 2733A at a desired timing.

[Settlement of total billed amount]
FIG. 36 is a diagram showing second virtual space 2711A and view image 3617A according to one embodiment. In FIG. 36(A), the live performance in the second virtual space 2711A has ended. Avatar objects 6B and 6C have left the second virtual space 2711A with the end of the live. Avatar objects 6A and 6D remain in the second virtual space 2711A after the live performance ends. Assume that the total billing amount charged to the user 5B at the end of the live performance is 12000 yen. In other words, the total billing amount represented by each piece of billing information registered in tag object 2733A is 12000 yen.

The processor 210A, for example, displays a view image 3617A corresponding to the second virtual space 2711A shown in FIG. 36(A) on the monitor 130A as shown in FIG. 36(B). User 5A recognizes that the live has ended by visually recognizing field-of-view image 3617A. User 5B further recognizes that the total billing amount charged to user 5A is 12,000 yen.

After confirming the total billing amount at the end of the live performance, the user 5A performs an operation for leaving the avatar object 6A from the second virtual space 2711A. This operation is, for example, the user 5A pressing any button of the right controller 300RA. In step S2616, the processor 210A causes the avatar object 6A to leave the second virtual space 2711A in response to detecting the user's 5A operation for leaving the avatar object 6A. Thereby, the processor 210A finishes providing the second virtual space 2711A to the user 5A.

FIG. 37 is a diagram showing first virtual space 2411A and view image 3717A according to one embodiment. In step S2617, the processor 210A causes the avatar object 6A that has left the second virtual space 2711A to enter the first virtual space 2411A again as shown in FIG. In other words, the processor 210A repositions the avatar object 6A in the first virtual space 2411A. At this time, processor 210A disassociates tag object 2733A from avatar object 6A. In other words, processor 210A does not place tag object 2733A in virtual left hand 1531LA in first virtual space 2411A.

In step S2618, processor 210A, as payment processing module 1429, executes processing related to payment according to the billing information registered in tag object 2733A. Processor 210A, for example, notifies server 600 of the total billing amount represented by each registered billing information. Server 600 settles the notified total billing amount. The server 600 executes credit card settlement of the total charge amount according to the credit card information of the user 5A registered in the HMD system 100, for example. As a result, the credit card issuer later bills the user 5A for the 12,000 yen charged to the user 5A. Another server different from the server 600 can also settle the total billing amount charged to the user 5A. Processor 210A can also settle the total billing amount charged to user 5A.

The processor 210A, for example, displays a view image 3717A corresponding to the first virtual space 2411A shown in FIG. 37(A) on the monitor 130A as shown in FIG. 37(B). Server 600 notifies computer 200A that the payment has been completed. In response to notification of the completion of payment from server 600, processor 210A notifies user 5A of the completion of payment on view image 3717A. Processor 210A, for example, superimposes notification image 3735 including a message explaining that payment has been completed on view image 3717A and displays it on monitor 130A. Processor 210A can also place a virtual object having the same appearance as notification image 3735 at any position within view area 15A in second virtual space 2711A. Processor 210A may alternatively include notification image 3735 at any location within view image 3717A when view image 3717A is generated. User 5B recognizes that avatar object 6A has been placed again in first virtual space 2411A after the end of the live performance by visually recognizing view image 3717A. Further, user 5A recognizes that settlement of the billing amount charged to user 5A during the live has been completed by visually recognizing notification image 3735 . Further, the user 5A will not be billed any more in the first virtual space 2411A according to the action of the avatar object 6A because the tag object 2733A is not arranged in the virtual left hand 7131LB in the first virtual space 2411A. to recognize

The processor 210A can also leave the tag object 2733A placed on the avatar object 6A after the avatar object 6A moves to the first virtual space 2411A. In this case, processor 210A displays a message on tag object 2733A to notify user 5A that the payment has been completed after the payment is completed. Processor 210A also displays an "accept" button on tag object 2733A. When avatar object 6A presses the "Approve" button, processor 210A disassociates tag object 2733A from avatar object 6A. As a result, the tag object 2733A is no longer placed on the avatar object 6A. Therefore, even if the user 5A moves the virtual left hand 1531LA into the viewing area 15A, the tag object 2733A cannot be visually recognized.

Processor 210B calculates live sales by totaling the total charges charged to each user 5 who is a viewer after the end of the live. Server 600 can also calculate sales and notify computer 200B. Processor 210B executes processing for paying user 5B a reward according to live sales. The processor 210B, for example, transmits request information requesting the server 600 to pay the reward to the server 600 after the live performance ends. The server 600 pays the user 5B as a reward at least part of the sales of the live show after completing the payment for each user 5 according to the received request information. The server 600 can pay a reward to the user 5B each time a live performance ends. Alternatively, the server 600 can collectively pay the user 5B a remuneration according to the total sales of each live performance held within a certain period of time (for example, one month) after a certain period of time has passed. The processor 210B, instead of the server 600, can also pay the user 5B a reward according to the sales.

[Main advantages of this embodiment]
As described above, processor 210A automatically registers billing information according to the behavior of avatar object 6A in second virtual space 2711A in association with avatar object 6A. Thus, the HMD system 100 can provide the user 5A with a suitable billing method in the second virtual space 2711A. Further, the processor 210A automatically registers billing information in the tag object 2733A without displaying on the monitor 130A a UI requesting the user 5A for input regarding billing superimposed on the view image 17A. As a result, the feeling of being immersed in the second virtual space 2711A by the user 5A can be maintained when the billing process is executed.

Further, the processor 210A automatically executes processing related to settlement when the avatar object 6A leaves the second virtual space 2711A. Thus, the HMD system 100 can provide the user 5A with a suitable payment method in the second virtual space 2711A. Further, the processor 210A automatically executes processing related to settlement without superimposing a UI requesting the user 5A for input related to settlement on the view image 17A and displaying it on the monitor 130A. As a result, the immersive feeling of the user 5A in the first virtual space 2411A is not impaired when the processing related to payment is executed.

Further, the processor 210A automatically executes processing related to payment without having the user 5A use a virtual object (virtual currency) for payment imitating a credit card or cash. As a result, the user 5A does not have to be bothered when the processing related to payment is executed.

[Warning to User 5A]
FIG. 38 is a diagram showing second virtual space 2711A and view image 3817A according to one embodiment. In the example of FIG. 38, the total billing amount is 25000 yen, which exceeds a certain percentage of the maximum billing amount. The fixed rate is, for example, 70% of the maximum billing amount (21,000 yen). Processor 210A warns user 5B that the total billing amount exceeds a certain percentage of the maximum billing amount. Processor 210A alerts user 5B by, for example, causing tag object 2733A to emit light, as shown in FIG. 38(A).

The processor 210A, for example, displays a view image 3817A corresponding to the second virtual space 2711A shown in FIG. 38(A) on the monitor 130A as shown in FIG. 38(B). User 5A recognizes that the current total billing amount is 25000 yen by visually recognizing view image 3817A. Further, user 5A recognizes that the warning that the total billing amount is approaching the maximum billing amount is given by visually recognizing the light emission of tag object 2733A. As a result, the user 5A can avoid the total billing amount exceeding the maximum billing amount without his knowledge.

After warning the user 5A, the processor 210A can increase the maximum billing amount in accordance with the operation of the user 5A. The user 5A inputs a new maximum billing amount into the tag object 2733A, for example, by operating the virtual right hand 1531RA. Processor 210A detects the entered new maximum billing amount. Processor 210A updates the maximum billing amount indicated by the maximum billing information registered in tag object 2733A to the detected new maximum billing amount.

FIG. 39 is a diagram showing second virtual space 2711A and view image 3917A according to one embodiment. In the example of FIG. 39, processor 210A controls the appearance of second virtual space 2711A according to the total billing amount. Specifically, the processor 210A arranges a monitor object 3936A for notifying the user 5A of the total billing amount in the second virtual space 2711A in association with the avatar object 6A. Processor 210A also generates total billing information representing the total billing amount and periodically transmits it to server 600 . Hereinafter, this total billing information will be referred to as total billing information of the avatar object 6A.

In the second virtual space 2711C (not shown), a tag object 2733C is placed on the virtual left hand 1531LC of the avatar object 6C. In the tag object 2733C, each billing information indicating each billing amount to be billed to the user 5C is registered. Processor 210C calculates the sum of each billing amount represented by each piece of billing information registered in tag object 2733C (total billing amount). Processor 210C also generates total billing information representing the total billing amount and periodically transmits it to server 600 . Hereinafter, this total billing information will be referred to as total billing information of the avatar object 6C.

In a second virtual space 2711D (not shown), a tag object 2733D is placed on the virtual left hand 1531LD of the avatar object 6D. In the tag object 2733D, each billing information indicating each billing amount to be billed to the user 5D is registered. Processor 210D calculates the sum of each billing amount represented by each piece of billing information registered in tag object 2733D (total billing amount). Processor 210D also generates total billing information representing the total billing amount and periodically transmits it to server 600 . Hereinafter, this total billing information will be referred to as total billing information of the avatar object 6D.

Processor 210A receives from server 600 total billing information for each of avatar objects 6C and 6D. The processor 210A controls the appearance of the avatar object 6C in the second virtual space 2711A according to the total billing amount represented by the total billing information of the avatar object 6C. Specifically, the processor 210A places a monitor object 3936C in which the total billing amount of the user 5C is visualized in the second virtual space 2711A in association with the avatar object 6C.

The processor 210A controls the appearance of the avatar object 6D in the second virtual space 2711A according to the total billing amount represented by the billing information. The processor 210A controls the appearance of the avatar object 6D in the second virtual space 2711A according to the total billing amount represented by the total billing information of the avatar object 6D. Specifically, the processor 210A places a monitor object 3936D in which the total billing amount of the user 5D is visualized in the second virtual space 2711A in association with the avatar object 6D.

A monitor object 3936A-3936C is placed on top of each of the avatar objects 6A-6C. Processor 210A controls the appearance of avatar object 6A by visualizing billing information for avatar object 6A on monitor object 3936A. Specifically, processor 210A displays user 5A's total billing amount in monitor object 3936A. Processor 210A controls the appearance of avatar object 6C by visualizing billing information for avatar object 6C on monitor object 3936C. Specifically, processor 210A displays the total billing amount for user 5C in monitor object 3936C. Processor 210A controls the appearance of avatar object 6D by visualizing billing information for avatar object 6D on monitor object 3936D. Specifically, processor 210D displays the total billing amount for user 5D in monitor object 3936D.

The processor 210A, for example, displays a view image 3917A corresponding to the second virtual space 2711A shown in FIG. 39(A) on the monitor 130A as shown in FIG. 39(B). User 5A recognizes that user 5C's current total billing amount is 20,000 yen and user 5D's current total billing amount is 5,000 yen by visually recognizing view image 3917A. Since the monitor object 3936A is not placed within the viewing area 15A, the user 5A does not visually recognize the user's 5A total billing amount displayed on the monitor object 3936A.

Since the monitor object 3936A displays the total billing amount of the user 5A, the larger the total billing amount, the higher the amount displayed in the monitor object 3936A. If the amount displayed in monitor object 3936A is low, the amount will not attract much attention from users 5B-5D. Therefore, avatar object 6A associated with monitor object 3936A is less noticeable to other users 5B-5D. The higher the amount displayed in monitor object 3936A, the more that amount attracts the attention of users 5B-5D. Therefore, avatar object 6A associated with monitor object 3936A appears more prominent to other users 5B-5D. In this way, processor 210A displays the total billing amount of user 5A on monitor object 3936A, thereby making the appearance of avatar object 6A more conspicuous as the total billing amount increases.

In view image 3917A, the total billing amount for user 5C (20,000 yen) displayed in association with avatar object 6C is greater than the total billing amount for user 5D (5,000 yen) displayed in association with avatar object 6D. Therefore, the user 5A recognizes that the appearance of the avatar object 6C is more conspicuous than the appearance of the avatar object 6D by viewing the view image 3917A.

FIG. 40 is a diagram showing second virtual space 2711B and view image 4017B according to an embodiment. Server 600 transmits the received total billing information of avatar objects 6A, 6C, and 6D to computer 200B by synchronous processing. Processor 210B receives total billing information for each of avatar objects 6A, 6C, and 6D transmitted from server 600. FIG. Processor 210B controls the appearance of each of avatar objects 6A, 6C and 6D in response to received total billing information for each of avatar objects 6A, 6C and 6D. Appearance control in this case is the same as the appearance control of each of avatar objects 6A, 6C, and 6D in second virtual space 2711A shown in FIG. 39, so detailed description will not be repeated.

The processor 210B, for example, displays a view image 4017B corresponding to the second virtual space 2711B shown in FIG. 40(A) on the monitor 130B as shown in FIG. 40(B). By visually recognizing the view image 4017B, the user 5B confirms that the current total billing amount for the user 5A is 7,000 yen, the current total billing amount for the user 5C is 20,000 yen, and the current total billing amount for the user 5D is 7,000 yen. is 5,000 yen. For user 5B, the appearance of avatar object 6C corresponding to the higher total billing amount appears more prominent than the appearances of avatar object 6A corresponding to the lower total billing amount and avatar object 6D associated with user 5D. . Thus, the user 5C can appeal to the user 5B through the appearance change of the avatar object 6C that the total billing amount of the user 5C is larger than that of the users 5B and 5D.

FIG. 41 is a diagram showing second virtual space 2711A and view image 4117A according to an embodiment. In the example of FIG. 41, the processor 210A controls the appearance colors of the avatar objects 6A, 6C and 6D according to the respective billing information of the avatar objects 6A, 6C and 6D. Specifically, the processor 210A applies textures 4137A, 4137C, and 4137D corresponding to each billing information to the avatar objects 6A, 6C, and 6D, thereby changing the appearance colors of the avatar objects 6A, 6C, and 6D. change.

Texture 4137D is a texture colored in the first color corresponding to the total billing amount exceeding the first threshold. Texture 4137A is a texture colored in a second color corresponding to the total billing amount exceeding the second threshold. Texture 4137C is a texture colored in a third color corresponding to the total billing amount exceeding the third threshold. The second color is a color that stands out more than the first color. The third color is a color that stands out more than the second color. The first color is for example copper, the second color is for example silver and the third color is for example gold.

The processor 210A, for example, displays a view image 4117A corresponding to the second virtual space 2711A shown in FIG. 41(A) on the monitor 130A as shown in FIG. 41(B). User 5A recognizes that the appearance of avatar object 6C has changed to the third color and the appearance of avatar object 6D has changed to the first color by visually recognizing view image 4117A. User 5A recognizes that the appearance of avatar object 6C is more conspicuous than the appearance of avatar object 6D by visually recognizing view image 3917A.

The processor 210A can also change the appearance of the avatar object 6A to look richer as the total billing amount increases. The processor 210A can place a bulletin board object (not shown) in the second virtual space 2711A and display the ranking of the total billing amount on the bulletin board object. The processor 210A displays, for example, the names of the top five users 5 with the highest total billing amounts on the bulletin board object. This makes it possible to make the avatar object 6 associated with the user 5 with the larger total billing amount more prominent in the second virtual space 2711A.

The processor 210A may also change some, but not all, of the appearance of the avatar object 6A. The processor 210A can also equip the avatar object 6A with accessories that can make the avatar object 6A look richer. Such accessories include, for example, gold-plated watches or necklaces.

[Watch live viewing]
FIG. 42 is a diagram showing second virtual space 4211B and live video 4239 according to one embodiment. Processor 210B arranges avatar objects 6A-6D, stage object 1532, and virtual camera 14B in second virtual space 4211B, as in second virtual space 2711B. Processor 210B further arranges virtual camera 4238 in second virtual space 4211B. The virtual camera 4238 is a virtual object that generates a live video image of the avatar object 6B live. The processor 210B places the virtual camera 4238 at a certain distance from the front of the avatar object 6B, for example. Processor 210B also controls the orientation of virtual camera 4238 to bring stage object 1532 and avatar object 6B within field of view 4215 of virtual camera 4238 .

The processor 210B, for example, controls the virtual camera 4238 so that the virtual camera 4238 captures a portion included in the field of view area 4215 in the second virtual space 4211B. Thereby, the processor 210B generates a live image 4239 corresponding to the field of view area 4215 as shown in FIG. 42(B). Live video 4239 is a kind of view image. A live video 4239 shows the avatar object 6B performing a performance on the stage object 1532 . Processor 210B distributes generated live video 4239 to each of the other computers 200, such as computer 200A, via server 600. FIG.

FIG. 43 is a diagram showing a second virtual space 4311A and a view image 4317A according to one embodiment. Processor 210A arranges avatar objects 6A, 6C, and 6D in second virtual space 4311A, similar to second virtual space 2711A. However, unlike the second virtual space 2711A, the processor 210A does not place the stage object 1532 and the avatar object 6B in the second virtual space 4311A. Instead, processor 210A places screen object 4340 in second virtual space 4311A.

Processor 210A receives live video 4239 delivered from computer 200B via server 600 . Processor 210A displays received live video 4239 on screen object 4340 . Thereby, the processor 210A causes the avatar object 6A to view the performance by the avatar object 6B displayed on the live video 4239. FIG. The processor 210A, for example, displays a view image 4317A corresponding to the second virtual space 4311A shown in FIG. 43(A) on the monitor 130A as shown in FIG. 43(B). User 5A visually recognizes live video 4239 displayed on screen object 4340 by visually recognizing view image 4317A. The live video 4239 shows the avatar object 6B performing on the stage object 1532 . By visually recognizing the live image 4239, the user 5A can obtain a virtual experience as if he/she were watching the live viewing by the avatar object 6B in the second virtual space 4311A.

Distribution of live viewing is not limited to the examples described above. For example, in a virtual space for live video shooting in which only the avatar object 6B, the stage object 1532, and the virtual camera 4238 are arranged, the live video of the avatar object 6B is shot, and the live video is placed in the second virtual space 4311A. Live video can also be distributed by a method of displaying on the screen object 4340 that is displayed.

The processor 210A can also register billing information according to the behavior of the avatar object 6A within the second virtual space 4311A in the tag object 2733A. For example, when the avatar object 6A takes an action leading to the purchase of the live video 4239 by the user 5A, the processor 210A registers billing information according to the action in the tag object 2733A.

[First area in virtual space]
FIG. 44 is a diagram showing second virtual space 4411A and view image 4417A according to an embodiment. Processor 210A arranges avatar objects 6A and 6C and stage object 1532 in second virtual space 4411A in the same manner as second virtual space 2711A. A second virtual space 4411A includes a first area 4451 . The first area 4451 corresponds to part of the second virtual space 4411A. The processor 210A arranges the gate object 4452 at one place on the outer edge of the first area 4451 .

In FIG. 44, the avatar object 6C and the stage object 1532 are placed within the first area 4451. In FIG. Avatar object 6A is placed outside first area 4451 . The first area 4451 is surrounded by non-illustrated opaque wall objects. A first area 4451 is a place where a live performance of the avatar object 6B is held. When the avatar object 6A is in the first area 4451, it can watch the performance by the avatar object 6B. When the avatar object 6A is outside the first area 4451, the avatar object 6B cannot view the performance. The avatar object 6A can enter or leave the first area 4451 by passing through the gate object 4452 .

After defining the second virtual space 4411A, the processor 210A arranges the avatar object 6A outside the first area 4451 in the second virtual space 4411A. Processor 210A does not place tag object 2733A in virtual left hand 1531LA while avatar object 6A is placed outside first area 4451 . The processor 210A moves the avatar object 6A from outside the first area 4451 into the first area 4451 through the gate object 4452 according to the operation of the user 5A. Thereby, the processor 210A causes the avatar object 6A to enter the first area 4451. FIG.

When the avatar object 6A enters the first area 4451, the processor 210A places the tag object 2733A on the virtual left hand 1531LA. Processor 210A registers billing information according to the action of avatar object 6A within first area 4451 in tag object 2733A. The processor 210A moves the avatar object 6A from inside the first area 4451 to outside the first area 4451 through the gate object 4452 according to the operation of the user 5A. Thereby, the processor 210A causes the avatar object 6A to leave the first area 4451.

When avatar object 6A leaves first area 4451, processor 210A executes processing related to payment according to billing information registered in tag object 2733A. Processor 210A also removes tag object 2733A from virtual left hand 1531LA.

[Modification]
Processor 210A can associate tag object 2733A with any part of avatar object 6A, not just virtual left hand 1531LA. Processor 210A, for example, associates tag object 2733A with virtual right hand 1531RA. Processor 210A associates tag object 2733A with, for example, the body or legs of avatar object 6A. Processor 210A preferably places tag object 2733A in virtual right hand 1531RA or virtual left hand 1531LA. This allows the user 5A to easily confirm the total billing amount displayed in the tag object 2733A.

Processor 210A does not necessarily associate tag object 2733A with avatar object 6A when avatar object 6A enters second virtual space 2711A. The processor 210A can associate billing information with the avatar object 6A and register it with the avatar object 6A. When the avatar object 6A exits from the second virtual space 2711A, the avatar object 6A can also execute settlement-related processing according to the billing information registered in the avatar object 6A.

The user 5A can pre-purchase points that can be used for settlement of the total billing amount. When the user 5A has purchased points in advance, the server 600 can settle the total billing amount by consuming points corresponding to the total billing amount among all the points possessed by the user 5A.

When processor 210A registers the billing information with tag object 2733A, processor 210A can store the billing information in storage 230A. When avatar object 6A illegally leaves second virtual space 2711A, processor 210A uses the billing information stored in storage 230A to execute processing related to payment. Avatar object 6A illegally leaves second virtual space 2711A, for example, when user 5A forcibly turns off computer 200A before avatar object 6A leaves second virtual space 2711A. can occur. Processor 210A transmits the billing information stored in storage 230A to server 600 after computer 200A is powered on. Processor 610 settles the total billing amount represented by the received billing information.

If the user 5A forcibly powers off the computer 200A before leaving the avatar object 6A from the second virtual space 2711A, the processor 210A restores the power of the avatar object 6A after the power of the computer 200A is turned on next time. Detects that an unauthorized exit has occurred. When processor 210A detects the occurrence of unauthorized withdrawal, processor 210A notifies server 600 to that effect. When computer 200A notifies processor 610 of the occurrence of unauthorized withdrawal, processor 610 settles the billing amount according to the billing information of user 5A at the time of occurrence of unauthorized withdrawal. Thereby, the HMD system 100 can prevent the user 5A from evading payment of the billing amount. The processor 610 can delete the account of the user 5A when the avatar object 6A frequently withdraws illegally or the settlement cannot be completed normally after the illegal withdrawal.

When the avatar object 6A enters the second virtual space 2711A, the processor 210A can register billing information corresponding to the entrance fee in the tag object 2733A and immediately execute processing related to settlement corresponding to the entrance fee. In other words, processor 210A can also execute processing related to payment after billing information is registered in tag object 2733A and before avatar object 6A exits second virtual space 2711A. Furthermore, even when billing information corresponding to the behavior of avatar object 6A is registered in tag object 2733A, processor 210A executes processing related to payment corresponding to the registered billing information before avatar object 6A leaves. can be done.

The processor 210A can cause the avatar object 6A to enter the second virtual space 2711A before the live performance of the avatar object 6B in the second virtual space 2711A. When avatar object 6A enters second virtual space 2711A before the live performance, processor 210A can cause avatar object 6A to purchase a ticket for the live performance from avatar object 6B in second virtual space 2711A.

The user 5A can cause the avatar object 6A to purchase a live ticket before the live is held. In one aspect, the processor 210B pre-detects the movement of the user 5B to sell tickets for the gig in advance of the gig. Processor 210B causes avatar object 6B to perform a first action for selling tickets in response to the detected motion. The processor 210B transmits to the server 600 avatar information of the avatar object 6B including motion information corresponding to the first action.

When avatar object 6A enters second virtual space 2711A before the live performance, processor 210A requests server 600 for the avatar information of avatar object 6B, which computer 200B has sent to server 600 in advance. Server 600 transmits the avatar information of avatar object 6B to computer 200A in response to the request. Processor 210A arranges avatar object 6B in second virtual space 2711A based on the received avatar information. The processor 210A causes the avatar object 6B to perform a first action for selling tickets based on the motion information included in the avatar information. The processor 210A, for example, causes the avatar object 6B to hand over the ticket to the avatar object 6A. Processor 210A also causes avatar object 6B to swing virtual left hand 1531LB.

In another aspect, processor 210A transmits avatar information of avatar object 6A to server 600 when avatar object 6A enters second virtual space 2711A during a specific time period. As a result of the synchronization processing by the server 600, the processor 210B receives the avatar information of the avatar object 6A from the server 600. FIG. Processor 210B arranges avatar object 6A in second virtual space 2711B according to the received avatar information. Processor 210B detects movement of user 5B to sell tickets for a live show after avatar object 6A is positioned. Processor 210B causes avatar object 6B to perform a second action for selling tickets to avatar object 6A in second virtual space 2711B in accordance with the detected movement of user 5B. The processor 210B transmits to the server 600 avatar information of the avatar object 6B including motion information corresponding to the second action.

Processor 210A receives the avatar information of avatar object 6B from server 600 as a result of the synchronization processing of server 600 . Based on the motion information included in the received avatar information, the processor 210A causes the avatar object 6B to perform a second action for selling tickets based on the motion information included in the avatar information. In other words, the processor 210A can cause the avatar object 6B to perform the action of selling tickets in response to the real-time movements of the user 5B. This allows the user 5A to interact with the user 5B in real time, thereby making the user 5A more satisfied.

[Embodiment 2]
In this embodiment, the processor 210A executes processing related to billing and settlement for the user 5A by the method using the tag object 2733A described in the first embodiment. However, without being limited to this, the processor 210A can also execute processing related to billing and payment for the user 5A by any method different from the method according to the first embodiment.

[Billing processing flow according to distance traveled]
FIG. 45 is a sequence chart representing part of the processing performed in HMD set 110A according to an embodiment. FIG. 46 is a diagram showing second virtual space 2711A and view image 4617A according to an embodiment. In FIG. 46, in the second virtual space 2711A, a performance is being performed by the avatar object 6B (second avatar) associated with the user 5B (second user). Only billing information corresponding to the entrance fee (500 yen) is registered in the tag object 2733A. Avatar object 6A (first avatar) associated with user 5A (first user) is placed at a first position 4661 far away from stage object 1532 in second virtual space 2711A. A user 5A is watching the performance of an avatar object 6B far away from the avatar object 6A.

User 5A wishes to view the performance closer to avatar object 6B by moving avatar object 6A to second position 4662 closer to stage object 1532 . User 5A moves the right hand so that the tip of virtual right hand 1531RA points to second position 4662 in order to fulfill this desire. At step S4501, the processor 210A detects movement of the right hand of the user 5A based on the output of the right controller 300RA. At step S4502, the processor 210A moves the virtual right hand 1531RA according to the movement of the right hand of the user 5A. As shown in FIG. 46A, after the virtual right hand 1531RA moves, the tip of the virtual right hand 1531RA points to the second position 4662 in the second virtual space 2711A. In step S4503, the processor 210A identifies the second position 4662 in the second virtual space 2711A pointed to by the tip of the virtual right hand 1531RA.

The processor 210A, for example, displays a view image 4617A corresponding to the second virtual space 2711A shown in FIG. 46(A) on the monitor 130A as shown in FIG. 46(B). At step S4504, the processor 210A notifies the user 5A on the view image 4617A of the mark 4663 indicating the second position 4662 in the second virtual space 2711A. Specifically, processor 210A displays mark 4663 superimposed on position 4664 in view image 4617A corresponding to second position 4662 on monitor 130A. Processor 210A can place mark 4663 as a virtual object at second location 4662 in second virtual space 2711A. Processor 210A may alternatively include mark 4663 at location 4664 corresponding to second location 4662 when generating view image 4617A. User 5A confirms that mark 4663 indicating second position 4662 in second virtual space 2711A is displayed on monitor 130A by viewing view image 4617A. By checking the mark 4663, the user 5A understands that the second position 4662 has been specified as the destination of the avatar object 6A.

After confirming the mark 4663, the user 5A performs the first operation to move the avatar object 6A to the second position 4662. The first operation is, for example, the user pressing any button of the right controller 300RA. At step S4505, the processor 210A detects a first operation by the user 5A for moving the avatar object 6A.

FIG. 47 is a diagram showing second virtual space 2711A and view image 4717A according to an embodiment. In step S4506, when the first operation by the user 5A is detected, the processor 210A operates as the avatar object control module 1424 to move the avatar object 6A to the first position within the second virtual space 2711A as shown in FIG. Discretely move from position 4661 to second position 4662 . Specifically, the processor 210A instantaneously moves the avatar object 6A from the first position 4661 to the second position 4662 without going through each position between the first position 4661 and the second position 4662 (warping). move). The processor 210A also discretely moves the virtual camera 14A in conjunction with the movement of the avatar object 6A.

In step S4507, processor 210A registers billing information according to moving distance 4775 of avatar object 6A from first position 4661 to second position 4662 in tag object 2733A. Specifically, processor 210A calculates the billing amount according to moving distance 4775 by multiplying moving distance 4775 by a billing unit price for each fixed distance. The charging unit price is determined in advance by the operator of the HMD system 100, for example. For example, when defining the second virtual space 2711A, the processor 210A sets the charging unit price represented by the unit price information included in the virtual space data in the computer 200A. In FIG. 47, 500 yen is calculated as the billing amount. Processor 210A newly registers billing information representing the calculated billing amount in tag object 2733A. Processor 210A displays on the surface of tag object 2733A the total billing amount (1000 yen) represented by the two registered billing information.

After the billing information is registered, processor 210A moves virtual left hand 1531LA into viewing area 15A as shown in FIG. 47(A) in accordance with the movement of user 5A's left hand. The processor 210A, for example, displays a view image 4717A corresponding to the second virtual space 2711A shown in FIG. 47(A) on the monitor 130A as shown in FIG. 47(B). User 5A recognizes that avatar object 6A has moved to second position 4662 near stage object 1532 by visually recognizing view image 4717A. The user 5A also recognizes that the avatar object 6B is in front of the avatar object 6A. This allows the user 5A to view the performance near the avatar object 6B, thereby enjoying the live performance more. User 5A further recognizes that the current total billing amount is 1000 yen by visually recognizing tag object 2733A. In other words, the user 5A understands that as a result of moving the avatar object 6A to the second position 4662, the user 5A has been newly charged 500 yen as a movement fee.

FIG. 48 is a diagram showing second virtual space 2711A and view image 4817A according to an embodiment. The second virtual space 2711A in FIG. 48 is the same as in FIG. Avatar object 6A is placed at first position 4661 . Only billing information corresponding to the entrance fee (500 yen) is registered in the tag object 2733A. The user 5A moves the right hand so that the tip of the virtual right hand 1531RA points to the third position 4866 farther from the stage object 1532 than the second position 4662. Processor 210A identifies third position 4866 in second virtual space 2711A pointed to by the tip of virtual right hand 1531RA.

The processor 210A, for example, displays a view image 4817A corresponding to the second virtual space 2711A shown in FIG. 48(A) on the monitor 130A as shown in FIG. 48(B). Processor 210A notifies user 5A on view image 4817A of mark 4663 indicating third position 4866 in second virtual space 2711A. Specifically, processor 210A superimposes mark 4663 on position 4867 in view image 4817A corresponding to third position 4866 in second virtual space 2711A and displays it on monitor 130A. Processor 210A can place mark 4663 as a virtual object at third position 4866 in second virtual space 2711A. Processor 210A may alternatively include mark 4663 at location 4867 corresponding to third location 4886 when generating view image 4817A. User 5A confirms that mark 4663 indicating third position 4866 in second virtual space 2711A is displayed on monitor 130A by viewing view image 4817A. By checking the mark 4663, the user 5A understands that the third position 4866 has been specified as the destination of the avatar object 6A.

FIG. 49 is a diagram showing second virtual space 2711A and view image 4917A according to an embodiment. The processor 210A detects a first operation by the user 5A to move the avatar object 6A to the third position 4866. When the first operation by user 5A is detected, processor 210A moves avatar object 6A and virtual camera 14A from first position 4661 to third position 4866 in second virtual space 2711A as shown in FIG. 49(A). discretely move up to Processor 210A registers billing information according to moving distance 4969 of avatar object 6A from first position 4661 to third position 4866 in tag object 2733A. Specifically, processor 210A calculates the billing amount according to moving distance 4969 by multiplying moving distance 4969 by the billing unit price. Since the moving distance 4969 is shorter than the moving distance 4775, 250 yen, which is cheaper than 500 yen, is calculated as the billing amount in FIG. Processor 210A newly registers billing information representing the calculated billing amount in tag object 2733A.

Processor 210A displays on the surface of tag object 2733A the total billing amount (750 yen) represented by the two registered billing information. The processor 210A, for example, displays a view image 4917A corresponding to the second virtual space 2711A shown in FIG. 49(A) on the monitor 130A as shown in FIG. 49(B). User 5A recognizes that avatar object 6A has moved to third position 4866, which is slightly away from stage object 1532, by visually recognizing view image 4917A. User 5A also recognizes that the current total billing amount is 750 yen. In other words, as a result of moving the avatar object 6A to the third position 4866, the user 5A understands that 250 yen has been newly charged to the user 5A as a movement fee. As shown in FIGS. 47 and 49, processor 210A registers billing information in tag object 2733A, which indicates that the longer the distance traveled, the higher the billing amount.

[Main advantages of this embodiment]
Processor 210A registers billing information according to the moving distance of avatar object 6A in tag object 2733A. Thus, the HMD system 100 can provide the user 5A with a suitable billing method in the second virtual space 2711A.

The user 5A can watch the live performance at his/her favorite position by moving the avatar object 6A to a desired position in the second virtual space 2711A during the live performance. The user 5A, for example, moves the avatar object 6A to a position near the avatar object 6B when performing a favorite performance. When the user 5A wants to view the performance at a different position depending on the contents of the performance performed by the avatar object 6B, the user 5A can move the avatar object 6A to his/her favorite position for each performance and view the live performance. For example, if the user 5A's favorite performance consists of two parts, the first part is viewed by moving the avatar object 6A to the front position of the avatar object 6B, and the second part is , the avatar object 6A can be moved to the upper side of the stage object 1532 and the avatar object 6B can be viewed from an oblique direction. When there are a plurality of avatar objects 6B performing performances in the second virtual space 2711A and the positions of the avatar objects 6B on the stage object 1532 are switched, the user 5A can move to the position of his/her favorite avatar object 6B. It is also possible to view the live performance by moving the avatar object 6A so as to follow it. As a result, the HMD system 100 can make the live performance more enjoyable according to the preferences of the user 5A.

[Movement of avatar object 6B]
Although the example in which the avatar object 6A can move at any timing has been described above, the movement of the avatar object 6A is not limited to this example. An example in which the avatar object 6A can be moved only at the timing designated by the avatar object 6B will be described in detail below.

FIG. 50 is a diagram showing second virtual space 2711A and view image 5017A according to an embodiment. In FIG. 50, the avatar object 6B is performing in the second virtual space 2711A. Only billing information corresponding to the entrance fee (500 yen) is registered in the tag object 2733A. Avatar object 6B is placed at first position 5070 near the right end on stage object 1532 . The avatar object 6A is placed at a second position 5071 opposite the first position 5070 near the stage object 1532 in the second virtual space 2711A. Thus, the avatar object 6A faces the avatar object 6B and watches the performance by the avatar object 6B near the avatar object 6B.

Processor 210A, for example, displays view image 5017A corresponding to second virtual space 2711A shown in FIG. 50(A) on monitor 130A as shown in FIG. 50(B). User 5A enjoys the performance of avatar object 6B near avatar object 6A by visually recognizing view image 5017A.

FIG. 51 is a diagram showing second virtual space 2711B and view image 5117B according to an embodiment. The second virtual space 2711B shown in FIG. 51 is synchronized with the second virtual space 2711A shown in FIG. The processor 210B, for example, displays a view image 5117B corresponding to the second virtual space 2711B shown in FIG. 51(A) on the monitor 130B as shown in FIG. 51(B). User 5B recognizes that avatar object 6A is watching the performance near avatar object 6B by visually recognizing view image 5117B.

FIG. 52 is a diagram showing second virtual space 2711A and view image 5217A according to an embodiment. The processor 210A prohibits the avatar object 6A from moving within the second virtual space 2711A when the conditions enabling the avatar object 6A to move within the second virtual space 2711A are not satisfied. In FIG. 52, processor 210A moves virtual right hand 1531RA so that the tip of virtual right hand 1531RA points to a desired position in second virtual space 2711A in accordance with the movement of user 5A's right hand. However, processor 210A does not identify the position pointed to by the tip of virtual right hand 1531RA in response to prohibition of movement of avatar object 6A.

The processor 210A, for example, displays a view image 5217A corresponding to the second virtual space 2711A shown in FIG. 52(A) on the monitor 130A as shown in FIG. 52(B). In response to not specifying the position pointed by the tip of virtual right hand 1531RA, processor 210A does not notify user 5A of mark 4663 indicating that position on view image 5217A. By visually recognizing the field-of-view image 5217A, the user 5A notices that the mark 4663 indicating the position is not displayed on the monitor 130A even though the tip of the virtual right hand 1531RA points to the desired position in the second virtual space 2711A. confirm. The user 5A also understands that the avatar object 6A cannot be moved because the mark 4663 is not displayed.

FIG. 53 is a diagram showing second virtual space 2711B and view image 5317B according to an embodiment. User 5B wishes to change the position at which avatar object 6B performs on stage object 1532 to third position 5372 near the left end of stage object 1532 . User 5B moves the left hand so that the tip of virtual left hand 1531LB points to third position 5372 in order to fulfill this desire. Processor 210B moves virtual left hand 1531LB in second virtual space 2711B in accordance with the movement of user 5B's left hand. Processor 210B identifies third position 5372 in second virtual space 2711B pointed to by the tip of virtual left hand 1531LB.

The processor 210B, for example, displays a view image 5317B corresponding to the second virtual space 2711B shown in FIG. 53(A) on the monitor 130B as shown in FIG. 53(B). Processor 210B notifies user 5B on view image 5317B of mark 4663 indicating third position 5372 in second virtual space 2711B. Specifically, processor 210B superimposes mark 4663 on position 5373 in view image 5317B corresponding to third position 5372 in second virtual space 2711B and displays it on monitor 130B. Processor 210B can place mark 4663 as a virtual object at third position 5372 in second virtual space 2711B. Processor 210B may alternatively include mark 4663 at location 5373 corresponding to third location 5372 when generating view image 5317B. User 5B confirms that mark 4663 indicating third position 5372 is displayed on monitor 130B by visually recognizing view image 5317B. By checking the mark 4663, the user 5B understands that the third position 5372 has been specified as the destination of the avatar object 6B.

After confirming the mark 4663, the user 5B performs a second operation for moving the avatar object 6B to the third position 5372. FIG. The second operation is, for example, the user 5B pressing any button of the right controller 300RA. Processor 210B detects a second operation by user 5B to move avatar object 6B.

FIG. 54 is a diagram showing second virtual space 2711B and view image 5417B according to one embodiment. When the second operation by the user 5B is detected, the processor 210B moves the avatar object 6B and the virtual camera 14B from the first position 5070 to the third position 5372 within the second virtual space 2711B as shown in FIG. 54(A). discretely move up to The processor 210B, for example, displays a view image 5417B corresponding to the second virtual space 2711B shown in FIG. 54(A) on the monitor 130B as shown in FIG. 54(B). User 5B recognizes that avatar object 6B has moved to third position 5372 by visually recognizing view image 5417B. The user 5B also recognizes that the avatar object 6A has disappeared from the field of view of the avatar object 6B as a result of the movement of the avatar object 6B.

FIG. 55 is a diagram showing second virtual space 2711A and view image 5517A according to an embodiment. After avatar object 6B has moved in second virtual space 2711B, processor 210A receives the latest avatar information of avatar object 6B from server 600 . Processor 210A discretely moves avatar object 6B from first position 5070 to third position 5372 in second virtual space 2711A based on the motion information included in the received avatar information. As a result, the avatar object 6B is placed outside the field of view 15A. Processor 210A, for example, displays view image 5517A corresponding to second virtual space 2711A shown in FIG. 55(A) on monitor 130A as shown in FIG. 55(B). The user 5A recognizes that the avatar object 6B has moved on the stage object 1532 and has left the field of view of the avatar object 6A by viewing the field-of-view image 5517A.

The processor 210A enables the avatar object 6A to move within the second virtual space 2711A when the avatar object 6B moves within the second virtual space 2711A. However, the conditions that allow the avatar object 6A to move within the second virtual space 2711A are not limited to this. The processor 210A notifies the user 5A that the avatar object 6A can move, for example, by voice. The processor 210A can also notify the user 5A that the avatar object 6A can now move by changing the lighting in the second virtual space 2711A.

FIG. 56 is a diagram showing second virtual space 2711A and view image 5617A according to an embodiment. In FIG. 56, processor 210A permits avatar object 6A to move within second virtual space 2711A in response to movement of avatar object 6B. The user 5A recognizes that the avatar object 6B has moved near the right end of the stage object 1532 by looking to the right. User 5A wishes to continue watching the live near avatar object 6B by moving avatar object 6A to fourth position 5674 in second virtual space 2711A near avatar object 6B after movement.

The user 5A moves the right hand so that the tip of the virtual right hand 1531RA points to the fourth position 5674 in order to fulfill this desire. The processor 210A moves the virtual right hand 1531RA according to the movement of the right hand of the user 5A. The processor 210A identifies the fourth position 5674 pointed to by the tip of the virtual right hand 1531RA after movement in response to the permission of movement of the avatar object 6A.

Processor 210A, for example, displays view image 5617A corresponding to second virtual space 2711A shown in FIG. 56(A) on monitor 130A as shown in FIG. 56(B). Processor 210A superimposes mark 4663 indicating fourth position 5674 in second virtual space 2711A on position 5675 in view image 4817A corresponding to fourth position 5674 in second virtual space 2711A and displays it on monitor 130A. Processor 210A can place mark 4663 as a virtual object at fourth position 5674 in second virtual space 2711A. Processor 210A may alternatively include mark 4663 at location 5675 corresponding to fourth location 5674 when generating view image 5617A. User 5A recognizes that avatar object 6B has moved to near the right end of stage object 1532 by visually recognizing view image 5617A. User 5A further confirms that fourth position 5674 has been designated as the destination of movement of avatar object 6A by confirming mark 4663 included in field-of-view image 5617A.

FIG. 57 is a diagram showing second virtual space 2711A and view image 5717A according to an embodiment. The processor 210A detects a first operation by the user 5A to move the avatar object 6A to the fourth position 5674. When the first operation by user 5A is detected, processor 210A moves avatar object 6A and virtual camera 14A from second position 5071 to fourth position 5674 in second virtual space 2711A as shown in FIG. 57(A). discretely move up to Processor 210A registers billing information according to moving distance 5776 of avatar object 6A from second position 5071 to fourth position 5674 in tag object 2733A.

The processor 210A, for example, displays a view image 5717A corresponding to the second virtual space 2711A shown in FIG. 57(A) on the monitor 130A as shown in FIG. 57(B). User 5A recognizes that avatar object 6A has moved to fourth position 5674 near avatar object 6B by visually recognizing view image 5717A. The user 5A also recognizes that the avatar object 6B is in front of the avatar object 6A. Thereby, the user 5A can continue to enjoy the live near the avatar object 6B.

In the forms shown in FIGS. 50 to 57, as a result of the avatar object 6B moving within the second virtual space 2711A, the user 5A has a strong motivation to move the avatar object 6A closer to the avatar object 6B. Thus, the user 5B can strongly urge the user 5A to move the avatar object 6A by moving the avatar object 6A. Therefore, it can be expected that the user 5A will actively move the avatar object 6A within the second virtual space 2711A, so it is possible to increase the possibility of charging the user 5A with the billing amount corresponding to the movement distance. As a result, it is possible to increase the possibility of increasing live sales of the avatar object 6B.

[Modification]
Processor 210A does not necessarily need to register billing information according to travel distance in tag object 2733A. The processor 210A can, for example, associate billing information according to the distance traveled with the avatar object 6A and register it in the avatar object 6A. Processor 210A can perform any processing related to charging based on moving distance instead of registering billing information based on moving distance in tag object 2733A. Processing related to billing is, for example, transmitting billing information to server 600 without registering it in tag object 2733A.

The processor 210A can also perform processing related to charging according to the number of times the avatar object 6A moves. Processor 210A, for example, registers billing information according to the number of times of movement in tag object 2733A. Processor 210A, for example, calculates a billing amount that increases as the number of times of movement increases, and registers billing information representing the calculated billing amount in tag object 2733A. In this case, the processor 210A can register optimal billing information in the tag object 2733A according to the number of movements of the avatar object 6A. The processor 210A can also prevent the avatar object 6A from obstructing the live performance of the avatar object 6B by unlimited movement in the second virtual space 2711A.

Processor 210A can perform billing processing according to at least one of the distance traveled and the number of times traveled. Processor 210A performs arbitrary processing among a first process related to charging based only on the distance traveled, a second process related to charging based only on the number of times of travel, and a third process related to charging based on both the distance traveled and the number of times traveled. can be executed.

The processor 210A can also perform processing related to charging according to the moving distance and moving direction of the avatar object 6A. For example, when the moving direction of the avatar object 6A is the direction closer to the avatar object 6B, the processor 210A increases the charging unit price. Processor 210A, for example, lowers the charging unit price when the moving direction of avatar object 6A is the direction away from avatar object 6B.

The processor 210A can also continuously move the avatar object 6A within the second virtual space 2711A. For example, the processor 210A moves the avatar object 6A from the first position 4661 to the second position so as to pass through all positions between the first position 4661 and the second position 4662 in response to the first operation by the user 5A. Move to 4662. In this case as well, processor 210A registers billing information representing the billing amount according to movement distance 4775 in tag object 2733A.

The processor 210A can also automatically move the avatar object 6A within the second virtual space 2711A without detecting the first operation by the user 5B. Even when the avatar object 6A is automatically moved, the processor 210A registers in the tag object 2733A billing information representing the billing amount according to the movement distance.

In the present embodiment, it is assumed that the processor 210A can move the avatar object 6A only to a position where no avatar object 6 different from the avatar object 6A is placed in the second virtual space 2711A, but the present invention is not limited to this. not something. In other words, it is assumed that the user 5A cannot designate the position where the avatar object 6 different from the avatar object 6A is placed as the position to which the avatar object 6A is to be transferred, but is limited to this. isn't it.

It is assumed that the processor 210A can move the avatar object 6A only to positions within a prescribed range in the second virtual space 2711A. The processor 210A for example forbids moving the avatar object 6A onto the stage object 1532 in principle. Processor 210A may also move avatar object 6A onto stage object 1532 if authorized by user 5B. However, it is not limited to this.

The user 5B can perform a second operation for permitting movement of the avatar object 6A. When processor 210B detects the second operation by user 5B, processor 210B transmits detection information indicating detection of the second operation to computer 200A via server 600 . Processor 210A receives detection information transmitted from computer 200B. Processor 210A detects the second operation by user 5B based on the received detection information. The processor 210A enables the avatar object 6A to move within the second virtual space 2711A when detecting the second operation. The processor 210A, for example, enables movement for a certain period of time after detecting the second operation. The second operation may not be the operation itself of the user 5B, but may be an action of the user 5B such as the contents of the speech of the user 5B, or may be information specifying a period during which the movement is allowed. Thereby, the processor 210A can prevent the avatar object 6A from moving freely without permission of the user 5B during the live. In other words, the user 5B can voluntarily specify the timing at which the avatar object 6A can move. As a result, the user 5B can control the performance of the live performance in a manner more desired by the user 5B.

Processor 210A can detect the occurrence of a first event in second virtual space 2711A. The first event is, for example, the avatar object 6B taking some action. The action of the avatar object 6B is, for example, moving the avatar object 6B to another position on the stage object 1532 as shown in FIG. The first event may be that the avatar object 6B performs a prescribed performance on the stage object 1532 for a certain period of time or longer. Processor 210A enables avatar object 6A to move within second virtual space 2711A when the first event is detected. This can prevent the avatar object 6A from randomly moving during a live performance.

In the HMD system 100, the charging unit price for each fixed distance may differ according to the size of the second virtual space 2711A. For example, the larger the size of the second virtual space 2711A, the cheaper the charge unit price. When the size of the defined second virtual space 2711A is the first size, the processor 210A sets the first billing unit price (eg, 50 yen) to the computer 200A as the billing unit price. When the size of the defined second virtual space 2711A is a second size larger than the first size, the processor 210A sends a second billing unit price (for example, 20 yen) lower than the first billing unit price to the computer 200A. set. Processor 210A can calculate the optimum billing amount according to both the size and movement distance of second virtual space 2711A by multiplying the billing unit price set in computer 200A by the movement distance of avatar object 6A. .

Even when avatar object 6A moves within second virtual space 4311A in which live video 4239 shown in FIG. 43 is reproduced, processor 210A executes processing related to charging according to at least one of the moving distance and the number of moving times. can be done. In addition, processor 210A also controls avatar object 6A to move in a virtual space in which performance by avatar object 6B is not performed and live video 4239 is not reproduced, according to at least one of the moving distance and the number of moving times. Billing-related processing can be executed.

[Embodiment 3]
In this embodiment, the processor 210A executes processing related to billing and settlement for the user 5A by the method using the tag object 2733A described in the first embodiment. However, without being limited to this, the processor 210A can also execute processing related to billing and payment for the user 5A by any method different from the method according to the first embodiment.

In this embodiment, the processor 210C executes processing related to billing and payment for the user 5C by the same method as the method using the tag object 2733A described in the first embodiment. However, without being limited to this, the processor 210C can also execute processing related to billing and payment for the user 5C by any method different from the method according to the first embodiment.

[Viewpoint sharing processing flow]
FIG. 58 is a sequence chart representing part of the processing performed in HMD set 110A according to one embodiment. FIG. 59 is a diagram showing second virtual space 2711C and view image 5917C according to one embodiment. In FIG. 59, the processor 210C controls the avatar object 6B (third user) associated with the user 5B (third user) in accordance with the movement of the user 5B in the second virtual space 2711C provided to the user 5C (second user). avatar) to perform the performance (first performance). Processor 210C causes avatar object 6C (second avatar) associated with user 5C to view the performance near avatar object 6B. The processor 210C causes the avatar object 6A (first avatar) associated with the user 5A (first user) to view the performance remotely from the avatar object 6B.

In the second virtual space 2711A that is synchronized with the second virtual space 2711C, the processor 210A causes the avatar object 6B to perform the performance according to the movements of the user 5B. Processor 210A causes avatar objects 6A and 6C to view the performance by avatar object 6B in second virtual space 2711A.

The processor 210C notifies the user 5C of a message guiding the movement of the avatar object 6C to the stage object 1532 during the live. Processor 210C, for example, displays the message in tag object 2733C, as shown in FIG. 59(A). The message includes text guiding the user 5C to move to the stage object 1532 and the billing amount to be charged to the user 5C when moving the avatar object 6C. In the example of FIG. 59, the billing amount for moving the avatar object 6C onto the stage object 1532 is 1000 yen.

After the message is displayed on tag object 2733C, processor 210C moves virtual left hand 1531LC into viewing area 15C as shown in FIG. 59(A) in response to the movement of user's 5C left hand. The processor 210C, for example, displays a view image 5917C corresponding to the second virtual space 2711C shown in FIG. 59(A) on the monitor 130C as shown in FIG. 59(B). User 5C confirms the message displayed on tag object 2733C by visually recognizing view image 5917C. Thereby, the user 5C recognizes that the avatar object 6C can be moved onto the stage object 1532 by paying a new charge of 1000 yen.

FIG. 60 is a diagram showing second virtual space 2711C and view image 6017C according to an embodiment. User 5C confirms the message displayed in tag object 2733C by viewing view image 6017C. User 5C wishes to move avatar object 6C onto stage object 1532 as a result of confirming the message. In order to fulfill this desire, the user 5C performs an operation for selecting the tag object 2733C. The processor 210C registers charging information corresponding to stage movement in the tag object 2733C in response to detecting the operation of the user 5C. Specifically, the processor 210C registers billing information representing the billing amount for moving the avatar object 6C onto the stage object 1532 in the tag object 2733C. After registering the billing information, the processor 210C moves the avatar object 6C onto the stage object 1532 as shown in FIG. 60(A). The processor 210C places the avatar object 6C after movement at a position in front of the avatar object 6B on the stage object 1532 . The processor 210C transmits the avatar information of the avatar object 6C to the server 600 after moving the avatar object 6C.

The processor 210C, for example, displays a view image 6017C corresponding to the second virtual space 2711C shown in FIG. 60(A) on the monitor 130C as shown in FIG. 60(B). The user 5C recognizes that the avatar object 6C has moved onto the stage object 1532 by viewing the view image 6017C. The user 5C can also enjoy a performance in front of the avatar object 6B and enjoy interaction with the avatar object 6B (in other words, with the user 5B).

FIG. 61 is a diagram showing second virtual space 2711A and view image 6117A according to an embodiment. At step S5801, the processor 210A receives avatar information of the avatar object 6C from the server 600 after the avatar object 6C moves onto the stage object 1532 in the second virtual space 2711C. In step S5802, processor 210A moves avatar object 6C onto stage object 1532 in second virtual space 2711A, as shown in FIG. 61A, based on the received avatar information. In other words, the processor 210A synchronizes the second virtual space 2711A with the second virtual space 2711C shown in FIG. The processor 210A arranges the avatar object 6C at a position on the stage object 1532 in front of the avatar object 6B.

After moving the avatar object 6C onto the stage object 1532, the processor 210A receives the avatar information of the avatar object 6B from the server 600. FIG. Processor 210A causes avatar object 6B to perform a performance (second performance) for avatar object 6C in accordance with the motion information included in the received avatar information. Thus, in the second virtual space 2711A, the avatar object 6C watches the performance of the avatar object 6B for the avatar object 6C in response to the billing information corresponding to the stage movement being registered in the tag object 2733C. .

After the avatar object 6C moves onto the stage object 1532, the processor 210A sends a message to the user 5A during the live to guide the avatar object 6A to share the viewpoint of another avatar object 6 different from the avatar object 6A. Notify the user 5A. Processor 210A, for example, displays the message in tag object 2733A. The message includes text guiding the user 5A to share the viewpoint, and the billing amount to be charged to the user 5A when sharing the viewpoint. In the example of FIG. 61, the billing amount charged to the user 5A for viewpoint sharing is 200 yen. After the message is displayed on tag object 2733A, processor 210A moves virtual left hand 1531LA into viewing area 15A, as shown in FIG. 61(A), according to the movement of user's 5A left hand.

The processor 210A, for example, displays a view image 6117A corresponding to the second virtual space 2711A shown in FIG. 61(A) on the monitor 130A as shown in FIG. 61(B). The user 5A recognizes that the avatar object 6C has moved onto the stage object 1532 by viewing the view image 6117A. User 5A also recognizes that avatar object 6C is viewing the performance on stage object 1532 near avatar object 6B. Further, by confirming the message displayed on the tag object 2733C, the user 5A can make the avatar object 6A share the viewpoint of any of the avatar objects 6 by paying a new charge of 200 yen. recognize.

[Viewpoint sharing by billing]
FIG. 62 is a diagram showing second virtual space 2711A and view image 3417A according to one embodiment. After confirming the message displayed in the tag object 2733A, the user 5A wishes to have the avatar object 6A share the viewpoint of the avatar object 6C. User 5A performs an operation to select tag object 2733A in order to fulfill this desire. In step S5803, processor 210A registers billing information corresponding to viewpoint sharing in tag object 2733A when an operation by user 5A is detected. Specifically, processor 210A newly registers billing information indicating the billing amount (200 yen) required for viewpoint sharing in tag object 2733A. Processor 210A enables avatar object 6A to share the viewpoint of other avatar object 6 in second virtual space 2711A in response to registration of billing information. The processor 210A notifies the user 5A that viewpoint sharing has become possible, for example, by voice. The processor 210A can also notify the user 5A that viewpoint sharing is enabled by changing the lighting in the second virtual space 2711A.

Processor 210A moves virtual right hand 1531RA so that the tip of 1RA of virtual right hand 153 points to avatar object 6C as shown in FIG. move. In step S5804, the processor 210A identifies the avatar object 6C pointed to by the tip of the virtual right hand 1531RA.

The processor 210A, for example, displays a view image 6217A corresponding to the second virtual space 2711A shown in FIG. 62(A) on the monitor 130A as shown in FIG. 62(B). At step 5805, processor 210A displays on monitor 130A a mark 6277 indicating avatar object 6C superimposed on avatar object 6C in view image 6217A. The processor 210A can also place the mark 6277 in the second virtual space 2711A as a virtual object in contact with or close to the avatar object 6C. Processor 210A may also cause, when generating view image 6217A, to include mark 6277 at the position of avatar object 6C within view image 4617A. User 5A recognizes that avatar object 6C has been designated by visually recognizing mark 6277 in view image 6217A. After confirming the mark 6277, the user 5A performs the first operation for sharing the viewpoint of the avatar object 6C with the avatar object 6A. The first operation is, for example, the user 5A pressing any button of the right controller 300RA.

FIG. 63 is a diagram showing second virtual space 2711A and view image 6317A according to an embodiment. At step S5806, processor 210A selects avatar object 6C in response to detection of the first operation. At step S5807, the processor 210A causes the avatar object 6A to share the viewpoint of the avatar object 6C viewing the performance for the avatar object 6C in response to the avatar object 6C being selected. Specifically, processor 210A moves virtual camera 14A from the position of avatar object 6A to the position of avatar object 6C. At this time, the processor 210A does not move the avatar object 6A. Therefore, after the virtual camera 14A moves, the positions at which the avatar object 6A and the virtual camera 14A are arranged in the second virtual space 2711A are different from each other. The processor 210A controls the orientation of the virtual camera 14A after movement according to the orientation of the head of the avatar object 6C. This allows the avatar object 6A to share the viewpoint of the avatar object 6C.

Processor 210A can also display a UI for selecting avatar object 6C on tag object 2733A when billing information corresponding to view sharing is registered in tag object 2733A. In this case, user 5A selects avatar object 6C through input to the UI displayed in tag object 2733A. The processor 210A causes the avatar object 6A to share the viewpoint of the avatar object 6C selected through the UI. When billing information corresponding to viewpoint sharing is registered in tag object 2733A, processor 210A can automatically cause avatar object 6A to share the viewpoint of avatar object 6C without causing user 5A to select avatar object 6C. can.

After sharing the viewpoint, at step S5808, the processor 210A receives the avatar information of the avatar object 6B from the server 600. FIG. In step S5809, processor 210A causes avatar object 6B to perform a performance (second performance) for avatar object 6C based on the motion information included in the received avatar information. In this way, the avatar object 6C watches the performance by the avatar object 6B for the avatar object 6C in the second virtual space 2711A in response to the billing information corresponding to stage movement being registered in the tag object 2733C.

The processor 210A, for example, displays a field-of-view image 6317A (first field-of-view image) corresponding to the second virtual space 2711A shown in FIG. 63(A) on the monitor 130A as shown in FIG. 63(B). The user 5A recognizes that the viewpoint of the avatar object 6C is shared with the avatar object 6A by visually recognizing the view image 6317A. In other words, the user 5A can watch the performance for the avatar object 6C in front of the avatar object 6B, like the user 5C.

In FIG. 63A, the viewpoint of avatar object 6C is shared with avatar object 6A, but avatar object 6A itself is located far away from avatar object 6B. Therefore, the user 5A cannot directly interact with the avatar object 6A with the avatar object 6B. For example, the user 5A cannot extend the virtual right hand 1531RA toward the avatar object 6B or wave it. The processor 210A may prevent the voice uttered by the user 5B from being output to the speaker 180A. This may prevent the user 5A from listening to the voice spoken by the user 5B toward the avatar object 6C. In this way, the user 5A can only visually recognize how the user 5C interacts with the avatar object 6B (user 5B) from the viewpoint of the avatar object 6C. Since the billing amount (200 yen) for viewpoint sharing is lower than the billing amount (1000 yen) for moving onto the stage object 1532, the user 5A exchanges the avatar object 6A with the avatar object 6B after viewpoint sharing. I agree that it cannot be done. Furthermore, the user 5A is motivated to try out viewpoint sharing.

FIG. 64 is a diagram showing second virtual space 2711B and view image 6417B according to one embodiment. In FIG. 64(A), avatar object 6B and avatar object 6C are placed on stage object 1532 . Three different avatar objects 6, including avatar object 6A, share the viewpoint of avatar object 6C. The processor 210B places a monitor object 6478 for notifying the user 5B of the number of avatar objects 6 sharing the viewpoint of the avatar object 6C in the second virtual space 2711B in association with the avatar object 6C. A monitor object 6478 is placed above the avatar object 6C. Processor 210B displays on monitor object 6478 the number of avatar objects 6 sharing the viewpoint of avatar object 6C.

The processor 210B, for example, displays a view image 6417B corresponding to the second virtual space 2711B shown in FIG. 64(A) on the monitor 130B as shown in FIG. 64(B). User 5B recognizes that avatar object 6C is watching the performance in front of avatar object 6B by visually recognizing view image 6417B. User 5B also recognizes that three other avatar objects 6 share the viewpoint of avatar object 6C by checking the number displayed on monitor object 6478 . In other words, the user 5B can grasp how many other users 5 have received the performance for the avatar object 6C.

[Movement of avatar object 6A by billing]
FIG. 65 is a diagram showing second virtual space 2711A and view image 6517A according to an embodiment. As shown in FIG. 65(A), after moving the avatar object 6C onto the stage object 1532, the processor 210A moves the avatar object 6C to the second stage if a prescribed condition (for example, a certain period of time has elapsed) is satisfied. It returns to the original position in the virtual space 2711A. At step S5810, the processor 210A ends the viewpoint sharing if the avatar object 6C is returned to its original position. Specifically, processor 210A moves virtual camera 14A to the position of avatar object 6A.

After the viewpoint sharing ends, the processor 210A notifies the user 5A of a message guiding the movement of the avatar object 6A onto the stage object 1532 during the live. Processor 210A, for example, displays the message in tag object 2733A. The message includes text guiding the user 5A to move to the stage object 1532 and the billing amount to be charged to the user 5A when moving the avatar object 6A onto the stage object 1532 . In the example of FIG. 65, the billing amount for moving the avatar object 6A onto the stage object 1532 is 1000 yen.

After displaying the message, processor 210A moves virtual left hand 1531LA into viewing area 15A as shown in FIG. 65(A) in accordance with the movement of user 5A's left hand. The processor 210A, for example, displays a view image 6517A corresponding to the second virtual space 2711A shown in FIG. 65(A) on the monitor 130A as shown in FIG. 65(B). The user 5A recognizes that the sharing of the viewpoint has ended by visually recognizing the field-of-view image 6517A. Further, by confirming the message displayed on tag object 2733A, user 5A recognizes that avatar object 6A can be moved onto stage object 1532 by paying a new charge of 1000 yen.

FIG. 66 is a diagram showing second virtual space 2711A and view image 6617A according to an embodiment. User 5A enjoys the experience of having avatar object 6A share the viewpoint of avatar object 6C and wishes to interact directly with avatar object 6B by moving avatar object 6A onto stage object 1532 . In order to fulfill this desire, the user 5A confirms the message displayed on the tag object 2733A and then performs an operation to select the tag object 2733A. In step S5811, when the processor 210A detects an operation by the user 5A for selecting the tag object 2733A, the processor 210A registers billing information corresponding to stage movement in the tag object 2733A. Specifically, processor 210A newly registers, in tag object 2733A, billing information representing the billing amount (1000 yen) corresponding to stage movement after the sharing of the viewpoint is completed.

In step S5812, when billing information corresponding to stage movement is registered in tag object 2733A, processor 210A moves avatar object 6A onto stage object 1532 as shown in FIG. 66(A). The processor 210A places the avatar object 6A after movement at a position in front of the avatar object 6B on the stage object 1532 . At step S5813, the processor 210A receives from the server 600 the avatar information of the avatar object 6B. In step S5814, the avatar object control module 1424 causes the avatar object 6B to perform a performance (second performance) for the avatar object 6A according to the motion information included in the received avatar information.

Processor 210A, for example, displays field-of-view image 6617A corresponding to second virtual space 2711A shown in FIG. 66(A) on monitor 130A as shown in FIG. 66(B). User 5A recognizes that avatar object 6A has moved onto stage object 1532 by viewing view image 6617A. The user 5A can also enjoy a performance in front of the avatar object 6B and enjoy interaction with the avatar object 6B (in other words, with the user 5B).

In FIG. 66, unlike the case of viewpoint sharing, the avatar object 6A is in front of the avatar object 6B in the second virtual space 2711A. Therefore, the user 5A can hold out the virtual right hand 1531RA toward the avatar object 6B and wave it. Processor 210A outputs the voice uttered by user 5B to speaker 180A. Therefore, the user 5A can hear the voice spoken by the user 5B toward the avatar object 6A. In this way, the user 5A can directly interact with the avatar object 6A with the avatar object 6B unlike in the case of shared viewpoints, so that the user 5A can be more satisfied than in the case of shared viewpoints.

[Main advantages of this embodiment]
The processor 210A can cause the avatar object 6A to share the viewpoint of another avatar object 6 different from the avatar object 6A in the second virtual space 2711A. This allows the user 5A to view the performance of the avatar object 6B not only from the viewpoint of the avatar object 6A, but also from the viewpoint of another avatar object 6 located at a position different from that of the avatar object 6A. For example, even if the user 5A cannot move the avatar object 6A to that position because the avatar object 6C is already at the position where the user 5A wants to move the avatar object 6A, the user 5A can move the viewpoint of the avatar object 6C by sharing the viewpoint. , the performance of the avatar object 6B can be viewed. In this way, the HMD system 100 can further enhance the interest of the user 5A in the second virtual space 2711A.

In this embodiment, the avatar object 6C watches the performance of the avatar object 6C by the avatar object 6B as a result of the user 5C being charged for the stage movement. When the user 5A is billed for viewpoint sharing, the avatar object 6A views the performance for the avatar object 6C as a result of the billing from the viewpoint of the avatar object 6C. This allows the user 5A to experience the performance for the avatar object 6C in advance, albeit incompletely. When the user 5B is satisfied with the performance of the avatar object 6B when the viewpoint is shared, the user 5B wants the avatar object 6A to perform the performance of the avatar object 6B. Thereby, the user 5A allows the user 5A to be billed for the stage movement.

In this way, the HMD system 100 charges the user 5C with the billing amount corresponding to the stage movement, and then bills the user 5A with the billing amount corresponding to the viewpoint sharing. The billing amount can also be billed to the user 5A. In this way, the HMD system 100 can realize a plurality of billing cycles for the user 5A, so that the user 5A can be billed with a larger billing amount. As a result, the user 5B can further increase live sales.

[Modification]
Charging the user 5A for viewpoint sharing is not necessarily required. Processor 210A can allow user 5A to share a view without registering billing information corresponding to view sharing in tag object 2733A. In other words, the processor 210A can cause the avatar object 6A to share the viewpoint of the other avatar object 6 according to the operation by the user 5A at any time during the live. Processor 210A can also cause avatar object 6A to automatically share the viewpoint of other avatar object 6 without detecting an operation by user 5A.

The processor 210A can cause the avatar object 6A to share the viewpoint of other avatar objects 6 that are not placed on the stage object 1532 . For example, when the avatar object 6D is selected by the user 5A, the processor 210A causes the avatar object 6A to share the viewpoint of the avatar object 6D.

The processor 210B can inform the user 5B of the number of other avatar objects 6 sharing a viewpoint for each of the plurality of different avatar objects 6. Thereby, the user 5B can grasp which viewpoint of the avatar object 6 in the second virtual space 2711B is gaining popularity among more users 5 .

Processor 210C does not necessarily need to register billing information corresponding to stage movement in tag object 2733C. The processor 210C can also associate billing information corresponding to stage movement with the avatar object 6C and register it in the avatar object 6C, for example. Instead of registering the billing information corresponding to the stage movement in the tag object 2733C, the processor 210C can perform any other processing related to the billing corresponding to the stage movement (second billing). The other processing is, for example, transmitting billing information corresponding to stage movement to server 600 without registering it in tag object 2733C.

Processor 210A does not necessarily need to register billing information corresponding to view sharing in tag object 2733A. The processor 210A can, for example, associate billing information corresponding to viewpoint sharing with the avatar object 6A and register it in the avatar object 6A. Instead of registering the billing information corresponding to view sharing in tag object 2733A, processor 210A can perform any other processing related to billing corresponding to view sharing (first billing). The other processing is, for example, transmitting billing information corresponding to viewpoint sharing to server 600 without registering it in tag object 2733A.

Processor 210A does not necessarily need to register billing information corresponding to stage movement in tag object 2733A. The processor 210A can, for example, associate billing information corresponding to stage movement with the avatar object 6A and register it in the avatar object 6A. Instead of registering the billing information corresponding to the stage movement in the tag object 2733A, the processor 210A can perform any other processing related to the billing corresponding to the stage movement (third billing). The other processing is, for example, transmitting billing information to server 600 without registering it in tag object 2733A.

The user 5B can perform a first operation for permitting viewpoint sharing. When processor 210B detects the first operation by user 5B, processor 210B transmits detection information indicating detection of the first operation to computer 200A via server 600 . Processor 210A receives detection information transmitted from computer 200B. Processor 210A detects the first operation by user 5B based on the received detection information. The processor 210A enables the avatar object 6A to share the viewpoint of any other avatar object 6 different from the avatar object 6A when the first operation is detected. For example, after detecting the first operation, the processor 210A enables the avatar object 6A to share the viewpoint of any other avatar object 6 different from the avatar object 6A for a certain period of time. The first operation may not be the operation of the user 5B itself, but may be an action of the user 5B such as the content of the speech of the user 5B, or may be information specifying a period during which the viewpoint can be shared. As a result, it is possible to prevent the viewpoint of another avatar object 6 from being shared by the avatar object 6A without the permission of the user 5B. In other words, the user 5B can voluntarily specify the timing at which the avatar object 6A can share the viewpoint of the other avatar object 6. FIG. As a result, the user 5B can control the performance of the live performance in a manner more desired by the user 5B.

Processor 210A can cause avatar object 6A to share the viewpoint of avatar object 6C without moving virtual camera 14A to the position of avatar object 6C. For example, when the processor 210A detects an operation by the user 5A to cause the avatar object 6C to share the viewpoint of the avatar object 6C, the processor 210A transmits request information for requesting viewpoint sharing to the computer 200C via the server 600. . When the request information is received, the processor 210C transmits a view image 6017C (second view image) generated by the virtual camera 14C and corresponding to the view area 15C from the avatar object 6C to the computer 200A via the server 600. do. When processor 210A receives view image 6017C, it temporarily stops controlling virtual camera 14A. Processor 210A also outputs received view image 6017C to monitor 130A. The user 5A can view the performance of the avatar object 6B from the viewpoint of the avatar object 6C by viewing the view image 6017C.

Processor 210A can detect the occurrence of a first event in second virtual space 2711A. The first event is, for example, the avatar object 6B taking some action. The action of the avatar object 6B is to move the avatar object 6B to another position on the stage object 1532, for example. The first event may be that the avatar object 6B performs a prescribed performance on the stage object 1532 for a certain period of time or longer. The first event may alternatively be the movement of the avatar object 6C onto the stage object 1532. FIG. Processor 210A enables avatar object 6A to share the viewpoint of other avatar objects 6 when the first event is detected. As a result, it is possible to prevent the viewpoint of another avatar object 6 from being shared by the avatar object 6A without permission.

After sharing the viewpoint, the processor 210A can control the virtual camera 14A in conjunction with the control of the virtual camera 14C. For example, after moving the virtual camera 14A to the position of the avatar object 6C, the processor 210A receives from the computer 200C via the server 600 control information representing the control details of the position and orientation of the virtual camera 14C. Processor 210A controls virtual camera 14A in conjunction with control of virtual camera 14C according to the received control information. This allows the user 5A to more clearly realize that the avatar object 6A shares the viewpoint of the avatar object 6C.

[Embodiment 4]
In this embodiment, the processor 210B executes processing related to billing and payment for the user 5B by the same method as the method using the tag object 2733A described in the first embodiment. However, without being limited to this, the processor 210B can also execute processing related to billing and payment for the user 5B by any method different from the method according to the first embodiment.

[Avatar selection flow]
FIG. 67 is a sequence chart representing part of the processing performed in HMD set 110B according to one embodiment. FIG. 68 is a diagram showing a third virtual space 6811B and a view image 6817B according to one embodiment. In step S6701, processor 210B of computer 200B (hereinafter simply "processor 210B") defines a third virtual space 6811B as shown in FIG. 68(A). This process corresponds to the process of step S1110 in FIG. Specifically, the processor 210B defines the third virtual space 6811B represented by the virtual space data by specifying the virtual space data. The third virtual space 6811B is a virtual space in which the avatar object 6B is arranged before the live performance in the second virtual space 2711B is started. The third virtual space 6811B is also a virtual space for allowing the user 5B to select an avatar object to be used by the user 5B during a live performance.

At step S6702, the processor 210B creates the virtual camera 14B and places it in the third virtual space 6811B. At step S6703, the processor 210B generates the avatar object 6B including the virtual right hand 1531RB and the virtual left hand 1531LB and places it in the third virtual space 6811B. Avatar object 6B is used by user 5B to select an avatar in third virtual space 6811B to be used by user 5B during a live performance. In this embodiment, the avatar object 6B is not used in the second virtual space 2711B. In step S6704, processor 210B generates UI panel 6850 and pen object 6853 and places them in third virtual space 6811B.

In this embodiment, a plurality of different avatar objects are prepared in the HMD system 100 in advance. The user 5B can select one avatar object to be used live in the second virtual space 2711B from multiple avatar objects.

UI panel 6850 is a type of UI object, and is used by user 5B to cause processor 210B to execute processing for selecting one avatar object from among a plurality of avatar objects. UI panel 6850 includes options 6851 and 6852 located in front of UI panel 6850 . Choices 6851 and 6852 include images showing the appearance of the avatar object that would be selected if that choice were selected by user 5B. Option 6851 is an item for selecting a first avatar object from among multiple avatar objects, and option 6852 is an item for selecting a second avatar object from among multiple avatars. Pen object 6853 is a kind of virtual object, and is grasped and used by virtual right hand 1531 RB or virtual left hand 1531 LB to select option 6851 or 6852 of UI panel 6850 .

In step S6705, the processor 210B determines the position and tilt of the virtual camera 14B in the third virtual space 6811B according to the movement of the HMD 120B. This process corresponds to part of the process of step S1140 in FIG. At step S6706, processor 210B displays view image 17B on monitor 130A. This processing corresponds to the processing of steps S1180 and S1190 in FIG.

The processor 210B, for example, displays a view image 6817B corresponding to the third virtual space 6811B shown in FIG. 68(A) on the monitor 130B as shown in FIG. 68(B). User 5B recognizes that it is necessary to select option 6851 or option 6852 on UI panel 6850 by viewing view image 6817B.

The processing of steps S6705 and S6706 described above (that is, the field-of-view image 17B is updated in accordance with the movement of the HMD 120B) is continuously and repeatedly performed while steps S6707 to S6709, which will be described later, are performed.

At step S6707, the processor 210B detects movement of the right hand of the user 5B based on the output of the right controller 300RB. At step S6708, the processor 210B moves the virtual right hand 1531RB in the third virtual space 6811B in accordance with the detected movement of the right hand of the user 5B. In one aspect, the processor 210B moves the virtual right hand 1531RB in the third virtual space 6811B so as to bring the virtual right hand 1531RB closer to the pen object 6853 in response to the movement of the right hand of the user 5B. After the virtual right hand 1531RB comes sufficiently close to the pen object 6853, the processor 210B selects (grabs) the pen object 6853 with the virtual right hand 1531RB based on the movement of the right hand of the user 5B.

FIG. 69 is a diagram showing a third virtual space 6811B and a view image 6917B according to one embodiment. As shown in FIG. 69A, after the virtual right hand 1531RB selects the pen object 6853, the processor 210B moves the tip of the pen object 6853 closer to the option 6851 based on the movement of the right hand of the user 5B. 3 Move virtual right hand 1531RB and pen object 6853 in virtual space 6811B. The processor 210B detects that the pen object 6853 has selected the option 6851 based on the collision between the tip of the pen object 6853 and the option 6851 . At step S6709, the processor 210B selects the avatar object 7106B corresponding to the selected option 6851 from among the plurality of avatar objects. The appearance and size of avatar object 7106B is different from the appearance and size of avatar object 6B.

Processor 210B, for example, displays view image 6917B corresponding to third virtual space 6811B shown in FIG. 69(A) on monitor 130A as shown in FIG. 69(B). User 5B recognizes that avatar object 7106B has been selected by selecting option 6851 on UI panel 6850 by viewing view image 6917B.

After selecting avatar object 7106B, processor 210B finishes arranging avatar object 6B in third virtual space 6811B. Thereafter, processor 210B continues definition of third virtual space 6811B. Further, the processor 210B keeps placing other virtual objects in the third virtual space 6811B even after the avatar object 6B is no longer placed. Processor 210B suspends provision of third virtual space 6811B to user 5B. Processor 210B further provides a second virtual space 2711B to user 5B by executing a series of processes that cause avatar object 7106B to perform a performance.

A method of selecting an avatar object by the user 5B is not limited to the example described above. The processor 210B may arrange, for example, the first avatar object indicated by the option 6851 and the second avatar object indicated by the option 6852 in the third virtual space 6811B. The user 5B, for example, brings the virtual right hand 1531RB into contact with the first avatar object or the second avatar object placed in the third virtual space 6811B. Processor 210B selects the avatar object that touches virtual right hand 1531RB.

[Performance execution processing flow]
FIG. 70 is a sequence chart representing part of the processing performed in HMD set 110B according to one embodiment. FIG. 71 is a diagram showing second virtual space 2711B and view image 7117A according to an embodiment. At step S7001, the processor 210B defines a second virtual space 2711B as shown in FIG. 71(A). In step S7002, the processor 210B creates the virtual camera 14B and places it in the second virtual space 2711B. In step S7003, the processor 210B creates a stage object 1532 and places it in the second virtual space 2711B. At step S7004, processor 210B receives from server 600 each avatar information of avatar objects 6A, 6C and 6D. At step S7005, processor 210B places avatar objects 6A, 6C, and 6D in second virtual space 2711B based on the received avatar information.

In step S7006, processor 210B arranges avatar object 7106B (first avatar) including virtual right hand 7131RB and virtual left hand 7131LB in second virtual space 2711B in association with user 5B (first user). Specifically, processor 210B places avatar object 7106B on stage object 1532 . Although not shown, the processor 210B generates avatar information of the avatar object 6B and transmits it to the server 600 at arbitrary timing.

At step S7007, the processor 210B creates a tag object 2733B and associates it with the avatar object 7106B. Specifically, processor 210B places tag object 2733B in virtual left hand 7131LB of avatar object 7106B. Processor 210B registers, in tag object 2733B, upper limit billing information representing the upper limit of the billing amount that can be billed to user 5B during a live performance.

After the avatar object 7106B is placed on the stage object 1532, a live performance by the avatar object 7106B is started in the second virtual space 2711B. FIG. 71(A) represents the second virtual space 2711B at the time when the live was started. Processor 210B does not register billing information in tag object 2733B at the start of the live. In step S7008, the processor 210B determines the position and tilt of the virtual camera 14B in the second virtual space 2711B according to the movement of the HMD 120B. Since this process is basically the same as the process in step S2305, detailed description will not be repeated. At step S7009, processor 210B displays view image 17B on monitor 130B. Since this process is basically the same as the process in step S2306, detailed description will not be repeated.

The processor 210B, for example, displays a view image 7117B corresponding to the second virtual space 2711B shown in FIG. 71(A) on the monitor 130B as shown in FIG. 71(B). Processor 210B displays the maximum billing amount indicated by the maximum billing information in tag object 2733B. Further, processor 210B displays 0 yen as the total billing amount in tag object 2733A in response to billing information not being registered in tag object 2733A. User 5B recognizes that avatar objects 6A, 6C, and 6D, who are live viewers, are in front of stage object 1532 by visually recognizing view image 7117B. User 5B also recognizes that the usage fee for avatar object 7106B has not been charged to user 5B at the start of the live.

FIG. 72 is a diagram showing second virtual space 2711A and view image 7217A according to an embodiment. The processor 210A receives the avatar information of the avatar object 6B from the server 600 in real time after the second virtual space 2711B is defined. Processor 210A arranges avatar object 7106B in second virtual space 2711A, as shown in FIG. 72(A), based on the avatar information of avatar object 6B received first. Specifically, processor 210 A places avatar object 7106 B on stage object 1532 .

The processor 210A, for example, displays a view image 7217A corresponding to the second virtual space 2711A shown in FIG. 72(A) on the monitor 130A as shown in FIG. 72(B). User 5A recognizes that avatar object 7106B, a live performer, has appeared on stage object 1532 by visually recognizing view image 7217A. User 5 also recognizes that a live performance by avatar object 7106B is about to start.

FIG. 73 is a diagram illustrating an example of a posture of user 5B according to an embodiment. After starting the live performance, the user 5B moves his or her body so as to take the posture shown in FIG. 73, for example. In step S7010, processor 210B detects the motion of user 5B when taking the posture shown in FIG. 73 based on the output of each motion sensor.

FIG. 74 is a diagram showing second virtual space 2711B and view image 7417B according to one embodiment. In step S7011, the processor 210B causes the avatar object 7106B to perform in the second virtual space 2711B according to the movement of the user 5B, as shown in FIG. 74(A). Specifically, the processor 210B causes the body movements of the user 5B shown in FIG. 73 to be reflected in the avatar object 7106B. Processor 210B transmits avatar information including movement information of avatar object 7106B when avatar object 7106B performs a performance to server 600 in real time.

The processor 210B, for example, displays a view image 7417B corresponding to the second virtual space 2711B shown in FIG. 74(A) on the monitor 130B as shown in FIG. 74(B). User 5B confirms the reactions of avatar objects 6A, 6C, and 6D to the performance by viewing view image 7417B.

FIG. 75 is a diagram showing second virtual space 2711A and view image 7517A according to an embodiment. After the performance of avatar object 7106B in second virtual space 2711B, processor 210A receives avatar information of avatar object 7106B from server 600 in real time. Processor 210A causes avatar object 7106B to perform a performance based on the motion information contained in the received avatar information. This allows the processor 210A to cause the avatar object 7106B to perform the performance according to the movement of the user 5B shown in FIG. Processor 210A causes avatar object 6A to view the performance by avatar object 7106B in second virtual space 2711A. The processor 210A, for example, displays a view image 7517A corresponding to the second virtual space 2711A shown in FIG. 75(A) on the monitor 130A as shown in FIG. 75(B). User 5A recognizes that avatar object 7106B performed the first performance on stage object 1532 by visually recognizing view image 7517A. Thereby, the user 5A can enjoy the performance by the avatar object 7106B from the viewpoint of the avatar object 6A.

[Registration of billing information]
FIG. 76 is a diagram showing second virtual space 2711B and view image 7617B according to an embodiment. In the second virtual space 2711B shown in FIG. 76(A), the live performance of the avatar object 7106B has already ended. In step S7012, the processor 210B registers billing information according to the performance of the avatar object 7106B in the second virtual space 2711B in the tag object 2733B. Processor 210B, for example, registers billing information representing the billing amount according to the time during which avatar object 7106B is used in second virtual space 2711B in tag object 2733B after the live performance ends. Processor 210B multiplies the elapsed time from the start of the live performance to the end of the live performance by the unit charge per fixed time set for avatar object 7106B, thereby calculating the charge for using avatar object 7106B. In FIG. 76, 1000 yen is calculated as the billing amount. Processor 210B registers billing information representing the calculated billing amount in tag object 2733B. Processor 210B visualizes the registered billing information on tag object 2733B.

The processor 210B, for example, displays a view image 7617B corresponding to the second virtual space 2711B shown in FIG. 76(A) on the monitor 130B as shown in FIG. 76(B). By visually recognizing the view image 7617B, the user 5B understands that 1000 yen has been charged to the user 5B as the usage fee for the avatar object 7106B.

After confirming the total billing amount at the end of the live performance, the user 5B performs an operation for leaving the avatar object 7106B from the second virtual space 2711B. This operation is, for example, the user 5B pressing any button of the right controller 300RB. In step S7013, the processor 210B causes the avatar object 7106B to leave the second virtual space 2711B in response to detecting the operation of the user 5B for leaving the avatar object 7106B. Thereby, the processor 210B finishes providing the second virtual space 2711B to the user 5B.

FIG. 77 is a diagram showing a third virtual space 6811B and a view image 7717B according to one embodiment. In step S7014, the processor 210B arranges the avatar object 6B again in the third virtual space 6811B as shown in FIG. 77(A). At this time, processor 210B does not place tag object 2733B in virtual left hand 1531LB. In step S7015, processor 210B executes processing related to payment according to the billing information registered in tag object 2733B. Since this process is basically the same as the process in step S2618, detailed description will not be repeated.

The processor 210B, for example, displays a view image 7717B corresponding to the third virtual space 6811B shown in FIG. 77(A) on the monitor 130B as shown in FIG. 77(B). Server 600 notifies computer 200B that the payment has been completed. Processor 210B notifies user 5B of the completion of payment on view image 7717B in response to notification of completion of payment from server 600 . Processor 210B displays, for example, notification image 3735 including a message explaining that payment has been completed on monitor 130B while superimposing it on view image 7717B. Processor 210B can also place a virtual object having the same appearance as notification image 3735 at any position within view area 15B in second virtual space 2711B. Processor 210B may alternatively include notification image 3735 at any location within view image 7717B when view image 7717B is generated. By visually recognizing the view image 7717B, the user 5B recognizes that the avatar object 6B has been placed again in the third virtual space 6811B after the end of the live performance. User 5B also sees notification image 3735 to understand that the payment of the billing amount corresponding to the usage fee of avatar object 7106B has been completed. User 5B also recognizes that since tag object 2733B is not placed on virtual left hand 7131LB in third virtual space 6811B, no further usage fee for avatar object 7106B will be charged.

At step S7016, the processor 210B determines a reward for the creator of the avatar object 7106B. Processor 210B determines the reward, in particular, according to billing information registered in tag object 2733B. Since the billing information is registered according to the performance of avatar object 7106B, determining the reward according to the billing information is synonymous with determining the reward according to the performance of avatar object 7106B. In the example of FIG. 76, billing information representing the billing amount according to the time during which avatar object 7106B is used in second virtual space 2711B is registered. Therefore, processor 210B determines a reward according to the time avatar object 7106B is used in second virtual space 2711B.

At step S7017, processor 210B performs processing to pay the determined reward to the creator of avatar object 7106B. Processor 210B, for example, notifies server 600 of the determined reward. After executing the payment, server 600 pays the notified reward to the creator of avatar object 7106B. Server 600 can pay a reward to the user of avatar object 7106B each time a live performance in which avatar object 7106B is used ends. Alternatively, the server 600 can collectively pay the creator of the avatar object 7106B a reward according to the total usage fee of the avatar object 7106B within a certain period of time (for example, one month) when the certain period of time has elapsed. Processor 210B, rather than server 600, may pay the creator of avatar object 7106B a reward according to billing information.

[Main advantages of this embodiment]
As described above, processor 210B registers billing information corresponding to the performance of avatar object 7106B in second virtual space 2711B in tag object 2733B. Thus, the HMD system 100 can provide the user 5B with a suitable billing scheme in the second virtual space 2711B.

The processor 210B causes the avatar object selected by the user 5B to perform the performance according to the position information of the user 5B and the rotation direction of the joints. As a result, it is possible to accurately reflect the movement of the user 5B even on an avatar object whose shape is significantly different from that of the user 5B. Thus, in the HMD system 100, there is no need to restrict the shape of the avatar object selectable by the user 5B to a specific shape. Therefore, various avatar objects with different shapes can be prepared for the HMD system 100 . As a result, it is possible to increase the number of avatar object options that can be selected by the user 5B, thereby increasing the variety of performers' live performances.

A creator of an avatar object can receive a reward according to the performance of the avatar object when the avatar object created by the creator is used by a performer. Therefore, the HMD system 100 can motivate many creators to create new avatar objects that can be used in the HMD system 100 . As a result, the types of avatar objects prepared for the HMD system 100 can be increased. As a result, it is possible to increase the number of avatar object options that can be selected by the performer, thereby increasing the variety of live performances by the performer.

[Modification]
Processor 210B need not necessarily register billing information corresponding to avatar use in tag object 2733B. Processor 210B may, for example, associate billing information corresponding to avatar use with avatar object 7106B and register it with avatar object 7106B. Instead of registering billing information for avatar use in tag object 2733B, processor 210B may perform any other processing related to billing for avatar use. The other processing is, for example, transmitting billing information corresponding to avatar use to server 600 without registering it in tag object 2733B.

The processor 210B can execute processing related to billing corresponding to the billing amount according to the live sales in the second virtual space 2711B. The processor 210B can further determine rewards according to live sales in the second virtual space 2711B. The processor 210B, for example, calculates part of the sales as a billing amount according to the live sales. Processor 210B registers billing information representing the billing amount according to sales in tag object 2733B. Accordingly, the higher the live sales, the higher the billing amount charged to the user 5B as the usage fee for the avatar object 7106B. Also, the higher the live sales, the higher the reward paid to the creator of avatar object 7106B. This allows HMD system 100 to motivate creators to create popular avatar objects 7106B that will lead to increased live sales.

Processor 210B can execute processing related to charging corresponding to the charging amount according to the contents of second virtual space 2711B in which avatar object 7106B is used. Processor 210B can further determine a reward according to the contents of second virtual space 2711B. For example, the larger the size of the second virtual space 2711B in which the avatar object 7106B is used, the processor 210B calculates a higher fee for using the avatar object 7106B. Processor 210B registers billing information representing the calculated billing amount in tag object 2733B. The larger the size of the second virtual space 2711B in which the avatar object 7106B is used, the higher the reward paid to the creator of the avatar object 7106B. This allows the HMD system 100 to motivate creators to create popular avatar objects 7106B to be used in the larger size second virtual space 2711B.

Processor 210B can generate a usage history of avatar object 7106B in second virtual space 2711B. Processor 210B generates, as a usage history, for example, a usage time history of avatar object 7106B in second virtual space 2711B. The processor 210A generates, as a usage history, for example, a history of live sales in which the avatar object 7106B was used in the second virtual space 2711B. Processor 210B, for example, generates a history of second virtual space 2711A in which avatar object 7106B has been used. Processor 210B notifies the creator of avatar object 7106B of the generated usage history. The creator can grasp the details of the past use of avatar object 7106B in second virtual space 2711B by viewing the notified usage history of avatar object 7106B. The creator can newly create a better avatar object by referring to the grasped usage status.

User 5B can also rent avatar object 7106B from the creator. In this case, the processor 210B charges the user 5B a charge corresponding to the rental period of the avatar object 7106B. Processor 210B also determines a reward for the rental period of avatar object 7106B.

[Embodiment 5]
In this embodiment, the processor 210A executes processing related to billing and settlement for the user 5A by the method using the tag object 2733A described in the first embodiment. However, without being limited to this, the processor 210A can also execute processing related to billing and payment for the user 5A by any method different from the method according to the first embodiment.

[Request for second performance]
FIG. 78 is a sequence chart showing an example of processing executed in the HMD system 100. FIG. In the following, in response to a second performance request made by the user 5A (first user), the user 5B (second user) causes the avatar object 6B (second avatar) to perform the second performance in the virtual space 11B. A series of processes for The sequence chart shown in FIG. 78 shows processing after step S2614 shown in FIG. 26 is executed. The request for the second performance may be made by the user 5C or by the user 5D.

FIG. 79 is a diagram showing second virtual space 2711A and view image 7917A according to one embodiment. The processor 210A notifies the user 5A during the live of a message guiding the user 5A to request the second performance for the avatar object 6B. Processor 210A, for example, displays the message in tag object 2733A. The message includes a text that guides the user 5A to the content of the second performance, and the billing amount required for the user 5A to request the second performance. In the example of FIG. 79, the content of the second performance is a pointing pose, and the billing amount required to request the pointing pose is 500 yen. A pointing pose is, for example, a performance in which the avatar object 6B points to the avatar object 6A (first avatar). Thus, the second performance may be a performance directed to the avatar object 6 associated with the user 5 who requested the second performance.

After the message is displayed on tag object 2733A, processor 210A moves virtual left hand 1531LA into viewing area 15A, as shown in FIG. 79(A), in response to the movement of user's 5A left hand. Processor 210A, for example, displays view image 7917A corresponding to second virtual space 2711A shown in FIG. 79(A) on monitor 130A as shown in FIG. 79(B). User 5A confirms the message displayed on tag object 2733A by visually recognizing view image 7917A. Thereby, the user 5A recognizes that the pointing pose can be requested to the avatar object 6B by paying a new charge of 500 yen.

After confirming the message displayed on the tag object 2733A, the user 5A performs an operation to request the pointing pose. Since this operation has been described with reference to FIG. 34 in Embodiment 1, it will not be repeated here. Processor 210A detects user 5A's operation for requesting the second performance in response to selection of tag object 2733A in which a message guiding the request for the second performance is displayed.

FIG. 80 is a diagram showing second virtual space 2711A and view image 8017A according to an embodiment. At step S7801, processor 210A registers billing information corresponding to the request for the second performance in tag object 2733A. In the example of FIG. 79, when avatar object 6A selects tag object 2733A with virtual right hand 1531RA, processor 210A sends billing information representing the billing amount (500 yen) required for requesting the pointing pose to tag object 2733A. Register again. It should be noted that the settlement of the billing amount indicated by the registered billing information may be performed by the method described in the first embodiment.

After the billing information is registered, processor 210A re-visualizes the billing information on tag object 2733A. The details of this process have been described with reference to FIG. 35 in the first embodiment, and will not be repeated here. Processor 210A displays, for example, view image 8017A corresponding to second virtual space 2711A shown in FIG. 80(A) on monitor 130A as shown in FIG. 80(B). By visually recognizing the view image 8017A, the user 5A recognizes that the current total billing amount has increased to 2000 yen as a result of requesting the pointing pose.

At step S7802, processor 210A sends a request for a second performance to computer 200B. Specifically, processor 210A transmits the request to server 600 . The server 600 transmits the received information about the second performance to the computer 200B by synchronous processing. It has already been described in the first embodiment with reference to FIG. 17 that information is transmitted and received between the computer 200A and the computer 200B via the server 600. FIG. Therefore, FIG. 78 does not show the processing executed by the server 600 .

FIG. 81 is a diagram showing second virtual space 2711B and view image 8117B according to an embodiment. In the example of FIG. 81, the avatar object 6B faces the avatar object 6C (the third avatar), and the avatar object 6A is placed at the first position 8176 outside the viewing area. That is, the avatar object 6A is not placed within the field of view 15B. Processor 210B, for example, displays view image 8117B corresponding to second virtual space 2711B shown in FIG. 81(A) on monitor 130A as shown in FIG. 81(B).

At step S7811, the processor 210B receives the request for the second performance sent from the processor 210A. At step S7812, the processor 210B notifies the user 5B of information regarding the requested second performance. Information about the requested second performance may be, for example, the content of the requested second performance. Processor 210B may also notify user 5B of information indicating the position (hereinafter, "first position") in second virtual space 2711B of avatar object 6A associated with user 5A who requested the second performance. good. Processor 210B, as an example, notifies information about the requested second performance and information indicating first location 8176 on view image 8117B. Processor 210B includes notification image 8181 and arrow image 8182 in view image 8117B and displays it on monitor 130, as shown in FIG. 81B, for example. Processor 210B can also place a virtual object having the same appearance as notification image 8181 in second virtual space 2711B. Processor 210B can also place a virtual object having the same appearance as arrow image 8182 in second virtual space 2711B.

The notification image 8181 includes, for example, that the second performance has been requested and the content of the requested second performance, as shown in FIG. 81(B). This allows the user 5B to recognize that the second performance has been requested and the contents of the requested second performance.

The arrow image 8182 is an image showing the first position 8176 when the avatar object 6A is not arranged in the view area 15B, as shown in FIG. 81(A). Specifically, arrow image 8182 is an image showing a second direction in view image 8117B, corresponding to first direction 8178 from second position 8177 within view area 15B toward first position 8176 where avatar object 6A is located. be. That is, in the example of FIG. 81, the arrow image 8182 is an image of an arrow pointing from left to right (rightward) in the view image 8117B. Since the arrow image 8182 is included in the view image 8117B, the user 5B moves his/her head to the right so that the avatar object 6 ( It can be easily recognized that the avatar object 6A) is included.

Processor 210B may specify first position 8176 of avatar object 6A in second virtual space 2711B using information identifying avatar object 6A included in the request. The information may be, for example, the ID of the user 5A or the avatar object 6A. The processor 210B then determines whether or not the avatar object 6A is placed in the view area 15B.

FIG. 82 is a diagram showing second virtual space 2711B and view image 8217B according to an embodiment. It is assumed that the avatar object 6B faces the direction shown in FIG. 82(A) as a result of the user 5B moving his/her head. Thereby, as shown in FIG. 82(A), the avatar object 6A is included in the view area 15B. In other words, processor 210B determines that avatar object 6A is included in viewing area 15B. Processor 210B displays view image 8117B corresponding to second virtual space 2711B shown in FIG. 82(A), including frame image 8283 surrounding avatar object 6A as shown in FIG. 82(B), on monitor 130B. do. This allows the user 5B to easily identify the avatar object 6 associated with the user 5 who requested the second performance. The processor 210B can also place a virtual object having the same appearance as the frame image 8283 in the second virtual space 2711B so as to surround the avatar object 6A.

FIG. 83 is a diagram showing an example of the posture of user 5B according to an embodiment. At step S7813, the processor 210B detects movement of the user 5B corresponding to the second performance. The user 5B who has viewed the notification image 8181 shown in FIG. 81B moves his/her body so as to take the posture shown in FIG. At this time, it is assumed that the user 5B operates (for example, presses) a button (not shown in FIG. 83) provided on the right controller 300RB. The posture shown in FIG. 83 is the posture corresponding to the second performance requested by the user 5A, that is, the pointing pose.

FIG. 84 is a diagram showing second virtual space 2711B and view image 8417B according to an embodiment. At step S7814, the processor 210B causes the avatar object 6B to perform the second performance according to the detected motion. As an example, processor 210B points to avatar object 6B as shown in FIG. 84(A) in response to the movement of user 5B taking the posture shown in FIG. perform a pose. Specifically, the processor 210B causes the avatar object 6B to extend the virtual right hand 1531RB forward and point to the extended right hand. Processor 210B, for example, displays view image 8417B corresponding to second virtual space 2711B shown in FIG. 84(A) on monitor 130A as shown in FIG. 84(B). The user 5B can recognize that the avatar object 6B is performing the pointing pose by visually recognizing the view image 8417B including the virtual right hand 1531RB.

When processor 210B causes avatar object 6B to perform the second performance, processor 210B hides notification image 8181 and frame image 8283 in view image 8417B as shown in FIG. 84(B). As an example, the processor 210B compares the information indicating the posture taken by the user 5B with the information indicating the posture of the second performance stored in the storage 230 in association with the contents of the second performance. As a result of the comparison, if the matching rate of the two pieces of information is equal to or higher than the predetermined threshold, processor 210B determines that avatar object 6B has performed the second performance, and places notification image 8181 and frame image 8283 in view image 8417B. Hide.

At step S7815, processor 210B transmits the avatar information corresponding to the execution of the second performance to computer 200A. Specifically, processor 210B transmits the avatar information to server 600 . Server 600 transmits the received avatar information to computer 200A through synchronization processing.

FIG. 85 is a diagram showing second virtual space 2711A and view image 8517A according to an embodiment. At step S7803, the processor 210A receives the avatar information transmitted from the processor 210B in real time. At step S7804, the processor 210A causes the avatar object 6B to perform the second performance based on the motion information included in the received avatar information. Thereby, the processor 210A can cause the avatar object 6B to perform the second performance according to the movement of the user 5B shown in FIG.

The processor 210A causes the avatar object 6A to view the second performance by the avatar object 6B in the second virtual space 2711A. Processor 210A, for example, displays view image 8517A corresponding to second virtual space 2711A shown in FIG. 85(A) on monitor 130A as shown in FIG. 85(B). The user 5A recognizes that the avatar object 6B performed the second performance requested by the user 5A by viewing the view image 8517A. As a result, the user 5A feels happy that the avatar object 6B performed the second performance requested by the user 5A. Specifically, the user 5A can virtually experience the performance of a performer directed at him/herself, which is difficult to experience in a real live performance, by paying the fee. As a result, the user 5A can be made to feel joy, and the user 5A can be more satisfied. In other words, it is possible to further enhance the interest of the live performance in the virtual space.

[Modification]
It may be possible for the user 5A to request the second performance without specifying the content. In this case, the processor 210B that received the request only notifies the user 5B that the second performance was requested. The user 5B himself decides the second performance to be executed by the avatar object 6B, and takes a posture corresponding to the decided second performance.

The billing amount required for requesting the second performance may be set according to the content of the requested second performance. As an example, a high billing amount may be set for a second performance that is difficult for the user 5B to execute, such as taking a long time to execute, being complicated, or requiring a large number of man-hours (operations).

The content of the second performance is not limited to the example described above. For example, the second performance may be the avatar object 6B moving from the stage object 1532, approaching the requesting avatar object 6, and performing any or requested content. The movement of the avatar object 6B from the stage object 1532 may be performed by the movement method described in the second embodiment. Also, the second performance may be a performance that does not target the avatar object 6 associated with the user 5 who requested the second performance. An example of such performance is jumping on stage object 1532 .

The notification image 8181 and the arrow image 8182 or the frame image 8283 may be included in the view image 17B at preset timing instead of immediately after receiving the request. The preset timing may be, for example, the timing at which the period set by the user 5B as the period for performing the second performance starts.

The billing amount required for requesting the second performance may be settled, for example, when an operation for requesting the second performance is performed.

The second performance request may be, for example, a bidding scheme. In this example, no billing amount is set for requesting the second performance. Processor 210A accepts an input of an amount of money (hereinafter, "first amount of money") as consideration for the second performance. In addition, if the first amount of money is higher than the amount (second amount of money) as consideration for the second performance input by the user 5 (third user) other than the user 5A, the processor 210A sets the billing amount representing the first amount of money. Register information in tag object 2733A. For example, the server 600 may compare the first amount and the second amount. That is, processor 210A transmits information indicating the first amount of money to server 600 . If the first amount is higher than the second amount, server 600 notifies computer 200A to that effect. When processor 210A receives the notification, processor 210A registers billing information representing the first amount of money in tag object 2733A.

If the request for the second performance is based on the bidding system, the user 5B may preset the content and execution timing of the second performance. Alternatively, the virtual space 11 may be divided into a plurality of areas and a bid may be made for each area. In this case, processor 210B sums up the billing amount for each area and includes an image showing the area with the highest summed amount in view image 17B. In the second performance in this case, for example, the avatar object 6B moves to the stage object arranged in the area with the highest sum of money among the stage objects arranged in each area, and the first performance is executed. There may be.

If the second performance request is bidding, a minimum bid amount may be set. If both the first price and the second price are lower than the minimum bid price, server 600 may notify computer 200B to that effect. When processor 210B receives the notification, processor 210B may notify user 5B that the amount bid has not reached the minimum bid amount. In this case, the user 5B does not have to cause the avatar object 6B to perform the second performance. For example, an example will be described in which the second performance is to approach the avatar object 6 associated with the requesting user 5 and perform the requested content. In this example, if the first amount and the second amount are lower than the minimum bid amount, the user 5B may cause the avatar object 6B to continue the first performance on the stage object 1532 instead of the second performance. .

Also, the request for the second performance may be made before the first performance by the avatar object 6B is started. As an example, the request may be made before user 5A begins a live virtual experience. In this example, the request may be made in the form of so-called crowdfunding. For example, before the start of the live virtual experience, the user 5B approaches the avatar object 6B and invites users 5 who wish to perform a predetermined performance to pay the charge. When the billing amount exceeds the set amount, the avatar object 6B approaches the avatar object 6 associated with the user 5 who has paid the billing amount and performs a predetermined performance live.

[Embodiment 6]
In this embodiment, the processor 210A executes processing related to billing and settlement for the user 5A by the method using the tag object 2733A described in the first embodiment. However, without being limited to this, the processor 210A can also execute processing related to billing and payment for the user 5A by any method different from the method according to the first embodiment.

FIG. 86 is a sequence chart showing an example of processing executed in the HMD system 100. FIG. Below, the user 5A (first user) controls the appearance of the avatar object 6A (first avatar) in the virtual space 11A, and performs the second performance targeting the avatar object 6A with the avatar object 6B (second avatar). A series of processes to be executed will be described. The sequence chart shown in FIG. 86 shows processing after step S2614 shown in FIG. 26 is executed. The control of the appearance of the avatar object 6 may be performed by the user 5C or the user 5D.

FIG. 87 is a diagram showing second virtual space 2711A and view image 8717A according to an embodiment. The processor 210A notifies the user 5A of a message during the live to guide the user 5A to control the appearance of the avatar object 6A. As an example, controlling the appearance is to make the appearance of the avatar object 6</b>A stand out (emphasize) from other avatar objects 6 . Processor 210A, for example, displays the message in tag object 2733A. The message includes text guiding appearance enhancement, and the billing amount required for the user 5A to enhance the appearance of the avatar object 6A.

After the message is displayed on tag object 2733A, processor 210A moves virtual left hand 1531LA into viewing area 15A as shown in FIG. 87(A) in accordance with the movement of user's 5A left hand. Processor 210A, for example, displays view image 8717A corresponding to second virtual space 2711A shown in FIG. 87(A) on monitor 130A as shown in FIG. 87(B). User 5A confirms the message displayed on tag object 2733A by visually recognizing view image 8717A. Thereby, the user 5A recognizes that the appearance of the avatar object 6A can be emphasized by paying a new charge of 500 yen.

After confirming the message displayed on tag object 2733A, user 5A performs an operation to emphasize the appearance of avatar object 6A. Since this operation has been described with reference to FIG. 34 in Embodiment 1, it will not be repeated here. Processor 210A detects user 5A's operation for emphasizing the appearance of avatar object 6A in response to selection of tag object 2733A displaying a message guiding emphasis of the appearance of avatar object 6A.

FIG. 88 is a diagram showing second virtual space 2711A and view image 8817A according to an embodiment. At step S8601, processor 210A registers billing information corresponding to highlighting of avatar object 6A in tag object 2733A. In the example of FIG. 88, when avatar object 6A selects tag object 2733A with virtual right hand 1531RA, processor 210A transmits charging information representing the charging amount (500 yen) required for emphasizing avatar object 6A to tag object 2733A. to register anew. It should be noted that the settlement of the billing amount represented by the registered billing information may be performed by the method described in the first embodiment.

After the billing information is registered, processor 210A re-visualizes the billing information on tag object 2733A. Details of this process have been described with reference to FIG. 35 in the first embodiment, and therefore will not be repeated here. Processor 210A, for example, displays view image 8817A corresponding to second virtual space 2711A shown in FIG. 88(A) on monitor 130A as shown in FIG. 88(B). User 5A recognizes that the current total billing amount has increased to 2000 yen as a result of emphasizing avatar object 6A by visually recognizing view image 8817A.

FIG. 89 is a diagram showing second virtual space 2711B and view image 8117B according to an embodiment. In step S8602, the processor 210A changes the appearance of the avatar object 6A to the avatar object 6 (third avatar, hereinafter "other avatar object 6") associated with the viewer-side user 5 (third user) other than the user 5A. ) to make it stand out more. As an example, processor 210A controls the appearance color of avatar object 6A so that it stands out from other avatar objects 6. FIG. Specifically, processor 210A changes the appearance color of avatar object 6A by applying texture 4137A to avatar object 6A, as shown in FIG. 89(A). The texture 4137A is a texture colored in the first color corresponding to the user 5A having been charged once with the charge (500 yen) necessary for emphasizing the avatar object 6A. Processor 210A, for example, displays view image 8917A corresponding to second virtual space 2711A shown in FIG. 89(A) on monitor 130A as shown in FIG. 89(B).

The processor 210A may make the avatar object 6A more conspicuous each time the user 5A is charged the amount necessary for highlighting the avatar object 6A. As an example, the color of the appearance of the avatar object 6A may be changed so as to stand out as the number of times the billing amount is charged increases. For example, if the billing amount is charged twice, a texture colored in a second color that is more conspicuous than the first color may be attached to the avatar object 6A. Also, when the billing amount is charged three times, a texture colored in a third color that is more conspicuous than the second color may be attached to the avatar object 6A. The first color is for example copper, the second color is for example silver and the third color is for example gold.

In step S8603, processor 210A transmits avatar information (first information) of appearance-controlled avatar object 6A to computer 200B. Specifically, processor 210A transmits the avatar information to server 600 . Server 600 transmits the received avatar information to computer 200B by synchronous processing. Note that the avatar information includes first user information indicating the user 5A. The first user information may be, for example, an ID indicating the user 5A or an ID indicating the avatar object 6A.

FIG. 90 is a diagram showing second virtual space 2711B and view image 9017B according to an embodiment. At step S8611, the processor 210B receives the avatar information transmitted from the processor 210A in real time. At step S8612, the processor 210B makes the appearance of the avatar object 6A stand out from the other avatar objects 6 based on the received avatar information. Specifically, processor 210B changes the appearance color of avatar object 6A by applying texture 4137A to avatar object 6A, as shown in FIG. 90(A), based on the received avatar information. Processor 210B displays, for example, view image 9017B corresponding to second virtual space 2711B shown in FIG. 90(A) on monitor 130A as shown in FIG. 90(B). User 5B (second user) recognizes that the appearance of avatar object 6A has been changed to be more conspicuous than other avatar objects 6 by visually recognizing view image 9017B. That is, user 5B recognizes that user 5A has been charged for making avatar object 6A stand out.

At step S8613, the processor 210B notifies the user 5B of the recommended second performance. Specifically, the processor 210B selects at least one second performance from among the plurality of second performances and preferentially notifies the user 5B. As an example, processor 210B selects and notifies user 5B of a second performance that will result in billing for user 5A with whom appearance-controlled avatar object 6A is associated.

FIG. 91 is a diagram showing an example of data referred to by processor 210B in order to notify user 5B of at least one second performance. The data is stored, for example, in the storage 230B. The "second performance" column stores information (hereinafter referred to as "second information") indicating the content of each second performance such as "pointing pose" and "waving hand". The column of "first user information" stores the first user information associated with each piece of second information. In FIG. 91, reference numerals 5A, 5C, and 5D attached to "user" in this specification are used as an example of the first user information.

The first user information is associated with the second information by processor 210B prior to the processing shown in FIG. As an example, after the avatar object 6B has performed the second performance, the processor 210B waits for reception of avatar information for a predetermined period of time from the performance of the second performance. If the avatar information is received within the predetermined time, the processor 210B associates the first user information contained in the avatar information with the second information indicating the second performance performed immediately before. In other words, if the user 5 on the audience side registers the billing information to make the avatar object 6 associated with him/herself stand out within a predetermined time after the second performance is performed, the processor 210B executes immediately before the second performance. The resulting second performance is set as the second performance leading to billing by the user 5 .

Processor 210B notifies user 5B of the second information associated with the first user information included in the received avatar information. As an example, when avatar information including first user information "5A" indicating user 5A is received, processor 210B refers to the data shown in FIG. "Pointing pose" is selected and notified to the user 5B. In other words, the processor 210B notifies the user 5B of the second performance that actually led to billing for the user 5A as a result of having the avatar object 6B performed in the past as the second performance that is highly likely to lead to billing. As shown in FIG. 90B, processor 210B displays view image 9017B including notification image 9084 including text indicating the content of the selected second performance on monitor 130B. Processor 210B can also place a virtual object having the same appearance as notification image 9084 in second virtual space 2711B. By visually recognizing the notification image 9084, the user 5B can recognize the second performance that is highly likely to lead to billing for the user 5A.

At step S8614, the processor 210B detects movement of the user 5B corresponding to the second performance. At step S8615, the processor 210B causes the avatar object 6B to perform the second performance according to the detected movement of the user 5B. The processor 210B causes the avatar object 6B to perform a pointing pose according to the movement of the user 5B. Details of the pointing pose and details of the movement of the user 5B to cause the avatar object 6B to perform the pointing pose have already been described in Embodiment 5 with reference to FIGS. 83 and 84, and will not be repeated here. .

After causing the avatar object 6B to perform the second performance, the processor 210B hides the notification image 9084 in the view image 17B. As an example of this process, the processor 210B may perform the same process as the process of not displaying the notification image 8181 and the frame image 8283 in the view image 17B described in the fifth embodiment.

At step S8616, processor 210B transmits the avatar information corresponding to the execution of the second performance to computer 200A. Specifically, processor 210B transmits the avatar information to server 600 . Server 600 transmits the received avatar information to computer 200A through synchronization processing.

FIG. 92 is a diagram showing second virtual space 2711A and view image 9217A according to an embodiment. At step S8604, the processor 210A receives the avatar information transmitted from the processor 210B in real time. At step S8605, the processor 210A causes the avatar object 6B to perform the second performance based on the motion information included in the received avatar information. Thereby, the processor 210A can cause the avatar object 6B to perform the second performance according to the movement of the user 5B.

The processor 210A causes the avatar object 6A to view the second performance by the avatar object 6B in the second virtual space 2711A. Processor 210A, for example, displays view image 9217A corresponding to second virtual space 2711A shown in FIG. 92(A) on monitor 130A as shown in FIG. 92(B). User 5A recognizes that avatar object 6B has performed the second performance requested by user 5A by visually recognizing view image 8517A. As a result, the user 5A made the appearance of the avatar object 6A associated with him stand out more than the avatar object 6 on the other viewer side by paying the billing amount, so that the avatar object 6B performed the second performance. Recognize and feel joy. In other words, it is possible to further enhance the interest of the live performance in the virtual space. Also, the user 5A may wish to perform the second performance directed to him/herself again, and may pay a further billing amount in order to make the avatar object 6A more conspicuous. This allows the user 5B to earn more income.

Also, consider a case where the user 5B causes the avatar object 6B to perform the notified second performance (recommended second performance). The second performance is a second performance that, when performed in the past, led to the user 5A paying the billing amount, in other words, a second performance that the user 5A likes. For the user 5A, by making the appearance of the avatar object 6A associated with him/herself stand out from the avatar objects 6 on the other viewer side by paying the billing amount, the avatar object 6B can be displayed in his/her preferred second I feel like I delivered a performance. This makes the user 5A more happy. In addition, the user 5A is more likely to pay the billing amount for making the avatar object 6A stand out more.

[Modification]
The control of the appearance of the avatar object 6A is not limited to the example of changing the color of the avatar object 6A described above, as long as the appearance control is such that the avatar object 6A stands out from the avatar object 6B. For example, the processor 210A may change the size or shape of the avatar object 6A. Processor 210A may also control the appearance of other avatar objects 6 to make avatar object 6A stand out. For example, processor 210A may make other avatar objects 6 transparent for a period of time.

The conditions for selecting the second performance to be notified to the user 5B are not limited to the example described above, that is, the "second performance leading to billing". Specifically, the processor 210B may further narrow down the second performances selected under the condition under another condition. The other condition may be, for example, "the second performance that the user 5B is good at". That is, when there are multiple second performances that have led to billing for user 5A, processor 210B may select a second performance that user 5B is good at and notify user 5B of it. The second performance that the user 5B is good at may be preset by the user 5B.

Also, the other condition may be, for example, "second performance that does not lower the evaluation of other users 5". In other words, when there are multiple second performances that have led to billing for user 5A, processor 210B may exclude second performances whose evaluation values indicating evaluations performed in past live performances are less than a threshold from options. The evaluation value is calculated, for example, based on individual evaluations made by each user on the viewer side. The individual evaluation is, for example, a value input by the viewer side user 5 by performing a predetermined operation with the right hand or the left hand, and the value is transmitted to the computer 200B. For example, the processor 210B calculates the evaluation value by averaging the received values. As described above, it is possible to prevent the evaluation of the avatar object 6B from being lowered or being criticized on the Internet as a result of performing the second performance leading to billing.

A viewer user 5 may enter comments on the second performance along with individual evaluations. Processor 210A may place a bulletin board object (not shown) in second virtual space 2711A and display the comment on the bulletin board object.

Further, the processor 210B may refer to the gender ratio, age distribution, etc. of the users 5 on the viewer side when selecting the second performance to be notified to the user 5B. For example, it is assumed that among the plurality of second performances that have led to billing for user 5A, there is a second performance whose evaluation value indicating the evaluation when performed in past live performances is less than the threshold. Assume that the evaluation value of the second performance is less than the threshold value in the women-only live performance, and the evaluation value is equal to or greater than the threshold value in the men-only live performance. Here, it is assumed that 80% of the users 5 on the audience side of this live show are male. In this case, the processor 210B may notify the user 5B that the second performance is the second performance that is well received by men, without excluding it from the options.

[Embodiment 7]
In this embodiment, the processor 210A executes processing related to billing and settlement for the user 5A by the method using the tag object 2733A described in the first embodiment. However, without being limited to this, the processor 210A can also execute processing related to billing and payment for the user 5A by any method different from the method according to the first embodiment.

FIG. 93 is a sequence chart showing an example of processing executed in the HMD system 100. FIG. Below, based on the input of the user 5A (1st user), a series of processes for performing live production|presentation control are demonstrated. The sequence chart shown in FIG. 93 shows processing after step S2613 shown in FIG. 26 is executed. In addition, in this embodiment, lighting control is mentioned as an example and demonstrated as production|presentation control which the user 5A is made to perform. However, the effect control is not limited to lighting control. For example, the presentation control may include sound control. In this example, processor 210A controls at least one of lighting and sound based on user 5A's input. Moreover, the user 5C may perform live production|presentation control, and the user 5D may perform.

FIG. 94 is a diagram showing second virtual space 2711A and view image 9417A according to an embodiment. As shown in FIG. 94A, a lighting object 9491, which is a type of virtual object, is arranged in the second virtual space 2711A. As an example, processor 210A may place illumination object 9491 in second virtual space 2711A along with stage object 1532 .

In step S9301, processor 210A causes avatar object 6B (second avatar) to perform a first performance based on the motion information included in the received avatar information. In step S9302, processor 210A executes a first lighting effect. The first lighting effect is a lighting effect preset by the user 5B before the start of the live performance. Information for executing the first lighting effect is transmitted from computer 200B via server 600 to computer 200A. Computer 200A may, for example, receive avatar information for arranging avatar object 6B in second virtual space 2711A together with the information, and execute the first lighting effect based on the information in step S9302. good.

Specifically, processor 210A, as virtual object control module 1427, emits light ray object 9492 from lighting object 9491 based on information for executing the first lighting effect, as shown in FIG. . Processor 210A may move lighting object 9491 based on the information. This allows processor 210A to move ray object 9492 . Note that the illumination object 9491 may be placed in the second virtual space 2711A at the same timing as the stage object 1532 is placed, for example.

The processor 210A notifies the user 5A during the live of a message that guides the user 5A to the lighting control. Processor 210A, for example, displays the message in tag object 2733A. The message includes text guiding lighting control and the billing amount required for user 5A to perform live lighting control.

After the message is displayed on tag object 2733A, processor 210A moves virtual left hand 1531LA into viewing area 15A as shown in FIG. 94(A) in accordance with the movement of user's 5A left hand. Processor 210A, for example, displays view image 9417A corresponding to second virtual space 2711A shown in FIG. 94(A) on monitor 130A as shown in FIG. 94(B). User 5A confirms the message displayed on tag object 2733A by visually recognizing view image 9417A. As a result, the user 5A recognizes that he or she can control live lighting by paying a new billing amount of 1000 yen.

After confirming the message displayed on the tag object 2733A, the user 5A performs an operation for himself/herself to control the lighting. Since this operation has been described with reference to FIG. 34 in Embodiment 1, it will not be repeated here. Processor 210A detects user 5A's operation for performing lighting control in response to selection of tag object 2733A displaying a message guiding lighting control.

FIG. 95 is a diagram showing second virtual space 2711A and view image 9517A according to an embodiment. At step S9303, processor 210A registers billing information corresponding to lighting control in tag object 2733A. In the example of FIG. 95 , when avatar object 6A (first avatar) selects tag object 2733A with virtual right hand 1531RA, processor 210A transmits charging information indicating the charging amount (1000 yen) required for lighting control to the tag. Newly register in object 2733A. It should be noted that the settlement of the billing amount represented by the registered billing information may be performed by the method described in the first embodiment.

After the billing information is registered, processor 210A re-visualizes the billing information on tag object 2733A. Details of this process have been described with reference to FIG. 35 in the first embodiment, and therefore will not be repeated here. Processor 210A, for example, displays view image 9517A corresponding to second virtual space 2711A shown in FIG. 95(A) on monitor 130A as shown in FIG. 95(B). User 5A recognizes that the current total billing amount has increased to 2500 yen by performing lighting control by visually recognizing view image 9517A.

FIG. 96 is a diagram showing second virtual space 2711A and view image 9617A according to one embodiment. At step S9304, the processor 210A moves the avatar object 6A to the first position for lighting control. The processor 210A, as the virtual object generation module 1421, generates the lighting control object 9693 and places it in front of the avatar object 6A that has moved to the first position. The lighting control object 9693 is used by the user 5C to cause the processor 210A to perform processing for controlling lighting effects. The lighting control object 9693 includes, for example, partial objects such as buttons, dials, and sliders that receive operations of the avatar object 6A. The processor 210A controls lighting effects by receiving operations on partial objects. For example, the processor 210A can change the color of the ray object 9492 or move the lighting object 9491 according to the received operation. Note that the lighting control object 9693 may be placed at a predetermined position determined based on the first position (the position shown in FIG. 96A) before the avatar object 6A moves. For example, the lighting control object 9693 may be arranged at the predetermined position at the same timing as the stage object 1532 is arranged.

Processor 210A, for example, displays view image 9617A corresponding to second virtual space 2711A shown in FIG. 96(A) on monitor 130A as shown in FIG. 96(B). User 5A recognizes that avatar object 6A has moved to the first position and lighting control object 9693 by visually recognizing view image 9617A. In other words, the user 5A recognizes that he or she can now control the lighting.

FIG. 97 is a diagram showing second virtual space 2711A and view image 9717A according to an embodiment. After moving to the first position, the user 5A performs an operation for controlling lighting. The user 5A moves his or her right hand so that the tip of the virtual right hand 1531RA approaches the lighting control object 9693. At step S9305, the processor 210A detects movement of the right hand of the user 5A based on the output of the right controller 300RA. In step S9306, the processor 210A moves the virtual right hand 1531RA in the second virtual space 2711A so that the tip of the virtual right hand 1531RA approaches the lighting control object 9693 according to the movement of the right hand of the user 5A.

At step S9307, the processor 210A operates the lighting control object 9693 according to the movement of the virtual right hand 1531RA. Specifically, when the tip of virtual right hand 1531RA and the partial object included in lighting control object 9693 are in the first positional relationship, processor 210A detects that the tip of virtual right hand 1531RA collides with the partial object. . The first positional relationship is, for example, that the distance between the tip of the virtual right hand 1531RA and the partial object is less than the first distance. Alternatively, the collision area defined at the tip of the virtual right hand 1531RA and the collision area set for the partial object at least partially collide. The processor 210A detects that the partial object has been manipulated by the virtual right hand 1531RA based on the collision between the tip of the virtual right hand 1531RA and the partial object.

In step S9308, the processor 210A executes the second lighting effect according to the operation on the lighting control object 9693, that is, the operation on the partial object. The second lighting effect is a lighting effect different from the first lighting effect, which is performed under the control of the user 5A. For example, processor 210A changes light ray object 9492 colored in the first color to light ray object 9792 colored in the second color, as shown in FIG. 97(A), according to the operation on the partial object. Processor 210A, for example, displays view image 9717A corresponding to second virtual space 2711A shown in FIG. 97(A) on monitor 130A as shown in FIG. 97(B). The user 5A confirms that the second lighting effect corresponding to his/her own operation is being executed in the second virtual space 2711A by viewing the view image 9717A.

Processor 210A transmits information for executing the second lighting effect to server 600 . The server 600 transmits the received information to the computers 200 of all users 5 . This enables the processor 210 of each computer 200 to execute the second lighting effect.

FIG. 98 is a diagram showing second virtual space 2711C and view image 9817C, according to an embodiment. Upon receiving the information for executing the second lighting effect, processor 210C generates UI panel 9894 and places it in second virtual space 2711C. The processor 210C, for example, displays a view image 9817C corresponding to the second virtual space 2711C shown in FIG. 98(A) on the monitor 130C as shown in FIG. 98(B). The user 5C (third user) recognizes that the UI panel 9894 has been placed in the second virtual space 2711C by viewing the view image 9817C.

The UI panel 9894 is a kind of UI object, and is used by the user 5C to cause the processor 210C to execute processing for selecting lighting effects. UI panel 9894 includes options 9895 and 9896 located in front of UI panel 9894 . Option 9895 and option 9896 include information describing the lighting effects that are selected when that option is selected by user 5C. The option 9895 is an item for selecting the first lighting effect, in other words, the default lighting effect, and the option 9896 is for selecting the second lighting effect, in other words, the lighting effect controlled by the user 5A. is the item. That is, the user 5C can select whether or not to view the first performance of the avatar object 6B with the second lighting effect.

In the example of FIG. 98, option 9895 is selected. Therefore, the processor 210C is executing the first lighting effect in the second virtual space 2711C. The processor 210C may change the color of the selected option 9895 and the unselected option 9896, as shown in FIG. 98(B). Thereby, the user 5C can recognize that the option 9895 is selected. Since the first lighting effect is the default lighting effect, the option 9855 may be automatically selected when the UI panel 9894 is placed in the second virtual space 2711C.

When the user 5C wishes to execute the second lighting effect in the second virtual space 2711C, the user 5C performs an operation for changing the lighting effect. As an example, user 5C moves his or her right hand so that the tip of virtual right hand 1531RC approaches option 9896 . The processor 210C detects movement of the right hand of the user 5C based on the output of the right controller 300RC. Then, the processor 210C moves the virtual right hand 1531RC in the second virtual space 2711C so that the tip of the virtual right hand 1531RC approaches the option 9896 according to the movement of the right hand of the user 5C.

The processor 210C moves the virtual right hand 1531RC in the second virtual space 2711C so that the tip of the virtual right hand 1531RC approaches the option 9896 based on the movement of the right hand of the user 5C. Processor 210C detects that option 9896 has been selected by virtual right hand 1531RC based on the collision between the tip of virtual right hand 1531RC and option 9896. FIG.

FIG. 99 is a diagram showing second virtual space 2711C and view image 9917C according to an embodiment. Processor 210C executes a second lighting effect in second virtual space 2711C in response to detecting selection of option 9896 . The processor 210C, for example, displays a view image 9917C corresponding to the second virtual space 2711C shown in FIG. 99(A) on the monitor 130C as shown in FIG. 99(B). The user 5C recognizes that the option 9896 has been selected and that the second lighting effect has started in the second virtual space 2711C by viewing the view image 9917C. When processor 210C executes the second lighting effect in second virtual space 2711C, processor 210C notifies computer 200A to that effect via server 600 .

FIG. 100 is a diagram showing second virtual space 2711A and view image 10017A according to one embodiment. At step S9309, the processor 210A notifies the users 5A of the number of users 5 viewing the first performance of the avatar object 6B with the second lighting effect. Specifically, processor 210A notifies user 5A of the number of notifications received from other computers 200 to the effect that the second lighting effect has been executed as the number of people. Processor 210A includes, for example, notification image 10085 including text indicating the number of people in view image 10017A corresponding to second virtual space 2711A shown in FIG. 100(A). Processor 210A then displays field-of-view image 10017A on monitor 130A, as shown in FIG. 100(B). Processor 210A can also place a virtual object having the same appearance as notification image 10085 in second virtual space 2711A. The user 5A recognizes the number of people watching the first performance of the avatar object 6B with the second lighting effect by visually recognizing the view image 10017A.

FIG. 101 is a diagram showing second virtual space 2711A and view image 10117A according to an embodiment. In FIG. 101(A), the live performance in the second virtual space 2711A has ended. Avatar objects 6B and 6C have left the second virtual space 2711A with the end of the live. Avatar objects 6A and 6D remain in the second virtual space 2711A after the live performance ends.

In step S9310, processor 210A changes the billing amount in the billing information registered in tag object 2733A based on another user's 5 evaluation of the second lighting effect. The evaluation may be, for example, the number of users 5 who have viewed the first performance of the avatar object 6B with the second lighting effect. Processor 210A may, for example, decrease the billing amount in the billing information registered in tag object 2733A as the number of people increases.

Processor 210A, for example, displays view image 10117A corresponding to second virtual space 2711A shown in FIG. 101(A) on monitor 130A as shown in FIG. 101(B). User 5A recognizes that the live has ended by visually recognizing field-of-view image 10117A. User 5A further recognizes that the total billing amount charged to user 5A has been changed from 2,500 yen (see FIG. 95(B)) to 2,300 yen. In other words, the user 5A recognizes that the amount of charge required for lighting control has decreased due to the evaluation of the lighting control performed by the user 5A.

As described above, the processor 210A allows the user 5A to control the performance of the live performance in accordance with the registration of the billing information. In this way, the HMD system 100 can allow the user 5A to virtually experience a live performance by the audience, which is impossible in a real live performance. As a result, it is possible to further enhance the interest of live performances in the virtual space. In addition, since live performance control requires payment of a billing amount, the number of users 5 who perform performance control can be optimized, and the performance of the avatar object 6B is not hindered by the performance of the users 5. can be prevented.

Also, when the performance control by the user 5A is highly evaluated, the user 5A can gain fame as a live performance staff in the virtual space. As a result, it is conceivable that the user 5A is employed as a production staff by the user 5B.

[Modification]
FIG. 102 is a diagram showing second virtual space 2711A and view image 10217A according to one embodiment. The processor 210A may enable the effect control by the user 5A when detecting the first operation by the user 5B (second user). Specifically, when the processor 210A receives a notification that the effect control is permitted from the computer 200B, it displays a message for guiding the effect control to the user 5A on the tag object 2733A. Processor 210A, for example, displays view image 10217A corresponding to second virtual space 2711A shown in FIG. 102(A) on monitor 130A as shown in FIG. 102(B). At this time, the processor 210A may include a notification image 10286 indicating that the effect control by the user 5A is enabled in the view image 10217A. Thereby, the user 5A can recognize that the effect control by himself/herself is possible by visually recognizing the field-of-view image 10217A. Note that the first operation may be, for example, a movement of the user 5B to cause the avatar object 6B to take a posture in which both hands are raised upward, as shown in FIG. 102(B). Processor 210A can also place a virtual object having the same appearance as notification image 10286 in second virtual space 2711A.

The billing amount required for effect control may be set according to the content of effect control. For example, when controlling sound and lighting, a higher billing amount may be set than when controlling only one of them. Also, the billing amount required for effect control may be determined according to the time during which effect control is performed. In the case of this example, the processor 210A newly registers billing information indicating the billing amount corresponding to the time required for the effect control in the tag object 2733A after the effect control is finished.

The user 5A may perform production control by operating an input device connected to the computer 200A. In this example, processor 210A displays view image 17A not only on monitor 130 but also on display 430A connected to computer 200A. Specifically, user 5A removes HMD 120 from the head after billing information is registered. Then, the orientation of the HMD 120 is adjusted so that the view image 17A suitable for effect control is displayed on the display 430A. The field-of-view image 17A suitable for effect control is, for example, a field-of-view image including the avatar object 6B and the ray object 9492. FIG. The user 5A then operates input devices such as a keyboard and a mouse connected to the computer 200A. The processor 210A executes effect control according to the operation.

The total billing amount may become a negative value as a result of changing the billing amount in the billing information registered in the tag object 2733A based on another user's 5 evaluation of the second lighting effect. For example, when the performance control performed by the user 5A is highly evaluated, the billing amount may be greatly reduced and the total billing amount may become a negative value. In this case, the user 5B or the platformer pays the user 5A the absolute value of the total billing amount as a reward.

The user 5A may not need to charge for the performance control by the user 5A.

[Embodiment 8]
In this embodiment, the processor 210A executes processing related to billing and settlement for the user 5A by the method using the tag object 2733A described in the first embodiment. However, without being limited to this, the processor 210A can also execute processing related to billing and payment for the user 5A by any method different from the method according to the first embodiment.

[Venue selection]
FIG. 103 is a diagram showing first virtual space 2411A (first region) and view image 10317A according to one embodiment. Processor 210A generates UI panel 10350 and pen object 2453 and places them in first virtual space 2411A. Note that the pen object 2453 has already been described with reference to FIG. 24 in the first embodiment, so description thereof will not be repeated here.

The UI panel 10350 is used by the user 5A to cause the processor 210A to execute processing for selecting a venue, similar to the UI panel 2450 (see FIG. 24) described in the first embodiment. UI panel 10350 includes options 10351 and 10352 located in front of UI panel 10350 . Options 10351 and 10352 include information describing the name of the venue to be selected when that option is selected by user 5A (first user). Also, options 10351 and 10352 include billing amounts when the option is selected by user 5A. The billing amount is, for example, a venue fee (first billing amount) and a performance fee (second billing amount). The venue fee is an amount charged according to the venue, and the performance fee is an amount charged according to the performance of the avatar object 6 .

The option 10351 is an item for selecting the first venue, and when the first venue is selected, the venue fee is 1000 yen and the performance fee is 1000 yen. The option 10352 is an item for selecting a second venue different from the first venue. When the second venue is selected, the venue fee is 500 yen and the performance fee is 800 yen. In other words, since the first venue and the second venue are different venues, the venue fees for the options 10351 and 10352 are different. In addition, the avatar object 6 performing at the first venue is different from the avatar object 6 performing at the second venue, and the content of the performance may be different. there is

Processor 210A, for example, displays view image 10317A corresponding to first virtual space 2411A shown in FIG. 103(A) on monitor 130A as shown in FIG. 103(B). User 5A recognizes that it is necessary to select option 10351 or option 10352 of UI panel 10350 by visually recognizing view image 10317A.

FIG. 104 is a diagram showing first virtual space 2411A and view image 10417A according to one embodiment. Processor 210A allows user 5A to select the first venue. As shown in FIG. 104A, after the virtual right hand 1531RA selects the pen object 2453, the processor 210A moves the tip of the pen object 2453 closer to the option 10351 based on the movement of the right hand of the user 5A. 1 Move the virtual right hand 1531RA and the pen object 2453 in the virtual space 2411A. The processor 210A detects that the pen object 2453 has selected the option 10351 based on the collision between the tip of the pen object 2453 and the option 10351 . Processor 210A then accepts selection of the first venue corresponding to selected option 10351 .

The processor 210A, for example, displays a view image 10417A corresponding to the first virtual space 2411A shown in FIG. 104(A) on the monitor 130A as shown in FIG. 104(B). User 5A recognizes selection of option 10351 of UI panel 10350 (that is, selection of the first meeting place) by visually recognizing view image 10417A.

FIG. 105 is a sequence chart representing part of the processing performed in the HMD set 110 according to one embodiment. In the following, the billing process after the user 5A selects the first venue will be described. The sequence chart shown in FIG. 105 shows the processing after step S2607 shown in FIG. 26 is executed.

FIG. 106 is a diagram showing second virtual space 2711A and view image 10617A according to one embodiment. In step S10501, processor 210A registers billing information representing the hall fee in tag object 2733A (performs processing relating to the first billing). In step S10502, processor 210A registers billing information representing the performance fee in tag object 2733A (performs processing relating to the second billing). Processor 210A visualizes billing information registered with tag object 2733A on tag object 2733A. In the example of FIG. 106A, processor 210A displays the sum of the venue fee and the performance fee (2000 yen) as the total billing amount in tag object 2733A. Note that the processes of steps S10501 and S10502 may be executed simultaneously, or the process of step S10502 may be executed before the process of step S10501.

After the billing information is registered, user 5A moves his/her left hand into his/her field of view as if checking the time on a wristwatch. Processor 210A detects the movement of user 5A's left hand based on the output of left controller 300LA. Processor 210A moves virtual left hand 1531LA into visual field 15A as shown in FIG. 106(A) according to the movement of user 5A's left hand. Processor 210A, for example, displays view image 10617A corresponding to second virtual space 2711A shown in FIG. 106(A) on monitor 130A as shown in FIG. 106(B). User 5A confirms the upper limit billing amount and total billing amount displayed in tag object 2733A by visually recognizing view image 10617A. In the example of FIG. 106B, the user recognizes that the maximum billing amount is 30000 yen and the current total billing amount is 2000 yen. In other words, the user 5A understands that the venue fee (1000 yen) and the performance fee (1000 yen) have been charged as a result of the avatar object 6A (first avatar) entering the second virtual space 2711A.

At step S10503, processor 210A receives avatar information of avatar object 6B (second avatar) from server 600 in real time. In step S10504, processor 210A arranges avatar object 6B in second virtual space 2711A based on the avatar information of avatar object 6B received first. Specifically, processor 210A places avatar object 6B on stage object 1532 .

As described with reference to FIG. 31 in the first embodiment, after the live performance starts, the user 5B (second user) moves his or her body. The processor 210B causes the avatar object 6B to perform the first performance in the second virtual space 2711B according to the movement of the user 5B. Processor 210B transmits avatar information including movement information of avatar object 6B when avatar object 6B performs the first performance to server 600 in real time.

At step S10505, the processor 210A receives the avatar information of the avatar object 6B from the server 600 in real time. At step S10506, the processor 210A causes the avatar object 6B to perform the first performance based on the motion information included in the received avatar information.

As described above, when the avatar object 6A enters the second virtual space 2711A in accordance with the selection of the venue by the user 5A, the processor 210A stores the billing information representing the hall fee and the billing information representing the performance fee as tag objects. 2733A. In this way, the HMD system 100 can charge the user 5A a more appropriate billing amount in consideration of the performance of the avatar object 6B and the venue where the performance is performed.

[Venue fee setting]
FIG. 107 is a diagram showing first virtual space 2411A and view image 10717A according to one embodiment. Processor 210A generates UI panel 10750 and pen object 2453 and places them in first virtual space 2411A. Assume that the first virtual space 2411A shown in FIG. 107 is a first virtual space generated at a different timing (for example, on a different day) than the first virtual space 2411A shown in FIG. In other words, FIG. 107 is a diagram showing an example in which user 5A selects a venue for viewing a live performance on a day different from the example in FIG.

In the example shown in FIG. 107, the first venue is different from the first venue shown in FIG. On the other hand, it is assumed that the performance performed at the first venue and the avatar object 6 performing the performance are the same as in the example of FIG. It is assumed that the second venue, the performance performed at the second venue, and the avatar object 6 performing the performance are the same as in the example of FIG. Therefore, on UI panel 10750 , the venue fee (500 yen) included in option 10751 is different from the venue fee (1000 yen) included in option 10351 . UI panel 10750 is similar to UI panel 10350 shown in FIG. 103 except for this point.

FIG. 108 is a diagram showing second virtual space 2711A and view image 10817A according to an embodiment. FIG. 108 is a diagram showing a state after the user 5A selects the first hall in the first virtual space 2411A shown in FIG. 107 and enters the first hall, that is, the second virtual space 2711A.

The second virtual space 2711A shown in FIG. 108(A) is narrower than the second virtual space 2711A shown in FIG. 106(A). In other words, the second virtual space 2711A shown in FIG. 108A has a smaller capacity than the second virtual space 2711A shown in FIG. 106A. Also, the stage object 10832 placed in the second virtual space 2711A shown in FIG. 108A is narrower than the stage object 1532 placed in the second virtual space 2711A shown in FIG. 106A. In other words, the second virtual space 2711A shown in FIG. 108A is less equipped than the second virtual space 2711A shown in FIG. 106A.

Therefore, the venue fee (500 yen) charged for entering the second virtual space 2711A shown in FIG. 108A is charged for entering the second virtual space 2711A shown in FIG. 106A. It is cheaper than the venue fee (1000 yen). Thus, in one aspect, the venue fee is set based on the attributes of the venue, that is, the second virtual space 2711A. Examples of the attributes of the second virtual space 2711A include the number of people that can be accommodated, the level of equipment availability, and the sound quality.

Processor 210A, for example, displays view image 10817A corresponding to second virtual space 2711A shown in FIG. 108(A) on monitor 130A as shown in FIG. 108(B). The user 5A recognizes that 1500 yen has been charged as a result of the avatar object 6A entering the second virtual space 2711A by visually recognizing the view image 10817A. Specifically, the user 5A recognizes that the venue fee of 500 yen and the performance fee of 1000 yen have been charged.

FIG. 109 is a diagram showing second virtual space 2711A and view image 10917A according to an embodiment. FIG. 109 is a diagram showing the state after the avatar object 6B has been placed in the second virtual space 2711A. Note that the avatar object 6 on the viewer side cannot enter the second virtual space 2711A after the avatar object 6B has been placed in the second virtual space 2711A, that is, after the live performance has started.

The processor 210A sends the user 5A a A venue fee to be charged may be set. In this example, the venue fee is not fixed at the time of venue selection in the first virtual space 2411A. Therefore, the options of the UI panel arranged in the first virtual space 2411A may be options that do not include the venue fee amount. In other words, the UI panel in this example may be UI panel 2450 shown in FIG.

Processor 210A calculates the number of other avatar objects 6 when avatar object 6B is placed in second virtual space 2711A. Processor 210A then determines the venue fee to be charged to avatar object 6A according to the calculated number, and registers charging information representing the venue fee in tag object 2733A. As an example, processor 210A may also register charging information representing the performance fee in tag object 2733A at this timing. Alternatively, processor 210A may register billing information representing the performance fee in tag object 2733A when avatar object 6A enters second virtual space 2711A.

As shown in FIG. 109A, three other avatar objects 6 are arranged in the second virtual space 2711A. In this example, processor 210A determines the venue fee to be ¥1,000, and registers billing information representing the venue fee and billing information representing the performance fee (¥1,000) in tag object 2733A.

Processor 210A, for example, displays view image 10917A corresponding to second virtual space 2711A shown in FIG. 109(A) on monitor 130A as shown in FIG. 109(B). User 5A recognizes that 2000 yen has been charged as the total amount of venue fee and performance fee as a result of avatar object 6A entering second virtual space 2711A by visually recognizing view image 10917A.

FIG. 110 is a diagram showing second virtual space 2711A and view image 11017A according to an embodiment. FIG. 110 is a diagram showing a state similar to FIG. Also in this example, after the avatar object 6B is placed in the second virtual space 2711A, that is, after the live is started, the avatar object 6 on the viewer side cannot enter the second virtual space 2711A. Also, the performance of the avatar object 6B in this example shall be the same as in the example of FIG. In other words, assume that the performance fee in the example of FIG. 110 is 1000 yen, as in the example of FIG.

In the example of FIG. 110, as shown in FIG. 110A, the number of other avatar objects 6 placed in the second virtual space 2711A is five, which is more than the example of FIG. In this example, processor 210A determines the venue fee to be 1,500 yen, which is higher than the example in FIG. 109, and registers charging information representing the venue fee and charging information representing the performance fee (1,000 yen) in tag object 2733A. . Processor 210A, for example, displays field-of-view image 11017A corresponding to second virtual space 2711A shown in FIG. 110(A) on monitor 130A as shown in FIG. 110(B). The user 5A recognizes, by visually recognizing the view image 11017A, that the avatar object 6A has entered the second virtual space 2711A and that 2500 yen has been charged as the total amount of the venue fee and the performance fee.

In this way, the processor 210A can set the venue fee higher and make the venue fee appropriate when the number of viewer-side avatar objects 6 is large, that is, when the live performance is popular.

Instead of the number of other avatar objects 6, the processor 210A sets the venue fee based on the attributes of the users 5 (third users, hereinafter referred to as "other users 5") with whom the other avatar objects 6 are associated. may Alternatively, processor 210A may set venue fees based on the number of other avatar objects 6 and attributes of other users 5 . Examples of such attributes include popularity, gender, occupation, and annual income. These pieces of information may be input, for example, when registering for a service for viewing live performances in virtual space. For example, the processor 210A may set the venue fee higher when the avatar object 6 associated with the celebrity is placed in the second virtual space 2711A as the avatar object 6 on the viewer's side. Also, for example, if the user 5A is male and the proportion of other avatar objects 6 associated with the female user 5 is equal to or greater than a predetermined value, the processor 210A sets the venue fee higher. good too.

Further, the processor 210A may set the venue fee based on other avatar objects 6 already placed in the second virtual space 2711A at the time when the avatar object 6A enters the second virtual space 2711A. . Specifically, the processor 210A may set the venue fee based on the number of the other avatar objects 6 and/or attributes of the user 5 associated with the other avatar objects 6 . In this example, the processor 210A may register billing information representing the venue fee in the tag object 2733A when the avatar object 6A enters the second virtual space 2711A.

The processor 210A may set the venue fee according to the matching rate between the attributes of the other users 5 preferred by the user 5A and the attributes of the other users 5 placed in the second virtual space 2711A. The attributes of other users 5 preferred by user 5A may be input in advance by user 5A, or determined by processor 210A based on the attributes of other users 5 in live performances that user 5A has viewed in the past. good too.

[Variation of venue selection]
The venue fee and performance fee may be set according to the time of viewing the live performance. In this example, the processor 210A may perform processing related to payment of the venue fee and the performance fee when the user 5A finishes viewing the live performance.

The venue fee may further be set according to the position where the avatar object 6A is arranged in the second virtual space 2711A. As an example, the processor 210A may set the venue fee higher if the avatar object 6A is positioned closer to the avatar object 6B. The processor 210A may also set the venue fee according to the position of the avatar object 6A in the second virtual space 2711A and the time the avatar object 6A existed at that position. In other words, the processor 210A may change the billing amount per unit time before and after the avatar object 6A moves its position.

The processor 210A may allow the user 5A to experience a preview from the best seat of each venue in the first virtual space 2411A, ie, when having the user 5A select the venue. The best seat may be, for example, the front row seat.

[Shuffle avatar object 6]
As an example, processor 210A selects each of a plurality of avatar objects 6 selected from an avatar group consisting of avatar object 6A and other avatar objects 6 in response to an input from user 5A, when processing relating to billing is executed. 2 to different positions in the virtual space 2711A. In one aspect, processor 210A swaps the positions of multiple avatar objects 6 selected from a group of avatars in second virtual space 2711A. Hereinafter, this replacement will be referred to as "avatar shuffle".

FIG. 111 is a diagram showing second virtual space 2711A and view image 11017A according to an embodiment. The processor 210A notifies the user 5A of a message during the live to guide the user 5A to avatar shuffle. Processor 210A, for example, displays the message in tag object 2733A. The message includes a text guiding avatar shuffle and the billing amount required for user 5A to perform avatar shuffle.

After the message is displayed on tag object 2733A, processor 210A moves virtual left hand 1531LA into viewing area 15A as shown in FIG. 111(A) in response to the movement of user's 5A left hand. Processor 210A, for example, displays view image 11117A corresponding to second virtual space 2711A shown in FIG. 111(A) on monitor 130A as shown in FIG. 111(B). User 5A confirms the message displayed on tag object 2733A by viewing view image 11117A. Thereby, the user 5A recognizes that the avatar shuffle can be performed by paying a new billing amount of 1000 yen.

After confirming the message displayed on tag object 2733A, user 5A performs an operation for avatar shuffle. Since this operation has been described with reference to FIG. 34 in Embodiment 1, it will not be repeated here. Processor 210A detects user 5A's operation for performing avatar shuffle in response to selection of tag object 2733A displaying a message guiding avatar shuffle. Upon detecting the operation, processor 210A registers billing information corresponding to avatar shuffle in tag object 2733A and executes avatar shuffle. In the example of FIG. 111, when avatar object 6A selects tag object 2733A with virtual right hand 1531RA, processor 210A newly adds billing information representing the billing amount (1000 yen) required for avatar shuffle to tag object 2733A. Register (perform processing related to third billing). It should be noted that the settlement of the billing amount represented by the registered billing information may be performed by the method described in the first embodiment.

FIG. 112 is a diagram showing second virtual space 2711A and view image 11217A according to one embodiment. As an example, the processor 210A randomly changes the positions of the six avatar objects 6 placed in the second virtual space 2711A, as shown in FIG. 112(A). Due to the avatar shuffle being executed, the avatar object 6A has moved to the position of the avatar object 6 in the front row in the example of FIG. Also, in the example of FIG. 111, the avatar object 6 closest to the avatar object 6A is the avatar object 6C. On the other hand, the avatar object 6 closest to the avatar object 6A has become the avatar object 6D as shown in FIG. 112(A) because the avatar shuffle has been executed.

Processor 210A, for example, displays view image 11217A corresponding to second virtual space 2711A shown in FIG. 112(A) on monitor 130A as shown in FIG. 112(B). The user 5A recognizes that the avatar object 6A has moved by avatar shuffle by viewing the view image 11217A.

When processor 210A executes avatar shuffle, processor 210A transmits avatar information including information indicating the position of each avatar object 6 after execution of avatar shuffle to server 600 . The server 600 transmits the received avatar information to each computer 200 by synchronous processing. As a result, the avatar shuffle is performed also in the virtual space 11 provided to each user 5 such as the users 5B, 5C, and 5D.

FIG. 113 is a diagram showing second virtual space 2711A and view image 11317A according to one embodiment. After the billing information is registered, processor 210A re-visualizes the billing information on tag object 2733A. Details of this process have been described with reference to FIG. 35 in the first embodiment, and therefore will not be repeated here. Processor 210A, for example, displays view image 11317A corresponding to second virtual space 2711A shown in FIG. 113(A) on monitor 130A as shown in FIG. 113(B). User 5A recognizes that the current total billing amount has increased to 2500 yen by performing avatar shuffle by visually recognizing view image 11317A.

As described above, processor 210A performs avatar shuffling in accordance with registration of billing information. As a result, the other avatar object 6 closest to the avatar object 6A is changed. As a result, the user 5 can achieve communication with various other users 5 by executing avatar shuffle during the live.

[Modified example of avatar shuffle]
FIG. 114 is a diagram showing second virtual space 4411A and view image 11417A according to an embodiment. Processor 210A arranges avatar objects 6A-6D and stage object 1532 in second virtual space 4411A in the same manner as second virtual space 2711A. A second virtual space 4411A includes a first area 4451 . The first area 4451 corresponds to part of the second virtual space 4411A. The processor 210A arranges the gate object 4452 at one place on the outer edge of the first area 4451 .

In FIG. 114, the avatar objects 6A-6C and the stage object 1532 are arranged within the first area 4451. In FIG. Avatar object 6D is placed outside first area 4451 . The first area 4451 is surrounded by non-illustrated opaque wall objects. A live performance of the avatar object 6B is held in the first area 4451 . The avatar object 6 in the first area 4451 can watch the performance by the avatar object 6B. The avatar object 6 outside the first area 4451 cannot view the performance by the avatar object 6B. That is, in the case of the example of FIG. 114, avatar objects 6A and 6C can view the performance by avatar object 6B. Avatar object 6D cannot view the performance by avatar object 6B.

Processor 210A, for example, displays view image 11417A corresponding to second virtual space 4411A shown in FIG. 114(A) on monitor 130A as shown in FIG. 114(B). User 5A recognizes that avatar object 6C is in first area 4451 by visually recognizing view image 11417A.

FIG. 115 is a diagram showing second virtual space 4411A and view image 11517A according to an embodiment. As an example, processor 210A executes avatar shuffle when processing related to billing is executed in response to user 5A's input, and moves other avatar objects 6 arranged in first area 4451 to outside first area 4451. , and another avatar object 6 placed outside the first area 4451 may be moved into the first area 4451 . Processor 210A swaps the position of avatar object 6C and the position of avatar object 6D, for example, by performing an avatar shuffle. As a result, as shown in FIG. 115A, the avatar object 6C moves out of the first area 4451 and the avatar object 6D moves into the first area.

Processor 210A, for example, displays view image 11517A corresponding to second virtual space 4411A shown in FIG. 115(A) on monitor 130A as shown in FIG. 115(B). The user 5A recognizes that the avatar shuffle has moved the avatar object 6D to the position where the avatar object 6C was, by visually recognizing the view image 11517A. Then, the user 5A can realize communication with another new user 5 (user 5D).

Note that, in the case of this modification, it is possible to notify the user 5 that the avatar object 6 may move from within the first area 4451 to outside the first area 4451 due to the avatar shuffle before starting the live. preferable. The processor 210A may also arrange the screen object 4340 (see FIG. 43) outside the first area 4451. FIG. In other words, the area outside the first area 4451 may be a public viewing venue for a live performance being held inside the first area 4451 .

Also, the processor 210A may enable avatar shuffle when detecting the first operation by the user 5B. Specifically, when processor 210A receives a notification from computer 200B to the effect that avatar shuffle is permitted, processor 210A displays a message guiding user 5A to avatar shuffle on tag object 2733A. At this time, processor 210A may include, in view image 17A, a notification image indicating that avatar shuffle is now possible. Thereby, the user 5A can recognize that the avatar shuffle is enabled by visually recognizing the view image 17A.

Also, as an example, the processor 210A may move one other avatar object 6 to a position closest to the avatar object 6A when processing related to billing is executed according to an input from the user 5A.

Avatar shuffling by user 5A may not require charging for user 5A.

The processor 210A may execute a billing-related process for excluding the avatar object 6A from avatar shuffle targets executed by other users 5 in response to an input from the user 5A.

Although the embodiments of the present disclosure have been described above, the technical scope of the present invention should not be construed to be limited by the description of the embodiments. It should be understood by those skilled in the art that this embodiment is merely an example, and that various modifications of the embodiment are possible within the scope of the invention described in the claims. The technical scope of the present invention should be determined based on the scope of the invention described in the claims and their equivalents.

[Additional notes]
The contents of one aspect of the present invention are listed below.

(Item 1) Explained the program. According to one aspect of the present disclosure, a program is executed by a computer (computer 200A) with a processor (processor 210A) to provide a first user (user 5A) with a virtual experience. The program instructs the processor to define a virtual space (second virtual space 2711A, 4411A) including a first area for providing a virtual experience to the first user (S2601); a step of allowing one avatar (avatar object 6A) to enter the first area (S2606), and a step of placing a second avatar (avatar object 6B) associated with a second user (user 5B) in the first area (S2612); , the step of causing the second avatar to perform a performance according to the movement of the second user (S10506); a step (S10502) of executing a process related to the second billing for the second billing amount according to .

(Item 2) In (Item 1), the program causes the processor to further execute a step of causing the first user to select the virtual space including the first area.

(Item 3) In (Item 1) or (Item 2), the program instructs the processor to move from the first avatar in the virtual space according to the posture of the head of the first user and the position of the first avatar in the virtual space. (S2608) of controlling the field of view (viewing area 15A) of the first avatar, defining a viewing image (viewing image 17A) corresponding to the field of view from the first avatar (S2609), and A step (S2609) of outputting the field-of-view image to the image display device (monitor 130A) is further executed.

(Item 4) In any of (Item 1) to (Item 3), the first billing amount is set based on the attribute of the first area.

(Item 5) In any one of (Item 1) to (Item 4), the program causes the processor to create third avatars (avatar objects 6C and 6D) respectively associated with a plurality of third users (users 5C and 5D). The step of arranging in the virtual space (S2605) is further executed.

(Item 6) In (Item 5), the program causes the processor to , to further execute the step of setting the first billing amount.

(Item 7) In (Item 5), the program further causes the processor to move each of the plurality of avatars selected from the avatar group consisting of the first avatar and the plurality of third avatars to different positions in the virtual space. let it run.

(Item 8) In (Item 7), the program further causes the processor to execute a process relating to the third billing according to the first user's input. In the moving step, each of the plurality of selected avatars is moved to a different position in the virtual space when the third billing process is executed.

(Item 9) In (Item 7) or (Item 8), in the step of arranging the plurality of third avatars in the virtual space, in the step of arranging and moving the plurality of third avatars in the first region, Each of the plurality of avatars is moved to a different position within the first area.

(Item 10) In (Item 7) or (Item 8), the plurality of selected avatars are the third avatar placed within the first area (4451) and the third avatar placed outside the first area. and In the moving step, the third avatar placed inside the first area is moved outside the first area, and the third avatar placed outside the first area is moved inside the first area.

(Item 11) In any one of (Item 7) to (Item 10), in the moving step, the positions of the plurality of selected avatars are exchanged.

(Item 12) In any one of (Item 7) to (Item 11), the program further causes the processor to detect the first operation by the second user. In the moving step, when the first operation is detected, each of the plurality of selected avatars can be moved to different positions in the virtual space.

(Item 13) The information processing apparatus has been described. According to one aspect of the present disclosure, the information processing device (computer 200A) stores a program executed by the information processing device (storage 230A) to provide a first user (user 5A) with a virtual experience. and a control unit (processor 210A) that controls the operation of the information processing apparatus by executing a program. The control unit defines a virtual space (second virtual space 2711A, 4411A) including a first area for providing a virtual experience to a first user, and a first avatar (avatar object 6A) associated with the first user. enters a first area, a second avatar (avatar object 6B) associated with a second user (user 5B) is placed in the first area, and the second avatar performs a performance according to the movement of the second user. Then, a process related to the first charging is executed for the first charging amount according to the first area, and a process related to the second charging is executed for the second charging amount according to the performance.

(Item 14) I explained how to run the program. According to one aspect of the present disclosure, a program is executed by a computer (computer 200A) with a processor (processor 210A) to provide a first user (user 5A) with a virtual experience. The method includes the steps of a processor defining (S2601) a virtual space including a first region (second virtual space 2711A, 4411A) for providing a virtual experience to a first user; a step of allowing one avatar (avatar object 6A) to enter the first area (S2606), and a step of placing a second avatar (avatar object 6B) associated with a second user (user 5B) in the first area (S2612); , the step of causing the second avatar to perform a performance according to the movement of the second user (S10506); and a step of executing a process related to the second billing for the second billing amount according to (S10502).

In the above embodiments, the virtual space (VR space) in which the user is immersed by the HMD has been exemplified and explained, but a transmissive HMD may be employed as the HMD. In this case, by outputting a visual field image obtained by synthesizing a part of an image constituting a virtual space with a real space visually recognized by a user through a transmissive HMD, an augmented reality (AR) space or mixed reality ( A virtual experience in MR (Mixed Reality) space may be provided to the user. In this case, an action may be generated on the target object in the virtual space based on the movement of the user's hand instead of the operation object. Specifically, the processor may identify the coordinate information of the position of the user's hand in the real space and define the position of the target object in the virtual space in relation to the coordinate information in the real space. As a result, the processor can grasp the positional relationship between the user's hand in the real space and the target object in the virtual space, and execute processing corresponding to the above-described collision control or the like between the user's hand and the target object. . As a result, it is possible to give an effect to the target object based on the movement of the user's hand.

2 network, 5, 5A, 5B, 5C, 5D user, 6, 6A, 6B, 6C, 6D avatar object, 11, 11A, 11B, 11C, 11D virtual space, 12 center, 13 panoramic image, 14, 14A, 14B virtual camera, 15, 15A, 15B, 15C field of view, 16 reference line of sight, 17, 17A, 17B field of view image, 18, 19 field, 100 HMD system, 110, 110A, 110B, 110C, 110D HMD set, 120, 120A, 120B, 120C, HMD, 130, 130A, 130B, 130C monitor, 140 gaze sensor, 150 first camera, 160 second camera, 170, 170A, 170B microphone, 180, 180A, 180B speaker, 190 sensor, 200, 200A, 200B computer, 210, 210A, 210B, 210C, 210D, 610 processor, 220, 620 memory, 230, 230A, 230B, 630 storage, 240, 640 input/output interface, 250, 650 communication interface, 260, 660 bus, 300, 300B controller, 300R right controller, 300L left controller, 310 grip, 320 frame, 330 top surface, 340, 340, 350, 370, 380 buttons, 360 infrared LED, 390 analog stick, 410 HMD sensor, 420, 420A motion sensor, 430, 430A display, 510 control module, 520 rendering module, 530 memory module, 540 communication control module, 600 server, 700 external device, 1421 virtual object generation module, 1422 virtual camera control module, 1423 operation object control module, 1424 avatar object Control module 1425 Motion detection module 1426 Collision detection module 1427 Virtual object control module 1428 Billing processing module 1429 Payment processing module 1517A, 1617B, 2417A, 2517A, 2717A, 2817A, 2917B, 3017A, 3217A, 3317A, 3417A ，３５１７Ａ，３６１７Ａ，３７１７Ａ，３８１７Ａ，３９１７Ａ，４０１７Ｂ，４１１７Ａ，４３１７Ａ，４４１７Ａ，４６１７Ａ，４７１７Ａ，４８１７Ａ，４９１７Ａ，５０１７Ａ，５１１７Ｂ，５２１７Ａ，５３１７Ｂ，５４１７Ｂ，５５１７Ａ，５６１７Ａ，５７１７Ａ，５９１７Ｃ，６０１７Ｃ，６１１７Ａ，６２１７Ａ ，６３１７Ａ，６４１７Ｂ，６５１７Ａ，６６１７Ａ，６８１７Ｂ，６９１７Ｂ，７１１７Ａ，７１１７Ｂ，７２１７Ａ，７４１７Ｂ，７５１７Ａ，７６１７Ｂ，７７１７Ｂ，７９１７Ａ，８０１７Ａ，８１１７Ｂ，８２１７Ｂ，８４１７Ｂ，８５１７Ａ，８７１７Ａ，８８１７Ａ，８９１７Ａ，９０１７Ｂ，９２１７Ａ，９４１７Ａ ，９５１７Ａ，９６１７Ａ，９７１７Ａ，９８１７Ｃ，９９１７Ｃ，１００１７Ａ，１０１１７Ａ，１０２１７Ａ，１０３１７Ａ，１０４１７Ａ，１０６１７Ａ，１０７１７Ａ，１０８１７Ａ，１０９１７Ａ，１１０１７Ａ，１１１１７Ａ，１１２１７Ａ，１１３１７Ａ，１１４１７Ａ，１１５１７Ａ，視界画像、１５３１ＬＡ，１５３１ＬＢ，１５３１ＬＣ， 1531LD virtual left hand, 1531RA, 1531RB, 1531RC virtual right hand, 1532, 10832 stage object, 1641, 1642, 1643 motion sensor, 1644 belt, 2411A first virtual space, 2450 2450, 6850, 9894, 10350, 10750, UI panel, UI Panels 2451, 2452, 6851, 6852, 9895, 9896, 10351, 10352, 10751 Choices 2453, 6853 Pen objects 2711A, 2711B, 2711C, 2711D, 4311B, 4411A Second virtual space 2733A, 2733B, 2733D, 2733C tag object, 3936A, 3936C, 3936D monitor object, 4137A, 4137C, 4137D texture, 4239 live video, 4340 screen object, 4451 first area, 4452 gate object, 3735, 8181, 9084, 10085, 10286, notification image, 4661, 5070, 8176 first position, 4662, 5071, 8177 second position, 4663, 6277 mark, 4664, 4867, 5373, 5675 position, 4665, 4969, 5776 movement distance, 4866, 5372 third position, 5674 fourth position, 6811B third virtual space, 8178 first direction, 8182 arrow image, 8283 frame image, 9491 lighting object, 9492, 9792 ray object, 9693 lighting control object

Claims (14)

A program executed by a computer with a processor to provide a virtual experience to a first user, comprising:
The program causes the processor to:
defining a virtual space including a first region for providing the virtual experience to the first user;
allowing a first avatar associated with the first user to enter the first area;
placing a second avatar associated with a second user in the first area;
causing the second avatar to perform a performance requested by the first user in response to the movement of the second user;
a first billing amount to be billed to the first user, which is a first billing amount according to the position of the first avatar in the first area;
and a step of executing processing relating to a second billing amount, which is a second billing amount to be billed to the first user, according to the performance. The program, in the step of causing the processor to execute processing relating to the first billing, sets the first billing amount higher as the position where the first avatar exists is closer to the position where the second avatar exists. 2. The program according to claim 1, causing The program causes the processor to:
3. The program according to claim 1, further causing the first user to select the virtual space including the first area. In the step of causing the processor to select, the program causes the first user to receive a preview from a specific position in the virtual space when the virtual space including the first area is selected by the first user. 4. The program according to claim 3, which is experienced. The program causes the processor to:
controlling a field of view from the first avatar in the virtual space according to the posture of the head of the first user and the position of the first avatar in the virtual space;
defining a view image corresponding to the view from the first avatar;
5. The program according to any one of claims 1 to 4 , further causing a step of outputting the field-of-view image to an image display device associated with the first user's head. 6. The program according to any one of claims 1 to 5 , wherein the program further causes the processor to place a plurality of third avatars respectively associated with a plurality of third users in the virtual space. The program instructs the processor, based on at least one of the number of the third avatars arranged in the first area and attributes of the third user with which the third avatars are associated, to perform the first lesson 7. The program according to claim 6 , further causing the step of setting an amount of money. The program causes the processor to:
7. The program according to claim 6 , further causing a step of moving each of a plurality of avatars selected from an avatar group consisting of said first avatar and said plurality of third avatars to different positions in said virtual space. The program further causes the processor to execute processing relating to a third billing in response to an input from the first user;
9. The program according to claim 8 , wherein, in said moving step, each of said plurality of selected avatars is moved to a different position in said virtual space when said processing relating to said third billing is executed. In the step of arranging the plurality of third avatars in the virtual space, arranging the plurality of third avatars in the first region;
10. The program according to claim 8 , wherein said moving step moves each of said plurality of selected avatars to different positions within said first area. The plurality of selected avatars includes the third avatar placed within the first area and the third avatar placed outside the first area,
In the moving step, the third avatar placed inside the first area is moved outside the first area, and the third avatar placed outside the first area is moved inside the first area. 10. A program according to claim 8 or 9 , for moving. 12. The program according to any one of claims 8 to 11 , wherein in said moving step, positions of said plurality of selected avatars are exchanged. The program further causes the processor to detect a first operation by the second user;
13. Any one of claims 8 to 12 , wherein in said moving step, each of said plurality of selected avatars can be moved to a different position in said virtual space when said first operation is detected. The program described in Section. An information processing device,
The information processing device is
a storage unit storing a program to be executed by a computer with a processor to provide a virtual experience to a first user;
A control unit that controls the operation of the information processing device by executing the program,
The control unit
defining a virtual space including a first region for providing the virtual experience to the first user;
causing a first avatar associated with the first user to enter the first area;
Placing a second avatar associated with a second user in the first region;
causing the second avatar to perform a performance requested by the first user according to the movement of the second user;
performing a first billing process for a first billing amount to be billed to the first user, the first billing amount corresponding to a position of the first avatar within the first area;
An information processing apparatus that executes processing related to a second billing amount, which is a second billing amount to be billed to the first user, according to the performance.

Images