JP7273328B2 - system, program and mobile - Google Patents

Description

The present disclosure relates to systems, programs and mobile objects.

Patent Literature 1 discloses an automatically operated vehicle that can continue operation using an automatic operation function even when communication is abnormal.

Japanese Patent Application Laid-Open No. 2021-9429

In recent years, in the automobile industry, technological innovation is progressing in a new area called CASE (Connected, Autonomous, Shared & Services, Electric). In Japan, the Strategic Council for the Promotion of Public and Private Data Utilization, Strategic Headquarters for the Promotion of an Advanced Information and Telecommunications Network Society, created a public-private ITS initiative and a roadmap showing policies for ITS (Intelligent Transport Systems) and automated driving. Expectations for self-driving cars are growing.

In view of the circumstances described above, the present disclosure provides a system, a program, and a mobile object that can give new stimuli to the user.

According to one aspect of the disclosure, a system is provided. The system is configured to be able to perform an acquisition step and an output step. The acquiring step acquires environmental sounds generated inside or outside the moving object. In the output step, based on an attribute selected from a plurality of predetermined attributes, output information corresponding to the environmental sound is output to a user riding on the mobile object.

2 is a block diagram showing the hardware configuration of the information processing system 1 according to the first embodiment; FIG. 2 is a block diagram showing the hardware configuration of the game device 3 according to the first embodiment; FIG. It is a block diagram which shows the functional structure of ECU53 of the mobile body 5 in 1st Embodiment. 3 is an activity diagram showing an example of the flow of processing executed by the information processing system 1; FIG. FIG. 4 is a conceptual diagram for explaining sound information; FIG. 4 is a conceptual diagram showing an example of a game image being executed by a user and a mode of outputting an additional sound corresponding to an environmental sound to the user; It is a block diagram which shows the hardware constitutions of the information processing system 1 in 2nd Embodiment. 4 is a diagram showing an example in which functions of an ECU 53 are implemented by an information processing device 2; FIG. FIG. 4 is a conceptual diagram showing a mode of outputting an additional image corresponding to environmental sound to the user;

[First embodiment]
Embodiments of the present disclosure will be described below with reference to the drawings. Various features shown in the embodiments shown below can be combined with each other.

By the way, the program for realizing the software appearing in this embodiment may be provided as a non-transitory computer-readable medium (Non-Transitory Computer-Readable Medium), or may be downloaded from an external server. It may be provided as possible, or may be provided so that the program is activated on an external computer and the function is realized on the client terminal (so-called cloud computing).

Further, in the present embodiment, the term “unit” may include, for example, a combination of hardware resources implemented by circuits in a broad sense and software information processing that can be specifically realized by these hardware resources. . In addition, various information is handled in the present embodiment, and these information include, for example, physical values of signal values representing voltage and current, and signal values as binary bit aggregates composed of 0 or 1. Expressed by high and low, or quantum superposition (so-called quantum bits), communication and operations can be performed on circuits in a broad sense.

A circuit in a broad sense is a circuit realized by at least appropriately combining circuits, circuits, processors, memories, and the like. Application Specific Integrated Circuits (ASICs); Programmable Logic Devices (e.g., Simple Programmable Logic Devices (SPLDs); Complex Programmable Logic Devices (CPLDs); and field It includes a programmable gate array (Field Programmable Gate Array: FPGA).

The system according to the first embodiment provides a user with a new stimulating experience. In particular, based on the above, it gives a new stimulus to the user who is riding in a mobile object such as a vehicle.

1. Description of Game FIG. 1 is a block diagram showing the hardware configuration of an information processing system 1 according to the present embodiment. In an information processing system 1 shown in FIG. 1, an information processing device 2 and a plurality of game devices 3 are communicably connected to each other via a communication network 11, and games are executed in the game devices 3. FIG. In addition, mobile object 5 is communicably connected to at least one of information processing device 2 and game device 3 via communication network 11 . The connection between the mobile object 5 and the game device 3 may be realized without the communication network 11 by Bluetooth (registered trademark) DCM (Data Communication Module) or the like. In other words, the communication network 11 in this embodiment also includes a route for establishing a P2P connection without going through the Internet.

A game according to the present embodiment is an online game executed by the information processing system 1 . In this game, the user of the game device 3 activates one or more player characters in a virtual game space, or pits the player characters against enemy characters that are non-player characters. Also, a character is an example of an object.

Games such as those described above can be played on a home game machine such as PlayStation (registered trademark), a portable game machine such as Nintendo Switch (registered trademark), or a game device 3 that is an electronic device such as a personal computer, a smartphone, or a tablet. is executed using Below, the case where the game device 3 is a smart phone is illustrated.

2. Outline of Information Processing System 1 As shown in FIG. 1 , the information processing system 1 includes an information processing device 2 and a plurality of game devices 3 . The information processing device 2 stores game programs and game data, and manages the game data (for each account information described below) of the game device 3 . The information processing device 2 is configured by, for example, a server. Each of the plurality of game devices 3 has the same configuration as each other.

The game device 3 executes a predetermined game based on user's operation. For this purpose, the game device 3 receives (more specifically, downloads and installs) a game program and game data from the information processing device 2 via the communication network 11 . Account information including identification information and a password is assigned to each user in association with the game device 3 . This account information is transmitted from the game device 3 to the information processing device 2 at the time of login, and is used for user authentication in the information processing device 2 .

Mutual communication between the information processing device 2 and the game device 3 becomes possible after the user authentication. After logging in, when the game device 3 receives data necessary for progressing the game (data relating to the progress of the game) from the information processing device 2, the game device 3 outputs game images and sounds to the display 4a and the speaker 4b based on the user's operation. , to advance the game.

The mobile object 5 is boarded by the user, and the game is executed by operating the game device 3 while the user is on board the mobile object 5 . Here, the moving body 5 is a vehicle. In this embodiment, the moving body 5 is an autonomously driven vehicle capable of autonomous travel. In other words, autonomous driving of the moving object is realized by automatic driving. Here, the level of automated driving is assumed to be level 2 (partially automated driving) or higher from level 0 (no automated driving) to level 5 (completely automated driving). As a result, the system executes the subtasks of vehicle motion control in both the longitudinal and lateral directions in the limited area, so that the user's operation is not required in the limited area, and the game can be executed by the game device 3.

Moreover, the vehicle of the mobile body 5 is provided with ECU53(Electronic Control Unit). The ECU 53 is a unit that electronically controls the behavior of the moving body 5, and is used for various objects such as airbags, transmissions such as AT (Automatic Transmission) and CVT (Continuously Variable Transmission), lane keeping systems, and inter-vehicle distance control systems. is a computer that electronically controls the

The game device 3 and the moving body 5 communicate with each other, and the sound or image generated from the game can be output from the display unit 6a or the speaker 6b of the moving body 5 instead of from the game device 3 4a or the speaker 4b. In this embodiment, the ECU 53 and the game device 3 communicate with each other to output various output information not inherent in the game according to the environmental sounds around the moving object 5, thereby providing a new user experience.

2.1 Hardware Configuration Each hardware configuration of the information processing system 1 will be described below with reference to FIG.

<Information processing device 2>
As shown in FIG. 1 , the information processing device 2 has a communication section 21 , a storage section 22 and a control section 23 . Communication unit 21 and storage unit 22 are electrically connected to control unit 23 via communication bus 20 .

The communication unit 21 is a so-called network interface that is communicably connected to each game device 3 or mobile object 5 via a communication network 11 such as the Internet and a LAN. The main information received by the information processing device 2 via the communication unit 21 includes game program download request information, gacha lottery request in response to the user's operation, quest execution request, auto-play execution/end request, Examples include account information and game data. The main information that the information processing device 2 transmits via the communication unit 21 includes information for confirming that the game device 3 has received the game program, information related to the game content obtained by the gacha, and the like. .

The storage unit 22 includes a HDD (Hard Disk Drive), a RAM (Random Access Memory), a ROM (Read Only Memory), an SSD (Solid State Drive), and the like. The storage unit 22 stores various programs including part of the game program according to the present embodiment, various data related to the game, and the like.

Specifically, for example, the storage unit 22 stores a user DB, a lottery list, and the like. In the user DB, information such as the user name, user rank, status related to the player character operated by the user, the amount of consumable media that can be used in the virtual game space, parameters related to each user identification number who plays the game Information and the like are associated and stored. The lottery list is used for a lottery process generally called gacha, and includes a plurality of pieces of information about game media to be selected. In the lottery list, information (name, ability parameter, rarity, level, etc.) related to game media is associated with the selection ratio by lottery. The ability parameters include, for example, strength, HP, attack power, defense power, intelligence, speed, etc. when the game medium is a player character.

The control unit 23 is composed of a microcomputer including a CPU and a semiconductor memory, and controls the operation of the information processing apparatus 2 itself. In particular, the control unit 23 reads out a predetermined program stored in the storage unit 22 to implement various functions related to the information processing device 2 as its own device. That is, information processing by software stored in the storage unit 22 is specifically realized by the control unit 23, which is an example of hardware, so that each function unit described later can be executed. Note that the control unit 23 is not limited to a single unit, and may be implemented to have a plurality of control units 23 for each function. A combination thereof may also be used.

The information processing executed by the control unit 23 includes, for example, billing settlement processing, user account authentication processing, gacha lottery selection processing, and the like. The billing settlement process is executed, for example, based on a billing request necessary for recovering a predetermined amount of game parameters. The user account authentication process is performed using the user identification information received from the game device 3, for example. The gacha lottery selection process is a process of selecting one or more game media from a lottery list by lottery based on the selection probability of each game media in response to a gacha lottery request. According to the gacha lottery selection process, the information about the selected game content and the identification information of the user who performed the operation that is the source of the lottery request are associated in the user DB. Game media won as a result of drawing gacha by yourself are given.

Here, to further supplement the above, "game media" is electronic data representing elements related to a game, including the name of a character used as a player character, items used by the player character in the virtual game space ( Weapons, armor, tools, etc.). Users can acquire game media directly by charging, by clearing quests, or by a lottery method called gacha. The acquired game media are stored and managed in the user DB in association with the identification information of the user who owns the game media. Furthermore, "gacha" is a method of selecting arbitrary game media from a lottery list by lottery in the information processing device 2 based on a predetermined selection ratio. Any selected game content is provided to the user's game device 3 . "A game medium selected by a gacha is given to the user/owned by the user" is synonymous with "the game medium selected by the lottery process is associated with/associated with identification information indicating the user." be.

<Game device 3>
As shown in FIG. 2, the game apparatus 3 has a display 4a, a speaker 4b, and a controller 4c externally connected or built therein. The game device 3 also has a communication section 31, a storage section 32, a control section 33, a graphics processing section 34a, an audio processing section 34b, and an operation section 34c. The communication unit 31 , the storage unit 32 , the graphics processing unit 34 a , the audio processing unit 34 b and the operation unit 34 c are electrically connected to the control unit 33 via the communication bus 30 .

The communication unit 31 is a so-called network interface that is communicably connected to the communication network 11 in order to transmit and receive various data between the game device 3 and the information processing device 2 or the mobile device 5 . The main information received by the game device 3 via the communication unit 31 includes account information, download request information for new game data, a gacha execution request, a quest execution request, and the like. The main information that the game device 3 transmits via the communication unit 31 includes new game data sent from the information processing device 2 in response to the download request information, information about game media selected by lottery processing, and the like. mentioned.

The storage unit 32 is composed of an HDD, an SSD, a RAM, a ROM, and the like. The storage unit 32 stores game data downloaded from the information processing device 2, various programs including part of the game program, account information of the game device 3 as the own device, user information, and the like. Note that the user information is at least part of the user DB stored in the storage unit 22 of the information processing device 2 . The user DB manages a master of user information, and the storage unit 32 of the game device 3 stores at least part of the information of this master distributed from the information processing device 2 .

The control unit 33 is composed of a microcomputer including a CPU and a semiconductor memory, and controls the operation of the game device 3 itself. In particular, the control unit 33 reads out a predetermined program stored in the storage unit 32 to implement various functions related to the game device 3 as its own device. That is, information processing by software stored in the storage unit 32 is specifically realized by the control unit 33, which is an example of hardware, so that each function unit described later can be executed. Note that the control unit 33 is not limited to a single unit, and may be implemented to have a plurality of control units 33 for each function. A combination thereof may also be used.

In particular, the control unit 33 is configured to execute the game according to the operation of the controller 4c by the user of the game apparatus 3, which is the own apparatus. Specifically, the control unit 33 reads data such as virtual game space objects and textures included in the game data from the storage unit 32, or uses the data received from the information processing device 2 to perform two-dimensional or three-dimensional processing. game image information. As the game image information is processed by the graphics processing unit 34a, the processed game images are sequentially displayed on the display 4a. In other words, the control unit 33 is configured to perform display control of the display 4a and audio output control of the speaker 4b according to the operation of the user of the game device 3, which is the own device, when executing the game.

The graphics processing unit 34a draws game images including characters and various objects related to the virtual game space in a moving image format according to the game image information output from the control unit 33. FIG. The graphics processing unit 34a is connected to, for example, a liquid crystal display 4a, and a game image drawn in a moving image format is displayed on the display 4a as a game screen. The audio processing unit 34b is connected to the speaker 4b, reproduces and synthesizes the game sound according to the instruction of the control unit 33, and outputs it from the speaker 4b. The operation unit 34c is connected to the controller 4c, and transmits/receives data regarding operation input to/from the controller 4c. The user inputs an operation signal to the game apparatus 3 by operating the controller 4c.

<Moving body 5>
As shown in FIG. 1, the moving body 5 has a display unit 6a, a speaker 6b, an imaging unit 6c, a sensor 6d, and a sound collecting unit 6e externally connected to or built in. The display unit 6a includes a display, a projector for projecting onto a window or the like, a head-mounted display, smart glasses, and the like. The speaker 6b is for generating sound.

The imaging unit 6c is composed of, for example, a camera, and is configured to be capable of imaging inside or outside the vehicle. The sensor 6d acquires parameters relating to the environment inside or outside the vehicle. The type of sensor 6d is not particularly limited. Acquire parameters related to the environment of the (eg ambient weather, temperature, humidity, wind speed, amount of rainfall, brightness, type or volume of sound occurring outside the vehicle). The sound collector 6e is composed of, for example, a microphone, and is configured to be able to collect sounds inside or outside the vehicle. Information acquired by the imaging unit 6 c , the sensor 6 d , and the sound collecting unit 6 e is input to the ECU 53 .

The mobile body 5 also has a communication unit 51, a storage unit 52, an ECU 53, a graphics processing unit 54a, and an audio processing unit 54b. Communication unit 51 , storage unit 52 , graphics processing unit 54 a and audio processing unit 54 b are electrically connected to ECU 53 via communication bus 50 .

The communication unit 51 is a so-called network interface that is communicatively connected to the communication network 11 in order to transmit and receive various data between the mobile body 5 and the information processing device 2 or the game device 3 . The main information received by the mobile unit 5 via the communication unit 51 includes account information, new game data download request information, gacha execution request, quest execution request, etc., as well as user ID, game progress, game Attributes of the game being executed on the device 3 (e.g. type of content, game title, game category), game image information about the screen displayed in the game, and game sound information about the game sound occurring in the game. mentioned. In addition, the communication unit 51 exchanges data with a computer outside the vehicle (including control devices such as a game server and a matching server) and communication units of other vehicles to obtain game data, vehicle state (moving body movement information, information about the surrounding environment of the moving object, information about the season or time, etc.), road conditions, information exchange between vehicles and between vehicles and infrastructure facilities, etc. Collect and analyze and exchange information (CASE (equivalent to Connected).

The storage unit 52 has the same configuration as the storage unit 32 . The storage unit 52 stores programs for controlling the ECU 53, images captured by the imaging unit 6c, information acquired by the sensor 6d, sounds collected by the sound collecting unit 6e, and the like.

The ECU 53 is composed of a microcomputer including a CPU and a semiconductor memory, and electronically controls various objects that constitute the moving body 5 . In particular, the ECU 53 implements various functions related to the mobile body 5 as its own device by reading a predetermined program stored in the storage unit 52 . That is, each functional unit described later can be executed by specifically realizing information processing by software stored in the storage unit 52 by the ECU 53, which is an example of hardware. In addition, the ECU 53 is not limited to a single unit, and a plurality of ECUs 53 may be provided for each function. A combination thereof may also be used. The ECU 53 downloads the latest version of the predetermined program via the communication unit 51 and updates the predetermined program stored in the storage unit 52 .

In particular, the ECU 53 is connected to the game device 3 operated by the user on board the mobile object 5 and implements various controls based on environmental sounds generated inside or outside the mobile object 5 .

The graphics processing unit 54a draws game images including characters and various objects related to the virtual game space in a moving image format according to the game image information output from the ECU 53. FIG. The graphics processing unit 54a is connected to the display unit 6a, and a game image drawn in a moving image format is displayed on the display unit 6a as a game screen. Here, the mobile object 5 does not communicate with the information processing device 2 or the game device 3 to acquire game data (eg, game image information, game sound information), but establishes a P2P connection with the game device 3 . When the screen displayed on the display 4a is mirrored, the graphics processing unit 54a converts the game image including the character and various objects related to the virtual game space into a moving image format according to the game image information output from the game device 3. draw with

The audio processing unit 54b is connected to the speaker 6b, reproduces and synthesizes game sound according to instructions from the ECU 53, and outputs the sound from the speaker 6b.

While the user is on the mobile object 5 and is playing the game using the game device 3, the ECU 53 causes the game screen to be output from at least one of the display 4a and the display unit 6a. In addition, the ECU 53 outputs a game sound or an additional sound, which will be described later, from at least one of the speaker 4b and the speaker 6b.

2.2 Functional Configuration Next, each functional configuration of the ECU 53 of the moving body 5 will be described. Here, the system according to the present embodiment is implemented by the information processing system 1, particularly by the ECU 53. FIG.

<ECU53>
FIG. 3 is a block diagram showing the functional configuration of the ECU 53 in the mobile body 5 according to this embodiment. The ECU 53 functions as an acquisition section 531 , an output section 532 , a selection section 533 and a noise canceling section 534 by executing various programs stored in the storage section 52 . That is, information processing by software stored in the storage unit 52 is specifically realized by the ECU 53 which is an example of hardware, and can be executed as each function unit included in the ECU 53 .

Acquisition unit 531 is configured to be able to execute an acquisition step. The acquisition unit 531 acquires environmental sounds generated inside or outside the moving object 5 . In this embodiment, the acquisition unit 531 acquires environmental sounds collected by the sound collection unit 6e. Here, environmental sounds include the engine sound reverberating inside the vehicle, the frictional noise between the tires and the road surface, the sound emitted by the road surface, the sound of the vehicle running such as wind noise, the sound of closing the vehicle doors and trunk, and the sound emitted at the time of a collision. collision sounds of other vehicles, horns of other vehicles, sirens, sounds caused or emitted by other vehicles related to running, announcements, sounds emitted from sound signals, etc., road construction sounds, coastal wave sounds, etc. is an external sound caused by The environmental sounds include the voice of the user on board the mobile object 5, the sound of music being listened to inside the mobile object 5, and the sound of video being viewed inside the mobile object 5. good too. In other words, the ambient sound may be any sound that can be picked up by the sound collector 6e (microphone). Further, the acquisition unit 531 acquires various information acquired by the sensor 6d.

The output unit 532 is configured to be able to execute the output step. The output unit 532 outputs output information corresponding to the environmental sound to the user on board the moving body 5 based on an attribute selected from a plurality of predetermined attributes. These attributes also include the type of content. At this time, the output unit 532 outputs output information based on the type of content executed by the computer based on the instruction of the user on board the mobile object 5 . Here, the types of content include games executed by the game device 3, moving images such as movies, and audio-only music.

The selection unit 533 is configured to be able to execute the selection step. The selection unit 533 selects sound information corresponding to the environmental sound. Here, the sound information is information representing sounds created for each game or game category.

The noise canceling section 534 is configured to be able to execute a noise canceling step. The noise canceling unit 534 performs noise canceling processing on sounds that can be collected inside the moving object among environmental sounds. Here, the noise canceling process is a process of canceling noise in environmental sounds, and is realized by an "active noise canceling (ANC)" technology that cancels noise by digital processing. Specifically, the noise can be removed by outputting, from the speaker 6b, a sound having an opposite phase to the noise collected from the sound collecting unit 6e.

Noise generated in automobiles is classified into road noise, wind noise, engine order, and HVAC (Heating, Ventilation, Air Conditioning: an air conditioning system including heating, ventilation, and air conditioning). It is also possible to reduce only the noise that you want to remove while leaving the unwanted sound. Also, by applying noise canceling processing, it is possible to make the user sitting in the driver's seat and the user sitting in the front passenger seat hear different sounds.

3. Description of Various Information and Terms In this section, various information and terms handled by the information processing system 1 will be described.

A “user name” is a name for identifying a user, which is composed of an arbitrary character string set by the user himself/herself. "User rank" is an index that indicates the user's level, and the rank is raised by clearing game stages, etc. When the rank is raised, new game stages are released, items can be purchased, etc. The width of the play spreads.

"Consumable media" are currency, points, etc. that can be used in the virtual game space, and are media that are consumed in the game as the game progresses. Consumable media are given to users as rewards for clearing game stages called "quests" or as billing in the real world.

"Stamina" is a parameter that is consumed as the game is played. Stamina is consumed, for example, by playing a game stage called "quest". Stamina recovers over time, regardless of quest progress. In addition, stamina can be recovered according to user instructions such as consumption of consumable media and various in-game items. In addition, stamina may be recovered without responding to the user's instructions, such as by increasing the user's rank as the quest progresses.

"Parameter-related information" is, for example, stamina-related information. The information about stamina includes, for example, the current value of stamina, the upper limit, or the user rank corresponding to the upper limit, the unit of time (for example, the unit of time required to restore the stamina value to "1"), and the recovery speed. (For example, the amount of stamina value recovered per minute (hereinafter also referred to as “recovery amount”)), or the time elapsed since the stamina recovered. In this embodiment, an example is used in which the upper limit of stamina is set to the upper limit of recovery over time, but the present invention is not limited to this.

A “quest” is a stage, which is a constituent unit of a game, and has tasks and goals set. In particular, a quest in which a plurality of users participate in and play a game at the same time is called a "multi-quest". End conditions are set for quests. In addition, a clear reward is given to the user for clearing the quest.

Quest end conditions include clearing conditions such as annihilation of opposing enemy characters, defeating boss characters, reaching the goal point, conditions related to play time such as over the time limit, and exceeding the number of turns in turn-based games. and the conditions of At least one of these end conditions is set in the quest. When multiple end conditions are set for a quest, which end condition is prioritized is set in advance. For example, when the time limit is given priority over the clear condition, the quest ends when the time limit expires when the boss character cannot be defeated and the clear condition is not satisfied.

Also, a quest may be played repeatedly by the same user. Playing a quest repeatedly is referred to as "quest laps." As a specific example of a quest that the user repeats a quest, a quest that can be cleared with a certain probability to obtain a predetermined reward, and a quest that repeats until the reward is obtained, or a quest that gives a high reward when cleared, For example, the quest may be repeated for the purpose of obtaining as many rewards as possible.

A “mission” is a task associated with each quest. That is, each quest is associated with at least one mission. Specific examples of missions include "clear the game without incapacitating anyone", "inflict 5,000 or more damage to enemy characters in one hit", and "defeat three or more enemy characters at the same time". etc. Also, a mission reward is given to the user for accomplishing the mission.

4. Flow of Operation of Information Processing System 1 In this section, an information processing method of the information processing system 1 described above will be described. FIG. 4 is an activity diagram showing the flow of information processing executed by the information processing system 1. As shown in FIG.

4.1 Overview of information processing Information processing executed by the information processing system 1 acquires environmental sounds inside or outside the moving body 5, and determines whether the movement is performed based on an attribute selected from a plurality of predetermined attributes. It outputs output information corresponding to environmental sounds to the user on board the body 5 .

Specifically, when the game being executed by the user is a racing game in which a vehicle is operated in a virtual game space, the type of content, that is, "game" corresponds to the attribute. Here, a game is an example of content. When it is raining heavily around the mobile object 5, the sound of the rain or white noise resembling the sound of the rain is output as an additional sound to the game sound of the game being executed by the user. Here, the additional sound may be a sound added separately from the game sound specific to the game, or the game sound specific to the game itself may be treated as the additional sound. For example, a heavy rain sound that is not stored as game data may be additionally reproduced, or a heavy rain sound for a game that is stored as game data may be called and reproduced. Furthermore, the virtual space may be displayed in a state of rain, such as changing the in-game weather to rain). As a result, various output information corresponding to environmental sounds generated inside or outside the moving body 5 can be output in addition to the game sounds caused by the progress of the game, depending on the attributes of the game being executed. It is possible to give new stimulus to the user.

In the present embodiment, an example in which a game screen and game sound of a game executed by the game device 3 are output from the display unit 6a and the speaker 6b of the moving object 5 will be described. Information processing that occurs after the user has boarded the moving object 5 will be described below.

First, the user on board the mobile object 5 operates the game device 3 to start the game (A20). Then, communication is established between the game device 3 and the ECU 53 (A11 and A21). In the present embodiment, the game executed by the game device 3 corresponds to the function provided by the ECU 53, and after the game is started, a communication establishment icon is displayed on the game screen, and the user selects the communication establishment icon. , the communication between the game device 3 and the ECU 53 is established by pairing the ECU 53 and the game device 3 by P2P connection. This allows the game device 3 to output various information about the game to the ECU 53 .

Subsequently, the game device 3 outputs the attribute of the started game from the communication unit 31 as an attribute selected from a plurality of predetermined attributes (A22). Here, such an attribute is the type of content. In this embodiment, the attribute is the type of game played by the computer. Specifically, games, movies, animations, dramas, and other moving images are the types of content or games. That is, in the example of FIG. 4, "game" is output as the attribute.

In addition, the game device 3 outputs the content category from the communication unit 31 . Specifically, when the content is a game, the category of the content corresponds to a racing game, a fighting game, an action game, a role-playing game, a romance game, and the like. When the user operates the game device 3 or another terminal to watch a movie, the content type is "movie", and the content category includes science fiction movies, action movies, horror movies, and romance movies. Equivalent to.

Subsequently, the game apparatus 3 causes the control section 33 to proceed with the game based on the operation of the controller 4c (A23). At this time, a game sound unique to the game is output from the communication section 31 (A24). The game sound output from the communication unit 31 is input to the ECU 53 via the communication unit 51, processed by the audio processing unit 54b, and output as the actual sound from the speaker 6b. Similarly, a game image unique to the game is output from the communication section 31 . The game image output from the communication unit 31 is input to the ECU 53 via the communication unit 51, processed by the graphics processing unit 54a, and displayed on the display unit 6a.

Subsequently, the game device 3 outputs the progress of the game from the communication section 31 (A25). Here, the progress of the game includes, for example, the progress of a specific quest being challenged in the game, the occurrence of a specific event, the ongoing battle with a boss, and the automatic progress without receiving user operations. It contains all the information about the progress of the game, such as being in a mode, standing in a racing game, using items, etc.

On the other hand, the acquisition unit 531 acquires the attribute output from the communication unit 31 via the communication unit 51 (A12: acquisition step). The acquired attributes are stored in the storage unit 52 . Thereby, the ECU 53 can recognize that the content being executed inside the vehicle of the mobile object 5 is a game.

Subsequently, the ECU 53 acquires environmental sounds from the sound collector 6e (A13). Also, the ECU 53 acquires status information from the sensor 6d or the Internet (A14). Here, the status information is at least one of information on movement of the mobile body, information on the environment around the mobile body, and information on time or time. This will be explained in more detail later.

Subsequently, the ECU 53 acquires the progress of the game output from the communication section 31 via the communication section 51 (A15). The acquired progress is stored in the storage unit 52 .

Subsequently, the noise canceling unit 534 performs noise canceling processing on sounds that can be collected inside the moving object among the environmental sounds (A16: noise canceling step). In this embodiment, the noise canceling unit 534 controls the noise canceling process based on at least one of the progress of the game, the situation of the moving body 5, and the surroundings of the moving body 5. The status of the mobile body 5 includes information on the movement of the mobile body 5, as well as the degree of energy-saving driving, the temperature of electronic components, the distance traveled, and the like. In addition to the information about the environment, the circumstances surrounding the mobile object 5 include information about the environment, such as the fact that the vehicle is traveling on a highway, the surrounding traffic congestion, the fact that an event is occurring, traffic restrictions, and the like.

Subsequently, the selection unit 533 selects sound information corresponding to the environmental sound (A17: selection step). Here, the sound information is information representing sounds created for each game or game category. Specifically, in the case of a famous game title, sound information is generated in advance for each game title and stored in the storage unit 52 . Also, in order to deal with game titles that are not famous, sound information is generated in advance for each category of game and stored in the storage unit 52 . This will be explained in more detail later.

Then, the output unit 532 outputs the output information (additional sound) corresponding to the environmental sound to the user boarding the moving body based on the attribute selected from a plurality of predetermined attributes (A18 : output step). Then, the additional sound is processed by the audio processing unit 54b and output as the actual sound from the speaker 6b in the same manner as the game sound. In the present embodiment, based on an attribute (game X) selected from a plurality of predetermined attributes (types of games or game categories shown in FIG. etc.), that is, sound information (additional sound X1, additional sound X2, . . . ) is output. That is, in this embodiment, the output unit 532 outputs sound information corresponding to the environmental sound as the additional sound. This will be explained in more detail later.

Here, if the game device 3 continues the same game after completing the process of A25, the process returns to A23. On the other hand, if another game is started, the process returns to A22, and the attributes of the newly started game are output to the ECU 53. FIG. On the other hand, when the game device 3 continues the game, the ECU 53 returns the process to A12. Then, when the game ends, the processing of the information processing system 1 ends.

4.2 Details of Information Processing Next, details of the information processing outlined above will be described.

<Acquisition of status information>
The acquisition of status information (A14) outlined above will be supplemented. As described above, the situation information is at least one of information about the movement of the mobile body, information about the environment around the mobile body, and information about time or time.

Specifically, the information about the movement of the moving body 5 includes the speed, acceleration, degree of rolling, degree of pitching, continuous boarding time (or continuous running time), etc. of the moving body 5 . Environmental information includes the surrounding weather, temperature, humidity, wind speed, amount of rainfall, brightness, type or volume of sounds generated outside the vehicle, color of traffic lights, signs, other vehicles in the vicinity, presence or absence of people, etc. state, whether the road is straight, whether the road is curved, and so on. Also, as for the period or time, the ECU 53 connects to the Internet via the communication network 11 to acquire information about the period or time.

<Noise canceling processing>
The execution timing of the noise canceling process outlined above is an example, and more preferably, the noise canceling section 534 executes the noise canceling process regardless of the start of the game. In the example of FIG. 4, the noise canceling process is executed at A16 for the convenience of explaining the acquisition of the situation information at A15, but the noise canceling process can be started before A11. Moreover, once the noise canceling process is executed, it continues for a predetermined time. Therefore, after starting the noise canceling process before A11, it is preferable to control the noise canceling process in consideration of various information in A16.

By taking these into consideration, for example, when the vehicle is traveling at high speed, the degree of noise canceling processing can be automatically increased. For example, on an expressway, the mobile object 5 travels at a higher speed than on a general road (it travels at a speed higher than the minimum speed stipulated by the Road Traffic Law). Become. At this time, by increasing the degree of noise canceling processing, it is possible to firmly cancel loud noise compared to general roads. Further, whether the vehicle is traveling at high speed may be determined from environmental sounds, or may be acquired from vehicle position information, a vehicle speedometer, or a navigation device (GPS).

On the other hand, when an ambulance siren or a person's poor physical condition is detected, or when an emergency situation for the moving body 5 occurs, the degree of noise canceling processing can be automatically reduced. In addition, the external sound can be positively output by switching to the external sound capturing mode. As a result, for noises that require some action from the user among the environmental sounds, cancellation of the noises by the noise canceling process can be suppressed, so that the user can easily notice the environmental sounds.

<Select sound information>
The selection of sound information (A17) outlined above will be supplemented. FIG. 5 is an example of sound information stored in the storage unit 52. As shown in FIG. For example, when sound information specific to game X is stored in the storage unit 52 for game X being executed by the user, the selection unit 533 selects sound information specific to game X from a plurality of sound information. At this time, appropriate sound information is selected from a plurality of sound information (additional sound X1, additional sound X2, . . . ) according to the environmental sound or the progress of the game.

On the other hand, if the sound information unique to game X is not stored in the storage unit 52 and the attribute acquired in A12 is “racing game” corresponding to the category of the game, the selection unit 533 selects the category of the racing game. Select the sound information corresponding to . At this time, appropriate sound information is selected from a plurality of sound information (additional sound 11, additional sound 12, . . . ) according to the environmental sound or the progress of the game. In this way, when game-specific sound information is stored in the storage unit 52, it is preferable to select the game-specific sound information in preference to the sound information stored for each game category. Actually, in addition to the example of FIG. 5, the sound information is further subdivided for each situation information and game progress.

<Additional sound output>
The additional sound output (A18) outlined above will be supplemented. In the present embodiment, the output unit 532 outputs output information based on the type of content executed by the computer based on instructions from the user on board the mobile object 5 . In this embodiment, since a game is executed as content, the output unit 532 outputs additional sounds corresponding to environmental sounds based on the type of game.

Specifically, the output unit 532 outputs the sound information selected by the selection unit 533 as additional sound. The additional sound output by the output unit 532 is output as an actual sound from the speaker 6b. At this time, an additional sound controlled by the ECU 53 is output from the speaker 6b in addition to the game sound unique to the game. As a result, various additional sounds can be output according to the environmental sounds around the moving body 5, compared to the case where the game is executed by the game device 3 alone, so that the environment can be played without disturbing the game (content). It is possible to provide a realistic user experience in harmony with sound.

Such processing will be described with reference to FIG. As shown in FIG. 6, a game image of the game being executed by the user is displayed on the display section 6a. In the example of FIG. 6, the game image is a multi-quest game image. The game image includes, for example, a display of consumed stamina QS, a message 101, a deck 102, a player character P, and an enemy character E, and a button for enabling detailed information display of skill information 103 and mission QM. are provided respectively.

Here, the message 101 is, for example, a text notifying that it is the user's turn in a turn-based game. The deck 102 is made up of player characters P selected by the user. The enemy character E is a character associated with a quest, and is a character against which the player character P fights. The display of the consumed stamina QS is configured so that the user can easily recognize whether the stamina has not yet been consumed or has already been consumed. For example, the consumption stamina QS display lights up or blinks when the stamina is consumed in the middle of the quest.

Next, with reference to FIG. 7, a manner of outputting output information corresponding to environmental sounds will be described. In the example of FIG. 7, the user U operates the controller 4c of the game device 3 to execute the game, the game image is displayed on the display unit 6a, and the game sound is output from the speaker 6b. The type of content being executed by the user U is a game, and the category of the game is a turn-based battle game.

As shown in FIG. 7, in addition to the game sound, additional sounds AS1 and AS2 are output from the speaker 6b. The additional sound AS1 is a sound that expresses the sound of burning flames, and the additional sound AS2 is a sound that expresses the sound of thunder. At this time, the output unit 532 outputs the sound information selected by the selection unit 533 to the speaker 6b, and the speaker 6b outputs the game sound and the additional sound together.

In summary, the system of the present embodiment is configured to be able to execute an acquisition step and an output step. Based on an attribute selected from a plurality of determined attributes, output information corresponding to the environmental sound is output to a user on board the moving object.

Further, the information processing method of the present embodiment includes an acquisition step and an output step, wherein the acquisition step acquires environmental sounds generated inside or outside the mobile body, and the output step acquires a plurality of predetermined sounds. Based on the attribute selected from the attributes, output information corresponding to the environmental sound is output to the user on board the moving body.

According to this embodiment, it is possible to provide a new stimulus to the user. In particular, it is possible to provide the user with a new experience with a sense of realism that corresponds to automatic driving.

In this way, with the development of CASE and MaaS (Mobility as a Service), next-generation mobility is spreading, and the experience provided by the information processing system 1 can be a new stimulus for users.

[Second embodiment]
FIG. 7 is a block diagram showing the hardware configuration of the information processing system 1 according to the second embodiment. In the first embodiment, the mobile body 5 and the information processing device 2 are communicably connected via the communication network 11, but in the second embodiment, the mobile body 5 and the information processing device 2 do not communicate, This is an example in which the mobile object 5 and the game device 3 communicate with each other.

In the second embodiment, game data is downloaded to the game device 3 and the game can be executed on the game device 3 . At this time, the game device 3 is paired with the ECU 53 , and the game screen and game sound of the game executed by the game device 3 are output from the display unit 6 a and the speaker 6 b of the moving body 5 . Then, the ECU 53 outputs the output information corresponding to the environmental sound to the user on board the moving body 5 based on the attribute selected from a plurality of predetermined attributes. Thus, the ECU 53 implements the system according to the present embodiment.

[others]
The following aspects may be adopted for the information processing system 1 according to the above-described embodiment.

As shown in FIG. 8 , each functional unit of the ECU 53 may be mounted on the information processing device 2 . At this time, the information processing device 2 is configured by a server. The control unit 23 of the information processing device 2 includes an acquisition unit 231 , an output unit 232 , a selection unit 233 and a noise canceling unit 234 . The functions of the acquisition unit 231 through the noise canceling unit 234 are the same as the functions of the acquisition unit 531 through the noise canceling unit 534 . In the example of FIG. 8, the system according to the present embodiment is realized by the information processing device 2 instead of the ECU 53 .

Also, an additional image may be output as the output information corresponding to the environmental sound. As shown in FIG. 9, the display unit 6a may display an additional image AI1 and an additional image AI2 in addition to the game image. The additional image AI1 is an image representing flames, and the additional image AI2 is an image representing lightning. At this time, the output unit 532 outputs the image information selected by the selection unit 533 to the display unit 6a, and the display unit 6a displays the game image and the additional image together. Here, the image information is information representing an additional image, and is information having the same structure as the sound information shown in FIG. Also, the speaker 6b may output the additional sound AS1 and the additional sound AS2 together with the game sound. At this time, the additional sound AS1 may correspond to the additional image AI1, and the additional sound AS2 may correspond to the additional image AI2. In the example of FIG. 9, the additional image AI1 and the additional sound AS1 are information expressing flames, and the additional image AI2 and the additional sound AS2 are information expressing thunder.

Further, in A18 of FIG. 4, sound information may be selected based on the table shown in FIG. 5 in consideration of the situation information acquired in A14 and the progress of the game acquired in A15. For example, as the situation information, the moving speed of the mobile object 5 is within a predetermined range, the weather around the mobile object 5 is rainy, and it is the morning of December when the mobile object 5 is boarded. is obtained, select the particular sound information. Also, as game situation information, during a boss battle or during a specific event, the volume of the additional sound may be increased, the volume of the additional sound may be decreased, or noise may be generated without outputting the additional sound. Only the canceling process may be executed. In addition, although the relationship (table) between the sound information and the environmental sound according to the game title or game category has been described with reference to FIG. 5, the present invention is not limited to this. For example, a relationship (table) between sound information and environmental sounds may be prepared for each piece of game situation information, and different additional sounds may be played for the same environmental sound during a battle with a boss and during a non-battle. Furthermore, when using a specific item, an additional sound related to the item may be output. For example, when an acceleration item is used in a racing game, an additional sound representing the acceleration state (eg, boom, hoon, here we go) may be output.

Here, a plurality of additional sounds having different volumes may be stored in the storage unit 52, and the output sound of a specific additional sound may be adjusted by controlling the speaker 6b. In this way, the output unit 532 may output output information based on status information. Furthermore, the output unit 532 may output an additional sound corresponding to the environmental sound based on the progress of the game.

Specific examples of additional sounds for outputting output information based on situation information are listed below.
<Specific example>
・When accelerating: “Boooooooooooooooooooooooooooooooooooooooo!”
・When stopped: "Kiki----!"
・When curving: “Kurukurukyuru!”
・There are other vehicles in the vicinity: "Boo (honk)""Hello!"
・When being overtaken by another vehicle: "Bye-bye""That's a nice car~"
・It's snowing: "Woohoo"
・It's raining: "Wow!"
・When passing through an intersection: “Soooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo”
・When passing through uneven roads: "Gasheran""Ponyon"

Further, when the output information is output based on the situation information, if the driving (or automatic driving) of the moving body 5 is smooth, noise cancellation processing may be executed or an additional sound may be output. Here, smooth operation (or automatic operation) of the moving body 5 means that the acceleration of the moving body 5 is constant, that the number of times of sudden braking is equal to or less than a predetermined value within a predetermined period, and that the car navigation system Alternatively, a comprehensive judgment may be made based on whether the deviation between the route to the destination set by the multifunction information terminal as the game device 3 and the actual travel route is less than a predetermined standard.

Further, noise cancellation processing may be executed or additional sound may be output according to the road surface condition. For example, the additional sound may be output while the vehicle is traveling on an unpaved road without outputting the additional sound while traveling on the asphalt. In addition, when the moving body 5 runs on something that should be avoided, for example, rumble strips and thin pavement installed on a highway, the user is notified that the moving body 5 is running on these. The additional sound may be output at a high volume to convey the message.

In addition, when output information is output based on the progress of the game, even if the same environmental sound is used, the sound of a draft can be used as an additional sound for an indoor conversation scene in the game, and the wind and plants can be used for an outdoor conversation scene. A shaking sound may be output. Further, when the moving object 5 passes through a step, the sound of a bomb being dropped near the character is played as an additional sound if the game being executed by the game device 3 is a fighting game, and the conversation scene in the game is played as an additional sound. A noise sound corresponding to the virtual space (for example, a restaurant, an office, or a school) may be output as an additional sound. Also, if the game progresses well, noise cancellation processing may be executed, or an additional sound may be output. Here, whether or not the game progresses well is determined based on the progress of the game acquired by the ECU 53, the operation history of the user operating the controller 4c, parameters in the game (rank, points, acquired score), and the like. be. Furthermore, noise cancellation processing may be executed or additional sound may be output when the score obtained in the game, parameters such as HP are high/low, or when there is an abnormality such as poison. For example, when the character's HP is low and dying, when the character is poisoned, when the rifle is aimed at the sights, when a penalty kick event occurs in a soccer game, or when a free throw event occurs in a basketball game. When it is time to concentrate, noise cancellation processing may be performed, or additional sound may be output. Also, at this time, only the noise canceling process may be executed without outputting the additional sound.

Further, in A18 of FIG. 4, the output information may be output based on the content type different from the content being executed by the game device 3. FIG. For example, when the game device 3 is executing the game X, an additional sound based on the world view of the game Y, which is another game, may be output. When the game X is a racing game for adults and the game Y is a racing game for children, in addition to the tense game sound of the game X, a comical or relaxing additional sound corresponding to the game Y is output. By doing so, it is possible to give a new stimulus to the user. Here, the additional sound corresponding to game Y is stored in the storage unit 52 in advance, and is selected by the selection unit 533 based on at least one of environmental sound, situation information, and progress of the game.

Also, between A17 and A18 in FIG. 4, an adjustment process for adjusting the sound information may be executed based on the environmental sound. This adjusts the volume, speed, pitch, and atmosphere (eg, rock style, ballad style, bass emphasis, high temperature emphasis) of sound information to be output according to environmental sounds. For example, normal sound information may be output when the environmental sound is low, and sound information with emphasized bass may be output when the environmental sound is loud.

Further, when the ECU 53 can acquire the game sound itself from the game corresponding to the function realized by the ECU 53, the game sound is adjusted based on the environmental sound between A17 and A18 in FIG. Adjustment processing may be performed. When the game sound itself is adjusted, the adjusted game sound becomes the additional sound.

At this time, the configuration may be such that the user can manually adjust the beat, pitch, etc. of the additional sound.

Further, instead of the selection step in A17 of FIG. 4, a conversion process for converting environmental sounds into output information may be executed. At this time, the conversion process may be executed based on an algorithm stored in the storage unit 52 for each content attribute. As a result, for example, environmental sounds can be converted into sound effects or character voices that match the world view of the content, and can be output from the speaker 6b. In addition, when the moving object 5 is a light vehicle, the environmental sound can be converted into a dynamic engine sound of a luxury sports car and output from the speaker 6b. Here, the conversion processing may be realized by converting basic sound information according to the attribute of the content based on the loudness, volume, pitch, duration, etc. of the environmental sound. Specifically, modulation may be performed so that the faster the beat of the environmental sound, the faster the beat of the basic sound information.

Also, instead of the selection step in A17 of FIG. 4, a generation process of generating output information may be executed based on environmental sounds. This uses a learning model generated by machine learning to generate sound information each time without selecting sound information stored in the storage unit 52 . At this time, the generating process may be executed based on an algorithm stored in the storage unit 52 for each game or game category.

Further, in A22 of FIG. 4, instead of outputting the attributes of the game being executed by the game device 3, the user operates the screen displayed on the display unit 6a of the moving object 5, and the user manually inputs the attributes. May be set. Then, even if the content is not being executed by the computer (game device 3) based on the user's instruction, the user manually sets the attribute, and the attribute selected from a plurality of predetermined attributes (that is, the user The output information corresponding to the environmental sound may be output to the user on board the moving body 5 based on the manually set attributes). Note that this manual attribute setting is performed not via the display unit 6a of the mobile object 5, but via an additional sound output application or a web browser installed in a smartphone, which is an example of the game device 3. good too. Also, if the moving body 5 can communicate with the information processing device 2 , the game attributes may be obtained from the information processing device 2 .

Also, the game sound and the environmental sound obtained from the sound collector 6e may be output from the speaker 4b in the following manner.
・Output the game sound from the speaker 4b, and output the environmental sound from the speaker 4b without processing. ・Output the game sound from the speaker 4b, process the environmental sound, and output it from the speaker 4b. ・Output the game sound from the speaker 4b. Output the additional sound corresponding to the environmental sound from the speaker 4b. Output the game sound from the speaker 4b, further process the additional sound corresponding to the environmental sound and output it from the speaker 4b. Alternatively, if game sound information related to the game sound can be obtained from the game device 3, the game sound is processed and output from the speaker 4b, and the environmental sound is output from the speaker 4b in any of the above modes.

Alternatively, the game sound may be output from the speaker 4b and the additional sound may be output from the speaker 6b. Conversely, the game sound may be output from the speaker 6b and the additional sound may be output from the speaker 4b. Also, the game sound and the additional sound may be output from the speaker 4b. Furthermore, game sounds and additional sounds may be output from the speaker 6b. Thus, the output destination and number of game sounds, additional sounds, and environmental sounds are not limited, and speakers include earphones, headphones, bone conduction devices, and the like. Output includes control for instructing output even if the playback volume is zero.

Also, as the output information, an image may be output instead of the sound information. Here, such an image is an additional image to the game image, and the additional image may be superimposed on the game image and displayed on the display section 6a of the moving body 5. FIG. Here, the additional image is an image displayed corresponding to the environmental sound, and the additional image corresponding to the environmental sound selected by the selection unit 533 from a plurality of images prepared in advance is displayed on the display unit 6a. may be displayed. Also, the additional image may display an image corresponding to an item used in the game. For example, when an acceleration item is used while a racing game is being executed on the game device 3, an image representing the acceleration state (eg, smoke, an object on a line) may be displayed. Furthermore, characters corresponding to onomatopoeia such as sound effects and characters' words may be displayed.

Here, the output destination of the output information is not limited to the display unit 6a or the speaker 6b, and the output information may be output to the display 4a or the speaker 4b of the game device 3. FIG. Furthermore, the game screen and game sound may be output to the display 4a and the speaker 4b, and the output information (additional image or additional sound) may be output to the display section 6a or the speaker 6b. Conversely, the game screen and game sound may be output to the display section 6a or the speaker 6b, and the output information (additional image or additional sound) may be output to the display 4a and the speaker 4b.

Also, the noise canceling process may identify the user's voice and exclude the user's voice from the noise canceling process.

Also, the additional sound may be output louder when the vehicle is traveling at high speeds than when traveling at low speeds, without executing the noise canceling process. In addition, when an emergency situation such as an ambulance siren sound or poor physical condition of a person is detected, or when an emergency situation for the moving body 5 occurs, the additional sound may be output at a lower level than in normal times. good. Furthermore, these processes may be executed in combination with the above-described process for controlling the degree of noise canceling.

Alternatively, position information acquired from a multifunctional information terminal functioning as the game device 3 or a car navigation system (GPS) (not shown) may be used to output output information. For example, when the moving body 5 is traveling near the sea, the sound of waves, the sound of the wind, and the cries of seagulls may be output from the speaker 6b. Further, when the surroundings of the moving body 5 are not the sea, the image of the sea may also be displayed on the display section 6a to allow the user to experience driving along the coast in a simulated manner. In addition, when the moving object 5 is traveling near an area associated with content among the areas where content such as animations, TV dramas, and movies are set, for example, characters in the animation The output information may be words (voice), BGM, sound effects, and the like.

Moreover, the weather information acquired from the multifunctional information terminal functioning as the game device 3 may be used to output the output information. For example, when it is snowing around the mobile object 5, a sound representing snow (eg, whistling) may be output from the speaker 6b. Also, when it is raining around the moving body 5, a sound representing rain (eg, za) may be output from the speaker 6b. Furthermore, when the temperature around the moving body 5 is high, a sound reminiscent of summer may be output from the speaker 6b. In the case of Japan, a cicada's cries (e.g., min min min) can be mentioned as a sound associated with summer.

Furthermore, when the state of the traffic light (red, yellow, green, etc.) is received via the communication unit 51, the state of the signal changing from red to green may be output as a sound corresponding to the attribute (for example, racing game). For example, 3 seconds before the traffic light changes from red to green, a countdown sound such as "3, 2, 1, start!" You may output surrounding safety as a sound. That is, the situation around the vehicle determined by the ECU 53 as the autonomous vehicle runs autonomously may be output as a voice corresponding to the attribute. As a result, the voice output by the self-driving car changes according to the attributes set by the user, so that the user is likely to become attached to the self-driving car. For example, if the attribute is a game, while listening to the game sound of the game, the user can know the situation of the self-driving car by the sound corresponding to the game. Here, the sound corresponding to the game has the following contents, for example.
<Specific example>
・"I have confirmed the safety around the vehicle. I will depart when the signal changes. 3, 2, 1."
・"The vehicle is following the vehicle in front and driving safely. The game is currently subjugating the boss character. Be careful as the vehicle will drive on a road with many curves. I pray for your consideration."
・Soon, the vehicle will enter an area where you can see the sea.
・"The vehicle will soon arrive at its destination. The game is just one step away from the area where the boss appears. Good work."

Such a system can be expressed as follows. Here, the environmental conditions refer to conditions related to the environment inside or outside the moving body 5, such as traffic conditions around the moving body 5, the presence or absence of obstacles, the position or number of pedestrians, the presence or absence of other vehicles, traffic lights, etc. state, weather, temperature, volume or type of environmental sound, etc. can be used.
a system,
Configured to be able to execute acquisition and output steps,
the acquiring step acquires an internal or external environmental condition of the mobile body;
in the output step, outputting information about the content based on an attribute selected from a plurality of predetermined attributes and the environmental situation;
the content is content being executed by a user on board the mobile object;
system.
Here, the output step is executed by the output unit 532 .

This enables communication between the mobile object 5 and the user according to environmental conditions. The information output by the output step may be any information that can be recognized by the user, such as sound, image (still image or moving image), smell, vibration, light, and the like. Here, an example of a case where the output step outputs audio related to the content is shown below.
<Specific example>
・"Don't worry, I'm doing my best to move the car."
・"I understand the game situation of the passengers. Good luck."
・"Driving may affect the game. Be careful."
・"The road surface is bad and the car is likely to shake. Be conscious of holding on to the straps of the train."
・"Why don't you stop playing the game and enjoy the drive?"

Alternatively, the display unit 6a or a head-mounted display (not shown) may be caused to display scenery of a foreign country, and output information corresponding to the country may be output. For example, a scene of a large-scale beer event held in Germany may be displayed on the display unit 6a or a head-mounted display (not shown), and German folk songs played at the event may be output from the speaker 6b. At this time, if there are mountains around the moving body 5, the scenery of the mountainous region in Germany may be displayed.

In addition, the display unit 6a may display information that visualizes sound information such as environmental sound, noise that has been subjected to noise canceling processing, or the frequency of output information. In addition to the information indicating the frequency, a landscape image that changes based on the noise state or a symbol indicating the noise state such as "circle, cross, triangle" may be displayed. Furthermore, the target object imaged by the imaging unit 6c may be recognized, and the virtual object may be superimposed on the target object. For example, when there is a pedestrian in front, the pedestrian may be surrounded by a rectangle and displayed.

In addition to the type of content (e.g. game, anime, drama, movie), the attributes are the attributes of the vehicle (e.g. luxury car, dump truck, train, ship, airplane), and the attributes of the atmosphere (e.g. cute, cool, etc.). (sad), location, time, period, attribute set for each region, driving situation, or user's mental state.

Further, when the ECU 53 can access setting information of the content, the ECU 53 may change the setting of the content according to the environmental sound. For example, if the content is a game, the ECU 53 controls the weather in the game, the difficulty level, the parameters of objects (eg characters, weapons, armor, items), and the parameters of appearing objects (eg : enemy characters, items), screen brightness, brightness or color, or volume. Such a system can be expressed as follows.
a system,
is configured to be able to execute an acquisition step and a setting control step;
the obtaining step obtains an environmental sound generated inside or outside the moving body;
In the setting control step, based on an attribute selected from a plurality of predetermined attributes, setting of content is controlled according to the environmental sound;
the content is content being executed by a user on board the mobile object;
system.
In addition, in the setting control step, setting of contents may be controlled according to the above-described environmental conditions instead of environmental sounds. A setting control step is performed by ECU53 or the control part 33, for example.

An aspect of this embodiment may be a program that causes a computer to execute each step of the information processing system 1 .

Further, the moving object 5 in this embodiment includes an acquisition unit 531 and an output unit 532 . The acquisition unit 531 is configured to be capable of acquiring environmental sounds generated inside or outside the moving object 5 . The output unit 532 is configured to be capable of outputting output information corresponding to environmental sounds to a user on board the moving body 5 based on an attribute selected from a plurality of predetermined attributes.

Further, the moving body 5 may include a display section 6a and a speaker 6b. The display unit 6a is configured to be able to display an image corresponding to the content being executed by the user on board the moving body 5. FIG. The speaker 6b is configured to output the output information as sound. Furthermore, the moving body 5 may be an automatically driving vehicle capable of autonomously traveling.

In addition, when boarding the mobile object 5, the user authenticates the user using a multifunctional information terminal (eg, smart phone, tablet terminal) or wearable device (eg, smart watch, smart glasses) functioning as the game device 3. , the door of the moving body 5 may be unlocked. Also, the door of the moving body 5 may be unlocked by biometric authentication (eg, fingerprint authentication, face authentication, voice authentication, vein authentication, iris authentication) using the user's vital data. Furthermore, the user is made to walk a predetermined distance while holding these devices, and the drinking level of the user is determined from the degree of wobble in walking. may be controlled.

Further, the moving body 5 is not limited to a vehicle, and may be a bicycle, a motorcycle, an open car, a train, a ship, an airplane, an elevator, an escalator, or the like in which a user using the game device 3 can board. Furthermore, when the user is running a marathon or running, the user can be regarded as a mobile object 5 on which the game device 3 is mounted (portable). In this case, it is assumed that the content executed by the user is not only a game but also music reproduction. Then, for example, when a live sound source is being reproduced, an additional sound reproducing cheers or the like may be output according to the environmental information.

Furthermore, it may be provided in each aspect described below.
In the system, the attribute is a type of content, and the output step outputs the output information based on the type of content executed by a computer based on an instruction from a user on board the mobile object. Do, system.
In the system, the output step outputs the output information based on situation information, and the situation information includes information on movement of the mobile body, information on the environment around the mobile body, and information on time or time. A system that is at least one of
In the system, the attribute is a type of game executed by a computer, the output information is an additional sound for the game, and the output step corresponds to the environmental sound based on the type of the game. A system that outputs the additional sound that
The system is configured to be capable of executing a selection step, wherein the selection step selects sound information corresponding to the environmental sound, and the sound information represents a sound created for each game or category of the game. information, wherein in the output step, the sound information corresponding to the environmental sound is output as the additional sound.
In the system, the output step outputs the additional sound corresponding to the environmental sound based on the progress of the game.
In the system, a noise canceling step can be performed, and in the noise canceling step, noise canceling processing is performed on sounds that can be collected inside the moving body among the environmental sounds. .
In the system, in the noise canceling step, the noise canceling process is controlled based on at least one of progress of the game, conditions of the moving object, and conditions around the moving object.
In the system, the mobile body is a vehicle, the vehicle includes an ECU (Electronic Control Unit), the ECU is a unit that electronically controls the behavior of the mobile body, and is configured to be able to execute the output step. system.
In the above system, the moving body is an autonomous vehicle capable of autonomous driving.
A program that causes a computer to perform each step of the system.
A mobile body comprising an acquisition unit and an output unit, the acquisition unit being configured to be capable of acquiring environmental sounds generated inside or outside the mobile body, and the output unit configured to acquire environmental sounds from a plurality of predetermined attributes A moving object configured to output output information corresponding to the environmental sound to a user on board the moving object based on the selected attribute.
The mobile object includes a display unit and a speaker, the display unit is configured to be able to display an image corresponding to content being executed by a user on board the mobile object, and the speaker outputs the output information. A moving body configured to be capable of being output as sound.
The moving body, wherein the moving body is an autonomously driving vehicle capable of autonomous travel.
Of course, this is not the only case.

Finally, while various embodiments of the present disclosure have been described, these are provided by way of example and are not intended to limit the scope of the disclosure. The novel embodiments can be embodied in various other forms, and various omissions, replacements, and modifications can be made without departing from the gist of the disclosure. The embodiments and modifications thereof are included in the scope and gist of the disclosure, and are included in the disclosure described in the claims and the equivalent range thereof.

1 : Information processing system 2 : Information processing device 3 : Game device 4a : Display 4b : Speaker 4c : Controller 5 : Moving body 6a : Display unit 6b : Speaker 6c : Imaging unit 6d : Sensor 6e : Sound collecting unit 11 : Communication network 20: communication bus 21: communication unit 22: storage unit 23: control unit 30: communication bus 31: communication unit 32: storage unit 33: control unit 34a: graphic processing unit 34b: audio processing unit 34c: operation unit 50: communication bus 51: communication unit 52: storage unit 53: ECU
54a: Graphic processing unit 54b: Audio processing unit 101: Message 102: Deck 103: Skill information 231: Acquisition unit 232: Output unit 233: Selection unit 234: Noise canceling unit 531: Acquisition unit 532: Output unit 533: Selection unit 534: Noise canceling part AI1: Additional image AI2: Additional image AS1: Additional sound AS2: Additional sound E: Enemy character P: Player character QM: Mission QS: Consumed stamina U: User X: Game X1: Additional sound X2: Additional Sound Y: game

Claims (11)

a system,
Configured to be able to execute acquisition and output steps,
the obtaining step obtains an environmental sound generated inside or outside the moving body;
In the output step, output information corresponding to the environmental sound is generated based on a first content type selected from a plurality of predetermined contents , a progress status of the first content, and status information. output to a user on board the mobile object;
the situation information is at least one of information on movement of the mobile body, information on the environment around the mobile body, and information on time or time;
the output information includes an additional sound added to the audio of the first content and an additional image output corresponding to the additional sound;
wherein the first content is a game executed by a computer;
system. The system of claim 1, wherein
In the output step, outputting the output information based on a type of content executed by a computer based on an instruction from a user on board the mobile object;
system. The system of claim 1 , wherein
configured to be able to execute the selection step,
In the selecting step, sound information corresponding to the environmental sound is selected;
The sound information is information representing a sound created for each game or category of the game,
In the output step, the sound information corresponding to the environmental sound is output as the additional sound.
system. In the system according to any one of claims 1 to 3 ,
configured to be able to execute the noise canceling step,
In the noise canceling step, noise canceling processing is performed on sounds that can be collected inside the moving object among the environmental sounds.
system. 5. The system of claim 4 , wherein
In the noise canceling step, the noise canceling process is controlled based on at least one of the progress of the game, the situation of the moving body, and the surrounding situation of the moving body.
system. In the system according to any one of claims 1 to 5 ,
the moving body is a vehicle,
The vehicle includes an ECU (Electronic Control Unit),
The ECU is a unit that electronically controls the behavior of the moving body, and is configured to be able to execute the output step.
system. In the system according to any one of claims 1 to 6 ,
The moving body is an autonomously driving vehicle capable of autonomous travel,
system. a program,
causing a computer to perform each step of the system according to any one of claims 1 to 7 ,
program. being mobile,
comprising an acquisition unit and an output unit,
The acquisition unit is configured to acquire environmental sounds generated inside or outside the moving object,
The output unit outputs output information corresponding to the environmental sound based on a first content type selected from a plurality of predetermined contents , a progress status of the first content, and status information. configured to be able to output to a user on board the mobile body,
the situation information is at least one of information on movement of the mobile body, information on the environment around the mobile body, and information on time or time;
the output information includes an additional sound added to the audio of the first content and an additional image output corresponding to the additional sound;
wherein the first content is a game executed by a computer;
Mobile. In the mobile body according to claim 9 ,
Equipped with a display and a speaker,
The display unit is configured to be able to display an image corresponding to content being executed by a user on board the mobile body,
The speaker is configured to output the output information as sound,
Mobile. In the moving body according to claim 9 or claim 10 ,
The moving body is an autonomously driving vehicle capable of autonomous travel,
Mobile.

Images