JP2020154994A - Agent system, agent server, control method of agent server, and program - Google Patents

Abstract

To provide an agent system, an agent server, a control method of the agent server, and a program that permit a user to easily determine an agent function.SOLUTION: An agent system according to an embodiment comprises: a plurality of agent function units which are mounted on a different plurality of objects, and each of which provides a service including a response based on a voice according to speech of a user; and an information provision unit that provides attribute information which is associated with each of the same kind of agent function unit by being included in response content by means of the same kind of agent function unit to a portable type mobile terminal of the user, when there is the same kind of agent function unit in the plurality of objects of the plurality of agent function units.SELECTED DRAWING: Figure 4

Description

The present invention relates to an agent system, an agent server, an agent server control method, and a program.

Conventionally, a technology related to an agent function that provides information on driving support according to a request of a occupant, vehicle control, other applications, etc. while interacting with a vehicle occupant has been disclosed (see, for example, Patent Document 1). ..

Japanese Unexamined Patent Publication No. 2006-335231

In recent years, practical application has been promoted for mounting a plurality of agent functions on an object such as a vehicle. However, when a user owns a plurality of objects equipped with the agent function, the load is mounted on which object. In some cases, it could not be determined whether the information was provided by the agent function.

The present invention has been made in consideration of such circumstances, and an object of the present invention is to provide an agent system, an agent server, an agent server control method, and a program that can make it easier for a user to identify an agent function. It is one of.

The agent system, the agent server, the control method of the agent server, and the program according to the present invention have adopted the following configurations.
(1): The agent system according to one aspect of the present invention includes a plurality of agent function units mounted on a plurality of different objects, each of which provides a service including a voice response in response to a user's utterance, and the plurality of agents. When the same type of agent function unit exists in the plurality of objects, the attribute information associated with each of the same type agent function unit is returned by the same type agent function unit. It is an agent system including an information providing unit provided to the user's portable mobile terminal, including the contents.

(2): In the aspect of (1) above, the plurality of objects include a vehicle associated with the user.

(3): In the aspect of (2) above, the attribute information includes information about the vehicle.

(4): In the aspect of (2) or (3) above, among the plurality of agent function units, a setting unit for setting an output mode of the response in a predetermined agent function unit is further provided.

(5): In the aspect of (4) above, the predetermined agent function unit is an agent function unit capable of controlling the vehicle.

(6): In the above aspect (4) or (5), the predetermined agent function unit acquires the response contents of other agent function units included in the plurality of agent function units.

(7): In any one of the above (4) to (6), the predetermined agent function unit substitutes the response content acquired from the other agent function unit to the other agent function unit. Is output to the mobile terminal.

(8): In the agent server according to another aspect of the present invention, an agent function unit that provides a service including a voice response in response to a user's utterance, and the agent function unit are associated with the user. When the agent function unit mounted on a plurality of objects and mounted on each of the plurality of objects is an agent function unit of the same type, the attribute information associated with each of the agent function units of the same type is described. It is an agent server including an information providing unit provided to the user's portable mobile terminal, including the response contents by the agent function unit of the same type.

(9): The agent server control method according to another aspect of the present invention is a service in which a computer activates an agent function unit and, as a function of the activated agent function unit, responds to a user's utterance. When the agent function unit is mounted on a plurality of objects associated with the user and the agent function unit mounted on each of the plurality of objects is an agent function unit of the same type, the above is provided. This is an agent server control method in which attribute information associated with each of the agent function units of the same type is included in the response content of the agent function unit of the same type and provided to the user's portable mobile terminal.
(10): In the program according to another aspect of the present invention, the computer activates the agent function unit, and as a function of the activated agent function unit, provides a service including a response in response to a user's utterance. When the agent function unit is mounted on a plurality of objects associated with the user and the agent function unit mounted on each of the plurality of objects is the same type of agent function unit, the same type of agent function unit is used. It is a program that includes the attribute information associated with each of the agent function units in the response content by the agent function unit of the same type and provides the user with the portable mobile terminal.

According to the above aspects (1) to (10), it is possible to make it easier for the user to discriminate the agent function.

It is a block diagram of the agent system 1 including the agent apparatus 100. It is a figure which shows the structure of the agent apparatus 100 which concerns on 1st Embodiment, and the apparatus mounted on the vehicle M. It is a figure which shows the arrangement example of a display / operation apparatus 20 and a speaker unit 30. It is a figure which shows the configuration of the agent server 200 and the mobile terminal 300 which concerns on 1st Embodiment, and a part of the configuration of agent apparatus 100. It is a figure which shows an example of the content of agent management information 260. It is a figure which shows an example of the image IM1 for setting the output mode of an agent. It is a figure which shows an example of the content which information is provided through a mobile terminal 300. It is a flowchart which shows an example of the flow of the process executed by the agent system 1 which concerns on 1st Embodiment. It is a figure which shows an example of the configuration of the agent server which concerns on 2nd Embodiment. It is a figure which shows an example of the information output from a mobile terminal 300 in 2nd Embodiment. It is a flowchart which shows an example of the flow of the process which agent server 200 # causes another agent server to make a proxy response. It is a flowchart which shows an example of the processing flow which own agent makes a proxy response based on the response content received from another agent server.

Hereinafter, the agent system, the agent server, the control method of the agent server, and the embodiment of the program of the present invention will be described with reference to the drawings. An agent device is a device that realizes a part or all of an agent system. Hereinafter, as an example of the agent device, an agent device mounted on a vehicle and having a plurality of types of agent functions will be described. A vehicle is an example of an object. The agent function is, for example, providing various information based on a request (command) included in a user's utterance, managing a user's schedule, and mediating a network service while interacting with a vehicle user. It is a function to do. The functions, processing procedures, controls, output modes and contents of the plurality of types of agents may be different from each other. In addition, some of the agent functions may have a function of controlling equipment in the vehicle (for example, equipment related to driving control and vehicle body control).

Agent functions include, for example, a voice recognition function that recognizes a user's voice (a function that converts voice into text), a natural language processing function (a function that understands the structure and meaning of text), a dialogue management function, and a network. It is realized by integratedly using a network search function or the like that searches for another device or a predetermined database owned by the own device. Some or all of these functions may be realized by AI (Artificial Intelligence) technology. In addition, a part of the configuration for performing these functions (particularly, the voice recognition function and the natural language processing interpretation function) is an agent server capable of communicating with the in-vehicle communication device of the vehicle M or the general-purpose communication device brought into the vehicle M. It may be mounted on (external device). In the following description, it is assumed that a part of the configuration is installed in the agent server, and the agent device and the agent server cooperate to realize the agent system. Further, a service provider (service entity) in which an agent device and an agent server cooperate to appear virtually is called an agent.

<Overall configuration>
FIG. 1 is a configuration diagram of an agent system 1 including an agent device 100. The agent system 1 includes, for example, agent devices 100A, 100B, ... Mounted on each of a plurality of vehicles MA, MB, ... Associated with the user U1, and a plurality of agent servers 200-1, 200-2, 200. -3, ... And the mobile terminal 300 associated with the user U1. "Associating with user U1" corresponds to, for example, being owned by user U1, managed by user U1, or assigned to user U1. Further, in the above, the number below the hyphen at the end of the code shall be an identifier for distinguishing agents. When it is not distinguished which agent server it is, it may be simply referred to as an agent server 200, and the same applies to other configurations. Although three agent servers 200 are shown in FIG. 1, the number of agent servers 200 may be two or four or more. Each agent server 200 is operated by, for example, different agent system providers. Therefore, the agents in this embodiment are agents realized by different providers. Examples of providers include automobile manufacturers, network service providers, e-commerce businesses, sellers and manufacturers of mobile terminals, and any entity (corporation, group, individual, etc.) is the provider of the agent system. Can be.

The agent devices 100A, 100B, ... Have almost the same functions except that they are mounted on different vehicles MA, MB, .... Therefore, when it is not distinguished which agent device, the agent devices 100A, 100B, ... It is called 100. Further, when it is not distinguished which vehicle the vehicle is, it is simply referred to as a vehicle M. Further, although FIG. 1 shows two agent devices 100, the number of agent devices may be three or more. The agent device 100 communicates with the agent server 200 via the network NW. Further, the agent server 200 communicates with the mobile terminal 300 via the network NW. Further, the agent device 100 may communicate with the mobile terminal 300 via the network NW. The network NW includes, for example, a part or all of the Internet, cellular network, Wi-Fi network, WAN (Wide Area Network), LAN (Local Area Network), public line, telephone line, wireless base station and the like. Various web servers 400 are connected to the network NW, and the agent device 100, the agent server 200, or the mobile terminal 300 can acquire web pages from the various web servers 400 via the network NW.

The agent device 100 interacts with the user U1, transmits the voice from the user U1 to the agent server 200, and sends the response content based on the answer obtained from the agent server 200 to the user U1 in the form of voice output or image display. provide. Here, the agent device 100 provides information using the display unit and the speaker unit mounted on the vehicle M when the user U1 exists in the vehicle, and when the user U1 does not exist in the vehicle M, the agent device 100 provides information. , Information may be provided to the mobile terminal 300 of the user U1. Further, the agent device 100 may control the vehicle device 50 or the like based on a request from the user.

<First Embodiment>
[vehicle]
FIG. 2 is a diagram showing the configuration of the agent device 100 according to the first embodiment and the equipment mounted on the vehicle M. The vehicle M includes, for example, one or more microphones 10, a display / operation device 20, a speaker unit 30, a navigation device 40, a vehicle device 50, an in-vehicle communication device 60, an occupant recognition device 80, and an agent device. 100 and are installed. Further, a general-purpose communication device 70 such as a smartphone may be brought into the vehicle interior and used as a communication device. These devices are connected to each other by a multiplex communication line such as a CAN (Controller Area Network) communication line, a serial communication line, a wireless communication network, or the like. The configuration shown in FIG. 2 is merely an example, and a part of the configuration may be omitted or another configuration may be added. A combination of the display / operation device 20 and the speaker unit 30 is an example of the “output unit”.

The microphone 10 is a voice input unit that collects sounds emitted in the vehicle interior. The display / operation device 20 is a device (or a group of devices) capable of displaying an image and accepting an input operation. The display / operation device 20 includes, for example, a display device configured as a touch panel. The display / operation device 20 may further include a HUD (Head Up Display) or a mechanical input device. The speaker unit 30 includes, for example, a plurality of speakers (audio output units) arranged at different positions in the vehicle interior. The display / operation device 20 may be shared by the agent device 100 and the navigation device 40. Details of these will be described later.

The navigation device 40 includes a navigation HMI (Human Machine Interface), a positioning device such as a GPS (Global Positioning System), a storage device that stores map information, and a control device (navigation controller) that performs route search and the like. .. A part or all of the microphone 10, the display / operation device 20, and the speaker unit 30 may be used as the navigation HMI. The navigation device 40 searches for a route (navigation route) for moving from the position of the vehicle M specified by the positioning device to the destination input by the user, so that the vehicle M can travel along the route. , Navi HMI is used to output guidance information. The route search function may be provided in a navigation server accessible via the network NW. In this case, the navigation device 40 acquires a route from the navigation server and outputs guidance information. The agent device 100 may be constructed based on the navigation controller. In that case, the navigation controller and the agent device 100 are integrally configured on the hardware.

The vehicle device 50 is, for example, a device mounted on the vehicle M. The vehicle equipment 50 includes, for example, a driving force output device such as an engine or a traveling motor, an engine start motor, a door lock device, a door opening / closing device, a window, a window opening / closing device, a window opening / closing control device, a seat, and a seat position. Control device, room mirror and its angle position control device, lighting device inside and outside the vehicle and its control device, wiper and defogger and their respective control devices, direction indicator and its control device, air conditioner, mileage and tire pressure information And vehicle information devices such as fuel level information.

The in-vehicle communication device 60 is, for example, a wireless communication device that can access the network NW using a cellular network or a Wi-Fi network.

The occupant recognition device 80 includes, for example, a seating sensor, a vehicle interior camera, an image recognition device, and the like. The seating sensor includes a pressure sensor provided at the lower part of the seat, a tension sensor attached to the seat belt, and the like. The vehicle interior camera is a CCD (Charge Coupled Device) camera or a CMOS (Complementary Metal Oxide Semiconductor) camera installed in the vehicle interior. The image recognition device analyzes the image of the vehicle interior camera and recognizes the presence / absence of a occupant (user) for each seat, the face orientation, and the like. The recognition result by the occupant recognition device 80 is output to the agent device 100 and the agent 200.

FIG. 3 is a diagram showing an arrangement example of the display / operation device 20 and the speaker unit 30. The display / operation device 20 includes, for example, a first display 22, a second display 24, and an operation switch ASSY 26. The display / operation device 20 may further include a HUD 28. Further, the display / operation device 20 may further include a meter display 29 provided on a portion of the instrument panel facing the driver's seat DS. A combination of the first display 22, the second display 24, the HUD 28, and the meter display 29 is an example of the “display unit”.

The vehicle M includes, for example, a driver's seat DS provided with a steering wheel SW and a passenger seat AS provided in the vehicle width direction (Y direction in the drawing) with respect to the driver's seat DS. The first display 22 is a horizontally long display device extending from an intermediate portion between the driver's seat DS and the passenger's seat AS on the instrument panel to a position facing the left end of the passenger's seat AS. The second display 24 is installed at the middle of the driver's seat DS and the passenger's seat AS in the vehicle width direction and below the first display. For example, both the first display 22 and the second display 24 are configured as a touch panel, and include an LCD (Liquid Crystal Display), an organic EL (Electroluminescence), a plasma display, and the like as display units. The operation switch ASSY26 is an integrated dial switch, button type switch, and the like. The HUD 28 is, for example, a device that superimposes an image on a landscape to visually recognize an image. As an example, a virtual image is visually recognized by an occupant by projecting light including an image onto a front windshield or a combiner of a vehicle M. The meter display 29 is, for example, an LCD, an organic EL, or the like, and displays instruments such as a speedometer and a rotational speedometer. The display / operation device 20 outputs the content of the operation performed by the occupant to the agent device 100. The content displayed by each of the above-mentioned display units may be determined by the agent device 100.

The speaker unit 30 includes, for example, speakers 30A to 30F. The speaker 30A is installed on a window pillar (so-called A pillar) on the driver's seat DS side. The speaker 30B is installed under the door near the driver's seat DS. The speaker 30C is installed on the window pillar on the passenger seat AS side. The speaker 30D is installed at the bottom of the door near the passenger seat AS. The speaker 30E is installed in the vicinity of the second display 24. The speaker 30F is installed on the ceiling (roof) of the vehicle interior. Further, the speaker unit 30 may be installed at the lower part of the door near the right rear seat or the left rear seat.

In such an arrangement, for example, when the speakers 30A and 30B exclusively output sound, the sound image is localized in the vicinity of the driver's seat DS. “The sound image is localized” means, for example, determining the spatial position of the sound source felt by the occupant by adjusting the loudness of the sound transmitted to the left and right ears of the occupant. Further, when the sound is output exclusively to the speakers 30C and 30D, the sound image is localized in the vicinity of the passenger seat AS. Further, when the sound is output exclusively to the speaker 30E, the sound image is localized near the front of the passenger compartment, and when the sound is output exclusively to the speaker 30F, the sound image is localized near the upper part of the passenger compartment. Become. Not limited to this, the speaker unit 30 can localize the sound image at an arbitrary position in the vehicle interior by adjusting the distribution of the sound output from each speaker by using a mixer or an amplifier.

[Agent device]
Returning to FIG. 2, the agent device 100 includes a management unit 110, agent function units 150-1, 150-2, 150-3, a pairing application execution unit 160, and a storage unit 170. The management unit 110 includes, for example, an sound processing unit 112, a WU (Wake Up) determination unit 114 for each agent, an agent setting unit 116, and an output control unit 120. Hereinafter, when it is not distinguished which agent function unit it is, it is simply referred to as an agent function unit 150. The three agent function units 150 are shown only as an example corresponding to the number of agent servers 200 in FIG. 1, and the number of agent function units 150 may be two or four or more. It may be. The software arrangement shown in FIG. 2 is shown briefly for the sake of explanation. In practice, for example, the management unit 110 may be optionally interposed between the agent function unit 150 and the in-vehicle communication device 60. It can be modified. Further, in the following, the agent that the agent function unit 150-1 and the agent server 200-1 collaborate to appear is “agent 1”, and the agent function unit 150-2 and the agent server 200-2 cooperate to appear. The agent may be referred to as "agent 2", and the agent caused by the agent function unit 150-3 and the agent server 200-3 working together may be referred to as "agent 3".

Each component of the agent device 100 is realized by, for example, a hardware processor such as a CPU (Central Processing Unit) executing a program (software). Some or all of these components are hardware such as LSI (Large Scale Integration), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), GPU (Graphics Processing Unit), etc. It may be realized by (including circuits), or it may be realized by the cooperation of software and hardware. The program may be stored in advance in a storage device such as an HDD (Hard Disk Drive) or a flash memory (a storage device including a non-transient storage medium), or a removable storage device such as a DVD or a CD-ROM. It is stored in a medium (non-transient storage medium) and may be installed by mounting the storage medium in a drive device.

The storage unit 170 is realized by the above-mentioned various storage devices. Various data and programs are stored in the storage unit 170.

The management unit 110 functions by executing a program such as an OS (Operating System) or middleware.

The sound processing unit 112 of the management unit 110 receives the sound collected from the microphone 10, and is suitable for recognizing a wake-up word (starting word) preset for each agent for the received sound. Sound processing is performed so as to become. The acoustic processing is, for example, noise removal by filtering such as a bandpass filter, sound amplification, and the like. Further, the sound processing unit 112 outputs the sound-processed voice to the WU determination unit 114 and the activated agent function unit together with the agent.

The WU determination unit 114 for each agent exists corresponding to each of the agent function units 150-1, 150-2, and 150-3, and recognizes a wakeup word predetermined for each agent. The WU determination unit 114 for each agent recognizes the meaning of the voice from the voice (voice stream) subjected to the acoustic processing. First, the WU determination unit 114 for each agent detects a voice section based on the amplitude and zero intersection of the voice waveform in the voice stream. The WU determination unit 114 for each agent may perform frame-by-frame speech recognition based on a mixture Gaussian mixture model (GMM) and section detection based on non-speech recognition.

Next, the WU determination unit 114 for each agent converts the voice in the detected voice section into text and converts it into character information. Then, the WU determination unit 114 for each agent determines whether or not the textual character information corresponds to the wakeup word. When it is determined that the word is a wakeup word, the WU determination unit 114 for each agent activates the corresponding agent function unit 150. The agent server 200 may be equipped with a function corresponding to the WU determination unit 114 for each agent. In this case, when the management unit 110 transmits the voice stream to which the sound processing has been performed by the sound processing unit 112 to the agent server 200 and determines that the agent server 200 is a wakeup word, the management unit 110 follows an instruction from the agent server 200. The agent function unit 150 starts. It should be noted that each agent function unit 150 may be always activated and may determine the wakeup word by itself. In this case, the management unit 110 does not need to include the WU determination unit 114 for each agent.

Further, the WU determination unit 114 for each agent recognizes the end word included in the spoken voice by the same procedure as the above-mentioned procedure, and the agent corresponding to the end word is activated ( Hereinafter, if necessary, it is referred to as "starting"), the running agent function unit is stopped (finished). The activated agent may stop the agent when it does not accept the voice input for a predetermined time or longer, or when it receives a predetermined instruction operation for terminating the agent.

The agent setting unit 116 sets the output mode at the time of response for each agent. The output mode is, for example, one or both of an agent image described later or a voice (hereinafter, referred to as an agent voice) for causing the listener (occupant) to recognize that the agent image is emitted in a pseudo manner. The agent setting unit 116 may be able to set only the agent image and the agent voice associated with the predetermined agent function unit among the plurality of agent function units 150-1 to 150-3. The predetermined agent function unit includes, for example, an agent function unit 150-1 capable of controlling the vehicle M. “The vehicle M can be controlled” means that, for example, it is possible to control the vehicle device 50, the navigation device 40, and the display / operation device 20. The details of the setting contents in the agent setting unit 116 will be described later.

The output control unit 120 provides the occupant with services and the like by causing the display unit or the speaker unit 30 to output information such as response contents in response to an instruction from the management unit 110 or the agent function unit 150. The output control unit 120 includes, for example, a display control unit 122 and a voice control unit 124.

The display control unit 122 causes the image to be displayed in at least a part of the area of the display unit in response to an instruction from the output control unit 120. Hereinafter, an image relating to the agent will be described as being displayed on the first display 22. The display control unit 122 generates an agent image under the control of the output control unit 120, and displays the generated agent image on the first display 22. The agent image is, for example, an image of an anthropomorphic agent communicating with an occupant in the vehicle interior. Further, the agent image is, for example, an image of a mode of talking to an occupant. The agent image may include, for example, a facial image such that the facial expression and the facial orientation are recognized by the viewer (occupant) at least. For example, in the agent image, parts imitating eyes and nose are represented in the face area, and the facial expression and face orientation may be recognized based on the positions of the parts in the face area. In addition, the agent image is felt three-dimensionally, and the viewer can recognize the face orientation of the agent by including the head image in the three-dimensional space, or the agent's image can be included by including the image of the main body (body and limbs). The movement, behavior, posture, etc. may be recognized. Further, the agent image may be an animation image. For example, the display control unit 122 may display the agent image in the display area close to the position of the occupant recognized by the occupant recognition device 80, or may generate and display the agent image with the face facing the position of the occupant. ..

The voice control unit 124 causes a part or all of the speakers included in the speaker unit 30 to output voice in response to an instruction from the output control unit 120. The voice control unit 124 may use a plurality of speaker units 30 to control the localization of the sound image of the agent voice at a position corresponding to the display position of the agent image. The position corresponding to the display position of the agent image is, for example, a position where the occupant is expected to feel that the agent image is speaking the agent voice. Specifically, the position is near the display position of the agent image (for example, 2). It is within ~ 3 [cm]).

The agent function unit 150 causes an agent to appear in cooperation with the corresponding agent server 200, and provides a service including a voice response in response to an utterance of a vehicle occupant. The agent function unit 150 may include one to which the authority to control the vehicle M (for example, the vehicle equipment 50) is granted. Further, the agent function unit 150 may be one that cooperates with the general-purpose communication device 70 via the pairing application execution unit 160 and communicates with the agent server 200. For example, the agent function unit 150-1 is given the authority to control the vehicle M (for example, the vehicle device 50). The agent function unit 150-1 communicates with the agent server 200-1 via the vehicle-mounted communication device 60. The agent function unit 150-2 communicates with the agent server 200-2 via the vehicle-mounted communication device 60. The agent function unit 150-3 cooperates with the general-purpose communication device 70 via the pairing application execution unit 160, and communicates with the agent server 200-3.

The pairing application execution unit 160 pairs with the general-purpose communication device 70 by, for example, Bluetooth (registered trademark), and connects the agent function unit 150-3 and the general-purpose communication device 70. The agent function unit 150-3 may be connected to the general-purpose communication device 70 by wired communication using USB (Universal Serial Bus) or the like.

[Agent server and mobile terminal]
FIG. 4 is a diagram showing a configuration of the agent server 200 and the mobile terminal 300 according to the first embodiment and a part of the configuration of the agent device 100. Hereinafter, along with the configuration of the agent server 200 and the mobile terminal 300, the operation of the agent function unit and the like corresponding to the agents of the same type mounted on the vehicle MA and the vehicle MB will be described. Hereinafter, the agent 1 will be used as an example of the agent of the same type, and the letters "A" or "B" are added to the codes of the respective constituent parts of the agent devices 100A and 100B to identify the respective configurations. It shall be. Further, in the following, the description of physical communication using the network NW will be omitted. Further, in the following, the operations of the agent device 100A and the agent server 200-1 will be mainly described.

The agent server 200-1 includes a server-side communication unit 210. The server-side communication unit 210 is, for example, a network interface such as a NIC (Network Interface Card). Further, the agent server 200-1 stores, for example, a voice recognition unit 220, a natural language processing unit 222, a dialogue management unit 224, a network search unit 226, a response content generation unit 228, and an information providing unit 230. A unit 250 is provided. These components are realized, for example, by a hardware processor such as a CPU executing a program (software). Some or all of these components may be realized by hardware such as LSI, ASIC, FPGA, GPU (including circuit part; circuitry), or realized by collaboration between software and hardware. May be good. The program may be stored in advance in a storage device such as an HDD or flash memory (a storage device including a non-transient storage medium), or a removable storage medium such as a DVD or a CD-ROM (non-transient). It is stored in a sex storage medium) and may be installed by attaching the storage medium to a drive device. A combination of the voice recognition unit 220 and the natural language processing unit 222 is an example of the "recognition unit".

The storage unit 250 is realized by the above-mentioned various storage devices. Data and programs such as a dictionary DB (database) 252, a personal profile 254, a knowledge base DB 256, a response rule DB 258, and agent management information 260 are stored in the storage unit 250.

In the agent device 100, the agent function unit 150A-1 transmits, for example, an audio stream input from the sound processing unit 112 or the like, or an audio stream that has undergone processing such as compression or coding to the agent server 200-1. .. When the agent function unit 150A-1 can recognize a command (request content) capable of local processing (processing that does not go through the agent server 200-1), the agent function unit 150A-1 may execute the processing requested by the command. The command capable of local processing is, for example, a command that can be responded to by referring to the storage unit 170A provided in each of the agent devices 100A. More specifically, the command capable of local processing is, for example, searching for the name of a specific person from the telephone directory data existing in the storage unit 170A and calling the telephone number associated with the matching name. It is a command (call the other party). Therefore, the agent function unit 150A-1 may have a part of the functions provided by the agent server 200-1.

When the voice stream is acquired, the voice recognition unit 220 performs voice recognition and outputs textual character information, and the natural language processing unit 222 interprets the character information with reference to the dictionary DB 252. The dictionary DB 252 is, for example, associated with abstract semantic information with respect to character information. The dictionary DB 252 may include list information of synonyms and synonyms. The stages of the processing of the voice recognition unit 220 and the processing of the natural language processing unit 222 are not clearly separated, and the voice recognition unit 220 corrects the recognition result in response to the processing result of the natural language processing unit 222. It may be done by influencing each other.

For example, when the natural language processing unit 222 recognizes the meanings such as "today's weather" and "how is the weather" as the recognition result, the natural language processing unit 222 generates a command replaced with the standard character information "today's weather". As a result, even if there is a character fluctuation in the voice of the request, it is possible to facilitate the dialogue according to the request. Further, the natural language processing unit 222 may recognize the meaning of character information by using artificial intelligence processing such as machine learning processing using probability, or may generate a command based on the recognition result.

Based on the input command, the dialogue management unit 224 outputs the response content to the occupant of the vehicle M (for example, the utterance content to the occupant and the output unit) while referring to the personal profile 254, the knowledge base DB 256, and the response rule DB 258. Image, sound) is decided. The personal profile 254 includes the user's personal information, hobbies and preferences, the history of past dialogues, etc. stored for each user (occupant). The user's personal information includes, for example, address information for communicating with the user's mobile terminal 300. The knowledge base DB 256 is information that defines the relationships between things. The response rule DB 258 is information that defines the actions (answers, device control contents, etc.) that the agent should perform in response to the command.

In addition, the dialogue management unit 224 may identify the user by collating with the personal profile 254 using the feature information obtained from the voice stream. In this case, in the personal profile 254, for example, personal information is associated with voice feature information. The voice feature information is, for example, information on the characteristics of how to speak such as voice pitch, intonation, and rhythm (sound pitch pattern), and the feature amount based on the Mel Frequency Cepstrum Coefficients. .. The voice feature information is, for example, information obtained by having the user utter a predetermined word or sentence at the time of initial registration of the user and recognizing the uttered voice.

The dialogue management unit 224 causes the network search unit 226 to perform a search when the command requests information that can be searched via the network NW. The network search unit 226 accesses various web servers 400 via the network NW and acquires desired information. The "information searchable via the network NW" may be, for example, an evaluation result by a general user of a restaurant in the vicinity of the vehicle M, or a weather forecast according to the position of the vehicle M on that day.

The response content generation unit 228 generates the response content so that the content of the utterance determined by the dialogue management unit 224 is transmitted to the occupant of the vehicle M, and transmits the generated response content to the agent device 100. The response content includes, for example, a response statement provided to the occupant, a control command for each controlled device, and the like. Further, the response content generation unit 228 acquires the recognition result by the occupant recognition device 80 from the agent device 100, and the occupant who made a speech including a command based on the acquired recognition result is a occupant registered in the personal profile 254. If specified, the occupant's name may be called or a response may be generated that resembles the occupant's speech.

The information providing unit 230 refers to the agent management information 260 stored in the storage unit 250 with respect to the response content generated by the response content generating unit 228, and generates the response content corresponding to the output mode of the agent. FIG. 5 is a diagram showing an example of the contents of the agent management information 260. In the agent management information 260, for example, attribute information and agent setting information are associated with a user ID which is identification information for identifying a user and a vehicle ID which is identification information for identifying a vehicle M. The attribute information is, for example, identification information for identifying an agent of the same type mounted on each of a plurality of vehicles associated with a user. The attribute information includes, for example, a vehicle name or a vehicle registration number written on a license plate. Further, the attribute information may be any information that allows the user to identify any one of a plurality of vehicles, and specifically includes information such as an automobile manufacturer, vehicle body color information and shape information, and purchase time. May be good. The agent setting information includes, for example, agent image information and agent audio information set by the agent setting unit 116.

For example, the information providing unit 230 refers to the agent management information 260 stored in the storage unit 250 by using the user ID and the vehicle ID transmitted from the agent function unit 150A-1 together with the voice, and uses the user ID and the vehicle ID as the user ID and the vehicle ID. Acquires the associated agent setting information. Then, the information providing unit 230 generates a response content corresponding to the agent image of the agent setting information and the agent voice, and transmits the generated response content to the agent function unit 150A-1 that has transmitted the voice.

When the agent function unit 150A-1 acquires the response content from the agent server 200-1, the agent function unit 150A-1 instructs the voice control unit 124 to perform voice synthesis or the like and output the voice. Further, the agent function unit 150A-1 generates an agent image in accordance with the voice output, and instructs the display control unit 122 to display the generated agent image, the image included in the response result, and the like. In this way, the agent function in which the virtually appearing agent responds to the occupant of the vehicle M is realized.

Further, the information providing unit 230 is, for example, information to be provided to the user U1 acquired from the agent function unit 150-1, and the user U1 is not on the vehicle MA or MB based on the recognition result from the occupant recognition device 80. In this case, the address information of the mobile terminal 300 is acquired from the personal information stored in the personal profile 254, and the response content is output to the mobile terminal 300 based on the acquired address information. The information provided to the user U1 is, for example, information that should be notified to the user U1 at an early stage, such as information about the vehicle and information reserved by dialogue with the user U1. The details of the function of the information providing unit 230 when providing information to the mobile terminal 300 will be described later.

The agent server 200-1 describes the agent function unit 150A-1 included in the agent device 100A mounted on the vehicle MA and the agent function unit 150B-1 included in the agent device 100B mounted on the vehicle MB, respectively. Perform the function you did.

[Mobile terminal]
The mobile terminal 300 is, for example, a portable terminal device not equipped with an agent function. The mobile terminal 300 is, for example, a smartphone, a tablet terminal, or a wearable terminal. The mobile terminal 300 includes, for example, a terminal-side communication unit 310, a terminal-side microphone 320, a terminal-side display / operation device 330, a terminal-side speaker 340, a terminal-side sound processing unit 350, and a terminal-side output control unit 360. To be equipped. These components are realized, for example, by a hardware processor such as a CPU executing a program (software). Some or all of these components may be realized by hardware such as LSI, ASIC, FPGA, GPU (including circuit part; circuitry), or realized by collaboration between software and hardware. May be good. The program may be stored in advance in a storage device such as an HDD or flash memory (a storage device including a non-transient storage medium), or a removable storage medium such as a DVD or a CD-ROM (non-transient). It is stored in a sex storage medium) and may be installed by attaching the storage medium to a drive device.

The terminal-side communication unit 310 communicates with the agent server 200 and other external devices via the network NW. The terminal-side microphone 320 is a voice input unit that collects the sound emitted by the user U1. The terminal-side display / operation device 330 is a device capable of displaying an image and accepting an input operation. The terminal-side display / operation device 330 includes, for example, a display device configured as a touch panel. The terminal-side display / operation device 330 includes an LCD, an organic EL, a plasma display, etc. as a display unit, and various buttons and a GUI (Graphical User Interface) switch on the screen as an operation unit. The terminal-side speaker 340 is an audio output unit that outputs audio.

The terminal-side sound processing unit 350 receives the sound collected from the terminal-side microphone 320, and performs sound processing on the received sound so that the agent server 200-1 can recognize the sound. The terminal-side sound processing unit 350 transmits the sound-processed voice to the agent server 200-1 via the terminal-side communication unit 310.

The terminal-side output control unit 360 causes the display unit of the terminal-side display / operation device 330 or the terminal-side speaker 340 to output information based on the response content input from the agent server 200-1. The terminal-side output control unit 360 includes, for example, a display control unit 362 and a voice control unit 364. The display control unit 362 causes various images such as an agent image to be displayed in at least a part of the display area of the display unit in response to an instruction from the terminal side output control unit 360. The voice control unit 364 causes the terminal side speaker 340 to output various voices such as agent voice in response to an instruction from the terminal side output control unit 360.

[Function of agent setting part]
Hereinafter, the function of the agent setting unit 116 will be specifically described. The agent setting unit 116 displays an image for setting the output mode of the agent, for example, when an agent setting instruction is received by a user operation on the display / operation device 20, or when the user first uses the agent. The display control unit 122 is generated, and the generated image is output to the display unit of the display / operation device 20 as an agent setting screen.

FIG. 6 is a diagram showing an example of the image IM1 for setting the output mode of the agent. The content, layout, etc. displayed on the image IM1 are not limited to this. The image IM1 includes, for example, an agent selection area A11, an agent image selection area A12, an audio selection area A13, and a GUI switch selection area A14. In addition to the above-mentioned areas, the image IM1 includes a setting area for setting a user ID, a vehicle ID, a vehicle color, and a shape, a setting area for setting attribute information, a setting area for setting an agent name, and the like. May be included.

In the agent selection area A11, for example, information for the user U1 to select an agent for which an output mode is set is displayed. For example, if agent 1 and agent 2 can be set and agent 3 cannot be set among agents 1 to 3, only the options of agent 1 and agent 2 are displayed in the agent selection area A11. .. The agent capable of setting (changing) the output mode of the agent includes an agent capable of controlling the vehicle M (here, agent 1). Since the agent 1 has a higher relationship with the vehicle M than other agents due to the peculiarity of being able to control the vehicle, the agent 1 is frequently used when notifying the mobile terminal 300 of information about the vehicle. Is expected. Therefore, by making it possible to change the output mode of the agent 1 for each vehicle, it is possible to make it easier for the user U1 to identify the vehicle from the information output to the mobile terminal 300. Further, in the example of FIG. 6, a pull-down menu (drop-down menu) is displayed in the agent selection area A11, but instead of this, an agent can be selected by using a GUI such as a list box or a radio button. It may be displayed.

In the agent image selection area A12, for example, one or more predetermined agent images are displayed. In the example of FIG. 6, agent images EI1 to EI3 (Character A to C) are displayed in the agent image selection area A12. The user U1 selects an agent image by touching the display area of any of the agent images EI1 to EI3. In the example of FIG. 6, the agent image selection area A12 shows the state in which the agent image EI1 is selected.

In the voice selection area A13, for example, one or more predetermined agent voices are displayed. The agent voice includes, for example, synthetic voice and information for identifying the voice of an actual person such as a voice actor, a celebrity, or a talent. In the example of FIG. 6, a pull-down menu (drop-down menu) is displayed in the voice selection area A13, but instead of this, the agent voice can be selected and displayed by using a GUI such as a list box or a radio button. May be done.

Various user-selectable GUI buttons are displayed in the GUI switch selection area A14. In the example of FIG. 6, in the GUI switch selection area A14, for example, a setting permission button (OK button) IC1 that permits setting with the contents selected in each area A11 to A13 and a cancel button IC2 that does not permit setting are provided. included. When the user U1 selects the setting permission button IC1, the output mode of the agent is set according to the contents selected in each of the areas A11 to A13. In this case, the agent setting unit 116 associates the setting information with a predetermined (or set) user ID, vehicle ID, attribute information, and the like, and the agent corresponding to the agent selected in the agent selection area A11. Send to server 200. In the example of FIG. 6, since the agent 1 is selected, the setting information (agent image "characterA", agent voice "VoiceA") is the user ID of the user U1 (for example, "U1") and the vehicle ID of the vehicle MA. (For example, "V001") and attribute information (for example, vehicle name "AAA") are transmitted to the agent server 200-1. The agent server 200-1 stores the information transmitted by the agent setting unit 116 in the storage unit 250 as the agent management information 260.

The agent setting unit 116 registers the setting information when, for example, the output mode (agent image, agent voice) set in association with the user ID overlaps with the output mode of another agent in the same vehicle. May be canceled and the user U1 may be notified to set again with a different content. As a result, it is possible to prevent it from becoming difficult to distinguish between a plurality of agents mounted on the same vehicle. Further, the agent setting unit 116 may perform control to set again in the same manner even when the output mode overlaps with the output mode of the agent of the same type of another vehicle that has already been registered.

When the cancel button IC2 is selected, the agent setting unit 116 clears the contents set by the image IM1. In the above example, an example of setting the agent image and the agent sound has been described. However, for example, an image for setting either the agent image or the agent sound is displayed, and the displayed contents are set. You may.

[Function of information provision department]
Next, the function of the information providing unit 230 will be specifically described. In the following, the function of the response content generation unit 228 related to the function of the information providing unit 230 will also be described. The response content generation unit 228 generates response content for notifying the user U1 of the information regarding the vehicle MA when the information regarding the vehicle MA is input as the information provided to the user from the agent function unit 150A-1, for example. The "information about the vehicle" includes, for example, the open / closed state of windows and doors of the vehicle M, the state of fuel and battery (for example, the remaining amount), the on / off state of lights and interior lighting, or the recognition result by the occupant recognition device 80. Of these, at least one is included.

Here, for example, it is assumed that the agent server 200-1 has acquired the information that "the rear seat window is open" from the agent function unit 150A-1 of the vehicle MA. Further, it is assumed that the agent server 200-1 has acquired the information that there is no occupant in the vehicles MA and MB based on the recognition results of the occupant recognition devices 80A and 80B mounted on the vehicles MA and MB.

In this case, the information providing unit 230 causes the mobile terminal 300 of the user U1 to output information about the vehicle MA. In this case, the information providing unit 230 acquires the attribute information corresponding to the user ID and the vehicle ID from the agent management information 260 stored in the storage unit 250, and generates a response content including the acquired attribute information. In the example of FIG. 5, when the user ID of the user U1 is "U1" and the vehicle ID of the vehicle MA is "V001", the response content generation unit 228 outputs the agent image "chargerA" and the agent voice "characterA". The response content of "VoiceA" is generated, the response content including the information of the attribute information (for example, the vehicle name "AAA") is generated, and the generated response content is transmitted to the mobile terminal 300.

The terminal-side output control unit 360 outputs an image corresponding to the response content transmitted by the agent server 200-1 to the display unit of the terminal-side display / operation device 330, and outputs a voice corresponding to the response content to the terminal-side speaker 340. Is output by.

FIG. 7 is a diagram showing an example of the content provided as information via the mobile terminal 300. In the example of FIG. 7, an example of the image IM2 generated by the display control unit 362 and displayed on the display unit of the terminal side display / operation device 330 and the sound output by the terminal side speaker 340 is shown. The image IM2 includes, for example, a character information display area A21 and an agent display area A22. In the character information display area A21, for example, information about an agent interacting with the user U1 is displayed. In the example of FIG. 7, in the character information display area A21, the character information "The agent 1 of" AAA "is in a dialogue" is displayed.

In the agent display area A22, for example, an agent image associated with the agent in conversation and the response content of the agent are displayed. In the example of FIG. 7, the agent image EI1 associated with the user U1 and the agent 1 of the vehicle MA is displayed in the agent display area A22. In this case, the agent image EI1 displays the agent image of the agent setting information set for each vehicle. Further, in the agent display area A22, the character information "The rear seat window is open. Do you want to close it?" Is displayed as the character information corresponding to the response content. In addition, the voice control unit 364 is the agent 1 of "AAA" as the voice corresponding to the response content. The rear seat window of "AAA" is open. Do you want to close it? ", And output the generated voice. In this case, the voice is output as the agent voice associated with the user U1 and the vehicle MA. In this way, by telling the user U1 which vehicle the agent itself is the agent of which vehicle by the character information displayed in the image, the agent image, and the agent voice, it is easy for the user U1 to identify the agent who is interacting with the agent. can do.

Further, when the user U1 utters "close" after the information as shown in FIG. 7 is provided to the mobile terminal 300, the sound processing for the voice of the utterance acquired by the terminal side microphone 320 is performed. The performed and sound-processed voice is transmitted from the terminal-side communication unit 310 to the agent server 200-1. The agent server 200-1 performs voice recognition for the voice received from the mobile terminal 300, generates a command for closing the window of the rear seat of the vehicle MA as a response based on the recognition result, and generates the generated command in the server-side communication unit. It is transmitted from 210 to the agent function unit 150A-1. The agent function unit 150A-1 controls the vehicle equipment 50 according to the received command, and executes the control of closing the rear seat window. As a result, the user U1 can give an appropriate instruction to the specified vehicle.

[Processing flow]
FIG. 8 is a flowchart showing an example of a processing flow executed by the agent system 1 according to the first embodiment. In the following, it is assumed that the user U1 owns two vehicles MA and MB and a mobile terminal 300, which are each equipped with a plurality of agent devices 100A and 100B having the same type of agent function. Further, in the following, the process of providing the information from the agent of the vehicle MA to the mobile terminal 300 will be mainly described. Further, in the following, it is assumed that the agent image and the agent voice of each agent have already been set for each vehicle by the agent setting unit 116. In addition, the following processing shall be repeatedly executed at a predetermined timing or cycle.

In the example of FIG. 8, first, the agent function unit 150 determines whether or not to respond to the user U1 (step S100). The response includes the case of simply providing information rather than during a dialogue. When it is determined that the response to the user U1 is performed, the response content generation unit 228 generates the response content of the output mode corresponding to the agent setting information (agent image, agent voice) based on the content of the information provision ( Step S102). Next, the information providing unit 230 determines whether or not to transmit the generated response content to the mobile terminal 300 of the user U1 (step S104). For example, when the user U1 is not on the vehicle MA or MB, the information providing unit 230 determines that the response content is transmitted to the mobile terminal 300.

When it is determined that the response content is transmitted to the mobile terminal 300, the information providing unit 230 generates the response content including the attribute information of the agent (step S106), and transmits the generated response content to the mobile terminal 300 (step S108). ). As a result, the response content including the attribute information of the agent is output from the mobile terminal 300, and it is possible to make it easy for the user U1 to determine from which vehicle the agent the information is provided.

Further, in the process of step S104, when it is determined not to transmit to the mobile terminal 300, the information providing unit 230 transmits the response content generated by the response content generating unit 228 to the vehicle on which the occupant is riding ( Step S110). The response to the user U1 is the information provided by the agent of the vehicle MA, and when the user U1 is in the vehicle MB (another vehicle), the output of the agent of the same type of the other vehicle MB The response content may be output depending on the mode, or the response content including the attribute information may be generated and the generated response content may be output. As a result, the processing of this flowchart ends. Further, in the process of step S100, when it is determined that the response to the user is not performed, the process of this flowchart ends.

According to the agent system 1 according to the first embodiment described above, when the user owns a plurality of vehicles equipped with a plurality of agent functions, the information provided by the agent is given the attribute information of the agent. , It is possible to make it easy for the user to distinguish which vehicle the agent is mounted on. Further, according to the agent system 1 according to the first embodiment, even when the mobile terminal 300 is not equipped with the agent function, it is possible to realize a dialogue linked with the agent on the vehicle side.

<Second Embodiment>
Next, the agent system according to the second embodiment will be described. For example, depending on the agent mounted on the object, the agent image and the agent voice may not be changed due to restrictions and privileges determined for each agent provider. In this situation, when providing information to the mobile terminal 300, the output modes of the same type of agents mounted on a plurality of objects are the same, so it may not be possible to distinguish which object the agent is. obtain. Therefore, in the second embodiment, when the agent image or the agent voice provides information from the agent that is not set to be distinguishable for each vehicle, the agent image or the agent sound is another agent mounted on the object. The agent (proxy response agent) for which the agent voice is set provides information instead. The agent system of the second embodiment differs from the agent system 1 of the first embodiment only in the configuration of the agent server. Therefore, in the following, the functions of the agent server will be mainly described. Further, in the following description, the same name and reference numerals will be given to the same configurations as those of the first embodiment, and specific description will be omitted.

FIG. 9 is a diagram showing an example of the configuration of the agent server according to the second embodiment. In the example of FIG. 9, as an example, the agent function units 150A-1 to 150A-3 and the agent servers 200 # -1 to 200 # 3 associated with a plurality of different agents 1 to 3 mounted on the vehicle MA are shown. It shall be. Although the agent servers 200 # -1 to 200 # -3 differ in the functions and databases that can be executed by each, the rough processing is the same. Therefore, the agent servers 200 # -1 will be described below. It shall be.

The agent server 200 # -1 is different from the agent server 200 according to the first embodiment in that it includes an information providing unit 230 # instead of the information providing unit 230, for example. Therefore, in the following, the information providing unit 230 # will be mainly described. In addition to having the same functions as the information providing unit 230, the information providing unit 230 # is a case where the response content is output to the mobile terminal 300, and the agent management information 260 contains agent setting information for each vehicle. If not, the response content is output as a proxy by another agent mounted on the same vehicle and having agent setting information for each vehicle.

Specifically, the information providing unit 230 # of the agent server 200 # -1 outputs the response content to the mobile terminal 300, and the agent management information 260 does not include the agent setting information for each vehicle. Inquires about whether or not a proxy response is possible by transmitting a user ID and a vehicle ID to another agent server (for example, agent servers 200 # -2 and 200 # -3).

The information providing unit 230 # of the other agent servers 200 # -2 and 200 # -3 refers to its own agent management information 260 by using the received user ID and vehicle ID, and there is matching agent management information. In this case, it is determined that the proxy response is possible, and if there is no matching agent management information, it is determined that the proxy response is not possible. Regarding the pass / fail judgment of the proxy response, instead of (or in addition to) the above example, the pass / fail judgment may be made based on the pass / fail flag set in advance by the user, and whether or not the proxy response can be transmitted to the mobile terminal 300 is determined. The pass / fail judgment may be made based on the above, or the pass / fail judgment may be made based on other conditions. The information providing unit 230 # of the other agent servers 200 # -2 and 200 # -3 transmits the inquiry result to the agent server 200 # -1.

The information providing unit 230 # of the agent server 200 # -1 sends the response content to the agent server of the other agent servers 200 # -2 and 200 # -3 that has permission for the proxy response, and the destination agent The agent associated with the server is made to output the response contents to the mobile terminal 300. When response permission is given from a plurality of agent servers, the information providing unit 230 # sends the response content to the agent server that first sent the response permission or the agent server having a predetermined high priority. Note that the information providing unit 230 # may forcibly send the response content to a specific agent server and output it as a proxy without making the above inquiry to another agent server.

Upon receiving the response content from the agent server 200 # -1, the information providing unit 230 # of the agent server adds the agent attribute information, the agent type, etc. to the response content to the mobile terminal 300 in the output mode of its own agent. Output.

FIG. 10 is a diagram showing an example of information output from the mobile terminal 300 in the second embodiment. FIG. 10 shows an example of the image IM3 displayed on the display unit of the terminal side display / operation device 330 of the mobile terminal 300. Further, the example of FIG. 10 shows a scene in which the agent 1 associated with the agent server 200 # -1 provides the response content generated by the agent server 200 # -3 of the agent 3. The image IM3 includes, for example, a character information display area A31 and an agent display area A32. In the character information display area A31, information about the agent currently interacting is displayed. In the example of FIG. 10, in the character information display area A31, the character information "The agent 1 of" AAA "is in a dialogue" is displayed.

In the agent display area A32, for example, an agent image during dialogue and a response result (provided information) are displayed. In the example of FIG. 10, in the agent display area A32, the agent image EI1 corresponding to the agent 1 that notifies the user U1 of the response content on behalf of the agent 3, and the response result generated by the agent 3 are "Today 15: The text information "Go out to 00 ○○." Is displayed. The response result output in this scene includes information about the own agent including attribute information, information about the agent that generated the response content, and the response content generated by the agent. In the example of FIG. 10, the agent display area A32 is the agent 1 of "AAA" as the voice information of the response result. "And" Agent 3 has contacted me about my plan to go to XX from 15:00 today. "

In the second embodiment, the proxy response agent may be set so that only a predetermined agent can perform the proxy response agent. In this case, the predetermined agent is, for example, an agent whose output mode of the agent can be set. Further, the predetermined agent includes an agent capable of controlling the vehicle M (here, agent 1). Further, the agent 1 is, for example, selected as a specific agent that has a higher priority for performing a proxy response than other agents or forcibly performs a proxy response. By causing the agent 1 having a high relevance to the vehicle M to make a proxy response, it is possible to make it easier for the user to determine from which vehicle the information is provided.

[Processing flow]
Hereinafter, the flow of processing executed by the agent system according to the second embodiment will be described. The processes executed by the agent system according to the second embodiment include, for example, a process in which the information providing unit 230 # of the agent server 200 # causes another agent server to make a proxy response, and a response content received from the other agent server. Based on the above, the processing is roughly divided into the processing in which the own agent responds by proxy, so each of them will be described separately.

FIG. 11 is a flowchart showing an example of a processing flow in which the agent server 200 # causes another agent server to respond by proxy. The flowchart in FIG. 11 is different from the flowchart in the first embodiment shown in FIG. 8 described above in that the processes of steps S120 to S128 are added. Therefore, in the following, the processing of steps S120 to S128 will be mainly described. When it is determined in the process of step S100 that the information is provided to the user, the information providing unit 230 # determines whether or not the agent management information 260 contains the agent setting information corresponding to the user ID and the vehicle ID. (Step S120). If it is determined that the agent setting information exists, the processes of steps S102 to S110 are executed.

If it is determined that the agent setting information does not exist, the response content generation unit 228 generates the response content (step S122). Next, the information providing unit 230 # inquires of another agent (more specifically, another agent server) mounted on the same vehicle whether or not a proxy response is possible (step S124). Next, the information providing unit 230 # determines whether or not the proxy response agent exists from the inquiry result for step S124 (step S126). If the proxy response agent does not exist, the processes of steps S104 to S110 are executed. If the proxy response agent exists, the information providing unit 230 # transmits the response content to the proxy response agent (step S128).

FIG. 12 is a flowchart showing an example of a processing flow in which the own agent makes a proxy response based on the response contents received from another agent server. In the example of FIG. 12, the information providing unit 230 # determines whether or not an inquiry as to whether or not a proxy response is possible has been received from another agent server (step S200). When receiving an inquiry as to whether or not a proxy response is possible, the information providing unit 230 # refers to the agent management information 260 based on the user ID and the vehicle ID acquired as the inquiry information, and the agent corresponding to the user ID and the vehicle ID. It is determined whether or not the setting information exists (step S202).

If the agent setting information exists, the information providing unit 230 # determines that the proxy response is possible, and notifies the inquired agent server that the proxy response is possible (step S204). Next, the information providing unit 230 # determines whether or not the response content has been received from the agent server that has notified that the proxy response is possible (step S206). When it is determined that the response content has been received, the information providing unit 230 # determines whether or not to transmit the received response content to the mobile terminal 300 of the user U1 (step S208). When it is determined that the response content is transmitted to the mobile terminal 300, the information providing unit 230 # generates the response content including the attribute information of the agent and the agent type that generated the response content (step S210), and generates the generated response content. It is transmitted to the mobile terminal 300 (step S212). As a result, the provided information output by the mobile terminal 300 also includes the attribute information of the agent and the agent type, so that the user U1 can easily identify the vehicle and the agent.

Further, in the process of step S208, when it is determined not to transmit to the mobile terminal 300, the information providing unit 230 # transmits the response content generated by the response content generating unit 228 to the vehicle on which the occupant is riding. (Step S214). If it is determined in the process of step S02 that the agent setting information does not exist, the information providing unit 230 # notifies the inquired agent server that the proxy response is impossible (step S216). As a result, the processing of this flowchart ends. Further, in the process of step S200, when it is determined that the inquiry as to whether or not the proxy response is possible is not accepted, or when it is determined in the process of step S206 that the response content is not received, the process of this flowchart ends. To do.

According to the agent system according to the second embodiment described above, the same effect as that of the agent system in the first embodiment is obtained, and the output mode is set according to the agent mounted on a plurality of vehicles owned by the user. By having other agents in the same vehicle respond on behalf of the user, it is possible to make it easier for the user to distinguish which agent is installed in which vehicle.

Each of the first embodiment and the second embodiment described above may be a combination of some or all of the other embodiments. Further, a part or all of the functions of the agent device 100 may be included in the agent server 200 (200 #). Further, a part or all of the functions of the agent server 200 (200 #) may be included in the agent device 100. That is, the division of functions between the agent device 100 and the agent server 200 (200 #) may be appropriately changed depending on the components of each device, the scale of the agent server 200 (200 #), the agent system, and the like. Further, the division of functions in the agent device 100 and the agent server 200 (200 #) may be set for each vehicle M.

Further, although the vehicle has been described as an example of the object in the first embodiment and the second embodiment, the terminal device installed in place of (or in addition to) the vehicle, for example, at a specific point such as a home or a company. Alternatively, it may be a moving body (for example, a ship, an airplane, a drone, a motorcycle) having an output unit other than a vehicle. Further, the mobile terminal 300 of the above-described embodiment has been described as a mobile terminal not equipped with an agent function, but the present invention is not limited to this, and even if the mobile terminal 300 is equipped with one or more agent functions. Good. In this case, the mobile terminal 300 is equipped with the same function as the agent function unit 150 described above, and the same function as the agent described above is realized via the agent server 200 or the like.

Although the embodiments for carrying out the present invention have been described above using the embodiments, the present invention is not limited to these embodiments, and various modifications and substitutions are made without departing from the gist of the present invention. Can be added.

1 ... Agent system, 10 ... Microphone, 20 ... Display / operation device, 30 ... Speaker unit, 40 ... Navigation device, 50 ... Vehicle equipment, 60 ... In-vehicle communication device, 70 ... General-purpose communication device, 80 ... Crew recognition device, 100 ... Agent device, 110 ... Management unit, 112 ... Sound processing unit, 114 ... WU determination unit for each agent, 116 ... Agent setting unit, 120 ... Output control unit, 122, 362 ... Display control unit, 124, 364 ... Voice control unit , 150 ... Agent function unit, 160 ... Pairing application execution unit, 170, 250 ... Storage unit, 200 ... Agent server, 210 ... Server side communication unit, 220 ... Voice recognition unit, 222 ... Natural language processing unit, 224 ... Dialogue Management unit, 226 ... Network search unit, 228 ... Response content generation unit, 230, 230 # ... Information provision unit, 300 ... Mobile terminal, 310 ... Terminal side communication unit, 320 ... Terminal side microphone, 330 ... Terminal side display / operation Device, 340 ... Terminal side speaker, 350 ... Terminal side sound processing unit, 360 ... Terminal side output control unit, 400 ... Various web servers

Claims (10)

Multiple agent function units mounted on multiple different objects, each of which provides a service including a voice response according to the user's utterance.
When the same type of agent function unit exists in the plurality of objects among the plurality of agent function units, the attribute information associated with each of the same type agent function unit is transmitted to the same type of agent function unit. The information providing unit provided to the user's portable mobile terminal, including the contents of the response by
Agent system with. The plurality of objects include a vehicle associated with the user.
The agent system according to claim 1. The attribute information includes information about the vehicle.
The agent system according to claim 2. Among the plurality of agent function units, a setting unit for setting an output mode of the response in a predetermined agent function unit is further provided.
The agent system according to claim 2 or 3. The predetermined agent function unit is an agent function unit capable of controlling the vehicle.
The agent system according to claim 4. The predetermined agent function unit acquires the response contents of other agent function units included in the plurality of agent function units.
The agent system according to claim 4 or 5. The predetermined agent function unit causes the mobile terminal to output the response content acquired from the other agent function unit on behalf of the other agent function unit.
The agent system according to any one of claims 4 to 6. An agent function unit that provides services including voice responses in response to user utterances,
When the agent function unit is mounted on a plurality of objects associated with the user and the agent function unit mounted on each of the plurality of objects is the same type of agent function unit, the same type of agent An information providing unit that includes the attribute information associated with each of the functional units in the response content of the agent function unit of the same type and provides the user's portable mobile terminal.
Agent server with. The computer
Start the agent function part and
As a function of the activated agent function unit, a service including a response is provided in response to a user's utterance.
When the agent function unit is mounted on a plurality of objects associated with the user and the agent function unit mounted on each of the plurality of objects is the same type of agent function unit, the same type of agent The attribute information associated with each of the functional units is included in the response content by the agent functional unit of the same type and provided to the user's portable mobile terminal.
How to control the agent server. The computer
Start the agent function part and
As a function of the activated agent function unit, a service including a response is provided in response to a user's utterance.
When the agent function unit is mounted on a plurality of objects associated with the user and the agent function unit mounted on each of the plurality of objects is the same type of agent function unit, the same type of agent The attribute information associated with each of the functional units is included in the response content of the agent functional unit of the same type, and is provided to the user's portable mobile terminal.
program.

Images