JP2020074044A - Electronic apparatus and control method for the same - Google Patents

Abstract

To provide: an electronic apparatus for controlling various devices connected through a network so as to match them with each user's lifestyle more; and a control method for the same.SOLUTION: An electronic apparatus determines the execution of control of devices on the basis of the content of a second voice that is input after the input of a first voice, depending on the content of the first voice input from the outside. The apparatus includes: management means that creates voice data for determination for determining that the first voice is a desired voice and manages them and that determines that the first voice is the desired voice by using the created and managed voice data for determination; and control means for executing the control of the devices on the basis of the content of the second voice. The control means executes the control of the devices in accordance with a determination result obtained by the management means, changes a content of the control of the devices on the basis of the second voice when the first voice is determined to be the desired voice, and changes the content of the control of the devices in accordance with types of the voice data for determination.SELECTED DRAWING: Figure 1

Description

  The embodiments of the present invention relate to an electronic device that controls a plurality of devices by voice and a control method thereof in the field of home automation in a home, office, or small business establishment.

  In the field of home automation, there have been voice recognition devices and methods for operating and controlling various devices in homes, offices, and small businesses by voice input.

  This voice recognition device and method analyze a voice input by a user to determine whether or not the input voice is a voice that turns on the function of the device, and the function of the device. When it is determined that the voice is a voice that turns on, the content of the voice that continues is analyzed and processing based on the analysis result is performed. Further, there is also one that recognizes the characteristics of the voice input by the user, identifies the user who uttered the voice, and performs processing suitable for the user.

International Publication No. 2015/029379 International Publication No. 2015/033523

  As a form of the home automation system, there is a system in which the respective devices are connected to each other by a home network, and a host device for totally controlling the connected plurality of devices is connected to the network. In this case, the host device controls the operation of each device connected by the network, and collects information about each device and manages it so that the user can browse it in a unified manner.

  By instructing the host device by voice, for example, the user can control each device connected to the host device via a network and can centrally browse information about each connected device. it can.

  In the home automation system having such a form, the devices to be controlled can be easily connected to the network, so that the number and types of connected devices tend to be large. In addition, there is a tendency that new participation in the network, setting change, and withdrawal from the network often occur due to addition, change, version upgrade, movement of the installation place, disposal, etc. of the device to be controlled. Further, since there are many types of operation contents and specifications of connected devices, home automation systems tend to be used in homes and offices regardless of age or sex. This tendency is becoming more and more remarkable with the recent miniaturization of devices and sensors having various functions.

  However, in conventional home automation systems, control of a wide variety of devices and support for a wide range of users have not been sufficient. For example, when using a home automation system at home, it cannot be said that the devices that match the lifestyle of each family member are sufficiently controlled.

  The present embodiment has been made in view of the above problems, and proposes an electronic device and a control method for controlling a wide variety of devices connected by a network so as to match each user's individual lifestyle. To aim.

  The electronic device according to the embodiment includes one or a plurality of devices based on the content of the first voice input from the outside, based on the content of the second voice input after the input of the first voice. In the electronic device that determines the execution of the control, the creation voice is used to manage the determination voice data for determining whether the first voice is the desired voice by the voice input from the outside multiple times. Management means for determining whether the first voice is a desired voice using the determination voice data, and controlling the one or more devices based on the content of the second voice. When the first voice is determined to be a desired voice by the management voice using the determination voice data, the control means changes the content of the second voice. Based on the above-mentioned one or multiple A part of the content of the second voice when the first voice is determined to be a desired voice by the management means using the determination voice data by controlling the single device. Alternatively, when the control content of the one or more devices is changed based on the entire content and the management unit has a plurality of the determination voice data, it is determined that the first voice is a desired voice. The control content of the one or more devices is changed according to the type of the determination voice data used for the determination.

FIG. 1 is a diagram showing an example of an overview of a home automation system according to an embodiment. FIG. 2 is a list showing another example of the sensor according to the embodiment. FIG. 3 is a diagram illustrating an example of a host device according to an embodiment. FIG. 4 is a functional block diagram of the host device according to the embodiment. FIG. 5A is a diagram showing an example of a processing sequence in registering a reserved word according to an embodiment. FIG. 5B is a diagram showing an example of a processing sequence in registering a reserved word according to the embodiment. FIG. 6A is a diagram showing an example of a processing sequence in registering a reserved word according to an embodiment. FIG. 6B is a diagram showing an example of a processing sequence in registering a reserved word according to the embodiment. FIG. 7A is a diagram showing an example of a processing sequence in registering a reserved word according to the embodiment. FIG. 7B is a diagram showing an example of a processing sequence in registering a reserved word according to the embodiment. FIG. 8A is a diagram showing an example of a processing sequence in recognizing a reserved word according to the embodiment. FIG. 8B is a diagram showing an example of a processing sequence in recognizing a reserved word according to the embodiment. FIG. 9A is a diagram showing an example of a processing sequence in recognizing a reserved word according to the embodiment. FIG. 9B is a diagram showing an example of a processing sequence in recognizing a reserved word according to the embodiment. FIG. 10A is a diagram showing an example of a processing sequence for controlling a corresponding device or sensor based on a word issued by the user for controlling the device or sensor after recognizing the reserved word according to the embodiment. is there. FIG. 10B is a diagram showing an example of a processing sequence for controlling a corresponding device or sensor based on a word issued by the user for controlling the device or sensor after recognizing the reserved word according to the embodiment. is there. FIG. 11A is a diagram showing an example of a processing sequence in the case where words for controlling devices and sensors issued by the user continuously after recognizing the reserved word according to the embodiment continue within a certain time. FIG. 11B is a diagram showing an example of a processing sequence in the case where words for controlling the devices and sensors issued by the user continuously after the recognition of the reserved word according to the embodiment continue within a certain time. FIG. 12A is a diagram showing an example of a processing sequence in the case where the words for controlling the devices and sensors issued by the user continuously after the recognition of the reserved word according to the embodiment continues for a predetermined time or longer. .. FIG. 12B is a diagram showing an example of a processing sequence in the case where words for controlling a device or a sensor issued by the user continuously after recognizing the reserved word according to the embodiment continue for a predetermined time or longer. .. FIG. 13 is a list specifically showing the contents of control information used when controlling a device or a sensor after recognizing a reserved word according to an embodiment. FIG. 14 is a list showing an example of operation contents changed according to a plurality of reserved words according to an embodiment. FIG. 15A is a diagram showing an example of a processing sequence in which, in registering a plurality of reserved words according to one embodiment, the operation content changed according to each reserved word is also registered. FIG. 15B is a diagram showing an example of a processing sequence in which, in registering a plurality of reserved words according to an embodiment, the operation content changed according to each reserved word is also registered. FIG. 16A is a diagram showing an example of a processing sequence for setting the operation content according to each reserved word in the recognition of the reserved word according to the embodiment. FIG. 16B is a diagram showing an example of a processing sequence for setting the operation content according to each reserved word in the recognition of the reserved word according to the embodiment. FIG. 17 is a list showing an example of operation contents set in accordance with a word continuing to the reserved word in the reserved word according to the embodiment. FIG. 18A is a diagram showing an example of a processing sequence in which, in recognition of a registered reserved word according to one embodiment, an operation content is set according to a word continuing to the reserved word. FIG. 18B is a diagram showing an example of a processing sequence for setting the operation content in accordance with a word continuing to the reserved word in recognizing the registered reserved word according to the embodiment. FIG. 18C is a diagram showing an example of a processing sequence in which, in recognition of a registered reserved word according to the embodiment, the operation content is set according to a word continuing to the reserved word. FIG. 18D is a diagram showing another example of the processing sequence for setting the operation content according to the word continuing to the reserved word in the recognition of the registered reserved word according to the embodiment. FIG. 18E is a diagram showing another example of the processing sequence for setting the operation content according to the word continuing to the reserved word in the recognition of the registered reserved word according to the embodiment. FIG. 19A is a diagram showing an example of a processing sequence in which, in the recognition of a reserved word according to the embodiment, the operation content is set according to a word continuing to the recognized reserved word. FIG. 19B is a diagram showing an example of a processing sequence in which, in the recognition of a reserved word according to the embodiment, the operation content is set according to a word continuing to the recognized reserved word. FIG. 20 is a list showing an example of types of voice recognition dictionaries used according to the reserved words in recognizing a plurality of reserved words according to an embodiment. FIG. 21A is a diagram showing an example of a processing sequence for changing the type of the voice recognition dictionary to be used according to the reserved words in recognizing a plurality of reserved words according to the embodiment. FIG. 21B is a diagram showing an example of a processing sequence for changing the type of the voice recognition dictionary to be used in accordance with the reserved words in recognizing a plurality of reserved words according to the embodiment. FIG. 22 is a list showing an example in which, in recognition of a plurality of reserved words according to an embodiment, the words that continue to the reserved words, the operation content set according to the reserved words, and the type of the voice recognition dictionary to be used are changed. FIG. 23 is a list showing an example of changing the type of the voice recognition dictionary according to the embodiment in accordance with the content other than the reserved word. FIG. 24 is a diagram showing a sequence of processing for registering the type of the voice recognition dictionary to be changed according to the content other than the reserved word in changing the type of the voice recognition dictionary according to the embodiment. FIG. 25 is a diagram showing a sequence of processing when changing the type of the voice recognition dictionary to be registered according to the content other than the reserved word in changing the type of the voice recognition dictionary according to the embodiment. FIG. 26 shows a reserved word (for relief) for displaying a reserved word and a reserved word corresponding to the reserved word when the user forgets the registered reserved word in the process according to the embodiment. It is a list which shows the example of a range. FIG. 27 is a functional block diagram of the host device according to the embodiment. In the processing according to the embodiment, FIG. 28 shows that when a scene for registering a reserved word, an additional word, or additional information occurs, or a scene for recognizing a reserved word, an additional word occurs, the host device 332 executes It is a figure showing an example of time lapse when recording a scene of registration, or a scene of recognition. FIG. 29 is a diagram showing an example of a state in which the data to be reproduced is displayed when reproducing each data of the recorded sound or the recorded scene according to the embodiment.

  FIG. 1 is a diagram showing an example of the overall configuration of a home automation system according to the present embodiment. The home automation system includes a cloud server 1 including a group of servers placed in a cloud, various sensors 310 and various equipment devices 320 connected to each other via a network 333 via a host device 332 having an HGW (HomeGate Way) function, and The home 3 includes various home appliances 340 and the Internet 2 that connects the cloud server 1 and the host device 332.

  The home 1 is a home, an office, or a small scale in which various sensors 310, various facility devices 320, and various home appliances 340 connected to each other via a home network 333 are arranged via a host device 332 having an HGW function. It is a business establishment and its scale does not matter.

  The host device 332 controls the devices and sensors connected to the network 333 based on preset information and information notified from the sensors connected to the network 333, and also relates to each device and sensor. It has the function of centrally managing information.

  Furthermore, the host device 332 is equipped with a microphone and can capture words spoken by the user 331. When the host device 332 recognizes a predetermined keyword (hereinafter referred to as a reserved word) from the words issued by the user 331, the host device 332 takes in the word issued by the user 331 subsequent to the reserved word, It has a function of returning a response according to the analysis result to the user 331 by analyzing the contents, or controlling a device or a sensor connected by the network 333 according to the analysis result.

  On the contrary, unless the host device 332 recognizes the reserved word from the words spoken by the user 331, the host machine 332 does not continuously take in the word spoken by the user 331. This prevents the host device 332 from picking up unnecessary surrounding sounds and operating them.

  The recognition of the reserved word is performed in the host device 332, the word issued by the user 331 following the reserved word is continuously taken in, and the content of the taken-in word is analyzed in the cloud server 1. Details of the function of the host device 332 will be described later.

  For convenience of description, the various equipments 320 and the various home appliances 340 mean the equipments that the equipments 320 are not so easy to move, and the various home appliances 340 are the equipments that are relatively easy to move. The illustrated names of equipment and home appliances do not limit the capabilities and functions of individual devices.

  Specific examples of the various sensors 310 include a security camera 311, a fire alarm 312, a motion sensor 313, and a temperature sensor 314. Further, as specific examples of the various equipment 320320, there are an intercom 325, a lighting 326, an air conditioner 327, and a water heater 328. Further, as specific examples of the various home appliances 340, there are a washing machine 341, a refrigerator 342, a microwave oven 343, a fan 344, a rice cooker 345, and a television 346.

  FIG. 2 shows another example of the various sensors 310 shown in FIG.

  FIG. 3 shows various examples of the host device 332 shown in FIG.

  The host device 332-1 is the host device 332 shown in FIG. 1 and is an example of a stationary type having a built-in HGW function. The host device 332-1 is connected to other devices and sensors arranged in the home 1 via the network 333, and is also connected to the cloud server 1 via the Internet 2. Since the host device 332-1 is a stationary type, it is an example in which a means for moving autonomously such as a motor is not mounted.

  The host device 332-2 is an example of a stationary type that does not have a built-in HGW function. Therefore, the host device 332-2 is connected to the HGW 330 via the network 333. The host device 332-2 is connected to other devices and sensors arranged in the home 1 via the network 333 via the HGW 330, and is also connected to the cloud server 1 via the Internet 2 via the HGW 330. Since the host device 332-2 is a stationary type, it is an example in which no means for autonomously moving such as a motor is mounted.

  The host device 332-3 is an example of a movable type having a built-in HGW function. The host device 332-3 is connected to other devices and sensors via the network 333, and is also connected to the cloud server 1 via the Internet 2. Since the host device 332-3 is a movable type, it is an example in which means for autonomously moving such as a motor is mounted.

  The host device 332-4 is an example of a movable type that does not have a built-in HGW function. Therefore, the host device 332-4 is connected to the HGW 330 via the network 333. The host device 332-4 is connected to other devices and sensors via the network 333 via the HGW 330, and is also connected to the cloud server 1 via the Internet 2 via the HGW 330. Since the host device 332-4 is a movable type, it is an example in which means for autonomously moving such as a motor is mounted.

  FIG. 4 shows functional blocks of the host device 332 shown in FIG. The host device 332 is connected to the system controller 402 that controls the entire internal processing, and the control management unit 401, the trigger setting unit 403, the trigger recognition unit 405, the input management unit 420, and the network 333 that control the respective functions by the system controller 402. Network I / F 427. The control management unit 401 internally manages a plurality of applications for controlling various operations of the host device 332, APP-Mg 401-1, initial settings of various functional blocks of the host device 332, various status settings, operation settings, etc. CONF-Mg 401-2 which manages the setting contents of.

  Further, the host device 332 serves as an interface (I / F) with the user 331, to the microphone 421 for capturing the words spoken by the user 331, the speaker 423 for outputting a response to the user 331 by voice, and the user 331. And a display unit 425 for notifying the state of the host device 332.

  The microphone 421 is connected to the input management unit 420. The input management unit 420 controls which of the trigger setting unit 403, the trigger recognition unit 405, and the voice processing unit 407 the voice data input from the microphone 421 is sent to, depending on the state managed internally. The display unit 425 notifies the user 331 of the state of the host device 332, and is, for example, an LED (Light Emitting Diode) or an LCD (Liquid Crystal Display).

  The memory 410 is divided into three areas: an operation mode storage area 410-1, a reserved word storage area 410-2, and a voice storage area 410-3. The contents of the information stored in each area will be described later.

  As described above, when the function of the host device 332 recognizes the reserved word from the words issued by the user 331, the word issued by the user 331 that continues to the reserved word is taken in, and the content of the taken word is read. By the analysis, it has a function of returning a response according to the analysis result to the user 331 and controlling the operation of devices and sensors connected through the network 333.

  In order to realize these functions, the host device 332 roughly performs four processes. The first process is the registration of reserved words. The second process is recognition of reserved words. The third process is registration of the control content of the device or sensor that controls the operation. The fourth process is control of devices and sensors whose control contents are registered.

First, the reserved word registration, which is the first process, will be described.
The host device 332 has a function of registering a reserved word in the host device 332. In order to register the reserved word, the host device 332 has a mode for registering the reserved word (hereinafter referred to as reserved word registration mode).

  5A and 5B show an example of the processing sequence of the host device 332 from the start of registration of a reserved word to the completion of registration in a state where the host device 332 is in the "reserved word registration mode" for registering a reserved word. Is shown.

  The host device 332 may change the mode by recognizing words spoken by the user 331 in a predetermined order for changing the mode. Alternatively, a menu screen may be displayed on the display unit 425, and the user 331 may operate the menu screen to change the mode. Alternatively, the mode may be changed by the user 331 operating a menu screen for changing the mode of the host device 332 displayed on a smartphone or tablet connected via the network I / F 427.

  When the user 331 issues a word to be registered as a reserved word, the host device 332 fetches the voice data input from the microphone 421 into the input management unit 420 (S501). The input management unit 420 has a function of determining the transfer destination of the input audio data according to the state managed internally. When the mode of the host device 332 is the setting mode, the input management unit 420 transfers the received voice data to the trigger setting unit 403 (S502). The trigger setting unit 403 saves the received voice data in the voice storage area 410-3 of the memory 410 (S503) and confirms whether the number of times of capturing the voice of the user 331 has reached the prescribed number (S504). ..

  When the trigger setting unit 403 determines that the number of times of capturing the voice of the user 331 has reached the specified number of times as a result of confirmation, the trigger setting unit 403 displays a message prompting the user 331 to issue the words to be registered. At the same time as performing (S507), an input continuation notification is sent to the input management unit 420 (S506). Upon receiving the input continuation notification, the input management unit 420 transitions the internal state to the state of waiting for the voice input from the microphone (S500).

  The display prompting the user 331 to input the word to be registered is a display in which the trigger setting unit 403 transmits a registration incomplete notification to the display device 425 (S505) and receives the registration incomplete notification. It is desirable that the device 425 perform a display method that the user 331 can recognize, such as causing a light emitting diode (LED) to blink red (S507). The user 331 may be prompted to input a word to be registered by using a voice method instead of the display method. In this case, the trigger setting unit 403 sends a registration incomplete notification to the speaker 423, and the speaker 423 that has received this registration incomplete notification announces to the user 331, for example, “Please input again”. But it's okay. Alternatively, the trigger setting unit 403 may use both the display method and the voice method to prompt the user 331 to input a word to be registered. Alternatively, when the host device 332 is movable, the trigger setting unit 403 may give an instruction to a moving unit (not shown) so that the host device 332 is repeatedly rotated and moved, for example, with a certain angular width.

  When the trigger setting unit 403 determines that the number of times of capturing the voice of the user 331 has reached the specified number of times as a result of confirmation, the trigger setting unit 403 stores it in the audio storage area 410-3 by then. Certain voice data is read (S508) and sent to the recognition data conversion unit 101-1 in the voice recognition cloud 101 in the cloud server 1 through the Internet 2 (S509).

  The recognition data conversion unit 101-1 converts the voice data sent from the trigger setting unit 403 into recognition data for recognizing as a reserved word (S510). When the conversion into the recognition data is completed, the recognition data conversion unit (101-1) sends the recognition data to the trigger setting unit 403 via the Internet 2 (S511). Upon receiving the recognition data, the trigger setting unit 403 stores the received data in the reserved word storage area 410-2 of the memory 410 (S512).

  The trigger setting unit 403 performs a display for notifying the user 331 that the reserved word registration is completed (S514). The trigger setting unit 403 sends a registration completion notification to the display device 425 (S514), and the display device 425 that has received the registration completion notification displays the notification that the registration of the reserved word is completed to the user 331. It is desirable to use a display method that can be recognized by the user 331, such as turning on the LED in green. Alternatively, the trigger setting unit 403 may use a voice method instead of the display method to notify the user 331 that the reserved word registration is completed. In this case, the trigger setting unit 403 may transmit a registration completion notification to the speaker 423, and the speaker 423 that has received the registration completion notification may announce to the user 331 that “registration is completed”, for example. .. Alternatively, the trigger setting unit 403 may use both the display method and the voice method to notify the user 331 that registration of the reserved word is completed. Alternatively, when the host device 332 is movable, the trigger setting unit 403 may give an instruction to a moving unit (not shown) so that the host device 332 repeatedly linearly moves with a certain movement width.

  As described above, the trigger setting unit 403 has a role of managing the flow of data when registering a reserved word.

  FIGS. 6A and 6B show another sequence example from the start of registration of a reserved word to the completion of registration. It may be insufficient to register the voice data captured by the host device 332 as a reserved word. An example of processing when the data thus captured is insufficient will be shown.

  The processing of S600 to S615 shown in FIGS. 6A and 6B is the same as the processing of S500 to S515 shown in FIGS. 5A and 5B, respectively. The difference between the processing in FIGS. 5A and 5B and the processing in FIGS. 6A and 6B is that the processing in S616 to S619 is added to the processing in FIG. 6B.

  If the trigger setting unit 403 determines that the number of times the words uttered by the user 331 have been captured has reached the specified number of times (S604), and determines that the number of times has reached the specified number of times, the trigger setting unit 403 will have the audio storage area by then. The voice data stored in 410-3 is read (S608) and sent to the recognition data conversion unit 101-1 in the voice recognition cloud 101 in the cloud server 1 via the Internet 2 (S609).

  When the trigger setting unit 403 determines that the number of times that the words uttered by the user 331 have been captured has not reached the specified number of times, the trigger setting unit 403 displays a message prompting the user 331 to utter the words to be registered (S607), and also the input management unit 403. An input continuation notification is sent to 420 (S606). Upon receiving the input continuation notification, the input management unit 420 transitions the internal state to the state of waiting for input of voice from the microphone (S600).

  The display prompting the user 331 to input a word to be registered is a display in which the trigger setting unit 403 transmits a registration incomplete notification to the display device 425 (S605) and receives the registration incomplete notification. It is desirable that the device 425 uses a display method that the user 331 can recognize, such as blinking the LED in red (S607). The user 331 may be prompted to input a word to be registered by using a voice method instead of the display method. In this case, the trigger setting unit 403 sends a registration incomplete notification to the speaker 423, and the speaker 423 that has received this registration incomplete notification announces to the user 331, for example, “Please input again”. But it's okay. Alternatively, the trigger setting unit 403 may use both the display method and the voice method to prompt the user 331 to input a word to be registered. Alternatively, when the host device 332 is movable, the trigger setting unit 403 may give an instruction to a moving unit (not shown) so that the host device 332 is repeatedly rotated and moved, for example, with a certain angular width.

  The recognition data conversion unit 101-1 determines, when converting all the voice data sent from the trigger setting unit 420 into the recognition data, whether the sent voice data can be converted into the recognition data. (S616). When it is determined that some of the sent voice data cannot be converted into recognition data, the recognition data conversion unit 101-1 transmits a voice data addition request to the trigger setting unit 403 via the Internet 2 (S617). To do. Upon receiving the voice data addition request, the trigger setting unit 403 sets the number of times the user 331 additionally inputs a word to be registered as a reserved word (S618), and notifies the input management unit 420 of an input continuation (S619). To notify.

  At the time when the trigger setting unit 403 sets the number of times of additional input to the user 331 (S618), for example, the LED of the display unit 425 remains lit in red. In accordance with this display, the user 331 emits the word to be registered as the reserved word for the number of times additionally set in S618.

  When the input management unit 420 receives the input continuation notification (S619), the input management unit 420 transitions the internal state to input waiting (S600), and waits for the input of the words issued by the user 331.

  The processes shown in FIGS. 5A and 5B and the processes shown in FIGS. 6A and 6B are performed after the number of times the input management unit 402 has captured the voice uttered by the user 331 reaches a prescribed number, and then the captured voice data is collected. This is an example of transmitting to the recognition data conversion unit 101-1 in the cloud server 1, but every time the input management unit 420 acquires the voice uttered by the user 331, the acquired voice data is converted to the recognition data conversion unit 101-1. 1 may be transmitted. 7A and 7B, each time the input management unit 420 captures a voice uttered by the user 331, the captured voice data is sequentially sent to the recognition data conversion unit 101-1 in the cloud server 1 for recognition. It is a sequence example at the time of converting into data.

  The processes of S700 to S702 shown in FIG. 7A are the same as the processes of S500 to S502 shown in FIG. 5A, respectively. Further, the processing of S703 and S704 shown in FIG. 7A is the same as the processing of S505 and S507 shown in FIG. 5A, respectively.

  When the user 331 issues a word to be registered as a reserved word, the host device 332 fetches the voice data input from the microphone 421 into the input management unit 420 (S701). Since the mode of the host device 332 is the reserved word registration mode, the input management unit 420 transfers the received voice data to the trigger setting unit 403 (S702). The trigger setting unit 403 sequentially transmits the received voice data to the recognition data conversion unit 101-1 in the cloud server 1 each time it is received (S706). When converting the voice data sent from the trigger setting unit 403 into the recognition data, the recognition data conversion unit 101-1 determines whether or not the sent voice data can be converted into the recognition data ( S707).

  When it is determined that the sent voice data cannot be converted into the recognition data, the recognition data conversion unit 101-1 transmits a voice data addition request to the trigger setting unit 403 via the Internet 2 (S708). Upon receiving the voice data addition request, the trigger setting unit 403 (S708) confirms whether the number of times of capturing the voice of the user 331 has reached the specified number (S714). When the trigger setting unit 403 determines that the number of times of capturing the voice of the user 331 has reached the specified number of times as a result of confirmation, the trigger setting unit 403 displays a message prompting the user 331 to issue the words to be registered. While continuing, by sending an input continuation notification to the input management unit 420 (S715), the input management unit 420 is transitioned to a state of waiting for input of voice from the microphone (S700). When the input management unit 420 receives the input continuation notification (S715), the input management unit 420 shifts the internal state to input waiting (S700) and waits for the input of the words issued by the user 331.

  When it is determined that the received voice data can be converted into the recognition data (S707), the recognition data conversion unit 101-1 converts the voice data into the recognition data (S709). The recognition data conversion unit 101-1 converts the voice data input from the microphone 421 by using all the recognition data including those already converted into the recognition data as a result of the conversion into the recognition data (S709). It is judged whether or not the accuracy of recognition as a reserved word is secured (S710).

  If it is determined that all the recognition data have sufficient accuracy to recognize the voice data input from the microphone 421 as a reserved word, the user 331 issues a word to be registered as a reserved word. In order to stop the recognition, the trigger setting unit 403 is notified of the recognition data (with the recognition data satisfaction notification) to which the information indicating that the recognition data is sufficient is added via the Internet 2 (S711). The trigger setting unit 403 that has received the recognition data (satisfaction data for recognition) is the recognition data received up to this point, and is sufficient recognition data for recognizing the voice data input from the microphone 421 as a reserved word. Even if the number of times that the voice of the user 331 has been captured has not reached the prescribed number, urging the user 331 to input a word to be registered is stopped (S712). The trigger setting unit 403 saves all the recognition data received up to this point in the reserved word storage area 410-2 (S716), and also stores them in the input management unit 420, the display unit 425, and the recognition data conversion unit 101-1. A registration completion notice is sent (S717) (S718) (S719). As a result, due to the accuracy of the converted recognition data, it is possible to stop the user 331 from issuing a word to be registered as a reserved word because the number of times that the voice of the user 331 has been captured does not reach the prescribed number. , It becomes possible to perform the process of registering a reserved word with more flexibility. The specified number of times can be changed by the user 331 as a setting value of the host device 332, and can be changed as one of additional information described later.

  If the recognition data conversion unit 101-1 determines that the recognition data created up to this point does not ensure sufficient accuracy to recognize the voice data input from the microphone 421 as a reserved word, , And sends only the converted recognition data to the trigger setting unit 403 (S713). Upon receiving the recognition data, the trigger setting unit 403 confirms whether the number of times that the voice of the user 331 has been captured has reached the specified number (S714). If the trigger setting unit 403 determines that the specified number of times has not been reached as a result of checking whether the number of times has reached the specified number of times, the trigger setting unit 403 continues the display prompting the user 331 to utter a word to be registered, By sending an input continuation notification (S715), the input management unit 420 is transitioned to a state of waiting for input of voice from the microphone (S700).

  The display prompting the user 331 to input the word to be registered is a display in which the trigger setting unit 403 transmits a registration incomplete notification to the display device 425 (S703) and receives the registration incomplete notification. It is preferable that the device 425 uses a display method that the user 331 can recognize, such as blinking the LED in red (S704). The user 331 may be prompted to input a word to be registered by using a voice method instead of the display method. In this case, the trigger setting unit 403 sends a registration incomplete notification to the speaker 423, and the speaker 423 that has received this registration incomplete notification announces to the user 331, for example, “Please input again”. But it's okay. Alternatively, the trigger setting unit 403 may use both the display method and the voice method to prompt the user 331 to input a word to be registered. Alternatively, when the host device 332 is movable, the trigger setting unit 403 may give an instruction to a moving unit (not shown) so that the host device 332 is repeatedly rotated and moved, for example, with a certain angular width.

  When the trigger setting unit 403 that has received the recognition data determines that the specified number of times has been reached as a result of checking whether the specified number of times has been reached (S714), a registration completion notification is input to the input management unit 420, the display unit 425, and the recognition. A registration completion notification is sent to the data conversion unit 101-1 (S717) (S718) (S719). Upon receiving the registration completion notification (S718), the recognition data conversion unit 101-1 clears the converted recognition data temporarily stored for performing the process of S710.

  Next, recognition of reserved words, which is the second process of the host device 332, will be described.

  When the host device 332 recognizes the reserved word from the words issued by the user 331, the host device 332 analyzes the content of the word issued by the user 331 continuously, and controls the device and the sensor based on the analysis result. It has a function. In order to recognize this reserved word and control the device or sensor after recognizing the reserved word, the host device 332 has a mode for recognizing the reserved word and controlling the device or sensor (hereinafter referred to as operation mode). ing.

  FIG. 8A and FIG. 8B show an example of a processing sequence of the host device 332 until the word issued by the user 331 is recognized as one of the registered reserved words in the operation mode.

  When the user 331 utters a word, the host device 332 takes in the voice data input from the microphone 421 to the input management unit 420 (S801). When the mode of the host device 332 is the operation mode, the input management unit 420 transfers the received voice data to the trigger recognition unit 405 (S802). When the trigger recognition unit 405 receives the voice data transferred from the input management unit 420, the trigger recognition unit 405 determines whether or not the transferred voice data is a reserved word, and reserves a reserved word storage area 410-2 in the memory 410. It is compared with the recognition data read from (S803) (S804).

  When the trigger recognition unit 405 determines that the input voice data cannot be recognized as the reserved word (S805), the trigger recognition unit 405 performs a display prompting the user 331 to issue the reserved word (S808), and inputs it to the input management unit 420. A continuation notice is sent (S807). The display prompting the user 331 to issue the reserved word is performed by the trigger recognizing unit 405 transmitting a recognition incompletion notification to the display unit 425 (S806), and the display unit 425 receiving the recognition incompletion notification is, for example, It is desirable to use a display method that allows the user 3313 to recognize, such as blinking the LED in red (S808). Also, the trigger setting unit 403 may prompt the user 331 to input a voice by using a voice method instead of the display method. In this case, the trigger recognition unit 405 also transmits a recognition incompletion notice to the speaker 423, and the speaker 423 that has received this recognition incompletion notice announces to the user 331, for example, "I did not hear". Good. Alternatively, the trigger recognition unit 405 may use both a display method and a voice method to prompt the user 331 to input a voice. Alternatively, when the host device 332 is movable, the trigger setting unit 403 may give an instruction to a moving unit (not shown) so that the host device 332 is repeatedly rotated and moved, for example, with a certain angular width.

  When the input voice data can be recognized as a reserved word (S805), the trigger recognition unit 405 displays that the voice uttered by the user 331 is recognized as a reserved word (S810). The display indicating that the voice uttered by the user 331 has been recognized as a reserved word is displayed on the display device that the trigger recognition unit 403 transmits a recognition completion notification to the display device 425 (S809) and receives the recognition completion notification. It is desirable to use a display method that the user 331 can recognize, such as 425 turning on the LED in green (S810). Further, the trigger recognition unit 405 may notify that the voice uttered by the user 331 is recognized as a reserved word by using a voice method instead of the display method. In this case, the trigger recognition unit 405 also transmits a recognition completion notification to the speaker 423, and the speaker 423 that has received the recognition completion notification announces to the user 331, for example, "Yes" or "I heard". Good. Alternatively, the trigger recognition unit 405 may use both the display method and the voice method to indicate that the voice uttered by the user 331 is recognized as a reserved word. Alternatively, when the host device 332 is movable, the trigger setting unit 403 may give an instruction to a moving unit (not shown) so that the host device 332 repeatedly linearly moves with a certain movement width.

  9A and 9B are another example of the processing sequence of the host device 332 until the word issued by the user 331 is recognized as one of the registered reserved words in the operation mode.

  The difference between the sequence examples of FIGS. 9A and 9B and the sequence examples of FIGS. 8A and 8B is that the recognition probability is taken into consideration in the process of recognizing a reserved word. The recognition probability means a level at which the recognition data is compared with characteristic points such as frequency components and strengths of the voice data transferred from the input management unit 420, and the two match. The processes of S900 to S912 shown in FIGS. 9A and 9B are the same as the processes of S800 to S812, respectively. The difference between the processes of FIGS. 9A and 9B and the processes of FIGS. 8A and 8B is that of S913 to S916. The point is that processing is added.

  Upon receiving the voice data transferred from the input management unit 420, the trigger recognition unit 405 reads the recognition data from the reserved word storage area 410-2 of the memory 410 (S903), and is transferred from the input management unit 420. It is compared with the audio data (S904).

  When the trigger recognition unit 405 determines that the input voice data has been recognized as a reserved word (S905), the trigger recognition unit 405 proceeds to recognition probability determination processing (S913).

  Here, the voice recognition processing performed by the trigger recognition unit 405 has characteristics such as frequency components and strength of the recognition data read from the reserved word storage area 410-2 of the memory 410 and the voice data transferred from the input management unit 420. It is determined that the voice data transferred from the input management unit 420 is the recognition data when the two points match with each other at a certain level or more.

  When the host device 332 compares the recognition data with the characteristic points such as the frequency component and the strength of the voice data transferred from the input management unit 420, the host device 332 sets a threshold value for determining the level at which the two match. It is also possible to provide a plurality. By doing so, when the host device 332 recognizes the reserved word from the words issued by the user, the host device 332 does not perform the two determinations that the reserved word can be recognized or the reserved word cannot be recognized. For example, it is possible to add a judgment that the reserved word is not recognized, the reserved word cannot be recognized, or the reserved word cannot be recognized. By thus providing a plurality of thresholds of recognition probabilities, when the user 331 does not remember the reserved word correctly, for example, the user 331 repeatedly utters a word close to the reserved word to capture the word uttered by the user 331. However, the host device 332 responds according to the determination result that "the reserved word cannot be recognized", and the user 331 who sees the response content can approach the correct reserved word.

  The examples of FIGS. 9A and 9B are examples in which two threshold values of the recognition probability are provided. Assuming that the threshold for recognizing a reserved word is threshold 1 and the threshold for not recognizing a reserved word is threshold 0, the result of the comparison in S904 is that if the recognition probability is 1 or more, the reserved word can be recognized. Becomes If the recognition probability is greater than or equal to the threshold value 0 and less than the threshold value 1, it is determined that the reserved word cannot be recognized. Further, when the recognition probability is less than the threshold value 0, it is determined that the reserved word cannot be recognized. Therefore, the process of S905 is a process of comparing the recognition probability with the threshold value 0. The process of S913 is a process of comparing the recognition probability with the threshold value 1 in magnitude.

  When the host device 332 determines that the recognition probability is greater than or equal to the threshold value 0 and less than the threshold value 1 (S913), the host device 332 displays a message prompting the user 331 to issue a reserved word (S915) and continues input to the input management unit 420. A notice is sent (S916). As for the display prompting the user 331 to issue the reserved word, the trigger recognition unit 405 sends a notification of insufficient recognition to the display unit 425 (S914), and the display unit 425 that has received the insufficient notification of recognition recognizes, for example, It is desirable to perform the display method that the user 331 can recognize, such as blinking the LED in green (S915).

  As described above, when the recognition probability is low, the display prompting the user 331 to issue the reserved word is changed to the display when the recognition fails (S908) or the display when the recognition succeeds (S910). The user 331 can recognize that the word that he / she has spoken is close to the reserved word but does not correctly emit the reserved word.

  Also, the trigger setting unit 403 may prompt the user 331 to input a voice by using a voice method instead of the display method. In this case, the trigger recognition unit 405 transmits a recognition insufficient notification to the speaker 423 (S914), and the speaker 423 that has received the recognition insufficient notification notifies the user 331, for example, "Did you call something?" An announcement method is also acceptable. Alternatively, the trigger recognition unit 405 may use both a display method and a voice method to prompt the user 331 to input a voice. Alternatively, when the host device 332 is movable, the trigger setting unit 403 may give an instruction to a moving unit (not shown) so that the host device 332 is repeatedly rotated and moved, for example, with a certain angular width.

  Next, regarding the third process of the host device 332, registration of the control content of the device or sensor for controlling the operation, and the fourth process of controlling the device or sensor with the control content registered explain.

  First, an overview of control of devices and sensors using the host device 332 will be described.

  When the host device 332 recognizes the reserved word from the words spoken by the user 331, the host device 332 continuously captures the word spoken by the user after the reserved word is recognized, and analyzes the content of the captured word. It has the function of controlling devices and sensors.

  10A and 10B show a case where the host device controls the device and the sensor based on the content of the voice data including the control content of the device and the sensor fetched from the microphone 421 after the recognition of the reserved word is completed. The example of the processing sequence of is shown. The internal state of the input management unit 420 has transitioned to recognized (S1000) because the recognition of the reserved word is completed.

  When the user 331 utters a word including content for controlling the device or sensor, the host device 332 takes in the voice data (control content) to the input management unit 420 through the microphone 421 (S1001) (S1002). Since the internal state has been recognized, the input management unit 420 transfers the input voice data (control content) to the voice processing unit 407 (S1002). The voice processing unit 407 sends the transferred voice data (control contents) to the voice text conversion unit 101-2 in the voice recognition cloud 101 in the cloud server 1 via the Internet 2.

  The voice / text conversion unit 101-2 performs a process of converting voice data sent via the Internet 2 into text data (S1004). By this processing, the voice originally uttered by the user 331 and taken in through the microphone 421 is converted into text data.

  When the conversion into text data is completed, the voice / text conversion unit 101-2 stores the converted text data therein and sends a conversion completion notice to the voice processing unit 407 (S1005).

  Upon receiving the conversion completion notification, the voice processing unit 407 transmits a text analysis request to the voice / text conversion unit 101-2 (S1006). Upon receiving the text analysis request, the voice / text conversion unit 101-2 sends the text analysis request to the text analysis unit 102-1 together with the data converted into the text stored inside (S1007). Upon receiving the text analysis request (S1007), the text analysis unit 102-1 analyzes the content of the accompanying text data (S1008). When the analysis of the content of the sent text data is completed, the text analysis unit 102-1 sends the analysis result to the response / action generation unit 102-2 as a text analysis result notification (S1009). Upon receiving the text analysis result (S1009), the response / action generation unit 102-2 generates a target device and a command for controlling the device based on the content (S1010), and returns the generated command as a response / action. It is sent to the voice processing unit 407 as a generation result notification (S1011).

  Upon receiving the response / action generation result notification (S1011), the voice processing unit 407 identifies the device or sensor to be controlled and its control content from the content of the response / action generation result notification (S1012). The voice processing unit 407 converts the specified control target device or sensor and its control content into a format that can be recognized by the control target device or sensor, and notifies the target device or target sensor of an action through the network 333 at a necessary timing. (S1013).

  Upon receiving the action notification (S1013), the device or sensor to be controlled, which is the notification destination of the action notification, operates based on the control content included in the notification (S1014).

  When the user 331 continuously emits a voice, the host device 332 determines that the continuous voice is a continuous voice and does not request the user 331 to issue a reserved word in the middle of the continuous voice. Can be captured. Conversely, when the user 331 utters a voice after a certain period of time, the host device 332 requests the input of the reserved word again. Each case will be described with reference to FIGS. 11A and 11B and FIGS. 12A and 12B.

  11A and 11B are examples of a processing sequence when the user 331 continuously speaks a word within a time T0 after the recognition of the reserved word is completed. When the host device 332 fetches the audio data (control content) input from the microphone 421 into the input management unit 420 (S1101), the input management unit 420 activates the input interval confirmation timer T. When the next voice data (control content) generated by the user 331 through the microphone 421 is captured in the input management unit 420 at the time T1 before the time (= T0) when the input interval confirmation timer T expires (S1121), the input management The unit 420 transfers the captured voice data (control contents) to the voice processing unit 407 (S1122). At the same time, the input interval confirmation timer T being activated is activated again. The voice processing unit 407 sends the transferred voice data (control contents) to the voice text conversion unit 101-2 in the voice recognition cloud 101 in the cloud server 1 via the Internet 2 (S1123). After that, similarly to the processing from S1104 to S1110, the processing of the voice data (S1123) sent in the voice recognition cloud 101 is continued.

  The input interval confirmation timer T is activated at the timing when the input management unit 420 captures the voice data input from the microphone 421. However, the input interval confirmation timer T is not limited to this, and the input management unit 420 is sent from the microphone 421, for example. It may be activated at the timing when the data is transferred to the trigger setting unit 403 or the voice processing unit 407. The input management unit 420 may be activated at the timing when the internal state of the input management unit 420 transits to "recognized" (S1100).

  12A and 12B are examples in which the user 331 does not continuously make a sound within the time T0. When the host device 332 fetches the voice data (control content) input from the microphone 421 into the input management unit 420 (S1201), the input management unit 420 activates the input interval confirmation timer T. When the input interval confirmation timer T expires (= T0), the input management unit 420 transitions the internal state to wait for input (S1220).

  When the host device 332 acquires the next audio data input from the microphone 421 after the time (= T0) at which the input interval confirmation timer T expires (S1224), the device 332 is based on the acquired audio data. The display for urging the user 331 to issue the reserved word is displayed without executing the process for controlling the or sensor.

  When the input interval confirmation timer T expires, the input management unit shifts the internal state to wait for input (S1220) and also notifies the voice processing unit 407 of a timeout notification (S1221). The voice processing unit 407 that has received the time-out notification transmits a recognition incomplete notification to the display unit 425 (S1222), and the display unit 425 that has received the recognition incomplete notification instructs the user 331 to issue a reserved word. A prompting message, for example, the LED is blinked in red (S1223).

  When the next voice data input from the microphone 421 is fetched after the input interval confirmation timer T expires (S1224), the input management unit 420 causes the internal state to transition to recognition (S1225) and the fetched voice. The data is transferred to the trigger recognition unit 405 (S1226). After that, the host device 332 performs the processing from S803 to S812 in FIGS. 8A and 8B or the processing from S903 to S916 in FIGS. 9A and 9B, and recognizes the reserved word again.

  Next, registration of control contents for controlling the devices and sensors using the host device 332 and control of the devices and sensors performed based on the registered control contents will be described.

  13 shows control information used when the host device 332 recognizes a reserved word and then controls various sensors 310, various equipment devices 320, and various home electric appliances 340 as shown in the sequence diagrams of FIGS. 10A and 10B. It shows a specific example of the content.

  Item 1 is information included in the response / action generation result notification transmitted from the response / action generation unit 102-2 for controlling the various sensors 310, the various facility devices 320, and the various home appliances 340 (hereinafter, the response / action). (Called information). This response / action generation information is made up of a “target” such as a device or a sensor controlled by the device 332, and a “command” indicating the content of controlling the control target. Upon receiving the response / action generation result notification, the host device 332 extracts the action information included therein, and controls the target device based on the content of the action information.

  Examples of the "instruction" are a "start instruction" for activating (operating) the device to be controlled, a "stop instruction" for terminating (stopping) the device, and an "operation change instruction" for changing the contents (mode) during operation. , ”“ Setting change command ”for changing the contents (mode) preset in the target device.

  In order for the response / action generation unit 102-2 to generate the response / action information included in the response / action generation result notification, the user 331 preliminarily controls the device to be controlled, its control content, and the host device to control the device. It is necessary to register a combination of words issued to 332 in the response / action generation unit 102-2 as an initial setting of the host device 332. Registration of response / action information in the initial setting of the host device 332 will be described below using the example of FIG. 13.

  Item 2 is a “target” that is a device controlled through the host device 332. The “target” is an identification name of a device or a sensor included in the various sensors 310, the various facility devices 320, and the various home appliances 340, and the air conditioner 1 is described as a specific example.

  Item 3 is a "command" that is the control content of the device shown in "item 2". This “command” describes the command of the air conditioner 1 given in item 2 as a specific example, and it is a “start command” to move the air conditioner, a “stop command” to stop the air conditioner, and a “change operation” to change the operation content of the air conditioner. The “command” and the “setting change command” for changing the setting contents of the air conditioner are described as examples.

  The product specifications of the devices and sensors of item 2 and item 3 are stored in advance in the product specification cloud server in which the information of the product specifications not described is stored. The user 331 obtains, from the product specification cloud server, the product specification information of item 2 and item 3 of the target device or target sensor to be controlled through the host device 332.

  Next, when the user 331 executes the control contents of item 2 and item 3 through the host device 332, the user 331 determines item 4 = “phrase” which is a word issued to the host device 332. This "phrase" is preferably the content corresponding to the command of the air conditioner 1 listed in item 3. For example, "start command" to move the air conditioner is "turn on the air conditioner" and "stop command to stop the air conditioner". "For air conditioner", "Air conditioner operation" is changed to "Dry" "Operation change command" is "Dry", air conditioner setting operation start time As for the “setting change command” that changes “to 10 o'clock at night”, “turn on air conditioner at 10 o'clock” is described as an example.

  The user 331 creates the combination of (target, command, phrase) determined as described above as the initial setting of the host device 332. The user 331 performs the same creation for all devices to be controlled through the host device 332, and finally generates a response / action information list in which (targets, commands, phrases) relating to all devices to be controlled are combined. To do. The created response / action information list is registered in the response / action generation unit 102-2 through the host device 332.

  When the response / action information list is registered in the response / action generation unit 102-2, as shown in FIGS. 10A and 10B, the host device 332 continues to output the words issued by the user 331 after the recognition of the reserved words is completed. By taking in and analyzing, it is possible to control devices and sensors.

  For example, when the word uttered by the user 331 = turn on the air conditioner, the voice / text conversion unit 101-2 converts the input voice data into a text "Eakon Tsukete", and the text analysis unit 102-1 , Analyze that the text data "Eakon" should have "Air conditioner". Based on this analysis result, the response / action generation unit 102-2 refers to the already-registered response / action information list, and responds / action information corresponding to the analysis result of the "phrase" "turn on the air conditioner". To search. As a result, the response / action information of (target = air conditioner 1, command = start operation) is extracted, and the response / action information of (target = air conditioner 1, command = start operation) is set in the response / action generation result notification. The audio processing unit 407 is notified.

  The voice processing unit 407 refers to the response / action information set in the received response / action generation result notification to identify the corresponding device or sensor among the various sensors 310, the various facility devices 320, and the various home appliances 340. Control.

  Next, in the case of controlling a device or a sensor using the host device 332, a description will be given of a case where the control content for controlling the device or the sensor under various conditions and the operation content of the host device 332 are changed.

  In FIG. 14, when a plurality of reserved words are registered in the host device 332, the host device 332 recognizes that the word issued by the user 331 is one of the reserved words, and performs the operation according to the recognized reserved word. 3 is a list of example contents.

  The host device 332 is capable of registering a plurality of reserved words, and when each of the plurality of reserved words is recognized, the operation content corresponding to the recognized reserved word (hereinafter referred to as additional information 1) Can be set.

  As shown in FIG. 14, it is assumed that the host device 332 has registered three reserved words, for example, “Iroha”, “Ore-sama”, and “sonya”. When the host device 332 recognizes the word uttered by the user 331 as the reserved word “Iroha”, the host device 332 does not change the operation content that has already been set, but the word uttered by the user 331 is the reserved word “I am like”. If it is recognized, the operation is changed so that, after recognizing the words spoken by the user 331, the user will be announced through the speaker 423 that "the master is pleased". Further, when the word spoken by the user 331 is recognized as the reserved word “sonya”, the host device 332 determines that the user 331 is a senior user, and in the case of a senior, the host device 332 tends to talk slowly. The setting is changed so that the expiration time T0 of the input interval confirmation timer shown in FIGS. 11A and 11B is set longer than the normal set time.

  The example of FIG. 14 shows an example in which the host device 332 changes the operation content of the host device itself, but the invention is not limited to this, and controls the operation of devices and sensors connected to the host device 332 by the network 333. May be.

  The host device 332 needs to register the additional information 1 for each reserved word in the host device 332 in advance in order to change the operation of the host device 332 according to a plurality of reserved words.

  The host device 332 has a mode (hereinafter referred to as a reserved word registration (additional information 1) mode) in which the additional information 1 corresponding to the reserved word to be registered is also registered when the reserved word is registered in the host device 332. ing.

  FIG. 15A and FIG. 15B show a reserved word in a state in which the host device 332 is in the “reserved word registration (additional information 1) mode” in order to register the reserved word and the additional information 1 corresponding thereto. An example of a processing sequence of the host device 332 from the start of registration to the completion of registration of the additional information 1 is shown. The processes of S1500 to S1514 shown in FIGS. 15A and 15B are the same as the processes of S500 to S514 shown in FIGS. 5A and 5B, respectively. 15A and 15B are different from those of FIGS. 5A and 5B in that S1515 is different from S515 and that S1516 to S1523 are added.

  The trigger setting unit 403 displays to notify the user 331 that the reserved word registration is completed (S1514). The display notifying the user that registration of the reserved word is completed (S1515) is performed by the trigger setting unit 403 by transmitting a registration completion notification to the display device 425 (S1514), and receiving the registration completion notification. It is desirable to use a display method that can be recognized by the user 331, such as the LED 425 blinking the LED in green. Thereby, the trigger setting unit 403 can prompt the user 331 to register the additional information 1.

  The user 331 who recognizes that the LED is blinking in green (S1515) can set the additional information 1 corresponding to the reserved word registered in S1511.

  The additional information 1 may be set by registering the voice generated by the user 331 by the host device 332 through the microphone 421 and analyzing the captured voice data. Alternatively, a menu for setting the additional information 1 may be displayed on the display device 425 so that the user 331 can register it by operating according to the menu. Alternatively, using an external device connected via the network I / F 427 shown in FIG. 4, for example, a smartphone or tablet, a menu for setting additional information 1 corresponding to the reserved word on the display screen of the smartphone or tablet is displayed. It may be displayed so that the user 331 can register by operating according to the displayed menu screen. FIG. 15A and FIG. 15B are examples of a processing sequence when the menu for setting the additional information 1 displayed on the display unit 425 is displayed and the user 331 operates the menu to register the additional information 1.

  When the LED blinks in green to prompt the user 331 to input the additional information 1 (S1515), a menu for registering the additional information 1 is displayed on the display unit 425. The user 331 creates the additional information 1 by operating according to the displayed menu screen. The additional information 1 that has been created is taken into the input management unit 420 (S1517). The input management unit 420 transfers the captured additional information 1 to the trigger setting unit 403. The trigger setting unit 403 stores the transferred additional information 1 in the reserved word storage area 410-2 of the memory 410 (S1519).

  Note that the trigger setting unit 403 stores the additional information 1 in association with the reserved word registered in S1513 when storing the additional information 1 in the reserved word storage area 410-2 of the memory 410.

  The voice processing unit 407 also performs a display (S1522) informing the user 331 that the registration of the additional information 1 has been completed. As for the display notifying the user 331 that the registration of the additional information 1 is completed (S1522), the voice processing unit 407 transmits a registration completion notification to the display device 425 (S1520), and receives the registration completion notification. It is desirable to use a display method that the user 331 can recognize, for example, the display device 425 turns on the LED in green.

  FIGS. 16A and 16B show recognition of a reserved word from words issued by the user 331 when the additional information 1 is stored in the reserved word storage area 410-2 of the memory 410 by the processing shown in FIGS. 15A and 15B. Then, the additional information 1 of the recognized reserved word is read from the reserved word storage area 410-2, and the operation is set in the host device 332.

  The processes of S1600 to S1612 shown in FIGS. 16A and 16B are the same as the processes of S800 to S812 shown in FIGS. 8A and 8B, respectively. The difference between the processes of FIGS. 16A and 16B and the processes of FIGS. 8A and 8B is that the processes of S1613 and S1614 are added.

  When the word recognized by the user 331 is recognized as a reserved word (S1605), the trigger recognition unit 405 reads the additional information 1 corresponding to the corresponding reserved word from the reserved word storage area 410-2 of the memory 410. The trigger recognition unit 405 that has read the additional information 1 sets the operation having the content of the read additional information 1 (S1613) in the host device 332 (S1614). When the content of the example shown in FIG. 14 is stored in the reserved word storage area 410-2, when “son” is recognized as the reserved word in S1605, the trigger recognition unit 405 determines the input interval in S1614. The expiration time T0 of the confirmation timer T is set so as to make the normal value longer.

  In FIG. 17A, when a word uttered by the user 331 is recognized as a reserved word registered in the host device 332, the host device responsive to the word continued by the reserved word recognized by the user 331. 332 is a list of examples of operation contents for performing a specific operation.

  When the host device 332 recognizes the word issued by the user 331 as a registered reserved word, the content of the word issued by the user 331 (hereinafter referred to as an additional word) following the recognized reserved word. The operation content (hereinafter referred to as additional information 2) can be set according to the above.

  For example, as shown in FIG. 17A, assume that "Iroha" is registered as a reserved word. In this case, when the host device 332 recognizes the reserved word “Iroha” and does not recognize the word issued by the user 331 following the reserved word “Iroha”, the host device 332 does not change the operation content that has already been set. When the host device 332 recognizes "chan" as the word spoken by the user 331 following the reserved word "iroha", the host device 332 determines that the user 331 is in a good mood, and when responding through the speaker 423, the response is made. Change the operation so that the tone is raised. When the host device 332 recognizes “ya” as the word spoken by the user 331 following the reserved word “iroha”, the host device 332 estimates that the user 331 is a senior user, and the user 331 tends to speak slowly. Therefore, the expiration time T0 of the input interval confirmation timer shown in FIGS. 11A and 11B is changed to be longer than the normal set time. Further, when the host device 332 recognizes “Ooi” as the word spoken by the user 331 following the reserved word “Iroha”, the host device 332 determines that the user 331 is angry and immediately says “Sorry” through the speaker 423. Make an announcement.

  In the example of FIG. 17A, the host device 332 sets the additional information 2 by setting a plurality of additional words for one reserved word and setting additional information 2 for each combination of a plurality of additional words for the reserved word. Although the example in which the operation content is changed based on the content of 1 is shown, the additional information 2 can be set for each combination of a plurality of reserved words and a plurality of additional words. As shown in FIG. 17B, for example, it is assumed that the host device 332 has registered three reserved words “Iroha”, “Ookini”, and “Ahsind”. In this case, an additional word may be defined for each reserved word, and the additional information 2 may be set for each combination of the reserved word and the additional word.

  In addition, some users may want to perform a specific operation only by issuing a reserved word. For example, when there is a habit of a certain individual, that habit is registered as a reserved word in the host device 332, and an operation corresponding to this reserved word is also registered in the host device 332, so that a device that meets the characteristics of the individual is registered. It is possible to easily control the operation of the sensor and the sensor. In the example of the reserved word “earth” in FIG. 17B, when the host device 332 recognizes the reserved word “earth”, the host device 332 recognizes the reserved word from the words issued by the user 331. Only by doing, it is possible to announce the information of beer stored in the refrigerator connected to the network 333 through the speaker 423.

  In order to change the operation according to the content of the additional word to the reserved word, the host device 332 combines the additional word corresponding to the reserved word and the additional information 2 which is the operation content for the combination of the reserved word and the additional word. It is necessary to register in the host device 332 in advance. Therefore, the host device 332 has a mode of additionally registering a corresponding additional word or additional information for the registered reserved word. The mode for adding the additional information 1 to the reserved word already registered in the host device 332 is called the additional information 1 additional registration mode, and the mode for adding the additional word and the additional information 2 is called the additional information 2 additional registration mode. And

  Similarly to the setting of the additional information 1, the additional information 2 may be registered by the host device 332 capturing the voice uttered by the user 331 through the microphone 421 and analyzing the captured voice data. Alternatively, a menu for setting the additional information 2 may be displayed on the display device 425 so that the user 331 can perform registration by operating according to the displayed menu. Alternatively, by using an external device connected via the network I / F 427 shown in FIG. 4, for example, a smartphone or tablet, additional information 2 corresponding to the reserved word and the additional word is set on the display screen of the smartphone or tablet. It is also possible to display a menu to be displayed and allow the user 331 to perform registration by operating in accordance with the displayed menu screen.

  FIGS. 18A, 18B, and 18C show the case where an additional word is registered for the registered reserved words shown in FIGS. 17A and 17B and the operation content (additional information 2) is registered for the additional word. Is an example of a processing sequence of.

  In order to additionally register the additional word for the registered reserved word, the user 331 changes the host device 332 to the “additional information 2 additional registration mode”. When the host device is changed to the “additional information 2 additional registration mode”, the user 331 issues a reserved word registered in the host device 332 and an additional word to be registered for the reserved word. The host device 332 first recognizes a reserved word from words spoken by the user 331 (S1805).

  The host device 332 takes in the words spoken by the user 331 into the input management unit 420 through the microphone 421 (S1801). When the input management unit 420 takes in the voice data, the input management unit 420 changes the internal state internally managed to being recognized (reserved word) (S1802), and transfers the input voice data to the trigger recognition unit 405 (S1803).

  When the trigger recognition unit 405 receives the voice data transferred from the input management unit 420, the trigger recognition unit 405 reads the recognition data from the reserved word storage area 410-2 of the memory 410 (S1804), and is transferred from the input management unit 420. It is compared with the audio data (S1805). When the input voice data can be recognized as a reserved word, the trigger recognition unit 405 notifies the input management unit 420 of a recognition completion notification (S1806). Upon receiving the recognition completion notification, the input management unit 420 causes the internal state internally managed to change from recognizing (reserved word) to waiting for input (additional word) (S1807).

  The host device 332 fetches the word issued by the user 331 following the reserved word into the input management unit 420 through the microphone 421 (S1808). The input management unit 420 transfers the input voice data to the trigger setting unit 403 because the internal state managed internally is waiting for input (additional word) (S1807) (S1809). After that, the trigger setting unit 403 stores the received voice data in the voice storage area 410-3 of the memory 410 (S1810), and performs the specified number of additional words in the same manner as the reserved word registration described in FIGS. 5A and 5B. Is taken in (S1811).

  If the trigger setting unit 403 determines that the number of times has reached the specified number of times as a result of checking whether the number of times has reached the specified number of times, the trigger setting unit 403 displays a message prompting the user 331 to input the voice of the additional word to be registered (S1812), and manages the input An input continuation notification is transmitted to the section 420 (S1814). As for the display prompting the user 331 to input a voice to be registered as an additional word (S1813), the trigger setting unit 403 transmits a registration incomplete notification to the display device 425 (S1812), and the registration incomplete notification is sent. It is desirable that the display device 425 that has received the message be displayed by a method that the user 331 can recognize, for example, blinking the LED in red. A voice method may be used instead of the display method to prompt the user 331 to input a voice to be registered. In this case, the trigger setting unit 403 sends a registration incomplete notification to the speaker 423, and the speaker 423 that has received this registration incomplete notification announces to the user 331, for example, “Please input again”. But it's okay. Alternatively, the trigger setting unit 403 may use both the display method and the voice method to prompt the user 331 to input the voice to be registered.

  When the trigger setting unit 403 determines that the number of times has reached the specified number as a result of checking whether the number of times has reached the specified number of times, the trigger setting unit 403 reads out the audio data stored in the audio storage area 410-3 up to then (S1815), and the Internet. 2 to the recognition data conversion unit 101-1 in the voice recognition cloud 101 in the cloud server 1 (S1816).

  The recognition data conversion unit 101-1 converts the voice data sent from the trigger setting unit 403 into recognition data for recognizing an additional word (S1817). When the conversion to the recognition data is completed, the recognition data conversion unit 101-1 sends the recognition data to the trigger setting unit 403 via the Internet 2 (S1818). The trigger setting unit 403 that has received the recognition data for recognizing the additional word (hereinafter referred to as the recognition data (additional word)) stores the received data in the reserved word storage area 410-2 of the memory 410 (S1819). ). When saving the recognition data (additional word), the trigger setting unit 403 saves it in association with the reserved word recognized in S1806. This makes it possible to store the recognition data (additional word) in association with the reserved word recognized in S1806.

  Further, the trigger setting unit 403 performs a display (S1822) informing the user 331 that the additional word registration is completed. The display notifying the user 331 that the reserved word registration is completed (S1822) is a display in which the trigger setting unit 403 transmits a registration completion notification to the display device 425 (S1821) and receives the registration completion notification. It is desirable that the device 425 perform a display method that the user 331 can recognize, such as making the LED blink green (S1822). Alternatively, the trigger setting unit 403 may use a voice method instead of the display method to notify the user 331 that the reserved word registration is completed. In this case, the trigger setting unit 403 sends a registration completion notification to the speaker 423 (S1821), and the speaker 423 that has received the registration completion notification announces to the user 331 that "registration is completed", for example. But it's okay. Alternatively, the trigger setting unit 403 may use both the display method and the voice method to notify the user 331 that the reserved word registration is completed. As a result, the user 331 can know the timing at which the content of the additional information 2, which is the operation content corresponding to the additional word, is uttered.

  When the LED blinks green to prompt the user 331 to input the additional information 2 (S1822), a menu for registering the additional information 2 is displayed on the display unit 425. The user 331 creates the additional information 2 by operating according to the displayed menu screen. After the creation is completed, the additional information 2 is fetched by the input management unit 420 (S1824). The input management unit 420 transfers the captured additional information 2 to the trigger setting unit 403 (S1825). The trigger setting unit 403 stores the transferred additional information 2 in the reserved word storage area 410-2 of the memory 410 (S1826).

  The trigger setting unit 403 stores the additional information 2 in association with the reserved word recognized in S1806 when storing the additional information 2 in the reserved word storage area 410-2 of the memory 410. This makes it possible to save the operation content (additional information 2) associated with the reserved word recognized in S1806 and with the additional word stored in S1819.

  It is also possible to add only the additional information to the registered reserved word later.

  18D and 18E are an example of a processing sequence in the case of adding only additional information to a registered reserved word, which is different from FIGS. 18A, 18B and 18C.

  The processes of S1850 to S1856 shown in FIG. 18D are the same as the processes of S1800 to S1806 shown in FIG. 18A, respectively. Further, the processing of S1871 to S1880 shown in FIGS. 18D and 18E is the same as the processing of S1821 to S1830 shown in FIG. 18C, respectively. The difference between the sequence example of FIGS. 18A, 18B and 18C and the sequence example of FIGS. 18D and 18E is that the process corresponding to the additional word registration process of S1807 to S1820 of FIGS. 18A, 18B and 18C is 18D and FIG. 18E do not have this point.

  When the LED blinks in green to prompt the user 331 to input the additional information 1 (S1871), a menu for registering the additional information 1 is displayed on the display unit 425. The user 331 creates the additional information 1 by operating according to the displayed menu screen. After the creation is completed, the additional information 1 is fetched by the input management unit 420 (S1874). The input management unit 420 transfers the captured additional information 1 to the trigger setting unit 403 (S1875). The trigger setting unit 403 stores the transferred additional information 1 in the reserved word storage area 410-2 of the memory 410 (S1876).

  The trigger setting unit 403 stores the additional information 1 in association with the reserved word recognized in S1856 when storing the additional information 1 in the reserved word storage area 410-2 of the memory 410. As a result, it becomes possible to save the operation content associated with the reserved word recognized in S1856.

  FIGS. 19A and 19B show the words issued by the user 331 when the additional word and the additional information 2 are stored in the reserved word storage area 410-2 of the memory 410 by the processing shown in FIGS. 18A, 18B, and 18C. When a reserved word and an additional word are recognized from the inside, the additional information 2 corresponding to the recognized combination of the reserved word and the additional word is read from the reserved word storage area 410-2, and the operation is set to the host device 332. Is an example of the sequence.

  The processes of S1900 to S1908 shown in FIG. 19A are the same as the processes of S1600 to S1608 shown in FIG. 16A, respectively. 19A and 19B are different from the processes of FIGS. 16A and 16B in that the additional word recognition process of S1909 to S1911 is added, and the additional information 2 reading process of S1912 to S1913 is added. Is the point to do.

  When the reserved word is successfully recognized in S1905 of FIG. 19A with respect to the data in which the words the user 311 has taken in, the trigger recognition unit 420 succeeds in recognizing the data in which the words the user 311 has taken. The voice data continuously input to the reserved word is compared with the recognition data (additional word) read from the reserved word storage area 410-2 of the memory 410 to determine whether it is an additional word. Is performed (S1911). When the voice data continuing to the reserved word is recognized as an additional word, the trigger recognition unit 405 reads the corresponding reserved word and additional information 2 corresponding to the additional word from the reserved word storage area 410-2 of the memory 410 ( S1912). The trigger recognition unit 405 that has read the additional information 2 sets the operation of the content of the read additional information 2 in the host device 332 (S1913).

  As described above, by registering the reserved word, the additional word, and the additional information in the host device 332, the host device 332 operates as the host device 332 and operates as to the devices and sensors connected to the host device 332 via the network. Can be controlled freely, and it is possible to control devices and sensors that suit the lifestyle of each individual.

  In FIG. 20, when a plurality of reserved words are registered in the host device 332, when the user 331 recognizes that one of the reserved words is one of the reserved words, the speech recognition cloud is displayed according to the recognized reserved word. 3 is a list of examples of changing the voice recognition dictionary used by the voice text conversion unit 101-2 of 101.

  The host device 332 can register a plurality of reserved words. When the host device 332 recognizes the word issued by the user 331 as one of the registered reserved words, the voice text conversion unit 101-2 of the voice recognition cloud 101 according to the recognized reserved word. It is possible to change the voice recognition dictionary for converting the voice used in to the text. For example, as shown in FIGS. 21A and 21B, the host apparatus 332 is assumed to have registered three "Hello", "Hello", "Ookini" as reserved words. In this case the host device 332, if you recognize the "Hello" reserved word, it is possible to issue an instruction to change the voice recognition dictionary used in speech-to-text conversion unit 101-2 of the speech recognition cloud 101 to Japanese dictionary .. When the reserved word “Hello” is recognized, the host device 332 issues a command to the voice text conversion unit 101-2 of the voice recognition cloud 101 to change the type of voice recognition dictionary to an English dictionary. be able to. Furthermore, when the reserved word “Ookini” is recognized, the host device 332 changes the type of the voice recognition dictionary used by the voice text conversion unit 101-2 of the voice recognition cloud 101 to a dialect dictionary (Kansai dialect). Can issue orders.

  In order to change the type of the voice recognition dictionary used by the voice text conversion unit 101-2 of the voice recognition cloud 101 according to the reserved word recognized by the host device 332, the user 331 registers the reserved word in the host device 332. In doing so, it is necessary to register the type of the voice recognition dictionary used by the voice text conversion unit 101-2 (hereinafter referred to as additional information 3) in association with the reserved word.

  The processing sequence for registering the type (additional information 3) of the voice recognition dictionary corresponding to the reserved word together with the registration of the reserved word is the processing sequence for registering the additional information 1 for the reserved word shown in FIGS. 15A and 15B. The additional information 1 may be selected and input instead of inputting the additional information 1 on the menu screen displayed on the display unit 425 (S1516). Hereinafter, the flow of processing for registering the additional information 3 will be described using the processing from S1514 of FIG. 15B. The additional information 1 described after S1514 in FIG. 15B will be replaced with the additional information 3 for description.

  When the LED blinks in green to prompt the user 331 to input the additional information 3 (S1514), a menu for registering the additional information 3 is displayed on the display unit 425. The user 331 can select the type of dictionary as the additional information 3 by inputting the additional information 3 according to the displayed menu screen. When the creation is completed, the additional information 3 is taken into the input management unit 420 (S1516). The input management unit 420 transfers the captured additional information 3 to the trigger setting unit 403. The trigger setting unit 403 stores the transferred additional information 3 in the reserved word storage area 410-2 of the memory 410.

  The trigger setting unit 403 stores the additional information 3 in association with the reserved word registered in S1513 when storing the additional information 3 in the reserved word storage area 410-2 of the memory 410.

  21A and 21B show that the voice text conversion unit 101-2 is used when a plurality of reserved words are registered in the host device 332 as shown in FIG. 20 and each reserved word is recognized by the host device 332. The example of a sequence which changes the kind of speech recognition dictionary used is shown. The processes of S2100 to S2113 shown in FIGS. 21A and 21B are the same as the processes of S1600 to S1613 shown in FIGS. 16A and 16B, respectively. 21A and 21B is different from the processes of FIGS. 16A and 16B in that in the processes of FIGS. 16A and 16B, after the trigger recognition unit 403 reads the additional information 1, the additional information 1 While the operation of the host device 332 is set based on the content (S1614), in the case of FIGS. 21A and 21B, after the trigger recognition unit 403 reads the additional information 3, the content of the additional information 3 is set. The point is that the voice-text conversion unit 101-2 exchanges with the voice-text conversion unit 101-2 in order to change the type of the voice-recognition dictionary used (S2114-1 to S2114-3).

  As for the display notifying the user that the reserved word recognition and the voice recognition dictionary change are completed, the trigger setting unit 403 sends a registration completion notification to the display device 425 (S2109), and the registration completion notification is sent. It is desirable that the display device 425 that has received the message illuminate the LED in green, for example, by a display method that can be recognized by the user 331. Alternatively, the trigger recognition unit 405 sends a recognition completion notification to the speaker 423, and the speaker 423 that has received the recognition completion notification asks, for example, “Yes, yes? By the way, the voice recognition dictionary is a dialect dictionary (Kansai). It has been changed to "valve". "Alternatively, a method of making an audio announcement to the user 331 may be used. Alternatively, the trigger recognition unit 405 notifies the user 331 that the recognition of the reserved word and the change of the voice recognition dictionary corresponding to the recognized reserved word are completed, in order to notify the user 331 of the method using the display device 425 and the speaker. Both voice methods using 423 may be used.

  Note that the operation content (additional information 1) corresponding to the reserved word shown in FIG. 14, the operation content for each additional word (additional information 2) with respect to the reserved word shown in FIGS. 17A and 17B, and the reservation shown in FIG. The types of the voice recognition dictionary (additional information 3) for words can be registered in combination.

  22. FIG. 22 shows the registration of the operation content corresponding to the reserved word shown in FIG. 14, the registration of the additional word for the reserved word shown in FIG. 17A, the registration of the operation content for the additional word, and the voice recognition of the reserved word shown in FIG. It is a list of combinations when registration of dictionary types is performed in combination. The host device 332 is, for example, a reserved word for the "Hello" is configured to use the Japanese dictionary as the type of voice recognition dictionary. The host device 332, also "Chan", "Ya" as an additional language to the reserved word "Hello" to register the "Hey", the host device so as to raise the tone at the time of response in the case of the additional word "chan" Change the operation contents of 332, change the setting contents to lengthen the expiration time T0 of the input interval confirmation timer T when the additional word is "ya", and if the additional word is "Oi", There is no such action "so that it will be announced immediately.

  FIG. 23 is a list of examples in which the type of the voice recognition dictionary used by the text conversion unit 101-2 is changed according to the contents other than the reserved words (hereinafter referred to as change conditions). For example, FIG. 23 (A) shows an example in which time is set as the change condition. The host device 332 instructs the text conversion unit 101-2 of the voice recognition cloud 101 to change the type of the voice recognition dictionary used when converting the voice data into text depending on the time when the voice recognition dictionary is used. An example is shown.

  For example, the host device 332 uses the family general dictionary from 05:00 to 08:00, the wife's dictionary from 08:00 to 16:00, and the time from 16:00 to 20:00. To the text conversion unit 101-2 via the Internet 2 to use the family general dictionary up to and from time 20:00 to 05:00.

  In addition, FIG. 23B is an example when the change condition = the operation status of the host device 332. The host device 332 can instruct to change the type of the voice recognition dictionary used by the text conversion unit 101-2 according to the type of the operation status of the host device 332 when using the voice recognition dictionary.

  The host device 332 uses, for example, the time / route search dictionary when the operation status = going to work, the general dictionary when the operation status = going out, and the refresh dictionary when the operation status = night mode. To the text conversion unit 101-2 via the Internet 2.

  The host device 332 has a mode (hereinafter referred to as a change condition registration mode) for registering change condition type information, which is information on the type of the voice recognition dictionary to be used according to the conditions.

  The user 331 needs to register the change condition type information in the host device 332 in advance in order to properly use the type of the voice recognition dictionary according to the change condition.

  The registration method for selectively using the type of the voice recognition dictionary according to the change condition is that the host device 332 captures the voice uttered by the user 331 through the microphone 421 and analyzes the captured voice data to enable registration. Good. Alternatively, a menu for setting the additional information 1 may be displayed on the display device 425 so that the user 331 can register it by operating according to the menu. Alternatively, using an external device connected via the network I / F 427 shown in FIG. 4, such as a smartphone or tablet, a menu for setting the additional information 1 to the reserved word is displayed on the display screen of the smartphone or tablet. The user 331 may be registered by operating according to the displayed menu screen.

  FIG. 24 is a process of displaying a menu for setting change condition type information displayed on the display unit 425, and operating the user 331 according to the menu to register the type of the voice recognition dictionary to be used according to the change condition. It is an example of a sequence. The processing of S2417 to S2423 shown in FIG. 24 is the same as the processing of S1517 to S1523 of FIG. 15B which is the registration sequence of the additional information 1.

  The user 331 operates the displayed menu screen to input the type of the voice recognition dictionary to be used according to the change condition. The change condition type information of which the input is completed is taken into the input management unit 420 (S2417). The input management unit 420 transfers the fetched change condition type information to the trigger setting unit 403 (S2418). The trigger setting unit 403 stores the transferred change condition type information in the reserved word storage area 410-2 of the memory 410 (S2419).

  FIG. 25 shows a case where the change condition type information for changing the type of the voice recognition dictionary according to the change condition is stored in the reserved word storage area 410-2 of the memory 410 as shown in FIG. It is an example of a processing sequence in the case where the host device 332 notifies the voice / text conversion unit 101-2 of the change of the voice recognition dictionary according to the contents of the change condition type information that has been performed.

  It is desirable that the processing of FIG. 25 be continuously performed after the reserved word recognition processing shown in FIG. 9B is completed (S911), for example. Alternatively, after the reserved word is recognized, as shown in FIGS. 10A and 10B, when the user 331 issues the word to the host device 332 to control the device or sensor, the host device 332 captures the word. It is desirable to carry out at the same timing (S1001).

  FIG. 25 shows the timing (S1001) when the user device 331 utters a word to the host device 332 in order to control the device or sensor as shown in FIGS. 10A and 10B, and the word is taken in by the host device 332 (S1001). This is an example of the case of determining the change of the voice recognition dictionary and notifying the result.

When the recognition of the reserved word is completed, the host device 332 continuously captures the voice uttered by the user into the input management unit 420 through the microphone 421 (S2501). The input management unit 420 transmits a read request (change condition type information) to the voice processing unit 407 to read the change condition type information at the timing of capturing the voice data (S2502), Pause. Upon receiving the read request (change condition type information), the voice processing unit 407 reads the change condition type information including the combination of the change condition and the voice recognition dictionary type from the reserved word storage area 410-2 of the memory 410. (S2503). The voice processing unit 407 analyzes the “change condition” of the read change condition type information, and determines whether the content matches the state of the host device 332 (S2504). When it is determined that the voice recognition dictionary is suitable, the voice processing unit 407 reads the “voice recognition dictionary type” corresponding to the “change condition” and notifies the voice recognition dictionary type notification of the changed voice recognition dictionary type. To the voice / text conversion unit 101-2 via the Internet 2 (2505). Upon receiving the voice recognition dictionary type notification, the voice text conversion unit 101-2 refers to the notified voice recognition dictionary type, and changes the currently used voice recognition dictionary type to the notified voice recognition dictionary type. (S2506)
When the change of the type of the voice recognition dictionary is completed, the voice / text conversion unit 101-2 notifies the voice processing unit 407 of a voice recognition dictionary change completion notification as a change completion notification (S2507).

  Upon receiving the voice recognition dictionary change completion notification (S2507), the voice processing unit 407 transmits a read completion notification to the input management unit 420 as a notification that the read of the change condition type information is completed (S2508). .. When the input management unit 420 receives the read completion notification (S2508), the input management unit 420 restarts the processing on the audio data captured in S2501.

  The user 331 may forget the reserved word registered in the host device 332. In preparation for such a case, it is desirable that the user 331 be able to confirm the registered reserved words by a simple method.

  In FIG. 26, when the user 331 who has registered a reserved word in the example of the processing sequence shown in FIGS. 5A and 5B forgets the registered reserved word, some or all of the registered reserved words are written by the user 331. A list of examples of reserved words (hereinafter referred to as “reservation reserved words”) and display contents (display range) for notifying the user is shown. For example, when the reserved word "I don't know" is displayed on the display unit 425 of all the reserved words registered in the host device 332, or in the display area of an external device connected to the host device 332. Is shown. For the reserved word "tell me a little", a predetermined part of the reserved words registered in the host device 332 is displayed on the display unit 425, or an external device connected to the host device 332. It shows the case of displaying in the display area of. For the reserved word “unused”, the reserved words registered in the host device 332 and having no usage history for the past year are displayed on the display unit 425, or connected to the host device 332. It shows the case of displaying in the display area of an external device. As an external device connected to the host device 332, for example, a device such as a smartphone, a tablet, or a liquid crystal television, which has a relatively large display screen and allows the user to refer to many reserved words at one time, is desirable.

  As described above, the registration of the reserved word for displaying the registered reserved word is performed by changing the mode of the host device to the setting mode (reserved word (for display)) and changing the reserved word shown in FIGS. 5A and 5B. The registration may be performed according to the registration processing sequence.

  The above example is an example in which when the user issues the "reservation reserved word" shown in FIG. 26, the corresponding reserved word is immediately displayed. However, the host device 332 may ask the user 331 for a secret word before displaying the corresponding reserved word. After the user issues the "reservation reserved word", the host device 332 emits a sound such as "mountain" through the speaker 423, and when the user 331 responds "kawa", the corresponding reserved word is issued. It may be displayed.

  Furthermore, the host device 332 can also capture the words issued by the user 331 and record or record the reserved word, the additional word, or the scene in which the additional information is registered. Alternatively, when the reserved word or the additional word is recognized, the recognized scene can be recorded or recorded.

  FIG. 27 shows that when the host device 332 captures a word issued by the user 331 and records a scene of a reserved word, an additional word, or additional information, and a reserved word or the recognition of the additional word, 3 shows a functional block diagram of 332. The difference from FIG. 4 is that the host device 2700 has a camera 2702 for recording a scene in which a reserved word, an additional word, or additional information is registered, or a scene for recognizing a reserved word or an additional word. The control management unit 2701 has EVT-Mg2701-3 in addition to APP-Mg2701-1 and CONF-Mg2701-2, and the system controller 402 has a reproduction control function for reproducing data of a recorded or recorded scene. That is the point. The EVT-Mg2701-3 has a function of performing recording or recording described later due to the occurrence of a scene in which a reserved word, an additional word, or additional information is registered, and the occurrence of a scene in which the reserved word or the additional word is recognized. have. Hereinafter, the host device 332 captures the words uttered by the user 331 and records or records a scene for registering reserved words, additional words, or additional information, and a scene for recognizing reserved words and additional words. The flow of a recording process or a recording process will be described.

  In FIG. 28, when a scene for registering a reserved word, an additional word, or additional information occurs, or a scene for recognizing a reserved word, an additional word occurs, the registered scene or the recognized scene is used as a host device. The reference numeral 332 indicates a time lapse when recording or recording.

  It is assumed that at time t1, the host device 332 starts registration with the word issued by the user as a reserved word. The registration of the reserved word may be started, for example, at the timing when the input management unit 420 performs the process of S502 in the reserved word registration sequence of FIGS. 5A and 5B. When the input management unit 420 recognizes the start of registration of a reserved word, it notifies the EVT-Mg2701-3 to that effect. The EVT-Mg2701-3 that has received the notification that the reserved word registration is started records the reserved word registration scene as Rec1 through the microphone 421 or the reserved word registration scene as Rec1 through the camera 2702. The reserved word registration may be ended at the timing when the input management unit 420 receives the registration completion notification of S512 in the reserved word registration sequence of FIGS. 5A and 5B, for example. The input management unit 420, which has recognized the end of the reserved word registration, notifies the EVT-Mg2701-3 to that effect. The EVT-Mg2701-3, which has received the notification that the reserved word registration is completed, terminates the recording of the reserved word registration scene performed through the microphone 421, or ends the reserved word registration scene recording performed through the camera 2702. Let

  Similarly, it is assumed that at time t2, the host device 332 starts recognition of a word uttered by the user as a reserved word. The recognition of the reserved word may be started, for example, at the timing when the input management unit 420 performs the process of S802 in the reserved word recognition sequence of FIGS. 8A and 8B. Upon recognizing the start of the recognition of the reserved word, the input management unit 420 notifies the EVT-Mg2701-3 to that effect. The EVT-Mg2701-3 that has received the notification that the reserved word recognition is started records the reserved word recognition scene as Rec2 through the microphone 421, or records the reserved word recognition scene as Rec2 through the camera 2702. The recognition of the reserved word may be ended at the timing when the input management unit 420 receives the recognition completion notification of S811 in the reserved word registration sequence of FIGS. 8A and 8B, for example. The input management unit 420, which has recognized the end of the reserved word registration, notifies the EVT-Mg2701-3 to that effect. The EVT-Mg2701-3, which has received the notification that the reserved word registration is completed, terminates the recording of the reserved word recognition scene performed through the microphone 421, or the recording of the reserved word recognition scene performed through the camera 2702. Let

  Similarly, record or record the registration or recognition event that occurred at t3 and t4.

  The host device 332 can play back the recorded scene or the recorded scene of registration or the scene of recognition.

  FIG. 29 shows an example of how the data to be reproduced is displayed when reproducing each data of the recorded or recorded scene. In the example of FIG. 29, four reproduction target data icons are displayed in a manner corresponding to the occurrence of events on the time axis of FIG. 28. The icon display of the reproduction target data may be displayed on the display unit 425, for example. Alternatively, it may be displayed on an external device connected to the host device 332, such as a smartphone, a tablet, or a liquid crystal television.

  The displayed icons represent recording or recording date and time, and contents of recording or recording target data. For example, when the display content of the icon is the reserved word registration “OOKINI”, it indicates that the content of the recorded or recorded data is a scene in which “OOKINI” is registered as the reserved word. Similarly, when the display content of the icon is the reserved word recognition “Ookini”, it indicates that the content of the recorded or recorded data is a scene in which “Ookini” is recognized as the reserved word.

  The user 331 can confirm the recording of the target data or the recorded contents by selecting the icon of the data to be reproduced.

  Furthermore, the host device 332 gives an instruction to a camera or a microphone connected via the network 333, and when a scene for registering a reserved word, an additional word, or additional information is generated by these cameras or microphones, or When a scene for recognizing a reserved word or an additional word occurs, a registered scene or a recognized scene may be recorded or recorded.

  As described above, the host device 332 recognizes the reserved word from the words issued by the user 331, and based on the content of the additional information corresponding to the reserved word, the device or the sensor connected to the network. Can be controlled. The control contents of the target device or sensor may require high security. For example, it is assumed that a reserved word in which the opening / closing operation of the safe door is set as the additional information is registered in the host device 332 so that the opening / closing control of the safe door can be performed using the host device. In this case, when the host device 332 recognizes the corresponding reserved word, the host device 332 opens and closes the door of the safe, and uses a microphone and a camera around the safe to record or record around the safe, which is the device to be controlled. By recording the video, it becomes possible to maintain the security of the opening / closing operation of the safe door. The user 331 confirms the content of the data recorded or recorded using the microphone or camera connected to the network, as well as the data recorded or recorded using the microphone or camera built in the host device 332. You can do it. If the host device 332 requires high security for the control content of the device or sensor to be controlled, the host device 332 may also perform the control content before the control device or the microphone around the sensor. The validity of the person who uttered the recorded voice or the person of the recorded video may be confirmed by using the voice or the video recorded by the camera. The host device 332 uses a feature point such as a voice or face of a specific person, which is registered in advance, and a microphone or a camera around the device or sensor to be controlled, before executing the control content of the specific additional information. It is also possible to compare the collected sound and the captured video and execute the corresponding control content only when the legitimacy of the person is confirmed.

  In the description of the above embodiment, the recognition data conversion unit 101-1, the voice text conversion unit 101-2, the text analysis unit 102-1 and the response / action generation unit 102-2 are all included in the cloud server 1. Although it has been described as being present, some or all of these may be present in the host device 332. Also in that case, the example of the operation sequence of each processing already described is the same as the one already described.

  Although some embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope and gist of the invention, and are also included in the invention described in the claims and the scope equivalent thereto.

  1 ... Cloud server, 2 ... Internet, 3 ... Home, 101 ... Voice recognition cloud, 102 ... Corresponding action generation cloud, 310 ... Various sensors, 320 ... Various equipment , 330 ... HGW (HomeGateway), 331 ... User, 332 ... Host device, 340 ... Various home appliances.

Claims (2)

Based on the content of the first voice input from the outside, it is determined whether to control one or a plurality of devices based on the content of the second voice input after the input of the first voice. In electronic devices,
The determination voice data for determining whether the first voice is a desired voice is created and managed by the voice input from the outside multiple times, and the determination voice data created and managed is used to perform the Management means for determining that the first voice is the desired voice,
A control unit that controls the one or more devices based on the content of the second voice, and the first voice is a desired voice using the determination voice data by the management unit. An electronic device that controls the one or more devices based on the content of the second voice by the control means when it is determined that
When the first voice is determined to be a desired voice by the management means using the determination voice data, based on a part or all of the contents of the second voice, Change the control content of one or more devices,
When the management unit has a plurality of the determination voice data, one or a plurality of the determination voice data are used according to the type of the determination voice data used to determine that the first voice is a desired voice. An electronic device that changes the control contents of the device. Based on the content of the first voice input from the outside, it is determined whether to control one or a plurality of devices based on the content of the second voice input after the input of the first voice. In the control method,
The determination voice data for determining whether the first voice is a desired voice is created and managed by the voice input from the outside multiple times, and the determination voice data created and managed is used to perform the Management means for determining that the first voice is the desired voice,
A control unit that controls the one or more devices based on the content of the second voice, and the first voice is a desired voice using the determination voice data by the management unit. When it is determined that the control means controls the one or a plurality of devices based on the content of the second voice,
When the first voice is determined to be a desired voice by the management means using the determination voice data, based on a part or all of the contents of the second voice, Change the control content of one or more devices,
When the management unit has a plurality of the determination voice data, one or a plurality of the determination voice data are used according to the type of the determination voice data used to determine that the first voice is a desired voice. Control method to change the control contents of the equipment of.

Images