JP2005072764A - Equipment control system and device thereof, and equipment control method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To achieve voice operation without specifying equipment to be operated by voice including existing home electric appliances of an infrared I/F by using a client server type voice recognition system. <P>SOLUTION: Barcodes 21, 31, 41 are provided at each of infrared communication equipment 20, radio communication equipment 30, and cable communication equipment 40; the image of the barcode of the equipment to be operated is inputted by a remote controller 1; and image data and voice data are transmitted to a server 10 when the operation details of the equipment to be operated are inputted by voice. In the server 10, the equipment to be operated is recognized from image data, and operation details are recognized from voice data, thus forming control codes. Based on the recognition, the control code is transmitted to the equipment 30 by radio when the equipment 30 is to be operated. When the equipment 40 is to be operated, control codes are transmitted to the equipment 40 by cables. When the equipment 20 is to be operated, the control codes are transmitted to the remote controller 1, and the control codes are transmitted to the equipment 20 by infrared communication from the remote controller 1. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to a device control system using a voice recognition function and an image recognition function, an apparatus therefor, and a device control method.

  When remotely controlling a controlled device such as a home appliance, a remote controller (hereinafter referred to as a remote controller) attached to the controlled device is usually used. This is because when a user operates a predetermined operation button provided on the remote controller, a control signal corresponding to the operation button is transmitted to the controlled device by infrared rays, and the controlled device is controlled by the control signal. To perform a predetermined operation. By the way, new functions are added to home appliances and so on, and the number of operation buttons increases, and the number of operation buttons increases. ing.

  By the way, there has been proposed a voice recognition system that uses a voice recognition function to recognize voice information input by a user, convert this into character information, and provide the text information to the user side (see, for example, Patent Document 1).

  There has also been proposed a system that recognizes voice information input from a remote controller by using such a voice recognition system, converts the voice information into a control signal, and controls a controlled device by the control signal (for example, a patent). Reference 2).

  Patent Document 2 describes a device control device as a remote control in which a device control device (remote control) is provided with a voice input function and a voice recognition function. In this device control device, when voice information for controlling a desired controlled device is input by a user, the voice information is converted into a recognized command, and the command is selected from previously stored control signals. Is selected, and this control signal is transmitted to the controlled apparatus desired by the user. Thereby, the controlled device desired by the user is controlled so as to be desired by the user.

  In addition, this Patent Document 2 also describes a control system using a client / server type speech recognition system. In this control system, a voice recognition function is provided on the home server side, and a voice input function is provided on the device control device on the client side so that communication can be performed between the device control device and the home server. In addition, a controlled device to be controlled is connected to the home server via a network.

  In such a configuration, when voice information for controlling a desired controlled device is input to the device control apparatus by the user, the voice information is transmitted from the device control apparatus to the home server. In the home server, the voice information is converted into a recognized command, a control signal corresponding to this command is selected from control signals stored in advance, and this control signal is transmitted to the controlled device desired by the user. Is done. Thereby, the controlled device desired by the user is controlled as desired by the user.

  In addition, a system using an image recognition technique for controlling a plurality of operated devices with a computer has been proposed (see, for example, Patent Document 3).

  This is a configuration in which a user terminal and a target as an operated device are connected via a network, and each target is provided with a visual marker for distinguishing it, and the user terminal It has a recognition function and is connected to a camera for photographing the target.

  In such a configuration, when the user captures a target that the user wants to operate with the camera, including the visual marker, the user terminal recognizes the captured image, and recognizes the target target by the visual marker. The user terminal is provided with a database in which the visual marker and the target address are associated with each other. The address corresponding to the visual marker acquired from the photographed image of the camera is acquired from this database, and the user uses this address to By accessing the target that the user desires to operate, this target is connected to the user terminal so as to be communicable.

  When connected in this way, the target can be operated from the user terminal. For example, when the target is a projector tool, the slide page stored in the user terminal can be displayed on the projector by an operation from the user terminal, and the slide page currently displayed on the projector is displayed. Can be displayed on the user terminal.

Note that it is not necessary to capture images in real time, a still image obtained by capturing the target may be used, and the target may be a remote device reflected on the screen in a video conference or the like as well as the current object. This system is characterized in that it is intuitively easy to understand that a connected target among a plurality of targets is a user looking through a user terminal.
Japanese Patent Laid-Open No. 7-222248 JP 2000-236585 A JP 2001-142825 A

  Based on the technique described in Patent Document 2, since the controlled device can be controlled by voice input by providing the remote control of the controlled device with a voice recognition function, the operation button operation is also omitted. The user can easily operate without memorizing the position. However, the remote control is generally expensive, and high functionality cannot be expected from the remote control. Therefore, even if the voice recognition algorithm is improved and the performance of the CPU or the like is improved, it is necessary to install a voice recognition function in the remote control because the remote control requires a processing circuit and a storage device for voice recognition. This is a problem because it increases the size and increases the cost of the remote control.

  On the other hand, as described in Patent Document 2, it is conceivable to use a client / server type voice recognition system that has a voice function on the home server side instead of a remote controller. In the device control system used, home appliances that are controlled devices must have a network function. In the future, home networks will become popular due to improvements in network technology and lower prices of network equipment, but in the current situation, it will take more time to achieve this. For this reason, a system that can be used in existing home appliances is required. Of course, even with existing home appliances, networking can be realized by connecting an adapter or the like having a network function, but this will incur costs.

  Further, when there are a plurality of controlled devices, it may be possible to control them with a single remote controller by using a voice recognition function. In such a case, in order to control the controlled device to be controlled, this controlled device must be designated by voice. For this purpose, a name or number must be assigned to each controlled device. Instead, in order to specify a desired controlled device, it is necessary to input a name or number corresponding to this to the device control apparatus by voice. For example, when the power of the television receiver is turned on, voice input is made by saying "TV ON" or the like. If there is only one television receiver, this may be sufficient. When a machine is to be controlled, for example, the target controlled device such as “living TV 1 ON” must be specified accurately, which makes it difficult for the user to use.

  As a means for solving this problem, it is conceivable to specify a target controlled device using image recognition technology as described in Patent Document 3 above. Providing the image recognition function requires a large-scale processing circuit and database for the image recognition, which increases the size of the remote control and greatly increases the cost.

  In addition, it is conceivable to use a client-server type image recognition system in which a home server has an image recognition function, as in the system described in Patent Document 3. Such a device that does not have a communication function with the server cannot be controlled using such a system.

  An object of the present invention is to solve such a problem and use a client / server type information recognition system to enable a device having only a control communication function with a portable control terminal to be controlled. A control system, an apparatus therefor, and a device control method are provided.

  To achieve the above object, a device control system according to the present invention has a voice input means, a portable control terminal that transmits voice data for controlling a controlled device input from the voice input means, and a portable control. A server that receives voice data transmitted from the terminal, recognizes the voice data, and transmits the recognition result to the portable control terminal as control information; the portable control terminal transmits the control information transmitted by the server; Is transmitted to the controlled device.

  Further, the present invention is a portable control terminal of a device control system including a server that receives voice data for controlling a controlled device, recognizes the voice data, and forms control information of the controlled device, Voice input means comprising voice input means for inputting voice data for control of the controlled device, wireless communication means for wireless communication with the server, and infrared communication means for infrared communication with the controlled device. Is transmitted to the server by the wireless communication means, the control information is received from the server by the wireless communication means, and is transmitted to the controlled device by the infrared communication means to control the controlled device. .

  The present invention also provides a server of a device control system for controlling a controlled device based on sound data for controlling the controlled device input from a portable control terminal, wherein the sound data is received from the portable control terminal. Radio communication means for receiving, registration data holding means for registering voice data inputted at the portable control terminal for controlling the controlled device and control information corresponding to the voice data, and holding information of the registration data holding means A control information converting means for recognizing the voice data received by the wireless communication means and converting the voice data into the corresponding control information, and transmitting the control information to the portable control terminal by the wireless communication means It is.

  In the device control method according to the present invention, the portable control terminal inputs voice data for controlling the controlled device, the server receives the voice data input from the portable control terminal, and receives the received voice data. The control information is recognized and converted into control information of the controlled device, the control information is transmitted to the portable control terminal, the control information is received by the portable control terminal, and the controlled device is controlled by the control information.

  Further, the present invention is a device control system that has a plurality of controlled devices with different communication methods and controls the controlled device by inputting voice data from a portable control terminal, and each controlled device is itself And a portable control terminal that performs infrared communication with at least one of the controlled devices, and is to be controlled among a plurality of controlled devices. Inputs audio information for device control and image data for the code of the controlled device to be controlled, recognizes the image data, determines the controlled device to be controlled, recognizes the audio data, and controls Control information of a controlled device to be controlled is generated, and a server that transmits the control information to the controlled device to be controlled based on the determination result by a corresponding communication means is provided.

  At least one of the controlled devices is a predetermined controlled device that performs infrared communication with the portable control terminal, and the server determines that the controlled device to be controlled is the predetermined controlled device. Control information is supplied to the portable control terminal by wireless communication, and the portable control terminal transmits the received control information to a predetermined controlled device by infrared communication.

  The present invention also receives audio data for controlling the controlled device and image data for identifying the controlled device, and recognizes the audio data and the image data to form control information for the controlled device. Audio control data for controlling a controlled device to be controlled, which is a portable control terminal of a device control system provided with a server that provides a visible code for identifying each controlled device. A voice input means for inputting the image, an image input means for inputting image data of the code of the controlled device to be controlled, a wireless communication means for wirelessly communicating with the server, and infrared communication with at least one of the controlled devices An infrared communication means for transmitting the voice data input from the voice input means and the image data input from the image input means to the server by wireless communication means, Receiving control information by the wireless communication means from the chromatography server, the infrared communication unit, and controls to send the at least one controlled device.

  In addition, the present invention is a server of a device control system that controls a plurality of controlled devices having different communication methods based on voice data for controlling the controlled device input from a portable control terminal. Wireless communication means for receiving audio data and image data indicating a controlled device to be controlled from a portable control terminal; image recognition means for recognizing the image data and determining a controlled device to be controlled; Registration data holding means for registering voice data input at the portable control terminal for controlling the control device and control information corresponding to the voice data, and reception by the wireless communication means based on the holding information of the registered data holding means Control information conversion means for recognizing the voice data and converting the voice data into the corresponding control information, and the controlled device to be controlled determined by the image recognition means The appropriate communication method, and transmits the control information.

  At least one of the plurality of controlled devices is a predetermined controlled device that performs infrared communication with the portable control terminal, and the controlled device to be controlled by the image recognition unit is the predetermined controlled device. Is determined, the control information is transmitted to the portable control terminal by wireless communication.

  In addition, the device control method according to the present invention is a portable control terminal that includes voice data for controlling a controlled device to be controlled and a controlled device to be controlled among a plurality of controlled devices having different communication methods. The image data for identification is input, the server receives the audio data and the image data input from the portable control terminal, recognizes the received audio data, and controls the control information of the controlled device to be controlled. Converting, recognizing the received image data to determine the controlled device to be controlled, and sending control information to the controlled device to be controlled based on the determination result by the corresponding communication method, It is something to control.

  At least one of the plurality of controlled devices is a predetermined controlled device that performs infrared communication with the portable control terminal, and the controlled device to be controlled by the image recognition unit is the predetermined controlled device. Is determined, the control information is transmitted to the portable control terminal by wireless communication, and the portable control terminal transmits the control information to a predetermined controlled device by infrared communication.

  ADVANTAGE OF THE INVENTION According to this invention, even if it is an existing household appliance, it becomes possible to control a device by voice using a client / server type voice recognition system without adding a network function.

  In addition, for a plurality of devices connected to a wireless or wired network, a client / server type speech recognition system can be used to control a device to be operated without specifying it by voice.

  Since a client-server type recognition system is used, the remote controller as a client does not need to be enhanced with a voice or image recognition function, and the cost does not increase. Even when a plurality of remote controllers are used, it is sufficient that the functions for voice recognition and image recognition are provided only in the home server, and the high cost of the system can be prevented.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
FIG. 1 is a schematic configuration diagram of an embodiment of a device control system according to the present invention, in which 1 is a portable control terminal (hereinafter referred to as a remote controller) which is a control device having a remote control function, 10 is a home server, Reference numerals 30 and 40 denote home appliances as controlled devices. In the following, it is assumed that the device control system is provided in the home, but it is needless to say that the device control system is not limited to this.

  In the figure, the home server 10 is equipped with a wireless communication I / F, a wired communication I / F, a voice recognition function, an image recognition function, and a database. The remote controller 1 is equipped with an imaging mechanism such as a camera (hereinafter referred to as a camera), a wireless communication I / F, and an infrared I / F, and is configured to be able to perform wireless communication with the home server 10. The home appliances 20, 30, and 40 are controlled devices to be controlled by the remote controller 1. For example, the home appliance 20 is a television (TV receiver), the home appliance 30 is an air conditioner (air conditioner), and the home appliance 40 is a video. (Video tape recorder).

  Here, it is assumed that the home appliance 20 is equipped with an existing infrared I / F (infrared communication function) and is configured to be able to communicate with the remote controller 1 by infrared. The home appliance 30 is assumed to be equipped with a wireless I / F (wireless communication function) and configured to be able to perform wireless communication with the home server 10. The home appliance 40 is assumed to be equipped with a wired I / F (wired communication function) and configured to be able to perform wired communication with the home server 10. These home appliances 20, 30, and 40 have their own communication I / F type (infrared, wireless, or wired communication method type) and their own address according to the communication I / F. Visible identifiers (hereinafter referred to as two-dimensional barcodes) 21, 31, and 41 such as coded two-dimensional barcodes are provided at locations on the outer surface where the user can see them well.

  The camera equipped in the remote controller 1 may be built in the remote controller 1 or externally attached. A typical example of such a remote controller is a mobile phone. Mobile phones are becoming more and more functional, and some are equipped with an infrared communication function and a low-power wireless function as well as with a camera. Also, because of the high penetration rate, mobile phones are a powerful remote control 1 It will be a thing.

  When the user wants to operate any desired home appliance 20, 30, or 40, the user images the desired controlled device with the camera of the remote controller 1 (in this case, the desired controlled device). The operation instruction is input to the remote control 1 by voice. The remote controller 1 wirelessly transmits the input audio data and the image data captured by the camera to the home server 10 from the wireless I / F. The home server 10 recognizes the image data and the audio data received from the remote controller 1 and detects a two-dimensional barcode from the image data as a result of the image recognition. From the two-dimensional barcode to the home appliance desired by the user. And the corresponding control code is acquired from the result of speech recognition. And this control code is transmitted to the household electrical appliance of the read address using the read communication means.

  When the home appliance at the read address, that is, the home appliance desired by the user is the home appliance 30 that is an air conditioner, for example, the home server 10 selects the wireless I / F for that purpose, and this wireless I / F The control code is transmitted to the home appliance 30 using / F. The home appliance 30 is controlled in accordance with this control code, and thereby the operation desired by the user is performed. If the desired home appliance that the user wants to operate is the home appliance 40 that is a video, the home server 10 selects a wired I / F for that purpose, and uses this wired I / F to send a control code to this home appliance. Send to device 40. The home appliance 40 is controlled in accordance with this control code, whereby the user's desired operation is performed. Furthermore, when the home appliance desired by the user is the home appliance 20 that is a television, the home server 10 selects a wireless I / F to the remote controller 1 and uses this wireless I / F to generate a control code. It transmits to this household appliances 20. The home appliance 20 transmits the received control code to the home appliance 20 using an infrared I / F. The home appliance 20 is controlled in accordance with this control code, and the operation desired by the user is thereby performed.

  Note that in the case of a controlled device that receives a control code by infrared communication, such as the home appliance 20, infrared transmission from the home server 10 to the controlled device is performed with the home server 10 in terms of directivity. The arrangement relationship with the control device must be taken into account, and restrictions are imposed on these arrangements, so that the control code is not transmitted directly from the home server 10 to the controlled device via the remote control 1. A control code is transmitted from the infrared I / F of the remote controller 1 to the controlled device.

  2 is a block diagram showing an embodiment of each unit in FIG. 1. 2 is an image input unit, 3 is an audio input unit, 4 is an application unit, 5 is a wireless communication unit, 6 is an infrared transmission unit, 11 is Wireless communication unit, 12 an image recognition unit, 13 a target device identification unit, 14 a voice recognition unit, 15 a control code conversion unit, 16 a database unit, 17 a control code transmission unit, 18 a wired communication unit, and 22 An infrared receiving unit, 23 is a control unit, 32 is a wireless communication unit, 33 is a control unit, 42 is a wired communication unit, and 43 is a control unit, and parts corresponding to those in FIG.

  In the figure, the remote controller 1 is input with an imaging mechanism such as a camera and an audio input means such as an image input unit 2 that performs processing such as converting the captured image into digital image data, and a microphone (hereinafter referred to as a microphone). An audio input unit 3 that performs processing such as conversion of analog audio signals into digital audio data, an application unit 4 that processes image data input from the image input unit 2 and audio data input from the audio input unit 3, and wireless A wireless communication unit 5 for performing wireless communication with the home server 10 having an I / F and a communication protocol, and an infrared transmission unit 6 having a function of generating an infrared I / F and an infrared transmission signal are provided. Note that a display or the like may be provided in order to confirm the image input from the image input unit 2.

  Further, the home server 10 has a wireless I / F and a communication protocol, a wireless communication unit 11 for performing wireless communication with the remote controller 1 and the home appliance 30, and image data received from the remote controller 1 by the wireless communication unit 11. The image recognition unit 12 for recognizing the two-dimensional barcode 21, the controlled device (target device) that is the operation target of the user is identified from the two-dimensional barcode 21 recognized by the image recognition unit 12, and its communication I / F And a target device identification unit 13 having a function of reading an address, a voice recognition unit 14 that recognizes voice data received from the remote controller 1 by the wireless communication unit 11, and a recognition result of the voice recognition unit 14 is converted into a corresponding control code. A control code conversion unit 15 for registering, and a database unit 16 for storing the registered voice data, control code, and voice data / control code conversion correspondence table A communication unit corresponding to the communication I / F read by the target device identification unit 13 is selected from the wireless communication unit 11 and the wired communication unit 18, and a control code is assigned to the controlled device at the address read by the target device identification unit 13. A control code transmission unit 17 that generates a transmission signal for transmitting the control code generated by the conversion unit 15 and supplies the transmission signal to the selected communication unit, and a wired I / F and a communication protocol such as Ethernet (Ethernet) And a wired communication unit 18 for performing wired communication with the home appliance 14.

  Note that when the controlled device having the wired I / F is not used, it is not necessary to provide the wired communication unit 18 in the home server 10. The image recognition unit 12, the target device identification unit 13, the voice recognition unit 14, the control code conversion unit 15, and the control code transmission unit 17 are respectively an image recognition program, a target device identification program, a voice recognition program, a control code conversion program, Since it can be realized by software such as a control code transmission program, the home server 10 may be a general-purpose PC (personal computer) instead of a dedicated server.

  The home appliance 20 includes an infrared receiving unit 22 that receives a control code by infrared from the remote controller 1, a control unit 23 that performs a control operation corresponding to the control code received by the infrared receiving unit 22, and the like. Is provided with a two-dimensional barcode 21 in which the type of communication I / F (in this case, the infrared I / F) is coded where it can be easily imaged by the camera of the remote controller 1. In this way, in the controlled device to which the control code is sent by the infrared I / F, when the two-dimensional barcode of the controlled device is imaged by the camera of the remote controller 1, that is, the camera is moved to the two-dimensional image. One where this two-dimensional barcode is provided near the infrared receiving unit so that the infrared receiving unit of the controlled device can receive infrared rays from the remote control 1 when facing the barcode. Is preferred.

  The home appliance 30 has a wireless I / F and a communication protocol, and a control operation corresponding to the control code received by the wireless communication unit 32 for receiving the control code wirelessly transmitted from the home server 10 or the wireless communication unit 32 In addition, the type of communication I / F (in this case, for example, a wireless LAN) and the communication I are provided in a place where the user can easily take an image with the camera of the remote controller 1. A two-dimensional bar code 31 that encodes its own address according to / F is provided.

  The home appliance 40 has a wired I / F and a communication protocol, and a control operation corresponding to the control code received by the wired communication unit 42 for receiving the control code wired from the home server 10 and the wired communication unit 42 In addition, the user can easily take an image with the camera of the remote controller 1 and the type of communication I / F (in this case, for example, Ethernet) and the communication I / F. A two-dimensional bar code 41 in which its own address according to F is encoded is provided.

  Home appliances that are currently popular are those having only an infrared I / F like the home appliance 20, and in this embodiment, such an existing home appliance is also provided with a two-dimensional barcode 21. It can be used as a control device.

  FIG. 3 is a flowchart showing a specific example of the operation procedure of the remote controller 1 in FIG. This will be described with reference to FIG.

  In FIG. 3, when the user turns on the power of the remote controller 1, the remote controller 1 is activated. First, it is determined whether or not data from the home server 10 is received by the wireless communication unit 5 (step S301), and there is no received data. In the case, it is determined whether or not a voice is input from the voice input unit 3 (step S302). If there is no voice input, the process returns to step S301. The remote controller 1 is normally in a state of repeating the operations of steps S301 to S302, that is, in a state of polling for the presence of received data or voice input.

  When the user wants to operate any one of the home appliances 20, 30, and 40, the user wants the camera mounted on the remote controller 1 to point at the controlled device and perform the operation on the controlled device. Information representing the operation is input by voice from the voice input unit 3 (step S302). A trigger is activated by this voice input, and an image captured by the camera by this trigger is input from the image input unit 2 (step S303). The image data input from the image input unit 2 and the audio data input from the audio input unit 3 are supplied to the application unit 4 and combined, and are wirelessly transmitted from the wireless communication unit 5 to the home server 10 as a transmission signal. (Step S304). And it is in the standby state which repeats the operation | movement of step S301-S302 until the next data reception or voice input is received.

  Thereafter, when the wireless communication unit 5 receives data from the home server 10 (step S301), it is determined whether or not the received data is an error code (step S305). If the received data is an error code, the user is notified that the reception has failed (step S306). The notification method may be a sound method using sound output means such as a speaker (not shown), a light method using an LED or the like, or a method of displaying a message or the like on a display (not shown). May be. Also, the type of error may be notified.

  If the received data is not an error code but a control code (step S305), since the received data is a control code for the controlled device having the infrared I / F, that is, the home appliance 20, the infrared data The control unit converts the control code into a control signal by the transmission unit 6, and transmits the control signal to the infrared reception unit 22 of the home appliance 20 by infrared rays (step S307). Thereafter, a standby state is repeated in which the operations of steps S301 to S302 are repeated.

  When a voice is input to the home appliances 30 and 40 in which the control code is wirelessly or wiredly communicated (step S302) and taken with a camera (step S303), the control code is sent from the home server 10 to the remote controller 1. However, if there is an erroneous voice input, an error code is transmitted to the remote controller 1, and the user is notified of this (steps S305, S306, S301).

  Such an error code is, for example, a two-dimensional barcode from the image data in the home server 10 even when voice information not related to the controlled device to which the camera is directed is inputted from the voice input unit 3 or there is voice input. This is due to the case where cannot be recognized. For example, when a voice input of “fast forward” for operation of the home appliance 40 which is a video tape recorder is made while the camera of the remote controller 1 is pointed at the home appliance 20 which is a TV, or the content of a conversation which has nothing to do is input. If it is, an error code is transmitted from the home server 10 to the remote controller 1.

  FIG. 4 is a flowchart showing a specific example of the operation procedure of the home server 10 in FIG. This will be described with reference to FIG.

  In FIG. 4, the home server 10 is in a state of polling whether there is wireless reception data, that is, in a standby state until the wireless communication unit 11 receives data transmitted from the remote controller 1 (step S401).

  If there is reception data in the wireless communication unit 11, the reception data is divided into image data and audio data (step S402), and the image data is supplied to the image recognition unit 12 and the audio data is supplied to the audio recognition unit 14, respectively. The image recognition unit 12 determines whether there is a two-dimensional barcode in the image data (step S403). When the two-dimensional barcode does not exist, an error code is generated and wirelessly transmitted from the wireless communication unit 11 to the remote controller 1. When the two-dimensional barcode exists, this is supplied to the target device identification unit 13. To do. In the target device identification unit 13, the type of communication I / F (infrared, wireless, and wired) of the controlled device targeted by the user from the supplied two-dimensional barcode and this controlled I / F according to the communication I / F. The device address is read (step S404), and information indicating the type and address of these communication I / Fs is supplied to the control code transmitter 17.

  Further, the voice recognition unit 14 determines whether or not the supplied voice data is registered in the database unit 16 (step S405). If the voice data is not registered, an error code is displayed. Created and wirelessly transmitted from the wireless communication unit 11 to the remote controller 1 (step S406), but if the audio data is registered, the control code conversion unit 15 stores the audio stored in the database unit 16 Based on the data control code conversion correspondence table, the voice data is converted into a corresponding control code (step S407). This control code is supplied to the control code transmission unit 17.

  The control code transmission unit 17 generates a control code transmission signal based on the information from the target device identification unit 13 and the control code conversion unit 15, and also transmits the communication I / O of the controlled device to be operated by the user. It is determined whether the type of F is infrared, wireless, or wired (step S408). When the communication I / F of the controlled device to be operated is infrared, this transmission signal is transmitted from the wireless communication unit 11 to the remote controller 1 (step S409), and the communication I / F of the controlled device to be operated is set. If F is not infrared, the above address is added to the transmission signal and transmitted via the communication I / F (step S410). Here, when the communication I / F of the controlled device to be operated is wired, this transmission signal is transmitted from the wired communication unit 18, and the communication I / F of the controlled device to be operated is wireless. In some cases, the transmission signal is transmitted from the wireless communication unit 11. In this way, the control code is transmitted to the home appliance 20 or the home appliance 30 or 40 to be operated.

  In FIG. 4, the processing of steps S403 to S404 and the processing of steps S405 to S406 may be reversed, or may be performed in parallel.

  FIG. 5 is a flowchart showing a specific example of the operation procedure of the controlled devices (home appliance 20, home appliance 30, and home appliance 40) in FIG. This will be described with reference to FIG.

  In the figure, the controlled device first determines whether or not a control signal or control code has been received by the communication unit (infrared receiving unit 22, wireless communication unit 32, wired communication unit 42) (step S501). If not, the process returns to step S501. That is, polling is performed in step S501 until a control signal or control code is received by the communication unit, and a reception waiting state is entered. When the control unit receives the control signal or control code, the control unit (the control unit 23, the control unit 33, or the control unit 43) performs a control operation according to the control signal or control code received (step S502), and the user The desired operation is performed. When this control operation ends, the process returns to step S501.

  By the way, depending on the room, for example, two or more home appliances may be arranged close to each other like a television and a video. In such a case, even if one of the home appliances is to be imaged with the camera of the remote controller 1, depending on the direction of the camera, the two-dimensional barcodes of the two home appliances may be simultaneously imaged. In this case, the image recognition unit 12 recognizes these two two-dimensional barcodes. In order to prevent such imaging, it is possible to prevent a user from imaging two or more home appliances at the same time by providing a display on the remote controller 1 and displaying an imaging screen of the camera on the remote controller 1. Two or more home appliances may be imaged at the same time.

  As described above, when two or more two-dimensional barcodes are recognized from one image data, an error code is generated, and the remote controller 1 instructs to re-input an image or sound, or two or more two-dimensional barcodes are One method is to display a selection screen of the controlled device on the display of the remote controller and select one of them. However, when two or more home appliances are arranged side by side, two or more two-dimensional barcodes are often imaged at the same time, and an error code is generated each time. It will be difficult.

  Therefore, in order to solve such a problem, when two or more two-dimensional barcodes are included in the same image data, which two-dimensional barcode belongs to the controlled device to be operated by the user. The home server 10 makes the discrimination. This is because the type of operation differs depending on the type of controlled device (TV, video, air conditioner, etc.) (for example, on a TV, channel switching, volume adjustment, etc. are types of operation, but with a video tape recorder, Select one of the two or more two-dimensional barcodes recognized simultaneously by the image recognition unit 12 according to the type of operation as a result of voice recognition, such as recording, playback, and fast-forwarding) It is what you want to do.

  For this reason, in the database unit 16, voice data for operation is classified for each of the home appliances 20, 30, and 40, and addresses of devices corresponding to the respective categories are associated and registered, and the voice recognition unit 14, when the received voice data is recognized, the voice data is checked against the voice data registered in the database unit 16. At this time, the registration voice data corresponding to the received voice data is registered together with the registration voice data. The address corresponding to the category to which the audio data belongs is read from the database unit 16.

  The target device identification unit 13 reads the type and address of the communication I / F from the two-dimensional barcode recognized by the image recognition unit 12, but the type and address of the communication I / F read from the two-dimensional barcode. Of these pairs, only the pair of the communication I / F type and address read from the two-dimensional bar code whose address matches the address fetched from the database unit 16 by the voice recognition unit 14 is sent to the control code transmission unit 17. Supply.

  Thereby, even if two or more two-dimensional barcodes are included in the same image data, the control code for the controlled device to be operated by the user can be definitely transmitted to the controlled device.

  In this way, audio data is registered in the database unit 16 for each address, and some or all types of operations are performed, such as a video tape recorder and a DVD (digital versatile disc). When two or more of the same controlled devices are used, for example, when the operation content of “play” is input by voice on the remote controller 1, in the home server 10, the voice recognizing unit 14 receives the address for the video tape recorder from the database unit 16 and the DVD Will read the address. However, since it is the same as the registered voice data of these addresses, the control code conversion unit 15 can obtain the correct control code from the database unit 16.

  By the way, in the situation where the database unit 16 is provided in the home server 10 and the controlled device of the same type, for example, a video tape recorder and a DVD are installed close to each other, these two-dimensional bars are used. The code may be imaged simultaneously with the camera. In such a case, the image recognizing unit 12 reads the addresses of the video tape recorder and the DVD, and the remote controller 1 has a voice input of the same operation content. Therefore, the addresses of the video tape recorder and the DVD are similarly read.

  In such a case, the home server 11 cannot recognize any controlled device as an operation target of the user, and transmits an error code to the remote controller 1. In this case, the error code may represent a message to the effect described above and a message for instructing re-input of voice and images from the remote controller 1, and the controlled device corresponding to these addresses, that is, a video tape recorder and the like. A selection screen for DVD is displayed on the display, and the user is allowed to designate one of the controlled devices.

  As described above, even when two-dimensional barcodes of two or more controlled devices are simultaneously imaged by the remote control camera, the control code can be transmitted to the controlled device to be operated by the user.

  Through the processing operation of each unit described above, that is, the image data and audio data input by the remote controller 1 are recognized by the home server 10, and the communication I / F appropriate for the controlled device to be operated is based on the recognition result. When a control signal or control code is sent via, it is possible to realize an operation by voice input of all controlled devices capable of communication, including existing controlled devices that are controlled by infrared rays.

  According to this embodiment, even with existing home appliances that do not have a network function, it is possible to operate home appliances by voice input using a client-server type speech recognition system. For home appliances connected via wireless or wired network, the client / server type image recognition system allows the user to take an image of the controlled device he wants to operate and input only voice operation instructions. This makes it possible to operate the controlled device to be operated. Of course, since it is a client-server type recognition system, the remote control serving as the client does not require a voice recognition unit and an image recognition unit, and therefore does not cost. Even if there are a plurality of remote controllers, the voice recognition unit and the image recognition unit need only be provided in one home server, and this can be realized without cost.

  In this embodiment, the control code generated as a result of voice recognition by the home server 10 is transmitted to the infrared I / F controlled device via the remote controller 1. By providing an infrared transmission unit, the control code may be transmitted directly from the home server 10 to the controlled device via the infrared I / F to the controlled device.

  Further, in this embodiment, the controlled device is a home appliance, but other controlled devices may be used. Further, although the infrared I / F home appliance 20, the wireless I / F home appliance 30, the wired I / F home appliance 30, the remote controller 1, and the home server 10 are shown as one unit, A plurality of all of them may be used.

  Furthermore, in this embodiment, the home server is used as the voice recognition unit and the image recognition unit, but other general servers and PCs are home appliances and electrical devices having a CPU that does not place a burden on voice recognition and image recognition. Also good.

  Furthermore, in this embodiment, in the home server, the speech recognition result is converted into a control code corresponding to this, and this control code is transmitted as control information to the remote controller or controlled device. The recognition result may be text data, which may be transmitted as control information to a remote controller or a controlled device. In this case, the text data is converted into a corresponding control code by the remote controller or controlled device.

It is a schematic block diagram which shows one Embodiment of the apparatus control system by this invention. It is a block diagram which shows one Embodiment of each apparatus in FIG. It is a flowchart which shows a specific example of the operation | movement procedure of the remote control in FIG. It is a flowchart which shows a specific example of the operation | movement procedure of the home server in FIG. 3 is a flowchart illustrating a specific example of an operation procedure of the controlled device in FIG. 1.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Remote control 2 Image input part 3 Audio | voice input part 4 Application part 5 Wireless communication part 6 Infrared transmission part 10 Home server 11 Wireless communication part 12 Image recognition part 13 Target apparatus identification part 14 Voice recognition part 15 Control code conversion part 16 Database part 17 Control code transmission unit 18 Wired communication unit 20 Home appliance with infrared I / F 21 Two-dimensional barcode 22 Infrared receiver 23 Control unit 30 Home appliance with wireless I / F 31 Two-dimensional barcode 32 Wireless communication unit 33 Control unit 40 Wired Home appliances with I / F 42 Wired communication unit 43 Control unit

Claims (11)

A portable control terminal that has voice input means and transmits voice data for controlling the controlled device input from the voice input means;
A server that receives the voice data transmitted from the portable control terminal, recognizes the voice data, and transmits the recognition result to the portable control terminal as control information;
The device control system, wherein the portable control terminal receives the control information transmitted by the server and transmits the control information to the controlled device. A portable control terminal of a device control system including a server that receives voice data for controlling a controlled device, recognizes the voice data, and forms control information of the controlled device;
Voice input means for inputting the voice data for controlling the controlled device;
Wireless communication means for wireless communication with the server;
An infrared communication means for performing infrared communication with the controlled device,
The voice data input from the voice input means is transmitted to the server by wireless communication means, control information is received from the server by the wireless communication means, and transmitted to the controlled device by the infrared communication means. A portable control terminal for controlling the controlled device. A server of a device control system for controlling the controlled device based on audio data for controlling the controlled device input by a portable control terminal,
Wireless communication means for receiving the audio data from the portable control terminal;
Registered data holding means for registering voice data input at the portable control terminal for controlling the controlled device and control information corresponding to the voice data;
Control information conversion means for recognizing the voice data received by the wireless communication means on the basis of the holding information of the registration data holding means and converting the voice data into corresponding control information;
A server, wherein the control information is transmitted to the portable control terminal by the wireless communication means. On the portable control terminal, input the audio data for controlling the controlled device,
The server receives the voice data input from the portable control terminal, recognizes the received voice data, converts it into control information of the controlled device, and transmits the control information to the portable control terminal. ,
A device control method, wherein the portable control terminal receives the control information and controls the controlled device according to the control information. A device control system that has a plurality of controlled devices with different communication methods and controls the controlled device by inputting voice data from a portable control terminal,
Each of the controlled devices is provided with a visible code for identifying itself,
The portable control terminal performs infrared communication with at least one of the controlled devices, and the audio information for controlling the controlled device to be controlled among the plurality of controlled devices and the Input the image data about the code of the controlled device to be controlled,
Recognizing the image data to determine the controlled device to be controlled, recognizing the audio data to generate control information of the controlled device to be controlled, and based on the determination result, A device control system comprising a server for transmitting information to a controlled device whose information is to be controlled by a corresponding communication means. In claim 5,
At least one of the controlled devices is a predetermined controlled device that performs infrared communication with the portable control terminal,
The server supplies the control information to the portable control terminal by wireless communication when the controlled device to be controlled is determined as the predetermined controlled device, and the portable control terminal receives the control information Is transmitted to the predetermined controlled device by infrared communication, and is controlled. A server that receives audio data for controlling a controlled device and image data for identifying the controlled device, recognizes the audio data and the image data, and forms control information for the controlled device A portable control terminal of a device control system provided with a visible code for identifying each of the controlled devices,
Voice input means for inputting the voice data for controlling the controlled device to be controlled;
Image input means for inputting the image data regarding the code of the controlled device to be controlled;
Wireless communication means for wireless communication with the server;
Infrared communication means for infrared communication with at least one of the controlled devices,
The audio data input from the audio input means and the image data input from the image input means are transmitted to the server by wireless communication means, and the control information is received from the server by the wireless communication means, A portable control terminal that is controlled by being transmitted to at least one of the controlled devices by the infrared communication means. A server of a device control system that controls a plurality of controlled devices having different communication methods based on audio data for controlling the controlled device input from a portable control terminal,
Wireless communication means for receiving the audio data and image data indicating the controlled device to be controlled from the portable control terminal;
Image recognition means for recognizing the image data and determining a controlled device to be controlled;
Registered data holding means for registering voice data input at the portable control terminal for controlling the controlled device and control information corresponding to the voice data;
Control information conversion means for recognizing the voice data received by the wireless communication means on the basis of the holding information of the registration data holding means and converting the voice data into corresponding control information;
A server characterized in that the control information is transmitted to the controlled device to be controlled determined by the image recognition means by a communication method corresponding to the controlled device. In claim 8,
At least one of the plurality of controlled devices is a predetermined controlled device that performs infrared communication with the portable control terminal,
When the image recognition means determines that the controlled device to be controlled is the predetermined controlled device, the server transmits the control information to the portable control terminal by wireless communication. Inputs audio data for controlling a controlled device to be controlled among a plurality of controlled devices with different communication methods and image data for identifying the controlled device to be controlled by a portable control terminal And
The server receives the audio data and the image data input from the portable control terminal, recognizes the received audio data, converts the control data into control information of the controlled device to be controlled, and receives Recognize the image data, determine the controlled device to be controlled, and transmit the control information to the controlled device to be controlled based on the determination result by the communication method corresponding to the controlled device A device control method. In claim 10,
At least one of the plurality of controlled devices is a predetermined controlled device that performs infrared communication with the portable control terminal,
When the image recognition means determines that the controlled device to be controlled is the predetermined controlled device, the control information is transmitted to the portable control terminal by wireless communication, and the portable control terminal is infrared A device control method comprising transmitting and controlling to the predetermined controlled device by communication.

Images