JP2004034273A - Robot and system for generating action program during utterance of robot - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To make a robot act properly in correspondent to its utterance and let a user know the start point and end point of robot's utterance. <P>SOLUTION: A plurality of action pattern data giving different values to different actions requiring different times are stored in a database 31. At an utterance time measuring section 32, the duration of an utterance of the robot is measured using audio waveform data 35a generated before the robot starts its utterance. At an action pattern combination generating section 33, two or more combinations of action patterns, and their reproduction order and timing are generated, using the contents of the database 31, so that the start and end of the utterance and that of the action of the robot coincide in accordance with the duration of the utterance. <P>COPYRIGHT: (C)2004,JPO

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to a technique for appropriately operating a robot while the robot is speaking, and for transmitting the start and end timings of speech to a user.
[0002]
[Prior art]
In the related art, during the utterance of the robot, (1) a predetermined motion is reproduced, and (2) the robot is operated according to waveform data (particularly, sound intensity) during the utterance.
[0003]
However, in the reproduction of the fixed motion of (1), since the utterance time is not constant, the start and end of the motion often do not exactly coincide with the start and end of the utterance. Has a disadvantage that it is difficult to grasp the remark timing. (2) In the robot operation according to the waveform data, since the operation pattern is a combination of the instantaneous operation with respect to the voice data, there is a disadvantage that the operation does not match the meaning of the utterance.
[0004]
[Problems to be solved by the invention]
An object of the present invention is to solve the above-mentioned drawbacks of the conventional technology, and to provide a technique capable of operating the robot appropriately while the robot speaks and transmitting the start and end timings of the speech to the user. To provide.
[0005]
[Means for Solving the Problems]
A first aspect of the present invention is an apparatus for generating an operation program during a robot utterance that solves the above-mentioned problem, comprising: a database having data of a plurality of operation patterns whose operation times are determined to be different from each other; Utterance time measuring means for measuring the length of utterance time before the start of utterance from waveform data of the generated speech, and starting and ending operations of utterance for the utterance time measured by the utterance time measuring means. An operation pattern combination generating means for generating a combination of two or more operation patterns, a reproduction order and a reproduction timing from the contents of the database so that the end coincides with the operation pattern.
[0006]
In a second aspect based on the first aspect, the operation pattern combination generating means inserts a non-operation time between adjacent operation patterns when the start and end of the operation do not coincide with the start and end of the utterance. An operation program generation device during robot utterance characterized by performing
According to a third invention, in the first invention, there is provided an operation program generation apparatus for uttering a robot, further comprising an operation type determination unit for determining an operation type from character data to be uttered.
[0007]
According to a fourth aspect, there is provided a robot having the motion program generation device for uttering a robot according to any one of the first to third aspects.
[0008]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, embodiments of the present invention will be described with reference to the drawings.
[0009]
Prior to the description of the present invention, an example of a robot to which the present invention is applied will be described with reference to FIG. The robot 1 shown in FIG. 3 simulates a human having a robot body 11 having a head 12, a chest 13, a torso (cart unit) 14, and left and right arms 15. Is rotatable by a drive mechanism (not shown), and the left and right traveling wheels 16 mounted on the body 14 can be steered and run by a drive mechanism (not shown), and move autonomously in the work space by battery driving. It is configured to The robot 1 coexists with a human in an indoor space such as a general home as a work space, and is used, for example, to assist, support, and care for a user's life in the general home. Therefore, the robot 1 has a function of talking with the user, watching the user's action, assisting the user's action, and acting together with the user by using the built-in CPU (computer) and various sensors. ing. Various shapes of the robot 1 are conceivable, such as an animal model for pets.
[0010]
Specifically, the robot 1 shown in FIG. 3 has a control unit using a CPU, two cameras 17 and two microphones 18 are mounted on the head 12, and a volume / sound source direction sensor and a focus sensor are provided at the center of the chest 13. A human detection sensor 19 composed of an electric infrared sensor and speakers 20a are mounted on the left and right. Further, an image display 20 b is mounted on the center of the chest 13, and an ultrasonic obstacle sensor 21 and a laser obstacle sensor 22 are mounted on the trunk 4. A keyboard and a touch sensor may be mounted.
[0011]
The camera 17 captures an image of the user or the room and outputs the image to the control unit, and the microphone 18 captures the user's voice and life sounds such as a telephone bell, a doorbell, and a television sound and outputs the captured image to the control unit. The sensor 19 detects the presence or absence of a user and outputs the result to the control unit. The speaker 20a is used for conversation with the user together with the microphone 18, and the image display 20b is used for providing information to the user. The keyboard and the touch sensor are data input devices for the user, and are used for inputting the life pattern data of the user and inputting the intention of the user. The speaker 20a is also used for providing information to a user. The control unit includes a calendar for a predetermined period and a calendar clock for measuring the current date and time.
[0012]
The control unit includes a data input device such as a keyboard, an external sensor that constantly monitors a user's life pattern such as a camera 17, a microphone 18, a human detection sensor 19, and obstacle sensors 21 and 22, and an internal sensor such as a calendar clock. Thus, information on the self-position of the robot 1, information on the position of the user, and information on the behavior of the user are input.
[0013]
The control unit has a self-position recognition function, a user recognition function, a database (data storage unit), a life pattern data processing function, and a drive control function.
[0014]
The self-position recognition function recognizes the position and azimuth (direction, posture) of the robot 1 itself based on image information captured by the camera 17. The user recognition function recognizes the content of the user's utterance (voice recognition) from the voice captured by the microphone 18, and performs personal recognition of the user from the image information captured by the camera 17 and the voice captured by the microphone 18. , The user's position, orientation, posture, and activity amount are recognized from the detection results of the camera 17 and the human detection sensor 19.
[0015]
The database includes voice data for conversation with the user and talking to the user, data of a gesture motion at the time of talking to the user and talking to the user, user's living information (room layout and furniture related to the user's residence, etc.). Location, personal daily life pattern of the user, user's hobby, health condition, etc.). A keyword such as a character string, video, or audio is added to the user's life information as needed. As the life pattern data, events related to activities such as getting up, going to bed, eating, taking medicine, taking a walk, free time, and taking a bath are listed, and these events are stored as time schedules together with time data.
[0016]
The life pattern data processing function stores life information directly input by a user from a data input device in a database with a keyword attached thereto, and various types of user recognition such as voice recognition results obtained by a camera 17, a microphone 18, and a human detection sensor 19. By processing the information, each piece of life information is attached to the database with a keyword and its date and time. Further, the life pattern data processing function selects data corresponding to the command from the database in response to an instruction or speech from the user, or according to time, and issues a command corresponding to the selected data to the operation control unit. Give and control.
[0017]
The drive control function drives the running wheels 16 to execute running and steering of the robot 1 in accordance with a command from the life pattern data processing function, and drives the head 12 and the arm 13 to rotate the head 12 and the arm 13. Perform the action. Further, the speaker 20a and the image display 20b are driven to execute a conversation with the user.
[0018]
Next, with reference to FIG. 1 and FIG. 2, an operation program generation device during robot utterance (hereinafter, simply referred to as an operation program generation device) according to an embodiment of the present invention will be described.
[0019]
The above-described robot 1 has an operation program generation device illustrated in FIG. 1, and the operation program generation device includes a database 31, a speech time measurement unit 32, and an operation pattern combination generation unit 33. It is.
[0020]
In FIG. 1, reference numeral 34 denotes a speech synthesizer provided in the robot, which comprises a speech waveform generation unit 35 and a waveform reproduction unit 36. The voice waveform generator 35 receives character data to be uttered and generates corresponding voice waveform data 35a before the start of utterance, and the waveform reproducer 36 reproduces the waveform data 35a into a real-time voice waveform. . The reproduced voice waveform is amplified by the amplifier 37 and provided to the speaker 20a, whereby the robot speaks.
[0021]
Here, if the robot suddenly starts talking, it is difficult for the user to hear. Therefore, character data such as "A-" or "Hmm" is inserted at the beginning of the character data. Thereby, the robot utters the meaningless words such as "A-" or "Well" and then utters the original utterance, thereby attracting the user's attention and making it easier to hear.
[0022]
The database 31 stores data of a plurality of operation patterns whose operation times are different from each other. The motion pattern is a pattern of motion such as a gesture of the head 12, the arm 15, and the body 14, and in the example, a motion pattern for one second as a type of motion that the robot speaks to the user, An operation pattern for 2 seconds, an operation pattern for 4 seconds, and an operation pattern for 8 seconds are stored. In addition, a motion pattern for one second and a motion pattern for two seconds are stored as types of motions that the robot asks the user. Further, a motion pattern for one second and a motion pattern for two seconds are stored as types of motions confirmed by the robot.
[0023]
The utterance time measuring unit 32 measures the length of the utterance time before the utterance starts from the waveform data 35a generated before the utterance of the robot by the voice waveform generation unit 35 or the like. Since the waveform data 35a itself is data before being reproduced as a real-time audio waveform, the utterance time can be instantaneously measured by checking the data length (the number of bytes) of the waveform data 35a.
[0024]
The motion pattern combination generation unit 33 performs a process on the speech time measured by the speech time measurement unit 32 such that the start and end of the robot's speech coincide with the start and end of the robot's speech. A combination of two or more operation patterns is selected, and a reproduction order and a reproduction timing thereof are generated.
[0025]
When the start and end of the operation do not exactly match the start and end of the utterance, the operation pattern combination generation unit 33 inserts a non-operation time between adjacent operation patterns before and after.
[0026]
It is preferable that the plurality of operation patterns selected by the operation pattern combination generation unit 33 are suitable for the utterance content as much as possible. For this purpose, the operation type selection signal 38a is used.
[0027]
In this example, the action type determining unit 38 analyzes the character data to be uttered, thereby determining the action type such as question, talk, and confirmation, and obtaining the action type selection signal 38a.
[0028]
The operation control device 39 reproduces the plurality of operation patterns selected by the operation pattern combination generation unit 33 in the reproduction order and the reproduction timing designated by the operation pattern combination generation unit 33, and reproduces the head 12, the arm 15, and the trunk 14 (Actuator) 40 is actually operated.
[0029]
FIG. 2 shows an example of a procedure for combining operation patterns.
[0030]
In FIG. 2, it is assumed that the utterance time is 14.5 seconds and the motion type is a question from the robot to the user based on the length of the waveform data 35a.
[0031]
Therefore, the motion pattern combination generation unit 33 first determines a motion pattern (questioning motion for two seconds) for a question that requires two seconds as the last motion to be performed based on the motion type = question. Next, as an operation to be performed first, an operation pattern of a speaking movement for 8 seconds and an operation pattern of a speaking movement for 4 seconds are combined to fill a total of 14 seconds. For the remaining 0.5 seconds, a non-operation time of 0.5 seconds is inserted between the operation pattern of the speaking movement for 4 seconds and the operation pattern of the interrogating movement for 2 seconds.
[0032]
Thereby, the start and end of the operation coincide exactly with the start and end of the utterance, and while the robot 1 is uttering, the robot 1 operates appropriately and informs the user of the start and end timing of the utterance. be able to.
[0033]
【The invention's effect】
The first invention is based on a database having data of a plurality of operation patterns whose operation times are determined to be different from each other, and determines the length of the utterance time from the waveform data of the voice generated before the robot starts uttering. The utterance time measuring means to measure before and the utterance time measured by the utterance time measuring means, two or more from the contents of the database so that the start and end of the operation coincide with the start and end of the utterance. And a motion pattern combination generating means for generating the combination of the motion patterns, the reproduction order and the reproduction timing, so that the robot can be appropriately operated in accordance with the utterance and the start and end timings of the utterance can be transmitted to the user. .
[0034]
According to the second invention, when the start and end of the operation do not coincide with the start and end of the utterance, the operation pattern combination generation unit inserts a non-operation time between adjacent front and rear operation patterns. The start and end of the operation exactly match the end.
[0035]
Since the third invention includes the operation type determining means for determining the operation type from the character data to be uttered, the robot can perform an operation suitable for the content of the utterance.
[0036]
In the fourth invention, since the robot has the operation program generating device during the robot utterance according to any one of the first invention to the third invention, the robot operates appropriately in accordance with the utterance and gives the user a start and end timing of the utterance. Can be conveyed.
[Brief description of the drawings]
FIG. 1 is a diagram showing a configuration of an operation program generation device during robot utterance according to an embodiment of the present invention.
FIG. 2 is a diagram showing an example of a procedure for combining operation patterns.
FIG. 3 is a diagram illustrating an example of a robot.
[Explanation of symbols]
1 Robot 12 Head 13 Chest 14 Torso (trolley)
20a speaker 31 database 32 utterance time measurement unit 33 operation pattern combination generation unit 34 audio synthesis device 35 audio waveform generation unit 35a audio waveform data 36 waveform reproduction unit 37 amplifier 38 operation type determination unit 38a operation type selection signal 39 operation control unit 40 Actuator

Claims (4)

An utterance that measures the length of utterance time before the start of utterance from a database having data of a plurality of operation patterns whose operation times are different from each other, and waveform data of speech generated before the robot starts uttering A combination of two or more operation patterns from the contents of the database so that the start and end of the utterance coincide with the utterance time measured by the utterance time measurement unit. And a motion pattern combination generating means for generating a reproduction order and a reproduction timing. 2. The method according to claim 1, wherein when the start and end of the utterance do not coincide with each other, the operation pattern combination generation unit inserts a non-operation time between adjacent front and rear operation patterns. A motion program generation device during robot utterance. 2. The motion program generation device according to claim 1, further comprising an operation type determination unit configured to determine an operation type from character data to be uttered. A robot comprising the motion program generation device during robot utterance according to any one of claims 1 to 3.

Images