KR101234289B1 - Congitive system realizing contextual human robot interaction for service robot and method thereof - Google Patents

Abstract

A cognitive system capable of contextual interaction for a service robot is disclosed. The system expresses the sensing unit for sensing external information of the robot, the acting unit for acting outside of the robot, the storage unit for storing the robot's memory, and the information detected by the sensing unit in sentences. And an inference management unit for inferring the memory stored in the storage unit and acting outside through the acting unit according to the inference result. This enables the contextual interaction of service robots.

Description

Cognitive system realizing contextual human robot interaction for service robot and method

The present invention relates to a technology related to a robot, and more particularly, to a cognitive technology for allowing a robot to communicate with a human.

With the development of robot technology, there is a growing interest in service robots where humans and robots share their daily lives in the living space. In the meantime, research on grounding has been conducted in terms of human robot interface. In the case of Roy, visual object information obtained by robots is modeled through mental imagery and The conversation was made possible. Siskind described the movement of objects and hands in the video as linguistic expressions with mechanical meaning. These studies have not suggested a sentence-based basic model that can handle both temporal and spatial situations at the same time, and there are no studies on storing and reproducing the basic information as memory.

On the other hand, in the field of cognitive science, research on the cognitive architecture that models and connects with the structure of the human mind is being conducted.In the case of ACT-R at Carnegie Mellon University and SOAR at the University of Michigan, human cognition is understood. And simulations. These studies primarily aim to mimic human cognitive function as similarly as possible, and do not fully consider the issue of efficiency when applied to robots. Therefore, it is difficult to use it as a robot's cognitive system, and the invention about the context-based interaction between humans and robots has not been achieved.

In order to become a service robot that shares human life with everyday life, it is necessary to share experiences with humans and interact based on time and space context. Robot-human interaction requires functions such as sensing from the outside, acting outward, memory of cognitive information, and contextual reproduction. To do this, it is necessary to describe and store the cognitive information in an appropriate form. In addition, in order to utilize the language for mutual communication, the robot needs the ability to understand the human language and speak as needed. These functions necessarily require combining or linking language and cognitive information, and can derive information for contextual interaction. As such, there is a need for a cognitive system that combines verbal expressions of cognitive information such as sensing, behavior, and memory with cognitive information.

An object of the present invention is to provide a technical solution that enables a human-robot interaction based on the context.

A cognitive system capable of contextual interaction for a service robot according to an aspect of the present invention for achieving the above technical problem includes a sensing unit for sensing external information of the robot, an acting unit for acting outside of the robot, A memory for storing the memory of the robot, and the information detected by the sensor is expressed in a sentence and stored in the memory, and the memory stored in the memory is inferred and the action is based on the result of the inference. It includes an inference management unit that causes an outside act.

On the other hand, the cognitive method for the contextual interaction of the service robot according to an aspect of the present invention for achieving the above technical problem is a step of interpreting the sentence, the speech is converted by the listening module of the robot, the interpreted sentence Storing in the storage unit of the robot, inferring contents of the interpreted sentence, and executing an action on the inference result.

According to another aspect of the present invention, there is provided a cognitive method for contextual interaction of a service robot, in which information about the same object is detected in a camera image input to the robot. Determining whether the object descriptor is registered with the object descriptor; if the determination result is not registered, newly registering information on the recognized object with the object descriptor; and generating a sentence for the new object recognition event to generate the sentence in the storage unit. Storing.

The present invention creates the effect of enabling the contextual interaction of the service robot through the function of expressing and remembering all events in sentences and reproducing them if necessary. In addition, the present invention can further enhance the contextual interaction of the service robot by providing auxiliary cognitive information such as spatial information as well as verbal expression.

1 is a conceptual diagram of a cognitive system according to an embodiment of the present invention.
2 is a block diagram of a cognitive system in accordance with an embodiment of the present invention.
3 is an interpreter block diagram in accordance with an embodiment of the present invention.
Figure 4 is a spatial inference example of the spatial inference according to an embodiment of the present invention.
5 is a flow chart of a cognitive method for events occurring in a visual module of a cognitive system in accordance with an embodiment of the present invention.
6 is a flow diagram of a cognitive method for events occurring in a listening module of a cognitive system in accordance with an embodiment of the present invention.
7 is a flow diagram of a cognitive method for an event occurring in an action module of a cognitive system in accordance with one embodiment of the present invention.
8 is a flow diagram of a cognitive method for events occurring in a speech module of a cognitive system in accordance with an embodiment of the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS The foregoing and further aspects of the present invention will become more apparent from the following detailed description of preferred embodiments with reference to the accompanying drawings. Hereinafter, the present invention will be described in detail to enable those skilled in the art to easily understand and reproduce the present invention.

1 is a conceptual diagram of a cognitive system according to an embodiment of the present invention.

1 is a conceptual diagram of a cognitive system to be equipped with a robot for interaction with a human, and shows a state in which cognitive information is connected to sentences. The information sensed or listened from the outside is expressed in sentences, stored or inferred, and executed based on sentences even when acting or speaking outward. That is, it connects cognitive information and language, stores sentences as basic units in memory, infers and plays them when necessary. However, only the sentences stored in the memory at the time of inference have limitations, and it is difficult to reason about cognitive information that is difficult to express in sentences because the memory is composed only of a series of sentences. Therefore, the cognitive system according to the present invention enables more effective reasoning by using additional cognitive information such as visual as an auxiliary means based on sentences. Therefore, the process of linking the verbal expression of cognitive information and auxiliary cognitive information is required.

On the other hand, according to the method as shown in Figure 1 has the following advantages. First, since the dialogue itself is a linguistic expression, the linguistic expression of the cognitive information can express the speech or listening information as it is, so there is no need to code it in another form. Second, internal reasoning as well as sensing information and behavior information detected by the robot can be expressed in a monologue language. Third, the cognitive information can be stored temporally and reproduced when necessary through the sequential storage including the time tag of the sentence. Fourth, a sentence is the smallest unit that can express the motion of a robot or an object. It is semantically complete and can be a basic unit of events.

2 is a block diagram of a cognitive system according to an embodiment of the present invention.

As shown, the cognitive system includes a sensing unit 100, an acting unit 200, a storage unit 300, and an inference management unit 400. The sensing unit 100 is a component for sensing external information of the robot. In one embodiment, the detector 100 includes at least one of the visual module 110, the sensor module 130, and the listening module 120. The visual module 110 is a component connected to the camera to acquire an image input to the camera. The listening module 120 is a component connected to a microphone to obtain an external voice input to the microphone. The listening module 120 may include a speech to text (STT) function for converting a microphone input voice into a sentence. The sensor module 130 is a component for acquiring data sensed from one or more sensors. A plurality of different purpose sensors may be connected to the sensor module 130, and the same sensor may be several. For example, the sensor may be a tactile sensor, a temperature sensor, a humidity sensor, or the like. The sensor 100 may detect an external situation of the robot through the visual module 110, the listening module 120, and the sensor module 130.

The acting unit 200 includes an operation module 210 and a speech module 220. The operation module 210 is a module for driving a plurality of actuators for the operation of the robot. Through the motion module 210, the movement of the robot's arms and legs as well as the facial expressions for expressing emotions may be possible. The ignition module 220 is configured to ignite the robot and is connected to the speaker. Preferably, the speech module 220 includes a text to speech (TTS) function for converting sentences into speech.

The storage unit 300 is a configuration for storing the memory of the robot. This storage unit 300 includes one or more memories. The memory is preferably a nonvolatile memory, and may be a flash memory. The storage unit 300 includes a sentence storage unit 310, an object descriptor 320, and an operation descriptor 330. The sentence storage unit 310, the object descriptor 320, and the operation descriptor 330 may be embodied in physically separated memories. Alternatively, the sentence storage unit 310, the object descriptor 320, and the operation descriptor 330 may be physically stored in one memory and logically divided. The sentence storage unit 310 is represented by a sentence related to the robot and stored. The object descriptor 320 stores the information when the posture of the new object or the object is changed by the visual module 110, and the motion descriptor 330 stores information about the operation of the robot and its operation sequence.

The inference management unit 400 expresses the information detected by the detection unit 100 in a sentence and stores the information in the storage unit 300, infers the memory stored in the storage unit 300, and acts according to the inference result 200. As a configuration for performing a role of acting externally through), it may be implemented as a computer processor. The reasoning management unit 400 includes an interpreter 410 for expressing all the information transmitted from the sensing unit 100 in a sentence and acts outside the robot, and knowledge for inferring knowledge about the occurrence of the event and information. It includes a reasoner 430, and includes a spatial reasoner 420 to represent the spatial reasoning and the current spatial situation.

The interpreter 410 expresses all the information generated by the sensing unit 100 or the acting unit 200, that is, an event, in a sentence, and attaches a time tag together so that it knows when the event occurred. The sentence storage unit 310 to store. This enables the robot's contextual interaction with humans. Table 1 shows an example in which the information input through the visual module 110 is expressed in sentences and stored in the sentence storage unit 310.

number time module sentence One Year month date hour minute second Time (S (NP_AGENT Tom) (VP gave (NP_THEME an apple)) (PP_BENEFICIARY to (NP Peter)) (PP_TIME at (NP 6))).) 2 ... ... ... 3 ... ... ...

For storing the sentences as shown in Table 1, the interpreter 410 is a visual interpreter 411 for interpreting the information input through the visual module 110, so that the information input through the visual module 110 can be represented in a sentence. It includes. In addition, the analyzer 410 further includes a sensor analyzer 412 to interpret information input through the sensor module 130, for example, tactile information, temperature information, humidity information, and the like. Also, the interpreter 410 further includes a sentence generator 413 for expressing the contents analyzed by the visual analyzer 411 and the sensor interpreter 412 into sentences.

When the contents of the camera image are interpreted by the visual analyzer 411, the sentence generator 413 generates the interpreted contents as sentences. The time interpreter 411 and the sentence generator 413 express time information in sentences. The sentence may be stored in the sentence storage unit 310 as it is, but the result interpreted through the syntax analysis and the semantic analysis may be stored in the sentence storage unit 310. To this end, the sentence parser 414 includes a syntactic parser 414A and a semantic parser 414B. Here, the software provided by natural language processing may be used as the syntax interpreter 414A and the semantic interpreter 414B.

For example, the parser 414A interprets a sentence according to the rule of Penn Treebank. Table 2 shows the tag sets of the Penn Treebank. The syntax interpreter 414A automatically determines the type of tag set for a phrase of a sentence to indicate the type of phrase.

One ADJP Adjective phrase 2 ADVP Adverb phrase 3 NP Noun phrase 4 PP Prepositional phrase 5 S Simple declarative clause 6 SBAR Clause introduced by subordinating conjunction or 0 7 SBAEQ Direct question introduced by wh-word or wh-phrase 8 SINV Declarative sentence with subject-aux inversion 9 SQ Subconstituent of SBARQ excluding wh-word or wh-phrase 10 VP Verb phrase 11 WHADVP wh-adverb phrase 12 WHNP wh-noun phrase 13 WHPP wh-propositional phrase 14 X Constituent of unknown or certain category Null elements One * "Understood" subject of infinitive or imperative 2 0 Zero variant of that in subordinate clause 3 T Trace-marks position where moved wh-constituent is interpreted 4 NIL Marks position where preposition is interpreted in pied-piping contexts

The semantic interpreter 414B functions to give a semantic range of each argument to a phrase unit of a sentence. According to the field of natural language processing research, the semantic domain is largely composed of arguments and additional terms. Table 3 below shows the types of arguments and supplementary tags.

Dispute tag One Agent Intentionally causing the behavior of the predicate to describe 2 Subject Experiencing change due to the behavior of the predicate 3 Experiencer Experiencing the mental and psychological state described by the predicate 4 Instrument A tool used in the actions predicates describe 5 Location Where the action described by the predicate is directed 6 Goal Where the action described by the predicate is directed 7 Source Where the object has moved in the behavior described by the predicate 8 Benefactive Receiving benefits from the actions described by the predicate 9 Causer Causing an event 10 Cauesee What happens by the minister Supplementary tag One Time When did the action take place 2 Manner Method of act 3 Reason Reason of act 4 Purpose Purpose of the act 5 Location The place where the act took place 6 Direction Direction of action

An example of the syntax interpreter 414A and the semantic interpreter 414B will be described as follows.

Tom gave an apple to Peter at 6

The syntax interpreter 414A is used to interpret phrases centering on verbs in sentences.

(S (NP Tom) (VP gave (NP an apple) (PP to (NP peter)) (PP at (NP 6))).

Passing it again through the meaning interpreter 414B gives the meaning of the phrase in the sentence.

(S (NP_AGENT Tom) (VP gave (NP_THEME an apple) (PP_BENEFICIARY to (NP Peter)) (PP_TIME at (NP 6))).)

Sentences stored in the sentence storage unit 310 is stored through the syntax analysis and semantic analysis as described above, alternatively, it is also possible to store the general sentence as it is, and to process the syntax analysis and semantic analysis if necessary.

On the other hand, the space reasoner 420 represents the arrangement of objects with respect to the space in which the robot exists in the world coordinate system, and when necessary recalls the past events, and functions to infer about the spatial situation of the past memory. 4 shows an example of a spatial reasoner. The spatial reasoner 420 indicates the current spatial situation in the visual module 110 by recognizing the shape of an object from a camera connected with the visual module 110 and displaying the result on the spatial reasoner 420. do. The spatial inference unit 420 configures the space of the world coordinate system with respect to the current state based on the previous visual recognition stored in the storage unit 300. Here, the object is distinguished from the image input by the camera, and the shape and posture of the object are found. Here, the posture represents the position and angle of the object in the world coordinate system, which is expressed in two dimensions (x, t, θ), and in three dimensions, consisting of position information and angle information (x, y, z, α, β, γ). When the spatial inference device 420 infers the memory of the past, the shape descriptor and the attitude information of the object descriptor 320 are searched to create the memory space of the past, and the front, rear, left, right, etc. based on the center position of the object. Spatial information is identified.

The knowledge reasoner 430 is a function of inferring knowledge related to the knowledge generated inside the cognitive system and has the following functions. For example, when an external human is asked to regenerate knowledge, when an external human query the robot about past memories, the human voice is converted into a sentence by the listening module 120 of the sensor 100. Then, the syntax interpreter 414A and the semantic interpreter 414B of the interpreter 410 undergo syntax analysis and semantic analysis. The contents of the query resulting from the semantic analysis are retrieved through the memory unit 300 by the knowledge inference unit 430 and, if necessary, are uttered by the speech module 220 or connected to the operation through the operation module 210. do. Therefore, the knowledge reasoner 430 becomes a calculation rule of IF-THEN type, the content corresponding to IF becomes conditional, and the content corresponding to THEN becomes a result of inference. For example, consider the following query.

When did Tom go to the back of Peter

The operation of the knowledge reasoner 430 calls a function as the result of the condition in the conditional statement and the result of the condition in the result statement.

IF (WHADVP TIME) (VP go) (TENSE PAST) (1PP1 to) (LOCATION the back of Peter);

THEN CALL search (TIME, AGENT, 1PP1, LOCATION);

The reasoning management unit 400 has a result inferred by at least one of the knowledge reasoner 430 and the spatial reasoner 420 and is generated as a sentence by the sentence generator 413 to be converted into a voice through the speech module 220. After the output to the speaker, or by combining the operation of the robot described in the operation descriptor 330 and its operation order to the action module to drive the robot's actuators to operate the robot. Alternatively, the inference manager 400 may perform both the act of speaking and acting according to the result of the inference.

On the other hand, the object descriptor 320 will be described using Table 4 as follows.

object name shape Characteristic Present posture O1 Tom

Blue x1, y1, θ1 O2 Peter

Red x1, y2, θ2 H1 Kim

Male x3, y3, θ3 H2 Park

Female x4, y4, θ4

The object descriptor 320 is a data space for storing data about an object. The object descriptor 320 stores a shape and a posture of an object recognized by the visual module 110, stores a feature of the object, and the recent posture known by a cognitive system. Save it.

The operation descriptor 330 will be described with reference to Table 5 below.

number sentence function S1 I found at x, y, θ find (X) S2 I approach X approach (X) S3 I open hand openHand () S4 I reach to grip position of X reachGripPosition (X) S5 I grip X B (grip X) S6 I bend arm B (bend arm)

The motion descriptor 330 is a set of motions that are the basis for the motion of the robot. Table 5 shows the actions taken when the robot receives a command to pick up an object, that is, a “hold X” command. Here each sentence is a basic action, and a combination of them makes a complete one meaningful action.

5 is a flowchart illustrating a method for an event occurring in a visual module of a cognitive system according to an embodiment of the present invention.

The visual module 110 continuously receives an image from a camera and recognizes an object inside the image when a change occurs in the input camera image (S510). Recognizing an object here means recognizing the position and rotation of the object and the shape of the object. When the recognition is completed, the inference management unit 400 compares the recognized object with the object data of the object descriptor 320 (S530). The inference management unit 400 determines whether the same object among the objects described in the object descriptor 320 exists with respect to the object recognized through the comparison (S540). If the same object exists, the inference management unit 400 compares the recognized object and the posture of the same object to determine whether the same posture (S550). If it is not the same posture, the reasoning management unit 400 changes the posture information to the object descriptor 320 (S560). On the other hand, if the same object does not exist, the reasoning management unit 400 registers it in the object descriptor 320 because the recognized object is a new contact with the robot (S570). In addition, when the object descriptor 320 is changed, since a new event occurs on the time module, the sentence is expressed as a sentence in the sentence storage unit 310 and registered (S580) (S590). At this time, the reasoning management unit 400 uses the visual interpreter 411 to express the event in a sentence.

The method for registering a sentence in the sentence storage unit 310 will be described in detail. When the visual module 110 detects the shape and posture information of objects, the objects stored in the object descriptor 320 in the visual interpreter 411. To determine if the event has changed. If the variation of the object is made, the sentence generator 413 makes a sentence about the event and stores it in the sentence storage unit 310. The event is divided into two main categories: the appearance of new objects and the change of attitude of stored objects. If a new object first appeared

A new object appeared at x1, y1, θ1

It can be expressed as. Here, 'A new object' refers to an arbitrary new object, 'appeared' serves as a verb indicating that a new object has appeared in the current scene on the spatial reasoner 420, and 'at' refers to the posture of the object. Indicates. Here, 'A new object', 'appeared' and 'at' are automatically generated by the sentence generator, and the value of (x1, y1, θ1) corresponds to the value recognized by the listening module.

The sensor module represents all sensors other than the human voice attached to the robot, and in the case of sound, the natural language recognition is the sensor of the listening module, the external noise or sound, and the voice intensity or mood appearing in the human voice. This is the module. The process of recognizing and interpreting the sensor module is similar to that of the visual module. Since the sensor uses the sensed information from the outside, the sensor interprets the information sensed from the outside and generates the result as a sentence. The sensed information from the sensor is largely the type of the sensor, the intensity of the sensed information, the temporal continuity of the sensing information and the characteristics of the signal over time. Such sensor information is recognized by the sensor module 120, the contents of which define the meaning in the sensor interpreter and generate the sentence in the sentence generator. For example, if a light tapping situation occurs in the tactile sensor,

Someone touches me.

If you hit hard with the tactile sensor,

Someone beats me.

Is expressed as a sentence. Here, "Someone" means an agent that has not been identified to date, and the agent may be a human or a robot animal. 'me' indicates that the target is the robot itself, not another agent. 'Touches' and 'beats' are determined by the interpretation of the sensor analyzer, which is determined when the intensity of the information of the tactile sensor detected by the sensor analyzer exceeds or exceeds the predefined criteria.

6 is a flowchart illustrating a method for an event occurring in a listening module of a cognitive system according to an embodiment of the present invention.

The listening module 120 receives a voice through a microphone (S610). The listening module 120 converts the input voice into a sentence through the built-in voice recognition function (S620). The sentence interpreter 414 of the reasoning management unit 400 is used to interpret the meaning of the converted sentence. Preferably, a syntactic analysis and a semantic interpretation process are performed (S630). The inference management unit 400 stores the sentence in which the syntactic analysis and the semantic analysis are completed in the sentence storage unit 310 (S640). At this time, the reasoning management unit 400 attaches the time tag together and stores the sentence in the sentence storage unit 310 so as to know when an event for the sentence occurred. The reasoning management unit 400 interprets the meaning of a sentence using at least one of the spatial reasoning machine 420 and the knowledge reasoning machine 430, and infers the meaning of each argument according to the meaning of each argument (S650). When the inference is completed, the inference management unit 400 outputs the inference result to the operation module 210 and / or the speech module 220 to execute the action on the inference result (S660).

7 is a flowchart illustrating a method for an event occurring in an action module of a cognitive system according to an embodiment of the present invention.

The reasoning management unit 400 issues an action command according to a generation rule using the knowledge reasoner 430, and as a result, searches for an operation in the operation descriptor 330 (S710). The reasoning management unit 400 drives the actuator through the operation module 210 according to the search result and executes the operation (S730).

8 is a flowchart illustrating a method for an event occurring in a speech module of a cognitive system according to an embodiment of the present invention.

If a new object is found, the reasoning management unit 400 confirms that the label of the object does not exist in the object descriptor 320 (S810). The inference management unit 400 confirms that there is no presence, and then causes the knowledge inference unit 430 to write a spoken sentence, and then converts the written sentence into a voice in the speech module 220 and outputs it (S820) (S830). ).

A detailed example of the operation module and the speech module corresponding to the acting unit together with FIG. 7 is as follows. The operation module functions to execute an operation made by an actuator such as a motor of the robot when the operation of the robot is requested by the inference management unit. For example, let's say that an external human is given the following command through a listening module.

Hold the cup.

The listening module makes the sentence and stores the sentence interpreted through the sentence interpreter 414 in the sentence storage unit 310. At the same time, the sentence interpreter 414 interprets the content and delivers it to the knowledge reasoner. As described above, the knowledge reasoner 430 is inferred in the IF-THEN form for a certain situation.

IF (VP_IMPERATIVE hold) (OBJ the cup)

THEN CALL hold (the cup)

By the result of inference, the action function called hold (the cup) is called. Accordingly, when the 'hold X' is present in the motion descriptor as shown in Table 5, the process of picking the cup is performed by calling the functions of the basic operations included in the 'hold X'.

Next, for example, the case corresponding to the speech module in conjunction with FIG. The speech module refers to the output of voice to the outside at the request of the inference management unit. When the speech module is operated, it is largely made by a request inside the cognitive system and a request from the outside world. In the case of an internal request, for example, as described above, when a new object appears, information about this is stored in the object descriptor. However, there is no label of an object in a label slot of the stored object descriptor 320. Therefore, you cannot talk to humans with this object. Therefore, when a new object is found, the human is immediately queried for the name of the object. In this process, the knowledge reasoner 430 checks the state of the object descriptor 320 at predetermined time intervals, and if there is no name, a sentence asking for a name is generated.

IF label (Obj) == 0;

THEN CALL sentence (Obj);

For example, if there is a round object of orange color, the knowledge reasoner generates the following sentence using the function 'sentence (Obj)' through the sentence generator.

What is the new orange object

Next, the case is ignited by the demands of the outside world, mainly the answer to external human questions. For example, if the following sentence is input to the listening module from the outside world,

When did Ele go to the back of Pupy

The meaning of this sentence is to request the reasoning of past events through the spatial reasoner and to output the result to the speech module. 'When' asks the time of occurrence of the event, 'go' indicates the event that the object called 'Ele' has moved, 'to' indicates the direction the object is facing, and indicates the target location of the "the back of Pupy". will be. On the other hand, the spatial inference machine uses the sentence memory and the information of the object descriptor to infer the interrelation of objects while moving the viewpoint from the present to the past. By comparing 'Pupy' defined in sentence memory with past positional information, we find the case where 'Ele' moves backward from the central position of 'Pupy' and when the event occurred. After finding, the event is made in the sentence generator with the following sentence and output in words through the speech module.

Ele went to the to the back of Pupy at 10:51

An example of a sentence storage unit representing such a series of processes is as follows. Each sentence represents only a sentence without an interpreter.

number time module sentence One 9-10: 40: 15 Time A new object appeared at 129,51, -8.1 2 9-10: 40: 18 Fire what is the new orange object 3 9-10: 40: 29 listening It is Pupy 4 9-10: 40: 49 Time A new object appeared at 77,72, -149.0 5 9-10: 40: 52 Fire What is the new green object. 6 9-10: 41: 1 listening It is Raby 7 9-10: 41: 27 Time A new object appeared at 146,141, -171.0 8 9-10: 41: 30 Fire what is the new blue object 9 9-10: 41: 46 listening It is Caty 10 9-10: 41: 57 Time A new object appeared 210,49, -118.6 11 9-10: 42: 01 Fire What is the new red object 12 9-10: 42: 37 listening It is Ele 13 9-10: 43: 56 Time Ele went to 170, -8, -63.4 14 9-10: 46: 29 Time Caty disappeared 15 9-10: 51: 55 Time Ele went to 123,142, -131.8 16 9-10: 52: 8 Time Caty reappeared at 201,30, -163.4 17 9-10: 54: 4 listening When did Ele go to the back of Pupy 18 9-10: 54: 9 Fire Ele went to the to the back of Pupy at 10:51 19 9-10: 56: 22 listening What did appear last 20 9-10: 56: 27 Fire Ele appeared last 21 9-10: 57: 47 listening What is foremost object 22 9-10: 52: 51 Fire Caty is foremost object

So far I looked at the center of the preferred embodiment for the present invention.

It will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims. Therefore, the disclosed embodiments should be considered in an illustrative rather than a restrictive sense. The scope of the present invention is defined by the appended claims rather than by the foregoing description, and all differences within the scope of equivalents thereof should be construed as being included in the present invention.

100: detector 110: visual module
120: listening module 130: sensor module
200: action unit 210: operation module
220: ignition module 300: storage unit
310: sentence storage unit 320: object descriptor
330: motion engineer 400: inference management
410: interpreter 411: visual analyzer
412 sensor interpreter 413 sentence generator
414: Parsing Interpreter 414A: Parsing Interpreter
414B: Semantic Interpreter 420: Spatial Reasoner
430: Knowledge Reasoner

Claims (22)

delete delete delete delete delete delete delete delete delete delete A sensing unit for sensing external information of the robot;
An acting unit for acting outside of the robot;
A storage unit for storing the memory of the robot;
A reasoning management unit expressing the information sensed by the sensing unit in a sentence and storing the information in the storage unit, inferring the memory stored in the storage unit, and acting externally through the acting unit according to the inference result;
The reasoning management unit stores the sentence in the storage unit along with the time tag for the contextual memory of the robot,
The detection unit and a visual module for obtaining a camera image;
A listening module for obtaining an external voice,
The sensing unit includes a sensor module for obtaining one or more sensing data,
The acting unit is an operation module for driving an actuator for the operation of the robot,
It has a speech module for converting the spoken sentence into a voice for outputting the robot to output a speaker,
The memory unit is a sentence storage unit for storing the sentence,
An object descriptor for storing object information obtained by the visual module;
An operation descriptor for storing information about the operation of the robot and its operation sequence;
The reasoning management unit interprets the information sensed through the sensing unit to express the sentence, and an interpreter for performing an action by controlling the acting unit according to the inference result;
A knowledge reasoner which infers the meaning-interpreted sentence through the storage unit;
And a spatial inference device for classifying objects in the camera image obtained by the visual module, grasping the shape and posture of the separated objects, and updating the object descriptors.
The parser includes a sentence parser including a syntactic parser that parses the sentence in phrase units and a semantic parser that interprets the meaning of the phrase of the parsed sentence. Cognitive system with contextual interaction for robots. The method of claim 11,
When the spatial inference machine deduces the past spatial memory of the robot, the shape descriptor and the posture information of the object descriptor are searched to create the past memory space and the spatial information is deduced from the front, rear, left, and right based on the center position of the object. Contextually cognitive system for service robots. The method of claim 12,
The reasoning management unit generates a sentence based on a result inferred by at least one of the knowledge reasoner and the spatial reasoner and outputs the sentence to the speech module or combines the motions and the sequence of operations of the robot described in the motion descriptor. A contextually interactive cognitive system for service robots characterized by modular output. delete delete delete Interpreting the sentence in which the voice is converted by the listening module of the robot;
Storing the interpreted sentence in a memory of the robot;
Infer contents of the interpreted sentence;
Executing an action on the inference result;
The interpreting step may include: parsing the sentence in which the speech is converted into phrases;
Interpreting the meaning of the phrase in the parsed sentence,
The storing step stores a sentence with a time tag for the contextual memory of the robot,
The reasoning step is a context of a service robot, characterized by inferring the contents of the interpreted sentence by searching the memory of the robot, in which an object descriptor storing information about objects acquired through a sentence and a camera image stored by time is stored. Cognitive method for enemy interaction. 18. The method of claim 17,
In the deduction step, when inferring the past spatial memory of the robot according to the contents of the interpreted sentence, the object descriptor searches for shape information and posture information of the object to create a past memory space and refers to the center position of the object. Cognitive method for contextual interaction of service robot characterized by inferring front, rear, left and right spatial information. 19. The method of claim 18,
The executing step may generate a sentence according to the inference result and output the sentence to the utterance module of the robot or combine the operation of the robot described in the operation descriptor of the storage unit and the sequence of the operation to the behavior module of the robot. Cognitive method for contextual interaction of service robots. delete delete Determining whether information about the same object is registered in the object descriptor of the storage unit of the robot with respect to the object recognized in the camera image input to the robot;
If it is not registered as a result of the determination, newly registering information about the recognized object with the object descriptor;
Generating a sentence for the new object recognition event and storing the sentence in the storage unit;
The information about the object registered to the object descriptor includes information about the shape and posture of the object,
If it is registered as a result of the determination, determining whether the recognized posture of the object is the same as the posture of the same object,
If the attitude is not the same, changing the attitude information of the same object to the object descriptor;
Generating a sentence in the form of a posture change event of the same object and storing it in the storage unit.

Images