CN1838237A - Emotion recognizing method and system - Google Patents
Emotion recognizing method and system Download PDFInfo
- Publication number
- CN1838237A CN1838237A CNA2005100590616A CN200510059061A CN1838237A CN 1838237 A CN1838237 A CN 1838237A CN A2005100590616 A CNA2005100590616 A CN A2005100590616A CN 200510059061 A CN200510059061 A CN 200510059061A CN 1838237 A CN1838237 A CN 1838237A
- Authority
- CN
- China
- Prior art keywords
- information
- unit
- mood
- sound
- emotional
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 62
- 230000008451 emotion Effects 0.000 title claims abstract description 61
- 230000008859 change Effects 0.000 claims abstract description 33
- 230000036651 mood Effects 0.000 claims description 203
- 230000002996 emotional effect Effects 0.000 claims description 120
- 238000001514 detection method Methods 0.000 claims description 74
- 239000000523 sample Substances 0.000 claims description 70
- 238000012545 processing Methods 0.000 claims description 48
- 230000011218 segmentation Effects 0.000 claims description 26
- 238000003860 storage Methods 0.000 claims description 22
- 230000015654 memory Effects 0.000 claims description 14
- 230000001755 vocal effect Effects 0.000 claims description 14
- 238000011835 investigation Methods 0.000 claims description 11
- 230000015572 biosynthetic process Effects 0.000 claims description 8
- 238000001228 spectrum Methods 0.000 claims description 8
- 238000004458 analytical method Methods 0.000 claims description 7
- 230000008676 import Effects 0.000 claims description 7
- 230000005236 sound signal Effects 0.000 claims description 5
- 238000012217 deletion Methods 0.000 claims description 3
- 230000037430 deletion Effects 0.000 claims description 3
- 230000008450 motivation Effects 0.000 abstract description 7
- 230000007246 mechanism Effects 0.000 description 27
- 230000033764 rhythmic process Effects 0.000 description 27
- 239000004744 fabric Substances 0.000 description 21
- 238000006243 chemical reaction Methods 0.000 description 20
- 208000027534 Emotional disease Diseases 0.000 description 16
- 230000008569 process Effects 0.000 description 16
- 230000001815 facial effect Effects 0.000 description 14
- 206010012374 Depressed mood Diseases 0.000 description 13
- 238000005259 measurement Methods 0.000 description 13
- 238000010202 multivariate logistic regression analysis Methods 0.000 description 11
- 230000000875 corresponding effect Effects 0.000 description 10
- 238000010586 diagram Methods 0.000 description 9
- 230000008921 facial expression Effects 0.000 description 8
- 230000008878 coupling Effects 0.000 description 7
- 238000010168 coupling process Methods 0.000 description 7
- 238000005859 coupling reaction Methods 0.000 description 7
- 239000000284 extract Substances 0.000 description 7
- 230000006870 function Effects 0.000 description 7
- 208000019901 Anxiety disease Diseases 0.000 description 6
- 230000036506 anxiety Effects 0.000 description 6
- 238000003786 synthesis reaction Methods 0.000 description 6
- 238000013473 artificial intelligence Methods 0.000 description 5
- 230000001149 cognitive effect Effects 0.000 description 5
- 230000001276 controlling effect Effects 0.000 description 5
- 230000007613 environmental effect Effects 0.000 description 5
- 210000004709 eyebrow Anatomy 0.000 description 5
- 230000009467 reduction Effects 0.000 description 5
- 210000000216 zygoma Anatomy 0.000 description 5
- 230000003340 mental effect Effects 0.000 description 4
- 230000000007 visual effect Effects 0.000 description 4
- 238000009826 distribution Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 230000001360 synchronised effect Effects 0.000 description 3
- 101000741965 Homo sapiens Inactive tyrosine-protein kinase PRAG1 Proteins 0.000 description 2
- 206010020400 Hostility Diseases 0.000 description 2
- 102100038659 Inactive tyrosine-protein kinase PRAG1 Human genes 0.000 description 2
- 230000009471 action Effects 0.000 description 2
- 230000007812 deficiency Effects 0.000 description 2
- 230000036541 health Effects 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 230000005039 memory span Effects 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 238000003058 natural language processing Methods 0.000 description 2
- 238000012163 sequencing technique Methods 0.000 description 2
- 230000000638 stimulation Effects 0.000 description 2
- 230000002123 temporal effect Effects 0.000 description 2
- 230000007704 transition Effects 0.000 description 2
- 241001269238 Data Species 0.000 description 1
- 208000003443 Unconsciousness Diseases 0.000 description 1
- 230000008485 antagonism Effects 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000002596 correlated effect Effects 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 230000004665 defense response Effects 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 210000000887 face Anatomy 0.000 description 1
- 210000003128 head Anatomy 0.000 description 1
- 230000013632 homeostatic process Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000001788 irregular Effects 0.000 description 1
- 210000000088 lip Anatomy 0.000 description 1
- 230000007787 long-term memory Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000004321 preservation Methods 0.000 description 1
- 238000012797 qualification Methods 0.000 description 1
- 230000035484 reaction time Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 230000000717 retained effect Effects 0.000 description 1
- 238000010998 test method Methods 0.000 description 1
- 238000004148 unit process Methods 0.000 description 1
- 238000013316 zoning Methods 0.000 description 1
Images
Landscapes
- Toys (AREA)
Abstract
An object of the invention is to provide an emotion detecting method capable of detecting emotion of a human accurately, and provide sensibility generating method capable of outputting sensibility akin to that of a human. An intensity, a tempo, and intonation in each word of a voice are detected based on an inputted voice signal, amounts of change are obtained for the detected contents, respectively, and signals expressing each states of emotion of anger, sadness, and pleasure are generated based on the amounts of change. A partner's emotion or situation information is inputted, and thus instinctive motivation information is generated. Moreover, emotion information including basic emotion parameters of pleasure, anger, and sadness is generated, which is controlled based on the individuality information.
Description
The application is that the application number submitted September 4 calendar year 2001 is 01815455.7 divides an application.
Technical field
The present invention relates to a kind of emotion recognizing method, a kind of feeling ability generation method, the system that realizes these methods and software.Emotion recognizing method of the present invention can be used to the mood detection of medical field, and can be used for various systems as the part of artificial intelligence with artificial feeling ability.And a kind of feeling ability generation method of the present invention can be used to control with several different methods the various systems of the feeling ability of visual human and robot.
Background technology
For instance, relevant with emotion recognizing method of the present invention conventional art is published in Nos.Hei5-12023, Hei9-22296 and the Heill-119791 of " Japanese Patent Application Publication communique " (Japanese Unexamined Patent ApplicationPublication).
" Japanese Patent Application Publication communique " No.Hei5-12023 announces, the intensity of sound of the perdurability of sound, the resonant frequency of sound and each frequency is all surveyed as the characteristic quantity of this sound respectively.And this communique has also been announced a difference between detection reference signal and each characteristic quantity, and adopts fuzzy reasoning to survey mood according to this difference that detects.
" Japanese Patent Application Publication communique " No.Hei9-22296 announces, fore-and-aft plane vibration frequency, volume and the sound spectrum of the generating rate of sound (the minor joint number of unit interval), sound all surveyed as the characteristic quantity of this sound.And this communique has also been announced according to this characteristic quantity that detects and by the result that statistical treatment HMM (hidden Markov model) obtains and has been surveyed mood.
" Japanese Patent Application Publication communique " No.Hei11-119791 has announced and has utilized HMM to survey mood according to the probability of the phoneme frequency spectrum in the sound transition stage.
On the other hand, as with the relevant conventional art of feeling ability of the present invention (sensibility) generation method, can lift " mood generation system and mood generation method " (" Emotion Generating System andEmotion Generation Method ") of announcing among " Japanese Patent Application Publication communique " No.Hei11-265239 and be example.
The mood basis sight at that time of expressing people's inherent state or the like has various variations." Japanese Patent Application Publication communique " No.Hei11-265239 has announced the technology that generates the mood in the unpredictable sight.
Specifically, should be taken into account that foreseeable sight estimates various sights, and the mood of generation system self.In addition, mood that the reality of analyzing over generates and sight at that time, and study uncertain indirect condition and the therewith corresponding mood special to various sights.When the sight of a new input satisfies these indirect conditions, just export the mood corresponding with these indirect conditions.
For instance, the emotional state that is generated by such system can have influence on the sound and the image of output.
Summary of the invention
Yet the precision that this tradition emotion recognizing method is surveyed mood is very low, even it can survey mood to the language of special qualification, can not accurately survey a people's actual mood.So this emotion recognizing method only just can obtain actual use to limited application scenario, for example, fairly simple game machine.
An object of the present invention is to provide a kind of emotion recognizing method that can accurately survey as a people's of an object (subject) mood.
And this tradition emotion recognizing method just directly generates mood according to the information relevant with the input sight.To the people of reality, comprise that the various parameters of instinct, reason, individual character or the like can produce complicated influence each other, the result changes action, speech, expression etc.This tradition mood generation method can not make this instinct, reason, individual character or the like influence these results.
Instinct and mood can be counted as affectivity.In addition, instinct has become basic biological affectivity and the motivation that its mood generates.And, it is generally acknowledged that people directly do not export mood, but the feeling ability that their output is controlled by this reason and individual character.
Another object of the present invention provides a kind of feeling ability generation method that can output class be similar to a people's feeling ability.
According to a first aspect of the present invention, a kind of emotion recognizing method of the mood of detected object comprises the steps: to import a voice signal; According to the voice signal of above-mentioned input, survey the intensity of sound and the tempo that this sound occurs respectively, and the intonation that detects Strength Changes pattern in each word of representing this sound formation, as a time value; The 3rd variable quantity that second variable quantity that the beat of obtain first variable quantity that the intensity of this sound that detects of indication changes along time-axis direction respectively, indicating this sound changes along time-axis direction and the intonation of this sound change along time shaft; And the signal that generates the emotional state of expression indignation at least, sadness and happiness according to described first variable quantity, second variable quantity and the 3rd variable quantity respectively.
In this first aspect of the present invention, conform to and survey mood with comprising indignation, sad emotional state respectively with happiness by making each variable quantity from intensity, beat and the intonation of the sound of this object input.Adopt a kind of like this method, can survey mood more accurately than this conventional art.
According to a second aspect of the present invention, the mood detection system of the mood of detected object comprises: the sound input block is used for input audio signal; The strength investigation unit is used for surveying according to the voice signal of tut input block input the intensity of sound; The beat probe unit is used for surveying the speed that this sound occurs according to the voice signal of tut input block input, and with this as beat; The intonation probe unit is used for surveying according to the voice signal of tut input block input the intonation of the word Strength Changes pattern of this sound of expression, as a time value; The variable quantity probe unit, the 3rd variable quantity that is used for obtaining respectively indicating second variable quantity that first variable quantity that the intensity by the detected sound in described intensity detection unit changes along time-axis direction, indication change along time-axis direction by the beat of detected this sound of beat detecting unit and the intonation that detects this sound by described intonation detecting unit to change along time shaft; And the mood detection unit, be used for signal according to the emotional state of exporting expression indignation at least, sadness and happiness by detected above-mentioned first, second and third variable quantity of described change amount detection respectively.
In the mood detection system of this second aspect of the present invention, this sound input block, this strength investigation unit, this beat probe unit, this intonation probe unit, this variable quantity probe unit and this mood detection unit are provided, therefore can have carried out aforesaid emotion recognizing method.
According to a third aspect of this aspect, the intonation probe unit in the mood detection system of this second aspect of the present invention comprises: a bandpass filter is used for extracting specific frequency component from the voice signal that each word is separated input; A regional separative element, the power spectrum that is used for the signal that will be extracted by this bandpass filter is separated into many zones according to the intensity of this power spectrum; And an intonation computing unit, be used for according to calculating an intonation value by the time interval between the center separately in these isolated these many zones of separative element, zone.
This band-pass filter unit extracts these specific frequency components from the voice signal that each word is separated input.This zone separative element is separated into many zones with this power spectrum that detects according to their intensity.This intonation computing unit is according to the numerical value that is calculated this intonation by the time interval between the center separately in these isolated many zones of separative element, zone.
In this third aspect of the present invention, an energy distribution pattern relevant with these specific frequency components of this sound is detected out the time value at interval as these many zones of expression in the word, and length that should the time is used as this intonation.
According to a fourth aspect of the present invention, the mood detection system of this second aspect of the present invention also comprises: an image-generating unit is used to receive and an image information that face is relevant of this object at least; An image identification unit is used for surveying and each part location information related of this face according to the image information that is received by this image-generating unit; An image reference information is preserved the unit, is used to preserve the reference information relevant with the characteristic quantity of this each part of face; And an image feature amount probe unit, be used for surveying an image feature amount according to the positional information that detects by this image identification unit and by the reference information that this image reference information preservation unit is preserved.An emotional state is estimated according to the variation in the image feature amount that is detected by this image feature amount probe unit in this mood detection unit.
In this fourth aspect of the present invention, except this sound, also estimate this emotional state according to the expression of this object face.Usually, because people's emotional state influences the expression of their face, so can catch these emotional states by the facial expression of surveying them.Therefore, in this fourth aspect of the present invention, this emotional state is estimated according to the variation of the image feature amount that is detected by this image feature amount probe unit in this mood detection unit.
According to one the 5th aspect of the present invention, the mood detection system of this second aspect of the present invention also comprises: an emotional information storage unit, be used for receiving the various information relevant in rapid succession, and be used for storing therein these various information with these emotional states that detect by this mood detection unit; And one forgotten processing unit, be used for from the various information relevant deletion and since this information was stored originally, stored the information of predetermined period, and be used for illustrating that from wanting deleted information get rid of at least the emotional change amount is greater than the information of scheduled volume and the information that meets the predetermined variation pattern with the emotional state that is stored in this emotional information storage unit in the past.
Aspect the of the present invention the 5th, can be with the information storage relevant to this emotional information storage unit with these past emotional states that detect.And, can from this emotional information storage unit, be deleted automatically owing to since being detected, stored old information very over a long time, so can reduce the needed memory capacity of this emotional information storage unit.
Yet, typical information, the thread variable quantity of for example understanding can be got rid of from the information of needs deletion automatically greater than the information of this scheduled volume and the information that meets this predetermined variation pattern.So even this typical information is run ragged, they also can in statu quo be kept among this emotional information storage unit.Therefore, similar to a man memory, even perhaps useful typical information is run ragged in the future, also can from this emotional information storage unit, read so that reappear.
According to one the 6th aspect of the present invention, the mood detection system of the present invention the 5th aspect also comprises: a statement recognition unit, be used for carrying out grammatical analysis by handling the sound of saying or the information relevant with the character of this object input with this object, and with the verbal information of the implication that generates a statement of expression; And a memory control unit, be used for storing the verbal information that generates by the statement recognition unit of this emotional information storage unit according to mode with the information synchronization that relates to these emotional states.
The sound that this statement recognition unit processes and this object send or with the character relevant information of this object with a keyboard or similar devices input, the lang method of going forward side by side analysis generates the verbal information of expressing this statement implication.
For instance, this grammatical analysis can obtain to express the verbal information of " 5W3H ", i.e. " Who (who) ", " What (what) ", " When (when) ", " Where (where) ", " Why (what because of) ", " How (how) ", " How long; How far, How tall and so on (many height or the like how long, how far) " and " How much (how much) ".
This memory control unit stores the verbal information that is generated by the statement recognition unit in this emotional information storage unit under this verbal information and the state of information synchronization that relates to these emotional states.
Aspect the of the present invention the 6th, by consulting this emotional information storage unit, not only can take out and the relevant information of mood in any moment in the past, and can take out the verbal information of this moment sight of expression.
The information of preserving in this emotional information storage unit can be applied to various use occasions.For instance, if the mood assessment function of this mood detection system itself is inaccurate, just can proofread and correct a database that is used for estimating this mood according to the past result of detection of preserving in this emotional information storage unit.
According to one the 7th aspect of the present invention, the mood detection system of this second aspect of the present invention also comprises: a noiseless timing unit is used for measuring a noiseless time of reference according to the emotional state of the emotional state that detects; And a statement segmentation probe unit, be used for by utilizing the noiseless time of measuring by this noiseless timing unit of reference to survey a segmentation of this sound statement.
When carrying out this voice recognition and mood detection, must survey this segmentation to each statement, and must extract each statement.In general, owing to have a noiseless part in the segmentation between these statements, so many statements can go out to be separated now in these noiseless parts.
Yet the length of these noiseless parts is not constant.Particularly, the length of this noiseless part changes with a speaker's emotional state.So when distributing threshold in order to measure this noiseless part, the failure possibility of surveying this statement segmentation also becomes big.
Aspect the of the present invention the 7th, for instance, the noiseless time of this reference is measured according to the emotional state that has just detected before this mensuration, and the segmentation of this sound statement was surveyed according to the noiseless time of this reference.So, even segmentation that also can this statement of correct detection when this speaker's emotional change.
According to an eight aspect of the present invention, software has comprised a mood detection program of the mood of being carried out by computing machine, be used to survey an object, and wherein this mood detection program comprises the steps: a step that voice signal is input to this mood detection program; One according to this input audio signal survey this sound intensity, represent the step that this sound the speed of speed occurs and represents the intonation of Strength Changes pattern in each word that this sound constitutes; The step of the variable quantity of each in intensity of sound, speed of sound and the sound intonation that acquisition has been detected; And step that generates the signal of angry, the sad and happy emotional state of expression at least according to the variable quantity that is obtained.
Just can realize the emotion recognizing method of this first aspect of the present invention with included mood detection program in the software of this eight aspect of computing machine execution the present invention.
According to one the 9th aspect of the present invention, a feeling ability generation method comprises the steps: to preserve in advance the various individual informations of reason, predetermined characteristic and the purpose of an object of decision generation feeling ability at least; Generate this Mechanical-power-producing mechanism information according to the state of a mood that shows the partner or the input context information of the residing environment of this partner, this this Mechanical-power-producing mechanism information comprises one second instinct parameter of one first instinct parameter representing happy degree, the angry degree of expression and one the 3rd instinct parameter that (achievement) and intensity of variation are finished in expression at least; This Mechanical-power-producing mechanism information according to this generation generates emotional information, and this emotional information comprises happy, an angry and sad at least basic emotion parameter; And the emotional information of controlling this generation according to this individual information.
Aspect the of the present invention the 9th, this Mechanical-power-producing mechanism information that excites mood to produce is imported context information (this partner's mood, purpose and situation) according to this and is generated.Specifically, this this Mechanical-power-producing mechanism information generates from this context information, and this emotional information generates according to this this Mechanical-power-producing mechanism information.And the emotional information that will generate is controlled according to this individual information.So, can export by this individual's reason and the mood of purpose control, i.e. feeling ability information.
In addition, because this emotional information is by this this Mechanical-power-producing mechanism information generation, so can more accurate, more easily control the mood that will generate.
The mood that produces when running into this danger sight suddenly in the state that the mood that produces when for example, running into this danger sight in a people is having realized that the state of dangerous sight and this people are not recognizing dangerous sight is diverse.A kind of like this difference that reappears in these moods is possible.
Preferably make this instinct parameter also comprise an attention level (refusal degree), a determinacy degree (fascination degree), tracking degree (self-confident degree) or the like every except aforementioned.And, preferably make the basic emotion parameter that constitutes emotional information except aforementioned also comprise every surprised, frightened, painful, detest, despise, convergence, escape, envy, envy, rely on, enrage, anxiety or the like.
According to one the tenth aspect of the present invention, a feeling ability generator comprises: an instinct determination unit, be used to import the plot context information of the emotional state, the residing environment of this partner and this partner's that show the partner purpose, and be used for generating this Mechanical-power-producing mechanism information according to this plot context information, this this Mechanical-power-producing mechanism information comprises that at least one second instinct parameter of one first instinct parameter representing happy degree, expression hazard level is finished with expression and one the 3rd instinct parameter of intensity of variation; A mood generation unit is used for generating emotional information according to this Mechanical-power-producing mechanism information by this instinct determination unit output, and this emotional information comprises happy, angry and sad at least basic emotion parameter; Property information provides the unit one by one, is used to provide the reason of an object that determines the generation feeling ability at least and the individual information of purpose; And an emotion control unit, be used for controlling the emotional information of exporting by this mood generation unit according to the individual information that this individual information provides the unit to provide.
Feeling ability generator of the present invention the tenth aspect, can provide unit and emotion control unit to carry out feeling ability generation method as claimed in claim 9 by instinct determination unit, mood generation unit, individual information are provided.
So, can export by individual's the reason and the mood of purpose control, in other words, about the information of feeling ability.And, because emotional information is by the generation of this Mechanical-power-producing mechanism information, so can more accurate, more easily control the mood that will produce.
According to of the present invention one the tenth on the one hand, the mood generation unit of the present invention the tenth aspect comprises: a life rhythm generation unit is used to generate the information of the life rhythm of environment that indication cycle changes or a life; And a random emotion control unit, be used for controlling the random mood of this mood generation unit according to the life rhythm information of this life rhythm generation unit output.
For instance, physical environment situation (such as temperature and humidity) cyclical variation is although rule not too takes place simultaneously with the variations of weather, season, time etc.And it is believed that everyone has a health rhythm, a mood rhythm, an intelligence rhythm or the like uniquely.This periodically variable rhythm is considered to that people's actual mood is had many and diverse influences.
In the tenth one side of this aspect, the life rhythm information that this random emotion control unit is exported according to this life rhythm generation unit is controlled the random mood in this mood generation unit.So the mood that will be output can change according to this environment or according to the life rhythm of this life.
According to the 12 aspect of the present invention, mood generation unit in the feeling ability generator of the present invention the tenth aspect comprises: instinct-emotional information is preserved the unit, is used to preserve make this basic emotion parameter and this this Mechanical-power-producing mechanism information pattern information in correspondence with each other; And a matching probability unit, being used for exporting the information that this this Mechanical-power-producing mechanism information of expression and this instinct-emotional information are preserved the coupling/mismatch probability between the pattern information of unit, this this Mechanical-power-producing mechanism information is exported from this instinct determination unit.
Aspect the of the present invention the 12, can obtain this this Mechanical-power-producing mechanism information and from the matching probability between the pattern information of this matching probability unit so that utilize its mensuration factor as this mood.
For instance, a people's mental status is during from first state variation to second state, this mental status via a third state in the process from this first status transition to this second state.So, exist in the possibility that this mental status temporarily meets the pattern information in this third state.Yet the pattern information that meets this mental status in this third state does not have a high utilization factor.By the matching probability that utilizes this matching probability unit to obtain, just can suppress to have the generation of mood of the pattern information of low probability.
According to the 13 aspect of the present invention, mood generation unit in the feeling ability generator of the present invention the tenth aspect comprises an emotional feedback control module, be used for importing at least it self emotional information of final generation, and be used to make this final informational influence that generates will generate subsequently its self emotional information to this mood generation unit.
It is believed that, import different motivations and can cause a people's mood that chain variation takes place.The angry degree that produces when for instance, making his indignation for a motivation of people that is in normal condition can be very different each other with giving an angry degree that has produced when motivation of people of indignation has made him angry.
Aspect the of the present invention the 13, providing of this emotional feedback control module makes the emotional state that generates before this feedback be sent back to input, and the mood that this emotional state influence will be produced subsequently.So, can produce the mood more approaching with a people's mood.
According to the 14 aspect of the present invention, the feeling ability generator of the present invention the tenth aspect has such characteristics, i.e. this emotion control unit emotional information that a life rhythm informational influence will be transfused to, and this life rhythm produces the individual character of an object of feeling ability just.
Aspect the of the present invention the 14, the information of this life rhythm can influence this feeling ability.For example, whether be ready to do something in the determination result of making, difference can occur according to a people by reason etc.This difference in this feeling ability can be reappeared by the influence of this life rhythm.
According to the 15 aspect of the present invention, the feeling ability generator of the present invention the tenth aspect also comprises: a knowledge data base is used to store the expression result's of sight, past plot and this past sight and plot context information in the past; A knowledge is checked the unit, is used for from the retrieval of this knowledge data base and extracts with the context information of newly importing and similarly pass by context information, and be used for providing this past context information to this emotion control unit; And data are upgraded control module, be used for revising the content of this knowledge data base with the result's of this new sight context information, and be used for deleting automatically with time sequencing according to the weight of these contents the context information of low priority from this knowledge data base according to the sight of a new input of expression.
Aspect the of the present invention the 15, represent that this past sight and result's thereof context information is stored into this knowledge data base.For example, the whether successful information of a net result that stores a sight of expression plot and store this plot.So, can from this knowledge data base, obtain with deserve before sight similarly pass by context information so that be used for controlling this mood.
By the way, newly-generated information must be added to this knowledge data base as time goes by in rapid succession.Yet the memory span that constitutes a system of this knowledge data base is limited.And when the quantity of information that stores increased, processing speed descended.
But aspect the of the present invention the 15, the context information of low priority can be deleted according to time sequencing from this knowledge data base under the control of this Data Update control module automatically.So, can realize and the similar result of forgeing of a people, and can prevent the decline of deficiency and this processing speed of this memory span.
According to the 16 aspect of the present invention, the feeling ability generator of the present invention the tenth aspect also comprises: a sound input block is used to import a voice signal; A strength investigation unit is used for surveying according to the voice signal of this sound input block input the intensity of this sound; Velocity detection unit, be used for according to the voice signal of this sound input block input survey speed that this sound occurs and with this as speed; An intonation probe unit is used for surveying according to the voice signal of this sound input block input the intonation of a word Strength Changes pattern of this sound of expression; A variable quantity probe unit, the variable quantity in the sound intonation that detects with this intonation probe unit in the intensity of sound that is used for obtaining respectively detecting this strength investigation unit, the speed of sound that this velocity detection unit detects; And a mood detection unit, be used for exporting the signal of representing angry, sad and happy emotional state at least respectively according to the variable quantity that this variable quantity probe unit detects.
Aspect the of the present invention the 16, this partner's emotional state can be surveyed according to the characteristic quantity that extracts from this sound.So, can generate self mood according to this partner's mood.
According to the 17 aspect of the present invention, the feeling ability generator of the present invention the 16 aspect comprises: an acoustic recognition unit is used to discern the sound from this sound input block input, and is used for output character information; And a natural speech processing unit, be used to make by the sound message pick-up nature speech of this acoustic recognition unit identification and handle, and with the implication information of the implication of the sound that generates this input of expression.
Aspect the of the present invention the 17, obtained the implication information relevant, so just can influence this self feeling ability by a result who understands this implication information gained with the said word of this partner.
According to one the tenth eight aspect of the present invention, software comprised can carry out by a computing machine, be used for program and data that feeling ability generates control, wherein this program comprises: basis shows that an input context information of partner's the state of mood or the residing environment of this partner generates the step of this Mechanical-power-producing mechanism information, and this this Mechanical-power-producing mechanism information comprises one first instinct parameter represent happy degree at least, represent one second instinct parameter of hazard level and expression is finished or one the 3rd instinct parameter of intensity of variation; One generates the step of emotional information according to this Mechanical-power-producing mechanism information of this generation, and that this emotional information comprises is happy at least, a basic emotion parameter of indignation and sadness; The step that individual information is provided, this individual information determine to produce the reason and the purpose of an object of feeling ability at least; And step of controlling the emotional information of this generation according to this individual information.
The software of the tenth eight aspect of the present invention is imported into a predetermined computation machine of this program of execution, thereby realizes the feeling ability generation method of the present invention the 9th aspect.
Description of drawings
Read following detailed description together in conjunction with appended accompanying drawing, character so of the present invention, principle and effectiveness will become apparent.In these accompanying drawings, identical parts are with duplicate reference number representation, wherein:
Fig. 1 is the width of cloth block scheme of structure of the mood detection system of explanation embodiment;
Fig. 2 is a width of cloth synoptic diagram of the structure of explanation intonation probe unit;
Fig. 3 is a width of cloth coordinate diagram, the relation between intensity, speed and the intonation of emotional state variation of its explanation and sound;
Fig. 4 is the time plot of sound signal processing process in this intonation probe unit of explanation.
Fig. 5 is width of cloth process flow diagram of forgeing the operational process of processing unit of explanation;
Fig. 6 is a width of cloth synoptic diagram, the ios dhcp sample configuration IOS DHCP of the information that stores among mood of its explanation and the feeling ability storer DB;
Fig. 7 is a width of cloth block scheme, and the ios dhcp sample configuration IOS DHCP of the system of a feeling ability generator is adopted in its expression;
Fig. 8 is a width of cloth block scheme of the configuration of an instinct information generating unit of explanation;
Fig. 9 is a width of cloth block scheme of an emotional information generation unit of explanation;
Figure 10 is a width of cloth synoptic diagram, represents an example of reaction pattern model among the emotional reactions pattern DB; And
Figure 11 is a width of cloth block scheme of the configuration of feeling ability of explanation and thought recognition unit.
Embodiment
(first embodiment)
Come a relevant embodiment of a kind of emotion recognizing method indoor and of the present invention below with reference to Fig. 1 to Fig. 6.
Fig. 1 is a width of cloth block scheme, and it represents the configuration of the mood detection system of this embodiment.Fig. 2 is a width of cloth block scheme of an intonation probe unit of explanation.Fig. 3 is a width of cloth coordinate diagram, the relation between the variation of an emotional state of its explanation and intensity, speed and the intonation of sound.Fig. 4 is a width of cloth time history, each process of its explanation sound signal processing in this intonation probe unit.Fig. 5 is the width of cloth process flow diagram that an operational process of processing unit is forgotten in explanation.Fig. 6 is a width of cloth synoptic diagram, an ios dhcp sample configuration IOS DHCP of the information that stores among its expression mood and the feeling ability storer DB.
Referring to Fig. 1, this mood detection system comprises: a microphone 11; An A/D converter 12; A signal processing unit 13; An acoustic recognition unit 20; A strength investigation unit 17; Velocity detection unit 18; An intonation probe unit 19; A temporary data memory unit 21; An emotional change probe unit 22; A sound mood detection unit 23; A mood pattern database (DB hereinafter referred to as) 24; A keyboard 25; A statement recognition unit 26; A television camera 31; An image identification unit 32; A facial model DB33; A facial emotions probe unit 34; A character recognition unit 39; Mood and feeling ability storer DB41; Forget processing unit 42 for one; A synchronous processing unit 43; One individual information D B44; Property information D B45 one by one; An occupational information DB46; And mood recognition unit 60.
And, a signal processing unit 13 is provided in acoustic recognition unit 20; A phoneme probe unit 14; A word probe unit 15; And statement probe unit 16.Acoustic recognition unit 20 also comprises the function of voice recognition (the natural speech processing) equipment that sell in the shop.
In Fig. 1, acoustic recognition unit 20, strength investigation unit 17, velocity detection unit 18, intonation probe unit 19, temporary data memory unit 21, emotional change probe unit 22 and sound mood detection unit 23 all are the circuit of surveying mood from sound.
This mood detection system comprises that microphone 11, keyboard 25 and television camera 31 are used as reading the input block of a people's information, and this people is a partner who surveys mood for it.Specifically, this people's (i.e. this partner) mood utilization is from a sound of microphone 11 input, surveyed from the character information of keyboard 25 inputs and from the information that comprises facial expression of television camera 31 inputs etc.
Note, this mood also can be only according to from the sound of microphone 11 inputs, from the character information or a kind of detection the from the facial expression of television camera 31 inputs of keyboard 25 inputs.But,, judge that synthetically the information that obtains from these a plurality of information sources is more effective from the viewpoint of the detection accuracy that improves this mood.
The processing relevant with this sound at first described.Sample through A/D converter 12 from a voice signal of microphone 11 inputs, be converted into a digital signal then.The digital signal of this sound that obtains at the output terminal of A/D converter 12 is imported into acoustic recognition unit 20.
Signal processing unit 13 extracts the necessary frequency component of strength investigation of this sound.Strength investigation unit 17 comes detected intensity from the signal that signal processing unit 13 extracts.For example, can be used as this intensity to the average result of gained of the amplitude of the amplitude of this voice signal.
For instance, be set to about 10 seconds an average period surveying this intensity of sound.Note,, so just average to the time period of detecting this segmentation beginning from this statement if survey the segmentation of each statement under less than 10 seconds situation in average period.Say exactly, survey the intensity of this sound each statement.
The segmentation that the phoneme probe unit 14 that provides in the acoustic recognition unit 20 is surveyed each factor that is input to sound wherein.For instance, when being imported with the form of sound, will detect segmentation, as " kyo/u/wa/i/i/te/n/ki/de/su/ne " (Japanese) to each phoneme by the statement of " kyou wa ii tenki desun " (Japanese) expression.
The segmentation that the word probe unit 15 that provides in the acoustic recognition unit 20 is surveyed each word that is input to sound wherein.For instance, when being imported with the form of sound, will detect segmentation, as " kyou/wa/ii/tenki/desune " (Japanese) to each word by the statement of " kyou wa ii tenki desun " (Japanese) expression.
The segmentation that the statement probe unit 16 that provides in the acoustic recognition unit 20 is surveyed each statement that is input to sound wherein.Detecting when having length-specific or longer silent state, just thinking segmentation to have occurred each statement.A threshold limit value (TLV) of this silent state length has been distributed an about value of 0.1 to 0.2 second.And this threshold limit value (TLV) is not constant, and this threshold limit value (TLV) is changed automatically so that the emotional state that its reflection has just detected.
The signal that velocity detection unit 18 receives by 14 outputs of phoneme probe unit to each phoneme segmentation, and survey the phoneme quantity that in the unit interval, occurs.For a detect cycle of this speed, for instance, approximately distribute a time of 10 seconds.But, when the segmentation of surveying this statement,, also want the counting of stop tone prime number amount even within 10 seconds, survey this statement segmentation, till the moment of detecting this statement segmentation, and to calculate a value of this speed.Say exactly, tackle each statement speed of detection.
Digital signal from A/D converter 12 is divided each word of surveying segmentation by word probe unit 15, and this voice signal is imported into intonation probe unit 19.From the voice signal that is input to intonation probe unit 19, intonation probe unit 19 is surveyed in each word of expression and the intonation of the intensity of sound changing pattern in each statement segmentation in the statement probe unit 16.Thereby, the characteristic strength pattern that intonation probe unit 19 is surveyed in this segmentation.
As shown in Figure 2, in intonation probe unit 19, provide a bandpass filter 51, absolute value converting unit 52, comparing unit 53, regional center probe unit 54 and one interregional every probe unit 55.Each input and the waveform example of signal SG1, SG2, SG3 and the SG4 of output terminal of intonation probe unit 19 are seen Fig. 4.Notice that the ordinate of each signal is represented amplitude or intensity among Fig. 4.And in the example of Fig. 4, the length of a word that takes out from this sound is approximately 1.2 seconds.
51 of bandpass filter are surveyed necessary frequency component from extract intonation to the signal SG1 of its input.In this embodiment, have only the frequency component of frequency in 800 to 1200Hz scopes to appear at the output terminal of bandpass filter as signal SG2.Can find referring to Fig. 4, appear among the signal SG2 by the caused Strength Changes pattern of the intonation of this word.
In order to simplify the computing of this signal, in intonation probe unit 19, provide absolute value converting unit 52.Absolute value converting unit 52 is converted to the amplitude of this input signal its absolute value.So signal SG3 shown in Figure 4 appears at an output terminal of absolute value converting unit 52.
Comparing unit 53 is compared amplitude and this threshold limit value (TLV) of signal SG3, and only exports component greater than this threshold limit value (TLV) as signal SG4.Say the big component of numerical value in the power spectrum of 53 output signal SG3 of comparing unit exactly.The threshold limit value (TLV) that is added to comparing unit 53 is similar to by the method that is called as method of decision analysis to be determined.
Referring to Fig. 4, corresponding with intonation pattern in this word of this sound two regional A1 and A2 appear among the signal SG4.Regional center probe unit 54 detection time t1 and t2, the position corresponding with two regional A1 and the center separately of A2 just appears at this two times.
Interregional every the probe unit 55 detections mistiming relevant with t2 with two time t1 that detect by regional center probe unit 54, interregional every Ty with this as one.This interregional value every Ty is corresponding to the intonation pattern in this word of this sound.In fact, an average result of gained of each interregional value every Ty is used as a value of this intonation.
In brief, perhaps 3 or multizone more appear in SG4.If 3 or multizone more, will be to per two zones adjacent one another are zoning interval T y respectively, average and a result obtaining just is used as the value of this intonation every Ty to these a plurality of gained are interregional.
For instance, a people's emotional state changes as shown in Figure 3.And, for the correct mood that comprises indignation, sadness and happiness etc. of grasping, just want the variation of detection feature amount inevitably, such as intensity, speed and intonation.
In mood detection system shown in Figure 1, in order to inquire about characteristic quantity in the past, by the intensity level of strength investigation unit 17 outputs, all be temporarily stored in the temporary data memory unit 21 by the velocity amplitude of velocity detection unit 18 outputs and by the intonation value of intonation probe unit 19 outputs.
And emotional change probe unit 22 receives by the current intensity level of strength investigation unit 17 outputs, by the present speed value of velocity detection unit 18 outputs and the current intonation value of being exported by intonation probe unit 19.Emotional change probe unit 22 also receives in the temporary data memory unit 21 intensity, speed and the intonation value in past of storing (this current time soon before).So emotional change probe unit 22 is surveyed the variation of this emotional state.Specifically, emotional change probe unit 22 is surveyed the variation in intensity, speed and the intonation of this sound respectively.
Sound mood detection unit 23 receives by the variation in intensity, speed and the intonation of 22 outputs of emotional change probe unit, and estimates the emotional state that this is current.In this embodiment, sound mood detection unit 23 estimates to comprise that indignation, sad and happy three state are used as this emotional state.
Being stored in the past among the mood pattern DB 24 is the relevant information of pattern that intensity, speed and intonation in angry state and this sound are changed, make sad state and intensity, speed in this sound information relevant with the pattern of intonation variation and make happy state and this sound in the pattern of intensity, speed and the intonation variation information of being correlated with.
Sound mood detection unit 23 is according to the pattern that is changed by the Strength Changes of emotional change probe unit 22 outputs, velocity variations and intonation, and with reference to the information of preserving among the mood pattern DB 24 as estimating the regular currency of estimating this mood.
The expression of being estimated by sound mood detection unit 23 comprises that the information of 3 class states of indignation, sadness and happiness is imported into mood recognition unit 60 and mood and feeling ability storer DB 41.Mood and feeling ability storer DB 41 receive and store the current state of the mood of 23 inputs from sound mood detection unit in rapid succession.
So, read the past state that the information that stores among mood and the feeling ability storer DB 41 just can reappear this mood.
Simultaneously, discerned by statement recognition unit 26 from the statement content (this partner's speech content) of microphone 11 inputs as a sound.Each phoneme corresponding characters information that identifies with acoustic recognition unit 20 and the information of representing this word segmentation and statement segmentation are imported into statement recognition unit 26.And the character information of importing from keyboard 25 also is imported into statement recognition unit 26.
Statement recognition unit 26 is analyzed its grammer so that grasp the content of this statement to input of character string of each word identification and as a kind of natural speech.In fact, the verbal information of statement recognition unit 26 identification expression " 5W3H ", i.e. " who ", " what ", " when ", " where ", " what because of ", " how ", " many height or the like how long, how far, " and " how much ".Verbal information by 26 identifications of statement recognition unit is imported into mood recognition unit 60.
Below the processing procedure of surveying mood according to the expression of this partner's face will be described.Television camera 31 is produced a face as that people of the object of the mood detection system of Fig. 1 at least.Image (image that promptly comprises this human face's expression) by television camera 31 productions is imported into image identification unit 32.
Notice that the information of television camera 31 image of taking the photograph is imported into character recognition unit 39.Specifically, when the image of a statement was produced by television camera 31, character recognition unit 39 was from each character of this statement of production image recognition.Character information by character recognition unit 39 identifications is imported into statement recognition unit 26.
Image identification unit 32 is from this input picture recognition feature key element.Specifically, image identification unit 32 is discerned eyes, face, eyebrow, the cheekbone various piece of these object faces, and surveys each relative position of these facial eyes, face, eyebrow, cheekbone.And image identification unit 32 is followed the tracks of each position of eyes, face, eyebrow, cheekbone all the time, so that along with their change in location is separately surveyed in the variation of facial expression, and surveys expression, for example shakes the head.
The information relevant with the reference position of the position separately of eyes on this face, face, eyebrow, cheekbone (being equivalent to the facial expression of this object when normal condition) is stored among the facial model DB 33 in advance.Notice that the content among the facial model DB 33 also can change arbitrarily.And, represent that this facial expression changes and the Rule Information of the corresponding relation of 6 class moods (happy, angry, sad, frightened, happy and surprised) is stored among the facial model DB 33 in advance.
Facial emotions probe unit 34 survey according to by the relevant information in each position of eyes, face, eyebrow and the cheekbone of image identification unit 32 identifications and be stored in the characteristic quantity that this expression is surveyed in reference position among the facial model DB 33, promptly the characteristic quantity of they and normal condition is poor.
And facial emotions probe unit 34 is according to the variable quantity and the speed of the characteristic quantity that detects, and estimates the state separately of 6 class moods (happy, angry, sad, frightened, happy and surprised) according to the Rule Information of preserving among the facial model DB 33.The information of estimated state of representing this 6 class mood is by facial emotions probe unit 34 output, and is imported into mood recognition unit 60 and mood and feeling ability storer DB 41.
The information of mood recognition unit 60 synthesis measurings expression emotional states (indignation, sad or happy) of 23 inputs from sound mood detection unit, from the verbal information of statement recognition unit 26 inputs and from the information of the expression emotional state (happy, angry, sad, frightened, happy or surprised) of facial emotions probe unit 34 inputs.Then, mood recognition unit 60 is estimated the end-state of this mood.As for this verbal information, comprising emotional state (happy, angry, sad, frightened, happy or surprised) can be estimated by the content (5W3H) of measuring the statement in this speech according to pre-defined rule.
Information that the speech content that expression is discerned according to the character of this sound or keyboard 25 inputs by sound mood detection unit 23 information according to the estimated emotional state of this sound, with statement recognition unit 26 is relevant and expression facial emotions recognition unit 34 are input to mood and feeling ability storer DB 41 respectively from the information of the estimated emotional state of facial expression, and are stored in wherein in rapid succession.The detection time and the date of the each several part information that stores among mood and the feeling ability storer DB 41 are added among this information.
In the information that is input to mood and feeling ability storer DB 41, the information relevant with the mood of 23 inputs from sound mood detection unit, the information relevant with the speech content of importing from statement recognition unit 26 and the information relevant with the mood of importing from facial emotions probe unit 34 must be grasped on ground associated with each other.
So time (input time) and date that the synchronous processing unit 43 polytype information among mood and the feeling ability storer DB 41 that will be stored in is detected according to these information are associated with each other.For example, the information of the emotional state that comprises indignation, sad and happiness that expression has been estimated by sound mood detection unit 23 and the information relevant with speech content (5W3H) are all associated with each other by their moment, as shown in Figure 6.
Say that by way of parenthesis mood and feeling ability storer DB 41 comprise an enough memory capacity that can store relatively large information.But, owing to there is restriction to this memory capacity, therefore, for the quantity of information of using for a long time this system just must restriction will store therein incessantly.
So, provide and forgotten processing unit 42.Forget processing unit 42 and from mood and feeling ability storer DB 41, delete old information automatically.Yet the information that is suitable for specified conditions is not deleted, will store even be run ragged yet.
Forgeing an operational process of processing unit 42 will be described with reference to figure 5.
In the step S11 of Fig. 5,, inquire about with each data the relevant information of time and date that is stored (or detection) in the mass data that is stored among mood and the feeling ability storer DB 41 each.
At step S12, distinguished since this related data is stored whether over and done with one section preset time according to the time of mentioning among current time and the step S11.If handle the legacy data that has stored a period of time since storing constantly, this processing enters step S13 and later flow process so.The newer data that also are not stored certain period then continue in statu quo to store.
At step S13,, so just check this change in information amount (a mood difference that incident is forward and backward) if these data are the information of this emotional state of expression.Because this processing skips to S17 from step S13 when this emotional change amount surpasses the predetermined threshold value, so also store by the meeting former state even these data are legacy datas.If the variable quantity of this mood equaled/less than this threshold limit value (TLV), this processing would enter S14 from step S13 so.
At step S14, survey the mood pattern relevant, and distinguish whether this associative mode conforms to the special pattern of being scheduled to these data.Specifically, be to check whether these emotional states conform to the state of special, expression " to people's strong impression " with the multiple combination of speech content.Because this processing skips to S17 from step S14 when this pattern that detects conforms to this AD HOC, so also can in statu quo store even these data are outmoded.If these patterns do not conform to each other, this processing just enters S15 from step S14.
At step S15,, so just distinguish whether these contents conform to the speech content of being scheduled to (giving the speech of the important impression of people) if these data are speech contents.Even two contents not exclusively conform to each other, if but similarity between the two is very high, and they also just are taken as to conform to treats.Because this processing skips to S17 from step S15 when the speech content of this related data conforms to this predetermined speech content, so also can in statu quo store even these data are outmoded.
If these two contents do not conform to each other in step S15, this related data will be deleted in step S16.
This above-mentioned processing is carried out all data among mood and the feeling ability storer DB 41.And the processing of forgeing shown in Figure 5 is repeated periodically.This performance period forgeing processing can change arbitrarily with individual's individual character.Notice that this processing is carried out by the pattern DB (not drawing) for preparing before the inquiry in step S14 and S15.Pattern DB hereto, content wherein can be input to information wherein and revised automatically by study.
Fig. 5 represents the processing procedure simplified.In fact, emotional change amount, mood pattern and speech content synthesis measuring in addition all.Say exactly,, will comprehensively determine their priority so if the information and the speech content that exist the very big information of emotional change amount, mood pattern to conform to this AD HOC are scheduled to the identical or similar information of speech content with this.Specifically, the speech content be endowed limit priority with this predetermined identical or similar information of speech content, and the information that the mood pattern conforms to this AD HOC is endowed second high priority, and the information that the emotional change amount is big then is endowed lowest priority.So, to forget in the processing at this, the speech content is unlikely deleted with this predetermined identical or similar information of speech content, also can keep as memory even it is run ragged.
For the legacy data among mood and the feeling ability storer DB 41, have only the big data of emotional change amount, its pattern be considered to " give people's strong impression " data, by the data that repeatedly repeat to import with and the speech content data of giving important impression just can be endowed priority according to their intensity and content, and in statu quo store according to the above-mentioned processing of forgeing in the processing unit 42.So the legacy data among mood and the feeling ability storer DB 41 only just becomes that some is retained in deficiency of data wherein.The content class of this data is similar to the unclean memory in the past in the human mind.
The emotional state that always is stored in these past among mood and the feeling ability storer DB 41 can be read out so that carry out data analysis with speech content in the past, thereby, for instance, the whether normally operation of this mood detection system can be measured, and the database of each unit that is used to estimate mood can be revised so that improve wherein content.
The data that are stored among mood and the feeling ability storer DB 41 are also further distributed according to their content, and are stored into human nature information D B 44, individual information DB 45 or occupational information DB 46.
In human nature information D B 44, store the information (such as sex, age, aggressiveness, cooperative and current mood) and the information relevant of this characteristics of objects of definition with the decision-making mode of an action.In individual information DB 45, the information of storage is individual address, the present situation, current environment and speech content (5W3H).In professional information D B 46, the information of storage is occupation, career, vocational ability and professional behavior decision-making mode.
From human nature information D B 44, individual information DB 45 and occupational information DB 46 outputs are moral pattern informations of individual.This partner's feeling ability can be understood according to this partner's moral pattern information and past mood.
If the function of mood detection system shown in Figure 1 realizes with the software of a computing machine, as long as the program that will be carried out by this computing machine and data necessary are prerecorded on a recording medium (for example CD-ROM) and just met the requirements so.
Notice that microphone 11 shown in Figure 1 can replace with a phone receiver, also can provide a mouse to be used as being used for the unit of input information (such as character).
And television camera 31 shown in Figure 1 also can be with any replacement the in the various image-generating units, for example an optical camera, a digital camera and a ccd video camera.
Adopt above-mentioned emotion recognizing method can survey the mood of this object more accurately than classic method.
(second embodiment)
Describe and the relevant embodiment of a kind of feeling ability generation method of the present invention referring to Fig. 7 to Figure 11 below.
Fig. 7 is a width of cloth block scheme, and an ios dhcp sample configuration IOS DHCP of the system of a feeling ability generator is adopted in its expression.Fig. 8 is a width of cloth block scheme of the configuration of explanation instinct information generating unit.Fig. 9 is a width of cloth block scheme of an emotional information generation unit of explanation.Figure 10 is a width of cloth synoptic diagram, an example of reaction pattern model among emotional reactions pattern DB of its expression.Figure 11 is a width of cloth block scheme of the configuration of feeling ability of explanation and thought recognition unit.
The design of system shown in Figure 7 is to realize nature and responsive dialogue between a careless people of supposition and the computing machine (visual human).A mood detection system 200 is provided in this example will be as that people's of the partner of this computing machine mood so that survey, and a feeling ability generator 100 is so that make individual character of this calculating itself and feeling ability influence this dialogue.
And, provide an environmental information input equipment 300 so that various environmental informations are input to feeling ability generator 100.For instance, 300 outputs of environmental information input equipment and date, time, weather, place and image-related information.
Feeling ability generator 100 also can be used to the system of an independent operating.For example, if, just can obtain to meet the reaction of this scene so from the output of feeling ability generator 100 with the information input feeling ability generator 100 relevant with the scene that produces in the past.Do not need mood detection system 200 in this case.
Although the output of feeling ability generator 100 has connected the plurality of devices that realizes that this dialogue is required in Fig. 7 example, the feeling ability data of feeling ability generator 100 outputs can be used for various purposes.
For example, in data communication, utilize under the situation of feeling ability generator 100, because output character information satisfactorily, so do not need to export a sound.And the feeling ability data of feeling ability generator 100 outputs also can influence image, music, information retrieval and machine control.
The structure and the operation of feeling ability generator 100 will be described below.Owing to suppose that in this embodiment mood detection system 200 is identical with the mood detection system of having described 200 shown in Figure 1, so omit its explanation here.
In fact, system shown in Figure 7 can be made of a computer system and a software program that is performed therein, perhaps can be used as a specialized hardware and realizes.And software program that will be used and data can be stored among any one recording medium in advance, and can read in computing machine so that carry out from this recording medium.Note, be called as a computing machine during system shown in Figure 7 itself illustrates hereinafter.
Divide roughly, be input to the two class data that have of feeling ability generator 100 input ends, i.e. data D1 and data D2.Data D1 is the information of this partner's mood of expression.Data D2 is the character information that has received natural language processing, and it comprises the information relevant with this partner's purpose, situation and environment.How long, how far, by natural language processing, data D2 is used as the information input of expression " 5W3H ", i.e. " who ", " what ", " when ", " where ", " what because of ", " how ", " many height or the like " and " how much ".
In fact, can utilize following listed various information to be used as the input of feeling ability generator 100.
(A) relate to the changing pattern of the sounding of time attribute, comprise stress, rhythm, speed, pause, scale, interval, tone, harmony, frequency or the like; And the degree of basic emotion (indignation, happiness, sadness, detest, surprised, frightened or the like).
(B) information relevant with the sounding that relates to the tone attribute comprises accent, overcast, intensive, alert and resourceful, ear-piercing, tone color (JIS-Z8109), resonance, intonation, makes the part of one section spoken language to illustrating very important conspicuousness of implication or the like.
(C) distribution of the stress in word, segment contents, the statement, suprasegmental features information, by the characteristic information that artificial intelligence generates, these all relate to the character of stress.
(D) text message of analyzing through dialogue, plot information (comprising implication information and the information of discerning by artificial intelligence) or the like.
In these information, information (A) is subjected to a speaker's intention and emotion influence with information (B).This mood can be surveyed by mood detection system 200.
As shown in Figure 7, feeling ability generator 100 comprises an instinct information generating unit 110, measurement pattern DB 121, instinct language definition dictionary 122, emotional information generation unit 130, emotional reactions pattern DB 141, temporary storage DB 142, feeling ability and thought recognition unit 150, knowledge DB 161, feeling ability DB 162, people DB 163 and a moral hazard DB 164 one by one.
The function of feeling ability generator 100 can be divided into 3 functional imperative substantially: instinct information generating unit 110, emotional information generation unit 130 and feeling ability and thought recognition unit 150.Instinct information generating unit 110 is at first described.
As shown in Figure 8, instinct information generating unit 110 comprises measurement pattern coupling recognition unit 111, an instinct parameter generating unit 112 and a dictionary retrieval unit 113.
A measurement pattern dictionary that is input to this computing machine (visual human) is stored among the measurement pattern DB 121 that is subjected to 111 inquiries of measurement pattern coupling recognition unit in advance.This yardstick is the rhythm key element of a speech, voice and harmonious sounds feature that its representative occurs a syllable, a word, a phrase, a statement and whole speech (than the long continuous sound of a word).Specifically, being used as personal information with input information (A) and (B) pattern information of suitable this computing machine itself is stored among the measurement pattern DB 121.
Measurement pattern coupling recognition unit 111 will be analyzed data D1 from partner's mood of mood detection system 200 inputs and be compared with the measurement pattern that is stored in the measurement pattern DB 121, and discern both synchronous and matching degrees.Expression exists the information of a strong tone and emotional change can appear at the output terminal of measurement pattern coupling recognition unit 111.
Simultaneously, stimulate relevant information to be recorded in advance in the instinct language definition dictionary 122 with instinct.On the contrary, the stress distribution pattern in word of expression or the statement and various information suprasegmental features, that relate to stress character be stored be one with the relevant dictionary of this instinct stimulation.
The data D2 (partner's purpose and situation) that dictionary retrieval unit 113 will be imported as character information is compared with the content of instinct language definition dictionary 122, and produces natural reaction information from the content of one section dialogue.
Instinct parameter generating unit 112 is according to producing this Mechanical-power-producing mechanism information D 4 by the information of measurement pattern coupling recognition unit 111 inputs, information and the data D3 that is imported by dictionary retrieval unit 113.Data D3 is the output terminal feedack from feeling ability generator 100, and it has the plot of this computing machine proposal and wishes reaction pattern.
In this example, this Mechanical-power-producing mechanism information D 4 comprises 6 instinct parameters: a determinacy degree (or fascination degree), a happy degree (or unhappy degree), a hazard level (or safe coefficient), an attention level (or refusal degree), a performance level (or intensity of variation) and a tracking degree (or self-confident degree).Instinct parameter generating unit 112 determines the numerical value of each instinct parameter with following method.
Happy degree (unhappy degree): when this computing machine during near the sight plot of an interior perhaps hope of suggestion, happy degree increases, otherwise degree reduces.And when this computing machine was predefined for happy yardstick near one, happy degree increased, otherwise reduces.
Hazard level (safe coefficient): when this computing machine near being considered to dangerous interior perhaps one when being assumed to be dangerous sight plot in advance, hazard level increases, otherwise reduces.And when this computing machine was predefined for dangerous yardstick near one, hazard level increased, otherwise reduces.
Performance level (intensity of variation): when this computing machine interior perhaps when a quilt is assumed to be the sight plot of success/realizations in advance near what be pre success/realizations, performance level increase, otherwise reduction.And when this computing machine during near a particular dimensions that is considered to thoroughly to adjust, performance level increases, otherwise reduces.
Attention level (refusal degree): when this computing machine is assumed to be the sight plot of refusal/negate in advance near interior perhaps quilt of being used as refusal/negate in advance, refusal degree increase (attention level reduction), otherwise reduction (increase).And, when this computing machine detects strongly or repeatedly self-confident or during near a strong yardstick, attention level increases.If this computing machine is confirmed as offending yardstick near one, the refusal degree increases so.
Tracking degree (self-confident degree): when this computing machine feel oneself inferior near being pre/self-sacrificing content and quilt be assumed to be in advance and feel oneself inferior/during self-sacrificing sight plot, the tracking degree increases (self-confident degree reduction), otherwise reduces (increase).If be defined as good content in advance, then self-confident degree increases (reduction of tracking degree).And if one be pre uncertain yardstick, self-confident degree increases.Note, if this computing machine near a strong yardstick, repels degree so or self-sacrificing degree increases sometimes.
Determinacy degree (fascination degree): when the approaching content that confuses of this computing machine and a sight plot of supposing in advance, under the situation of the discrimination in the various stimulations (input) relevant very low (such as 70% or lower) with instinct, fascination degree and this discrimination journey inverse ratio.This discrimination is measured by the acoustic tones and the content of a dialogue.
In order to realize above-mentioned control, the yardstick of desirable content of this computing machine and sight plot will be measured as individual character in advance.As mentioned above, this partner's emotional information stimulates the individual instinct of this computing machine, thereby changes the numerical value of each instinct parameter.
This Mechanical-power-producing mechanism information D 4 by 110 outputs of instinct information generating unit is imported into emotional information generation unit 130.Below emotional information generation unit 130 will be described.
As shown in Figure 9, emotional information generation unit 130 comprises a reaction pattern retrieval unit 134, study processing unit 135, multivariable analysis unit 136, a random emotion control unit 137 and a basic emotion parameter generating unit 133.
Reaction pattern retrieval unit 134, study processing unit 135 have constituted an answering system 131 with emotional reactions pattern DB 141.Multivariable analysis unit 136 constitutes an operating system 132 with random emotion control unit 137.
The purpose that answering system 131 is provided is the mood that is caused by boost inductor in order to generate.The purpose that operating system 132 is provided is in order to export random mood (Libido (libido)).
Relating to the information of representing a reaction pattern model of corresponding relation between this this Mechanical-power-producing mechanism information D 4 and this basic emotion parameter is stored among the emotional reactions pattern DB 141 in advance for answering system 131 uses.For instance, this reaction pattern model can be as shown in figure 10.
Under the situation of the personality that reappears many people by a computing machine selectively, each reaction pattern model is recorded among the emotional reactions pattern DB 141 in advance, wherein each reaction pattern model all corresponding to one or a class individual character of these many philtrums, can select according to this elected's individual character by a reaction pattern model.
In this example, adopted to be used as above-mentioned 6 instinct parameters that this Mechanical-power-producing mechanism information D 4 is imported, they are: determinacy degree (fascination degree), happy degree (unhappy degree), hazard level (safe coefficient), attention level (refusal degree), performance level (intensity of variation) and tracking degree (self-confident degree).
Following 15 class parameters are used as from the basic emotion parameter of emotional information generation unit 130 outputs.The instinct parameter of this basic emotion parameter influence represented to be subjected in term in the bracket.
1. angry (unhappiness)
2. happy/happy (happiness)
3. sad (not finishing/stagnate/unhappiness)
4. surprised (finishing/impact)
5. frightened (danger/anxiety)
6. painful (danger/anxiety/unhappiness)
7. detest (resistance/refusal/unhappiness)
8. despise (resistance/weakness)
9. convergence (happiness/safety)
10. escape/avoidance (danger/anxiety/unhappiness)
11. envy (unhappiness/indignation/envy/note)
(safety/happiness/determinacy) 12. certainly
13. rely on (finishing/follow the trail of)
14. enrage/conflict (confidence/stagnation/unhappiness/danger)
15. anxiety (danger/anxiety/fascination/unhappiness)
Represent the reaction pattern of the relation of each and one or more basic emotion parameters in the 15 class basic emotion parameters to be stored among the emotional reactions pattern DB 141.
The reaction pattern of the basic emotion parameter among the reaction pattern retrieval unit 134 retrieval emotional reactions pattern DB 141, check the coupling/mismatch situation of this Mechanical-power-producing mechanism information D 4 of they and this input, and the information of exporting this basic emotion parameter that conforms to is as data D6.
Simultaneously, environmental information (D2) then is imported into (for example comprising Weather information, season information, temporal information etc.) input end of operating system 132.The environmental information (D2) of the 136 pairs of various inputs in multivariable analysis unit is carried out multivariable analysis, thus output life rhythm information.
In this life rhythm information, there is rule (sinusoidal waveform) rhythm with constant cycle, such as short period rhythm (as 1 hours period), a life rhythm (as 24 hours periods), a mood long period rhythm (as 28 day cycle), a health long period rhythm (as 23 day cycle) and an intelligence rhythm (as 33 day cycle), also there is irregular rhythm, such as temperature, humidity and weather.
Arbitrarily emotion control unit 137 is according to the random mood (Libido) in the life rhythm information of the output of the probability in preset range multivariable analysis unit 136 outputs.
Basic emotion parameter and matching rate and the operating system 132 random mood relevant information resulting result that export of basic emotion parameter generating unit 133 output by synthesis measuring and answering system 131 outputs, with this as self mood D5.In this case, this result is the information that is made of this 15 class basic emotion parameter.
And self emotional information D5 of this output is temporarily stored among the temporary storage DB 142, and is fed back to the input end of basic emotion parameter generating unit 133.Basic emotion parameter generating unit 133 receives from temporary storage DB 142 feedacks as last one constantly self mood, and makes it influence next mood measurement result constantly.
When basic emotion parameter generating unit 133 was carried out synthesis measuring, it just influenced priority and degree according to what the individual characteies of determining as individual information 143 decided each unit.
For instance, in reproduction during an impulse type mood, the influence degree of answering system 131 increases (80% or more than), and the influence of self mood just now also increases.During a thoughtful type mood, the influence degree of answering system 131 reduces (30% or following) in reproduction, and the influence of self mood just now also reduces in the stable environment of the output of operating system 132.
Be imported into feeling ability and thought recognition unit 150 from self mood D5 of emotional information generation unit 130 output.As shown in figure 11, emotional information generation unit 130 comprises a weighting processing unit 151, collation process unit 152, multivariable analysis unit 153, a comprehensive decision package 154 and the modification processing unit 156 of inspiring.
Weighted units 151 is according to self emotional information D5 weighting of 155 pairs of inputs of individual information.Self emotional information after the weighting is again from weighting processing unit 151 outputs.
Simultaneously, comprise the three unities of representing partner's environment of living in and situation and comprise that this partner's purpose and result's thereof character information (5W3H) is used as the input end that data D2 is input to collation process unit 152.
The implication information of this past plot, its result and their implications of expression with the form of character information (5W3H) be used as Knowledge Storage to knowledge DB 161 for 152 inquiries of collation process unit.And the various knowledge among the knowledge DB 161 comprise the temporal information that obtains each data, and are arranged according to the order of these times.
In this example, the various knowledge among the knowledge DB 161 can be classified as a long term memory, a statement storer and a program storage.This statement storer is a storer that stores by word, and representative is as the plot information of the incident in a special time/space linguistic context and implication information as general knowledge.This program storage representative is about the storer of method or technology.
This plot information comprises personal information, affectivity, intention (target), attitude, personal relationship of time, place, content, purpose (agree with, oppose, support or the like), personnel, quantity, weight, sight, state, partner or the like.This implication information is equivalent to language dictionary and feeling ability dictionary.Imagination is as temper, personality, social adaptiveness's (sociability), hope, conflict, attitude, the sense of superiority, love knot, interest, correctness, morals, thought pattern, emotional characteristics are arranged, the adhering to content (and degree), avoid language of personal information, taste, kind/evil standard or the like.
In this example, knowledge information is stored among the knowledge DB 161 according to the following grammer that will illustrate, still, the constitution content of this database is different because of object.
Story=Scene+Plot+Solution (story=scene+story of a play or opera+solution)
Scene=Character+Place+Time (scene=personage+place+time)
Theme=(Event)+Target (theme=(incident)+target)
Plot=Episode (story of a play or opera=plot)
Episode=Lower target+Attempt+Result (plot=than low target+attempt+result)
Attempt=Event+Episode (attempt=incident+plot)
Result=Event+State (result=incident+state)
Solution=Event+State (solution=incident+state)
Lower target, Target=Desirable state (than the state of low target, target=hope)
Character, Place, Time=State (personage, place, time=state)
And fresh information is added to knowledge DB 161 by the operation of revising processing unit 156 with keeping.Moreover unwanted information is forgotten processing and is deleted from this knowledge automatically by repeating.Specifically, except the data with higher priority, these data are deleted from the knowledge that wears according to time reference with being kept.For example, the knowledge of Shi Yonging is endowed priority with assert the data with strong impression repeatedly, even they wear, and can be not deleted yet.Each data forget degree and priority can change according to this individual character.
Collation process unit 152 is retrieved from knowledge DB 161 and is extracted and import approaching past plot and the result thereof of data D2 according to the data D2 of input.Then, the knowledge of collation process unit 152 these extractions of usefulness is checked this input data.
A study disposal system 157 generates and the information that the people is relevant to the self-value notion of this input plot by the result of study according to this input plot.Specifically, study processing unit 157 is made satisfied, happy and offending degree according to the result of this input plot.
The 153 pairs of following information in multivariable analysis unit are carried out multivariable analysis: from the weighting emotional information of weighting processing unit 151 inputs; The plot information and the object information of 152 inputs from the collation process unit; From study processing unit 157 information input, relevant with a people's self value notion and from individual DB 163 inputs, an and people's self the purpose information relevant with instinct.Then, multivariable analysis unit 153 is to comprehensive inspiration decision package 154 these analysis results of output.
The comprehensive decision package 154 that inspires utilizes individual DB 163 and the content of moral hazard DB 164 to measure the information that dictionary comes synthesis measuring 153 inputs from the multivariable analysis unit as one, and the content that will arbitrarily be carried out of output and an one result are as data D3.
The various information that will describe below are used as dictinary information and are stored among the individual DB 163.
1. individual information
(a) according to the mensuration criterion of the degree of every class individual character: the type of imagination have stereotyping, human-like towards him, towards inner, towards traditional, towards offensive, towards cooperation type, stress the strike type, stress release type or the like.Finish the motivation degree and the antagonism degree also can be used as the mensuration criterion.
(b) the mensuration criterion of Cognitive Style: distinguish Cognitive Style according to the difference of " reflection-type " and " impulsive style " and " dependent form " with the difference of " self " and be defined as measuring criterion.
(c) according to the mensuration criterion of personality:, be used as the mensuration criterion by the following project of personality test method and TPI (Todai inventory of personality table) classification to the Japanese.These classification items are: temper, personality, emotionality, social adaptiveness's (sociability), hope, conflict, attitude, love knot, interest, correctness, morals, thought pattern, mood characteristic, adhere to content (and degree), avoid language, taste, good and evil standard, ashamed standard, fault standard, happy and unhappy standard or the like.
(d) the mensuration criterion of negative/prejudice: NACK messages is added a prejudice so that grasp identical NACK messages in a large number, and they just can be used to form a personality then.
(e) follow/adhere to the mensuration criterion of time: determine to partner's cognitive information, plot and emotional information adhere to degree and to its reaction response time.
This I/unconscious reaction reference information:
(a) word lexicon and clause's dictionary, each dictionary all have the content that stimulates instinct.
(b) adhering to degree, follow the reference value of the various natural reaction time of degree and candid degree each individual.
(c) as individual character ego instinct pattern decision, corresponding with a partner's mood.
3. the reference information of homeostasis (constraining): attempt to keep the stable mensuration criterion of whole instinct output coordinating.
4. self-consciousness is reacted the reference time: the mensuration criteria information of representing people self purpose according to individual character.
And, be included in having in this mensuration dictionary: be used to discern and measure and the information of differentiating mensuration, for example true/false, correct/incorrect, suitable/inappropriate; Be used for the information that instinct is measured, such as happiness/unhappiness; Be used for the cognitive information of measuring of object individual, such as complicacy, weight coefficient or the like; Be used between object the cognitive relatively information of measuring, for example be equal to, size, difference be to similar; Be used for the information that metamemory is measured, for example the reliability standard and the knowledge accuracy of memory; The information that is used for abstract mensuration, such as truth, virtue, love or the like; Be used to conclude the information of mensuration; Or the like.
Dictinary information about professional ethics, individual morality, cardinal virtues etc. is stored among the moral hazard DB 164.
For example,, can write down " as an architect, I require a complete calculating ", " my override is considered my work ", " I am that a professional takes pride in for me " or the like as professional ethics.And,, can write down " I am proud for my native place ", " I am that the Japanese is proud for me " or the like as the individual morality.And, can write down " kill a person is bad thing ", " I the careful father and mother that look after me ", " I am a man (woman) " or the like as cardinal virtues.
Comprehensive inspire decision package 154 analyze by emotional information generation unit 130, weighting processing unit 151, collation process unit 152 and multivariable analysis unit 153 that generate with the relevant information of self mood.Then, comprehensively inspire decision package 154 according to the mensuration dictionary among the individual DB 163 that represents this computing machine individual character and purpose and according to the mensuration dictionary among the moral hazard DB 164 suppress this by analysis, with the relevant information of self mood.Subsequently, inspire comprehensively what decision package 154 decision will export, that is a kind of and what self emotional reactions (feeling ability).Under this decision situation, will reflect environment and sight and this partner purpose at that time at a partner place.
Feeling ability and thought recognition unit 150 comprise following function.
1, when surveying a strong impression, vocabulary or basic emotional change, a mensuration cycle changes according to this individual character.For example, when showing a strong content loudly suddenly, this mensuration cycle just shortens.
2, when a biorhythm people oneself, that depend on this individual character is responded, whether be ready to do something according to a people, the realization that feeling ability is measured has a great difference.
3, the happiness/unhappiness according to a people self realizes different feeling ability mensuration with the mood amount.
4, according to the knowledge among the knowledge DB 161 information of the expression situation of presence is carried out rational value judgement, reflect the influence of the judged result of this mood, thereby and determine final purpose.
5, when carrying out value judgement, this judgement is carried out according to social value, occupation value, daily life value, individual value or the like various viewpoints.And, be worth these social values, occupation, in daily life value and the individual value each distinguished in more detail and judge.For example, for social value, calculate value from the various viewpoints of religion, aesthetics, society, politics, economy and ethics.
6, to carry out value judgement to various factors, for example with satisfaction/dissatisfied, gain and loss/interests, safety/danger or the like judgement data as this purpose decision-making.For instance, when the value judgement of carrying out about safety, judgement should be carried out with following method.
(a) when a third party applies " unhappiness " to self, will produce value and a defense response about hostility.
(b) when self this third party being applied " unhappiness ", will produce value and an attack reaction about hostility.
(c) when other people will apply " unhappiness " to this third party, and self to stand in this third party on one side the time, will produce about the value of supporting mood and react with a cooperation attack.
7, the value information of this generation is stored into feeling ability DB 162, and is used as judgement data after this.
Note, because feeling ability and thought recognition unit 150 comprise and various learning functionalities like the mankind, so individual DB 163 can be modified by setting up experience with content among the feeling ability DB 162 with keeping.
Because feeling ability and 150 outputs of thought recognition unit are based on the synthesis measuring result of digital value (for example various value), so it carries out reasoning from logic or mensuration unlike artificial intelligence.Saying exactly, is the feeling ability information that the inspiration measurement function by this computing machine itself is obtained from the data D3 of feeling ability and thought recognition unit 150 outputs.
As previously mentioned, in feeling ability generation method of the present invention, produce as this Mechanical-power-producing mechanism information of the motivation that produces this mood context information (partner's mood, surrounding environment or the like) according to this input, emotional information then produces according to this this Mechanical-power-producing mechanism information.And the emotional information of this generation is controlled according to this individual information.
So it can export the mood by the reason of this individual character and purpose control, i.e. feeling ability.And, because this emotional information is by this this Mechanical-power-producing mechanism information generation, so can more accurate, more easily control the mood of this generation.
This emotion recognizing method according to the invention can be used to the mood detection of medical field, also can be used as the part of artificial intelligence or artificial feeling ability in various systems.And to the feeling ability control of a visual human or robot, feeling ability generation method of the present invention can be used to various purposes in various systems.In addition, can be by making up emotion recognizing method of the present invention and feeling ability generation method is constructed the various systems that comprise a computing machine and a person-to-person dialogue function.
The present invention is not limited to the foregoing description, can make various modifications and does not depart from spirit of the present invention and scope.All can make any improvement to some or all parts.
Claims (7)
1, a kind of emotion recognizing method that is used for the mood of detected object comprises the steps:
Import a voice signal;
According to the voice signal of above-mentioned input, survey the intensity of sound and the tempo that this sound occurs respectively, and the intonation that detects Strength Changes pattern in each word of representing this sound formation, as a time value;
The 3rd variable quantity that second variable quantity that the beat of obtain first variable quantity that the intensity of this sound that detects of indication changes along time-axis direction respectively, indicating this sound changes along time-axis direction and the intonation of this sound change along time shaft; And
Generate the signal of the emotional state of expression indignation at least, sadness and happiness respectively according to described first variable quantity, second variable quantity and the 3rd variable quantity.
2, mood detection system that is used for the mood of detected object comprises:
The sound input block is used for input audio signal;
The strength investigation unit is used for surveying according to the voice signal of tut input block input the intensity of sound;
The beat probe unit is used for surveying the speed that this sound occurs according to the voice signal of tut input block input, and with this as beat;
The intonation probe unit is used for surveying according to the voice signal of tut input block input the intonation of the word Strength Changes pattern of this sound of expression, as a time value;
The variable quantity probe unit, the 3rd variable quantity that is used for obtaining respectively indicating second variable quantity that first variable quantity that the intensity by the detected sound in described intensity detection unit changes along time-axis direction, indication change along time-axis direction by the beat of detected this sound of beat detecting unit and the intonation that detects this sound by described intonation detecting unit to change along time shaft; And
The mood detection unit is used for the signal according to the emotional state of being exported expression indignation at least, sadness and happiness by detected above-mentioned first, second and third variable quantity of described change amount detection respectively.
3, mood detection as claimed in claim 2 system, wherein above-mentioned intonation probe unit comprises:
Band-pass filter unit is used for extracting specific frequency component from the voice signal that separates input for each word;
The zone separative element is used for according to the intensity of the power spectrum of the signal that is extracted by above-mentioned band-pass filter unit this power spectrum being separated into many zones; And
The intonation computing unit is used for calculating the intonation value according to the time interval between the center separately in these many zones of being separated by the above-mentioned zone separative element.
4, mood detection as claimed in claim 2 system also comprises:
Image-generating unit is used to receive at least an image information that face is relevant with this object;
Image identification unit is used for surveying each part location information related with this face from the image information that is received by above-mentioned image-generating unit;
Image reference information is preserved the unit, is used to preserve the reference information relevant with the characteristic quantity of this each part of face; And
The image feature amount probe unit, be used for according to the positional information that detects by above-mentioned image identification unit and preserve the reference information of preserving the unit by above-mentioned image reference information and come the detection image characteristic quantity, and emotional state is estimated according to the variation in the image feature amount that is detected by above-mentioned image feature amount probe unit in wherein above-mentioned mood detection unit.
5, mood detection as claimed in claim 2 system also comprises:
The emotional information storage unit is used for continuously receiving the various information relevant with the emotional state that is detected by above-mentioned mood detection unit, and is used for storing therein various information; And
Forget processing unit, be used for from being stored in various information in the above-mentioned emotional information storage unit, relevant with emotional state in the past, deletion has stored the information of a predetermined period since this information was stored originally, and is used for from the above-mentioned variable quantity of wanting deleted information to get rid of at least to illustrate the mood greater than the information of a scheduled volume and the information that conforms to a predetermined variation pattern.
6, mood detection as claimed in claim 5 system also comprises:
The statement recognition unit is used for carrying out grammatical analysis by handling the sound that sends with this object and any relevant information of character of this object input, and is used to generate the verbal information of the implication of representing statement; And
Memory control unit is used for storing the verbal information that the above-mentioned statement recognition unit by this emotional information storage unit generates according to the mode with the information synchronization that relates to above-mentioned emotional state.
7, mood detection as claimed in claim 2 system also comprises:
Noiseless timing unit is used for measuring a noiseless time of reference according to the emotional state of these emotional states that detect; And
Statement segmentation probe unit is used for surveying by the noiseless time of reference of utilizing above-mentioned noiseless timing unit to measure the statement segmentation of this sound.
Applications Claiming Priority (4)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| JP2000278397A JP3676969B2 (en) | 2000-09-13 | 2000-09-13 | Emotion detection method, emotion detection apparatus, and recording medium |
| JP278397/2000 | 2000-09-13 | ||
| JP7726/2001 | 2001-01-16 | ||
| JP2001007726A JP3676981B2 (en) | 2001-01-16 | 2001-01-16 | KANSEI GENERATION METHOD, KANSEI GENERATION DEVICE, AND SOFTWARE |
Related Parent Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CNB018154557A Division CN100370414C (en) | 2000-09-13 | 2001-09-04 | Emotion recognizing method, sensibility creating method, system, and software |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN1838237A true CN1838237A (en) | 2006-09-27 |
| CN1838237B CN1838237B (en) | 2010-04-28 |
Family
ID=18763572
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2005100590616A Expired - Lifetime CN1838237B (en) | 2000-09-13 | 2001-09-04 | Emotion recognizing method and system |
Country Status (2)
| Country | Link |
|---|---|
| JP (1) | JP3676969B2 (en) |
| CN (1) | CN1838237B (en) |
Cited By (26)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101645961A (en) * | 2008-08-06 | 2010-02-10 | 深圳富泰宏精密工业有限公司 | Mobilephone and method for achieving caller emotion identification |
| CN103425247A (en) * | 2013-06-04 | 2013-12-04 | 深圳市中兴移动通信有限公司 | User reaction based control terminal and information processing method thereof |
| CN103543979A (en) * | 2012-07-17 | 2014-01-29 | 联想(北京)有限公司 | Voice outputting method, voice interaction method and electronic device |
| CN103720210A (en) * | 2014-01-10 | 2014-04-16 | 宁波多尔贝家居制品实业有限公司 | Family atmosphere adjustment wardrobe and method for controlling work of family atmosphere adjustment wardrobe |
| CN103745575A (en) * | 2014-01-10 | 2014-04-23 | 宁波多尔贝家居制品实业有限公司 | Family atmosphere regulating device and work control method thereof |
| CN103829958A (en) * | 2014-02-19 | 2014-06-04 | 广东小天才科技有限公司 | Method and device for monitoring human emotion |
| CN104939810A (en) * | 2014-03-25 | 2015-09-30 | 上海斐讯数据通信技术有限公司 | Method and device for controlling emotion |
| CN105339926A (en) * | 2013-08-06 | 2016-02-17 | 英特尔公司 | Emotion-related query processing |
| CN105596016A (en) * | 2015-12-23 | 2016-05-25 | 王嘉宇 | Human body psychological and physical health monitoring and managing device and method |
| CN105723360A (en) * | 2013-09-25 | 2016-06-29 | 英特尔公司 | Improving natural language interactions using emotional modulation |
| CN105792758A (en) * | 2013-12-05 | 2016-07-20 | Ｐｓｔ株式会社 | Estimation device, program, estimation method, and estimation system |
| CN106331586A (en) * | 2015-06-16 | 2017-01-11 | 杭州萤石网络有限公司 | Smart household video monitoring method and system |
| CN106537294A (en) * | 2016-06-29 | 2017-03-22 | 深圳狗尾草智能科技有限公司 | Method, system and robot for generating interactive content of robot |
| CN106575500A (en) * | 2014-09-25 | 2017-04-19 | 英特尔公司 | Method and apparatus to synthesize voice based on facial structures |
| CN106851478A (en) * | 2017-02-10 | 2017-06-13 | 深圳市笨笨机器人有限公司 | Multi-channel information processing method and system |
| CN106874265A (en) * | 2015-12-10 | 2017-06-20 | 深圳新创客电子科技有限公司 | A kind of content outputting method matched with user emotion, electronic equipment and server |
| WO2017166994A1 (en) * | 2016-03-31 | 2017-10-05 | 深圳光启合众科技有限公司 | Cloud-based device and operating method therefor |
| CN108093526A (en) * | 2017-12-28 | 2018-05-29 | 美的智慧家居科技有限公司 | Control method, device and the readable storage medium storing program for executing of LED light |
| WO2018188097A1 (en) * | 2017-04-14 | 2018-10-18 | 格兰比圣（深圳）科技有限公司 | Method and device for regulating family atmosphere |
| CN108682419A (en) * | 2018-03-30 | 2018-10-19 | 京东方科技集团股份有限公司 | Sound control method and equipment, computer readable storage medium and equipment |
| CN108694169A (en) * | 2018-07-27 | 2018-10-23 | 海南大学 | Individualized emotion visualization system based on content and environment |
| CN109009170A (en) * | 2018-07-20 | 2018-12-18 | 深圳市沃特沃德股份有限公司 | Detect the method and apparatus of mood |
| CN110246519A (en) * | 2019-07-25 | 2019-09-17 | 深圳智慧林网络科技有限公司 | Emotion identification method, equipment and computer readable storage medium |
| CN110970017A (en) * | 2018-09-27 | 2020-04-07 | 北京京东尚科信息技术有限公司 | Human-computer interaction method and system and computer system |
| CN112656401A (en) * | 2019-10-15 | 2021-04-16 | 梅州市青塘实业有限公司 | Intelligent monitoring method, device and equipment |
| CN112765875A (en) * | 2021-01-04 | 2021-05-07 | 山东师范大学 | Crowd evacuation simulation method and system considering influence of knowledge propagation on depolarization emotions |
Families Citing this family (32)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20040038419A (en) * | 2002-11-01 | 2004-05-08 | 에스엘투(주) | A method and apparatus for recognizing emotion from a speech |
| JP4546767B2 (en) * | 2004-06-09 | 2010-09-15 | 日本放送協会 | Emotion estimation apparatus and emotion estimation program |
| JP4871552B2 (en) * | 2004-09-10 | 2012-02-08 | パナソニック株式会社 | Information processing terminal |
| JP2006178063A (en) * | 2004-12-21 | 2006-07-06 | Toyota Central Res & Dev Lab Inc | Interactive processing device |
| JP4670431B2 (en) * | 2005-03-30 | 2011-04-13 | 日本電気株式会社 | Mobile terminal with mail creation function by voice emotion recognition, portable terminal system, and mail creation method by voice emotion recognition |
| JP4728868B2 (en) * | 2006-04-18 | 2011-07-20 | 日本電信電話株式会社 | Response evaluation apparatus, method, program, and recording medium |
| KR100790494B1 (en) | 2006-06-22 | 2008-01-02 | 중앙대학교 산학협력단 | System for recognizing emotion and method for selecting feature set therein |
| US8386257B2 (en) | 2006-09-13 | 2013-02-26 | Nippon Telegraph And Telephone Corporation | Emotion detecting method, emotion detecting apparatus, emotion detecting program that implements the same method, and storage medium that stores the same program |
| JP2008076905A (en) * | 2006-09-22 | 2008-04-03 | Univ Of Tokyo | Feeling discrimination method |
| JP4941966B2 (en) * | 2006-09-22 | 2012-05-30 | 国立大学法人 東京大学 | Emotion discrimination method, emotion discrimination device, atmosphere information communication terminal |
| KR101009604B1 (en) * | 2007-06-16 | 2011-01-20 | 김길호 | Methode for recognizing sensibility using musical elements |
| JP5244627B2 (en) * | 2009-01-21 | 2013-07-24 | Ｋｄｄｉ株式会社 | Emotion estimation method and apparatus |
| JP4992925B2 (en) * | 2009-03-23 | 2012-08-08 | トヨタ自動車株式会社 | Spoken dialogue apparatus and program |
| JP5146434B2 (en) * | 2009-10-05 | 2013-02-20 | 株式会社ナカヨ通信機 | Recording / playback device |
| JP2011154665A (en) * | 2010-01-25 | 2011-08-11 | Opn Corp | Expression change analysis system |
| JP5834449B2 (en) | 2010-04-22 | 2015-12-24 | 富士通株式会社 | Utterance state detection device, utterance state detection program, and utterance state detection method |
| JP5477153B2 (en) * | 2010-05-11 | 2014-04-23 | セイコーエプソン株式会社 | Service data recording apparatus, service data recording method and program |
| JP5602653B2 (en) * | 2011-01-31 | 2014-10-08 | インターナショナル・ビジネス・マシーンズ・コーポレーション | Information processing apparatus, information processing method, information processing system, and program |
| JP5708155B2 (en) | 2011-03-31 | 2015-04-30 | 富士通株式会社 | Speaker state detecting device, speaker state detecting method, and computer program for detecting speaker state |
| JP5685488B2 (en) * | 2011-05-19 | 2015-03-18 | 佐藤 英典 | Telephone device and control method thereof |
| JP5772448B2 (en) * | 2011-09-27 | 2015-09-02 | 富士ゼロックス株式会社 | Speech analysis system and speech analysis apparatus |
| US8751042B2 (en) | 2011-12-14 | 2014-06-10 | Toyota Motor Engineering & Manufacturing North America, Inc. | Methods of robot behavior generation and robots utilizing the same |
| JP6105337B2 (en) * | 2013-03-14 | 2017-03-29 | 日本写真印刷株式会社 | Evaluation system and evaluation method |
| JP6122816B2 (en) | 2014-08-07 | 2017-04-26 | シャープ株式会社 | Audio output device, network system, audio output method, and audio output program |
| JP6392578B2 (en) * | 2014-08-11 | 2018-09-19 | オリンパス株式会社 | Audio processing apparatus, audio processing method, and audio processing program |
| JP6601214B2 (en) * | 2015-12-25 | 2019-11-06 | ブラザー工業株式会社 | Music selection device, karaoke system, and music selection program |
| JP6759927B2 (en) | 2016-09-23 | 2020-09-23 | 富士通株式会社 | Utterance evaluation device, utterance evaluation method, and utterance evaluation program |
| KR102651253B1 (en) * | 2017-03-31 | 2024-03-27 | 삼성전자주식회사 | An electronic device for determining user's emotions and a control method thereof |
| CN110555346A (en) * | 2018-06-01 | 2019-12-10 | 杭州海康威视数字技术股份有限公司 | Driver emotion detection method and device, electronic equipment and storage medium |
| JP6641045B1 (en) * | 2019-02-27 | 2020-02-05 | みんとる合同会社 | Content generation system and content generation method |
| CN111341417A (en) * | 2020-02-11 | 2020-06-26 | 山西泉新科技有限公司 | Computer social cognitive assessment and correction system |
| US20240273946A1 (en) * | 2021-06-11 | 2024-08-15 | Life Quest Inc. | Emotion inferring apparatus, emotion inferring method and program |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH0512023A (en) * | 1991-07-04 | 1993-01-22 | Omron Corp | Emotion recognizing device |
| JPH06139044A (en) * | 1992-10-28 | 1994-05-20 | Sony Corp | Interface method and device |
| JPH08339446A (en) * | 1995-06-09 | 1996-12-24 | Sharp Corp | Interactive system |
| JPH10143351A (en) * | 1996-11-13 | 1998-05-29 | Sharp Corp | Interface unit |
| US6185534B1 (en) * | 1998-03-23 | 2001-02-06 | Microsoft Corporation | Modeling emotion and personality in a computer user interface |
| JP2000200103A (en) * | 1998-08-06 | 2000-07-18 | Yamaha Motor Co Ltd | Control method for object to be controlled using pseudo feeling and pseudo character, autonomous device operated by being adapted to user and method for adapting action of device to feature of user |
| JP2000187435A (en) * | 1998-12-24 | 2000-07-04 | Sony Corp | Information processing device, portable apparatus, electronic pet device, recording medium with information processing procedure recorded thereon, and information processing method |
-
2000
- 2000-09-13 JP JP2000278397A patent/JP3676969B2/en not_active Expired - Lifetime
-
2001
- 2001-09-04 CN CN2005100590616A patent/CN1838237B/en not_active Expired - Lifetime
Cited By (33)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101645961A (en) * | 2008-08-06 | 2010-02-10 | 深圳富泰宏精密工业有限公司 | Mobilephone and method for achieving caller emotion identification |
| CN103543979A (en) * | 2012-07-17 | 2014-01-29 | 联想(北京)有限公司 | Voice outputting method, voice interaction method and electronic device |
| CN103425247A (en) * | 2013-06-04 | 2013-12-04 | 深圳市中兴移动通信有限公司 | User reaction based control terminal and information processing method thereof |
| CN105339926A (en) * | 2013-08-06 | 2016-02-17 | 英特尔公司 | Emotion-related query processing |
| CN105723360A (en) * | 2013-09-25 | 2016-06-29 | 英特尔公司 | Improving natural language interactions using emotional modulation |
| US10485467B2 (en) | 2013-12-05 | 2019-11-26 | Pst Corporation, Inc. | Estimation device, program, estimation method, and estimation system |
| CN105792758B (en) * | 2013-12-05 | 2019-05-03 | Ｐｓｔ株式会社 | Estimating device, recording medium and deduction system |
| CN105792758A (en) * | 2013-12-05 | 2016-07-20 | Ｐｓｔ株式会社 | Estimation device, program, estimation method, and estimation system |
| CN103745575B (en) * | 2014-01-10 | 2016-04-20 | 宁波多尔贝家居制品实业有限公司 | A kind of family climate regulating device and operation control method thereof |
| CN103745575A (en) * | 2014-01-10 | 2014-04-23 | 宁波多尔贝家居制品实业有限公司 | Family atmosphere regulating device and work control method thereof |
| CN103720210A (en) * | 2014-01-10 | 2014-04-16 | 宁波多尔贝家居制品实业有限公司 | Family atmosphere adjustment wardrobe and method for controlling work of family atmosphere adjustment wardrobe |
| CN103720210B (en) * | 2014-01-10 | 2016-04-27 | 宁波多尔贝家居制品实业有限公司 | A kind of family climate regulates wardrobe and operation control method thereof |
| CN103829958A (en) * | 2014-02-19 | 2014-06-04 | 广东小天才科技有限公司 | Method and device for monitoring human emotion |
| CN104939810A (en) * | 2014-03-25 | 2015-09-30 | 上海斐讯数据通信技术有限公司 | Method and device for controlling emotion |
| CN106575500A (en) * | 2014-09-25 | 2017-04-19 | 英特尔公司 | Method and apparatus to synthesize voice based on facial structures |
| CN106331586A (en) * | 2015-06-16 | 2017-01-11 | 杭州萤石网络有限公司 | Smart household video monitoring method and system |
| CN106874265A (en) * | 2015-12-10 | 2017-06-20 | 深圳新创客电子科技有限公司 | A kind of content outputting method matched with user emotion, electronic equipment and server |
| CN106874265B (en) * | 2015-12-10 | 2021-11-26 | 深圳新创客电子科技有限公司 | Content output method matched with user emotion, electronic equipment and server |
| CN105596016A (en) * | 2015-12-23 | 2016-05-25 | 王嘉宇 | Human body psychological and physical health monitoring and managing device and method |
| WO2017166994A1 (en) * | 2016-03-31 | 2017-10-05 | 深圳光启合众科技有限公司 | Cloud-based device and operating method therefor |
| CN106537294A (en) * | 2016-06-29 | 2017-03-22 | 深圳狗尾草智能科技有限公司 | Method, system and robot for generating interactive content of robot |
| CN106851478A (en) * | 2017-02-10 | 2017-06-13 | 深圳市笨笨机器人有限公司 | Multi-channel information processing method and system |
| WO2018188097A1 (en) * | 2017-04-14 | 2018-10-18 | 格兰比圣（深圳）科技有限公司 | Method and device for regulating family atmosphere |
| CN108093526A (en) * | 2017-12-28 | 2018-05-29 | 美的智慧家居科技有限公司 | Control method, device and the readable storage medium storing program for executing of LED light |
| CN108682419A (en) * | 2018-03-30 | 2018-10-19 | 京东方科技集团股份有限公司 | Sound control method and equipment, computer readable storage medium and equipment |
| CN109009170A (en) * | 2018-07-20 | 2018-12-18 | 深圳市沃特沃德股份有限公司 | Detect the method and apparatus of mood |
| CN108694169A (en) * | 2018-07-27 | 2018-10-23 | 海南大学 | Individualized emotion visualization system based on content and environment |
| CN110970017A (en) * | 2018-09-27 | 2020-04-07 | 北京京东尚科信息技术有限公司 | Human-computer interaction method and system and computer system |
| CN110246519A (en) * | 2019-07-25 | 2019-09-17 | 深圳智慧林网络科技有限公司 | Emotion identification method, equipment and computer readable storage medium |
| CN112656401A (en) * | 2019-10-15 | 2021-04-16 | 梅州市青塘实业有限公司 | Intelligent monitoring method, device and equipment |
| CN112656401B (en) * | 2019-10-15 | 2023-08-22 | 梅州市青塘实业有限公司 | Intelligent monitoring method, device and equipment |
| CN112765875A (en) * | 2021-01-04 | 2021-05-07 | 山东师范大学 | Crowd evacuation simulation method and system considering influence of knowledge propagation on depolarization emotions |
| CN112765875B (en) * | 2021-01-04 | 2022-07-22 | 山东师范大学 | Crowd evacuation simulation method and system considering influence of knowledge propagation on vital emotion |
Also Published As
| Publication number | Publication date |
|---|---|
| JP3676969B2 (en) | 2005-07-27 |
| JP2002091482A (en) | 2002-03-27 |
| CN1838237B (en) | 2010-04-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN1838237A (en) | Emotion recognizing method and system | |
| CN1455916A (en) | Emotion recognizing method, sensibility creating method, system, and software | |
| CN1199149C (en) | Dialogue processing equipment, method and recording medium | |
| US10579912B2 (en) | User registration for intelligent assistant computer | |
| CN1194337C (en) | Voice identifying apparatus and method, and recording medium with recorded voice identifying program | |
| CN1290034C (en) | Robot device and behavoir control method for robot device | |
| JP7070544B2 (en) | Learning device, learning method, speech synthesizer, speech synthesizer | |
| CN1168068C (en) | Speech synthesizing system and speech synthesizing method | |
| US20210249035A1 (en) | Emotion detection using speaker baseline | |
| CN1462428A (en) | Sound processing apparatus | |
| CN1244902C (en) | Voice recognition apparatus and voice recognition method | |
| CN1310207C (en) | System and method for multi-modal focus detection, referential ambiguity resolution and mood classification using multi-modal input | |
| CN1637740A (en) | Conversation control apparatus, and conversation control method | |
| US8606580B2 (en) | Speech data process unit and speech data process unit control program for speech recognition | |
| CN1461463A (en) | Voice synthesis device | |
| CN1409527A (en) | Terminal device, server and voice identification method | |
| CN1196103C (en) | Voice identifying apparatus and method, and recording medium with recorded voice identifying program | |
| CN1474379A (en) | Voice identfying/responding system, voice/identifying responding program and its recording medium | |
| JP2012093972A (en) | Conversation processing device | |
| CN1302056A (en) | Information processing equiopment, information processing method and storage medium | |
| CN1453767A (en) | Speech recognition apparatus and speech recognition method | |
| CN1507617A (en) | Learning apparatus, learning method, and robot apparatus | |
| CN1282151C (en) | Speech recognition system, method and recording medium of recording speech recognition program | |
| JP6462936B1 (en) | Speech recognition system and speech recognition device | |
| CN1647069A (en) | Conversation control system and conversation control method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| REG | Reference to a national code |
Ref country code: HK Ref legal event code: DE Ref document number: 1096483 Country of ref document: HK |
|
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| REG | Reference to a national code |
Ref country code: HK Ref legal event code: GR Ref document number: 1096483 Country of ref document: HK |
|
| ASS | Succession or assignment of patent right |
Owner name: AGI CO., LTD. Free format text: FORMER OWNER: KYOWA MACHINERY CO., LTD. A G I Effective date: 20121129 |
|
| C41 | Transfer of patent application or patent right or utility model | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20121129 Address after: Tokyo, Japan, Japan Patentee after: KK AGI Address before: Tokyo, Japan Patentee before: A.G.I. Inc. |
|
| EE01 | Entry into force of recordation of patent licensing contract |
Application publication date: 20060927 Assignee: PST CORP INC Assignor: Hikaruyoshi Junji|KK, AGI Contract record no.: 2013990000856 Denomination of invention: Emotion recognizing method and system Granted publication date: 20100428 License type: Exclusive License Record date: 20131217 |
|
| LICC | Enforcement, change and cancellation of record of contracts on the licence for exploitation of a patent or utility model | ||
| CX01 | Expiry of patent term |
Granted publication date: 20100428 |
|
| CX01 | Expiry of patent term |