article

Free access

Learning behavior-selection by emotions and cognition in a multi-goal robot task

Author:

Sandra Clara GadanhoAuthors Info & Claims

The Journal of Machine Learning Research, Volume 4

Pages 385 - 412

Published: 01 December 2003 Publication History

PDF eReader

Abstract

The existence of emotion and cognition as two interacting systems, both with important roles in decision-making, has been recently advocated by neurophysiological research (LeDoux, 1998, Damasio, 1994. Following that idea, this paper presents the ALEC agent architecture which has both emotive and cognitive learning, as well as emotive and cognitive decision-making capabilities to adapt to real-world environments. These two learning mechanisms embody very different properties which can be related to those of natural emotion and cognition systems. The reported experiments test ALEC within a simulated autonomous robot which learns to perform a multi-goal and multi-step survival task when faced with real world conditions, namely continuous time and space, noisy sensors and unreliable actuators. Experimental results show that both systems contribute positively to the learning performance of the agent.

References

[1]

James S. Albus. The role of world modeling and value judgment in perception. In A. Meystel, J. Herath, and S. Gray, editors, Proceedings of the 5th IEEE International Symposium on Intelligent Control. Los Alamitos, CA: IEEE Computer Society Press, 1990.

Google Scholar

[2]

Minoru Asada. An agent and an environment: A view on body scheme. In Jun Tani and Minoru Asada, editors, Proceedings of the 1996 IROS Workshop on Towards real autonomy, pages 19-24, Senri Life Science Center, Osaka, Japan, 1996.

Google Scholar

[3]

Penny Baillie and Dickson Lukose. Affect appraisals for decision making in artificial intelligences. In Robert Trappl, editor, Cybernetics and Systems 2002 -- Proc. of the 16'th European Meeting on Cybernetics and Systems Research, volume 2, pages 745-750, Austria, April 2002. University of Vienna.

Google Scholar

[4]

Christian Balkenius. Natural Intelligence in artificial creatures. Lund University Cognitive Studies 37, 1995.

Google Scholar

[5]

Bruce Blumberg. Old Tricks, New Dogs: Ethology and interactive creatures. PhD thesis, MIT, 1996.

Crossref

Google Scholar

[6]

Stevo Bozinovski. A self-learning system using secondary reinforcement. In R. Trappl, editor, Cybernetics and Systems, pages 397-402. Elsevier Science Publishers, North Holland, 1982.

Google Scholar

[7]

Cynthia Breazeal. Robot in society: Friend or appliance? In Agents'99 workshop on emotion-based agent architectures, pages 18-26, Seattle, WA, 1999.

Google Scholar

[8]

Richard E. Cytowic. The man who tasted shapes. Abacus, London, 1993.

Google Scholar

[9]

Antonio Damasio. The feeling of what happens. Harcout Brace & Company, New York, 1999.

Google Scholar

[10]

Antonio R. Damasio. Descartes' error--Emotion, reason and human brain. Picador, London, 1994.

Google Scholar

[11]

Clark Elliott. The affective reasoner: A process model of emotions in a multi-agent system. PhD thesis, Northwestern University, Evanston, Illinois, 1992. Field of Computer Science.

Crossref

Google Scholar

[12]

Gérald Foliot and Olivier Michel. Learning object significance with an emotion based process. In SAB'98 Workshop on Grounding Emotions in Adaptive Systems, pages 25-30, Zurich, Switzerland, 1998.

Google Scholar

[13]

Masahiro Fujita, Rika Hasegawa, Gabriel Costa, Tsuyoshi Takagi, Jun Yokono, and Hideki Shimomura. Physically and emotionally grounded symbol acquisition for autonomous robots. In Lola Cañmero, editor, AAAI Fall Symposium on Emotional and Intelligent II: The tangled knot of social cognition, pages 43-48. Menlo Park, California: AAAI Press, 2001. Technical report FS-01-02.

Google Scholar

[14]

Sandra Clara Gadanho. Reinforcement Learning in Autonomous Robots: An Empirical Investigation of the Role of Emotions. PhD thesis, University of Edinburgh, 1999.

Google Scholar

[15]

Sandra Clara Gadanho and John Hallam. Emotion-triggered learning in autonomous robot control. Cybernetics and Systems -- Special Issue: Grounding emotions in adaptive systems, 32(5):531-559, July 2001a.

Google Scholar

[16]

Sandra Clara Gadanho and John Hallam. Robot learning driven by emotions. Adaptive Behavior, 9 (1), 2001b.

Crossref

Google Scholar

[17]

Joseph E. LeDoux. The Emotional Brain. Phoenix, London, 1998.

Google Scholar

[18]

Long-Ji Lin. Reinforcement learning for robots using neural networks. PhD thesis, Carnegie Mellon University, 1993. Technical report CMU-CS-93-103.

Crossref

Google Scholar

[19]

Márcia Maçãs, Paulo Couto, Carlos Pinto-Ferreira, Luis Custódio, and Rodrigo Ventura. Experiments with an emotion-based agent using the DARE architecture. In Proceedings of the AISB'01 Symposium on Emotion, Cognition and Affective Computing, pages 105-112, University of York, U. K., March 2001.

Google Scholar

[20]

Sridhar Mahadevan and Jonathan Connell. Automatic programming of behavior-based robots using reinforcement learning. Artificial intelligence, 55:311-365, 1992.

Crossref

Google Scholar

[21]

Yuval Marom and Gillian Hayes. Maintaining attentional capacity in a social robot. In R. Trappl, editor, Cybernetics and Systems 2000: Proceedings of the 15th European Meeting on Cybernetics and Systems Research. Symposium on Autonomy Control -- Lessons from the emotional, volume 1, pages 693-698, Vienna, Austria, April 2000.

Google Scholar

[22]

Maja J. Mataric. Reward functions for accelerated learning. In William W. Cohen and Haym Hirsh, editors, Machine Learning: Proceedings of the Eleventh International Conference, pages 181-189. San Francisco, CA: Morgan Kaufmann Publishers, 1994.

Google Scholar

[23]

Lee McCauley and Stan Franklin. An architecture for emotion. In Dolores Canamero, editor, AAAI Fall Symposium on Emotional and Intelligent: The tangled knot of cognition, Technical Report FS-98-03, pages 122-127. Menlo Park, CA: AAAI Press, 1998.

Google Scholar

[24]

Olivier Michel. Khepera Simulator package version 2.0: Freeware mobile robot simulator written at the University of Nice Sophia-Antipolis, March 1996. Downloadable from the World Wide Web at http://diwww.epfl.ch/lami/team/michel/khep-sim/.

Google Scholar

[25]

Justus H. Piater, Paul R. Cohen, Xiaoqin Zhang, and Michael Atighetchi. A randomized ANOVA procedure for comparing performance curves. In J. Shavlik, editor, Machine Learning: Proceedings of the Fifteenth International Conference, pages 430-438. Morgan Kaufmann Publishers, San Francisco, CA, 1999.

Crossref

Google Scholar

[26]

Miguel Rodriguez and Jean-Pierre Muller. Towards autonomous cognitive animats. In F. Morán, A. Moreno, J.J. Merelo, and P. Chacón, editors, Advances in artificial life -- Proceedings of the Third European Conference on Artificial Life, Lecture Notes in Artificial Intelligence Volume 929, Berlin, Germany, 1995. Springer-Verlag.

Google Scholar

[27]

Rui Sadio, Gonçalo Tavares, Rodrigo Ventura, and Luis Custódio. An emotion-based agent architecture application with real robots. In Lola Cañmero, editor, AAAI Fall Symposium on Emotional and Intelligent II: The tangled knot of social cognition, pages 117-122. Menlo Park, California: AAAI Press, 2001. Technical report FS-01-02.

Google Scholar

[28]

Mathias Scheutz. The evolution of simple affective states in multi-agent environments. In Lola Cañmero, editor, Emotional and Intelligent II: The tangled knot of social cognition, pages 123-128, Menlo Park, California, 2001. AAAI Press. Technical report FS-01-02.

Google Scholar

[29]

Magy Seif El-Nasr, John Yen, and Thomas Ioerger. Flame - a fuzzy logic adaptive model of emotions. Autonomous Agents and Multi-agent Systems, 1999.

Google Scholar

[30]

H. A. Simon. Motivational and emotional controls of cognition. Psychological Review, 74:29-39, 1967.

Google Scholar

[31]

Aaron Sloman and Monica Croucher. Why robots will have emotions. In IJCAI'81 -- Proceedings of the Seventh International Joint Conference on Artificial Intelligence, pages 2369-71, 1981. Also available as Cognitive Science Research Paper 176, Sussex University.

Google Scholar

[32]

R. Sun, E. Merrill, and T. Peterson. From implicit skills to explicit knowledge: a bottom-up model of skill learning. Cognitive Science, 25(2):203-244, 2001.

Google Scholar

[33]

R. Sun and T. Peterson. Autonomous learning of sequential tasks: experiments and analysis. IEEE Transactions on Neural Networks, 9(6): 1217-1234, November 1998.

Crossref

Google Scholar

[34]

Richard S. Sutton and Andrew G. Barto. Reinfrocement Learning. The MIT Press, 1998.

Crossref

Google Scholar

[35]

Silvan S. Tomkins. Affect theory. In Klaus R. Scherer and Paul Ekman, editors, Approaches to Emotion. Lawrence Erlbaum, London, 1984.

Google Scholar

[36]

Juan D. Velásquez. A computational framework for emotion-based control. In SAB'98 Workshop on Grounding Emotions in Adaptive Systems, pages 62-67, Zurich, Switzerland, 1998.

Google Scholar

[37]

Rodrigo Ventura and Carlos Pinto-Ferreira. Emotion-based agents: Three approaches to implementation (preliminary report). In Juan D. Velsquez, editor, Workshop on Emotion-Based Agent Architectures, Seattle, U. S. A., 1999. Workshop of the Third International Conference on Autonomous Agents.

Google Scholar

[38]

C. Watkins. Learning from delayed rewards. PhD thesis, King's College, Cambridge, 1989.

Google Scholar

[39]

Ian Wright. Reinforcement learning and animat emotions. In Pattie Maes, Maja J. Mataric, Jean-Arcady Meyer, Jordan Pollack, and Stewart W. Wilson, editors, From animals to animats 4 -- Proceedings of the Fourth International Conference on Simulation of Adaptive Behavior, pages 273-281. The MIT Press, 1996.

Google Scholar

Cited By

View all

Broekens JChetouani M(2021)Towards Transparent Robot Learning Through TDRL-Based Emotional ExpressionsIEEE Transactions on Affective Computing10.1109/TAFFC.2019.289334812:2(352-362)Online publication date: 1-Apr-2021
https://dl.acm.org/doi/10.1109/TAFFC.2019.2893348
Verma BChoudhary A(2021)Affective state recognition from hand gestures and facial expressions using Grassmann manifoldsMultimedia Tools and Applications10.1007/s11042-020-10341-680:9(14019-14040)Online publication date: 1-Apr-2021
https://dl.acm.org/doi/10.1007/s11042-020-10341-6
Doboli ADoboli S(2021)A novel agent-based, evolutionary model for expressing the dynamics of creative open-problem solving in small groupsApplied Intelligence10.1007/s10489-020-01919-651:4(2094-2127)Online publication date: 1-Apr-2021
https://dl.acm.org/doi/10.1007/s10489-020-01919-6
Show More Cited By

Index Terms

Learning behavior-selection by emotions and cognition in a multi-goal robot task
1. Computer systems organization
  1. Embedded and cyber-physical systems
    1. Robotics
      1. Robotic autonomy
2. Computing methodologies
  1. Artificial intelligence
    1. Distributed artificial intelligence
      1. Intelligent agents
      2. Multi-agent systems
  2. Machine learning
    1. Learning settings

Recommendations

Perceiving emotions in robot body language

Reliance on socioemotional assistive robots is projected to increase, yet little is known about how our ability to perceive their emotional expression is impacted by psychological factors. In high-risk and high-tension domains such as emergency services ...
Social Sharing of Emotions with Robots and the Influence of a Robot’s Nonverbal Behavior on Human Emotions
Social Robotics
Abstract
People share their emotional experiences with one another. This is called social sharing of emotions (SSE). SSE can affect one’s own positive and negative emotions. As robots become increasingly capable of engaging in conversations with people, ...
Using Emotions for Behaviour-Selection Learning
Proceedings of the 2006 conference on ECAI 2006: 17th European Conference on Artificial Intelligence August 29 -- September 1, 2006, Riva del Garda, Italy

Emotions play a very important role in human behaviour and social interaction. In this paper we present a control architecture which uses emotions in the behaviour selection process of autonomous and social agents. The state of the agent is determined ...

Reviews

Reviewer: Angelica de Antonio

ALEC, an architecture for an autonomous and adaptive robot/agent controller, is described in this paper. The controller is responsible for deciding what behavior the robot/agent should exhibit at any moment, and when to switch from one behavior to another one. The robot/agent is adaptive because it is expected to learn to master a complex task, involving multiple and possibly conflicting goals. The first part of the paper motivates the need for such an architecture, and describes its components, while the second part reports the results of several experiments comparing the performance and learning capabilities of this architecture (in three variants) with other architectures. The originality and power of ALEC lies in the combination of two components. The first component is described as an emotion system (although emotions are not explicitly represented, its design is inspired by emotion theories). This component is able to learn the long-term utility of behaviors. The second component is a cognitive system that is able to learn decision rules (suggesting behaviors with successful results in certain past situations) from the agent-environment interaction. The results of the experiments support the author's claim that emotion and cognition both play important and complementary roles in decision making, and they demonstrate the advantages of ALEC when compared to similar architectures. The paper is well written and organized, and provides sufficient background information for nonexpert readers to understand its contribution, and to get a vision of the state-of-the-art in this research area.

Access critical reviews of Computing literature here

Become a reviewer for Computing Reviews.

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image The Journal of Machine Learning Research

The Journal of Machine Learning Research Volume 4, Issue

12/1/2003

1486 pages

ISSN:1532-4435

EISSN:1533-7928

Issue’s Table of Contents

Publisher

JMLR.org

Publication History

Published: 01 December 2003

Published in JMLR Volume 4

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

22
Total Citations
View Citations
1,791
Total Downloads

Downloads (Last 12 months)56
Downloads (Last 6 weeks)7

Reflects downloads up to 24 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Broekens JChetouani M(2021)Towards Transparent Robot Learning Through TDRL-Based Emotional ExpressionsIEEE Transactions on Affective Computing10.1109/TAFFC.2019.289334812:2(352-362)Online publication date: 1-Apr-2021
https://dl.acm.org/doi/10.1109/TAFFC.2019.2893348
Verma BChoudhary A(2021)Affective state recognition from hand gestures and facial expressions using Grassmann manifoldsMultimedia Tools and Applications10.1007/s11042-020-10341-680:9(14019-14040)Online publication date: 1-Apr-2021
https://dl.acm.org/doi/10.1007/s11042-020-10341-6
Doboli ADoboli S(2021)A novel agent-based, evolutionary model for expressing the dynamics of creative open-problem solving in small groupsApplied Intelligence10.1007/s10489-020-01919-651:4(2094-2127)Online publication date: 1-Apr-2021
https://dl.acm.org/doi/10.1007/s10489-020-01919-6
Li GDibeklioğlu HWhiteson SHung H(2020)Facial feedback for reinforcement learning: a case study and offline analysis using the TAMER frameworkAutonomous Agents and Multi-Agent Systems10.1007/s10458-020-09447-w34:1Online publication date: 12-Feb-2020
https://dl.acm.org/doi/10.1007/s10458-020-09447-w
Williams JZhang X(2019)Combining affective intelligence with learning to improve action selection in decision-making agentsInternational Journal of Hybrid Intelligent Systems10.3233/HIS-18025915:1(27-53)Online publication date: 1-Jan-2019
https://dl.acm.org/doi/10.3233/HIS-180259
Masuyama NLoo CSeera M(2018)Personality affected robotic emotional model with associative memory for human-robot interactionNeurocomputing10.1016/j.neucom.2017.06.069272:C(213-225)Online publication date: 10-Jan-2018
https://dl.acm.org/doi/10.1016/j.neucom.2017.06.069
Moerland TBroekens JJonker C(2018)Emotion in reinforcement learning agents and robotsMachine Language10.1007/s10994-017-5666-0107:2(443-480)Online publication date: 1-Feb-2018
https://dl.acm.org/doi/10.1007/s10994-017-5666-0
Barros PJirak DWeber CWermter S(2015)Multimodal emotional state recognition using sequence-dependent deep hierarchical featuresNeural Networks10.1016/j.neunet.2015.09.00972:C(140-151)Online publication date: 1-Dec-2015
https://dl.acm.org/doi/10.1016/j.neunet.2015.09.009
Ma Z(2015)Towards computational models of animal cognition, an introduction for computer scientistsCognitive Systems Research10.1016/j.cogsys.2014.08.00133:C(42-69)Online publication date: 1-Mar-2015
https://dl.acm.org/doi/10.1016/j.cogsys.2014.08.001
Ramnani SGorthi RHartmann KSchuller BScherer KBöck R(2014)A Model to Incorporate Emotional Sensitivity into Human Computer InteractionsProceedings of the 2014 workshop on Emotion Representation and Modelling in Human-Computer-Interaction-Systems10.1145/2668056.2668059(25-30)Online publication date: 16-Nov-2014
https://dl.acm.org/doi/10.1145/2668056.2668059
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Abstract

References

Cited By

Index Terms

Recommendations

Perceiving emotions in robot body language

Social Sharing of Emotions with Robots and the Influence of a Robot’s Nonverbal Behavior on Human Emotions

Using Emotions for Behaviour-Selection Learning

Reviews

Access critical reviews of Computing literature here

Comments

Information

Published In

Publisher

Publication History

Qualifiers

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

View options

PDF

eReader

Login options

Full Access

Figures

Other

Share

Share this Publication link

Share on social media

Affiliations