Abstract
In this paper, we propose an adaptation of four common appraisal dimensions that evaluate the relation of an agent with its environment into reward features within an intrinsically motivated reinforcement learning framework. We show that, by optimizing the relative weights of such features for a given environment, the agents attain a greater degree of fitness while overcoming some of their perceptual limitations. This optimization process resembles the evolutionary adaptive process that living organisms are subject to. We illustrate the application of our method in several simulated foraging scenarios.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Aberdeen, D.: A (revised) survey of approximate methods for solving partially observable Markov decision processes. Technical report, NICTA (2003)
Ahn, H., Picard, R.: Affective cognitive learning and decision making: The role of emotions. In: EMCSR 2006: The 18th Europ. Meet. on Cyber. and Syst. Res. (2006)
Broekens, D.: Affect and learning: a computational analysis. Doctoral Thesis, Leiden University (2007)
Cardinal, R., Parkinson, J., Hall, J., Everitt, B.: Emotion and motivation: The role of the amygdala, ventral striatum, and prefrontal cortex. Neuroscience and Biobehavioral Reviews 26(3), 321–352 (2002)
Dawkins, M.: Animal minds and animal emotions. American Zoologist 40(6), 883–888 (2000)
El-Nasr, M., Yen, J., Ioerger, T.: FLAME - Fuzzy logic adaptive model of emotions. Auton. Agents and Multiagent Systems 3(3), 219–257 (2000)
Ellsworth, P., Scherer, K.: Appraisal processes in emotion. In: Handbook of Affective Sciences, pp. 572–595. Oxford University Press, Oxford (2003)
Gadanho, S., Hallam, J.: Robot learning driven by emotions. Adaptive Behavior 9(1), 42–64 (2001)
Leventhal, H., Scherer, K.: The relationship of emotion to cognition: A functional approach to a semantic controversy. Cognition & Emotion 1(1), 3–28 (1987)
Littman, M.: Memoryless policies: Theoretical limitations and practical results. From Animals to Animats 3, 238–245 (1994)
Moore, A., Atkeson, C.: Prioritized sweeping: Reinforcement learning with less data and less real time. Machine Learning 13, 103–130 (1993)
Ng, A., Harada, D., Russel, S.: Policy invariance under reward transformations: Theory and application to reward shaping. In: Proc. 16th Int. Conf. Machine Learning, pp. 278–287 (1999)
Niekum, S., Barto, A., Spector, L.: Genetic programming for reward function search. IEEE Trans. Autonomous Mental Development 2(2), 83–90 (2010)
Salichs, M., Malfaz, M.: Using emotions on autonomous agents. The role of Happiness, Sadness and Fear. In: AISB 2006: Adaption in Artificial and Biological Systems, pp. 157–164 (2006)
Singh, S., Jaakkola, T., Jordan, M.: Learning without state-estimation in partially observable Markovian decision processes. In: Proc. 11th Int. Conf. Machine Learning, pp. 284–292 (1994)
Singh, S., Lewis, R., Barto, A., Sorg, J.: Intrinsically motivated reinforcement learning: An evolutionary perspective. IEEE Trans. Autonomous Mental Development 2(2), 70–82 (2010)
Sorg, J., Singh, S., Lewis, R.: Internal rewards mitigate agent boundedness. In: Proc. 27th Int. Conf. Machine Learning, pp. 1007–1014 (2010)
Sutton, R., Barto, A.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (1998)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Sequeira, P., Melo, F.S., Paiva, A. (2011). Emotion-Based Intrinsic Motivation for Reinforcement Learning Agents. In: D’Mello, S., Graesser, A., Schuller, B., Martin, JC. (eds) Affective Computing and Intelligent Interaction. ACII 2011. Lecture Notes in Computer Science, vol 6974. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-24600-5_36
Download citation
DOI: https://doi.org/10.1007/978-3-642-24600-5_36
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-24599-2
Online ISBN: 978-3-642-24600-5
eBook Packages: Computer ScienceComputer Science (R0)