Abstract
When used as an interface in the context of Ambient Assisted Living (AAL), a social robot should not just provide a task-oriented support. It should also try to establish a social empathic relation with the user. To this aim, it is crucial to endow the robot with the capability of recognizing the user’s affective state and reason on it for triggering the most appropriate communicative behavior. In this paper we describe how such an affective reasoning has been implemented in the NAO robot for simulating empathic behaviors in the context of AAL. In particular, the robot is able to recognize the emotion of the user by analyzing communicative signals extracted from speech and facial expressions. The recognized emotion allows triggering the robot’s affective state and, consequently, the most appropriate empathic behavior. The robot’s empathic behaviors have been evaluated both by experts in communication and through a user study aimed at assessing the perception and interpretation of empathy by elderly users. Results are quite satisfactory and encourage us to further extend the social and affective capabilities of the robot.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Notes
The name of the project, NICA, is also used to indicate the robot.
In this work, since the robot had to deal with European elderly people, we used color combinations that are suitable for the European culture. Of course, different cultures may require different colors for conveying the same feelings, but this does not change the basics of our approach (a different color can be easily used for each culture).
References
Anderson C, Keltner D (2002) The role of empathy in the formation and maintenance of social bonds. Behav Brain Sci 25:21–22
Baron-Cohen S (2011) The science of evil: on empathy and the origins of cruelty. Basic Books, New York, NY
Beadle JN, Sheehan AH, Dahlben B, Gutchess AH (2013) Aging, empathy, and prosociality. J Gerontol Ser B Psychol Sci Soc Sci:1–10
Boersma P, Weenink D (2007) Praat: doing phonetics by computer (version 4.5.15) [computer program]. http://www.praat.org/. Retrieved 24.02.2007
Broekens J, Heerink M, Rosendal H (2009) Assistive social robots in elderly care: a review. Gerontechnology 8(2):94–103. doi:10.4017/gt.2009.08.02.002.00
Cesta A, Cortellessa G, Pecora F, Rasconi R (2007) Supporting interaction in the robocare intelligent assistive environment, AAAI Spring Symposium: Interaction Challenges for Intelligent Assistants, pp. 18–25
Coradeschi S, Cesta A, Cortellessa G, Coraci L, Galindo C, Gonzalez J, Karlsson L, Forsberg A, Frennert S, Furfari F, Loutfi A, Orlandini A, Palumbo F, Pecora F, von Rump S, Stimec A, Ullberg J, Ötslund B (2014) GiraffPlus: a system for monitoring activities and physiological parameters and promoting social interaction for elderly in human-computer systems interaction: backgrounds and applications 3. Adv Intell Syst Comput 300:261–271. doi:10.1007/978-3-319-08491-6_22
Cramer H, Goddijn J, Wielinga B, Evers V (2010) Effects of (In) accurate empathy and situational valence on attitudes towards robots. ACM/IEEE International Conference on Human Robot Interaction (HRI). 141-2. New York. March 2005
Davis M (1980) A multidimensional approach to individual differences in empathy. Cat Sel Doc Psychol 10:85
De Carolis B, Ferilli S, Greco D (2013) Towards a caring home for assisted living. In: Proceedings of the workshop “the challenge of ageing society: technological roles and opportunities for artificial intelligence” co-located with the 13th Conference of the Italian Association for Artificial Intelligence (AI*IA 2013), Turin, 6 Dec 2013. http://ceur-ws.org/Vol-1122/
De Carolis B, Mazzotta I, Novielli N, Pizzutilo S (2013) User modeling in social interaction with a Caring Agent. In: UMADR - User modeling and adaptation for daily routines, Springer HCI Series. Springer
de Rosis F, De Carolis B, Carofiglio V, Pizzutilo S (2003) Shallow and inner forms of emotional intelligence in advisory dialog simulation. In: Prendinger H, Ishizuka M (eds) “Life-like characters. Tools, affective functions and applications. pp 271-294. Springer
Eisenberg N, Miller PA (1987) The relation of empathy to prosocial and related behaviors. Psychol Bull 101:91–119
Ekman P, Oster H (1979) Facial expressions of emotion. Annu Rev Psychol 30:527–554
Elliott C, Siegle G (1993) Variables influencing the intensity of simulated affective states. In Proceedings of the AAAI Spring Symposium on Mental States.’93:58-67
Graf B, Hans M, Shaft RD (2004) Care-O-bot II – development of a next generation robotics home assistant. Auton Robots 16(2):193–205
Hoffman ML (1981) The development of empathy. In: Rushton J, Sorrentino R (eds) Altruism and helping behavior: Social personality and developmental perspectives. Erlbaum, Hillsdale, pp. 41–63
Jensen FV (2001) Bayesian networks and decision graphs, statistics for engineering and information science. Springer New York, New York, doi:10.1007/978-1-4757-3502-4
Jiang H, Vidal JM, Huhns MN (2007) EBDI: an architecture for emotional agents, Proceedings of the 6th international joint conference on Autonomous agents and multiagent systems, May 14-18, Honolulu, Hawaii, ACM, p. 11
Johnson DO, Cuijpers RH, van der Pol D (2013) Imitating human emotions with artificial facial expressions. Int J Soc Robot 5(4):503–513
Klein J, Moon Y, Picard R (1999) This computer responds to user frustration. In Proceedings of the Conference on Human Factors in Computing Systems. Pittsburgh, USA, ACM Press, pp 242–243
Leite I, Martinho C, Paiva A (2013) Social robots for long-term interaction: a survey. Int J Soc Robot. doi:10.1007/s12369-013-0178-y
Mataric M, Tapus A, Feil-Seifer D (2007) Personalized socially assistive robotics, workshop on intelligent systems for assisted cognition, rochester, New York, USA
Mehrabian A (1968) Communication without words. Psychol Today 2:53–56
Milborrow S, Nicolls F (2008) Locating facial features with an extended active shape model. In Computer Vision–ECCV 2008 (pp. 504-513). Springer Berlin Heidelberg
Milborrow S, Nicolls F (2014) Active shape models with SIFT descriptors and MARS, In Proceedings of the 9th International Conference on Computer Vision Theory and Applications, Lisbon, pp. 5–8
Nehmer J, Becker M, Karshmer A, Lamm R (2006) Living assistance systems: an ambient intelligence approach, in International Conference on Software Engineering, ACM, pp. 43–50
Niewiadomski R, Ochs M, Pelachaud C (2008) Expressions of empathy in ECAs, in Proceedings of the 8t Int.Conf. on IVA, LNAI, vol. 5208. Springer-Verlag, Berlin, Heidelberg, pp 37-44
Nijholt A, de Ruyter B, Heylen D, Privender S (2006) Social interfaces for ambient intelligence environments. Chapter 14. In: Aarts E, Encarnaçao J (eds) True visions: the emergence of ambient intelligence. Springer, New York, pp. 275–289
Oatley K, Johnson-Laird PN (1987) Towards a cognitive theory of emotions. Cognit Emot 29–50
Ortony A, Clore GL, Collins A (1988) The cognitive structure of emotions. Cambridge University Press
Paiva A, Dias J, Sobral D, Aylett R, Sobreperez P, Woods S, Hall L (2004) Caring for agents and agents that care: building empathic relations with synthetic agents. In Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems-Volume 1 (pp. 194-201). IEEE Computer Society., ACM Press
Paiva A et al. (2011) Empathy in social agents. Int J sVirtual Real 10(1):65–68
Palestra G, Pettinicchio A, Del Coco M, Carcagnì P, Leo M, Distante C (2015) Improved performance in facial expression recognition using 32 geometric features. ICIAP, lecture notes in computer science, Springer, pp. 518–528
Picard R (1997) Affective computing. MIT Press, Cambridge, MA
Pineau J, Montemerlo M, Pollack M, Roy N, Thrun S (2003) Towards robotic assistants in nursing homes: challenges and results, robotics and autonomous systems 42(3–4):271–281
Reeves B, Nass C (1996) The media equation: how people treat computers, television, and new media like real people and places. Cambridge University Press, New York, NY
Reiter E, Dale R (2000) Building natural language generation systems. Studies in natural language processing. Cambridge University Press, Cambridge, United Kingdom. ISBN 0-521-62036-8
Russell J (1980) A circumplex model of affect. J Pers Soc Psychol 39:1161–1178
Speigel C. Greczek J, Matarić M (2016) Creating a baseline for empathy in socially assistive robotics. http://robotics.usc.edu/~cspiegel/naoempathy4.pdf
van Ruiten AM, Haitas D, Bingley P, Hoonhout HCM, Meerbeek BW, Terken JMB (2007) Attitude of elderly towards a robotic game-and-train- buddy: evaluation of empathy and objective control. In: Cowie R, de Rosis F (eds) Proceedings of the Doctoral consortium, in the scope of ACII2007 Conference
Viola PA, Jones MJ (2004) Robust real-time face detection. Int J Comput Vis 57(2):137–154
Vogt T, Andrié E, Bee N (2008) EmoVoice - a framework for online recognition of emotions from voice. In Proceedings of the 4th IEEE tutorial and research workshop on Perception and Interactive Technologies for Speech-Based Systems: Perception in Multimodal Dialogue Systems (PIT '08), Springer-Verlag, Berlin, 188-199
Acknowledgments
This work fulfills the research objectives of the PON02_00563_3489339 project "PUGLIA@SERVICE - funded by the Italian Ministry of University and Research (MIUR). Our thanks go to Isabella Poggi and Francesca D’Errico for their useful suggestions on the evaluation of the robot’s behaviors.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
De Carolis, B., Ferilli, S. & Palestra, G. Simulating empathic behavior in a social assistive robot. Multimed Tools Appl 76, 5073–5094 (2017). https://doi.org/10.1007/s11042-016-3797-0
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-016-3797-0