research-article

Using electrodermal activity to recognize ease of engagement in children during social interactions

Authors:

Javier Hernandez,

Gregory D. Abowd,

Rosalind W. PicardAuthors Info & Claims

UbiComp '14: Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing

Pages 307 - 317

https://doi.org/10.1145/2632048.2636065

Published: 13 September 2014 Publication History

Abstract

The recent emergence of comfortable wearable sensors has focused almost entirely on monitoring physical activity, ignoring opportunities to monitor more subtle phenomena, such as the quality of social interactions. We argue that it is compelling to address whether physiological sensors can shed light on quality of social interactive behavior. This work leverages the use of a wearable electrodermal activity (EDA) sensor to recognize ease of engagement of children during a social interaction with an adult. In particular, we monitored 51 child-adult dyads in a semi-structured play interaction and used Support Vector Machines to automatically identify children who had been rated by the adult as more or less difficult to engage. We report on the classification value of several features extracted from the child's EDA responses, as well as several other features capturing the physiological synchrony between the child and the adult.

Supplementary Material

MOV File (p307-hernandez.mov)

Download
208.81 MB

References

[1]

Adamson, L. B., Early Interests and Joint Engagement in Typical Development, Autism, and Down Syndrome, Journal of Autism Developmental Disorders, 40 (6), (2010), 665--676.

[2]

Benedek, M., and Kaernbach, C. Decomposition of skin conductance data by means of nonnegative deconvolution. Psychophysiology, 47 (4), (2010), 647--658.

[3]

Boser, B. E., Guyon, I., and, Vapnik V. A training algorithm for optimal margin classifiers. In Proc. of the Fifth Annual Workshop on Computational Learning Theory, (1992), 144--152.

Digital Library

[4]

Boucsein W. Electrodermal Activity (2^nd Ed). New York: Springer, (2012).

[5]

Calvo, R. A., and DMello, S. Affect detection: An interdisciplinary review of models, methods, and their applications. In IEEE Transactions on Affective Computing, 1(1), (2010), 18--37.

Digital Library

[6]

Chang, C. C., and Lin, C. J. LIBSVM: a library for Support Vector Machines, 2001. Software: http://www.csie.ntu.edu.tw/~cjlin/libsvm

Digital Library

[7]

Chaspari T., Goodwin M., Wilder-Smith O., Gulsrud A., Mucchetti C., Kasari C., and Narayanan S. A Non-Homogeneous Poisson Process Model of Skin Conductance Responses Integrated with Observed Regulatory Behaviors for Autism Intervention. In Proceedings of IEEE International Conference on Audio, Speech and Signal Processing (2014).

[8]

Ekman P., and Friesen W. Facial Action Coding System: a technique for the measurement of facial movement, Palo Alto, CA: Consulting Psychologists Press, 1978.

[9]

Ekman, I., Chanel, G., Kivikangas, J. M., Salminen, M., Jarvela, S., and Ravaja, N. Social interaction in games: measuring physiological linkage and social presence. Journal Simulation and Gaming, 43(3), (2012), 321--338.

Digital Library

[10]

Feldman, R. Parent-infant synchrony and the construction of shared timing: physiological precursors, developmental outcomes, and risk conditions. Journal of Child Psychology and Psychiatry, 48 (3-4), (2007), 329--354.

[11]

Guyon I., and Elisseeff A. An introduction to variable and feature selection. Journal of Machine Learning Research, 3, (2003), 1157--1182.

Digital Library

[12]

Hatfield, E., Cacioppo, J., and Rapson, R. Emotional contagion. New York, Cambridge University Press, (1994).

[13]

Hedman E., Miller L., Schoen S., Nielsen D., Goodwin M., and Picard R. W. Measuring autonomic arousal during therapy. In Proc. of Design and Emotion, (2012), 11--14.

[14]

Hernandez, J., Liu, Z., Hulten G., DeBarr, D., Krum, K., and Zhang, Z. Measuring the engagement level of TV Viewers. In Automatic Face and Gesture Recognition, (2013), 1--7.

[15]

Hernandez, J., Morris, R. R., and Picard, R. W. Call Center Stress Recognition with Person-Specific Models. In Proc. of the Affective Computing and Intelligent Interaction, (2011), 125--134.

Digital Library

[16]

Huang, Y. M., and Du, S. X. Weighted support vector machine for classification with uneven training class sizes. International Conference on Machine Learning and Cybernetics, 7, (2005), 4365--4369.

[17]

Jo J., Lee, S. J., Jung H. G., Park, K. R., and Kim J. Vision-based method for detecting driver drowsiness and distraction in driver monitoring system. In Optical Engineering, 50 (12), (2011), 127202/1--24.

[18]

Kühn S. S., Müller B. C. N., Van der Leij A., Dijksterhuis A., Brass M., and Van Baaren R. B. Neural correlates of emotional synchrony. In Social Cognitive and Affective Neuroscience, 6 (3), (2011), 368--374.

[19]

LaBarbera, P., and Tucciarone J. GSR reconsidered: a behavior based approach to evaluating and improving the sales potency of advertising, In Journal of Advertising Research, 35 (5), (1995), 33--53.

[20]

Lang A. Involuntary attention and physiological arousal evoked by structural features and mild emotion in TV commercials. In Proceedings of Communication Research, 17 (3), (1990), 275--299.

[21]

Levenson, R. W., and Gottman, J. M. Marital interaction: physiological linkage and affective exchange. Journal of Personality and Social Psychology, 45 (3), (1983), 587--597.

[22]

Levenson, R. W., and Ruef, A. M. Physiological aspects of emotional knowledge and rapport. In W. Ickes (Eds.), Empathic Accuracy, 44--72. New York: Guilford Press, (1997).

[23]

Lykken, D. T., and Venables, P. H. Direct measurement of skin conductance: A proposal for standardization. Psychophysiology, 8 (5), (1971), 656--672.

[24]

Marci, C. D., J. Ham, E. Moran, and Orr. S. P. Physiologic Correlates of Perceived Therapist Empathy and Social-emotional Process during Psychotherapy. Journal of Nervous and Mental Disease, 195 (2), (2007), 103--111.

[25]

Mundy, P. Joint attention and social-emotional approach behavior in children with autism. Development and Psychopathology, 7 (1), (1995), 63--82.

[26]

Mundy P., and Burnette C. Joint attention and neurodevelopment. In F. Volkmar, A. Klin, and R. Paul, editors, Handbook of Autism and Pervasive Developmental Disorders. 3, 650--681. Hoboken, NJ: John Wiley, 2005.

[27]

Poels K., and Dewitte S. How to capture the heart? Reviewing 20 years of emotion measurement in advertising. Journal of Advertising Research, 46 (1), (2006).

[28]

Rehg J. M., Abowd G. D., Rozga A., Romero M., Clements M. A., Sclaroff S., Essa I., Ousley O. Y., Li Y., Kim C., Rao H., Kim J. C., Presti L. L., Zhang J., Lantsman D., Bidwell J., and Ye Z. Decoding Children's Social Behavior. In Proceedings of Computer Vision and Pattern Recognition, (2013), 3414--3421.

Digital Library

[29]

Silveira F., Eriksson B., Sheth A., and Sheppard A. Predicting audience responses to movie content from electro-dermal activity signals. In Proceedings of Pervasive and Ubiquitous Computing, (2013), 707--716.

Digital Library

[30]

Voit M., and Stiefelhagen R. Deducing the visual focus of attention from head pose estimation in dynamic multi-view meeting scenarios, In Proceedings of the International Conference on Multimodal Interfaces, (2008), 173--180.

Digital Library

[31]

Wedel M., and Pieters R. Eye fixations on advertisements and memory for brands: a model and findings. Marketing Sciences, 19 (4), (2000), 297--312.

Digital Library

[32]

Zhang Z., Hu Y., Liu M., and Huang T. Head pose estimation in seminar room using multi view face detectors. In Multimodal Technologies for Perception of Humans, (2007), 299--304.

Digital Library

Cited By

Gopalakrishnan AZhou XVenkataraman RGururajan RChan KZhu GHiggins N(2025)Prenatal depression level prediction using ensemble based deep learning modelInternational Journal of Cognitive Computing in Engineering10.1016/j.ijcce.2024.12.0026(267-279)Online publication date: Dec-2025
https://doi.org/10.1016/j.ijcce.2024.12.002
Nandipati KPal SMitra R(2024)Electrodermal Activity (EDA) Morphologies and Prediction of Engagement with Simple Moving Average Crossover: A Mixed-Method StudySensors10.3390/s2414456524:14(4565)Online publication date: 14-Jul-2024
https://doi.org/10.3390/s24144565
Li GZarei MAlibakhshi GLabbafi A(2024)Teachers and educators’ experiences and perceptions of artificial-powered interventions for autism groupsBMC Psychology10.1186/s40359-024-01664-212:1Online publication date: 11-Apr-2024
https://doi.org/10.1186/s40359-024-01664-2
Show More Cited By

Index Terms

Using electrodermal activity to recognize ease of engagement in children during social interactions

Recommendations

Towards long term monitoring of electrodermal activity in daily life

Manic depression, also known as bipolar disorder, is a common and severe form of mental disorder. The European research project MONARCA aims at developing and validating mobile technologies for multi-parametric, long term monitoring of physiological and ...
Measuring Electrodermal Activity to Capture Engagement in an Afterschool Maker Program
FabLearn '16: Proceedings of the 6th Annual Conference on Creativity and Fabrication in Education

In this paper, we describe a new approach for exploring individual participants' engagement in youth maker activities. Participants were outfitted with wearable first person point-of-view still-image cameras and wrist-based electrodermal sensors. The ...
Robot’s adaptive emotional feedback sustains children’s social engagement and promotes their vocabulary learning: a long-term child–robot interaction study

In this article, we present an emotion and memory model for a social robot. The model allowed the robot to create a memory account of a child’s emotional events over four individual sessions. The robot then adapted its behaviour based on the developed ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

UbiComp '14: Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing

September 2014

973 pages

ISBN:9781450329682

DOI:10.1145/2632048

General Chairs:
AJ Brush
Microsoft Research
,
Adrian Friday
Lancaster University, UK
,
Program Chairs:
Julie Kientz
University of Washington
,
James Scott
Microsoft Research, UK
,
Junehwa Song
Korea Advanced Institute of Science and Technology (KAIST), KR

Copyright © 2014 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

In-Cooperation

SIGSPATIAL: ACM Special Interest Group on Spatial Information

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 13 September 2014

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Division of Computing and Communication Foundations

Conference

UbiComp '14

Sponsor:

UbiComp '14: The 2014 ACM Conference on Ubiquitous Computing

September 13 - 17, 2014

Washington, Seattle

Acceptance Rates

Overall Acceptance Rate 764 of 2,912 submissions, 26%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

125
Total Citations
View Citations
1,316
Total Downloads

Downloads (Last 12 months)108
Downloads (Last 6 weeks)13

Reflects downloads up to 16 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Gopalakrishnan AZhou XVenkataraman RGururajan RChan KZhu GHiggins N(2025)Prenatal depression level prediction using ensemble based deep learning modelInternational Journal of Cognitive Computing in Engineering10.1016/j.ijcce.2024.12.0026(267-279)Online publication date: Dec-2025
https://doi.org/10.1016/j.ijcce.2024.12.002
Nandipati KPal SMitra R(2024)Electrodermal Activity (EDA) Morphologies and Prediction of Engagement with Simple Moving Average Crossover: A Mixed-Method StudySensors10.3390/s2414456524:14(4565)Online publication date: 14-Jul-2024
https://doi.org/10.3390/s24144565
Li GZarei MAlibakhshi GLabbafi A(2024)Teachers and educators’ experiences and perceptions of artificial-powered interventions for autism groupsBMC Psychology10.1186/s40359-024-01664-212:1Online publication date: 11-Apr-2024
https://doi.org/10.1186/s40359-024-01664-2
Costa-Cordella SGrasso-Cladera AParada F(2024)The Future of Psychotherapy Research and Neuroscience: Introducing the 4E/MoBI Approach to the Study of Patient–Therapist InteractionReview of General Psychology10.1177/1089268023122439928:2(143-165)Online publication date: 27-Jan-2024
https://doi.org/10.1177/10892680231224399
Zhang BLi XZhou YLiu JZhou CLiu WBian Y(2024)Are We in the Zone? Exploring the Features and Method of Detecting Simultaneous Flow Experiences Based on EEG SignalsProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36997748:4(1-42)Online publication date: 21-Nov-2024
https://dl.acm.org/doi/10.1145/3699774
Akiri SJoshi VTaherzadeh SWilliams GMentis HKleinsmith A(2024)Design and Preliminary Evaluation of a Stress Reflection System for High-Stress Training EnvironmentsCompanion Proceedings of the 26th International Conference on Multimodal Interaction10.1145/3686215.3690148(11-15)Online publication date: 4-Nov-2024
https://dl.acm.org/doi/10.1145/3686215.3690148
Akiri SMisal VTaherzadeh SJenkins JWilliams GMentis HKleinsmith A(2024)Enhancing Stress Understanding Through Team Reflection: Technology-Driven Insights in High-Stress Training ScenariosProceedings of the 3rd Annual Meeting of the Symposium on Human-Computer Interaction for Work10.1145/3663384.3663385(1-18)Online publication date: 25-Jun-2024
https://dl.acm.org/doi/10.1145/3663384.3663385
Aloi DBouzit SLi J(2024)Biometric Methods for User Research: Three Case StudiesExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3637132(1-8)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613905.3637132
Chatterjee IGoršič MKaya RErion CClapp JNovak V(2024)Regression of Multiple Conversation Aspects using Dyadic Physiological Measurements2024 46th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC)10.1109/EMBC53108.2024.10782976(1-5)Online publication date: 15-Jul-2024
https://doi.org/10.1109/EMBC53108.2024.10782976
Sharma KGiannakos M(2024)Carry-forward effect: providing proactive scaffolding to learning processesBehaviour & Information Technology10.1080/0144929X.2024.2411592(1-40)Online publication date: 16-Oct-2024
https://doi.org/10.1080/0144929X.2024.2411592
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten