research-article

Public Access

Social Influences on Executive Functioning in Autism: Design of a Mobile Gaming Platform

Authors:

Amy Yeojin Ahn,

Madeline Aubertine,

Sarah Corrigan,

Tanya St. John,

Marilena Mademtzi,

Frederick ShicAuthors Info & Claims

CHI '18: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems

Paper No.: 443, Pages 1 - 13

https://doi.org/10.1145/3173574.3174017

Published: 21 April 2018 Publication History

Abstract

Most studies of executive function (EF) in Autism Spectrum Disorder (ASD) focus on cognitive information processing, emphasizing less the social interaction deficits core to ASD. We designed a mobile game that uses social and nonsocial stimuli to assess children's EF skills. The game comprised three components involving different EF skills: cognitive flexibility (shifting/inference), inhibitory control, and short-term memory. By recruiting 65 children with and without ASD to play the mobile game, we investigated the potential of such platforms for capturing important phenotypic characteristics of individuals with autism. Results highlighted between-diagnostic-group differences in playing patterns with children with ASD showing broad patterns of EF deficits, but with relative strengths in nonsocial short-term memory, and preserved response to emotional inhibition cues. We showed the system could predict IQ, an important target for clinical treatment, towards the goal of developing platforms to act as long-term, efficient, and effective behavioral biomarkers for ASD.

Supplementary Material

suppl.mov (pn3681-file5.mp4)

Supplemental video

Download
9.82 MB

MP4 File (pn3681.mp4)

Download
203.21 MB

References

[1]

Adham Atyabi, Beibin Li, Yeojin Amy Ahn, Minah Kim, Erin Barney, and Frederick Shic. 2017. An exploratory analysis targeting diagnostic classification of AAC app usage patterns. In Neural Networks (IJCNN), 2017 International Joint Conference on. IEEE, 1633-- 1640.

[2]

American Psychiatric Association. 2013. Diagnostic and Statistical Manual of Mental Disorders, 5th Edition: DSM-5. American Psychiatric Publishing.

[3]

Anna Anzulewicz, Krzysztof Sobota, and Jonathan T Delafield-Butt. 2016. Toward the Autism Motor Signature: Gesture patterns during smart tablet gameplay identify children with autism. Scientific reports 6 (2016), 31107.

[4]

Anna M. Ern. 2014. The use of gamification and serious games within interventions for children with autism spectrum disorder. Retrieved February 20, 2016 from http://essay.utwente.nl/64780/

[5]

Beibin Li, Laura Boccanfuso, Quan Wang, et al. Human Robot Activity Classification based on Accelerometer and Gyroscope{C}//2016 25th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN). Presented at the 2016 25th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN). 2016: 423--424.

[6]

Ben Popple, Carla Wall, Lilli Flink, Kelly Powell, Keri Discepolo, Douglas Keck, Marilena Mademtzi, Fred Volkmar, and Frederick Shic. 2016. Brief Report: Remotely Delivered Video Modeling for Improving Oral Hygiene in Children with ASD: A Pilot Study. Journal of autism and developmental disorders 46, no. 8: 2791--2796.

[7]

Bih-Ching Shu, For-Wey Lung, Allen Y. Tien, and Bor-Chih Chen. 2001. Executive function deficits in non-retarded autistic children. Autism 5, 2: 165--174.

[8]

Bretagne Abirached, Yan Zhang, Ji Hyun Park, Bretagne Abirached, Yan Zhang, and Ji Hyun Park. 2012. Understanding User Needs for Serious Games for Teaching Children with Autism Spectrum Disorders Emotions. 1054-- 1063. Retrieved January 6, 2017 from https://www.learntechlib.org/p/40883/

[9]

Catalin Voss, Peter Washington, Nick Haber, Aaron Kline, Jena Daniels, Azar Fazel, Titas De, Beth McCarthy, Carl Feinstein, Terry Winograd, and Dennis Wall. 2016. Superpower Glass: Delivering Unobtrusive Real-time Social Cues in Wearable Systems. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct (UbiComp '16), 1218--1226.

Digital Library

[10]

Charlotte Sanderson and Melissa L. Allen. 2013. The specificity of inhibitory impairments in autism and their relation to ADHD-type symptoms. Journal of autism and developmental disorders 43, no. 5: 1065--1079.

[11]

David A Grant and Esta Berg. 1948. A behavioral analysis of degree of reinforcement and ease of shifting to new responses in a Weigltype card-sorting problem. Journal of experimental psychology 38, 4 (1948), 404.

[12]

Diane L Williams, Gerald Goldstein, Patricia A Carpenter, and Nancy J Minshew. 2005. Verbal and spatial working memory in autism. Journal of autism and developmental disorders, 35(6), 747.

[13]

Elisabeth L Hill. 2004. Executive dysfunction in autism. Trends in cognitive sciences 8, 1: 26--32.

[14]

Emily K. Farran, Amanda Branson, and Ben J. King. 2011. Visual search for basic emotional expressions in autism; impaired processing of anger, fear and sadness, but a typical happy face advantage. Research in Autism Spectrum Disorders 5, 1: 455--462.

[15]

Fahd Albinali, Matthew S. Goodwin, and Stephen S. Intille. 2009. Recognizing Stereotypical Motor Movements in the Laboratory and Classroom: A Case Study with Children on the Autism Spectrum. In Proceedings of the 11th International Conference on Ubiquitous Computing (UbiComp '09), 71-- 80.

Digital Library

[16]

Fernando Ferná ndez-Aranda, Susana Jimé nezMurcia, Juan J. Santamarí a, Katarina Gunnard, Antonio Soto, Elias Kalapanidas, Richard GA Bults, and others. 2012. Video games as a complementary therapy tool in mental disorders: PlayMancer, a European multicentre study. Journal of Mental Health 21, 4: 364--374.

[17]

Francesca Happé, Rhonda Booth, Rebecca Charlton, and Claire Hughes. 2006. Executive function deficits in autism spectrum disorders and attention-deficit/hyperactivity disorder: examining profiles across domains and ages. Brain and cognition 61, 1: 25--39.

[18]

Gnanathusharan Rajendran and Peter Mitchell. 2007. Cognitive theories of autism. Developmental Review 27, 2: 224--260.

[19]

Gregory D. Abowd, Kristin Vadas, Tracy Westeyn, Thad Starner, and Xuehai Bian. 2005. Recognizing Mimicked Autistic Self-Stimulatory Behaviors Using HMMs. In In proceedings of the 16th International Symposium on Wearable Computers, 164--169.

Digital Library

[20]

H. Tanaka, S. Sakti, G. Neubig, H. Negoro, H. Iwasaka, and S. Nakamura. 2016. Automated social skills training with audiovisual information. In 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), 2262--2265.

[21]

Jacob Poushter. 2016. Smartphone ownership and internet usage continues to climb in emerging economies. Pew Research Center 22.

[22]

James M. Rehg, Gregory D. Abowd, Agata Rozga, Mario Romero, Mark A. Clements, Stan Sclaroff, Irfan Essa, Opal Y. Ousley, Yin Li, Chanho Kim, Hrishikesh Rao, Jonathan C. Kim, Liliana Lo Presti, Jianming Zhang, Denis Lantsman, Jonathan Bidwell, and Zhefan Ye. 2013. Decoding Children's Social Behavior. 3414--3421.

Digital Library

[23]

Jill Boucher and Vicky Lewis. 1992. Unfamiliar face recognition in relatively able autistic children. Journal of Child Psychology and Psychiatry, 33(5), pp.843--859.

[24]

Joaquin A. Anguera, Jacqueline Boccanfuso, James L. Rintoul, Omar Al-Hashimi, Farhoud Faraji, Jacqueline Janowich, Eric Kong, and others. 2013. Video game training enhances cognitive control in older adults. Nature 501, 7465: 97--101.

[25]

Jules Morgan. 2016. Gaming for dementia research: a quest to save the brain. The Lancet Neurology 15, 13: 1313.

[26]

Lizbeth Escobedo, David H. Nguyen, LouAnne Boyd, Sen Hirano, Alejandro Rangel, Daniel Garcia-Rosas, Monica Tentori, and Gillian Hayes. 2012. MOSOCO: a mobile assistive tool to support children with autism practicing social skills in real-life situations. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI 2012): 2589--2598.

Digital Library

[27]

Loisa Bennetto, Bruce F. Pennington, and Sally J. Rogers. 1996. Intact and impaired memory functions in autism. Child development 67, no. 4: 1816--1835.

[28]

LouAnne E. Boyd, Alejandro Rangel, Helen Tomimbang, Andrea Conejo-Toledo, Kanika Patel, Monica Tentori, and Gillian R. Hayes. 2016. SayWAT: Augmenting Face-to-Face Conversations for Adults with Autism. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16), 4872--4883.

Digital Library

[29]

M. Indumathi and others. 2015. Emotional Based Smart Accessing and Controlling For Autism Spectrum Disordered People. IJSEAT 3, 3: 78-- 85.

[30]

Mary Weng, Carla A. Wall, Elizabeth S. Kim, Litton Whitaker, Michael Perlmutter, Quan Wang, Eli R. Lebowitz, and Frederick Shic. 2015. Linking volitional preferences for emotional information to social difficulties: A game approach using the microsoft kinect. In IEEE International Conference on Affective Computing and Intelligent Interaction (ACII): 588--594.

Digital Library

[31]

Meia Chita-Tegmark. 2016. Social attention in ASD: A review and meta-analysis of eyetracking studies. Research in developmental disabilities, 48, 79--93.

[32]

Micah O Mazurek and Christopher R Engelhardt. 2013. Video game use in boys with autism spectrum disorder, ADHD, or typical development. Pediatrics 132, 2 (2013), 260--266.

[33]

Michelle Dawson, Isabelle Souliè res, Morton Ann Gernsbacher, and Laurent Mottron. 2007. The level and nature of autistic intelligence. Psychological science 18, no. 8: 657--662.

[34]

Mooi Choo Chuah and M. Diblasio. 2012. Smartphone based autism social alert system. In Mobile Ad-hoc and Sensor Networks (MSN), 2012 Eighth International Conference on, 6--13. Retrieved March 5, 2016 from http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnum ber=6516458

Digital Library

[35]

National Institutes of Health and Northwestern University. 2006. NIH Toolbox for Assessment of Neurological and Behavioral Function Administrator's Manual (iPad App Version 1.13). Retrieved Jan 7, 2018 from https://nihtoolbox.desk.com/customer/en/portal/a rticles/2191534-administrator-s-manual-andelearning-course

[36]

Nim Tottenham, James W. Tanaka, Andrew C. Leon, Thomas McCarry, Marcella Nurse, Todd A. Hare, David J. Marcus, Alissa Westerlund, B. J. Casey, and Charles Nelson. 2009. The NimStim set of facial expressions: judgments from untrained research participants. Psychiatry research 168, 3: 242--249.

[37]

Ouriel Grynszpan, Jacqueline Nadel, Noelle Carbonell, Jerome Simonin, Jacques Constant, Florence Le Barillier, Jean Claude Martin, Matthieu Courgeon. 2009. A new virtual environment paradigm for high functioning autism intended to help attentional disengagement in a social context bridging the gap between relevance theory and executive dysfunction. Virtual Rehabilitation International Conference on IEEE: 51--58.

[38]

Patricia Howlin, Iliana Magiati, and Tony Charman. 2009. Systematic Review of Early Intensive Behavioral Interventions for Children with Autism. American Journal on Intellectual and Developmental Disabilities 114, 1: 23--41.

[39]

Peter Leijdekkers, Valerie Gay, and Frederick Wong. 2013. CaptureMyEmotion: A mobile app to improve emotion learning for autistic children using sensors. In IEEE 26th International Symposium on Computer-Based Medical Systems (CBMS 2013): 381--384.

[40]

Philip David Zelazo, Jacob E Anderson, Jennifer Richler, Kathleen Wallner-Allen, Jennifer L Beaumont, and Sandra Weintraub. 2013. II. NIH Toolbox Cognition Battery (CB): Measuring executive function and attention. Monographs of the Society for Research in Child Development 78, 4 (2013), 16--33.

[41]

Quan Wang, Lauren DiNicola, Perrine Heymann, Michelle Hampson, and Katarzyna Chawarska. 2017. Impaired value learning for faces in preschoolers with autism spectrum disorder. Journal of the American Academy of Child & Adolescent Psychiatry.

[42]

Sandra Weintraub, Sureyya S Dikmen, Robert K Heaton, David S Tulsky, Philip D Zelazo, Patricia J Bauer, Noelle E Carlozzi, Jerry Slotkin, David Blitz, Kathleen Wallner-Allen, and others. 2013. Cognition assessment using the NIH Toolbox. Neurology 80, 11 Supplement 3 (2013), S54--S64.

[43]

Scott Draves. 2005. The Electric Sheep ScreenSaver: A Case Study in Aesthetic Evolution. In EvoWorkshops 5: 458--467.

Digital Library

[44]

Sofiane Boucenna, Antonio Narzisi, Elodie Tilmont, Filippo Muratori, Giovanni Pioggia, David Cohen, and Mohamed Chetouani. 2014. Interactive Technologies for Autistic Children: A Review. Cognitive Computation 6, 4: 722--740.

[45]

Thomas Insel, Bruce Cuthbert, Marjorie Garvey, Robert Heinssen, Daniel S. Pine, Kevin Quinn, Charles Sanislow, and Philip Wang. 2010. Research domain criteria (RDoC): toward a new classification framework for research on mental disorders. The American Journal of Psychiatry 167, 7: 748--751.

[46]

Thomas R. Insel, and Jeffrey A. Lieberman. 2013. DSM-5 and RDoC: shared interests. The National Institute of Mental Health.

[47]

Xue Ming, Michael Brimacombe, and George C. Wagner. 2007. Prevalence of motor impairment in autism spectrum disorders. Brain and Development 29.9: 565--570.

[48]

Yael Leitner. 2014. The co-occurrence of autism and attention deficit hyperactivity disorder in children--what do we know?. Frontiers in human neuroscience 8.

Cited By

Kim WSeong MDelPreto JMatusik WRus DKim SKostakos VKay JHoang T(2024)Exploring Potential Application Areas of Artificial Intelligence-Infused System for Engagement Recognition: Insights from Special Education ExpertsCompanion of the 2024 on ACM International Joint Conference on Pervasive and Ubiquitous Computing10.1145/3675094.3678376(803-808)Online publication date: 5-Oct-2024
https://dl.acm.org/doi/10.1145/3675094.3678376
Kim WSeong MKim KKim S(2024)Engagnition: A multi-dimensional dataset for engagement recognition of children with autism spectrum disorderScientific Data10.1038/s41597-024-03132-311:1Online publication date: 15-Mar-2024
https://doi.org/10.1038/s41597-024-03132-3
Toma MTurcu CTurcu CVlad STiliute DPascu P(2023)Extended Reality-Based Mobile Application Solutions for the Therapy of Children with Autism Spectrum Disorders: A Systematic Literature Review (Preprint)JMIR Serious Games10.2196/49906Online publication date: 13-Jun-2023
https://doi.org/10.2196/49906
Show More Cited By

Index Terms

Social Influences on Executive Functioning in Autism: Design of a Mobile Gaming Platform

Recommendations

Behavioral inhibition and prefrontal cortex in decision-making
2006 Special issue: Neurobiology of decision making

Every day we make innumerable decisions; some require no effort at all whereas others require considerable deliberation and weighing of various options. Despite the importance of decision-making in our lives and increased research interest, the specific ...
Is there a link between media-multitasking and the executive functions of filtering and response inhibition?

Media-multitasking refers to utilising at least two forms of media simultaneously. This study examined the link between media-multitasking and the inhibitory control executive function. Performance on measures of filtering (flanker task) and inhibitory ...
Executive functioning in addicts following health mobile cognitive stimulation: evidence from alcohol and heroin patients
PervasiveHealth '13: Proceedings of the 7th International Conference on Pervasive Computing Technologies for Healthcare

The consequences of alcohol and opioid dependence are severe and may range from physical diseases to neuropsychological deficits in several cognitive domains. Alcohol and opioid abuse has also been related to brain dysfunction specifically in the ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '18: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems

April 2018

8489 pages

ISBN:9781450356206

DOI:10.1145/3173574

General Chairs:
Regan Mandryk
University of Saskatchewan, Canada
,
Mark Hancock
University of Waterloo, Canada
,
Program Chairs:
Mark Perry
Brunel University London, UK
,
Anna Cox
University College London, UK

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 21 April 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

IES
NIH
NSF
Simons

Conference

CHI '18

Sponsor:

SIGCHI

CHI '18: CHI Conference on Human Factors in Computing Systems

April 21 - 26, 2018

Montreal QC, Canada

Acceptance Rates

CHI '18 Paper Acceptance Rate 666 of 2,590 submissions, 26%;

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI '25

Sponsor:
sigchi

CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

19
Total Citations
View Citations
1,219
Total Downloads

Downloads (Last 12 months)255
Downloads (Last 6 weeks)22

Reflects downloads up to 21 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Kim WSeong MDelPreto JMatusik WRus DKim SKostakos VKay JHoang T(2024)Exploring Potential Application Areas of Artificial Intelligence-Infused System for Engagement Recognition: Insights from Special Education ExpertsCompanion of the 2024 on ACM International Joint Conference on Pervasive and Ubiquitous Computing10.1145/3675094.3678376(803-808)Online publication date: 5-Oct-2024
https://dl.acm.org/doi/10.1145/3675094.3678376
Kim WSeong MKim KKim S(2024)Engagnition: A multi-dimensional dataset for engagement recognition of children with autism spectrum disorderScientific Data10.1038/s41597-024-03132-311:1Online publication date: 15-Mar-2024
https://doi.org/10.1038/s41597-024-03132-3
Toma MTurcu CTurcu CVlad STiliute DPascu P(2023)Extended Reality-Based Mobile Application Solutions for the Therapy of Children with Autism Spectrum Disorders: A Systematic Literature Review (Preprint)JMIR Serious Games10.2196/49906Online publication date: 13-Jun-2023
https://doi.org/10.2196/49906
Williams RPark C(2023)Cyborg Assemblages: How autistic adults construct sociotechnical networks to support cognitive functionProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581556(1-15)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3581556
Chinta UAtyabi A(2023)A Framework Pipeline to Address Imbalanced Class Distribution Problem in Real-world Datasets2023 IEEE 13th Annual Computing and Communication Workshop and Conference (CCWC)10.1109/CCWC57344.2023.10099163(0746-0753)Online publication date: 8-Mar-2023
https://doi.org/10.1109/CCWC57344.2023.10099163
de Carvalho ABraz Cdos Santos SFerreira RPrates R(2023)Serious Games for Children with Autism Spectrum Disorder: A Systematic Literature ReviewInternational Journal of Human–Computer Interaction10.1080/10447318.2023.219405140:14(3655-3682)Online publication date: 11-Apr-2023
https://doi.org/10.1080/10447318.2023.2194051
López-Bouzas Ndel Moral-Pérez M(2023)Gamified Environments and Serious Games for Students With Autistic Spectrum Disorder: Review of ResearchReview Journal of Autism and Developmental Disorders10.1007/s40489-023-00381-7Online publication date: 11-May-2023
https://doi.org/10.1007/s40489-023-00381-7
Gallardo-Montes CRodríguez Fuentes ACaurcel Cara MCapperucci D(2022)Functionality of Apps for People with Autism: Comparison between Educators from Florence and GranadaInternational Journal of Environmental Research and Public Health10.3390/ijerph1912701919:12(7019)Online publication date: 8-Jun-2022
https://doi.org/10.3390/ijerph19127019
Wu QXu RLiu YLottridge DNanayakkara S(2022)Players and Performance: Opportunities for Social Interaction with Augmented Tabletop Games at Centres for Children with AutismProceedings of the ACM on Human-Computer Interaction10.1145/35677166:ISS(161-184)Online publication date: 14-Nov-2022
https://dl.acm.org/doi/10.1145/3567716
de Carvalho ABraz CPrates RSantos CBento Villela MRios da Hora Rodrigues Kde Gois Ribeiro Darin T(2022)How are games for autistic children being evaluated?Proceedings of the 21st Brazilian Symposium on Human Factors in Computing Systems10.1145/3554364.3559127(1-13)Online publication date: 17-Oct-2022
https://dl.acm.org/doi/10.1145/3554364.3559127
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents