research-article

Touch or touchless?: evaluating usability of interactive displays for persons with autistic spectrum disorders

Authors:

Stefania Serafin,

Davide Rocchesso,

Salvatore SorceAuthors Info & Claims

PerDis '19: Proceedings of the 8th ACM International Symposium on Pervasive Displays

Article No.: 10, Pages 1 - 7

https://doi.org/10.1145/3321335.3324946

Published: 12 June 2019 Publication History

Abstract

Interactive public displays have been exploited and studied for engaging interaction in several previous studies. In this context, applications have been focused on supporting learning or entertainment activities, specifically designed for people with special needs. This includes, for example, those with Autism Spectrum Disorders (ASD). In this paper, we present a comparison study aimed at understanding the difference in terms of usability, effectiveness, and enjoyment perceived by users with ASD between two interaction modalities usually supported by interactive displays: touch-based and touchless gestural interaction. We present the outcomes of a within-subject setup involving 8 ASD users (age 18-25 y.o., IQ 40-60), based on the use of two similar user interfaces, differing only by the interaction modality. We show that touch interaction provides higher usability level and results in more effective actions, although touchless interaction is more effective in terms of enjoyment and engagement.

References

[1]

Ali Adjorlu, Emil Rosenlund Høeg, Luca Mangano, and Stefania Serafin. 2017. Daily Living Skills Training in Virtual Reality to Help Children with Autism Spectrum Disorder in a Real Shopping Scenario. In 2017 IEEE International Symposium on Mixed and Augmented Reality (ISMAR-Adjunct). 294--302.

[2]

Mary Ainley and John Ainley. 2011. Student engagement with science in early adolescence: The contribution of enjoyment to students' continuing interest in learning about science. Contemporary Educational Psychology 36, 1 (2011), 4 -- 12. Students' Emotions and Academic Engagement.

[3]

American Psychiatric Association et al. 2013. Diagnostic and statistical manual of mental disorders (DSM-5®). American Psychiatric Pub.

[4]

Rachel Bailey, Kevin Wise, and Paul Bolls. 2009. How Avatar Customizability Affects Children's Arousal and Subjective Presence During Junk FoodâĂŞSponsored Online Video Games. CyberPsychology & Behavior 12, 3 (jun 2009), 277--283.

[5]

Aaron Bangor, Philip Kortum, and James Miller. 2009. Determining What Individual SUS Scores Mean: Adding an Adjective Rating Scale. J. Usability Studies 4, 3 (May 2009), 114--123. http://dl.acm.org/citation.cfm?id=2835587.2835589

Digital Library

[6]

Laura Bartoli, Franca Garzotto, Mirko Gelsomini, Luigi Oliveto, and Matteo Valoriani. 2014. Designing and Evaluating Touchless Playful Interaction for ASD Children. In Proceedings of the 2014 Conference on Interaction Design and Children (IDC '14). ACM, New York, NY, USA, 17--26.

Digital Library

[7]

A. Battocchi, F. Pianesi, D. Tomasini, M. Zancanaro, G. Esposito, P. Venuti, A. Ben Sasson, E. Gal, and P. L. Weiss. 2009. Collaborative Puzzle Game: A Tabletop Interactive Game for Fostering Collaboration in Children with Autism Spectrum Disorders (ASD). In Proceedings of the ACM International Conference on Interactive Tabletops and Surfaces (ITS '09). ACM, New York, NY, USA, 197--204.

Digital Library

[8]

Nadia Bianchi-Berthouze. 2013. Understanding the Role of Body Movement in Player Engagement. Human-Computer Interaction 28, 1 (2013), 40--75.

[9]

John Brooke. 1996. SUS: A 'quick and dirty' usability scale. In Usability Evaluation in Industry, P. W. Jordan, B. Weerdmeester, A. Thomas, and I. L. Mclelland (Eds.). Taylor and Francis, London.

[10]

Michela Candini, Virginia Giuberti, Alessandra Manattini, Serenella Grittani, Giuseppe di Pellegrino, and Francesca Frassinetti. 2017. Personal space regulation in childhood autism: Effects of social interaction and person's perspective. Autism Research 10, 1 (2017), 144--154.

[11]

Franceli L. Cibrian, Oscar Pena, Vianey Vazquez, Carlos Cardenas, and Monica Tentori. 2016. Designing a deformable musical surface for children with autism. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing Adjunct - UbiComp '16. ACM Press, New York, New York, USA, 977--982.

Digital Library

[12]

Sarah Clinch, Jason Alexander, and Sven Gehring. 2016. A Survey of Pervasive Displays for Information Presentation. IEEE Pervasive Computing 15, 3 (jul 2016), 14--22.

[13]

Jorgos Coenen, Sandy Claes, and Andrew Vande Moere. 2017. The concurrent use of touch and mid-air gestures or floor mat interaction on a public display. In Proceedings of the 6th ACM International Symposium on Pervasive Displays -PerDis '17. ACM Press, New York, New York, USA, 1--9.

Digital Library

[14]

Peter Dalsgaard and Kim Halskov. 2010. Designing urban media façades: cases and challenges. In Proceedings of the 28th international conference on Human factors in computing systems - CHI '10. ACM Press, New York, New York, USA, 2277.

Digital Library

[15]

O. Damm, K. Malchus, P. Jaecks, S. Krach, F. Paulus, M. Naber, A. Jansen, I. Kamp-Becker, W. Einhaeuser-Treyer, P. Stenneken, and B. Wrede. 2013. Different gaze behavior in human-robot interaction in Asperger's syndrome: An eye-tracking study. In 2013 IEEE RO-MAN. 368--369.

[16]

Nigel Davies, Sarah Clinch, and Florian Alt. 2014. Pervasive Displays: Understanding the Future of Digital Signage (1st ed.). Morgan & Claypool Publishers.

Digital Library

[17]

René de la Barré, Paul Chojecki, Ulrich Leiner, Lothar Mühlbach, and Detlef Ruschin. 2009. Touchless Interaction-Novel Chances and Challenges. In Human-Computer Interaction. Novel Interaction Methods and Techniques, Julie A. Jacko (Ed.). Springer Berlin Heidelberg, Berlin, Heidelberg, 161--169.

Digital Library

[18]

Franca Garzotto, Mirko Gelsomini, Luigi Oliveto, and Matteo Valoriani. 2014. Motion-based Touchless Interaction for ASD Children: A Case Study. In Proceedings of the 2014 International Working Conference on Advanced Visual Interfaces (AVI '14). ACM, New York, NY, USA, 117--120.

Digital Library

[19]

Franca Garzotto, Matteo Valoriani, and Laura Bartoli. 2014. Touchless Motion-Based Interaction for Therapy of Autistic Children. Springer Berlin Heidelberg, Berlin, Heidelberg, 471--494.

[20]

Vito Gentile, Mohamed Khamis, Salvatore Sorce, and Florian Alt. 2017. They are looking at me! Understanding how Audience Presence Impacts on Public Display Users. In Proceedings of The 6th ACM International Symposium on Pervasive Displays (PerDis '17). ACM, Article 11, 7 pages.

Digital Library

[21]

Vito Gentile, Salvatore Sorce, Alessio Malizia, Fabrizio Milazzo, and Antonio Gentile. 2017. Investigating how user avatar in touchless interfaces affects perceived cognitive load and two-handed interactions. In Proceedings of The 6th ACM International Symposium on Pervasive Displays. ACM.

Digital Library

[22]

Vito Gentile, Salvatore Sorce, Alessio Malizia, Dario Pirrello, and Antonio Gentile. 2016. Touchless Interfaces For Public Displays: Can We Deliver Interface Designers From Introducing Artificial Push Button Gestures?. In Proceedings of the International Working Conference on Advanced Visual Interfaces. ACM, 40--43.

Digital Library

[23]

Vito Gentile, Salvatore Sorce, Giuseppe Russo, Dario Pirrone, and Antonio Gentile. 2016. A Multimodal Fruition Model for Graphical Contents in Ancient Books. In Proceedings of the 17th International Conference on Computer Systems and Technologies 2016 (CompSysTech '16). ACM, New York, NY, USA, 65--72.

Digital Library

[24]

Charles P. Gerba, Adam L. Wuollet, Peter Raisanen, and Gerardo U. Lopez. 2016. Bacterial contamination of computer touch screens. American Journal of Infection Control 44, 3 (mar 2016), 358--360.

[25]

Lynne Hall, Colette Hume, and Sarah Tazzyman. 2016. Five Degrees of Happiness: Effective Smiley Face Likert Scales for Evaluating with Children. In Proceedings of the 15th International Conference on Interaction Design and Children - IDC'16. 311--321.

Digital Library

[26]

Mikkel R. Jakobsen, Yvonne Jansen, Sebastian Boring, and Kasper Hornbæk. 2015. Should I Stay or Should I Go? Selecting Between Touch and Mid-Air Gestures for Large-Display Interaction. In Human-Computer Interaction - INTERACT 2015, Julio Abascal, Simone Barbosa, Mirko Fetter, Tom Gross, Philippe Palanque, and Marco Winckler (Eds.). Springer International Publishing, Cham, 455--473.

Digital Library

[27]

Mohamed Khamis, Christian Becker, Andreas Bulling, and Florian Alt. 2018. Which One is Me?: Identifying Oneself on Public Displays. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18). ACM, New York, NY, USA, Article 287, 12 pages.

Digital Library

[28]

Mohamed Khamis, Alexander Klimczak, Martin Reiss, Florian Alt, and Andreas Bulling. 2017. EyeScout: Active Eye Tracking for Position and Movement Independent Gaze Interaction with Large Public Displays. In Proceedings of the 30th Annual ACM Symposium on User Interface Software & Technology (UIST '17). ACM, New York, NY, USA, 155--166.

Digital Library

[29]

Kamran Khowaja, Siti Salwah Salim, Adeleh Asemi, Sumbul Ghulamani, and Asadullah Shah. 2019. A systematic review of modalities in computer-based interventions (CBIs) for language comprehension and decoding skills of children with autism spectrum disorder (ASD). Universal Access in the Information Society (02 2019).

[30]

Ville Mäkelä, Mohamed Khamis, Lukas Mecke, Jobin James, Markku Turunen, and Florian Alt. 2018. Pocket Transfers: Interaction Techniques for Transferring Content from Situated Displays to Mobile Devices. (2018), 13.

Digital Library

[31]

Emanuela Marchetti and Andrea Valente. 2016. What a Tangible Digital Installation for Museums Can Offer to Autistic Children and Their Teachers. International Journal of Game-Based Learning (IJGBL) 6, 2 (2016), 29--45.

Digital Library

[32]

Aleksandar Matic, Gillian R. Hayes, Monica Tentori, Maryam Abdullah, and Sabrina Schuck. 2014. Collective Use of a Situated Display to Encourage Positive Behaviors in Children with Behavioral Challenges. In Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp '14). ACM, New York, NY, USA, 885--895.

Digital Library

[33]

Jörg Müller, Robert Walter, Gilles Bailly, Michael Nischt, and Florian Alt. 2012. Looking glass: a field study on noticing interactivity of a shop window. In Proceedings of the 2012 ACM annual conference on Human Factors in Computing Systems - CHI '12. ACM Press, New York, New York, USA, 297--306.

Digital Library

[34]

Jörg Müller, Dennis Wilmsmann, Juliane Exeler, Markus Buzeck, Albrecht Schmidt, Tim Jay, and Antonio Krüger. 2009. Display Blindness: The Effect of Expectations on Attention towards Digital Signage. In Pervasive Computing, Hideyuki Tokuda, Michael Beigl, Adrian Friday, A. J. Bernheim Brush, and Yoshito Tobe (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 1--8.

Digital Library

[35]

Pai Chet Ng, James She, Kang Eun Jeon, and Matthias Baldauf. 2017. When Smart Devices Interact With Pervasive Screens: A Survey. ACM Trans. Multimedia Comput. Commun. Appl. 13, 4, Article 55 (Aug. 2017), 23 pages.

Digital Library

[36]

Timo Ojala, Vassilis Kostakos, Hannu Kukka, Tommi Heikkinen, Tomas Linden, Marko Jurmu, Simo Hosio, Fabio Kruger, and Daniele Zanni. 2012. Multipurpose Interactive Public Displays in the Wild: Three Years Later. Computer 45, 5 (May 2012), 42--49.

Digital Library

[37]

Peter Peltonen, Esko Kurvinen, Antti Salovaara, Giulio Jacucci, Tommi Ilmonen, John Evans, Antti Oulasvirta, and Petri Saarikko. 2008. It's Mine, Don'T Touch!: Interactions at a Large Multi-touch Display in a City Centre. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '08). ACM, New York, NY, USA, 1285--1294.

Digital Library

[38]

Elisa Rubegni, Vito Gentile, Alessio Malizia, Salvatore Sorce, and Niko Kargas. 2019. Child-Display Interaction: Exploring Avatar-based Touchless Gestural Interfaces. In Proceedings of The 8th ACM International Symposium on Pervasive Displays (PerDis '19). ACM.

Digital Library

[39]

Andrea Serino, Jean-Paul Noel, Robin Mange, Elisa Canzoneri, Elisa Pellencin, Javier Bello Ruiz, Fosco Bernasconi, Olaf Blanke, and Bruno Herbelin. 2018. Peripersonal Space: An Index of Multisensory BodyâĂŞEnvironment Interactions in Real, Virtual, and Mixed Realities. Frontiers in ICT 4 (2018), 31.

[40]

Salvatore Sorce, Vito Gentile, Cristina Enea, Antonio Gentile, Alessio Malizia, and Fabrizio Milazzo. 2017. A Touchless Gestural System for Extended Information Access Within a Campus. In Proceedings of the 2017 ACM Annual Conference on SIGUCCS. ACM, 37--43.

Digital Library

[41]

Salvatore Sorce, Vito Gentile, Debora Oliveto, Rossella Barraco, Alessio Malizia, and Antonio Gentile. 2018. Exploring Usability and Accessibility of Avatar-based Touchless Gestural Interfaces for Autistic People. In Proceedings of The 7th ACM International Symposium on Pervasive Displays (PerDis '18). ACM, 2.

Digital Library

[42]

Fabius Steinberger, Marcus Foth, and Florian Alt. 2014. Vote With Your Feet: Local Community Polling on Urban Screens. In Proceedings of The International Symposium on Pervasive Displays (PerDis '14). ACM, New York, NY, USA, Article 44, 6 pages.

Digital Library

[43]

Monica Tentori, Lizbeth Escobedo, Carlos Hernandez, Aleksandar Matic, and Gillian R. Hayes. 2016. Pervasive Displays in Classrooms of Children with Severe Autism. IEEE Pervasive Computing 15, 3 (jul 2016), 48--57.

[44]

Robert Walter, Gilles Bailly, Nina Valkanova, and Jörg Müller. 2014. Cuenesics: Using Mid-air Gestures to Select Items on Interactive Public Displays. In Proceedings of the 16th International Conference on Human-computer Interaction with Mobile Devices & Services (MobileHCI '14). ACM, New York, NY, USA, 299--308.

Digital Library

[45]

David Wechsler. 1955. Manual for the Wechsler adult intelligence scale. Psychological Corporation.

[46]

Lesley Xie, Alissa N. Antle, and Nima Motamedi. 2008. Are Tangibles More Fun?: Comparing Children's Enjoyment and Engagement Using Physical, Graphical and Tangible User Interfaces. In Proceedings of the 2Nd International Conference on Tangible and Embedded Interaction (TEI '08). ACM, New York, NY, USA, 191--198.

Digital Library

[47]

Hanan Makki Zakari, Minhua Ma, and David Simmons. 2014. A Review of Serious Games for Children with Autism Spectrum Disorders (ASD). In Serious Games Development and Applications, Minhua Ma, Manuel Fradinho Oliveira, and Jannicke Baalsrud Hauge (Eds.). Springer International Publishing, Cham, 93--106.

Cited By

Cornelio PHughes SGeorgiou OFrier WMaunsbach MVasudevan MObrist M(2023)Responsible Innovation of Touchless Haptics: A Prospective Design Exploration in Social InteractionProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581228(1-16)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3581228
Valencia KRusu CBotella FJamet E(2022)A Methodology to Evaluate User Experience for People with Autism Spectrum DisorderApplied Sciences10.3390/app12221134012:22(11340)Online publication date: 8-Nov-2022
https://doi.org/10.3390/app122211340
Huang JMao LQi MZhang DAn MHan RJi T(2021)Differences in muscle activity, kinematics, user performance, and subjective assessment between touchscreen and mid-air interactions on a tabletBehaviour & Information Technology10.1080/0144929X.2021.197022741:14(3028-3043)Online publication date: 25-Aug-2021
https://doi.org/10.1080/0144929X.2021.1970227
Show More Cited By

Index Terms

Touch or touchless?: evaluating usability of interactive displays for persons with autistic spectrum disorders
1. Human-centered computing
  1. Human computer interaction (HCI)

Recommendations

Touchless Interfaces For Public Displays: Can We Deliver Interface Designers From Introducing Artificial Push Button Gestures?
AVI '16: Proceedings of the International Working Conference on Advanced Visual Interfaces

Public displays have lately become ubiquitous thanks to the decreasing cost of such technology and public policies supporting the development of smart cities. Depending on form factor, those displays might use touchless gestural interfaces that ...
Demonstrating Proxemic Cursor Input for Touchless Displays
SUI '23: Proceedings of the 2023 ACM Symposium on Spatial User Interaction

Touchless gesture interfaces often use cursor-based interactions, where widgets are targeted by a movable cursor and activated with a mid-air gesture. Proxemic cursor interactions are a novel alternative that facilitate faster selection without the need ...
Proxemic Cursor Interactions for Touchless Widget Control
SUI '23: Proceedings of the 2023 ACM Symposium on Spatial User Interaction

Touchless gesture interfaces often use cursor-based interactions, where widgets are targeted by a movable cursor and activated with a mid-air gesture (e.g., push or Pinch). Continuous interactions like slider manipulation can be challenging in mid-air ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

PerDis '19: Proceedings of the 8th ACM International Symposium on Pervasive Displays

June 2019

223 pages

ISBN:9781450367516

DOI:10.1145/3321335

General Chairs:
Mohamed Khamis
University of Glasgow, United Kingdom
,
Salvatore Sorce
Università degli Studi di Palermo, Italy
,
Program Chairs:
Jessica R. Cauchard
Ben Gurion University of the Negev, Israel
,
Vito Gentile
Università degli Studi di Palermo, Italy

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 June 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

PerDis '19

PerDis '19: The 8th ACM International Symposium on Pervasive Displays

June 12 - 14, 2019

Palermo, Italy

Acceptance Rates

PerDis '19 Paper Acceptance Rate 26 of 67 submissions, 39%;

Overall Acceptance Rate 213 of 384 submissions, 55%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
261
Total Downloads

Downloads (Last 12 months)47
Downloads (Last 6 weeks)2

Reflects downloads up to 26 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Cornelio PHughes SGeorgiou OFrier WMaunsbach MVasudevan MObrist M(2023)Responsible Innovation of Touchless Haptics: A Prospective Design Exploration in Social InteractionProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581228(1-16)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3581228
Valencia KRusu CBotella FJamet E(2022)A Methodology to Evaluate User Experience for People with Autism Spectrum DisorderApplied Sciences10.3390/app12221134012:22(11340)Online publication date: 8-Nov-2022
https://doi.org/10.3390/app122211340
Huang JMao LQi MZhang DAn MHan RJi T(2021)Differences in muscle activity, kinematics, user performance, and subjective assessment between touchscreen and mid-air interactions on a tabletBehaviour & Information Technology10.1080/0144929X.2021.197022741:14(3028-3043)Online publication date: 25-Aug-2021
https://doi.org/10.1080/0144929X.2021.1970227
Spitale MGelsomini MBeccaluva EViola LGarzotto FGamberini LPittarello FSpagnolli A(2019)Meeting the needs of people with neuro-developmental disorder through a phygital approachProceedings of the 13th Biannual Conference of the Italian SIGCHI Chapter: Designing the next interaction10.1145/3351995.3352055(1-10)Online publication date: 23-Sep-2019
https://dl.acm.org/doi/10.1145/3351995.3352055
Caria SPaternò FSantoro CGamberini LPittarello FSpagnolli A(2019)Understanding ASD individuals' difficulties with managing moneyProceedings of the 13th Biannual Conference of the Italian SIGCHI Chapter: Designing the next interaction10.1145/3351995.3352038(1-5)Online publication date: 23-Sep-2019
https://dl.acm.org/doi/10.1145/3351995.3352038

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents