research-article

Open access

Eyelid gestures for people with motor impairments

Authors:

Franklin Mingzhe LiAuthors Info & Claims

Communications of the ACM, Volume 65, Issue 1

Pages 108 - 115

https://doi.org/10.1145/3498367

Published: 17 December 2021 Publication History

All formats PDF

Abstract

Although eye-based interactions can be beneficial for people with motor impairments, they often rely on clunky or specialized equipment (e.g., stationary eye-trackers) and focus primarily on gaze and blinks. However, two eyelids can open and close in different orders and for different duration to form rich eyelid gestures. We take a first step to design, detect, and evaluate a set of eyelid gestures for people with motor impairments on mobile devices. We present an algorithm to detect nine eyelid gestures on smartphones in real time and evaluate it with 12 able-bodied people and 4 people with severe motor impairments in two studies. The results of the study with people with motor-impairments show that the algorithm can detect the gestures with .76 and .69 overall accuracy in user-dependent and user-independent evaluations. Furthermore, we design and evaluate a gesture mapping scheme for people with motor impairments to navigate mobile applications only using eyelid gestures. Finally, we discuss considerations for designing and using eyelid gestures for people with motor impairments.

References

[1]

Albert, M.V., Toledo, S., Shapiro, M., Koerding, K. Using mobile phones for activity recognition in Parkinson's patients. Front. Neurol. 3, 158 (2012), 7.

[2]

Araujo, J.M., Zhang, G., Hansen, J.P.P., Puthusserypady, S. Exploring eye-gaze wheelchair control. In ACM Symposium on Eye Tracking Research and Applications (2020), ACM Press, New York, USA, 1--8.

Digital Library

[3]

A. Association. Who gets als? ALS Association, 2020. https://www.als.org/understanding-als/who-gets-als.

[4]

Fan, M., Li, Z., Li, F.M. Eyelid gestures on mobile devices for people with motor impairments. In The 22^nd International ACM SIGACCESS Conference on Computers and Accessibility, (New York, NY, USA, 2020), Association for Computing Machinery, NY, USA.

Digital Library

[5]

N. C. for Health Statistics et al. Summary Health Statistics: National Health Interview Survey, 2018.

[6]

Jota, R., Wigdor, D. Palpebrae superioris: Exploring the design space of eyelid gestures. In Proceedings of the 41^st Graphics Interface Conference (Toronto, Ontario, Canada, 2015), Canadian Human-Computer Communications Society, ACM Press, New York, USA, 3--5.

[7]

Leiva, L., Böhmer, M., Gehring, S., Krüger, A. Back to the app: the costs of mobile application interruptions. In Proceedings of the 14^th International Conference on Human-Computer Interaction with Mobile Devices And Services (New York, New York, USA, 2012), ACM Press, New York, USA, 291.

Digital Library

[8]

Li, Z., Fan, M., Han, Y., Truong, K.N. iWink: Exploring eyelid gestures on mobile devices. In Proceedings of the 1^st International Workshop on Human-Centric Multimedia Analysis, (New York, NY, USA, 2020), Association for Computing Machinery, New York, USA, 83--89.

Digital Library

[9]

G. LLC. Mobile Vision | Google Developers, 2019.

[10]

MacKenzie, I.S., Ashtiani, B. Blinkwrite: Efficient text entry using eye blinks. Universal Access Information Soc. 10, 1 (2011), 69--80.

[11]

Morris, M.R., Wobbrock, J.O., Wilson, A.D. Understanding users' preferences for surface gestures. In Proceedings of Graphics Interface (Toronto, Ontario, Canada, 2010), Canadian Information Processing Society, ACM Press, New York, USA, 261--268.

[12]

Naftali, M., Findlater, L. Accessibility in context: understanding the truly mobile experience of smartphone users with motor impairments. In Proceedings of the 16^th International ACM SIGACCESS Conference on Computers & Accessibility (2014), ACM Press, New York, USA, 209--216.

Digital Library

[13]

Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., et al. Scikit-learn: Machine learning in python. J. Mach. Learn, Res. 12, (2011), 2825--2830.

Digital Library

[14]

Pedrosa, D., Pimentel, M.D.G., Wright, A., Truong, K.N. Filteryedping: Design challenges and user performance of dwell-free eye typing. ACM Trans. Accessible Comput. 6, 1 (2015), 1--37.

Digital Library

[15]

Pradhan, A., Mehta, K., Findlater, L. "Accessibility came by accident" use of voice-controlled intelligent personal assistants by people with disabilities. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (2018), ACM Press, New York, USA, 1--13.

Digital Library

[16]

Printy, B.P., Renken, L.M., Herrmann, J.P., Lee, I., Johnson, B., Knight, E., Varga, G., Whitmer, D. Smartphone application for classification of motor impairment severity in parkinson's disease. In 2014 36^th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (2014), IEEE, 2686--2689.

[17]

Rozado, D., Niu, J., Lochner, M. Fast human-computer interaction by combining gaze pointing and face gestures. ACM Trans. Accessible Comput. 10, 3 (2017), 1--18.

Digital Library

[18]

Ruiz, J., Li, Y., Lank, E. User-defined motion gestures for mobile interaction. In Proceedings of the 2011 Annual Conference on Human Factors in Computing Systems (New York, New York, USA, 2011), ACM Press, New York, USA.

Digital Library

[19]

Sears, A., Karat, C.-M., Oseitutu, K., Karimullah, A., Feng, J. Productivity, satisfaction, and interaction strategies of individuals with spinal cord injuries and traditional users interacting with speech recognition software. Universal Access Information Soc. 1, 1 (2001), 4--15.

[20]

Stern, J.A., Walrath, L.C., Goldstein, R. The endogenous eyeblink. Psychophysiology 21, 1 (1984), 22--33.

[21]

White, N.-H., Black, N.-H. Spinal cord injury (sci) facts and figures at a glance. In National Spinal Cord Injury Statistical Center, Facts and Figures at a Glance, Birmingham (2016).

[22]

Wobbrock, J.O., Rubinstein, J., Sawyer, M.W., Duchowski, A.T. Longitudinal evaluation of discrete consecutive gaze gestures for text entry. In Proceedings of the 2008 Symposium on Eye Tracking Research & Applications (2008), ACM Press, New York, USA, 11--18.

Digital Library

[23]

Zhang, X., Kulkarni, H., Morris, M.R. Smartphone-based gaze gesture communication for people with motor disabilities. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (2017), ACM Press, New York, USA, 2878--2889.

Digital Library

Cited By

Lewis BKirupaharan PRanalli TVenkatasubramanian K(2024)A3C: An Image-Association-Based Computing Device Authentication Framework for People with Upper Extremity ImpairmentsACM Transactions on Accessible Computing10.1145/365252217:2(1-37)Online publication date: 28-May-2024
https://dl.acm.org/doi/10.1145/3652522
Tian JWang YYu KXu LXie JLi FNiu YFan M(2024)Designing Upper-Body Gesture Interaction with and for People with Spinal Muscular Atrophy in VRProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642884(1-19)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642884
Han HLi FBaca Vazquez AByrne DMartelaro NFox S(2024)Co-design Accessible Public Robots: Insights from People with Mobility Disability, Robotic Practitioners and Their CollaborationsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642875(1-20)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642875
Show More Cited By

Index Terms

Eyelid gestures for people with motor impairments
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction techniques
  2. Ubiquitous and mobile computing

Recommendations

Eyelid Gestures on Mobile Devices for People with Motor Impairments
ASSETS '20: Proceedings of the 22nd International ACM SIGACCESS Conference on Computers and Accessibility

Eye-based interactions for people with motor impairments have often used clunky or specialized equipment (e.g., eye-trackers with non-mobile computers) and primarily focused on gaze and blinks. However, two eyelids can open and close for different ...
Understanding Gesture Input Articulation with Upper-Body Wearables for Users with Upper-Body Motor Impairments
CHI '22: Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems

We examine touchscreen stroke-gestures and mid-air motion-gestures articulated by users with upper-body motor impairments with devices worn on the wrist, finger, and head. We analyze users’ gesture input performance in terms of production time, ...
Stroke-Gesture Input for People with Motor Impairments: Empirical Results & Research Roadmap
CHI '19: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems

We examine the articulation characteristics of stroke-gestures produced by people with upper body motor impairments on touchscreens as well as the accuracy rates of popular classification techniques, such as the $-family, to recognize those gestures. ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Communications of the ACM

Communications of the ACM Volume 65, Issue 1

January 2022

106 pages

ISSN:0001-0782

EISSN:1557-7317

DOI:10.1145/3507640

Editor:
Andrew A. Chien
Association for Computing Machinery, New York, NY

Issue’s Table of Contents

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 December 2021

Published in CACM Volume 65, Issue 1

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article
Popular
Refereed

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

13
Total Citations
View Citations
17,153
Total Downloads

Downloads (Last 12 months)2,314
Downloads (Last 6 weeks)27

Reflects downloads up to 22 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Lewis BKirupaharan PRanalli TVenkatasubramanian K(2024)A3C: An Image-Association-Based Computing Device Authentication Framework for People with Upper Extremity ImpairmentsACM Transactions on Accessible Computing10.1145/365252217:2(1-37)Online publication date: 28-May-2024
https://dl.acm.org/doi/10.1145/3652522
Tian JWang YYu KXu LXie JLi FNiu YFan M(2024)Designing Upper-Body Gesture Interaction with and for People with Spinal Muscular Atrophy in VRProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642884(1-19)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642884
Han HLi FBaca Vazquez AByrne DMartelaro NFox S(2024)Co-design Accessible Public Robots: Insights from People with Mobility Disability, Robotic Practitioners and Their CollaborationsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642875(1-20)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642875
Sun QHu YFan MLi JWang S(2024)“Can It Be Customized According to My Motor Abilities?”: Toward Designing User-Defined Head Gestures for People with DystoniaProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642378(1-11)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642378
Xiong ZFu SZhu YZhu CMa XFan M(2024)“It is hard to remove from my eye”: Design Makeup Residue Visualization System for Chinese Traditional Opera (Xiqu) PerformersProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642261(1-16)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642261
Han HLi FMartelaro NByrne DFox S(2023)The Robot in Our Path: Investigating the Perceptions of People with Motor Disabilities on Navigating Public Space Alongside Sidewalk RobotsProceedings of the 25th International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3597638.3614508(1-6)Online publication date: 22-Oct-2023
https://dl.acm.org/doi/10.1145/3597638.3614508
Vatavu R(2023)From Natural to Non-Natural Interaction: Embracing Interaction Design Beyond the Accepted Convention of NaturalProceedings of the 25th International Conference on Multimodal Interaction10.1145/3577190.3616122(684-688)Online publication date: 9-Oct-2023
https://dl.acm.org/doi/10.1145/3577190.3616122
Bilius LUngurean OVatavu R(2023)Understanding Wheelchair Users’ Preferences for On-Body, In-Air, and On-Wheelchair GesturesProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3580929(1-16)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3580929
Cruz CTatsuya NIchihara MShibata FKimura A(2023)Sequential Eyelid Gestures for User Interfaces in VR2023 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW)10.1109/VRW58643.2023.00201(717-718)Online publication date: Mar-2023
https://doi.org/10.1109/VRW58643.2023.00201
Liu MKuznetsov AKim YChang JKittur AMyers B(2022)Wigglite: Low-cost Information Collection and TriageProceedings of the 35th Annual ACM Symposium on User Interface Software and Technology10.1145/3526113.3545661(1-16)Online publication date: 29-Oct-2022
https://dl.acm.org/doi/10.1145/3526113.3545661
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Digital Edition

View this article in digital edition.

Digital Edition

Magazine Site

View this article on the magazine site (external)

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

Media

Figures

Other

Tables

View Issue’s Table of Contents