research-article

Effect of Motion-Gesture Recognizer Error Pattern on User Workload and Behavior

Authors:

Keiko Katsuragawa,

Edward LankAuthors Info & Claims

IUI '17: Proceedings of the 22nd International Conference on Intelligent User Interfaces

Pages 439 - 449

https://doi.org/10.1145/3025171.3025234

Published: 07 March 2017 Publication History

Abstract

Bi-level thresholding is a motion gesture recognition technique that mediates between false positives, and false negatives by using two threshold levels: a tighter threshold that limits false positives and recognition errors, and a looser threshold that prevents repeated errors (false negatives) by analyzing movements in sequence. In this paper, we examine the effects of bi-level thresholding on the workload and acceptance of end-users. Using a wizard-of-Oz recognizer, we hold recognition rates constant and adjust for fixed versus bi-level thresholding. Given identical recognition rates, we show that systems using bi-level thresholding result in significant lower workload scores on the NASA-TLX and accelerometer variance. Overall, these results argue for the viability of bi-level thresholding as an effective technique for balancing between false positives, recognition errors and false negatives.

References

[1]

Android studio. https://developer.android.com/studio/index.html.

[2]

Motox. https://motorola-global-portal.custhelp.com/app/answers/prod_answer_detail/a_id/96251/p/30,6720,8696.

[3]

Smart profile. https://motorola-global-portal.custhelp.com/app/answers/detail/a_id/51540/~/droid-x--smart-profile.

[4]

Ashbrook, D., and Starner, T. Magic: A motion gesture design tool. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI '10, ACM (NY, NY, USA, 2010), 2159--2168.

Digital Library

[5]

Bartlett, J. F. Rock 'n' scroll is here to stay. IEEE Comput. Graph. Appl. 20, 3 (May 2000), 40--45.

Digital Library

[6]

Bolt, R. "Put-that-there". In Proceedings of the 7th annual conference on Computer graphics and interactive techniques SIGGRAPH '80 (1980), 262--270.

Digital Library

[7]

Dai, L., Sears, A., and Goldman, R. Shifting the focus from accuracy to recallability: A study of informal note-taking on mobile information technologies. ACM Trans. Comput.-Hum. Interact. 16, 1 (Apr. 2009), 4:1--4:46.

Digital Library

[8]

Duda, R. O., Hart, P. E., and Stork, D. G. Pattern classification. John Wiley & Sons, 2012.

Digital Library

[9]

Fawcett, T. An introduction to roc analysis. Pattern Recogn. Lett. 27, 8 (June 2006), 861--874.

Digital Library

[10]

Flash, T., and Hogans, N. The coordination of arm movements: An experimentally confirmed mathematical model. Journal of neuroscience 5 (1985), 1688--1703.

[11]

Haque, F., Nancel, M., and Vogel, D. Myopoint: Pointing and clicking using forearm mounted electromyography and inertial motion sensors. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, CHI '15, ACM (NY, NY, USA, 2015), 3653--3656.

Digital Library

[12]

Hart, G. S., and Staveland, E. L. Development of nasa-tlx (task load index): results of empirical and theoretical research. Human Mental Workload 52 (1988), 139--183.

[13]

Hartmann, B., Abdulla, L., Mittal, M., and Klemmer, S. R. Authoring sensor-based interactions by demonstration with direct manipulation and pattern recognition. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI '07, ACM (NY, NY, USA, 2007), 145--154.

Digital Library

[14]

Hastie, T., Tibshirani, R., and Friedman, J. The Elements of Statistical Learning, 2 ed. 2009.

[15]

Hinckley, K., Pierce, J., Sinclair, M., and Horvitz, E. Sensing techniques for mobile interaction. In Proceedings of the 13th Annual ACM Symposium on User Interface Software and Technology, UIST '00, ACM (NY, NY, USA, 2000), 91--100.

Digital Library

[16]

Jones, E., Alexander, J., Andreou, A., Irani, P., and Subramanian, S. Gestext: Accelerometer-based gestural text-entry systems. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI '10, ACM (NY, NY, USA, 2010), 2173--2182.

Digital Library

[17]

Kamal, A., Li, Y., and Lank, E. Teaching motion gestures via recognizer feedback. In Proceedings of the 19th International Conference on Intelligent User Interfaces, IUI '14, ACM (NY, NY, USA, 2014), 73--82.

Digital Library

[18]

Katsuragawa, K., Pietroszek, K., Wallace, J. R., and Lank, E. Watchpoint: Freehand pointing with a smartwatch in a ubiquitous display environment. In Proceedings of the International Working Conference on Advanced Visual Interfaces, AVI '16, ACM (NY, NY, USA, 2016), 128--135.

Digital Library

[19]

Katsuragawa, K., Wallace, J. R., and Lank, E. Gestural text input using a smartwatch. In Proceedings of the International Working Conference on Advanced Visual Interfaces, AVI '16, ACM (NY, NY, USA, 2016), 220--223.

Digital Library

[20]

Li, F. C. Y., Dearman, D., and Truong, K. N. Virtual shelves: Interactions with orientation aware devices. In Proceedings of the 22Nd Annual ACM Symposium on User Interface Software and Technology, UIST '09, ACM (NY, NY, USA, 2009), 125--128.

Digital Library

[21]

Li, Y. Protractor: A fast and accurate gesture recognizer. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI '10, ACM (NY, NY, USA, 2010), 2169--2172.

Digital Library

[22]

Liu, J., Zhong, L., Wickramasuriya, J., and Vasudevan, V. User evaluation of lightweight user authentication with a single tri-axis accelerometer. In Proceedings of the 11th International Conference on Human-Computer Interaction with Mobile Devices and Services, MobileHCI '09, ACM (NY, NY, USA, 2009), 15:1--15:10.

Digital Library

[23]

Mignot, C., Valot, C., and Carbonell, N. An experimental study of future "natural" multimodal human-computer interaction. In INTERACT '93 and CHI '93 Conference Companion on Human Factors in Computing Systems, CHI '93, ACM (NY, NY, USA, 1993), 67--68.

Digital Library

[24]

Nagy, G. 29 optical character recognition - theory and practice. Handbook of statistics 2 (1982), 621--649.

[25]

Negulescu, M., Ruiz, J., and Lank, E. A recognition safety net: Bi-level threshold recognition for mobile motion gestures. In Proceedings of the 14th International Conference on Human-computer Interaction with Mobile Devices and Services, MobileHCI '12, ACM (NY, NY, USA, 2012), 147--150.

Digital Library

[26]

Negulescu, M., Ruiz, J., Li, Y., and Lank, E. Tap, swipe, or move: Attentional demands for distracted smartphone input. In Proceedings of the International Working Conference on Advanced Visual Interfaces, AVI '12, ACM (NY, NY, USA, 2012), 173--180.

Digital Library

[27]

Partridge, K., Chatterjee, S., Sazawal, V., Borriello, G., and Want, R. Tilttype: Accelerometer-supported text entry for very small devices. In Proceedings of the 15th Annual ACM Symposium on User Interface Software and Technology, UIST '02, ACM (NY, NY, USA, 2002), 201--204.

Digital Library

[28]

Rekimoto, J. Tilting operations for small screen interfaces. In Proceedings of the 9th Annual ACM Symposium on User Interface Software and Technology, UIST '96, ACM (NY, NY, USA, 1996), 167--168.

Digital Library

[29]

Rigoll, G., Kosmala, A., and Eickeler, S. High performance real-time gesture recognition using hidden markov models. In In Proc. Gesture Workshop, Springer (1998), 69--80.

Digital Library

[30]

Rubine, D. Specifying gestures by example. SIGGRAPH Comput. Graph. 25, 4 (July 1991), 329--337.

Digital Library

[31]

Ruiz, J., and Li, Y. Doubleflip: A motion gesture delimiter for mobile interaction. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI '11, ACM (NY, NY, USA, 2011), 2717--2720.

Digital Library

[32]

Ruiz, J., Li, Y., and Lank, E. User-defined motion gestures for mobile interaction. In Proceedings of the 2011 annual conference on Human factors in computing systems, CHI '11, ACM (NY, NY, USA, 2011), 197--206.

Digital Library

[33]

Sezgin, T. M., and Davis, R. Hmm-based efficient sketch recognition. In Proceedings of the 10th International Conference on Intelligent User Interfaces, IUI '05, ACM (NY, NY, USA, 2005), 281--283.

Digital Library

[34]

Small, D., and Ishii, H. Design of spatially aware graspable displays. In CHI '97 Extended Abstracts on Human Factors in Computing Systems, CHI EA '97, ACM (NY, NY, USA, 1997), 367--368.

Digital Library

[35]

Uchida, S., and Sakoe, H. A survey of elastic matching techniques for handwritten character recognition. IEICE Trans. Inf. Syst. E88-D, 8 (Aug. 2005), 1781--1790.

Digital Library

[36]

Vintsyuk, T. Speech discrimination by dynamic programming. Kybernetika (1968).

[37]

Vogel, D., and Balakrishnan, R. Distant freehand pointing and clicking on very large, high resolution displays. In Proceedings of the 18th Annual ACM Symposium on User Interface Software and Technology, UIST '05, ACM (NY, NY, USA, 2005), 33--42.

Digital Library

[38]

Voida, S., Podlaseck, M., Kjeldsen, R., and Pinhanez, C. A study on the manipulation of 2d objects in a projector/camera-based augmented reality environment. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI '05, ACM (NY, NY, USA, 2005), 611--620.

Digital Library

[39]

Weberg, L., Brange, T., and Hansson, A. W. A piece of butter on the pda display. In CHI '01 Extended Abstracts on Human Factors in Computing Systems, CHI EA '01, ACM (NY, NY, USA, 2001), 435--436.

Digital Library

[40]

Wigdor, D., and Balakrishnan, R. Tilttext: Using tilt for text input to mobile phones. In Proceedings of the 16th Annual ACM Symposium on User Interface Software and Technology, UIST '03, ACM (NY, NY, USA, 2003), 81--90.

Digital Library

[41]

Wobbrock, J. O., Morris, M. R., and Wilson, A. D. User-defined gestures for surface computing. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI '09, ACM (NY, NY, USA, 2009), 1083--1092.

Digital Library

[42]

Wobbrock, J. O., Wilson, A. D., and Li, Y. Gestures Without Libraries, Toolkits or Training: A $1 Recognizer for User Interface Prototypes. In Proceedings of the 20th Annual ACM Symposium on User Interface Software and Technology, UIST '07, ACM (NY, NY, USA, 2007), 159--168.

Digital Library

[43]

Yesilada, Y., Stevens, R., Harper, S., and Goble, C. Evaluating dante: Semantic transcoding for visually disabled users. ACM Trans. Comput.-Hum. Interact. 14, 3 (Sept. 2007).

Digital Library

Cited By

Henderson JJonker TLank EWigdor DLafreniere B(2022)Investigating Cross-Modal Approaches for Evaluating Error Acceptability of a Recognition-Based Input TechniqueProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/35172626:1(1-24)Online publication date: 29-Mar-2022
https://dl.acm.org/doi/10.1145/3517262
Xia HGlueck MAnnett MWang MWigdor D(2022)Iteratively Designing Gesture Vocabularies: A Survey and Analysis of Best Practices in the HCI LiteratureACM Transactions on Computer-Human Interaction10.1145/350353729:4(1-54)Online publication date: 5-May-2022
https://dl.acm.org/doi/10.1145/3503537
Rehman IUllah SKhan D(2022)FPSI-Fingertip pose and state-based natural interaction techniques in virtual environmentsMultimedia Tools and Applications10.1007/s11042-022-13824-w82:14(20711-20740)Online publication date: 7-Oct-2022
https://doi.org/10.1007/s11042-022-13824-w
Show More Cited By

Index Terms

Effect of Motion-Gesture Recognizer Error Pattern on User Workload and Behavior
1. Human-centered computing
  1. Human computer interaction (HCI)

Recommendations

Bi-Level Thresholding: Analyzing the Effect of Repeated Errors in Gesture Input
Special Issue on Highlights of ACM IUI 2017

In gesture recognition, one challenge that researchers and developers face is the need for recognition strategies that mediate between false positives and false negatives. In this article, we examine bi-level thresholding, a recognition strategy that ...
Face and body gesture recognition for a vision-based multimodal analyzer
VIP '05: Proceedings of the Pan-Sydney area workshop on Visual information processing

For the computer to interact intelligently with human users, computers should be able to recognize emotions, by analyzing the human's affective state, physiology and behavior. In this paper, we present a survey of research conducted on face and body ...
A Hybrid Method of Feature Extraction for Facial Expression Recognition
SITIS '11: Proceedings of the 2011 Seventh International Conference on Signal Image Technology & Internet-Based Systems

Facial Expression Recognition is necessary for designingany human-machine interface. The main issue of FacialExpression Recognition is to decide what features are requiredto represent a Facial Expression. In this paper, we proposethe hybrid technique ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

IUI '17: Proceedings of the 22nd International Conference on Intelligent User Interfaces

March 2017

654 pages

ISBN:9781450343480

DOI:10.1145/3025171

General Chairs:
George A. Papadopoulos
The University of Cyprus, Cyprus
,
Tsvi Kuflik
The University of Haifa, Israel
,
Program Chairs:
Fang Chen
Data61, CSIRO Australia
,
Carlos Duarte
The University of Lisbon, Portugal
,
Wai-Tat Fu
The University of Illinois at Urbana-Champaign, USA

Copyright © 2017 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 March 2017

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Conference

IUI'17

Sponsor:

IUI'17: 22nd International Conference on Intelligent User Interfaces

March 13 - 16, 2017

Limassol, Cyprus

Acceptance Rates

IUI '17 Paper Acceptance Rate 63 of 272 submissions, 23%;

Overall Acceptance Rate 746 of 2,811 submissions, 27%

Upcoming Conference

IUI '25

Sponsor:
sigai
sigai

30th International Conference on Intelligent User Interfaces

March 24 - 27, 2025

Cagliari , Italy

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

8
Total Citations
View Citations
332
Total Downloads

Downloads (Last 12 months)26
Downloads (Last 6 weeks)2

Reflects downloads up to 19 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Henderson JJonker TLank EWigdor DLafreniere B(2022)Investigating Cross-Modal Approaches for Evaluating Error Acceptability of a Recognition-Based Input TechniqueProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/35172626:1(1-24)Online publication date: 29-Mar-2022
https://dl.acm.org/doi/10.1145/3517262
Xia HGlueck MAnnett MWang MWigdor D(2022)Iteratively Designing Gesture Vocabularies: A Survey and Analysis of Best Practices in the HCI LiteratureACM Transactions on Computer-Human Interaction10.1145/350353729:4(1-54)Online publication date: 5-May-2022
https://dl.acm.org/doi/10.1145/3503537
Rehman IUllah SKhan D(2022)FPSI-Fingertip pose and state-based natural interaction techniques in virtual environmentsMultimedia Tools and Applications10.1007/s11042-022-13824-w82:14(20711-20740)Online publication date: 7-Oct-2022
https://doi.org/10.1007/s11042-022-13824-w
Hertzum M(2021)Associations among workload dimensions, performance, and situational characteristics: a meta-analytic review of the Task Load IndexBehaviour & Information Technology10.1080/0144929X.2021.2000642(1-13)Online publication date: 10-Nov-2021
https://doi.org/10.1080/0144929X.2021.2000642
Völkel SSchneegass CEiband MBuschek DPaternò FOliver NConati CSpano LTintarev N(2020)What is "intelligent" in intelligent user interfaces?Proceedings of the 25th International Conference on Intelligent User Interfaces10.1145/3377325.3377500(477-487)Online publication date: 17-Mar-2020
https://dl.acm.org/doi/10.1145/3377325.3377500
Roy QZhang FVogel DBrewster SFitzpatrick GCox AKostakos V(2019)Automation Accuracy Is Good, but High Controllability May Be BetterProceedings of the 2019 CHI Conference on Human Factors in Computing Systems10.1145/3290605.3300750(1-8)Online publication date: 2-May-2019
https://dl.acm.org/doi/10.1145/3290605.3300750
Katsuragawa KKamal ALiu QNegulescu MLank E(2019)Bi-Level ThresholdingACM Transactions on Interactive Intelligent Systems10.1145/31816729:2-3(1-30)Online publication date: 2-Apr-2019
https://dl.acm.org/doi/10.1145/3181672
Feng SFeng ZCao L(2019)Many-to-One Gesture-to-Command Flexible Mapping Approach for Smart Teaching Interface InteractionIEEE Access10.1109/ACCESS.2019.29573657(179517-179531)Online publication date: 2019
https://doi.org/10.1109/ACCESS.2019.2957365

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents