research-article

Differences and Similarities between Finger and Pen Stroke Gestures on Stationary and Mobile devices

Authors:

Shumin ZhaiAuthors Info & Claims

ACM Transactions on Computer-Human Interaction (TOCHI), Volume 22, Issue 5

Article No.: 22, Pages 1 - 39

https://doi.org/10.1145/2797138

Published: 10 August 2015 Publication History

Abstract

This study investigated differences and similarities between finger and pen gestures on stationary devices (sitting posture) and mobile devices (sitting and walking postures). The recorded gestures were analyzed according to multiple gesture features. We found (1) pen and index finger gestures were different in features like size ratio but similar in features like angle difference; (2) implement (pen vs. index finger vs. thumb) interacted with gesture complexity and size in features like articulation time; (3) features like time and shape distance, were different between the pen and index finger on mobile devices (walking) but similar on stationary devices; (4) one-handed thumb gestures had worse performances than index finger gestures by time and accuracy in sitting but similar performances in walking; and (5) for the three implements, gesture drawing time and accuracy on mobile devices reduced from sitting to walking condition. We discuss these findings with implications for future gesture design and research.

References

[1]

Dzimitry Aliakseyeu, Pourang Irani, Andrés Lucero, and Sriram Subramanian. 2008. Multi-flick: An evaluation of flick-based scrolling techniques for pen interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’08). ACM, New York, NY, USA, 1689--1698. org/10.1145/1357054.1357319

Digital Library

[2]

Tue Haste Andersen and Shumin Zhai. 2010. “Writing with music”: Exploring the use of auditory feedback in gesture interfaces. ACM Trans. Appl. Percept. 7, 3, Article 17 (June 2010), 24 pages. org/10.1145/1773965.1773968

Digital Library

[3]

Fraser Anderson and Walter F. Bischof. 2013. Learning and performance with gesture guides. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’13). ACM, New York, NY, USA, 1109--1118.

Digital Library

[4]

Michelle Annett and Walter F. Bischof. 2013. Your left hand can do it too&excl;: Investigating intermanual, symmetric gesture transfer on touchscreens. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’13). ACM, New York, NY, USA, 1119--1128.

Digital Library

[5]

Lisa Anthony, Radu-Daniel Vatavu, and Jacob O. Wobbrock. 2013. Understanding the consistency of users’ pen and finger stroke gesture articulation. In Proceedings of the 2013 Graphics Interface Conference (GI’13). Canadian Information Processing Society, Toronto, ON, Canada, 87--94.

Digital Library

[6]

Caroline Appert and Olivier Bau. 2010. Scale detection for a priori gesture recognition. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’10). ACM, New York, NY, USA, 879--882.

Digital Library

[7]

Caroline Appert and Shumin Zhai. 2009. Using strokes as command shortcuts: Cognitive benefits and toolkit support. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’09). ACM, New York, NY, USA, 2289--2298.

Digital Library

[8]

Shiri Azenkot and Shumin Zhai. 2012. Touch behavior with different postures on soft smartphone keyboards. In Proceedings of the 14th International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI’12). ACM, New York, NY, USA, 251--260.

Digital Library

[9]

Olivier Bau and Wendy E. Mackay. 2008. OctoPocus: A dynamic guide for learning gesture-based command sets. In Proceedings of the ACM Symposium on User Interface Software and Technology (UIST’08). ACM, New York, NY, USA, 37--46.

Digital Library

[10]

Xiaojun Bi, Ciprian Chelba, Tom Ouyang, Kurt Partridge, and Shumin Zhai. 2012. Bimanual gesture keyboard. In Proceedings of the ACM Symposium on User Interface Software and Technology (UIST’12). ACM, New York, NY, USA, 137--146.

Digital Library

[11]

Rachel Blagojevic, Samuel Hsiao-Heng Chang, and Beryl Plimmer. 2010. The power of automatic feature selection: Rubine on steroids. In Proceedings of the Seventh Sketch-Based Interfaces and Modeling Symposium (SBIM’10). Eurographics Association, Aire-la-Ville, Switzerland, 79--86. http://dl.acm.org/citation. cfm?id=1923363.1923377.

Digital Library

[12]

Andrew Bragdon, Eugene Nelson, Yang Li, and Ken Hinckley. 2011. Experimental analysis of touch-screen gesture designs in mobile environments. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’11). ACM, New York, NY, USA, 403--412.

Digital Library

[13]

Andrew Bragdon, Robert Zeleznik, Brian Williamson, Timothy Miller, and Joseph J. LaViola, Jr. 2009. GestureBar: Improving the approachability of gesture-based interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’09). ACM, New York, NY, USA, 2269--2278.

Digital Library

[14]

Xiang Cao and Shumin Zhai. 2007. Modeling human performance of pen stroke gestures. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’07). ACM, New York, NY, USA, 1495--1504.

Digital Library

[15]

Mathias Frisch, Jens Heydekorn, and Raimund Dachselt. 2009. Investigating multi-touch and pen gestures for diagram editing on interactive surfaces. In Proceedings of the ACM International Conference on Interactive Tabletops and Surfaces (ITS’09). ACM, New York, NY, USA, 149--156.

Digital Library

[16]

Vittorio Fuccella, Poika Isokoski, and Benoit Martin. 2013. Gestures and widgets: Performance in text editing on multi-touch capable mobile devices. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’13). ACM, New York, NY, USA, 2785--2794.

Digital Library

[17]

Mayank Goel, Alex Jansen, Travis Mandel, Shwetak N. Patel, and Jacob O. Wobbrock. 2013. ContextType: Using hand posture information to improve mobile touch screen text entry. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’13). ACM, New York, NY, USA, 2795--2798.

Digital Library

[18]

Mayank Goel, Jacob Wobbrock, and Shwetak Patel. 2012. GripSense: Using built-in sensors to detect hand posture and pressure on commodity mobile phones. In Proceedings of the 25th Annual ACM Symposium on User Interface Software and Technology (UIST’12). ACM, New York, NY, USA, 545--554.

Digital Library

[19]

David Goldberg and Cate Richardson. 1993. Touch-typing with a stylus. In Proceedings of the INTERACT’93 and Conference on Human Factors in Computing Systems (CHI’93). ACM, New York, NY, USA, 80--87.

Digital Library

[20]

Ken Hinckley, Patrick Baudisch, Gonzalo Ramos, and Francois Guimbretiere. 2005. Design and analysis of delimiters for selection-action pen gesture phrases in scriboli. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’05). ACM, New York, NY, USA, 451--460.

Digital Library

[21]

Ken Hinckley, Michel Pahud, Hrvoje Benko, Pourang Irani, François Guimbretière, Marcel Gavriliu, Xiang ‘Anthony’ Chen, Fabrice Matulic, William Buxton, and Andrew Wilson. 2014. Sensing techniques for tablet+stylus interaction. In Proceedings of the 27th Annual ACM Symposium on User Interface Software and Technology (UIST’14). ACM, New York, NY, USA, 605--614.

Digital Library

[22]

Ken Hinckley, Koji Yatani, Michel Pahud, Nicole Coddington, Jenny Rodenhouse, Andy Wilson, Hrvoje Benko, and Bill Buxton. 2010. Pen + touch = new tools. In Proceedings of the ACM Symposium on User Interface Software and Technology (UIST’10). ACM, New York, NY, USA, 27--36.

Digital Library

[23]

HP 2012. TouchSmart tx2 Notebook PC - Maintenance and Service Guide. Retrieved from http://www.hp.com.

[24]

HTC 2009. HTC Touch2 T3333. Retrieved from http://www.htc.com.

[25]

HTC 2010. HTC HD Mini. Retrieved from http://www.htc.com.

[26]

Poika Isokoski. 2001. Model for unistroke writing time. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’01). ACM, New York, NY, USA, 357--364.

Digital Library

[27]

Shaun K. Kane, Jacob O. Wobbrock, and Richard E. Ladner. 2011. Usable gestures for blind people: Understanding preference and performance. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’11). ACM, New York, NY, USA, 413--422.

Digital Library

[28]

Amy K. Karlson and Benjamin B. Bederson. 2006. Understanding Single-Handed Mobile Device Interaction. Technical Report Tech report HCIL-2006-02. Computer Science Dept., University of Maryland, College Park, MD, USA.

[29]

Per-Ola Kristensson and Shumin Zhai. 2004. SHARK²: A large vocabulary shorthand writing system for pen-based computers. In Proceedings of the ACM Symposium on User Interface Software and Technology (UIST’04). ACM, New York, NY, USA, 43--52.

Digital Library

[30]

Gordon P. Kurtenbach, Abigail J. Sellen, and William A. S. Buxton. 1993. An empirical evaluation of some articulatory and cognitive aspects of marking menus. Hum.-Comput. Interact. 8, 1 (March 1993), 1--23.

Digital Library

[31]

Francesco Lacquaniti, Carlo Terzuolo, and Paolo Viviani. 1983. The law relating the kinematic and figural aspects of drawing movements. Acta Psychologica 54, 1--3, 115--130.

[32]

Yang Li. 2010. Gesture search: A tool for fast mobile data access. In Proceedings of the ACM Symposium on User Interface Software and Technology (UIST’10). ACM, New York, NY, USA, 87--96.

Digital Library

[33]

Allan Christian Long, Jr., James A. Landay, and Lawrence A. Rowe. 1999. Implications for a gesture design tool. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’99). ACM, New York, NY, USA, 40--47.

Digital Library

[34]

A. Chris Long, Jr., James A. Landay, Lawrence A. Rowe, and Joseph Michiels. 2000. Visual similarity of pen gestures. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’00). ACM, New York, NY, USA, 360--367.

Digital Library

[35]

Meredith Ringel Morris, Anqi Huang, Andreas Paepcke, and Terry Winograd. 2006. Cooperative gestures: Multi-user gestural interactions for co-located groupware. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’06). ACM, New York, NY, USA, 1201--1210.

Digital Library

[36]

Alexander Ng, Stephen A. Brewster, and John H. Williamson. 2014. Investigating the effects of encumbrance on one-and two-handed interactions with mobile devices. In Proceedings of the 32nd Annual ACM Conference on Human Factors in Computing Systems. ACM, New York, NY, USA, 1981--1990.

Digital Library

[37]

Ntrig 2013. DuoSense Digitizer Product Brief. Retrieved from http://www.n-trig.com.

[38]

Keith B. Perry and Juan Pablo Hourcade. 2008. Evaluating one handed thumb tapping on mobile touchscreen devices. In Proceedings of Graphics Interface (GI’08). Canadian Information Processing Society, Toronto, ON, Canada, 57--64.

Digital Library

[39]

Xiangshi Ren and Shinju Moriya. 2000. Improving selection performance on pen-based systems: A study of pen-based interaction for selection tasks. ACM Trans. Comput.-Hum. Interact. 7, 3 (September 2000), 384--416.

Digital Library

[40]

Xiangshi Ren and Xiaolei Zhou. 2009. The optimal size of handwriting character input boxes on PDAs. Int. J. Hum.-Comput. Interact. 25, 8, 762--784.

[41]

Dean Rubine. 1991. Specifying gestures by example. SIGGRAPH Comput. Graph. 25, 4 (July 1991), 329--337.

Digital Library

[42]

Samsung 2013. Samsung Galaxy Note 3 smart phone. Retrieved from http://www.samsung.com/.

[43]

R. A. Schmidt. 1988. Motor Control and Learning: A Behavioral Emphasis. Human Kinetics, Champaign, IL, USA. http://books.google.co.uk/books?id=KappAAAAMAAJ.

[44]

Huawei Tu and Xiangshi Ren. 2013. Optimal entry size of handwritten chinese characters in touch-based mobile phones. Int. J. Hum.-Comput. Interact. 29, 1, 1--12.

[45]

Huawei Tu, Xiangshi Ren, and Shumin Zhai. 2012. A comparative evaluation of finger and pen stroke gestures. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’12). ACM, New York, NY, USA, 1287--1296.

Digital Library

[46]

Radu-Daniel Vatavu. 2011. The effect of sampling rate on the performance of template-based gesture recognizers. In Proceedings of the 13th International Conference on Multimodal Interfaces (ICMI’11). ACM, New York, NY, USA, 271--278.

Digital Library

[47]

Radu-Daniel Vatavu, Géry Casiez, and Laurent Grisoni. 2013. Small, medium, or large?: Estimating the user-perceived scale of stroke gestures. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’13). ACM, New York, NY, USA, 277--280.

Digital Library

[48]

Jacob O. Wobbrock, Meredith Ringel Morris, and Andrew D. Wilson. 2009. User-defined gestures for surface computing. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’09). ACM, New York, NY, USA, 1083--1092.

Digital Library

[49]

Jacob O. Wobbrock, Andrew D. Wilson, and Yang Li. 2007. Gestures without libraries, toolkits or training: A &dollar;1 recognizer for user interface prototypes. In Proceedings of the ACM Symposium on User Interface Software and Technology (UIST’07). ACM, New York, NY, USA, 159--168.

Digital Library

[50]

C. E. Wright. 1990. Generalized motor programs: Reexamining claims of effector independence in writing. In Attention and Performance XIII: Motor Representation and Control. Erlbaum, Hillsdale, NJ, USA, 294--320.

[51]

Ying Yin, Tom Yu Ouyang, Kurt Partridge, and Shumin Zhai. 2013. Making touchscreen keyboards adaptive to keys, hand postures, and individuals: A hierarchical spatial backoff model approach. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’13). ACM, New York, NY, USA, 2775--2784.

Digital Library

[52]

S. Zhai, P. O. Kristensson, C. Appert, T. H. Anderson, and X. Cao. 2012. Foundational issues in touch-surface stroke gesture design—An integrative review. Foundations Trends Hum-Comput. Interact. 5, 2, 97--205.

Digital Library

[53]

Shumin Zhai and Per-Ola Kristensson. 2003. Shorthand writing on stylus keyboard. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’03). ACM, New York, NY, USA, 97--104.

Digital Library

[54]

Shumin Zhai and Per Ola Kristensson. 2012. The word-gesture keyboard: Reimagining keyboard interaction. Commun. ACM 55, 9 (September 2012), 91--101.

Digital Library

[55]

Shumin Zhai, Per Ola Kristensson, Pengjun Gong, Michael Greiner, Shilei Allen Peng, Liang Mico Liu, and Anthony Dunnigan. 2009. Shapewriter on the iphone: From the laboratory to the real world. In CHI’09 Extended Abstracts on Human Factors in Computing Systems (CHI EA’09). ACM, New York, NY, USA, 2667--2670.

Digital Library

Cited By

Raj HSamia Basumatary JBhowmick S(2025)Culturally Relevant Novel Interaction Methods for Immersive Video Streaming Experience in Virtual RealityHuman-Computer Interaction. Design and Research10.1007/978-3-031-80829-6_12(259-272)Online publication date: 14-Feb-2025
https://doi.org/10.1007/978-3-031-80829-6_12
Tu HGao BWu HLyu F(2023)Text Pin: Improving text selection with mode-augmented handles on touchscreen mobile devicesInternational Journal of Human-Computer Studies10.1016/j.ijhcs.2023.103028175(103028)Online publication date: Jul-2023
https://doi.org/10.1016/j.ijhcs.2023.103028
Wang MQin GSun MZhao JQin J(2023)An exploration of pressure input with bare finger for Mobile interaction in stationary and Mobile situationsMultimedia Tools and Applications10.1007/s11042-023-14503-082:17(25711-25731)Online publication date: 14-Feb-2023
https://dl.acm.org/doi/10.1007/s11042-023-14503-0
Show More Cited By

Index Terms

Differences and Similarities between Finger and Pen Stroke Gestures on Stationary and Mobile devices
1. Human-centered computing
  1. Human computer interaction (HCI)

Recommendations

A comparative evaluation of finger and pen stroke gestures
CHI '12: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

This paper reports an empirical investigation in which participants produced a set of stroke gestures with varying degrees of complexity and in different target sizes using both the finger and the pen. The recorded gestures were then analyzed according ...
Exploring pinch and spread gestures on mobile devices
MobileHCI '13: Proceedings of the 15th international conference on Human-computer interaction with mobile devices and services

Pinching and spreading gestures are prevalent in mobile applications today, but these gestures have not yet been studied extensively. We conducted an exploratory study of pinch and spread gestures with seated participants on a phone and a tablet device. ...
iWink: Exploring Eyelid Gestures on Mobile Devices
HuMA'20: Proceedings of the 1st International Workshop on Human-centric Multimedia Analysis

Although gaze has been widely studied for mobile interactions, eyelid-based gestures are relatively understudied and limited to few basic gestures (e.g., blink). In this work, we propose a gesture grammar to construct both basic and compound eyelid ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Transactions on Computer-Human Interaction

ACM Transactions on Computer-Human Interaction Volume 22, Issue 5

October 2015

217 pages

ISSN:1073-0516

EISSN:1557-7325

DOI:10.1145/2814459

Editor:
Shumin Zhai
Google, Inc.

Issue’s Table of Contents

Copyright © 2015 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 10 August 2015

Accepted: 01 June 2015

Revised: 01 May 2015

Received: 01 April 2014

Published in TOCHI Volume 22, Issue 5

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

Japan and NSFC of China
Grant-in-Aid for Scientific Research

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

28
Total Citations
View Citations
869
Total Downloads

Downloads (Last 12 months)22
Downloads (Last 6 weeks)2

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Raj HSamia Basumatary JBhowmick S(2025)Culturally Relevant Novel Interaction Methods for Immersive Video Streaming Experience in Virtual RealityHuman-Computer Interaction. Design and Research10.1007/978-3-031-80829-6_12(259-272)Online publication date: 14-Feb-2025
https://doi.org/10.1007/978-3-031-80829-6_12
Tu HGao BWu HLyu F(2023)Text Pin: Improving text selection with mode-augmented handles on touchscreen mobile devicesInternational Journal of Human-Computer Studies10.1016/j.ijhcs.2023.103028175(103028)Online publication date: Jul-2023
https://doi.org/10.1016/j.ijhcs.2023.103028
Wang MQin GSun MZhao JQin J(2023)An exploration of pressure input with bare finger for Mobile interaction in stationary and Mobile situationsMultimedia Tools and Applications10.1007/s11042-023-14503-082:17(25711-25731)Online publication date: 14-Feb-2023
https://dl.acm.org/doi/10.1007/s11042-023-14503-0
Magrofuoco NRoselli PVanderdonckt J(2022)µV: An Articulation, Rotation, Scaling, and Translation Invariant (ARST) Multi-stroke Gesture RecognizerProceedings of the ACM on Human-Computer Interaction10.1145/35322006:EICS(1-25)Online publication date: 17-Jun-2022
https://dl.acm.org/doi/10.1145/3532200
Xia HGlueck MAnnett MWang MWigdor D(2022)Iteratively Designing Gesture Vocabularies: A Survey and Analysis of Best Practices in the HCI LiteratureACM Transactions on Computer-Human Interaction10.1145/350353729:4(1-54)Online publication date: 5-May-2022
https://dl.acm.org/doi/10.1145/3503537
Sung CCheng PChen L(2022)Effect of colour sensing and drawing pen on children’s colour perception and feature detection of objects when drawingBehaviour & Information Technology10.1080/0144929X.2022.207176442:9(1311-1323)Online publication date: 6-May-2022
https://doi.org/10.1080/0144929X.2022.2071764
Li QZeng HPeng ZMa X(2021)Understanding Players’ Interaction Patterns with Mobile Game App UI via VisualizationsProceedings of the Ninth International Symposium of Chinese CHI10.1145/3490355.3490357(9-21)Online publication date: 16-Oct-2021
https://dl.acm.org/doi/10.1145/3490355.3490357
Tu HHuang JLiang HSkarbez RTian FDuh HKitamura YQuigley AIsbister KIgarashi TBjørn PDrucker S(2021)Distractor Effects on Crossing-Based InteractionProceedings of the 2021 CHI Conference on Human Factors in Computing Systems10.1145/3411764.3445340(1-13)Online publication date: 6-May-2021
https://dl.acm.org/doi/10.1145/3411764.3445340
Shukri SOmar RRubegni EVasalou AParés NSawhney N(2020)The graphic routines of closed and open shapesProceedings of the 2020 ACM Interaction Design and Children Conference: Extended Abstracts10.1145/3397617.3397844(229-234)Online publication date: 21-Jun-2020
https://dl.acm.org/doi/10.1145/3397617.3397844
Akpinar EYeşilada YTemizer S(2020)The Effect of Context on Small Screen and Wearable Device Users’ Performance - A Systematic ReviewACM Computing Surveys10.1145/338637053:3(1-44)Online publication date: 28-May-2020
https://dl.acm.org/doi/10.1145/3386370
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Issue’s Table of Contents