Nothing Special   »   [go: up one dir, main page]

skip to main content
research-article

Differences and Similarities between Finger and Pen Stroke Gestures on Stationary and Mobile devices

Published: 10 August 2015 Publication History

Abstract

This study investigated differences and similarities between finger and pen gestures on stationary devices (sitting posture) and mobile devices (sitting and walking postures). The recorded gestures were analyzed according to multiple gesture features. We found (1) pen and index finger gestures were different in features like size ratio but similar in features like angle difference; (2) implement (pen vs. index finger vs. thumb) interacted with gesture complexity and size in features like articulation time; (3) features like time and shape distance, were different between the pen and index finger on mobile devices (walking) but similar on stationary devices; (4) one-handed thumb gestures had worse performances than index finger gestures by time and accuracy in sitting but similar performances in walking; and (5) for the three implements, gesture drawing time and accuracy on mobile devices reduced from sitting to walking condition. We discuss these findings with implications for future gesture design and research.

References

[1]
Dzimitry Aliakseyeu, Pourang Irani, Andrés Lucero, and Sriram Subramanian. 2008. Multi-flick: An evaluation of flick-based scrolling techniques for pen interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’08). ACM, New York, NY, USA, 1689--1698. org/10.1145/1357054.1357319
[2]
Tue Haste Andersen and Shumin Zhai. 2010. “Writing with music”: Exploring the use of auditory feedback in gesture interfaces. ACM Trans. Appl. Percept. 7, 3, Article 17 (June 2010), 24 pages. org/10.1145/1773965.1773968
[3]
Fraser Anderson and Walter F. Bischof. 2013. Learning and performance with gesture guides. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’13). ACM, New York, NY, USA, 1109--1118.
[4]
Michelle Annett and Walter F. Bischof. 2013. Your left hand can do it too!: Investigating intermanual, symmetric gesture transfer on touchscreens. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’13). ACM, New York, NY, USA, 1119--1128.
[5]
Lisa Anthony, Radu-Daniel Vatavu, and Jacob O. Wobbrock. 2013. Understanding the consistency of users’ pen and finger stroke gesture articulation. In Proceedings of the 2013 Graphics Interface Conference (GI’13). Canadian Information Processing Society, Toronto, ON, Canada, 87--94.
[6]
Caroline Appert and Olivier Bau. 2010. Scale detection for a priori gesture recognition. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’10). ACM, New York, NY, USA, 879--882.
[7]
Caroline Appert and Shumin Zhai. 2009. Using strokes as command shortcuts: Cognitive benefits and toolkit support. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’09). ACM, New York, NY, USA, 2289--2298.
[8]
Shiri Azenkot and Shumin Zhai. 2012. Touch behavior with different postures on soft smartphone keyboards. In Proceedings of the 14th International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI’12). ACM, New York, NY, USA, 251--260.
[9]
Olivier Bau and Wendy E. Mackay. 2008. OctoPocus: A dynamic guide for learning gesture-based command sets. In Proceedings of the ACM Symposium on User Interface Software and Technology (UIST’08). ACM, New York, NY, USA, 37--46.
[10]
Xiaojun Bi, Ciprian Chelba, Tom Ouyang, Kurt Partridge, and Shumin Zhai. 2012. Bimanual gesture keyboard. In Proceedings of the ACM Symposium on User Interface Software and Technology (UIST’12). ACM, New York, NY, USA, 137--146.
[11]
Rachel Blagojevic, Samuel Hsiao-Heng Chang, and Beryl Plimmer. 2010. The power of automatic feature selection: Rubine on steroids. In Proceedings of the Seventh Sketch-Based Interfaces and Modeling Symposium (SBIM’10). Eurographics Association, Aire-la-Ville, Switzerland, 79--86. http://dl.acm.org/citation. cfm?id=1923363.1923377.
[12]
Andrew Bragdon, Eugene Nelson, Yang Li, and Ken Hinckley. 2011. Experimental analysis of touch-screen gesture designs in mobile environments. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’11). ACM, New York, NY, USA, 403--412.
[13]
Andrew Bragdon, Robert Zeleznik, Brian Williamson, Timothy Miller, and Joseph J. LaViola, Jr. 2009. GestureBar: Improving the approachability of gesture-based interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’09). ACM, New York, NY, USA, 2269--2278.
[14]
Xiang Cao and Shumin Zhai. 2007. Modeling human performance of pen stroke gestures. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’07). ACM, New York, NY, USA, 1495--1504.
[15]
Mathias Frisch, Jens Heydekorn, and Raimund Dachselt. 2009. Investigating multi-touch and pen gestures for diagram editing on interactive surfaces. In Proceedings of the ACM International Conference on Interactive Tabletops and Surfaces (ITS’09). ACM, New York, NY, USA, 149--156.
[16]
Vittorio Fuccella, Poika Isokoski, and Benoit Martin. 2013. Gestures and widgets: Performance in text editing on multi-touch capable mobile devices. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’13). ACM, New York, NY, USA, 2785--2794.
[17]
Mayank Goel, Alex Jansen, Travis Mandel, Shwetak N. Patel, and Jacob O. Wobbrock. 2013. ContextType: Using hand posture information to improve mobile touch screen text entry. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’13). ACM, New York, NY, USA, 2795--2798.
[18]
Mayank Goel, Jacob Wobbrock, and Shwetak Patel. 2012. GripSense: Using built-in sensors to detect hand posture and pressure on commodity mobile phones. In Proceedings of the 25th Annual ACM Symposium on User Interface Software and Technology (UIST’12). ACM, New York, NY, USA, 545--554.
[19]
David Goldberg and Cate Richardson. 1993. Touch-typing with a stylus. In Proceedings of the INTERACT’93 and Conference on Human Factors in Computing Systems (CHI’93). ACM, New York, NY, USA, 80--87.
[20]
Ken Hinckley, Patrick Baudisch, Gonzalo Ramos, and Francois Guimbretiere. 2005. Design and analysis of delimiters for selection-action pen gesture phrases in scriboli. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’05). ACM, New York, NY, USA, 451--460.
[21]
Ken Hinckley, Michel Pahud, Hrvoje Benko, Pourang Irani, François Guimbretière, Marcel Gavriliu, Xiang ‘Anthony’ Chen, Fabrice Matulic, William Buxton, and Andrew Wilson. 2014. Sensing techniques for tablet+stylus interaction. In Proceedings of the 27th Annual ACM Symposium on User Interface Software and Technology (UIST’14). ACM, New York, NY, USA, 605--614.
[22]
Ken Hinckley, Koji Yatani, Michel Pahud, Nicole Coddington, Jenny Rodenhouse, Andy Wilson, Hrvoje Benko, and Bill Buxton. 2010. Pen + touch = new tools. In Proceedings of the ACM Symposium on User Interface Software and Technology (UIST’10). ACM, New York, NY, USA, 27--36.
[23]
HP 2012. TouchSmart tx2 Notebook PC - Maintenance and Service Guide. Retrieved from http://www.hp.com.
[24]
HTC 2009. HTC Touch2 T3333. Retrieved from http://www.htc.com.
[25]
HTC 2010. HTC HD Mini. Retrieved from http://www.htc.com.
[26]
Poika Isokoski. 2001. Model for unistroke writing time. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’01). ACM, New York, NY, USA, 357--364.
[27]
Shaun K. Kane, Jacob O. Wobbrock, and Richard E. Ladner. 2011. Usable gestures for blind people: Understanding preference and performance. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’11). ACM, New York, NY, USA, 413--422.
[28]
Amy K. Karlson and Benjamin B. Bederson. 2006. Understanding Single-Handed Mobile Device Interaction. Technical Report Tech report HCIL-2006-02. Computer Science Dept., University of Maryland, College Park, MD, USA.
[29]
Per-Ola Kristensson and Shumin Zhai. 2004. SHARK2: A large vocabulary shorthand writing system for pen-based computers. In Proceedings of the ACM Symposium on User Interface Software and Technology (UIST’04). ACM, New York, NY, USA, 43--52.
[30]
Gordon P. Kurtenbach, Abigail J. Sellen, and William A. S. Buxton. 1993. An empirical evaluation of some articulatory and cognitive aspects of marking menus. Hum.-Comput. Interact. 8, 1 (March 1993), 1--23.
[31]
Francesco Lacquaniti, Carlo Terzuolo, and Paolo Viviani. 1983. The law relating the kinematic and figural aspects of drawing movements. Acta Psychologica 54, 1--3, 115--130.
[32]
Yang Li. 2010. Gesture search: A tool for fast mobile data access. In Proceedings of the ACM Symposium on User Interface Software and Technology (UIST’10). ACM, New York, NY, USA, 87--96.
[33]
Allan Christian Long, Jr., James A. Landay, and Lawrence A. Rowe. 1999. Implications for a gesture design tool. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’99). ACM, New York, NY, USA, 40--47.
[34]
A. Chris Long, Jr., James A. Landay, Lawrence A. Rowe, and Joseph Michiels. 2000. Visual similarity of pen gestures. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’00). ACM, New York, NY, USA, 360--367.
[35]
Meredith Ringel Morris, Anqi Huang, Andreas Paepcke, and Terry Winograd. 2006. Cooperative gestures: Multi-user gestural interactions for co-located groupware. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’06). ACM, New York, NY, USA, 1201--1210.
[36]
Alexander Ng, Stephen A. Brewster, and John H. Williamson. 2014. Investigating the effects of encumbrance on one-and two-handed interactions with mobile devices. In Proceedings of the 32nd Annual ACM Conference on Human Factors in Computing Systems. ACM, New York, NY, USA, 1981--1990.
[37]
Ntrig 2013. DuoSense Digitizer Product Brief. Retrieved from http://www.n-trig.com.
[38]
Keith B. Perry and Juan Pablo Hourcade. 2008. Evaluating one handed thumb tapping on mobile touchscreen devices. In Proceedings of Graphics Interface (GI’08). Canadian Information Processing Society, Toronto, ON, Canada, 57--64.
[39]
Xiangshi Ren and Shinju Moriya. 2000. Improving selection performance on pen-based systems: A study of pen-based interaction for selection tasks. ACM Trans. Comput.-Hum. Interact. 7, 3 (September 2000), 384--416.
[40]
Xiangshi Ren and Xiaolei Zhou. 2009. The optimal size of handwriting character input boxes on PDAs. Int. J. Hum.-Comput. Interact. 25, 8, 762--784.
[41]
Dean Rubine. 1991. Specifying gestures by example. SIGGRAPH Comput. Graph. 25, 4 (July 1991), 329--337.
[42]
Samsung 2013. Samsung Galaxy Note 3 smart phone. Retrieved from http://www.samsung.com/.
[43]
R. A. Schmidt. 1988. Motor Control and Learning: A Behavioral Emphasis. Human Kinetics, Champaign, IL, USA. http://books.google.co.uk/books?id=KappAAAAMAAJ.
[44]
Huawei Tu and Xiangshi Ren. 2013. Optimal entry size of handwritten chinese characters in touch-based mobile phones. Int. J. Hum.-Comput. Interact. 29, 1, 1--12.
[45]
Huawei Tu, Xiangshi Ren, and Shumin Zhai. 2012. A comparative evaluation of finger and pen stroke gestures. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’12). ACM, New York, NY, USA, 1287--1296.
[46]
Radu-Daniel Vatavu. 2011. The effect of sampling rate on the performance of template-based gesture recognizers. In Proceedings of the 13th International Conference on Multimodal Interfaces (ICMI’11). ACM, New York, NY, USA, 271--278.
[47]
Radu-Daniel Vatavu, Géry Casiez, and Laurent Grisoni. 2013. Small, medium, or large?: Estimating the user-perceived scale of stroke gestures. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’13). ACM, New York, NY, USA, 277--280.
[48]
Jacob O. Wobbrock, Meredith Ringel Morris, and Andrew D. Wilson. 2009. User-defined gestures for surface computing. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’09). ACM, New York, NY, USA, 1083--1092.
[49]
Jacob O. Wobbrock, Andrew D. Wilson, and Yang Li. 2007. Gestures without libraries, toolkits or training: A $1 recognizer for user interface prototypes. In Proceedings of the ACM Symposium on User Interface Software and Technology (UIST’07). ACM, New York, NY, USA, 159--168.
[50]
C. E. Wright. 1990. Generalized motor programs: Reexamining claims of effector independence in writing. In Attention and Performance XIII: Motor Representation and Control. Erlbaum, Hillsdale, NJ, USA, 294--320.
[51]
Ying Yin, Tom Yu Ouyang, Kurt Partridge, and Shumin Zhai. 2013. Making touchscreen keyboards adaptive to keys, hand postures, and individuals: A hierarchical spatial backoff model approach. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’13). ACM, New York, NY, USA, 2775--2784.
[52]
S. Zhai, P. O. Kristensson, C. Appert, T. H. Anderson, and X. Cao. 2012. Foundational issues in touch-surface stroke gesture design—An integrative review. Foundations Trends Hum-Comput. Interact. 5, 2, 97--205.
[53]
Shumin Zhai and Per-Ola Kristensson. 2003. Shorthand writing on stylus keyboard. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’03). ACM, New York, NY, USA, 97--104.
[54]
Shumin Zhai and Per Ola Kristensson. 2012. The word-gesture keyboard: Reimagining keyboard interaction. Commun. ACM 55, 9 (September 2012), 91--101.
[55]
Shumin Zhai, Per Ola Kristensson, Pengjun Gong, Michael Greiner, Shilei Allen Peng, Liang Mico Liu, and Anthony Dunnigan. 2009. Shapewriter on the iphone: From the laboratory to the real world. In CHI’09 Extended Abstracts on Human Factors in Computing Systems (CHI EA’09). ACM, New York, NY, USA, 2667--2670.

Cited By

View all
  • (2023)Text Pin: Improving text selection with mode-augmented handles on touchscreen mobile devicesInternational Journal of Human-Computer Studies10.1016/j.ijhcs.2023.103028175(103028)Online publication date: Jul-2023
  • (2023)An exploration of pressure input with bare finger for Mobile interaction in stationary and Mobile situationsMultimedia Tools and Applications10.1007/s11042-023-14503-082:17(25711-25731)Online publication date: 14-Feb-2023
  • (2022)µV: An Articulation, Rotation, Scaling, and Translation Invariant (ARST) Multi-stroke Gesture RecognizerProceedings of the ACM on Human-Computer Interaction10.1145/35322006:EICS(1-25)Online publication date: 17-Jun-2022
  • Show More Cited By

Index Terms

  1. Differences and Similarities between Finger and Pen Stroke Gestures on Stationary and Mobile devices

    Recommendations

    Reviews

    Franz J Kurfess

    Before the advent of the iPhone, touch-based interaction between users and personal computational devices was dominated by pen-shaped styli, such as in handheld devices by Palm, or in laptops that could be converted to tablet-style devices. Although both pen-based and smartphone-style finger-based interactions rely on touch gestures as input methods, there are differences between the two: a finger has a larger surface area, and is used in a different manner from a stylus. Intuitively, a pen is better for more precise control, whereas fingers are more convenient since they are always at hand. There have been few thorough investigations of the two gesture methods, however. The authors of this publication conducted three experiments with different implements (pen, index finger, and thumb) and postures (sitting versus walking). To facilitate a systematic comparison, they assembled a set of gestures in three categories (simple, medium, complex) and used characteristics like articulation time (start to finish), gesture size ratio (target versus drawn gesture), aperture (between start and end point), corner shape distance (sharp versus broad angles), indicative angle difference ("tilt" between target and drawn gesture), and the proportional shape distance (sum of the distances between sample points of the target and drawn gestures). Overall, the results confirm the intuitive assessments: Stylus gestures perform better for more complex patterns and in smaller interaction areas, and the main advantage of thumb gestures is for one-handed device use. Styli often are available as an alternative input method for smartphones. Interaction designers can benefit from this publication in at least two ways: It is an excellent example of experiments that examine intuitive assumptions, and it contains some concrete recommendations for the appropriate use of stylus- and finger-based gesture interaction. Online Computing Reviews Service

    Access critical reviews of Computing literature here

    Become a reviewer for Computing Reviews.

    Comments

    Please enable JavaScript to view thecomments powered by Disqus.

    Information & Contributors

    Information

    Published In

    cover image ACM Transactions on Computer-Human Interaction
    ACM Transactions on Computer-Human Interaction  Volume 22, Issue 5
    October 2015
    217 pages
    ISSN:1073-0516
    EISSN:1557-7325
    DOI:10.1145/2814459
    Issue’s Table of Contents
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 10 August 2015
    Accepted: 01 June 2015
    Revised: 01 May 2015
    Received: 01 April 2014
    Published in TOCHI Volume 22, Issue 5

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. Pen gestures
    2. finger gestures
    3. gesture design
    4. mobile devices

    Qualifiers

    • Research-article
    • Research
    • Refereed

    Funding Sources

    • Japan and NSFC of China
    • Grant-in-Aid for Scientific Research

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)26
    • Downloads (Last 6 weeks)6
    Reflects downloads up to 13 Nov 2024

    Other Metrics

    Citations

    Cited By

    View all
    • (2023)Text Pin: Improving text selection with mode-augmented handles on touchscreen mobile devicesInternational Journal of Human-Computer Studies10.1016/j.ijhcs.2023.103028175(103028)Online publication date: Jul-2023
    • (2023)An exploration of pressure input with bare finger for Mobile interaction in stationary and Mobile situationsMultimedia Tools and Applications10.1007/s11042-023-14503-082:17(25711-25731)Online publication date: 14-Feb-2023
    • (2022)µV: An Articulation, Rotation, Scaling, and Translation Invariant (ARST) Multi-stroke Gesture RecognizerProceedings of the ACM on Human-Computer Interaction10.1145/35322006:EICS(1-25)Online publication date: 17-Jun-2022
    • (2022)Iteratively Designing Gesture Vocabularies: A Survey and Analysis of Best Practices in the HCI LiteratureACM Transactions on Computer-Human Interaction10.1145/350353729:4(1-54)Online publication date: 5-May-2022
    • (2022)Effect of colour sensing and drawing pen on children’s colour perception and feature detection of objects when drawingBehaviour & Information Technology10.1080/0144929X.2022.207176442:9(1311-1323)Online publication date: 6-May-2022
    • (2021)Understanding Players’ Interaction Patterns with Mobile Game App UI via VisualizationsProceedings of the Ninth International Symposium of Chinese CHI10.1145/3490355.3490357(9-21)Online publication date: 16-Oct-2021
    • (2021)Distractor Effects on Crossing-Based InteractionProceedings of the 2021 CHI Conference on Human Factors in Computing Systems10.1145/3411764.3445340(1-13)Online publication date: 6-May-2021
    • (2020)The graphic routines of closed and open shapesProceedings of the 2020 ACM Interaction Design and Children Conference: Extended Abstracts10.1145/3397617.3397844(229-234)Online publication date: 21-Jun-2020
    • (2020)The Effect of Context on Small Screen and Wearable Device Users’ Performance - A Systematic ReviewACM Computing Surveys10.1145/338637053:3(1-44)Online publication date: 28-May-2020
    • (2020)Classification of children’s drawing strategies on touch-screen of seriation objects using a novel deep learning hybrid modelAlexandria Engineering Journal10.1016/j.aej.2020.06.019Online publication date: Jun-2020
    • Show More Cited By

    View Options

    Get Access

    Login options

    Full Access

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media