research-article

Shortcut Gestures for Mobile Text Editing on Fully Touch Sensitive Smartphones

Authors:

Maximilian Weiß,

Jonas Vogelsang,

Henrike Weingärtner,

Niels HenzeAuthors Info & Claims

ACM Transactions on Computer-Human Interaction (TOCHI), Volume 27, Issue 5

Article No.: 33, Pages 1 - 38

https://doi.org/10.1145/3396233

Published: 17 August 2020 Publication History

Abstract

While advances in mobile text entry enable smartphone users to type almost as fast as on hardware keyboards, text-heavy activities are still not widely adopted. One reason is the lack of shortcut mechanisms. In this article, we determine shortcuts for text-heavy activities, elicit shortcut gestures, implement them for a fully touch-sensitive smartphone, and conduct an evaluation with potential users. We found that experts perform around 800 keyboard shortcuts per day, which are not available on smartphones. Interviews revealed the lack of shortcuts as a major limitation that prevents mobile text editing. Therefore, we elicited gestures for the 22 most important shortcuts for smartphones that are touch-sensitive on the whole device surface. We implemented the gestures for a fully touch-sensitive smartphone using deep learning and evaluated them in realistic scenarios to gather feedback. We show that the developed prototype is perceived as intuitive and faster than recent commercial approaches.

References

[1]

Jason Alexander, Andy Cockburn, and Richard Lobb. 2008. AppMonitor: A tool for recording user actions in unmodified Windows applications. Behavior Research Methods 40, 2 (2008), 413--421.

[2]

Lisa Anthony and Jacob O. Wobbrock. 2012. $N-protractor: A fast and accurate multistroke recognizer. In Proceedings of Graphics Interface 2012 (GI’12). Canadian Information Processing Society, 117--120. Retrieved from http://dl.acm.org/citation.cfm?id=2305276.2305296.

Digital Library

[3]

Sajid Anwar, Kyuyeon Hwang, and Wonyong Sung. 2017. Structured pruning of deep convolutional neural networks. Journal on Emerging Technologies in Computing Systems 13, 3 (Feb. 2017), 18 pages.

[4]

Patrick Bader, Valentin Schwind, Niels Henze, Stefan Schneegass, Nora Broy, and Albrecht Schmidt. 2014. Design and evaluation of a layered handheld 3D display with touch-sensitive front and back. In Proceedings of the 8th Nordic Conference on Human-Computer Interaction: Fun, Fast, Foundational (NordiCHI’14). ACM, New York, NY, 315--318.

Digital Library

[5]

Gilles Bailly, Thomas Pietrzak, Jonathan Deber, and Daniel J. Wigdor. 2013. MéTamorphe: Augmenting hotkey usage with actuated keys. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’13). ACM, New York, NY, 563--572.

[6]

Olivier Bau and Wendy E. Mackay. 2008. OctoPocus: A dynamic guide for learning gesture-based command sets. In Proceedings of the 21st Annual ACM Symposium on User Interface Software and Technology (UIST’08). ACM, New York, NY, 37--46.

[7]

Patrick Baudisch and Gerry Chu. 2009. Back-of-device interaction allows creating very small touch devices. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’09). ACM, New York, NY, 1923--1932.

Digital Library

[8]

Xiaojun Bi and Shumin Zhai. 2016. IJQwerty: What difference does one key change make? Gesture typing keyboard optimization bounded by one key position change from qwerty. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI’16). ACM, New York, NY, 49--58.

Digital Library

[9]

John Brooke. 1996. SUS-A quick and dirty usability scale. Usability Evaluation in Industry 189, 194 (1996), 4--7.

[10]

Wook Chang, Kee Eung Kim, Hyunjeong Lee, Joon Kee Cho, Byung Seok Soh, Jung Hyun Shim, Gyunghye Yang, Sung-jung Cho, and Joonah Park. 2006. Recognition of grip-patterns by using capacitive touch sensors. In Proceedings of the 2006 IEEE International Symposium on Industrial Electronics. Vol. 4. 2936--2941.

[11]

Lung-Pan Cheng, Fang-I. Hsiao, Yen-Ting Liu, and Mike Y. Chen. 2012. iRotate grasp: Automatic screen rotation based on grasp of mobile devices. In Proceedings of the 25th Annual ACM Symposium on User Interface Software and Technology (UIST Adjunct Proceedings’12). ACM, New York, NY, 15--16.

[12]

Lung-Pan Cheng, Hsiang-Sheng Liang, Che-Yang Wu, and Mike Y. Chen. 2013. iGrasp: Grasp-based adaptive keyboard for mobile devices. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’13). ACM, New York, NY, 3037--3046.

[13]

CooTek. 2019. TouchPal. Retrieved April 23, 2020 from http://www.touchpal.com/.

[14]

Gennaro Costagliola, Vittorio Fuccella, and Michele Di Capua. 2011. Text entry with KeyScretch. In Proceedings of the 16th International Conference on Intelligent User Interfaces (IUI’11). ACM, New York, NY, 277--286.

Digital Library

[15]

Wenzhe Cui, Jingjie Zheng, Blaine Lewis, Daniel Vogel, and Xiaojun Bi. 2019. HotStrokes: Word-gesture shortcuts on a trackpad. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI’19). ACM, New York, NY, Article 165, 13 pages.

Digital Library

[16]

Alexander De Luca, Emanuel von Zezschwitz, Ngo Dieu Huong Nguyen, Max-Emanuel Maurer, Elisa Rubegni, Marcello Paolo Scipioni, and Marc Langheinrich. 2013. Back-of-device authentication on smartphones. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’13). ACM, New York, NY, 2389--2398.

Digital Library

[17]

Tilman Dingler, Rufat Rzayev, Alireza Sahami Shirazi, and Niels Henze. 2018. Designing consistent gestures across device types: Eliciting RSVP controls for phone, watch, and glasses. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI’18). ACM, New York, NY, Article 419, 12 pages.

Digital Library

[18]

Jeff Donahue, Lisa Anne Hendricks, Marcus Rohrbach, Subhashini Venugopalan, Sergio Guadarrama, Kate Saenko, and Trevor Darrell. 2017. Long-term recurrent convolutional networks for visual recognition and description. IEEE Transactions on Pattern Analysis and Machine Intelligence 39, 4 (April 2017), 677--691.

Digital Library

[19]

Yin Fan, Xiangju Lu, Dian Li, and Yuanliu Liu. 2016. Video-based emotion recognition using CNN-RNN and C3D hybrid networks. In Proceedings of the 18th ACM International Conference on Multimodal Interaction (ICMI’16). ACM, New York, NY, 445--450.

Digital Library

[20]

Anna Maria Feit, Daryl Weir, and Antti Oulasvirta. 2016. How we type: Movement strategies and performance in everyday typing. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI’16). ACM, New York, NY, 4262--4273.

Digital Library

[21]

Dustin Freeman, Hrvoje Benko, Meredith Ringel Morris, and Daniel Wigdor. 2009. ShadowGuides: Visualizations for in-situ learning of multi-touch and whole-hand gestures. In Proceedings of the ACM International Conference on Interactive Tabletops and Surfaces (ITS’09). ACM, New York, NY, 165--172.

Digital Library

[22]

Vittorio Fuccella, Poika Isokoski, and Benoit Martin. 2013. Gestures and widgets: Performance in text editing on multi-touch capable mobile devices. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’13). ACM, New York, NY, 2785--2794.

Digital Library

[23]

Hyunjae Gil, DoYoung Lee, Seunggyu Im, and Ian Oakley. 2017. TriTap: Identifying finger touches on smartwatches. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI’17). ACM, New York, NY, 3879--3890.

Digital Library

[24]

Google Commerce Ltd. 2019. Google Docs for Android. Retrieved April 23, 2020 from https://play.google.com/store/apps/details?id=com.google.android.apps.docs.editors.docs.

[25]

Tovi Grossman, Pierre Dragicevic, and Ravin Balakrishnan. 2007. Strategies for accelerating on-line learning of hotkeys. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’07). ACM, New York, NY, 1591--1600.

Digital Library

[26]

Jefferson Y. Han. 2005. Low-cost multi-touch sensing through frustrated total internal reflection. In Proceedings of the 18th Annual ACM Symposium on User Interface Software and Technology (UIST’05). ACM, New York, NY, 115--118.

Digital Library

[27]

Song Han, Huizi Mao, and William J. Dally. 2015. Deep compression: Compressing deep neural network with pruning, trained quantization and huffman coding. CoRR abs/1510.00149 (2015). arxiv:1510.00149 http://arxiv.org/abs/1510.00149.

[28]

Gunnar Harboe and Elaine M. Huang. 2015. Real-world affinity diagramming practices: Bridging the paper-digital gap. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI’15). ACM, New York, NY, 95--104.

[29]

Niels Henze, Enrico Rukzio, and Susanne Boll. 2011. 100,000,000 taps: Analysis and improvement of touch performance in the large. In Proceedings of the 13th International Conference on Human Computer Interaction with Mobile Devices and Services (MobileHCI’11). ACM, New York, NY, 133--142.

Digital Library

[30]

Niels Henze, Enrico Rukzio, and Susanne Boll. 2012. Observational and experimental investigation of typing behaviour using virtual keyboards for mobile devices. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’12). ACM, New York, NY, 2659--2668.

Digital Library

[31]

David Holman, Andreas Hollatz, Amartya Banerjee, and Roel Vertegaal. 2013. Unifone: Designing for auxiliary finger input in one-handed mobile interactions. In Proceedings of the 7th International Conference on Tangible, Embedded and Embodied Interaction (TEI’13). ACM, New York, NY, 177--184.

Digital Library

[32]

Chih-Wei Hsu, Chih-Chung Chang, Chih-Jen Lin, et al. 2003. A practical guide to support vector classification. (2003).

[33]

Sergey Ioffe and Christian Szegedy. 2015. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In Proceedings of the 32nd International Conference on Machine Learning (PCML’15). 448--456. http://proceedings.mlr.press/v37/ioffe15.pdf.

[34]

Poika Isokoski. 2004. Performance of menu-augmented soft keyboards. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’04). ACM, New York, NY, 423--430.

Digital Library

[35]

John Karat, James E. McDonald, and Matt Anderson. 1986. A comparison of menu selection techniques: Touch panel, mouse and keyboard. International Journal of Man-Machine Studies 25, 1 (1986), 73--88.

Digital Library

[36]

Kee-Eung Kim, Wook Chang, Sung-Jung Cho, Junghyun Shim, Hyunjeong Lee, Joonah Park, Youngbeom Lee, and Sangryoung Kim. 2006. Hand grip pattern recognition for mobile user interfaces. In Proceedings of the 18th Conference on Innovative Applications of Artificial Intelligence.

Digital Library

[37]

Sunjun Kim, Jihyun Yu, and Geehyuk Lee. 2012. Interaction techniques for unreachable objects on the touchscreen. In Proceedings of the 24th Australian Computer-Human Interaction Conference (OzCHI’12). ACM, New York, NY, 295--298.

Digital Library

[38]

Anne Köpsel and Nikola Bubalo. 2015. Benefiting from legacy bias. Interactions 22, 5 (Aug. 2015), 44--47.

Digital Library

[39]

Per Ola Kristensson and Keith Vertanen. 2012. Performance comparisons of phrase sets and presentation styles for text entry evaluations. In Proceedings of the 2012 ACM International Conference on Intelligent User Interfaces (IUI’12). ACM, New York, NY, 29--32.

Digital Library

[40]

Per Ola Kristensson and Shumin Zhai. 2004. SHARK2: A large vocabulary shorthand writing system for pen-based computers. In Proceedings of the 17th Annual ACM Symposium on User Interface Software and Technology (UIST’04). ACM, New York, NY, 43--52.

Digital Library

[41]

Per Ola Kristensson and Shumin Zhai. 2007. Command strokes with and without preview: Using pen gestures on keyboard for command selection. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’07). ACM, New York, NY, 1137--1146.

Digital Library

[42]

Abinaya Kumar, Aishwarya Radjesh, Sven Mayer, and Huy Viet Le. 2019. Improving the input accuracy of touchscreens using deep learning. In Proceedings of the Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems (CHI EA’19). ACM, New York, NY, Article Paper LBW1514, 6 pages.

Digital Library

[43]

David M. Lane, H. Albert Napier, S. Camille Peres, and Aniko Sandor. 2005. Hidden costs of graphical user interfaces: Failure to make the transition from menus and icon toolbars to keyboard shortcuts. International Journal of Human-Computer Interaction 18, 2 (2005), 133--144.

[44]

Huy Viet Le, Patrick Bader, Thomas Kosch, and Niels Henze. 2016. Investigating screen shifting techniques to improve one-handed smartphone usage. In Proceedings of the 9th Nordic Conference on Human-Computer Interaction (NordiCHI’16). ACM, New York, NY, Article 27, 10 pages.

Digital Library

[45]

Huy Viet Le, Thomas Kosch, Patrick Bader, Sven Mayer, and Niels Henze. 2018. PalmTouch: Using the palm as an additional input modality on commodity smartphones. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI’18). ACM, New York, NY, 10.

Digital Library

[46]

Huy Viet Le, Sven Mayer, Patrick Bader, and Niels Henze. 2017. A smartphone prototype for touch interaction on the whole device surface. In Proceedings of the 19th International Conference on Human-Computer Interaction with Mobile Devices and Services Adjunct (MobileHCI’17). ACM, New York, NY.

Digital Library

[47]

Huy Viet Le, Sven Mayer, Patrick Bader, and Niels Henze. 2018. Fingers’ range and comfortable area for one-handed smartphone interaction beyond the touchscreen. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI’18). ACM, New York, NY, Article Paper 31, 12 pages.

Digital Library

[48]

Huy Viet Le, Sven Mayer, and Niels Henze. 2018. InfiniTouch: Finger-aware interaction on fully touch sensitive smartphones. In Proceedings of the 31th Annual ACM Symposium on User Interface Software and Technology (UIST’18). ACM, New York, NY, 14.

Digital Library

[49]

Huy Viet Le, Sven Mayer, and Niels Henze. 2019. Investigating the feasibility of finger identification on capacitive touchscreens using deep learning. In Proceedings of the 24th International Conference on Intelligent User Interfaces (IUI’19). ACM, New York, NY, 637--649.

Digital Library

[50]

Huy Viet Le, Sven Mayer, Benedict Steuerlein, and Niels Henze. 2019. Investigating unintended inputs for one-handed touch interaction beyond the touchscreen. In Proceedings of the 21st International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI’19). ACM, New York, NY, Article Article 34, 14 pages.

Digital Library

[51]

Huy Viet Le, Sven Mayer, Katrin Wolf, and Niels Henze. 2016. Finger placement and hand grasp during smartphone interaction. In Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA’16). ACM, New York, NY, 2576--2584.

Digital Library

[52]

Yang Li. 2010. Gesture search: A tool for fast mobile data access. In Proceedings of the 23rd Annual ACM Symposium on User Interface Software and Technology (UIST’10). ACM, New York, NY, 87--96.

Digital Library

[53]

Sylvain Malacria, Gilles Bailly, Joel Harrison, Andy Cockburn, and Carl Gutwin. 2013. Promoting hotkey use through rehearsal with ExposeHK. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’13). ACM, New York, NY, 573--582.

Digital Library

[54]

Joseph Malloch, Carla F. Griggio, Joanna McGrenere, and Wendy E. Mackay. 2017. Fieldward and pathward: Dynamic guides for defining your own gestures. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI’17). ACM, New York, NY, 4266--4277.

[55]

Jennifer Mankoff and Gregory D. Abowd. 1998. Cirrin: A word-level unistroke keyboard for pen input. In Proceedings of the 11th Annual ACM Symposium on User Interface Software and Technology (UIST’98). ACM, New York, NY, 213--214.

[56]

Sven Mayer, Huy Viet Le, Markus Funk, and Niels Henze. 2019. Finding the sweet spot: Analyzing unrestricted touchscreen interaction in-the-wild. In Proceedings of the 2019 ACM International Conference on Interactive Surfaces and Spaces (ISS’19). ACM, New York, NY, 171--179.

Digital Library

[57]

Sven Mayer, Huy Viet Le, and Niels Henze. 2017. Estimating the finger orientation on capacitive touchscreens using convolutional neural networks. In Proceedings of the 2017 ACM International Conference on Interactive Surfaces and Spaces (ISS’17). ACM, New York, NY, 220--229.

Digital Library

[58]

Hugh McLoone, Ken Hinckley, and Edward Cutrell. 2003. Bimanual interaction on the Microsoft Office keyboard. In Proceedings of the IFIP TC13 International Conference on Human-Computer Interaction. 49--56.

[59]

Microsoft Corporation. 2019. Google Play: Microsoft Excel. Retrieved April 23, 2020 from https://play.google.com/store/apps/details?id=com.microsoft.office.excel.

[60]

Microsoft Corporation. 2019. Google Play: Microsoft PowerPoint. Retrieved April 23, 2020 from https://play.google.com/store/apps/details?id=com.microsoft.office.powerpoint.

[61]

Microsoft Corporation. 2019. Google Play: Microsoft Word. Retrieved April 23, 2020 from https://play.google.com/store/apps/details?id=com.microsoft.office.word.

[62]

Takashi Miyaki and Jun Rekimoto. 2009. GraspZoom: Zooming and scrolling control model for single-handed mobile interaction. In Proceedings of the 11th International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI’09). ACM, New York, NY, Article 11, 4 pages.

Digital Library

[63]

Mohammad Faizuddin Mohd Noor, Andrew Ramsay, Stephen Hughes, Simon Rogers, John Williamson, and Roderick Murray-Smith. 2014. 28 frames later: Predicting screen touches from back-of-device grip changes. In Proceedings of the 32nd Annual ACM Conference on Human Factors in Computing Systems (CHI’14). ACM, New York, NY, 2005--2008.

Digital Library

[64]

Meredith Ringel Morris, Andreea Danielescu, Steven Drucker, Danyel Fisher, Bongshin Lee, m. c. schraefel, and Jacob O. Wobbrock. 2014. Reducing legacy bias in gesture elicitation studies. Interactions 21, 3 (May 2014), 40--45.

Digital Library

[65]

Meredith Ringel Morris, Jacob O. Wobbrock, and Andrew D. Wilson. 2010. Understanding users’ preferences for surface gestures. In Proceedings of Graphics Interface 2010 (GI’10). Canadian Information Processing Society, 261--268. Retrieved from http://dl.acm.org/citation.cfm?id=1839214.1839260.

[66]

Miguel A. Nacenta, Yemliha Kamber, Yizhou Qiang, and Per Ola Kristensson. 2013. Memorability of pre-designed and user-defined gesture sets. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’13). ACM, New York, NY, 1099--1108.

Digital Library

[67]

Jakob Nielsen. 1992. Finding usability problems through heuristic evaluation. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’92). ACM, New York, NY, 373--380.

Digital Library

[68]

Michael Nielsen, Moritz Störring, Thomas B. Moeslund, and Erik Granum. 2004. A procedure for developing intuitive and ergonomic gesture interfaces for HCI. In Gesture-Based Communication in Human-Computer Interaction. Springer, Berlin, 409--420.

[69]

Nuance Communications. 2019. Swype Keyboard. Retrieved April 23, 2020 from http://www.swype.com/.

[70]

Daniel L. Odell, Richard C. Davis, Andrew Smith, and Paul K. Wright. 2004. Toolglasses, marking menus, and hotkeys: A comparison of one and two-handed command selection techniques. In Proceedings of Graphics Interface 2004 (GI’04). Canadian Human-Computer Communications Society, 17--24. Retrieved from http://dl.acm.org/citation.cfm?id=1006058.1006061.

Digital Library

[71]

Antti Oulasvirta, Anna Reichel, Wenbin Li, Yan Zhang, Myroslav Bachynskyi, Keith Vertanen, and Per Ola Kristensson. 2013. Improving two-thumb text entry on touchscreen devices. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’13). ACM, New York, NY, 2765--2774.

Digital Library

[72]

Benjamin Poppinga, Alireza Sahami Shirazi, Niels Henze, Wilko Heuten, and Susanne Boll. 2014. Understanding shortcut gestures on mobile touch devices. In Proceedings of the 16th International Conference on Human-computer Interaction with Mobile Devices 8 Services (MobileHCI’14). ACM, New York, NY, 173--182.

Digital Library

[73]

A Poston. 2000. Human Engineering Design Data Digest. Washington, DC: Department of Defense Human Factors Engineering Technical Advisory Group.

[74]

Shyam Reyal, Shumin Zhai, and Per Ola Kristensson. 2015. Performance and user experience of touchscreen and gesture keyboards in a lab setting and in the wild. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI’15). ACM, New York, NY, 679--688.

Digital Library

[75]

Jochen Rick. 2010. Performance optimizations of virtual keyboards for stroke-based text entry on a touch-based tabletop. In Proceedings of the 23rd Annual ACM Symposium on User Interface Software and Technology (UIST’10). ACM, New York, NY, 77--86.

Digital Library

[76]

Anne Roudaut, Stéphane Huot, and Eric Lecolinet. 2008. TapTap and MagStick: Improving one-handed target acquisition on small touch-screens. In Proceedings of the Working Conference on Advanced Visual Interfaces (AVI’08). ACM, New York, NY, 146--153.

Digital Library

[77]

Jaime Ruiz, Yang Li, and Edward Lank. 2011. User-defined motion gestures for mobile interaction. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’11). ACM, New York, NY, 197--206.

Digital Library

[78]

Jaime Ruiz and Daniel Vogel. 2015. Soft-constraints to reduce legacy and performance bias to elicit whole-body gestures with low arm fatigue. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI’15). ACM, New York, NY, 3347--3350.

Digital Library

[79]

Joey Scarr, Andy Cockburn, Carl Gutwin, and Andrea Bunt. 2012. Improving command selection with CommandMaps. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’12). ACM, New York, NY, 257--266.

Digital Library

[80]

Joey Scarr, Andy Cockburn, Carl Gutwin, Andrea Bunt, and Jared E. Cechanowicz. 2014. The usability of CommandMaps in realistic tasks. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’14). ACM, New York, NY, 2241--2250.

[81]

Joey Scarr, Andy Cockburn, Carl Gutwin, and Philip Quinn. 2011. Dips and ceilings: Understanding and supporting transitions to expertise in user interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’11). ACM, New York, NY, 2741--2750.

Digital Library

[82]

Jean-Baptiste Scheibel, Cyril Pierson, Benoît Martin, Nathan Godard, Vittorio Fuccella, and Poika Isokoski. 2013. Virtual stick in caret positioning on touch screens. In Proceedings of the 25th Conference on L’Interaction Homme-Machine (IHM’13). ACM, New York, NY, Article 107, 8 pages.

Digital Library

[83]

Ben Schneiderman. 1986. Eight golden rules of interface design. In Disponible en.

[84]

Robin Schweigert, Jan Leusmann, Simon Hagenmayer, Maximilian Weiß, Huy Viet Le, Sven Mayer, and Andreas Bulling. 2019. KnuckleTouch: Enabling knuckle gestures on capacitive touchscreens using deep learning. In Proceedings of Mensch Und Computer 2019 (MuC’19). ACM, New York, NY, 387--397.

Digital Library

[85]

Erh-li Early Shen, Sung-sheng Daniel Tsai, Hao-hua Chu, Yung-jen Jane Hsu, and Chi-wen Euro Chen. 2009. Double-side multi-touch input for mobile devices. In Proceedings of the CHI’09 Extended Abstracts on Human Factors in Computing Systems (CHI EA’09). ACM, New York, NY, 4339--4344.

Digital Library

[86]

Shaikh Shawon Arefin Shimon, Sarah Morrison-Smith, Noah John, Ghazal Fahimi, and Jaime Ruiz. 2015. Exploring user-defined back-of-device gestures for mobile devices. In Proceedings of the 17th International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI’15). ACM, New York, NY, 227--232.

Digital Library

[87]

Katie A. Siek, Yvonne Rogers, and Kay H. Connelly. 2005. Fat finger worries: How older and younger users physically interact with PDAs. In Proceedings of the 2005 IFIP TC13 International Conference on Human-Computer Interaction (INTERACT’05). Springer, Berlin, 267--280.

[88]

Smith Aaron. 2015. A “Week in the Life” Analysis of Smartphone Users. Retrieved April 23, 2020 from http://www.pewinternet.org/2015/04/01/chapter-three-a-week-in-the-life-analysis-of-smartphone-users/.

[89]

Kenji Suzuki, Kazumasa Okabe, Ryuuki Sakamoto, and Daisuke Sakamoto. 2016. Fix and slide: Caret navigation with movable background. In Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI’16). ACM, New York, NY, 478--482.

Digital Library

[90]

Tijmen Tieleman and Geoffrey Hinton. 2012. Lecture 6.5-rmsprop: Divide the gradient by a running average of its recent magnitude. COURSERA: Neural Networks for Machine Learning 4, 2 (2012), 26--31.

[91]

TouchType Ltd. 2019. SwiftKey. Retrieved March 23, 2020 from http://swiftkey.com.

[92]

Radu-Daniel Vatavu. 2012. User-defined gestures for free-hand TV control. In Proceedings of the 10th European Conference on Interactive TV and Video (EuroITV’12). ACM, New York, NY, 45--48.

Digital Library

[93]

Radu-Daniel Vatavu. 2019. The dissimilarity-consensus approach to agreement analysis in gesture elicitation studies. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI’19). ACM, New York, NY, Article 224, 13 pages.

Digital Library

[94]

Radu-Daniel Vatavu and Jacob O. Wobbrock. 2015. Formalizing agreement analysis for elicitation studies: New measures, significance test, and toolkit. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI’15). ACM, New York, NY, 1325--1334.

[95]

Radu-Daniel Vatavu and Jacob O. Wobbrock. 2016. Between-subjects elicitation studies: Formalization and tool support. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI’16). ACM, New York, NY, 3390--3402.

[96]

Keith Vertanen, Haythem Memmi, Justin Emge, Shyam Reyal, and Per Ola Kristensson. 2015. VelociTap: Investigating fast mobile text entry using sentence-based decoding of touchscreen keyboard input. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI’15). ACM, New York, NY, 659--668.

Digital Library

[97]

Daniel Vogel and Patrick Baudisch. 2007. Shift: A technique for operating pen-based interfaces using touch. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’07). ACM, New York, NY, 657--666.

Digital Library

[98]

Panagiotis Vogiatzidakis and Panayiotis Koutsabasis. 2018. Gesture elicitation studies for mid-air interaction: A review. Multimodal Technologies and Interaction 2, 4 (2018), 65.

Digital Library

[99]

Daryl Weir, Simon Rogers, Roderick Murray-Smith, and Markus Löchtefeld. 2012. A user-specific machine learning approach for improving touch accuracy on mobile devices. In Proceedings of the 25th Annual ACM Symposium on User Interface Software and Technology (UIST’12). ACM, New York, NY, 465--476.

Digital Library

[100]

Daniel Wigdor, Clifton Forlines, Patrick Baudisch, John Barnwell, and Chia Shen. 2007. Lucid touch: A see-through mobile device. In Proceedings of the 20th Annual ACM Symposium on User Interface Software and Technology (UIST’07). ACM, New York, NY, 269--278.

Digital Library

[101]

Graham Wilson, Stephen Brewster, and Martin Halvey. 2013. Towards utilising one-handed multi-digit pressure input. In Proceedings of the CHI’13 Extended Abstracts on Human Factors in Computing Systems (CHI EA’13). ACM, New York, NY, 1317--1322.

Digital Library

[102]

Jacob O. Wobbrock, Htet Htet Aung, Brandon Rothrock, and Brad A. Myers. 2005. Maximizing the guessability of symbolic input. In CHI’05 Extended Abstracts on Human Factors in Computing Systems (CHI EA’05). ACM, New York, NY, 1869--1872.

[103]

Jacob O. Wobbrock, Leah Findlater, Darren Gergle, and James J. Higgins. 2011. The aligned rank transform for nonparametric factorial analyses using only anova procedures. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’11). ACM, New York, NY, 143--146.

Digital Library

[104]

Jacob O. Wobbrock, Meredith Ringel Morris, and Andrew D. Wilson. 2009. User-defined gestures for surface computing. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’09). ACM, New York, NY, 1083--1092.

Digital Library

[105]

Jacob O. Wobbrock, Andrew D. Wilson, and Yang Li. 2007. Gestures without libraries, toolkits or training: A $1 recognizer for user interface prototypes. In Proceedings of the 20th Annual ACM Symposium on User Interface Software and Technology (UIST’07). ACM, New York, NY, 159--168.

Digital Library

[106]

Katrin Wolf, Christina Dicke, and Raphael Grasset. 2011. Touching the void: Gestures for auditory interfaces. In Proceedings of the 5th International Conference on Tangible, Embedded, and Embodied Interaction (TEI’11). ACM, New York, NY, 305--308.

Digital Library

[107]

Katrin Wolf, Marilyn McGee-Lennon, and Stephen Brewster. 2012. A study of on-device gestures. In Proceedings of the 14th International Conference on Human-computer Interaction with Mobile Devices and Services Companion (MobileHCI’12). ACM, New York, NY, 11--16.

Digital Library

[108]

Katrin Wolf, Robert Schleicher, and Michael Rohs. 2014. Ergonomic characteristics of gestures for front- and back-of-tablets interaction with grasping hands. In Proceedings of the 16th International Conference on Human-computer Interaction with Mobile Devices 8 Services (MobileHCI’14). ACM, New York, NY, 453--458.

Digital Library

[109]

Shi Xingjian, Zhourong Chen, Hao Wang, Dit-Yan Yeung, Wai-Kin Wong, and Wang-chun Woo. 2015. Convolutional LSTM network: A machine learning approach for precipitation nowcasting. In Advances in Neural Information Processing Systems (NIPS). 802--810. Retrieved from http://papers.nips.cc/paper/5955-convolutional-lstm-network-a-machine-learning-approach-for-precipitation-nowcasting.pdf.

[110]

Shumin Zhai and Per Ola Kristensson. 2003. Shorthand writing on stylus keyboard. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’03). ACM, New York, NY, 97--104.

Digital Library

[111]

Shumin Zhai, Per Ola Kristensson, Pengjun Gong, Michael Greiner, Shilei Allen Peng, Liang Mico Liu, and Anthony Dunnigan. 2009. Shapewriter on the Iphone: From the laboratory to the real world. In Proceedings of the CHI’09 Extended Abstracts on Human Factors in Computing Systems (CHI EA’09). ACM, New York, NY, 2667--2670.

Digital Library

[112]

Yu Zhang, William Chan, and Navdeep Jaitly. 2017. Very deep convolutional networks for end-to-end speech recognition. In Proceedings of the 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP’17). 4845--4849.

[113]

Jingjie Zheng, Blaine Lewis, Jeff Avery, and Daniel Vogel. 2018. FingerArc and FingerChord: Supporting novice to expert transitions with guided finger-aware shortcuts. In Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology (UIST’18). ACM, New York, NY, 347--363.

Digital Library

[114]

Jingjie Zheng and Daniel Vogel. 2016. Finger-aware shortcuts. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI’16). ACM, New York, NY, 4274--4285.

Digital Library

Cited By

Li CYu JHe KFeng JZhou J(2024)SwivelTouch: Boosting Touchscreen Input with 3D Finger Rotation GestureProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36595848:2(1-30)Online publication date: 15-May-2024
https://dl.acm.org/doi/10.1145/3659584
Yeh YIrudayaraj AVogel D(2024)Single-handed Folding Interactions with a Modified Clamshell Flip PhoneProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642554(1-14)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642554
Song JShi RLi YGao BLiang H(2024)Exploring Controller-based Techniques for Precise and Rapid Text Selection in Virtual Reality2024 IEEE Conference Virtual Reality and 3D User Interfaces (VR)10.1109/VR58804.2024.00047(244-253)Online publication date: 16-Mar-2024
https://doi.org/10.1109/VR58804.2024.00047
Show More Cited By

Index Terms

Shortcut Gestures for Mobile Text Editing on Fully Touch Sensitive Smartphones
1. Human-centered computing
  1. Human computer interaction (HCI)
  2. Ubiquitous and mobile computing
    1. Ubiquitous and mobile devices
      1. Smartphones

Recommendations

Gestures and widgets: performance in text editing on multi-touch capable mobile devices
CHI '13: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

We describe the design and evaluation of a gestural text editing technique for touchscreen devices. The gestures are drawn on top of the soft keyboard and interpreted as commands for moving the caret, performing selections, and controlling the ...
A Shortcut for Caret Positioning on Touch-Screen Phones
MobileHCI '19: Proceedings of the 21st International Conference on Human-Computer Interaction with Mobile Devices and Services

Moving a caret to the desired position in a text field is challenging and inefficient, especially when the user has only one hand available to hold and interact with a mobile phone. We propose a shortcut on keyboards that enables precise and efficient ...
Touch+: Expanding Touch Input Vocabulary using a Smartphone and a Smartwatch
CHI EA '15: Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems

In this video, we present a new interaction technique, called Touch+, which expands touch input vocabulary by using both mobile devices and wrist-worn devices. This technique enables to recognize twisting, rolling, and lifting of a finger while it is ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Transactions on Computer-Human Interaction

ACM Transactions on Computer-Human Interaction Volume 27, Issue 5

October 2020

270 pages

ISSN:1073-0516

EISSN:1557-7325

DOI:10.1145/3415023

Editor:
Kristina Höök
KTH Royal Institute of Technology, Sweden

Issue’s Table of Contents

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 August 2020

Online AM: 07 May 2020

Accepted: 01 April 2020

Revised: 01 February 2020

Received: 01 February 2019

Published in TOCHI Volume 27, Issue 5

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

16
Total Citations
View Citations
656
Total Downloads

Downloads (Last 12 months)112
Downloads (Last 6 weeks)14

Reflects downloads up to 10 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Li CYu JHe KFeng JZhou J(2024)SwivelTouch: Boosting Touchscreen Input with 3D Finger Rotation GestureProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36595848:2(1-30)Online publication date: 15-May-2024
https://dl.acm.org/doi/10.1145/3659584
Yeh YIrudayaraj AVogel D(2024)Single-handed Folding Interactions with a Modified Clamshell Flip PhoneProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642554(1-14)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642554
Song JShi RLi YGao BLiang H(2024)Exploring Controller-based Techniques for Precise and Rapid Text Selection in Virtual Reality2024 IEEE Conference Virtual Reality and 3D User Interfaces (VR)10.1109/VR58804.2024.00047(244-253)Online publication date: 16-Mar-2024
https://doi.org/10.1109/VR58804.2024.00047
Villarreal-Narvaez SSluÿters AVanderdonckt JVatavu R(2023)Brave New GES World: A Systematic Literature Review of Gestures and Referents in Gesture Elicitation StudiesACM Computing Surveys10.1145/363645856:5(1-55)Online publication date: 7-Dec-2023
https://dl.acm.org/doi/10.1145/3636458
Rakhmetulla GRen YArif A(2023)GeShort: One-Handed Mobile Text Editing and Formatting with Gestural Shortcuts and a Floating ClipboardProceedings of the ACM on Human-Computer Interaction10.1145/36042597:MHCI(1-23)Online publication date: 13-Sep-2023
https://dl.acm.org/doi/10.1145/3604259
Xie HHan PLi CChen YZeng S(2023)Lightweight Midrange Arm-Gesture Recognition System From mmWave Radar Point CloudsIEEE Sensors Journal10.1109/JSEN.2022.321667623:2(1261-1270)Online publication date: 15-Jan-2023
https://doi.org/10.1109/JSEN.2022.3216676
De Rosa MFuccella VCostagliola GAlbanese MGalasso FGalasso L(2023)Arrow2edit: A Technique for Editing Text on SmartphonesHuman-Computer Interaction10.1007/978-3-031-35596-7_27(416-432)Online publication date: 9-Jul-2023
https://doi.org/10.1007/978-3-031-35596-7_27
Steuerlein BMayer S(2022)Conductive Fiducial Tangibles for EveryoneProceedings of the ACM on Human-Computer Interaction10.1145/35467186:MHCI(1-22)Online publication date: 20-Sep-2022
https://dl.acm.org/doi/10.1145/3546718
Liu MKuznetsov AKim YChang JKittur AMyers B(2022)Wigglite: Low-cost Information Collection and TriageProceedings of the 35th Annual ACM Symposium on User Interface Software and Technology10.1145/3526113.3545661(1-16)Online publication date: 29-Oct-2022
https://dl.acm.org/doi/10.1145/3526113.3545661
Iijima RShitara AOchiai Y(2022)Designing Gestures for Digital Musical Instruments: Gesture Elicitation Study with Deaf and Hard of Hearing PeopleProceedings of the 24th International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3517428.3544828(1-8)Online publication date: 23-Oct-2022
https://dl.acm.org/doi/10.1145/3517428.3544828
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Issue’s Table of Contents