Nothing Special   »   [go: up one dir, main page]

skip to main content
research-article

Single-tap Latency Reduction with Single- or Double- tap Prediction

Published: 13 September 2023 Publication History

Abstract

Touch surfaces are widely utilized for smartphones, tablet PCs, and laptops (touchpad), and single and double taps are the most basic and common operations on them. The detection of single or double taps causes the single-tap latency problem, which creates a bottleneck in terms of the sensitivity of touch inputs. To reduce the single-tap latency, we propose a novel machine-learning-based tap prediction method called PredicTaps. Our method predicts whether a detected tap is a single tap or the first contact of a double tap without having to wait for the hundreds of milliseconds conventionally required. We present three evaluations and one user evaluation that demonstrate its broad applicability and usability for various tap situations on two form factors (touchpad and smartphone). The results showed PredicTaps reduces the single-tap latency from 150--500 ms to 12 ms on laptops and to 17.6 ms on smartphones without reducing usability.

Supplementary Material

MP4 File (v7mhci224.mp4)
Supplemental video

References

[1]
Liliana Barrios, Pietro Oldrati, David Lindlbauer, Marc Hilty, Helen Hayward-Koennecke, Christian Holz, and Andreas Lutterotti. 2020. A Rapid Tapping Task on Commodity Smartphones to Assess Motor Fatigability. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (CHI '20). Association for Computing Machinery, New York, NY, USA, 1--10. https://doi.org/10.1145/3313831.3376588
[2]
Tobias Boceck, Sascha Sprott, Huy Viet Le, and Sven Mayer. 2019. Force Touch Detection on Capacitive Sensors Using Deep Neural Networks. In Proceedings of the 21st International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI '19). Association for Computing Machinery, New York, NY, USA, Article Article 42, 6 pages. https://doi.org/10.1145/3338286.3344389
[3]
Kim Byoungeul and Lim Yeongkyu. 2012. US Patent 20130271487A1, Position lag reduction for computer drawing. https://patents.google.com/patent/US20130271487
[4]
Kim Byoungeul and Lim Yeongkyu. 2014. WO Patent 2014129753A1, Mobile terminal and touch coordinate predicting method thereof. https://www.google.com/patents/WO2014129753A1?cl=en.
[5]
Elie Cattan, Amélie Rochet-Capellan, Pascal Perrier, and François Bérard. 2015. Reducing Latency with a Continuous Prediction: Effects on Users' Performance in Direct-Touch Target Acquisitions. In Proceedings of the 2015 International Conference on Interactive Tabletops & Surfaces (ITS '15). ACM, New York, NY, USA, 205--214. https://doi.org/10.1145/2817721.2817736
[6]
A. Cockburn, D. Masson, C. Gutwin, P. Palanque, A. Goguey, M. Yung, C. Gris, and C. Trask. 2019. Design and evaluation of braced touch for touchscreen input stabilisation. International Journal of Human-Computer Studies 122 (2019), 21 -- 37. https://doi.org/10.1016/j.ijhcs.2018.08.005
[7]
Jacob Cohen. 2013. Statistical power analysis for the behavioral sciences. Academic press.
[8]
Microsoft Corp. 2018. DoubleClickSpeed. https://docs.microsoft.com/en-us/previous-versions/windows/it-pro/windows-2000-server/cc978662(v=technet.10)?redirectedfrom=MSDN
[9]
Microsoft Corp. 2018. Windows Dev Center. https://docs.microsoft.com/ja-jp/windows/desktop/Controls/ttm-setdelaytime
[10]
Jonathan Deber, Bruno Araujo, Ricardo Jota, Clifton Forlines, Darren Leigh, Steven Sanders, and Daniel Wigdor. 2016. Hammer Time!: A Low-Cost, High Precision, High Accuracy Tool to Measure the Latency of Touchscreen Devices. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). ACM, New York, NY, USA, 2857--2868. https://doi.org/10.1145/2858036.2858394
[11]
Jonathan Deber, Ricardo Jota, Clifton Forlines, and Daniel Wigdor. 2015. How Much Faster is Fast Enough?: User Perception of Latency & Latency Improvements in Direct and Indirect Touch. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). ACM, New York, NY, USA, 1827--1836. https://doi.org/10.1145/2702123.2702300
[12]
Tobias Grosse-Puppendahl, Christian Holz, Gabe Cohn, Raphael Wimmer, Oskar Bechtold, Steve Hodges, Matthew S. Reynolds, and Joshua R. Smith. 2017. Finding Common Ground: A Survey of Capacitive Sensing in Human-Computer Interaction. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17). ACM, New York, NY, USA, 3293--3315. https://doi.org/10.1145/3025453.3025808
[13]
Anhong Guo, Robert Xiao, and Chris Harrison. 2015. CapAuth: Identifying and Differentiating User Handprints on Commodity Capacitive Touchscreens. In Proceedings of the 2015 International Conference on Interactive Tabletops & Surfaces (ITS '15). ACM, New York, NY, USA, 59--62. https://doi.org/10.1145/2817721.2817722
[14]
Niels Henze, Markus Funk, and Alireza Sahami Shirazi. 2016. Software-reduced Touchscreen Latency. In Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI '16). ACM, New York, NY, USA, 434--441. https://doi.org/10.1145/2935334.2935381
[15]
Niels Henze, Sven Mayer, Huy Viet Le, and Valentin Schwind. 2017. Improving Software-reduced Touchscreen Latency. In Proceedings of the 19th International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI '17). ACM, New York, NY, USA, Article 107, 8 pages. https://doi.org/10.1145/3098279.3122150
[16]
Niels Henze, Enrico Rukzio, and Susanne Boll. 2011. 100,000,000 taps: analysis and improvement of touch performance in the large. In Proceedings of the 13th International Conference on Human Computer Interaction with Mobile Devices and Services (MobileHCI '11). Association for Computing Machinery, New York, NY, USA, 133--142. https://doi.org/10.1145/2037373.2037395
[17]
Seongkook Heo, Jiseong Gu, and Geehyuk Lee. 2014. Expanding Touch Input Vocabulary by Using Consecutive Distant Taps. In Proceedings of the 32Nd Annual ACM Conference on Human Factors in Computing Systems (CHI '14). ACM, New York, NY, USA, 2597--2606. https://doi.org/10.1145/2556288.2557234
[18]
Ken Hinckley, Seongkook Heo, Michel Pahud, Christian Holz, Hrvoje Benko, Abigail Sellen, Richard Banks, Kenton O'Hara, Gavin Smyth, and William Buxton. 2016. Pre-Touch Sensing for Mobile Interaction. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). ACM, New York, NY, USA, 2869--2881. https://doi.org/10.1145/2858036.2858095
[19]
Christian Holz, Senaka Buthpitiya, and Marius Knaust. 2015. Bodyprint: Biometric User Identification on Mobile Devices Using the Capacitive Touchscreen to Scan Body Parts. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). ACM, New York, NY, USA, 3011--3014. https://doi.org/10.1145/2702123.2702518
[20]
Wenson Hsieh. 2015. More Responsive Tapping on iOS. https://webkit.org/blog/5610/more-responsive-tapping-on-ios/
[21]
Kaori Ikematsu, Masaaki Fukumoto, and Itiro Siio. 2019. Ohmic-Sticker: Force-to-Motion Type Input Device that Extends Capacitive Touch Surface. In Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology (UIST '19). Association for Computing Machinery, New York, NY, USA, 1021--1030. https://doi.org/10.1145/3332165.3347903
[22]
Kaori Ikematsu, Shota Yamanaka, and Kota Tsubouchi. 2020. PredicTaps: Latency Reduction Technique for Single-taps Based on Prediction for Single-tap or Double-tap. In Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems (CHI EA '20). ACM, New York, NY, USA, 6 pages.
[23]
Kaori Ikematsu, Shota Yamanaka, and Kota Tsubouchi. 2020. PredicTaps: Machine Learning-Based Latency Reduction Technique for Single-taps. In Proceedings of Interaction2020. IPSJ.
[24]
Apple Inc. 2019. Change Speaking Rate, Double-Tap Timeout, and Audio Ducking. https://support.apple.com/en-in/HT208434
[25]
Valeriu Manuel Ionescu. 2016. Using cross platform development libraries. telerik mobile. In 2016 15th RoEduNet Conference: Networking in Education and Research. IEEE, 1--6.
[26]
Kevin Jiokeng, Gentian Jakllari, and André-Luc Beylot. 2022. I Want to Know Your Hand: Authentication on Commodity Mobile Phones Based on Your Hand's Vibrations. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 6, 2, Article 58 (jul 2022), 27 pages. https://doi.org/10.1145/3534575
[27]
Ricardo Jota, Albert Ng, Paul Dietz, and Daniel Wigdor. 2013. How Fast is Fast Enough?: A Study of the Effects of Latency in Direct-touch Pointing Tasks. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '13). ACM, New York, NY, USA, 2291--2300. https://doi.org/10.1145/2470654.2481317
[28]
Steven W Keele and Michael I Posner. 1968. Processing of visual feedback in rapid movements. Journal of experimental psychology 77, 1 (1968), 155.
[29]
Dong Yoon Kim and Scott A Miller. 2013. US8442797B2, Directional tap detection algorithm using an accelerometer. https://patents.google.com/patent/US8442797B2
[30]
Naoki Kimura. 2022. Self-Supervised Approach for Few-Shot Hand Gesture Recognition. In Adjunct Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology (UIST '22 Adjunct). Association for Computing Machinery, New York, NY, USA, Article 21, 4 pages. https://doi.org/10.1145/3526114.3558707
[31]
Naoki Kimura. 2022. Self-Supervised Approach for Few-Shot Hand Gesture Recognition. In The Adjunct Publication of the 35th Annual ACM Symposium on User Interface Software and Technology (UIST '22 Adjunct). Association for Computing Machinery, New York, NY, USA, Article 21, 4 pages. https://doi.org/10.1145/3526114.3558707
[32]
Jason E King. 2008. Binary logistic regression. Best practices in quantitative methods (2008), 358--384.
[33]
Takumi Kusano and Takashi Komuro. 2015. 3D Tabletop User Interface with High Synchronization Accuracy Using a High-speed Stereo Camera. In Proceedings of the 2015 International Conference on Interactive Tabletops & Surfaces (ITS '15). ACM, New York, NY, USA, 39--42. https://doi.org/10.1145/2817721.2817724
[34]
Gierad Laput, Karan Ahuja, Mayank Goel, and Chris Harrison. 2018. Ubicoustics: Plug-and-Play Acoustic Activity Recognition. In Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology (UIST '18). Association for Computing Machinery, New York, NY, USA, 213--224. https://doi.org/10.1145/3242587.3242609
[35]
Huy Viet Le, Thomas Kosch, Patrick Bader, Sven Mayer, and Niels Henze. 2018. PalmTouch: Using the Palm As an Additional Input Modality on Commodity Smartphones. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18). ACM, New York, NY, USA, Article 360, 13 pages. https://doi.org/10.1145/3173574.3173934
[36]
Huy Viet Le, Sven Mayer, and Niels Henze. 2019. Investigating the Feasibility of Finger Identification on Capacitive Touchscreens Using Deep Learning. In Proceedings of the 24th International Conference on Intelligent User Interfaces (IUI '19). ACM, New York, NY, USA, 637--649. https://doi.org/10.1145/3301275.3302295
[37]
Huy Viet Le, Valentin Schwind, Philipp Göttlich, and Niels Henze. 2017. PredicTouch: A System to Reduce Touchscreen Latency Using Neural Networks and Inertial Measurement Units. In Proceedings of the 2017 ACM International Conference on Interactive Surfaces and Spaces (ISS '17). ACM, New York, NY, USA, 230--239. https://doi.org/10.1145/3132272.3134138
[38]
Darren Leigh, Clifton Forlines, Ricardo Jota, Steven Sanders, and Daniel Wigdor. 2014. High Rate, Low-Latency Multi-Touch Sensing with Simultaneous Orthogonal Multiplexing. In Proceedings of the 27th Annual ACM Symposium on User Interface Software and Technology (UIST '14). Association for Computing Machinery, New York, NY, USA, 355--364. https://doi.org/10.1145/2642918.2647353
[39]
Jiandong Liang, Chris Shaw, and Mark Green. 1991. On Temporal-spatial Realism in the Virtual Reality Environment. In Proceedings of the 4th Annual ACM Symposium on User Interface Software and Technology (UIST '91). ACM, New York, NY, USA, 19--25. https://doi.org/10.1145/120782.120784
[40]
Hyunchul Lim, Jungmin Chung, Changhoon Oh, SoHyun Park, Joonhwan Lee, and Bongwon Suh. 2018. TouchFinger: Extending Touch-Based User Interface Capabilities with ?Idle" Finger Gestures in the Air. In Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology (UIST '18). Association for Computing Machinery, New York, NY, USA, 335--346. https://doi.org/10.1145/3242587.3242651
[41]
Quingkui Man and Xinbin Liu. 2014. CN Patent 103902086A, Curve fitting based touch trajectory smoothing method and system. https://www.google.ca/patents/CN103902086A?cl=en.
[42]
Fabrice Matulic, Daniel Vogel, and Raimund Dachselt. 2017. Hand Contact Shape Recognition for Posture-Based Tabletop Widgets and Interaction. In Proceedings of the 2017 ACM International Conference on Interactive Surfaces and Spaces (ISS '17). Association for Computing Machinery, New York, NY, USA, 3--11. https://doi.org/10.1145/3132272.3134126
[43]
Mathieu Nancel, Stanislav Aranovskiy, Rosane Ushirobira, Denis Efimov, Sebastien Poulmane, Nicolas Roussel, and Géry Casiez. 2018. Next-Point Prediction for Direct Touch Using Finite-Time Derivative Estimation. In Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology (UIST '18). ACM, New York, NY, USA, 793--807. https://doi.org/10.1145/3242587.3242646
[44]
Mathieu Nancel, Daniel Vogel, Bruno De Araujo, Ricardo Jota, and Géry Casiez. 2016. Next-Point Prediction Metrics for Perceived Spatial Errors. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology (UIST '16). ACM, New York, NY, USA, 271--285. https://doi.org/10.1145/2984511.2984590
[45]
Albert Ng, Julian Lepinski, Daniel Wigdor, Steven Sanders, and Paul Dietz. 2012. Designing for Low-latency Direct-touch Input. In Proceedings of the 25th Annual ACM Symposium on User Interface Software and Technology (UIST '12). ACM, New York, NY, USA, 453--464. https://doi.org/10.1145/2380116.2380174
[46]
Walter Ritter, Guido Kempter, and Tobias Werner. 2015. User-Acceptance of Latency in Touch Interactions. In Universal Access in Human-Computer Interaction. Access to Interaction, Margherita Antona and Constantine Stephanidis (Eds.). Springer International Publishing, Cham, 139--147.
[47]
Shlomo Sawilowsky. 2009. New Effect Size Rules of Thumb. Journal of Modern Applied Statistical Methods 8 (11 2009), 597--599. https://doi.org/10.22237/jmasm/1257035100
[48]
Richard A Schmidt, Howard N Zelaznik, and James S Frank. 1978. Sources of inaccuracy in rapid movement. In Information processing in motor control and learning. Elsevier, 183--203.
[49]
Craig Shultz, Daehwa Kim, Karan Ahuja, and Chris Harrison. 2022. TriboTouch: Micro-Patterned Surfaces for Low Latency Touchscreens. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (CHI '22). Association for Computing Machinery, New York, NY, USA, Article 347, 13 pages. https://doi.org/10.1145/3491102.3502069
[50]
Michael W Smith, Joseph Sharit, and Sara J Czaja. 1999. Aging, motor control, and the performance of computer mouse tasks. Human factors 41, 3 (1999), 389--396.
[51]
Zixiong Su, Shitao Fang, and Jun Rekimoto. 2023. LipLearner: Customizable Silent Speech Interactions on Mobile Devices. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (CHI '23). Association for Computing Machinery, New York, NY, USA, Article 696, 21 pages. https://doi.org/10.1145/3544548.3581465
[52]
P Tsoi and J Xiao. 2015. Advanced touch input on iOS: Increasing responsiveness by reducing latency. https://developer.apple.com/videos/play/wwdc2015/233
[53]
Ruolin Wang, Chun Yu, Xing-Dong Yang, Weijie He, and Yuanchun Shi. 2019. EarTouch: Facilitating Smartphone Use for Visually Impaired People in Mobile and Public Scenarios. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI '19). ACM, New York, NY, USA, Article 24, 13 pages. https://doi.org/10.1145/3290605.3300254
[54]
Wujun Wang, Xinbin Liu, and Zhou Guangdao. 2013. WO Patent 2013170521A1, Multi-touch tracking method. https://www.google.com/patents/WO2013170521A1?cl=en.
[55]
Jason Wu, Chris Harrison, Jeffrey P. Bigham, and Gierad Laput. 2020. Automated Class Discovery and One-Shot Interactions for Acoustic Activity Recognition. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (CHI '20). Association for Computing Machinery, New York, NY, USA, 1--14. https://doi.org/10.1145/3313831.3376875
[56]
Haijun Xia, Ricardo Jota, Benjamin McCanny, Zhe Yu, Clifton Forlines, Karan Singh, and Daniel Wigdor. 2014. Zero-latency Tapping: Using Hover Information to Predict Touch Locations and Eliminate Touchdown Latency. In Proceedings of the 27th Annual ACM Symposium on User Interface Software and Technology (UIST '14). ACM, New York, NY, USA, 205--214. https://doi.org/10.1145/2642918.2647348
[57]
Robert Xiao, Julia Schwarz, and Chris Harrison. 2015. Estimating 3D Finger Angle on Commodity Touchscreens. In Proceedings of the 2015 International Conference on Interactive Tabletops & Surfaces (ITS '15). ACM, New York, NY, USA, 47--50. https://doi.org/10.1145/2817721.2817737
[58]
Xuhai Xu, Jun Gong, Carolina Brum, Lilian Liang, Bongsoo Suh, Shivam Kumar Gupta, Yash Agarwal, Laurence Lindsey, Runchang Kang, Behrooz Shahsavari, Tu Nguyen, Heriberto Nieto, Scott E Hudson, Charlie Maalouf, Jax Seyed Mousavi, and Gierad Laput. 2022. Enabling Hand Gesture Customization on Wrist-Worn Devices. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (CHI '22). Association for Computing Machinery, New York, NY, USA, Article 496, 19 pages. https://doi.org/10.1145/3491102.3501904
[59]
Xiang Zhang, Kaori Ikematsu, Kunihiro Kato, and Yuta Sugiura. 2022. ReflecTouch: Detecting Grasp Posture of Smartphone Using Corneal Reflection Images. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (CHI '22). Association for Computing Machinery, New York, NY, USA, Article 289, 8 pages. https://doi.org/10.1145/3491102.3517440

Cited By

View all
  • (2024)User Performance in Consecutive Temporal Pointing: An Exploratory StudyProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642904(1-15)Online publication date: 11-May-2024
  • (2024)Performance and Reliability of Touch Screen: A Game-Oriented Research2024 5th International Conference on Machine Learning and Human-Computer Interaction (MLHMI)10.1109/MLHMI63000.2024.00016(31-36)Online publication date: 14-Mar-2024

Recommendations

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Proceedings of the ACM on Human-Computer Interaction
Proceedings of the ACM on Human-Computer Interaction  Volume 7, Issue MHCI
MHCI
September 2023
1017 pages
EISSN:2573-0142
DOI:10.1145/3624512
Issue’s Table of Contents
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 13 September 2023
Published in PACMHCI Volume 7, Issue MHCI

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. double tap
  2. latency reduction
  3. prediction
  4. single tap
  5. touch surface

Qualifiers

  • Research-article

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)107
  • Downloads (Last 6 weeks)7
Reflects downloads up to 16 Nov 2024

Other Metrics

Citations

Cited By

View all
  • (2024)User Performance in Consecutive Temporal Pointing: An Exploratory StudyProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642904(1-15)Online publication date: 11-May-2024
  • (2024)Performance and Reliability of Touch Screen: A Game-Oriented Research2024 5th International Conference on Machine Learning and Human-Computer Interaction (MLHMI)10.1109/MLHMI63000.2024.00016(31-36)Online publication date: 14-Mar-2024

View Options

Login options

Full Access

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media