research-article

Software-reduced touchscreen latency

Authors:

Alireza Sahami ShiraziAuthors Info & Claims

MobileHCI '16: Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services

Pages 434 - 441

https://doi.org/10.1145/2935334.2935381

Published: 06 September 2016 Publication History

Abstract

Devices with touchscreens have an inherent latency. When a user's finger drags an object across the screen the object follows with a latency of around 100ms for current devices. Previous work showed that latencies down to 25ms reduce users' performance and that even 10ms latency is noticeable. In this paper we demonstrate an approach that reduces latency using a predictive model. Extrapolating the finger's movement we predict where the finger will be in the next moment. Comparing different prediction approaches we show for three different tasks that prediction using neural networks is more precise than linear and polynomial extrapolation. Furthermore, we show through a Fitts' Law dragging experiment that reducing touch latency can significantly increases users' performance. As the approach is software-based it can easily be integrated into existing mobile applications and systems.

References

[1]

Fakhreddine Ababsa, Malik Mallem, and David Roussel. 2004. Comparison between particle filter approach and Kalman filter-based technique for head tracking in augmented reality systems. In Proceedings or the IEEE International Conference on Robotics and Automation, Vol. 1. IEEE, 1021--1026.

[2]

Robert S. Allison, Laurence R. Harris, Michael Jenkin, Urszula Jasiobedzka, and James E. Zacher. 2001. Tolerance of temporal delay in virtual environments. In Virtual Reality, 2001. Proceedings. IEEE. IEEE, 247--254.

Digital Library

[3]

Glen Anderson, Rina Doherty, and Subhashini Ganapathy. 2011. User Perception of Touch Screen Latency. In Design, User Experience, and Usability. Theory, Methods, Tools and Practice. Springer, 195--202.

[4]

François Bérard and Renaud Blanch. 2013. Two touch system latency estimators: high accuracy and low overhead. In Proceedings of the 2013 ACM international conference on Interactive tabletops and surfaces. ACM, 241--250.

Digital Library

[5]

H. Braun and M. Riedmiller. 1992. RPROP: a fast adaptive learning algorithm. In Proceedings of the International Symposium on Computer and Information Science VII.

[6]

Timothy J. Buker, Dennis A. Vincenzi, and John E. Deaton. 2012. The Effect of Apparent Latency on Simulator Sickness While Using a See-Through Helmet-Mounted Display Reducing Apparent Latency With Predictive Compensation. Human Factors: The Journal of the Human Factors and Ergonomics Society 54, 2 (2012), 235--249.

[7]

Elie Cattan, Amélie Rochet-Capellan, Pascal Perrier, and François Bérard. 2015. Reducing Latency with a Continuous Prediction: Effects on Users' Performance in Direct-Touch Target Acquisitions. In Proceedings of the 2015 International Conference on Interactive Tabletops & Surfaces. ACM, New York, NY, USA, 205--214.

Digital Library

[8]

Stephen R. Ellis, François Bréant, Brian M. Menges, Richard H. Jacoby, and Bernard D. Adelstein. 1997. Operator interaction with virtual objects: effect of system latency. Advances in human factors/ergonomics (1997), 973--976.

[9]

J. M. Hannan and J. M. Bishop. 1997. A comparison of fast training algorithms over two real problems. In Artificial Neural Networks, Fifth International Conference on (Conf. Publ. No. 440). IET, 1--6.

[10]

Lars Kai Hansen and Peter Salamon. 1990. Neural network ensembles. IEEE Transactions on Pattern Analysis & Machine Intelligence 10 (1990), 993--1001.

Digital Library

[11]

Jeff Heaton. 2015. Encog: Library of Interchangeable Machine Learning Models for Java and C#. Journal of Machine Learning Research 16 (2015), 1243--1247. http://jmlr.org/papers/v16/heaton15a.html

Digital Library

[12]

Niels Henze and Martin Pielot. 2013. App Stores: External Validity for Mobile HCI. interactions 20, 2 (March 2013), 33--38.

Digital Library

[13]

Niels Henze, Martin Pielot, Benjamin Poppinga, Torben Schinke, and Susanne Boll. 2011. My app is an experiment: Experience from user studies in mobile app stores. International Journal of Mobile Human Computer Interaction (IJMHCI) 3, 4 (2011), 71--91.

Digital Library

[14]

Ricardo Jota, Albert Ng, Paul Dietz, and Daniel Wigdor. 2013. How fast is fast enough?: a study of the effects of latency in direct-touch pointing tasks. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 2291--2300.

Digital Library

[15]

Topi Kaaresoja and Stephen Brewster. 2010. Feedback is... late: measuring multimodal delays in mobile device touchscreen interaction. In International Conference on Multimodal Interfaces and the Workshop on Machine Learning for Multimodal Interaction. ACM, 2.

Digital Library

[16]

Bekir Karlik and A. Vehbi Olgac. 2011. Performance analysis of various activation functions in generalized MLP architectures of neural networks. International Journal of Artificial Intelligence and Expert Systems 1, 4 (2011), 111--122.

[17]

Edward Lank, Yi-Chun Nikko Cheng, and Jaime Ruiz. 2007. Endpoint prediction using motion kinematics. In Proceedings of the SIGCHI conference on Human factors in computing systems. ACM, 637--646.

Digital Library

[18]

I. Scott MacKenzie and R. William Soukoreff. 2003. Phrase sets for evaluating text entry techniques. In CHI'03 extended abstracts on Human factors in computing systems. ACM, 754--755.

Digital Library

[19]

I. Scott MacKenzie and Colin Ware. 1993. Lag as a determinant of human performance in interactive systems. In Proceedings of the INTERACT'93 and CHI'93 conference on Human factors in computing systems. ACM, 488--493.

Digital Library

[20]

Michael Meehan, Sharif Razzaque, Mary C. Whitton, and Frederick P. Brooks Jr. 2003. Effect of latency on presence in stressful virtual environments. In Virtual Reality, 2003. Proceedings. IEEE. IEEE, 141--148.

Digital Library

[21]

W. Todd Nelson, Merry M. Roe, Robert S. Bolia, and Rebecca M. Morley. 2000. Assessing simulator sickness in a see-through HMD: Effects of time delay, time on task, and task complexity. Technical Report. DTIC Document.

[22]

Albert Ng, Julian Lepinski, Daniel Wigdor, Steven Sanders, and Paul Dietz. 2012. Designing for low-latency direct-touch input. In Proceedings of the 25th annual ACM symposium on User interface software and technology. ACM, 453--464.

Digital Library

[23]

Phillip T. Pasqual and Jacob O. Wobbrock. 2014. Mouse pointing endpoint prediction using kinematic template matching. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 743--752.

Digital Library

[24]

Andriy Pavlovych and Carl Gutwin. 2012. Assessing Target Acquisition and Tracking Performance for Complex Moving Targets in the Presence of Latency and Jitter. In Proceedings of Graphics Interface 2012. Canadian Information Processing Society, Toronto, Ont., Canada, Canada, 109--116. http://dl.acm.org/citation.cfm?id=2305276.2305295

Digital Library

[25]

Andriy Pavlovych and Wolfgang Stuerzlinger. 2009a. The Tradeoff Between Spatial Jitter and Latency in Pointing Tasks. In Proceedings of the 1st ACM SIGCHI Symposium on Engineering Interactive Computing Systems. ACM, New York, NY, USA, 187--196.

Digital Library

[26]

Andriy Pavlovych and Wolfgang Stuerzlinger. 2009b. The tradeoff between spatial jitter and latency in pointing tasks. In Proceedings of the 1st ACM SIGCHI symposium on Engineering interactive computing systems. ACM, 187--196.

Digital Library

[27]

Andriy Pavlovych and Wolfgang Stuerzlinger. 2011. Target Following Performance in the Presence of Latency, Jitter, and Signal Dropouts. In Proceedings of Graphics Interface 2011. Canadian Human-Computer Communications Society, School of Computer Science, University of Waterloo, Waterloo, Ontario, Canada, 33--40. http://dl.acm.org/citation.cfm?id=1992917.1992924

Digital Library

[28]

Martin Pielot, Niels Henze, and Susanne Boll. 2011. Experiments in app stores-how to ask users for their consent. In Proceedings of the workshop on Ethics, logs & videotape.

[29]

Martin Riedmiller and Heinrich Braun. 1993. A direct adaptive method for faster backpropagation learning: The RPROP algorithm. In Neural Networks, 1993., IEEE International Conference on. IEEE, 586--591.

[30]

Wolfram Schiffmann, Merten Joost, and Randolf Werner. 1993. Comparison of optimized backpropagation algorithms. In In Proceedings of the European Symposium on Artificial Neural Networks, Vol. 93. Citeseer, 97--104.

[31]

Richard H. Y. So and German K. M. Chung. 2005. Sensory motor responses in virtual environments: Studying the effects of image latencies for target-directed hand movement. In Engineering in Medicine and Biology Society, 2005. IEEE-EMBS 2005. 27th Annual International Conference of the. IEEE, 5006--5008.

[32]

Haijun Xia, Ricardo Jota, Benjamin McCanny, Zhe Yu, Clifton Forlines, Karan Singh, and Daniel Wigdor. 2014. Zero-latency tapping: using hover information to predict touch locations and eliminate touchdown latency. In Proceedings of the 27th annual ACM symposium on User interface software and technology. ACM, 205--214.

Digital Library

[33]

Brian Ziebart, Anind Dey, and J. Andrew Bagnell. 2012. Probabilistic pointing target prediction via inverse optimal control. In Proceedings of the 2012 ACM international conference on Intelligent User Interfaces. ACM, 1--10.

Digital Library

Cited By

Schmid AHalbhuber DFischer TWimmer RHenze N(2023)Small Latency Variations Do Not Affect Player Performance in First-Person ShootersProceedings of the ACM on Human-Computer Interaction10.1145/36110277:CHI PLAY(197-216)Online publication date: 4-Oct-2023
https://dl.acm.org/doi/10.1145/3611027
Nishida NIkematsu KSato JYamanaka STsubouchi K(2023)Single-tap Latency Reduction with Single- or Double- tap PredictionProceedings of the ACM on Human-Computer Interaction10.1145/36042717:MHCI(1-26)Online publication date: 13-Sep-2023
https://dl.acm.org/doi/10.1145/3604271
Wiese JHenze N(2023)Predicting Mouse Positions Beyond a System’s Latency Can Increase Throughput and User Experience in Linear Steering TasksProceedings of Mensch und Computer 202310.1145/3603555.3603556(101-115)Online publication date: 3-Sep-2023
https://dl.acm.org/doi/10.1145/3603555.3603556
Show More Cited By

Index Terms

Software-reduced touchscreen latency
1. Human-centered computing
  1. Human computer interaction (HCI)

Recommendations

PredicTouch: A System to Reduce Touchscreen Latency using Neural Networks and Inertial Measurement Units
ISS '17: Proceedings of the 2017 ACM International Conference on Interactive Surfaces and Spaces

Touchscreens are the dominant input mechanism for a variety of devices. One of the main limitations of touchscreens is the latency to receive input, refresh, and respond. This latency is easily perceivable and reduces users' performance. Previous work ...
Improving software-reduced touchscreen latency
MobileHCI '17: Proceedings of the 19th International Conference on Human-Computer Interaction with Mobile Devices and Services

The latency of current mobile devices' touchscreens is around 100ms and has widely been explored. Latency down to 2ms is noticeable, and latency as low as 25ms reduces users' performance. Previous work reduced touch latency by extrapolating a finger's ...
Designing for low-latency direct-touch input
UIST '12: Proceedings of the 25th annual ACM symposium on User interface software and technology

Software designed for direct-touch interfaces often utilize a metaphor of direct physical manipulation of pseudo "real-world" objects. However, current touch systems typically take 50-200ms to update the display in response to a physi-cal touch action. ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

MobileHCI '16: Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services

September 2016

567 pages

ISBN:9781450344081

DOI:10.1145/2935334

General Chairs:
Fabio Paternò
CNR-ISTI, Italy
,
Kaisa Väänänen
Tampere University of Technology, Finland
,
Program Chairs:
Karen Church
Intercom, USA
,
Jonna Häkkilä
University of Lapland, Finland
,
Antonio Krüger
DFKI, Germany
,
Marcos Serrano
University of Toulouse, France

Copyright © 2016 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 06 September 2016

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Deutsche Forschungsgemeinschaft
Ministerium für Wissenschaft, Forschung und Kunst Baden-Württemberg

Conference

MobileHCI '16

Sponsor:

SIGCHI

MobileHCI '16: 18th International Conference on Human-Computer Interaction with Mobile Devices and Services

September 6 - 9, 2016

Florence, Italy

Acceptance Rates

Overall Acceptance Rate 202 of 906 submissions, 22%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

33
Total Citations
View Citations
591
Total Downloads

Downloads (Last 12 months)36
Downloads (Last 6 weeks)1

Reflects downloads up to 26 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Schmid AHalbhuber DFischer TWimmer RHenze N(2023)Small Latency Variations Do Not Affect Player Performance in First-Person ShootersProceedings of the ACM on Human-Computer Interaction10.1145/36110277:CHI PLAY(197-216)Online publication date: 4-Oct-2023
https://dl.acm.org/doi/10.1145/3611027
Nishida NIkematsu KSato JYamanaka STsubouchi K(2023)Single-tap Latency Reduction with Single- or Double- tap PredictionProceedings of the ACM on Human-Computer Interaction10.1145/36042717:MHCI(1-26)Online publication date: 13-Sep-2023
https://dl.acm.org/doi/10.1145/3604271
Wiese JHenze N(2023)Predicting Mouse Positions Beyond a System’s Latency Can Increase Throughput and User Experience in Linear Steering TasksProceedings of Mensch und Computer 202310.1145/3603555.3603556(101-115)Online publication date: 3-Sep-2023
https://dl.acm.org/doi/10.1145/3603555.3603556
Nicosia JWang BAschenbrenner ASliwinski MYabiku SRoque NGermine LBateman RMorris JHassenstab J(2022)To BYOD or not: Are device latencies important for bring-your-own-device (BYOD) smartphone cognitive testing?Behavior Research Methods10.3758/s13428-022-01925-1Online publication date: 11-Aug-2022
https://doi.org/10.3758/s13428-022-01925-1
Halbhuber DSchlenczek MBogon JHenze N(2022)Better be quiet about it! The Effects of Phantom Latency on Experienced First-Person Shooter PlayersProceedings of the 21st International Conference on Mobile and Ubiquitous Multimedia10.1145/3568444.3568448(172-181)Online publication date: 27-Nov-2022
https://dl.acm.org/doi/10.1145/3568444.3568448
Li DGe XMa QMehra BLiu JHan TLiu C(2022)Evaluating Three Touch Gestures for Moving Objects across Folded ScreensProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/35503096:3(1-28)Online publication date: 7-Sep-2022
https://dl.acm.org/doi/10.1145/3550309
Halbhuber DSchwind VHenze N(2022)Don't Break my Flow: Effects of Switching Latency in Shooting Video GamesProceedings of the ACM on Human-Computer Interaction10.1145/35494926:CHI PLAY(1-20)Online publication date: 31-Oct-2022
https://dl.acm.org/doi/10.1145/3549492
Zhang THu ZGupta AWu CBenko HJonker T(2022)RIDS: Implicit Detection of a Selection Gesture Using Hand Motion Dynamics During Freehand Pointing in Virtual RealityProceedings of the 35th Annual ACM Symposium on User Interface Software and Technology10.1145/3526113.3545701(1-12)Online publication date: 29-Oct-2022
https://dl.acm.org/doi/10.1145/3526113.3545701
Yu DDesai RZhang TBenko HJonker TGupta A(2022)Optimizing the Timing of Intelligent Suggestion in Virtual RealityProceedings of the 35th Annual ACM Symposium on User Interface Software and Technology10.1145/3526113.3545632(1-20)Online publication date: 29-Oct-2022
https://dl.acm.org/doi/10.1145/3526113.3545632
Halbhuber D(2022)To Lag or Not to Lag: Understanding and Compensating Latency in Video GamesExtended Abstracts of the 2022 Annual Symposium on Computer-Human Interaction in Play10.1145/3505270.3558364(370-373)Online publication date: 2-Nov-2022
https://dl.acm.org/doi/10.1145/3505270.3558364
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents