extended-abstract

Ka-Boom!!! Visually Exploring Latency Measurements for XR

Authors:

Jan-Philipp Stauffert,

Kristof Korwisi,

Florian Niebling,

Marc Erich LatoschikAuthors Info & Claims

CHI EA '21: Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems

Article No.: 20, Pages 1 - 9

https://doi.org/10.1145/3411763.3450379

Published: 08 May 2021 Publication History

Abstract

Latency can be detrimental for the experience of Virtual Reality. High latency can lead to loss of performance and cybersickness. There are simple approaches to measure approximate latency and more elaborated for more insight into latency behavior. Yet there are still researchers who do not measure the latency of the system they are using to conduct VR experiments.

This paper provides an illustrated overview of different approaches to measure latency of VR applications, as well as a small decision-making guide to assist in the choice of the measurement method. The visual style offers a more approachable way to understand how to measure latency.

References

[1]

Armin Becher, Jens Angerer, and Thomas Grauschopf. 2018. Novel Approach to Measure Motion-To-Photon and Mouth-To-Ear Latency in Distributed Virtual Reality Systems. arXiv:1809.06320 [cs] (Sept. 2018). http://arxiv.org/abs/1809.06320 arXiv:1809.06320.

[2]

Markus Billeter, Gerhard Röthlin, Jan Wezel, Daisuke Iwai, and Anselm Grundhöfer. 2016. A LED-Based IR/RGB End-to-End Latency Measurement Device. In 2016 IEEE International Symposium on Mixed and Augmented Reality (ISMAR-Adjunct). IEEE, 184–188. https://doi.org/10.1109/ISMAR-Adjunct.2016.0072

[3]

Chun-Ming Chang, Cheng-Hsin Hsu, Chih-Fan Hsu, and Kuan-Ta Chen. 2016. Performance Measurements of Virtual Reality Systems: Quantifying the Timing and Positioning Accuracy. In Proceedings of the 24th ACM international conference on Multimedia. ACM Press, 655–659. https://doi.org/10.1145/2964284.2967303

Digital Library

[4]

Massimiliano Di Luca. 2010. New method to measure end-to-end delay of virtual reality. Presence: Teleoperators and Virtual Environments 19, 6(2010), 569–584. http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=6797715

Digital Library

[5]

Ilja T. Feldstein and Stephen R. Ellis. 2020. A Simple, Video-Based Technique for Measuring Latency in Virtual Reality or Teleoperation. IEEE Transactions on Visualization and Computer Graphics (2020). https://doi.org/10.1109/TVCG.2020.2980527 Conference Name: IEEE Transactions on Visualization and Computer Graphics.

Digital Library

[6]

Sebastian Friston and Anthony Steed. 2014. Measuring Latency in Virtual Environments. IEEE Transactions on Visualization and Computer Graphics 20, 4 (April 2014), 616–625. https://doi.org/10.1109/TVCG.2014.30

Digital Library

[7]

Robert Gruen, Eyal Ofek, Anthony Steed, Ran Gal, Mike Sinclair, and Mar Gonzalez-Franco. 2020. Measuring System Visual Latency through Cognitive Latency on Video See-Through AR Devices. (2020), 791–799.

[8]

Ding He, Fuhu Liu, Dave Pape, Greg Dawe, and Dan Sandin. 2000. Video-based measurement of system latency. In International Immersive Projection Technology Workshop, Vol. 111. Citeseer. http://www.researchgate.net/profile/Dave_Pape/publication/2628137_Video-Based_Measurement_of_System_Latency/links/542166dc0cf203f155c66ad6.pdf

[9]

Zenja Ivkovic, Ian Stavness, Carl Gutwin, and Steven Sutcliffe. 2015. Quantifying and Mitigating the Negative Effects of Local Latencies on Aiming in 3D Shooter Games. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. ACM Press, 135–144. https://doi.org/10.1145/2702123.2702432

Digital Library

[10]

Ryugo Kijima and Kento Miyajima. 2016. Measurement of Head Mounted Display’s latency in rotation and side effect caused by lag compensation by simultaneous observation — An example result using Oculus Rift DK2. In 2016 IEEE Virtual Reality (VR). IEEE, 203–204. https://doi.org/10.1109/VR.2016.7504724 ISSN: 2375-5334.

[11]

Jiandong Liang, Chris Shaw, and Mark Green. 1991. On temporal-spatial realism in the virtual reality environment. In Proceedings of the 4th annual ACM symposium on User interface software and technology. 19–25.

Digital Library

[12]

Dorian Miller and Gary Bishop. 2002. Latency Meter: A device to measure end-to-end latency of VE systems. In Stereoscopic Displays and Virtual Reality Systems IX, Vol. 4660. International Society for Optics and Photonics, 7.

[13]

Mark Mine. 1993. Characterization of end-to-end delays in head-mounted display systems. The University of North Carolina at Chapel Hill, TR93-001 (1993). http://www0.cs.ucl.ac.uk/teaching/VE/Papers/93-001.pdf

Digital Library

[14]

Giorgos Papadakis, Katerina Mania, and Eftichios Koutroulis. 2011. A system to measure, control and minimize end-to-end head tracking latency in immersive simulations. In Proceedings of the 10th International Conference on Virtual Reality Continuum and Its Applications in Industry - VRCAI ’11. ACM Press, Hong Kong, China, 581–584. https://doi.org/10.1145/2087756.2087869

Digital Library

[15]

Sebastian Pape, Marcel Krüger, Jan Müller, and Torsten W. Kuhlen. 2020. Calibratio: A small, low-cost, fully automated Motion-to-Photon Measurement Device. In 2020 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW). IEEE, 234–237. https://doi.org/10.1109/VRW50115.2020.00050

[16]

Kjetil Raaen and Ivar Kjellmo. 2015. Measuring Latency in Virtual Reality Systems. In Entertainment Computing - ICEC 2015, Konstantinos Chorianopoulos, Monica Divitini, Jannicke Baalsrud Hauge, Letizia Jaccheri, and Rainer Malaka(Eds.). Vol. 9353. Springer International Publishing, Cham, 457–462. https://doi.org/10.1007/978-3-319-24589-8_40 Series Title: Lecture Notes in Computer Science.

Digital Library

[17]

Lisa Rebenitsch and Charles Owen. 2016. Review on cybersickness in applications and visual displays. Virtual Reality 20, 2 (June 2016), 101–125. https://doi.org/10.1007/s10055-016-0285-9

Digital Library

[18]

David Roberts, Toby Duckworth, Carl Moore, Robin Wolff, and John O’Hare. 2009. Comparing the End to End Latency of an Immersive Collaborative Environment and a Video Conference. In 2009 13th IEEE/ACM International Symposium on Distributed Simulation and Real Time Applications. IEEE, IEEE, 89–94. https://doi.org/10.1109/DS-RT.2009.43

Digital Library

[19]

Min-Woo Seo, Song-Woo Choi, Sang-Lyn Lee, Eui-Yeol Oh, Jong-Sang Baek, and Suk-Ju Kang. 2017. Photosensor-Based Latency Measurement System for Head-Mounted Displays. Sensors 17, 5 (May 2017), 1112. https://doi.org/10.3390/s17051112

[20]

Jan-Philipp Stauffert, Florian Niebling, and Marc Erich Latoschik. 2020. Simultaneous Run-Time Measurement of Motion-to-Photon Latency and Latency Jitter. In 2020 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). IEEE, IEEE, Atlanta, GA, USA, 636–644. https://doi.org/10.1109/VR46266.2020.1581339481249

[21]

Anthony Steed. 2008. A Simple Method for Estimating the Latency of Interactive, Real-time Graphics Simulations. In Proceedings of the 2008 ACM Symposium on Virtual Reality Software and Technology(VRST ’08). ACM, New York, NY, USA, 123–129. https://doi.org/10.1145/1450579.1450606

Digital Library

[22]

Colin Swindells, John C. Dill, and Kellogg S. Booth. 2000. System lag tests for augmented and virtual environments. In Proceedings of the 13th annual ACM symposium on User interface software and technology. ACM, 161–170. http://dl.acm.org/citation.cfm?id=354444

Digital Library

[23]

Matthias Walter, Tim Wendisch, and Klaus Bengler. 2018. In the Right Place at the Right Time? A View at Latency and Its Implications for Automotive Augmented Reality Head-Up Displays. In Congress of the International Ergonomics Association. Springer, 353–358. https://doi.org/10.1007/978-3-319-96074-6_38

[24]

Weixin Wu, Yujie Dong, and Adam Hoover. 2013. Measuring Digital System Latency from Sensing to Actuation at Continuous 1-ms Resolution. Presence: Teleoperators and Virtual Environments 22, 1 (Feb. 2013), 20–35. https://doi.org/10.1162/PRES_a_00131

Cited By

Souchet ALourdeaux DBurkhardt JHancock P(2023)Design guidelines for limiting and eliminating virtual reality-induced symptoms and effects at work: a comprehensive, factor-oriented reviewFrontiers in Psychology10.3389/fpsyg.2023.116193214Online publication date: 9-Jun-2023
https://doi.org/10.3389/fpsyg.2023.1161932
Pelikan HHofstetter E(2023)Managing Delays in Human-Robot InteractionACM Transactions on Computer-Human Interaction10.1145/356989030:4(1-42)Online publication date: 12-Sep-2023
https://dl.acm.org/doi/10.1145/3569890
Döllinger NWolf EBotsch MLatoschik MWienrich C(2023)Are Embodied Avatars Harmful to our Self-Experience? The Impact of Virtual Embodiment on Body AwarenessProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3580918(1-14)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3580918
Show More Cited By

Recommendations

Fine-grained latency and loss measurements in the presence of reordering
SIGMETRICS '11: Proceedings of the ACM SIGMETRICS joint international conference on Measurement and modeling of computer systems

Modern trading and cluster applications require microsecond latencies and almost no losses in data centers. This paper introduces an algorithm called FineComb that can estimate fine-grain end-to-end loss and latency measurements between edge routers in ...
Fine-grained latency and loss measurements in the presence of reordering
Performance evaluation review

Modern trading and cluster applications require microsecond latencies and almost no losses in data centers. This paper introduces an algorithm called FineComb that can estimate fine-grain end-to-end loss and latency measurements between edge routers in ...
Using NetMagic to observe fine-grained per-flow latency measurements
SIGCOMM '11

We introduce NetMagic to demonstrate the efficacy of RLI architecture RLI for the fine-grained per-flow latency measurements. In this demo, the main function of RLI is implemented in NetMagic, which is the key component of our experimental network ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI EA '21: Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems

May 2021

2965 pages

ISBN:9781450380959

DOI:10.1145/3411763

Editors:
Yoshifumi Kitamura
Tohoku University, Japan
,
Aaron Quigley
University of New South Wales, Australia
,
Katherine Isbister
University of California Santa Cruz, USA
,
Takeo Igarashi
The University of Tokyo, Japan

Copyright © 2021 Owner/Author.

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 08 May 2021

Check for updates

Author Tags

Qualifiers

Extended-abstract
Research
Refereed limited

Conference

CHI '21

Sponsor:

SIGCHI

CHI '21: CHI Conference on Human Factors in Computing Systems

May 8 - 13, 2021

Yokohama, Japan

Acceptance Rates

Overall Acceptance Rate 6,164 of 23,696 submissions, 26%

Upcoming Conference

CHI 2025

Sponsor:
sigchi

ACM CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

7
Total Citations
View Citations
318
Total Downloads

Downloads (Last 12 months)71
Downloads (Last 6 weeks)5

Reflects downloads up to 28 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Souchet ALourdeaux DBurkhardt JHancock P(2023)Design guidelines for limiting and eliminating virtual reality-induced symptoms and effects at work: a comprehensive, factor-oriented reviewFrontiers in Psychology10.3389/fpsyg.2023.116193214Online publication date: 9-Jun-2023
https://doi.org/10.3389/fpsyg.2023.1161932
Pelikan HHofstetter E(2023)Managing Delays in Human-Robot InteractionACM Transactions on Computer-Human Interaction10.1145/356989030:4(1-42)Online publication date: 12-Sep-2023
https://dl.acm.org/doi/10.1145/3569890
Döllinger NWolf EBotsch MLatoschik MWienrich C(2023)Are Embodied Avatars Harmful to our Self-Experience? The Impact of Virtual Embodiment on Body AwarenessProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3580918(1-14)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3580918
Mal DWolf EDöllinger NWienrich CLatoschik M(2023)The Impact of Avatar and Environment Congruence on Plausibility, Embodiment, Presence, and the Proteus Effect in Virtual RealityIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.324708929:5(2358-2368)Online publication date: 1-May-2023
https://dl.acm.org/doi/10.1109/TVCG.2023.3247089
Wolf EDöllinger NMal DWenninger SBartl ABotsch MLatoschik MWienrich C(2022)Does distance matter? Embodiment and perception of personalized avatars in relation to the self-observation distance in virtual realityFrontiers in Virtual Reality10.3389/frvir.2022.10310933Online publication date: 21-Dec-2022
https://doi.org/10.3389/frvir.2022.1031093
Wolf EMal DFrohnapfel VDollinger NWenninger SBotsch MLatoschik MWienrich C(2022)Plausibility and Perception of Personalized Virtual Humans between Virtual and Augmented Reality2022 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)10.1109/ISMAR55827.2022.00065(489-498)Online publication date: Oct-2022
https://doi.org/10.1109/ISMAR55827.2022.00065
Misiak MFuhrmann ALatoschik M(2021)Impostor-based Rendering Acceleration for Virtual, Augmented, and Mixed RealityProceedings of the 27th ACM Symposium on Virtual Reality Software and Technology10.1145/3489849.3489865(1-10)Online publication date: 8-Dec-2021
https://dl.acm.org/doi/10.1145/3489849.3489865

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents