research-article

Open access

Systems research is running out of time

Authors:

Michael WeiAuthors Info & Claims

HotOS '21: Proceedings of the Workshop on Hot Topics in Operating Systems

Pages 65 - 71

https://doi.org/10.1145/3458336.3465293

Published: 03 June 2021 Publication History

Abstract

Most sciences conduct experiments with a thorough understanding of the accuracy and precision of the instruments used for making measurements. Time is the most frequently used measurement in systems research, yet most of the literature does not consider the precision and accuracy of clocks. In this paper, we argue for the importance of understanding timekeeping and providing precise and accurate time for general systems research.

References

[1]

[2/3] x86/tsc: Fix erroneous TSC rate on skylake xeon. https://lore.kernel.org/patchwork/patch/866472/.

[2]

c15, cycle counter register (CCNT). https://developer.arm.com/documentation/ddi0360/f/control-coprocessor-cp15/register-descriptions/c15--cycle-counter-register-ccnt-.

[3]

chrony. https://chrony.tuxfamily.org.

[4]

excessive system clock drift? (2+ minutes per hour). https://askubuntu.com/questions/1014285/excessive-system-clock-drift-2-minutes-per-hour.

[5]

Getting high precision timing on android. https://www.gamasutra.com/view/feature/171774/getting_high_precision_timing_on_.php?print=1.

[6]

The HPET bug: What it is and what it isn't. https://www.overclockers.at/articles/the-hpet-bug-what-it-is-and-what-it-isnt.

[7]

Ieee 1588-2019 - ieee standard for a precision clock synchronization protocol for networked measurement and control systems. https://standards.ieee.org/standard/1588-2019.html.

[8]

The linux kernel disabling HPET for intel coffee lake. https://www.phoronix.com/scan.php?page=news_item&px=Linux-Disabling-HPET-CoffeeLake.

[9]

Mdn docs: performance.now(). https://developer.mozilla.org/en-US/docs/Web/API/Performance/now.

[10]

native_calibrate_tsc(): possibly incorrect tsc frequency on newest intel skylake-x cpus, i7-7820x in particular. https://bugzilla.kernel.org/show_bug.cgi?id=197843.

[11]

ntpd - network time protocol (ntp) daemon. http://doc.ntp.org/4.1.0/ntpd.htm.

[12]

[patch 2/2 v2] x86/tsc: Add additional intel cpu models to crystal_khz whitelist. https://lore.kernel.org/lkml/[email protected]/.

[13]

[patch] x86/tsc: Don't use cpuid 0x16 leaf to determine cpu speed. https://lkml.org/lkml/2019/12/6/492.

[14]

[patch] x86/tsc: Don't use cpuid 0x16 leaf to determine cpu speed. https://lkml.org/lkml/2019/12/5/670.

[15]

performance.now(). https://developer.mozilla.org/en-US/docs/Web/API/Performance/now.

[16]

Pitfalls of tsc usage. http://oliveryang.net/2015/09/pitfalls-of-TSC-usage/.

[17]

Pps - pulse per second. https://www.kernel.org/doc/html/latest/driver-api/pps.html.

[18]

Queryperformancecounter function (profileapi.h). https://docs.microsoft.com/en-us/windows/win32/api/profileapi/nf-profileapi-queryperformancecounter.

[19]

Re: [patch] x86: Export tsc related information in sysfs. https://lwn.net/Articles/388286/.

[20]

Running a database on ec2? your clock could be slowing you down. https://heap.io/blog/engineering/clocksource-aws-ec2-vdso.

[21]

Smd microprocessor crystal. https://www.mouser.com/datasheet/2/3/ABLS7M2-1774905.pdf.

[22]

Supermicro FAQ 21337. https://www.supermicro.com/support/faqs/faq.cfm?faq=21337.

[23]

Supermicro ultra. https://www.supermicro.com/en/products/ultra/.

[24]

A tsc_freq_khz driver for everyone. https://github.com/trailofbits/tsc_freq_khz.

[25]

A timely discovery: Examining our amd 2nd gen ryzen results. https://www.anandtech.com/show/12678/a-timely-discovery-examining-amd-2nd-gen-ryzen-results.

[26]

TSC frequency for all: Better profiling and benchmarking. https://blog.trailofbits.com/2019/10/03/tsc-frequency-for-all-better-profiling-and-benchmarking/.

[27]

Charlie Curtsinger and Emery D. Berger. Stabilizer: Statistically sound performance evaluation. In Proceedings of the Eighteenth International Conference on Architectural Support for Programming Languages and Operating Systems, ASPLOS '13, page 219--228, New York, NY, USA, 2013. Association for Computing Machinery.

Digital Library

[28]

Yilong Geng, Shiyu Liu, Zi Yin, Ashish Naik, Balaji Prabhakar, Mendel Rosenblum, and Amin Vahdat. Exploiting a natural network effect for scalable, fine-grained clock synchronization. In 15th {USENIX} Symposium on Networked Systems Design and Implementation ({NSDI} 18), pages 81--94, 2018.

Digital Library

[29]

Tomas Kalibera and Richard Jones. Quantifying performance changes with effect size confidence intervals. arXiv preprint arXiv:2007.10899, 2020.

[30]

Paul Kocher, Jann Horn, Anders Fogh, Daniel Genkin, Daniel Gruss, Werner Haas, Mike Hamburg, Moritz Lipp, Stefan Mangard, Thomas Prescher, et al. Spectre attacks: Exploiting speculative execution. In 2019 IEEE Symposium on Security and Privacy (SP), pages 1--19. IEEE, 2019.

[31]

Yuliang Li, Gautam Kumar, Hema Hariharan, Hassan Wassel, Peter Hochschild, Dave Platt, Simon Sabato, Minlan Yu, Nandita Dukkipati, Prashant Chandra, et al. Sundial: Fault-tolerant clock synchronization for datacenters. In 14th {USENIX} Symposium on Operating Systems Design and Implementation ({OSDI} 20), pages 1171--1186, 2020.

[32]

Erik Van Der Kouwe, Gernot Heiser, Dennis Andriesse, Herbert Bos, and Cristiano Giuffrida. Benchmarking flaws undermine security research. IEEE Security & Privacy, 18(3):48--57, 2020.

[33]

Vincent M Weaver. Advanced hardware profiling and sampling (pebs, ibs, etc.): creating a new papi sampling interface. Technical report, Technical Report UMAINE-VMWTR-PEBS-IBS-SAMPLING-2016-08. University of Maine, 2016.

Cited By

Clausecker RLemire D(2023)Transcoding unicode characters with AVX‐512 instructionsSoftware: Practice and Experience10.1002/spe.326153:12(2430-2462)Online publication date: 12-Sep-2023
https://doi.org/10.1002/spe.3261
Scheuner JBertilsson MGronqvist OTao HLagergren HSteghofer JLeitner P(2022)TriggerBench: A Performance Benchmark for Serverless Function Triggers2022 IEEE International Conference on Cloud Engineering (IC2E)10.1109/IC2E55432.2022.00018(96-103)Online publication date: Sep-2022
https://doi.org/10.1109/IC2E55432.2022.00018

Index Terms

Systems research is running out of time

Recommendations

Research on High Precision Time Synchronization of Android Based on NTP
ICIIP '19: Proceedings of the 4th International Conference on Intelligent Information Processing

Network Time Protocol (NTP) is one of the key technologies for time synchronization of mobile devices. It is very necessary for its research on Android platform. This paper designs an Android time synchronization system based on NTP for the requirement ...
Operations research
Engineering the System of Healthcare Delivery

In Evita, Andrew Lloyd Webber and Tim Rice wrote: Politics, the Art of the Possible. To those of us in the operations research community, we postulate: Operations Research, the Science of Better - (i.e. better processes, better systems and better ...
Towards reproducibility in recommender-systems research

Numerous recommendation approaches are in use today. However, comparing their effectiveness is a challenging task because evaluation results are rarely reproducible. In this article, we examine the challenge of reproducibility in recommender-system ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

HotOS '21: Proceedings of the Workshop on Hot Topics in Operating Systems

June 2021

251 pages

ISBN:9781450384384

DOI:10.1145/3458336

General Chair:
Sebastian Angel
University of Pennsylvania
,
Program Chairs:
Baris Kasikci
University of Michigan
,
Eddie Kohler
Harvard University

Copyright © 2021 Owner/Author.

This work is licensed under a Creative Commons Attribution International 4.0 License.

Sponsors

SIGOPS: ACM Special Interest Group on Operating Systems

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 03 June 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article

Conference

HotOS '21

Sponsor:

SIGOPS

HotOS '21: Workshop on Hot Topics in Operating Systems

June 1 - 3, 2021

Michigan, Ann Arbor

Upcoming Conference

HOTOS '25

Sponsor:
sigops

Workshop on Hot Topics in Operating Systems

May 14 - 16, 2025

Banff or Lake Louise , AB , Canada

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
445
Total Downloads

Downloads (Last 12 months)88
Downloads (Last 6 weeks)16

Reflects downloads up to 20 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Clausecker RLemire D(2023)Transcoding unicode characters with AVX‐512 instructionsSoftware: Practice and Experience10.1002/spe.326153:12(2430-2462)Online publication date: 12-Sep-2023
https://doi.org/10.1002/spe.3261
Scheuner JBertilsson MGronqvist OTao HLagergren HSteghofer JLeitner P(2022)TriggerBench: A Performance Benchmark for Serverless Function Triggers2022 IEEE International Conference on Cloud Engineering (IC2E)10.1109/IC2E55432.2022.00018(96-103)Online publication date: Sep-2022
https://doi.org/10.1109/IC2E55432.2022.00018

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents