research-article

Open access

Compiler-support for Critical Data Persistence in NVM

Authors:

Mohammad Alshboul,

Akihiro Hayashi,

Keiji KimuraAuthors Info & Claims

ACM Transactions on Architecture and Code Optimization (TACO), Volume 16, Issue 4

Article No.: 54, Pages 1 - 25

https://doi.org/10.1145/3371236

Published: 26 December 2019 Publication History

All formats PDF

Abstract

Non-volatile Main Memories (NVMs) offer a promising way to preserve data persistence and enable computation recovery in case of failure. While the use of NVMs can significantly reduce the overhead of failure recovery, which is the case with High-Performance Computing (HPC) kernels, rewriting existing programs or writing new applications for NVMs is non-trivial. In this article, we present a compiler-support that automatically inserts complex instructions into kernels to achieve NVM data-persistence based on a simple programmer directive. Unlike checkpointing techniques that store the whole system state, our technique only persists user-designated objects as well as some parameters required for safe recovery such as loop induction variables. Also, our technique can reduce the number of data transfer operations, because our compiler coalesces consecutive memory-persisting operations into a single memory transaction per cache line when possible.

Our compiler-support is implemented in the LLVM tool-chain and introduces the necessary modifications to loop-intensive computational kernels (e.g., TMM, LU, Gauss, and FFT) to force data persistence. The experiments show that our proposed compiler-support outperforms the most recent checkpointing techniques while its performance overheads are insignificant.

References

[1]

Saad Ahmed, Naveed Anwar Bhatti, Muhammad Hamad Alizai, Junaid Haroon Siddiqui, and Luca Mottola. 2019. Efficient intermittent computing with differential checkpointing. In Proceedings of the 20th ACM SIGPLAN/SIGBED International Conference on Languages, Compilers, and Tools for Embedded Systems (LCTES’19). ACM, New York, NY, 70--81.

Digital Library

[2]

Mohammad Alshboul, Hussein Elnawawy, Reem Elkhouly, Keiji Kimura, James Tuck, and Yan Solihin. 2019. Efficient checkpointing with recompute scheme for non-volatile main memory. ACM Trans. Archit. Code Optim. 16, 2, Article 18 (May 2019), 27 pages.

Digital Library

[3]

M. Alshboul, J. Tuck, and Y. Solihin. 2018. Lazy persistency: A high-performing and write-efficient software persistency technique. In Proceedings of the ACM/IEEE 45th International Symposium on Computer Architecture (ISCA). 439--451.

[4]

Dhruva R. Chakrabarti, Hans-J. Boehm, and Kumud Bhandari. 2014. Atlas: Leveraging locks for non-volatile memory consistency. ACM SIGPLAN Not. 49, 10 (Oct. 2014), 433--452.

Digital Library

[5]

Jongouk Choi, Qingrui Liu, and Changhee Jung. 2019. CoSpec: Compiler directed speculative intermittent computation. In Proceedings of the 52nd IEEE/ACM International Symposium on Microarchitecture (MICRO’52). ACM, New York, NY, 399--412.

Digital Library

[6]

Joel Coburn, Adrian M. Caulfield, Ameen Akel, Laura M. Grupp, Rajesh K. Gupta, Ranjit Jhala, and Steven Swanson. 2012. NV-Heaps: Making persistent objects fast and safe with next-generation, non-volatile memories. ACM SIGPLAN Not. 47, 4 (2012), 105--118.

[7]

Nachshon Cohen, David T. Aksun, Hillel Avni, and James R. Larus. 2019. Fine-grain checkpointing with in-cache-line logging. CoRR abs/1902.00660 (2019). arxiv:1902.00660 http://arxiv.org/abs/1902.00660.

[8]

Jeremy Condit, Edmund B. Nightingale, Christopher Frost, Engin Ipek, Benjamin Lee, Doug Burger, and Derrick Coetzee. 2009. Better I/O through byte-addressable, persistent memory. In Proceedings of the ACM SIGOPS 22nd Symposium on Operating Systems Principles (SOSP’09). ACM, New York, NY, 133--146.

Digital Library

[9]

Intel Corporation. 2016. Intel architecture instruction set extensions programming reference. Technical Report, 319433--030, Intel Corporation, Mountain View, CA.

[10]

Thomas H. Cormen, Charles E. Leiserson, Ronald L. Rivest, and Clifford Stein. 2009. Introduction to Algorithms. The MIT Press.

[11]

Tudor David, Aleksandar Dragojević, Rachid Guerraoui, and Igor Zablotchi. 2018. Log-free concurrent data structures. In Proceedings of the USENIX Annual Technical Conference (USENIX ATC’18). USENIX Association, 373--386. https://www.usenix.org/conference/atc18/presentation/david.

[12]

Linda DeMichiel and Michael Keith. 2006. Java persistence API. Presented in JavaOne Conference. https://docs.huihoo.com/javaone/2006/java-ee/ts-3395.pdf.

[13]

Joel E. Denny, Seyong Lee, and Jeffrey S. Vetter. 2016. NVL-C: Static analysis techniques for efficient, correct programming of non-volatile main memory systems. In Proceedings of the IEEE International Symposium on High Performance Distributed Computing (HPDC’16). ACM, 125--136.

[14]

J. E. Denny, S. Lee, and J. S. Vetter. 2017. Language-based optimizations for persistence on nonvolatile main memory systems. In Proceedings of the IEEE International Parallel and Distributed Processing Symposium (IPDPS’17). 1163--1173.

[15]

Moslem Didehban, Sai Ram Dheeraj Lokam, and Aviral Shrivastava. 2017. InCheck: An in-application recovery scheme for soft errors. In Proceedings of the 54th Design Automation Conference (DAC’17). ACM, New York, NY, Article 40, 6 pages.

Digital Library

[16]

Subramanya R. Dulloor, Sanjay Kumar, Anil Keshavamurthy, Philip Lantz, Dheeraj Reddy, Rajesh Sankaran, and Jeff Jackson. 2014. System software for persistent memory. In Proceedings of the 9th European Conference on Computer Systems (EuroSys’14). ACM, New York, NY, Article 15, 15 pages.

Digital Library

[17]

H. Elnawawy, M. Alshboul, J. Tuck, and Y. Solihin. 2017. Efficient checkpointing of loop-based codes for non-volatile main memory. In Proceedings of the 26th International Conference on Parallel Architectures and Compilation Techniques (PACT’17). 318--329.

[18]

Jian Huang, Karsten Schwan, and Moinuddin K. Qureshi. 2014. NVRAM-aware logging in transaction systems. Proc. VLDB Endow. 8, 4 (Dec. 2014), 389--400.

Digital Library

[19]

Yingchao Huang, Kai Wu, and Dong Li. 2017. High performance data persistence in non-volatile memory for resilient high performance computing. CoRR abs/1705.00264 (2017). arxiv:1705.00264 http://arxiv.org/abs/1705.00264.

[20]

Intel. 2016. Persistent collections for Java. https://github.com/pmem/pcj.

[21]

Joseph Izraelevitz, Terence Kelly, and Aasheesh Kolli. 2016. Failure-atomic persistent memory updates via JUSTDO logging. ACM SIGPLAN Not. 51, 4 (Mar. 2016), 427--442.

Digital Library

[22]

J. Jeong, C. H. Park, J. Huh, and S. Maeng. 2018. Efficient hardware-assisted logging with asynchronous and direct-update for persistent memory. In Proceedings of the 51st IEEE/ACM International Symposium on Microarchitecture (MICRO’18). 520--532.

[23]

Marcel Köppen, Jana Traue, Christoph Borchert, Jörg Nolte, and Olaf Spinczyk. 2019. Cache-line transactions: Building blocks for persistent kernel data structures enabled by AspectC++. In Proceedings of the 10th Workshop on Programming Languages and Operating Systems (PLOS’19). ACM, New York, NY, 38--44.

Digital Library

[24]

Chris Lattner and Vikram Adve. 2004. LLVM: A compilation framework for lifelong program analysis 8 transformation. In Proceedings of the International Symposium on Code Generation and Optimization: Feedback-directed and Runtime Optimization. IEEE Computer Society, 75.

[25]

Q. Liu, J. Izraelevitz, S. K. Lee, M. L. Scott, S. H. Noh, and C. Jung. 2018. iDO: Compiler-directed failure atomicity for nonvolatile memory. In Proceedings of the 51st IEEE/ACM International Symposium on Microarchitecture (MICRO’18). 258--270.

[26]

Qingrui Liu, Changhee Jung, Dongyoon Lee, and Devesh Tiwari. 2015. Clover: Compiler directed lightweight soft error resilience. ACM SIGPLAN Not. 50, 5, Article 2 (June 2015), 10 pages.

Digital Library

[27]

Sihang Liu, Korakit Seemakhupt, Gennady Pekhimenko, Aasheesh Kolli, and Samira Khan. 2019. Janus: Optimizing memory and storage support for non-volatile memory systems. In Proceedings of the 46th International Symposium on Computer Architecture (ISCA’19). ACM, New York, NY, 143--156.

Digital Library

[28]

C. D. Martino, W. Kramer, Z. Kalbarczyk, and R. Iyer. 2015. Measuring and understanding extreme-scale application resilience: A field study of 5,000,000 HPC application runs. In Proceedings of the 45th IEEE/IFIP International Conference on Dependable Systems and Networks. 25--36.

[29]

A. Memaripour and S. Swanson. 2018. Breeze: User-level access to non-volatile main memories for legacy software. In Proceedings of the IEEE 36th International Conference on Computer Design (ICCD’18). 413--422.

[30]

T. Nguyen and D. Wentzlaff. 2018. PiCL: A software-transparent, persistent cache log for nonvolatile main memory. In Proceedings of the 51st IEEE/ACM International Symposium on Microarchitecture (MICRO’18). 507--519.

[31]

David Patterson, Aaron Brown, Pete Broadwell, George Candea, Mike Chen, James Cutler, Patricia Enriquez, Armando Fox, Emre Kiciman, Matthew Merzbacher et al. 2002. Recovery-oriented Computing (ROC): Motivation, Definition, Techniques, and Case Studies. Technical Report. Technical Report UCB//CSD-02-1175, UC Berkeley Computer Science.

[32]

Steven Pelley, Peter M. Chen, and Thomas F. Wenisch. 2014. Memory persistency. SIGARCH Comput. Archit. News 42, 3 (June 2014), 265--276.

Digital Library

[33]

J. S. Plank, Kai Li, and M. A. Puening. 1998. Diskless checkpointing. IEEE Trans. Parallel Distrib. Syst. 9, 10 (Oct. 1998), 972--986.

Digital Library

[34]

Georgios Psaropoulos, Ismail Oukid, Thomas Legler, Norman May, and Anastasia Ailamaki. 2019. Bridging the latency gap between NVM and DRAM for latency-bound operations. In Proceedings of the 15th International Workshop on Data Management on New Hardware. 8.

Digital Library

[35]

George A. Reis, Jonathan Chang, and David I. August. 2007. Automatic instruction-level software-only recovery. IEEE Micro 27, 1 (Jan. 2007), 36--47.

Digital Library

[36]

Jinglei Ren, Qingda Hu, Samira Khan, and Thomas Moscibroda. 2017. Programming for non-volatile main memory is hard. In Proceedings of the 8th Asia-Pacific Workshop on Systems. ACM, 13.

Digital Library

[37]

Jie Ren, Kai Wu, and Dong Li. 2018. Understanding application recomputability without crash consistency in non-volatile memory. In Proceedings of the Workshop on Memory Centric High Performance Computing (MCHPC’18). ACM, New York, NY, 27--36.

Digital Library

[38]

J. Ren, J. Zhao, S. Khan, J. Choi, Y. Wu, and O. Mutiu. 2015. ThyNVM: Enabling software-transparent crash consistency in persistent memory systems. In Proceedings of the 48th IEEE/ACM International Symposium on Microarchitecture (MICRO’15). 672--685.

[39]

Bianca Schroeder and Garth A. Gibson. 2007. Understanding failures in petascale computers. J. Phys.: Conf. Series 78 (July 2007), 012022.

[40]

F. Shahzad, J. Thies, M. Kreutzer, T. Zeiser, G. Hager, and G. Wellein. 2019. CRAFT: A library for easier application-level checkpoint/restart and automatic fault tolerance. IEEE Trans. Parallel Distrib. Syst. 30, 3 (Mar. 2019), 501--514.

[41]

Seunghee Shin, Satish Kumar Tirukkovalluri, James Tuck, and Yan Solihin. 2017. Proteus: A flexible and fast software supported hardware logging approach for NVM. In Proceedings of the 50th IEEE/ACM International Symposium on Microarchitecture (MICRO’17). ACM, New York, NY, 178--190.

Digital Library

[42]

Thomas Shull, Jian Huang, and Josep Torrellas. 2019. AutoPersist: An easy-to-use Java NVM framework based on reachability. In Proceedings of the 40th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI’19). ACM, New York, NY, 316--332.

Digital Library

[43]

Thomas Shull, Jian Huang, and Josep Torrellas. 2019. QuickCheck: Using speculation to reduce the overhead of checks in NVM frameworks. In Proceedings of the 15th ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments (VEE’19). ACM, New York, NY, 137--151.

Digital Library

[44]

Daniel J. Sorin, Milo M. K. Martin, Mark D. Hill, and David A. Wood. 2002. SafetyNet: Improving the availability of shared memory multiprocessors with global checkpoint/recovery. SIGARCH Comput. Archit. News 30, 2 (May 2002), 123--134.

Digital Library

[45]

Willi-Hans Steeb. 1999. The Nonlinear Workbook: Chaos, Fractals, Cellular Automata, Neural Networks, Genetic Algorithms, Fuzzy Logic with C++, Java, Symbolic C++ and Reduce Programs. World Scientific Publishing Company.

[46]

TensorFlow. 2019. Multi-Level Intermediate Representation. https://github.com/tensorflow/mlir.

[47]

D. Tiwari, S. Gupta, and S. S. Vazhkudai. 2014. Lazy checkpointing: Exploiting temporal locality in failures to mitigate checkpointing overheads on extreme-scale systems. In Proceedings of the 44th IEEE/IFIP International Conference on Dependable Systems and Networks. 25--36.

[48]

N. H. Vaidya. 1997. Impact of checkpoint latency on overhead ratio of a checkpointing scheme. IEEE Trans. Comput. 46, 8 (Aug. 1997), 942--947.

Digital Library

[49]

Haris Volos, Andres Jaan Tack, and Michael M. Swift. 2011. Mnemosyne: Lightweight persistent memory. ACM SIGPLAN Not. 47, 4 (Mar. 2011), 91--104.

[50]

Chundong Wang, Sudipta Chattopadhyay, and Gunavaran Brihadiswarn. 2019. Crash recoverable ARMv8-oriented B+-tree for byte-addressable persistent memory. In Proceedings of the 20th ACM SIGPLAN/SIGBED International Conference on Languages, Compilers, and Tools for Embedded Systems (LCTES’19). ACM, New York, NY, 33--44.

Digital Library

[51]

Michael E. Wolf and Monica S. Lam. 1991. A data locality optimizing algorithm. In ACM SIGPLAN Not., Vol. 26. ACM, 30--44.

[52]

Mingyu Wu, Ziming Zhao, Haoyu Li, Heting Li, Haibo Chen, Binyu Zang, and Haibing Guan. 2018. Espresso: Brewing Java for more non-volatility with non-volatile memory. ACM SIGPLAN Not. 53, 2 (Mar. 2018), 70--83.

Digital Library

[53]

Song Wu, Fang Zhou, Xiang Gao, Hai Jin, and Jinglei Ren. 2019. Dual-page checkpointing: An architectural approach to efficient data persistence for in-memory applications. ACM Trans. Archit. Code Optim. 15, 4, Article 57 (Jan. 2019), 27 pages.

Digital Library

[54]

M. Ye, K. Zubair, A. Mohaisen, and A. Awad. 2019. Towards low-cost mechanisms to enable restoration of encrypted non-volatile memories. IEEE Trans. Depend. Sec. Comput. (2019), 1--1.

[55]

Litong You, Hao Xu, Qipeng Zhang, Tianyou Li, Chen Li, Yuting Chen, and Linpeng Huang. 2019. JDap: Supporting in-memory data persistence in JavaScript using Intel’s PMDK. J. Syst. Archit. 101 (2019), 101662.

Digital Library

[56]

Lu Zhang and Steven Swanson. 2019. Pangolin: A fault-tolerant persistent memory programming library. In Proceedings of the USENIX Annual Technical Conference (USENIX ATC’19). USENIX Association, Berkeley, CA, 897--911. http://dl.acm.org/citation.cfm?id=3358807.3358884.

[57]

M. Zhang, X. Yao, and C. Wang. 2018. NVCL: Exploiting NVRAM in cache-line granularity differential logging. In Proceedings of the IEEE 7th Non-Volatile Memory Systems and Applications Symposium (NVMSA’18). 37--42.

[58]

Pengfei Zuo, Yu Hua, and Yuan Xie. 2019. SuperMem: Enabling application-transparent secure persistent memory with low overheads. In Proceedings of the 52nd IEEE/ACM International Symposium on Microarchitecture (MICRO’19). ACM, New York, NY, 479--492.

Digital Library

Cited By

Bansal S(2023)StaticPersist: Compiler Support for PMEM ProgrammingVerification, Model Checking, and Abstract Interpretation10.1007/978-3-031-24950-1_3(44-65)Online publication date: 16-Jan-2023
https://dl.acm.org/doi/10.1007/978-3-031-24950-1_3
Köppen MFriesel DBorchert CSpinczyk O(2021)Decoupling Application Logic from Persistent Memory Frameworks with AspectC++Proceedings of the 11th Workshop on Programming Languages and Operating Systems10.1145/3477113.3487269(24-30)Online publication date: 25-Oct-2021
https://dl.acm.org/doi/10.1145/3477113.3487269
Wang HTang XSheng LLi WShen JTao J(2021)scList: A PCRAM-based Hybrid Memory Management Scheme2021 IEEE 19th International Conference on Embedded and Ubiquitous Computing (EUC)10.1109/EUC53437.2021.00010(1-7)Online publication date: Oct-2021
https://doi.org/10.1109/EUC53437.2021.00010

Index Terms

Compiler-support for Critical Data Persistence in NVM
1. Hardware
  1. Emerging technologies
    1. Analysis and design of emerging devices and systems
      1. Emerging languages and compilers
2. Software and its engineering
  1. Software notations and tools
    1. Compilers
      1. Runtime environments
      2. Source code generation
  2. Software organization and properties
    1. Contextual software domains
      1. Operating systems
        Memory management
        Main memory

Recommendations

Designing a secure DRAM+NVM hybrid memory module
CF '19: Proceedings of the 16th ACM International Conference on Computing Frontiers

Non-Volatile Memory (NVM) such as PCM has emerged as a potential alternative for main memory due to its high density and low leakage power. However, an NVM main-memory system faces three challenges when compared to Dynamic Random Access Memory (DRAM) - ...
DEUCE: Write-Efficient Encryption for Non-Volatile Memories
ASPLOS'15

Phase Change Memory (PCM) is an emerging Non Volatile Memory (NVM) technology that has the potential to provide scalable high-density memory systems. While the non-volatility of PCM is a desirable property in order to save leakage power, it also has the ...
Energy Aware Persistence: Reducing Energy Overheads of Memory-based Persistence in NVMs
PACT '16: Proceedings of the 2016 International Conference on Parallel Architectures and Compilation

Next generation byte addressable nonvolatile memories (NVMs) such as PCM, Memristor, and 3D X-Point are attractive solutions for mobile and other end-user devices, as they offer memory scalability as well as fast persistent storage. However, NVM's ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Transactions on Architecture and Code Optimization

ACM Transactions on Architecture and Code Optimization Volume 16, Issue 4

December 2019

572 pages

ISSN:1544-3566

EISSN:1544-3973

DOI:10.1145/3366460

Editor:
Koen De Bosschere
Ghent University, Belgium

Issue’s Table of Contents

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 26 December 2019

Accepted: 01 November 2019

Revised: 01 October 2019

Received: 01 June 2019

Published in TACO Volume 16, Issue 4

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
1,135
Total Downloads

Downloads (Last 12 months)171
Downloads (Last 6 weeks)27

Reflects downloads up to 25 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Bansal S(2023)StaticPersist: Compiler Support for PMEM ProgrammingVerification, Model Checking, and Abstract Interpretation10.1007/978-3-031-24950-1_3(44-65)Online publication date: 16-Jan-2023
https://dl.acm.org/doi/10.1007/978-3-031-24950-1_3
Köppen MFriesel DBorchert CSpinczyk O(2021)Decoupling Application Logic from Persistent Memory Frameworks with AspectC++Proceedings of the 11th Workshop on Programming Languages and Operating Systems10.1145/3477113.3487269(24-30)Online publication date: 25-Oct-2021
https://dl.acm.org/doi/10.1145/3477113.3487269
Wang HTang XSheng LLi WShen JTao J(2021)scList: A PCRAM-based Hybrid Memory Management Scheme2021 IEEE 19th International Conference on Embedded and Ubiquitous Computing (EUC)10.1109/EUC53437.2021.00010(1-7)Online publication date: Oct-2021
https://doi.org/10.1109/EUC53437.2021.00010

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

Media

Figures

Other

Tables

View Issue’s Table of Contents