Integrating Non-Volatile Main Memory in a Deterministic Database
Pages 672 - 686
Abstract
Deterministic databases provide strong serializability while avoiding concurrency-control related aborts by establishing a serial ordering of transactions before their execution. Recent work has shown that they can also handle skewed and contended workloads effectively. These properties are achieved by batching transactions in epochs and then executing the transactions within an epoch concurrently and deterministically. However, the predetermined serial ordering of transactions makes these databases more vulnerable to long-latency transactions. As a result, they have mainly been designed as main-memory databases, which limits the size of the datasets that can be supported.
We show how to integrate non-volatile main memory (NVMM) into deterministic databases to support larger data-sets at a lower cost per gigabyte and faster failure recovery. We describe a novel dual-version checkpointing scheme that takes advantage of deterministic execution, epoch-based processing and NVMM's byte addressability to avoid persisting all updates to NVMM. Our approach reduces NVMM accesses, provides better access locality, and reduces garbage collection costs, thus lowering the performance impact of using NVMM. We show that our design enables scaling the dataset size while reducing the impacts of using NVMM, achieving up to 79% of DRAM performance. Our design supports efficient failure recovery and outperforms alternative failure recovery designs, especially under contended workloads, by up to 56%.
References
[1]
Mohammad Alomari, Michael Cahill, Alan Fekete, and Uwe Rohm. 2008. The Cost of Serializability on Platforms That Use Snapshot Isolation. In 2008 IEEE 24th International Conference on Data Engineering. 576--585.
[2]
Joy Arulraj, Matthew Perron, and Andrew Pavlo. 2016. Write-behind Logging. Proc. VLDB Endow. 10, 4 (Nov. 2016), 337--348.
[3]
Brian F. Cooper, Adam Silberstein, Erwin Tam, Raghu Ramakrishnan, and Russell Sears. 2010. Benchmarking Cloud Serving Systems with YCSB. In Proceedings of the 1st ACM Symposium on Cloud Computing (Indianapolis, Indiana, USA) (SoCC '10). Association for Computing Machinery, New York, NY, USA, 143--154.
[4]
James Cowling and Barbara Liskov. 2012. Granola: Low-Overhead Distributed Transaction Coordination. In Proceedings of the 2012 USENIX Annual Technical Conference (Boston, MA) (USENIX ATC'12). USENIX Association, USA, 21--33.
[5]
Jose M. Faleiro and Daniel J. Abadi. 2015. Rethinking Serializable Multiversion Concurrency Control. Proc. VLDB Endow. 8, 11 (July 2015), 1190--1201.
[6]
Jose M. Faleiro, Daniel J. Abadi, and Joseph M. Hellerstein. 2017. High Performance Transactions via Early Write Visibility. Proc. VLDB Endow. 10, 5 (Jan. 2017), 613--624.
[7]
Michael Haubenschild, Caetano Sauer, Thomas Neumann, and Viktor Leis. 2020. Rethinking Logging, Checkpoints, and Recovery for High-Performance Storage Engines. In Proceedings of the 2020 ACM SIGMOD International Conference on Management of Data (Portland, OR, USA) (SIGMOD '20). Association for Computing Machinery, New York, NY, USA, 877--892.
[8]
Yihe Huang, William Qian, Eddie Kohler, Barbara Liskov, and Liuba Shrira. 2020. Opportunities for Optimism in Contended Main-Memory Multicore Transactions. Proc. VLDB Endow. 13, 5 (Jan. 2020), 629--642.
[9]
Deukyeon Hwang, Wook-Hee Kim, Youjip Won, and Beomseok Nam. 2018. Endurable Transient Inconsistency in Byte-Addressable Persistent B+-Tree. In 16th USENIX Conference on File and Storage Technologies (FAST 18). USENIX Association, Oakland, CA, 187--200. https://www.usenix.org/conference/fast18/presentation/hwang
[10]
Robert Kallman, Hideaki Kimura, Jonathan Natkins, Andrew Pavlo, Alexander Rasin, Stanley Zdonik, Evan P. C. Jones, Samuel Madden, Michael Stonebraker, Yang Zhang, John Hugg, and Daniel J. Abadi. 2008. H-Store: A High-Performance, Distributed Main Memory Transaction Processing System. Proc. VLDB Endow. 1, 2 (Aug. 2008), 1496--1499.
[11]
Se Kwon Lee, Jayashree Mohan, Sanidhya Kashyap, Taesoo Kim, and Vijay Chidambaram. 2019. Recipe: Converting Concurrent DRAM Indexes to Persistent-Memory Indexes. In Proceedings of the 27th ACM Symposium on Operating Systems Principles (Huntsville, Ontario, Canada) (SOSP '19). Association for Computing Machinery, New York, NY, USA, 462--477.
[12]
Viktor Leis, Michael Haubenschild, Alfons Kemper, and Thomas Neumann. 2018. LeanStore: In-Memory Data Management beyond Main Memory. In 2018 IEEE 34th International Conference on Data Engineering (ICDE). 185--196.
[13]
Yu-Shan Lin, Shao-Kan Pi, Meng-Kai Liao, Ching Tsai, Aaron Elmore, and Shan-Hung Wu. 2019. MgCrab: Transaction Crabbing for Live Migration in Deterministic Database Systems. Proc. VLDB Endow. 12, 5 (Jan. 2019), 597--610.
[14]
Gang Liu, Leying Chen, and Shimin Chen. 2021. Zen: A High-Throughput Log-Free OLTP Engine for Non-Volatile Main Memory. Proc. VLDB Endow. 14, 5 (Jan. 2021), 835--848.
[15]
Baotong Lu, Xiangpeng Hao, Tianzheng Wang, and Eric Lo. 2020. Dash: Scalable Hashing on Persistent Memory. Proc. VLDB Endow. 13, 8 (April 2020), 1147--1161.
[16]
Yi Lu, Xiangyao Yu, Lei Cao, and Samuel Madden. 2020. Aria: A Fast and Practical Deterministic OLTP Database. Proc. VLDB Endow. 13, 12 (July 2020), 2047--2060.
[17]
Yi Lu, Xiangyao Yu, Lei Cao, and Samuel Madden. 2021. Epoch-Based Commit and Replication in Distributed OLTP Databases. Proc. VLDB Endow. 14, 5 (Jan. 2021), 743--756.
[18]
Senthil Nathan, Chander Govindarajan, Adarsh Saraf, Manish Sethi, and Praveen Jayachandran. 2019. Blockchain Meets Database: Design and Implementation of a Blockchain Relational Database. Proc. VLDB Endow. 12, 11 (July 2019), 1539--1552.
[19]
Ismail Oukid, Johan Lasperas, Anisoara Nica, Thomas Willhalm, and Wolfgang Lehner. 2016. FPTree: A Hybrid SCM-DRAM Persistent and Concurrent B-Tree for Storage Class Memory. In Proceedings of the 2016 International Conference on Management of Data (San Francisco, California, USA) (SIGMOD '16). Association for Computing Machinery, New York, NY, USA, 371--386.
[20]
Dai Qin, Angela Demke Brown, and Ashvin Goel. 2021. Caracal: Contention Management with Deterministic Concurrency Control. In Proceedings of the ACM SIGOPS 28th Symposium on Operating Systems Principles (Virtual Event, Germany) (SOSP '21). Association for Computing Machinery, New York, NY, USA, 180--194.
[21]
Kun Ren, Dennis Li, and Daniel J. Abadi. 2019. SLOG: Serializable, Low-Latency, Geo-Replicated Transactions. Proc. VLDB Endow. 12, 11 (July 2019), 1747--1761.
[22]
Alexander Thomson and Daniel J. Abadi. 2010. The Case for Determinism in Database Systems. Proc. VLDB Endow. 3, 1--2 (Sept. 2010), 70--80.
[23]
Alexander Thomson, Thaddeus Diamond, Shu-Chun Weng, Kun Ren, Philip Shao, and Daniel J. Abadi. 2012. Calvin: Fast Distributed Transactions for Partitioned Database Systems. In Proceedings of the 2012 ACM SIGMOD International Conference on Management of Data (Scottsdale, Arizona, USA) (SIGMOD '12). Association for Computing Machinery, New York, NY, USA, 1--12.
[24]
Transaction Processing Performance Council (TPC). 2010. TPC Benchmark C Standard Specification, Revision 5.11.
[25]
Stephen Tu, Wenting Zheng, Eddie Kohler, Barbara Liskov, and Samuel Madden. 2013. Speedy Transactions in Multicore In-Memory Databases. In Proceedings of the Twenty-Fourth ACM Symposium on Operating Systems Principles (Farminton, Pennsylvania) (SOSP '13). Association for Computing Machinery, New York, NY, USA, 18--32.
[26]
Alexander van Renen, Viktor Leis, Alfons Kemper, Thomas Neumann, Takushi Hashida, Kazuichi Oe, Yoshiyasu Doi, Lilian Harada, and Mitsuru Sato. 2018. Managing Non-Volatile Memory in Database Systems. In Proceedings of the 2018 International Conference on Management of Data (Houston, TX, USA) (SIGMOD '18). Association for Computing Machinery, New York, NY, USA, 1541--1555.
[27]
Xinjing Zhou, Joy Arulraj, Andrew Pavlo, and David Cohen. 2021. Spitfire: A Three-Tier Buffer Manager for Volatile and Non-Volatile Memory. In Proceedings of the 2021 International Conference on Management of Data (Virtual Event, China) (SIGMOD/PODS '21). Association for Computing Machinery, New York, NY, USA, 2195--2207.
[28]
Xinjing Zhou, Lidan Shou, Ke Chen, Wei Hu, and Gang Chen. 2019. DPTree: Differential Indexing for Persistent Memory. Proc. VLDB Endow. 13, 4 (Dec. 2019), 421--434.
Index Terms
- Integrating Non-Volatile Main Memory in a Deterministic Database
Recommendations
Redesign the Memory Allocator for Non-Volatile Main Memory
Special Issue on Hardware and Algorithms for Learning On-a-chip and Special Issue on Alternative Computing SystemsThe non-volatile memory (NVM) has the merits of byte-addressability, fast speed, persistency and low power consumption, which make it attractive to be used as main memory. Commonly, user process dynamically acquires memory through memory allocators. ...
Managing Non-Volatile Memory in Database Systems
SIGMOD '18: Proceedings of the 2018 International Conference on Management of DataNon-volatile memory (NVM) is a new storage technology that combines the performance and byte addressability of DRAM with the persistence of traditional storage devices like flash (SSD). While these properties make NVM highly promising, it is not yet ...
Preliminary Analysis of a Write Reduction Method for Non-volatile Main Memory on Jikes RVM
CANDAR '13: Proceedings of the 2013 First International Symposium on Computing and NetworkingThere are many researches and developments on byte accessible non-volatile memory (NVM). Byte accessible NVM can be used as not only storage but also main memory. Non-volatile main memory has two significant merits: reducing power consumption of main ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
May 2023
910 pages
ISBN:9781450394871
DOI:10.1145/3552326
- General Co-chairs:
- Giuseppe Antonio Di Luna,
- Leonardo Querzoni,
- Program Co-chairs:
- Alexandra Fedorova,
- Dushyanth Narayanan
Copyright © 2023 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 08 May 2023
Check for updates
Badges
Author Tags
Qualifiers
- Research-article
Conference
EuroSys '23
Sponsor:
Acceptance Rates
Overall Acceptance Rate 241 of 1,308 submissions, 18%
Upcoming Conference
EuroSys '25
- Sponsor:
- sigops
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 348Total Downloads
- Downloads (Last 12 months)150
- Downloads (Last 6 weeks)14
Reflects downloads up to 18 Nov 2024
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in