Nothing Special   »   [go: up one dir, main page]
Skip to main content
Springer Nature Link
Log in

Resource abstraction and data placement for distributed hybrid memory pool

  • Research Article
  • Published:
Frontiers of Computer Science Aims and scope Submit manuscript

    We’re sorry, something doesn't seem to be working properly.

    Please try refreshing the page. If that doesn't work, please contact support so we can address the problem.

Abstract

Emerging byte-addressable non-volatile memory (NVM) technologies offer higher density and lower cost than DRAM, at the expense of lower performance and limited write endurance. There have been many studies on hybrid NVM/DRAM memory management in a single physical server. However, it is still an open problem on how to manage hybrid memories efficiently in a distributed environment. This paper proposes Alloy, a memory resource abstraction and data placement strategy for an RDMA-enabled distributed hybrid memory pool (DHMP). Alloy provides simple APIs for applications to utilize DRAM or NVM resource in the DHMP, without being aware of the hardware details of the DHMP. We propose a hotness-aware data placement scheme, which combines hot data migration, data replication and write merging together to improve application performance and reduce the cost of DRAM. We evaluate Alloy with several micro-benchmark workloads and public benchmark workloads. Experimental results show that Alloy can significantly reduce the DRAM usage in the DHMP by up to 95%, while reducing the total memory access time by up to 57% compared with the state-of-the-art approaches.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Kapoor R, Porter G, Tewari M, Voelker G M, Vahdat A. Chronos: predictable low latency for data center applications. In: Proceedings of the 3rd ACM Symposium on Cloud Computing. 2012, 1–14

  2. Ousterhout J K, Agrawal P, Erickson D, Kozyrakis C, Leverich J, Mazières D, Mitra S, Narayanan A, Parulkar G M, Rosenblum M, Rumble S M, Stratmann E, Stutsman R. The case for ramclouds: scalable high-performance storage entirely in DRAM. Operating Systems Review, 2009, 43(4): 92–105

    Article  Google Scholar 

  3. Xiao Y, Nazarian S, Bogdan P. Prometheus: processing-in-memory heterogeneous architecture design from a multi-layer network theoretic strategy. In: Proceedings of the 2018 Design, Automation & Test in Europe Conference & Exhibition. 2018, 1387–1392

  4. Wu C, Zhang G, Li K. Rethinking computer architectures and software systems for phase-change memory. ACM Journal on Emerging Technologies in Computing Systems, 2016, 12(4): 1–40

    Article  Google Scholar 

  5. Peng I B, Gioiosa R, Kestor G, Vetter J S, Cicotti P, Laure E, Markidis S. Characterizing the performance benefit of hybrid memory system for HPC applications. Parallel Computing, 2018, 76: 57–69

    Article  Google Scholar 

  6. Peng I B, Gioiosa R, Kestor G, Cicotti P, Laure E, Markidis S. RTHMS: a tool for data placement on hybrid memory system. In: Proceedings of the 2017 ACM SIGPLAN International Symposium on Memory Management. 2017, 82–91

  7. Wu K, Huang Y, Li D. Unimem: runtime data management on nonvolatile memory-based heterogeneous main memory. In: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis. 2017, 1–14

  8. Lee S M, Yoon S K, Kim J G, Kim S D. Adaptive correlated prefetch with large-scale hybrid memory system for stream processing. The Journal of Supercomputing, 2018, 74(9): 4746–4770

    Article  Google Scholar 

  9. Li Y, Ghose S, Choi J, Sun J, Wang H, Mutlu O. Utility-based hybrid memory management. In: Proceedings of the 2017 IEEE International Conference on Cluster Computing. 2017, 152–165

  10. Xie Y, Feng D, Wang F, Zhang L. Non-sequential striping encoder from replication to erasure coding for distributed storage system. Frontiers of Computer Science, 2019, 13(6): 1356–1358

    Article  Google Scholar 

  11. Gutierrez-Garcia J O, Ramirez-Nafarrate A. Collaborative agents for distributed load management in cloud data centers using live migration of virtual machines. IEEE Transactions on Services Computing, 2015, 8(6): 916–929

    Article  Google Scholar 

  12. Desai M R, Patel H B. Efficient virtual machine migration in cloud computing. In: Proceedings of the 15th International Conference on Communication Systems and Network Technologies. 2015, 1015–1019

  13. Yin J, Wang J, Zhou J, Lukasiewicz T, Huang D, Zhang J. Opass: analysis and optimization of parallel data access on distributed file systems. In: Proceedings of the 2015 IEEE International Parallel and Distributed Processing Symposium. 2015, 623–632

  14. Kalia A, Kaminsky M, Andersen D G. FaSST: fast, scalable and simple distributed transactions with two-sided (RDMA) datagram RPCs. In: Proceedings of the 12th USENIX Symposium on Operating Systems Design and Implementation. 2016, 185–201

  15. Duan Z, Liu H, Liao X, Jin H. HME: a lightweight emulator for hybrid memory. In: Proceedings of the 2018 Design, Automation & Test in Europe Conference & Exhibition. 2018, 1375–1380

  16. Dragojević A, Narayanan D, Castro M, Hodson O. FaRM: fast remote memory. In: Proceedings of the 11th USENIX Symposium on Networked Systems Design and Implementation. 2014, 401–414

  17. Lu Y, Shu J, Chen Y, Li T. Octopus: an RDMA-enabled distributed persistent memory file system. In: Proceedings of the 2017 USENIX Annual Technical Conference. 2017, 773–785

  18. Stuedi P, Trivedi A, Pfefferle J, Klimovic A, Schüpbach A, Metzler B. Unification of temporary storage in the nodekernel architecture. In: Proceedings of the 2019 USENIX Annual Technical Conference. 2019, 767–782

  19. Nelson J, Holt B, Myers B, Briggs P, Ceze L, Kahan S, Oskin M. Latency-tolerant software distributed shared memory. In: Proceedings of the 2015 USENIX Annual Technical Conference. 2015, 291–305

  20. Shan Y, Tsai S Y, Zhang Y. Distributed shared persistent memory. In: Proceedings of the 2017 Symposium on Cloud Computing. 2017, 323–337

  21. Vavilapalli V K, Murthy A C, Douglas C, Agarwal S, Konar M, Evans R, Graves T, Lowe J, Shah H, Seth S, Saha B, Curino C, O’Malley O, Radia S, Reed B, Baldeschwieler E. Apache hadoop YARN: yet another resource negotiator. In: Proceedings of the 4th ACM Symposium on Cloud Computing. 2013, 1–16

  22. Ghemawat S, Gobioff H, Leung S T. The google file system. In: Proceedings of the 19th ACM Symposium on Operating Systems Principles. 2003, 29–43

  23. Munetomo M, Takai Y, Sato Y. A genetic approach to dynamic load balancing in a distributed computing system. In: Proceedings of the 1st IEEE Conference on Evolutionary Computation, IEEE World Congress on Computational Intelligence. 1994, 419–421

  24. Peng X, Liu X, Cao H, Zhang Z. An efficient energy aware virtual network migration based on genetic algorithm. Frontiers of Computer Science, 2019, 13(2): 440–442

    Article  Google Scholar 

  25. Chen X, Lin J, Ma Y, Lin B, Wang H, Huang G. Self-adaptive resource allocation for cloud-based software services based on progressive QoS prediction model. Science China Information Sciences, 2019, 62(11): 1–3

    Google Scholar 

  26. Xia Y, Ren R, Cai H, Vasilakos A V, Lv Z. Daphne: a flexible and hybrid scheduling framework in multi-tenant clusters. IEEE Transactions on Network and Service Management, 2018, 15(1): 330–343

    Article  Google Scholar 

  27. Hindman B, Konwinski A, Zaharia M, Ghodsi A, Joseph A D, Katz R H, Shenker S, Stoica I. Mesos: a platform for fine-grained resource sharing in the data center. In: Proceedings of the 8th USENIX Symposium on Networked Systems Design and Implementation. 2011, 295–308

  28. Ghodsi A, Zaharia M, Hindman B, Konwinski A, Shenker S, Stoica I. Dominant resource fairness: fair allocation of multiple resource types. In: Proceedings of the 8th USENIX Symposium on Networked Systems Design and Implementation. 2011, 323–336

  29. Sanchez V A B, Kim W, Eom Y, Jin K, Nam M, Hwang D, Kim J, Nam B. EclipseMR: distributed and parallel task processing with consistent hashing. In: Proceedings of the 2017 IEEE International Conference on Cluster Computing. 2017, 322–332

  30. Shen L, Wu J, Wang Y, Liu L. Towards load balancing for LSH-based distributed similarity indexing in high-dimensional space. In: Proceedings of the 20th IEEE International Conference on High Performance Computing and Communications; the 16th IEEE International Conference on Smart City; the 4th IEEE International Conference on Data Science and Systems. 2018, 384–391

  31. Kulkarni C, Kesavan A, Zhang T, Ricci R, Stutsman R. Rocksteady: fast migration for low-latency in-memory storage. In: Proceedings of the 26th Symposium on Operating Systems Principles. 2017, 390–405

  32. Psychas K, Ghaderi J. Randomized algorithms for scheduling multi-resource jobs in the cloud. IEEE/ACM Transactions on Networking, 2018, 26(5): 2202–2215

    Article  Google Scholar 

  33. Islam N S, Wasi-ur-Rahman M, Lu X, Panda D K. Efficient data access strategies for hadoop and spark on HPC cluster with heterogeneous storage. In: Proceedings of the 2016 IEEE International Conference on Big Data. 2016, 223–232

  34. Zhou P, Huang J, Qin X, Xie C. PaRS: a popularity-aware redundancy scheme for in-memory stores. IEEE Transactions on Computers, 2019, 68(4): 556–569

    Article  MathSciNet  Google Scholar 

  35. Zhao N, Anwar A, Cheng Y, Salman M, Li D, Wan J, Xie C, He X, Wang F, Butt A R. Chameleon: an adaptive wear balancer for flash clusters. In: Proceedings of the 2018 IEEE International Parallel and Distributed Processing Symposium. 2018, 1163–1172

  36. Atikoglu B, Xu Y, Frachtenberg E, Jiang S, Paleczny M. Workload analysis of a large-scale key-value store. In: Proceedings of the 12th ACM SIGMETRICS/PERFORMANCE Joint International Conference on Measurement and Modeling of Computer Systems. 2012, 53–64

  37. Zhang W, Hwang J, Wood T, Ramakrishnan K K, Huang H H. Load balancing of heterogeneous workloads in memcached clusters. In: Proceedings of the 9th International Workshop on Feedback Computing. 2014, 1–8

  38. Salkhordeh R, Asadi H. An operating system level data migration scheme in hybrid DRAM-NVM memory architecture. In: Proceedings of the 2016 Design, Automation & Test in Europe Conference & Exhibition. 2016, 936–941

  39. Zhou Y, Alagappan R, Memaripour A, Badam A, Wentzlaff D. HNVM: hybrid NVM enabled datacenter design and optimization. Microsoft Research Technical Report, 2017

  40. Calheiros R N, Ranjan R, Beloglazov A, Rose C A F D, Buyya R. Cloudsim: a toolkit for modeling and simulation of cloud computing environments and evaluation of resource provisioning algorithms. Software: Practice and Experience, 2011, 41(1): 23–50

    Google Scholar 

  41. Izraelevitz J, Yang J, Zhang L, Kim J, Liu X, Memaripour A, Soh Y J, Wang Z, Xu Y, Dulloor S R, Zhao J, Swanson S. Basic performance measurements of the intel optane DC persistent memory module. 2019, arXiv preprint arXiv:1903.05714

  42. Hong Y J, Thottethodi M. Understanding and mitigating the impact of load imbalance in the memory caching tier. In: Proceedings of the ACM Symposium on Cloud Computing. 2013, 1–17

  43. Ou J, Shu J, Lu Y, Yi L, Wang W. EDM: an endurance-aware data migration scheme for load balancing in SSD storage clusters. In: Proceedings of the 28th IEEE International Parallel and Distributed Processing Symposium. 2014, 787–796

  44. Cooper B F, Silberstein A, Tam E, Ramakrishnan R, Sears R. Benchmarking cloud serving systems with YCSB. In: Proceedings of the 1st ACM Symposium on Cloud Computing. 2010, 143–154

Download references

Acknowledgements

The authors would like to thank the anonymous reviewers for their insightful comments. This work was supported jointly by National Key Research and Development Program of China (2017YFB1001603), and National Natural Science Foundation of China (NSFC) (Grants Nos. 61672251, 61732010, 61825202).

Author information

Authors and Affiliations

  1. National Engineering Research Center for Big Data Technology and System, Services Computing Technology and System Lab/Cluster and Grid Computing Lab, School of Computing Science and Technology, Huazhong University of Science and Technology, Wuhan, 430074, China

    Tingting Chen, Haikun Liu, Xiaofei Liao & Hai Jin

Authors
  1. Tingting Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Haikun Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiaofei Liao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hai Jin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Haikun Liu.

Additional information

Tingting Chen received the BS degree in computer science and technology from Huazhong University of Science and Technology (HUST), China in 2017. She is now working toward the MS degree at HUST, China. Her research interests include hybrid memory and distributed systems.

Haikun Liu received his PhD degree from Huazhong University of Science and Technology (HUST), China. He is an associate professor in School of Computer Science and Technology, HUST, China. His current research interests include in-memory computing, virtualization technologies, cloud computing. He was the recipient of outstanding doctoral dissertation award in Hubei province, China. He is senior member of the CCF, and a member of the IEEE.

Xiaofei Liao received his PhD degree in computer science and engineering from Huazhong University of Science and Technology (HUST), China in 2005. He is now a professor in the school of Computer Science and Engineering at HUST, China. He has served as a reviewer for many conferences and journal papers. His research interests are in the areas of system software, P2P system, cluster computing and streaming services. He is a member of the IEEE and the IEEE Computer society.

Hai Jin is a Cheung Kung Scholars Chair Professor of computer science and engineering at Huazhong University of Science and Technology (HUST) in China. He received his PhD in computer engineering from HUST, China in 1994. In 1996, he was awarded a German Academic Exchange Service fellowship to visit the Technical University of Chemnitz, Germany. He worked at The University of Hong Kong, China between 1998 and 2000, and as a visiting scholar at the University of Southern California, USA between 1999 and 2000. He was awarded Excellent Youth Award from the National Science Foundation of China in 2001. He is the chief scientist of ChinaGrid, the largest grid computing project in China, and the chief scientists of National 973 Basic Research Program Project of Virtualization Technology of Computing System, and Cloud Security. He is a Fellow of CCF, a Fellow of the IEEE and a member of the ACM. He has co-authored 22 books and published over 800 research papers. His research interests include computer architecture, virtualization technology, cluster computing and cloud computing, peer-to-peer computing, network storage, and network security.

Electronic supplementary material

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, T., Liu, H., Liao, X. et al. Resource abstraction and data placement for distributed hybrid memory pool. Front. Comput. Sci. 15, 153103 (2021). https://doi.org/10.1007/s11704-020-9448-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11704-020-9448-7

Keywords

Search

Navigation