Abstract
Non-Volatile Memory with byte-addressability invites a new paradigm to access persistent data directly. However, this paradigm brings new challenges to the Non-Uniform Memory Access (NUMA) architecture. Since data accesses cross NUMA node can incur significant performance loss, and, traditionally, OS moves data to the NUMA node where the process accessing it locates to reduce the access latency. However, we find challenges when migrating data on NVM, which motivates us to migrate the process instead. We propose SysMon-N, an OS-level sampling module, to obtain access information about NVM in low overhead. Furthermore, we propose N-Policy to utilize the data collected by SysMon-N to guide process migration. We evaluate SysMon-N and N-Policy on off-the-shelf NVM devices. The experimental results show that they provide 5.9% to \(3.62\times \) bandwidth improvement in the case where cross-node memory accesses happen.
This project is supported by the National Key Research and Development Program of China under Grant No.2017YFB1001602 and the NSFC under grants No.61502452, 61902206, 61702286. This work originates from L. Liu’s series of studies in ISCA, PACT, TPDS, TC, etc. [5,6,7,8,9,10,11,12] on memory systems conducted in Sys-Inventor Lab. More details refer to Sys-Inventor Lab - https://liulei-sys-inventor.github.io.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Direct Access for files. https://www.kernel.org/doc/Documentation/filesystems/dax.txt
Fio - Flexible I/O tester. https://fio.readthedocs.io/en/latest/
Persistent Memory Concepts. https://docs.pmem.io/ndctl-user-guide/concepts
Rao, D.S., et al., System software for persistent memory. In: EuroSys (2014)
Xie, M., et al.: Sysmon: monitoring memory behaviorsvia OS approach. In: APPT (2017)
L. Liu, et al.: Hierarchical hybrid memory management in OS for tiered memory systems. In: IEEE TPDS (2019)
X. Li, et al.: Thinking about a new mechanism for huge page management. In: APSys (2019)
Liu, L., et al.: Going vertical in memory management: handling multiplicity by multi-policy. In: ISCA (2014) (revised version)
Liu, L., et al.: BPM/BPM+: software-based dynamic memory partitioning mechanisms for mitigating DRAM Bank-/channel-level interferences in multicore systems. In: ACM TACO (2014) (revised version)
Liu, L., et al.: A software memory partition approach for eliminating bank-level interference in multicore systems. In: PACT (2012) (revised version)
Liu, L., et al.: Rethinking memory management in modern operating system: horizontal, vertical or random?. In: IEEE Trans. Comput. (TC) (2016)
Liu, L., et al.: Memos: a full hierarchy hybrid memory management framework. In: ICCD (2016)
Chen, S., et al.: Efficient GPU NVRAM persistence with helper warps. In: DAC (2019)
Lv, F., Cui, H.-M., Wang, L., Liu, L., Wu, C.-G., Feng, X.-B., Yew, P.-C.: Dynamic I/O-Aware scheduling for batch-mode applications on chip multiprocessor systems of cluster platforms. J. Comput. Sci. Technol. 29(1), 21–37 (2014). https://doi.org/10.1007/s11390-013-1409-2
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 IFIP International Federation for Information Processing
About this paper
Cite this paper
Yang, S. et al. (2021). Monitoring Memory Behaviors and Mitigating NUMA Drawbacks on Tiered NVM Systems. In: He, X., Shao, E., Tan, G. (eds) Network and Parallel Computing. NPC 2020. Lecture Notes in Computer Science(), vol 12639. Springer, Cham. https://doi.org/10.1007/978-3-030-79478-1_33
Download citation
DOI: https://doi.org/10.1007/978-3-030-79478-1_33
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-79477-4
Online ISBN: 978-3-030-79478-1
eBook Packages: Computer ScienceComputer Science (R0)