research-article

RESTRAIN: A dynamic and cost-efficient resource management scheme for addressing performance interference in NFV-based systems

Authors:

Venkatarami Reddy Chintapalli,

Madhura Adeppady,

Bheemarjuna Reddy Tamma,

Antony Franklin A.Authors Info & Claims

Volume 201, Issue C

https://doi.org/10.1016/j.jnca.2021.103312

Published: 01 May 2022 Publication History

Abstract

Network Functions Virtualization (NFV) replaces the conventional middleboxes by their software counterparts known as Virtual Network Functions (VNFs) which run on general-purpose hardware platforms and promise several benefits like reduced cost, ease of deployment, flexibility, etc. However, NFV faces some critical challenges as VNFs running on the same physical hardware still have to compete for shared resources such as Last Level Cache (LLC) and different levels of Memory Bandwidth (MB) (between L2 cache & LLC and LLC & main memory), which might result in unpredictable and variable performance interference to the co-located VNFs deployed. Some recent works have explored mechanisms for allocating LLC dynamically using Cache Allocation Technology (CAT) but they did not look into MB contentions among the co-located VNFs. Dynamic allocation of both LLC and MB to the co-located VNFs remains unexplored. To address this, in this work, by leveraging Intel’s CAT and Memory Bandwidth Allocation (MBA) technologies, we profile different VNFs to determine their minimum LLC and MB resource requirements to achieve performance isolation for different input traffic rates. We then propose a dynamic, joint resource allocation scheme, RESTRAIN, that takes each VNF’s input traffic rate as an input and dynamically adjusts LLC ways and MB resources allocated among them to avoid performance interference and thereby improves the overall resource utilization of the underlying hardware system and the number of VNFs meeting their QoS guarantees. Experimental results on a prototype system show that the proposed RESTRAIN scheme guarantees performance isolation. Further, it improves performance by 30% over a static allocation mechanism and 17% over ResQ, a state-of-the-art scheme.

References

[1]

Sun, Chen, Bi, Jun, Zheng, Zhilong, Yu, Heng, Hu, Hongxin, 2017. Nfp: Enabling network function parallelism in nfv. In: Proc. of ACM SIGCOMM. pp. 43–56.

[2]

Panda, Aurojit, Han, Sangjin, Jang, Keon, Walls, Melvin, Ratnasamy, Sylvia, Shenker, Scott, 2016. Netbricks: Taking the v out of nfv. In: Proc. of ACM OSDI. pp. 203–216.

[3]

Katsikas, Georgios P., Barbette, Tom, Kostic, Dejan, Steinert, Rebecca, Maguire, Gerald Q., Jr., 2018. Metron: Nfv service chains at the true speed of the underlying hardware. In: Proc. of 15th USENIX NSDI. pp. 171–186.

[4]

Intel, Intel, DPDK: Data Plane Development Kit: Programmer’s Guide, Intel Corporation, 2017.

[5]

Intel, 2012. DDIO: Intel Data Direct I/O Technology Overview. Intel White Paper.

[6]

Dobrescu, Mihai, Argyraki, Katerina, Ratnasamy, Sylvia, 2012. Toward predictable performance in software packet processing platforms. In: Proc. of ACM NSDI. pp. 141–154.

[7]

Mars, Jason, Tang, Lingjia, Hundt, Robert, Skadron, Kevin, Soffa, Mary Lou, 2011. Bubble-up: Increasing utilization in modern warehouse scale computers via sensible co-locations. In: Proc. of IEEE/ACM MICRO. pp. 248–259.

[8]

Tootoonchian, Amin, Panda, Aurojit, Lan, Chang, Walls, Melvin, Argyraki, Katerina, Ratnasamy, Sylvia, Shenker, Scott, 2018. Resq: Enabling slos in network function virtualization. In: Proc. of 15th USENIX NSDI. pp. 283–297.

[9]

Veitch, Paul, Curley, Edel, Kantecki, Tomasz, 2017. Performance evaluation of cache allocation technology for nfv noisy neighbor mitigation. In: Proc. of IEEE NetSoft. pp. 1–5.

[10]

Schramm, Norbert, Runge, Torsten M., Wolfinger, Bernd E., 2019. The impact of cache partitioning on software-based packet processing. In: Proc. of IEEE NetSys. pp. 1–6.

[11]

Li, Bin, Wang, Yipeng, Wang, Ren, Tai, Charlie, Iyer, Ravi, Zhou, Zhu, Herdrich, Andrew, Zhang, Tong, Haj-Ali, Ameer, Stoica, Ion, Asanovic, Krste, 2020b. Rldrm: Closed loop dynamic cache allocation with deep reinforcement learning for network function virtualization. In: Proc. of IEEE Conference on NetSoft.

[12]

Yu Heng, Zheng Zhilong, Shen Junxian, Miao Congcong, Sun Chen, Hu Hongxin, Bi Jun, Wu Jianping, Wang Jilong, Octans: Optimal placement of service function chains in many-core systems, IEEE Trans. Parallel Distrib. Syst. 32 (9) (2021) 2202–2215.

[13]

Cai, Qizhe, Chaudhary, Shubham, Vuppalapati, Midhul, Hwang, Jaehyun, Agarwal, Rachit, 2021. Understanding host network stack overheads. In: Proc. of ACM SIGCOMM. pp. 65–77.

[14]

Sieber, Christian, Durner, Raphael, Ehm, Maximilian, Kellerer, Wolfgang, Sharma, Puneet, 2017. Towards optimal adaptation of nfv packet processing to modern cpu memory architectures. In: Proceedings of the 2nd Workshop on Cloud-Assisted Networking. pp. 7–12.

[15]

Zhong Youbing, Zhou Zhou, Liu Xuan, Li Da, Guo Meijun, Zhang Shuai, Liu Qingyun, Guo Li, Bpa: The optimal placement of interdependent vnfs in many-core system, in: International Conference on Collaborative Computing: Networking, Applications and Worksharing, Springer, 2020, pp. 305–319.

[16]

Zeng, C., Liu, F., Chen, S., Jiang, W., Li, M., 2018. Demystifying the performance interference of co-located virtual network functions. In: Proc. of IEEE INFOCOM. pp. 765–773.

[17]

Mijumbi R., Serrat J., Gorricho J., Bouten N., De Turck F., Boutaba R., Network function virtualization: State-of-the-art and research challenges, IEEE Commun. Surv. Tutor. 18 (1) (2016) 236–262.

Digital Library

[18]

Kuo T., Liou B., Lin K.C., Tsai M., Deploying chains of virtual network functions: On the relation between link and server usage, IEEE/ACM Trans. Netw. 26 (4) (2018) 1562–1576.

[19]

Zhang, Q., Xiao, Y., Liu, F., Lui, J.C.S., Guo, J., Wang, T., 2017. Joint optimization of chain placement and request scheduling for network function virtualization. In: Proc. of IEEE ICDCS. pp. 731–741.

[20]

Sang, Y., Ji, B., Gupta, G.R., Du, X., Ye, L., 2017. Provably efficient algorithms for joint placement and allocation of virtual network functions. In: Proc. of IEEE INFOCOM. pp. 1–9.

[21]

Cohen, R., Lewin-Eytan, L., Naor, J.S., Raz, D., 2015. Near optimal placement of virtual network functions. In: Proc. of IEEE INFOCOM. pp. 1346–1354.

[22]

Jin, P., Fei, X., Zhang, Q., Liu, F., Li, B., 2020. Latency-aware vnf chain deployment with efficient resource reuse at network edge. In: Proc. of IEEE INFOCOM. pp. 267–276.

[23]

Nam, Jaehyun, Seo, Junsik, Shin, Seungwon, 2018. Probius: Automated approach for VNF and Service Chain Analysis in Software-Defined NFV. In: Proc. of ACM Symposium on SDN Research. pp. 1–13.

[24]

Herdrich, Andrew, Verplanke, Edwin, Autee, Priya, Illikkal, Ramesh, Gianos, Chris, Singhal, Ronak, Iyer, Ravi, 2016. Cache qos: From concept to reality in the intel® xeon® processor e5-2600 v3 product family. In: Proc. of IEEE Symposium HPCA. pp. 657–668.

[25]

ETSI GSZSM, Zero-touch network and service management (zsm); reference architecture, in: ETSI GS ZSM 002, V1.1.1, 2019.

[26]

Zhang Qixia, Liu Fangming, Zeng Chaobing, Online adaptive interference-aware vnf deployment and migration for 5g network slice, IEEE/ACM Trans. Netw. 29 (5) (2021) 2115–2128.

Digital Library

[27]

Mu, Yanyan, Wang, Lei, Zhao, Jin, 2021. Energy-efficient and interference-aware vnf placement with deep reinforcement learning. In: Proc. of IEEE IFIP Networking. pp. 1–9.

[28]

Nikas, Konstantinos, Papadopoulou, Nikela, Giantsidi, Dimitra, Karakostas, Vasileios, Goumas, Georgios, Koziris, Nectarios, 2019. Dicer: Diligent cache partitioning for efficient workload consolidation. In: Proc. of ACM ICPP. pp. 1–10.

[29]

Xu, Cong, Rajamani, Karthick, Ferreira, Alexandre, Felter, Wesley, Rubio, Juan, Li, Yang, 2018. Dcat: Dynamic cache management for efficient, performance-sensitive infrastructure-as-a-service. In: Proc. of ACM EuroSys. pp. 1–13.

[30]

Li, Yusen, Liu, Haoyuan, Wang, Xiwei, Pu, Lingjun, Marbach, Trent, Tang, Shanjiang, Wang, Gang, Liu, Xiaoguang, 2019a. Themis: Efficient and adaptive resource partitioning for reducing response delay in cloud gaming. In: Proc. of ACM MM. pp. 491–499.

[31]

Intel, Intel, Introduction to Cache Allocation Technology in the Intel Xeon Processor E5 V4 Family, Intel Corporation, 2016.

[32]

Chen, Shuang, Delimitrou, Christina, Martínez, José F., 2019. Parties: Qos-aware resource partitioning for multiple interactive services. In: Proc. of ACM ASPLOS. pp. 107–120.

[33]

Park, Jinsu, Park, Seongbeom, Baek, Woongki, 2019. Copart: Coordinated partitioning of last-level cache and memory bandwidth for fairness-aware workload consolidation on commodity servers. In: Proc. of ACM EuroSys. pp. 1–16.

[34]

Lo, David, Cheng, Liqun, Govindaraju, Rama, Ranganathan, Parthasarathy, Kozyrakis, Christos, 2015. Heracles: improving resource efficiency at scale. In: Proc. of ACM ISCA. pp. 450–462.

[35]

Yuan, Yifan, Alian, Mohammad, Wang, Yipeng, Wang, Ren, Kurakin, Ilia, Tai, Charlie, Kim, Nam Sung, 2021. Don’t forget the i/o when allocating your llc. In: Proc. of ACM/IEEE ISCA.

[36]

Bienia Christian, Benchmarking Modern Multiprocessors, (Ph.D. thesis) Princeton University, 2011.

Digital Library

[37]

Henning John L., Spec cpu2006 benchmark descriptions, SIGARCH Comput. Archit. News 34 (4) (2006) 1–17.

Digital Library

[38]

Manousis, Antonis, Sharma, Rahul Anand, Sekar, Vyas, Sherry, Justine, 2020. Contention-aware performance prediction for virtualized network functions. In: Proc. of ACM SIGCOMM. pp. 270–282.

[39]

Intel, Intel, Resource Director Technology in Linux, Intel Corporation, 2017.

[40]

Anon, 2022. User space software for intel resource director technology. https://github.com/01org/intel-cmt-cat/tree/master/pqos.

[41]

Zhang, Wei, Liu, Guyue, Zhang, Wenhui, Shah, Neel, Lopreiato, Phillip, Todeschi, Gregoire, Ramakrishnan, K.K., Wood, Timothy, 2016. Opennetvm: A platform for high performance network service chains. In: Proc. of ACM HotMiddlebox. pp. 26–31.

[42]

Inc. GitHub., Pktgen - traffic generator powered by dpdk, 2019, https://github.com/Pktgen/Pktgen-DPDK/.

[43]

Anon, 2020. https://wiki.ubuntu.com/Kernel/Reference/stress-ng.

[44]

Palkar, Shoumik, Lan, Chang, Han, Sangjin, Jang, Keon, Panda, Aurojit, Ratnasamy, Sylvia, Rizzo, Luigi, Shenker, Scott, 2015. E2: A framework for nfv applications. In: Proc. of ACM SOSP. pp. 121–136.

[45]

Martins, Joao, Ahmed, Mohamed, Raiciu, Costin, Olteanu, Vladimir, Honda, Michio, Bifulco, Roberto, Huici, Felipe, 2014. Clickos and the art of network function virtualization. In: Proc. of ACM NSDI. pp. 459–473.

[46]

Savi Marco, Tornatore Massimo, Verticale Giacomo, Impact of processing-resource sharing on the placement of chained virtual network functions, IEEE Trans. Cloud Comput. (2019).

[47]

Reddy, Venkatarami, Garg, Gaurav, Tamma, Bheemarjuna Reddy, Antony, Franklin A., 2019. Interference aware network function selection algorithm for next generation networks. In: Proc. of IEEE NetSoft. pp. 54–59.

[48]

Rahman Mahfuzur, Graham Peter, Compatibility-based static vm placement minimizing interference, J. Netw. Comput. Appl. 84 (1) (2017) 68–81.

[49]

Yokoyama Daniel, Schulze Bruno, Kloh Henrique, Bandini Matheus, Rebello Vinod, Affinity aware scheduling model of cluster nodes in private clouds, J. Netw. Comput. Appl. 95 (1) (2017) 94–104.

[50]

Li Jingwei, Qi Yong, Wei Wei, Lin Jinwei, Wozniak Marcin, Damasevicius Robertas, dccpi-predictor: A state-aware approach for effectively predicting cross-core performance interference, Future Gener. Comput. Syst. 105 (1) (2020) 184–195.

[51]

Xu, Xin, Zhang, Na, Cui, Michael, He, Michael, Surana, Ridhi, 2019. Characterization and prediction of performance interference on mediated passthrough gpus for interference-aware scheduler. In: Proc. of the ACM USENIX HotCloud.

[52]

Li, Yusen, Shan, Chuxu, Chen, Ruobing, Tang, Xueyan, Cai, Wentong, Tang, Shanjiang, Liu, Xiaoguang, Wang, Gang, Gong, Xiaoli, Zhang, Ying, 2019b. Gaugur: Quantifying performance interference of colocated games for improving resource utilization in cloud gaming. In: Proc. of ACM HPDC. pp. 231–242.

[53]

Javadi Seyyedahmad, Gandhi Anshul, User-centric interference-aware load balancing for cloud-deployed applications, IEEE Trans. Cloud Comput. (2019).

[54]

Savić Mihajlo, Ljubojević Miloš, Gajin Slavko, A novel approach to client-side monitoring of shared infrastructures, IEEE Access 8 (2020) 44175–44189.

[55]

Mukherjee Joydeep, Krishnamurthy Diwakar, Rolia Jerry, Resource contention detection in virtualized environments, IEEE Trans. Netw. Serv. Manag. 12 (2) (2015) 217–231.

[56]

Mukherjee Joydeep, Krishnamurthy Diwakar, Wang Mea, Subscriber-driven interference detection for cloud-based web services, IEEE Trans. Netw. Serv. Manag. 14 (1) (2017) 48–62.

[57]

Mukherjee Joydeep, Krishnamurthy Diwakar, Prima: Subscriber-driven interference mitigation for cloud services, IEEE Trans. Netw. Serv. Manag. 17 (2) (2019) 958–971.

[58]

Romero, Francisco, Delimitrou, Christina, 2018. Mage: Online and interference-aware scheduling for multi-scale heterogeneous systems. In: Proc. of ACM PACT. pp. 1–13.

[59]

Han, Jaeung, Jeon, Seungheun, Choi, Young-ri, Huh, Jaehyuk, 2016. Interference management for distributed parallel applications in consolidated clusters. In: Proc. of ACM ASPLOS. pp. 443–456.

[60]

Rossem Steven Van, Tavernier Wouter, Colle Didier, Pickavet Mario, Demeester Piet, Profile-based resource allocation for virtualized network functions, IEEE Trans. Netw. Serv. Manag. 16 (4) (2019) 1374–1388.

[61]

Zheng, Peng, Feng, Wendi, Narayanan, Arvind, Zhang, Zhi-Li, 2020. Nfv performance profiling on multi-core servers. In: Proc. of IEEE IFIP Networking Conference. pp. 91–99.

[62]

Khan, Michel Gokan, Bastani, Saeed, Taheri, Javid, Kassler, Andreas, Deng, Shuiguang, 2018. Nfv-inspector: A systematic approach to profile and analyze virtual network functions. In: Proc. of IEEE CloudNet. pp. 1–7.

[63]

Khan, M.G., Taheri, J., Khoshkholghi, M.A., Kassler, A., Cartwright, C., Darula, M., Deng, S., 2020. A performance modelling approach for sla-aware resource recommendation in cloud native network functions. In: Proc. of IEEE Conference on NetSoft. pp. 292–300.

[64]

Koehler Anne B., Snyder Ralph D., Keith Ord J., Forecasting models and prediction intervals for the multiplicative holt–winters method, Int. J. Forecast. 17 (2) (2001) 269–286.

[65]

Akbar Md Mostofa, Sohel Rahman M., Kaykobad Mohammad, Manning Eric G., Shoja Gholamali C., Solving the multidimensional multiple-choice knapsack problem by constructing convex hulls, Comput. Oper. Res. 33 (5) (2006) 1259–1273.

[66]

Chintapalli, Venkatarami Reddy, Kondepu, Koteswararao, Sgambelluri, Andrea, Antony, Franklin A., Tamma, Bheemarjuna Reddy, Castoldi, Piero, Valcarenghi, Luca, 2020. Orchestrating edge- and cloud-based predictive analytics services. In: Proc. of IEEE Conference on EuCNC. pp. 214–218.

[67]

Barlacchi Gianni, Nadai Marco De, Larcher Roberto, Casella Antonio, Chitic Cristiana, Torrisi Giovanni, Antonelli Fabrizio, Vespignani Alessandro, Pentland Alex, Lepri Bruno, A multi-source dataset of urban life in the city of milan and the province of trentino, Sci. Data 2 (1) (2015) 1–15.

Index Terms

RESTRAIN: A dynamic and cost-efficient resource management scheme for addressing performance interference in NFV-based systems

Index terms have been assigned to the content through auto-classification.

Recommendations

Batch scheduling of consolidated virtual machines based on their workload interference model

The use of virtualization technology (VT) has become widespread in modern datacenters and Clouds in recent years. In spite of their many advantages, such as provisioning of isolated execution environments and migration, current implementations of VT do ...
Quantifying the performance isolation properties of virtualization systems
ExpCS '07: Proceedings of the 2007 workshop on Experimental computer science

In this paper, we present the design of a performance isolation benchmark that quantifies the degree to which a virtualization system limits the impact of a misbehaving virtual machine on other well-behaving virtual machines running on the same physical ...
vCache: architectural support for transparent and isolated virtual LLCs in virtualized environments
MICRO-48: Proceedings of the 48th International Symposium on Microarchitecture

A key role of virtualization is to give an illusion that a consolidated workload runs on a dedicated machine although the underlying resources are actively shared by multiple workloads. Technical advances have enabled a virtual machine (VM) to exercise ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Journal of Network and Computer Applications

Journal of Network and Computer Applications Volume 201, Issue C

May 2022

192 pages

ISSN:1084-8045

Issue’s Table of Contents

Elsevier Ltd.

Publisher

Academic Press Ltd.

United Kingdom

Publication History

Published: 01 May 2022

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

0
Total Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 23 Feb 2025

Other Metrics

View Author Metrics

Citations

View Options

View options

Figures

Tables

Media

View Issue’s Table of Contents