A hierarchical reinforcement learning approach for energy‐aware service function chain dynamic deployment in IoT
Abstract
Traffic volume is increasing dramatically due to the quick development of technologies like online gaming, on‐demand video streaming, and the Internet of Things (IoT). The telecommunications industry's large‐scale expansion is increasing its energy usage and carbon footprint. Given the desire to minimize energy consumption and carbon emissions, one of the most essential concerns of future communication networks is ensuring rigorous performance restrictions of IoT services while improving energy efficiency. In this regard, a convolutional neural network‐based hierarchical reinforcement learning approach is provided to lower total energy consumption and carbon emissions in the dynamic service function chaining situations. This method can more effectively lower energy consumption and carbon emissions when compared to other hierarchical algorithms based on conventional deep neural networks and non‐hierarchical algorithms. The suggested method is tested in three typical complicated networks with different network parameters to show its suitability in different network scenarios.
Graphical Abstract
We provide a convolutional neural network‐based hierarchical reinforcement learning approach to lower total energy consumption and carbon emissions in the dynamic service function chaining situations. This method can more effectively lower energy consumption and carbon emissions when compared to other hierarchical algorithms based on conventional deep neural networks and non‐hierarchical algorithms. And the suggested method is tested in three typical complicated networks with different network parameters to show its suitability in different network scenarios.
References
[1]
Bari, M.F., Chowdhury, S.R., Boutaba, R.: ESSO: An Energy Smart Service Function Chain Orchestrator. IEEE Trans. Netw. Serv. Manag. 16(4), 1345–1359 (2019)
[2]
Joyce, T., Okrasinski, T.A., Schaeffer, W.: Estimating the Carbon Footprint of Telecommunications Products: A Heuristic Approach. J. Mech. Des. 132(9), 094502 (2010)
[3]
Federal renewable energy projects and technologies. http://energy.gov/eere/femp/federal‐renewable‐energy‐projects‐and‐technologies. Accessed 27 December 2021
[4]
Ikebe, H., Yamashita, N., Nishii, R.: Green Energy for Telecommunications. In: INTELEC 07‐29th International Telecommunications Energy Conference, pp. 1–6. IEEE, Piscataway (2008)
[5]
Mechtri, M., Ghribi, C., Soualah, O., Zeghlache, D.: NFV orchestration framework addressing SFC challenges. IEEE Commun. Mag. 55(6), 16–23 (2017)
[6]
Kobusińska, A., Leung, C., Hsu, C.H., et al.: Emerging trends, issues and challenges in Internet of Things, Big Data and cloud computing. Future Gener. Comput. Syst. 87, 416–419 (2018)
[7]
Chang, V., Ramachandran, M.: Towards achieving data security with the cloud computing adoption framework. IEEE Trans. Serv. Comput. 9(1), 138–151 (2016)
[8]
Taleb, T., Samdanis, K., Mada, B., et al.: On multi‐access edge computing: A survey of the emerging 5G network edge cloud architecture and orchestration. IEEE Commun. Surveys Tuts. 19(3), 1657–1681 (2017)
[9]
Medhat, A.M., Taleb, T., Elmangoush, A., et al.: Service function chaining in next generation networks: State of the art and research challenges. IEEE Commun. Mag. 55(2), 216–223 (2017)
[10]
Pham, C., Tran, N.H., Ren, S., et al.: Traffic‐aware and energy‐efficient vNF placement for service chaining: Joint sampling and matching approach. IEEE Trans. Serv. Comput. 13(1), 172–185 (2020)
[11]
Sun, G., Li, Y., Yu, H., et al.: Energy‐efficient and traffic‐aware service function chaining orchestration in multi‐domain networks. Future Gener. Comput. Syst. 91, 347–360 (2019)
[12]
Farkiani, B., Bakhshi, B., MirHassani, S.A.: A fast near‐optimal approach for energy‐aware SFC deployment. IEEE Trans. Netw. Serv. Manag. 16(4), 1360–1373 (2019)
[13]
Mai, L., Ding, Y., Zhang, X., et al.: Energy efficiency with service availability guarantee for network function virtualization. Future Gener. Comput. Syst. 119, 140–153 (2021)
[14]
Sun, G., Zhou, R., Sun, J., et al.: Energy‐efficient provisioning for service function chains to support delay‐sensitive applications in network function virtualization. IEEE Internet Things J. 7(7), 6116–6131 (2020)
[15]
Dalgkitsis, A., Garrido, L.A., Rezazadeh, F., et al.: SCHE2MA: Scalable, energy‐aware, multidomain orchestration for beyond‐5G URLLC services. IEEE Trans. Intell. Transp. Syst. 24(7), 7653–7663 (2023)
[16]
Kulkarni, T.D., Narasimhan, K.R., Saeedi, A., et al.: Hierarchical deep reinforcement learning: Integrating temporal abstraction and intrinsic motivation. arXiv preprint, arXiv:1604.06057 (2016)
[17]
Li, Z., Liu, F., Yang, W., et al.: A survey of convolutional neural networks: Analysis, applications, and prospects. IEEE Trans. Neural Networks Learn. Syst. 33(12), 6999–7019 (2022)
[18]
Ren, W., Sun, Y., Luo, H., et al.: A new scheme for IoT service function chains orchestration in SDN‐IoT network systems. IEEE Syst. J. 13(4), 4081–4092 (2019)
[19]
Wang, J., Qi, H., Li, H., Zhou, X.: PRSFC‐IoT: A performance and resource aware orchestration system of service function chaining for Internet of Things. IEEE Internet Things J. 5(3), 1400–1410 (2018)
[20]
Sun, G., Li, Y.Y., Li, Y., et al.: Low‐latency orchestration for workflow‐oriented service function chain in edge computing. Future Gener. Comput. Syst. 85, 116–128 (2018)
[21]
Fu, X., Yu, F.R., Wang, J., et al.: Dynamic service function chain embedding for NFV‐enabled IoT: A deep reinforcement learning approach. IEEE Trans. Wireless Commun. 19(1), 507–519 (2020)
[22]
Liu, Y., Lu, H., Li, X., et al.: A novel approach for service function chain dynamic orchestration in edge clouds. IEEE Commun. Lett. 24(10), 2231–2235 (2020)
[23]
Liu, Y., Lu, H., Li, X., et al.: Dynamic service function chain orchestration for NFV/MEC‐enabled IoT networks: A deep reinforcement learning approach. IEEE Internet Things J. 8(9), 7450–7465 (2021)
[24]
Kang, J., et al.: Blockchain‐empowered federated learning for healthcare metaverses: User‐centric incentive mechanism with optimal data freshness. IEEE Tran. Cognit. Commun. Networking 10(1), 348–362 (2024)
[25]
Akbari, M., Syed, A., et al.: AoI‐aware energy‐efficient SFC in UAV‐aided smart agriculture using asynchronous federated learning. IEEE Open J. Commun. Soc. 5, 1222–1242 (2024)
[26]
Xu, M., Niyato, D., Zhang, H., et al.: Cached model‐as‐a‐resource: Provisioning large language model agents for edge intelligence in space‐air‐ground integrated networks. arXiv preprint, arXiv:2403.05826 (2024)
[27]
Kang, J., et al.: Personalized saliency in task‐oriented semantic communications: Image transmission and performance analysis. IEEE J. Sel. Areas Commun. 41(1), 186–201 (2023)
Index Terms
- A hierarchical reinforcement learning approach for energy‐aware service function chain dynamic deployment in IoT
Index terms have been assigned to the content through auto-classification.
Recommendations
Uncertainty-aware Energy Harvest Prediction and Management for IoT Devices
Internet of things (IoT) devices are popular in several high-impact applications such as mobile healthcare and digital agriculture. However, IoT devices have limited operating lifetime due to their small form factor. Harvesting energy from ambient sources ...
Energy-aware system design for batteryless LPWAN devices in IoT applications
AbstractInterconnected LPWAN devices, which create the IoT, are usually powered by batteries that significantly limit their operational lifetime. The main disadvantage of batteries is that they must be periodically/manually replaced with new ...
Reinforcement Learning and Energy-Aware Routing
FlexNets '21: Proceedings of the 4th FlexNets Workshop on Flexible Networks Artificial Intelligence Supported Network Flexibility and AgilityWe present an approach that uses Reinforcement Learning (RL) with the Random Neural Network (RNN) acting as an adaptive critic, to route traffic in a SDN network, so as to minimize a composite Goal function that includes both packet delay and energy ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
© 2024 The Author(s). IET Communications published by John Wiley & Sons Ltd on behalf of The Institution of Engineering and Technology.
This is an open access article under the terms of the Creative Commons Attribution‐NonCommercial‐NoDerivs License, which permits use and distribution in any medium, provided the original work is properly cited, the use is non‐commercial and no modifications or adaptations are made.
Publisher
John Wiley & Sons, Inc.
United States
Publication History
Published: 23 August 2024
Author Tags
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 08 Dec 2024
Other Metrics
Citations
View Options
View options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in