research-article

Open access

Multi-Agent Deep Reinforcement Learning for Weighted Multi-Path Routing

Authors:

Theodoros Theodoropoulos,

Dimitrios Kafetzis,

Antonios Makris,

Konstantinos TserpesAuthors Info & Claims

FRAME '23: Proceedings of the 3rd Workshop on Flexible Resource and Application Management on the Edge

Pages 7 - 11

https://doi.org/10.1145/3589010.3594888

Published: 15 August 2023 Publication History

Abstract

Traditional multi-path routing methods distribute evenly traffic across multiple paths in a network, which can lead to inefficient use of resources if some paths are significantly longer or less reliable than others. Weighted multi-path routing addresses this issue by introducing weights to appropriately distribute traffic across the available paths based on their state. This paper proposes a novel approach to weighted multi-path routing using a multi-agent actor-critic framework, in a manner that is aligned with the need to keep up with the Quality of Service requirements of contemporary, bandwidth-intensive applications.

References

[1]

Davide Andreoletti, Tanya Velichkova, Giacomo Verticale, Massimo Tornatore, and Silvia Giordano. 2020. A privacy-preserving reinforcement learning algorithm for multi-domain virtual network embedding. IEEE Transactions on Network and Service Management 17, 4 (2020), 2291--2304.

Digital Library

[2]

Farah Chahlaoui, Hamza Dahmouni, and Hassan El Alami. 2022. Multipathrouting based load-balancing in SDN networks. In 2022 5th Conference on Cloud and Internet of Things (CIoT). 180--185. https://doi.org/10.1109/CIoT53061.2022. 9766801

[3]

Long Chen, Bin Hu, Zhi-Hong Guan, Lian Zhao, and Xuemin Shen. 2021. Multiagent meta-reinforcement learning for adaptive multipath routing optimization. IEEE Transactions on Neural Networks and Learning Systems 33, 10 (2021), 5374-- 5386.

[4]

Kai-Cheng Chiu, Chien-Chang Liu, and Li-Der Chou. 2022. Reinforcement Learning-Based Service-Oriented Dynamic Multipath Routing in SDN. Wireless Communications and Mobile Computing 2022 (Jan. 2022), 1--16. https://doi.org/ 10.1155/2022/1330993

[5]

I. Cidon, R. Rom, and Y. Shavitt. 1999. Analysis of multi-path routing. IEEE/ACM Transactions on Networking 7, 6 (1999), 885--896. https://doi.org/10.1109/90.811453

Digital Library

[6]

Ashwini R Doke and K Sangeeta. 2018. Deep reinforcement learning based load balancing policy for balancing network traffic in datacenter environment. In 2018 Second International Conference on Green Computing and Internet of Things (ICGCIoT). IEEE, 1--5.

[7]

Sven Gronauer and Klaus Diepold. 2022. Multi-Agent Deep Reinforcement Learning: A Survey. Artif. Intell. Rev. 55, 2 (feb 2022), 895--943. https://doi.org/ 10.1007/s10462-021-09996-w

Digital Library

[8]

Xuancheng Guo, Hui Lin, Zhiyang Li, and Min Peng. 2019. Deep-reinforcementlearning- based QoS-aware secure routing for SDN-IoT. IEEE Internet of things journal 7, 7 (2019), 6242--6251.

[9]

Muhammad Ibrar, Lei Wang, Gabriel-Miro Muntean, Jenhui Chen, Nadir Shah, and Aamir Akbar. 2020. IHSF: An intelligent solution for improved performance of reliable and time-sensitive flows in hybrid SDN-based FC IoT systems. IEEE Internet of Things Journal 8, 5 (2020), 3130--3142.

[10]

Vijay Konda and John Tsitsiklis. 1999. Actor-critic algorithms. Advances in neural information processing systems 12 (1999).

[11]

Ryan Lowe, Yi I Wu, Aviv Tamar, Jean Harb, OpenAI Pieter Abbeel, and Igor Mordatch. 2017. Multi-agent actor-critic for mixed cooperative-competitive environments. Advances in neural information processing systems 30 (2017).

Digital Library

[12]

Antonios Makris, Abderrahmane Boudi, Massimo Coppola, Luís Cordeiro, Massimiliano Corsini, Patrizio Dazzi, Ferran Diego Andilla, Yago González Rozas, Manos Kamarianakis, Maria Pateraki, Thu Le Pham, Antonis Protopsaltis, Aravindh Raman, Alessandro Romussi, Luís Rosa, Elena Spatafora, Tarik Taleb, Theodoros Theodoropoulos, Konstantinos Tserpes, Enrico Zschau, and Uwe Herzog. 2021. Cloud for Holography and Augmented Reality. In 2021 IEEE 10th International Conference on Cloud Networking (CloudNet). 118--126. https: //doi.org/10.1109/CloudNet53349.2021.9657125

[13]

Dmitry Mukhutdinov, Andrey Filchenkov, Anatoly Shalyto, and Valeriy Vyatkin. 2019. Multi-agent deep learning for simultaneous optimization for time and energy in distributed routing system. Future Generation Computer Systems 94 (2019), 587--600.

Digital Library

[14]

Tran Anh Quang Pham, Yassine Hadjadj-Aoul, and Abdelkader Outtagarts. 2019. Deep reinforcement learning based qos-aware routing in knowledge-defined networking. In Quality, Reliability, Security and Robustness in Heterogeneous Systems: 14th EAI International Conference, Qshine 2018, Ho Chi Minh City, Vietnam, December 3--4, 2018, Proceedings 14. Springer, 14--26.

[15]

Justus Rischke, Peter Sossalla, Hani Salah, Frank HP Fitzek, and Martin Reisslein. 2020. QR-SDN: towards reinforcement learning states, actions, and rewards for direct flow routing in software-defined networks. IEEE Access 8 (2020), 174773-- 174791.

[16]

Giorgio Stampa, Marta Arias, David Sánchez-Charles, Victor Muntés-Mulero, and Albert Cabellos. 2017. A deep-reinforcement learning approach for softwaredefined networking routing optimization. arXiv preprint arXiv:1709.07080 (2017).

[17]

José Suárez-Varela, Albert Mestres, Junlin Yu, Li Kuang, Haoyu Feng, Pere Barlet- Ros, and Albert Cabellos-Aparicio. 2019. Routing based on deep reinforcement learning in optical transport networks. In Optical Fiber Communication Conference. Optica Publishing Group, M2A--6.

[18]

Penghao Sun, Yuxiang Hu, Julong Lan, Le Tian, and Min Chen. 2019. TIDE: Time-relevant deep reinforcement learning for routing optimization. Future Generation Computer Systems 99 (2019), 401--409.

Digital Library

[19]

Penghao Sun, Junfei Li, Julong Lan, Yuxiang Hu, and Xin Lu. 2018. RNN deep reinforcement learning for routing optimization. In 2018 IEEE 4th International Conference on Computer and Communications (ICCC). IEEE, 285--289.

[20]

Tarik Taleb, Abderrahmane Boudi, Luis Rosa, Luis Cordeiro, Theodoros Theodoropoulos, Konstantinos Tserpes, Patrizio Dazzi, Antonis I. Protopsaltis, and Richard Li. 2023. Toward Supporting XR Services: Architecture and Enablers. IEEE Internet of Things Journal 10, 4 (2023), 3567--3586. https://doi.org/10.1109/JIOT.2022. 3222103

[21]

Theodoros Theodoropoulos, Antonios Makris, Abderrahmane Boudi, Tarik Taleb, Uwe Herzog, Luis Rosa, Luis Cordeiro, Konstantinos Tserpes, Elena Spatafora, Alessandro Romussi, et al. 2022. Cloud-based xr services: A survey on relevant challenges and enabling technologies. Journal of Networking and Network Applications 2, 1 (2022), 1--22.

[22]

Theodoros Theodoropoulos, Angelos-Christos Maroudis, John Violos, and Konstantinos Tserpes. 2021. An encoder-decoder deep learning approach for multistep service traffic prediction. In 2021 IEEE Seventh International Conference on Big Data Computing Service and Applications (BigDataService). IEEE, 33--40.

[23]

Wenfeng Xia, Yonggang Wen, Chuan Heng Foh, Dusit Niyato, and Haiyong Xie. 2014. A survey on software-defined networking. IEEE Communications Surveys & Tutorials 17, 1 (2014), 27--51.

Digital Library

[24]

Changhe Yu, Julong Lan, Zehua Guo, and Yuxiang Hu. 2018. DROM: Optimizing the Routing in Software-Defined Networks With Deep Reinforcement Learning. IEEE Access 6 (2018), 64533--64539. https://doi.org/10.1109/ACCESS.2018.2877686

Cited By

Xiao YYang YYu HLiu J(2024)Scalable QoS-Aware Multipath Routing in Hybrid Knowledge-Defined Networking With Multiagent Deep Reinforcement LearningIEEE Transactions on Mobile Computing10.1109/TMC.2024.337919123:11(10628-10646)Online publication date: Nov-2024
https://doi.org/10.1109/TMC.2024.3379191
Benmerar TTheodoropoulos TFevereiro DRosa LRodrigues JTaleb TBarone PGiuliani GTserpes KCordeiro L(2024)Towards establishing intelligent multi-domain edge orchestration for highly distributed immersive services: a virtual touring use caseCluster Computing10.1007/s10586-024-04413-727:4(4223-4253)Online publication date: 26-Apr-2024
https://dl.acm.org/doi/10.1007/s10586-024-04413-7
Theodoropoulos TRosa LBenzaid CGray PMarin EMakris ACordeiro LDiego FSorokin PGirolamo MBarone PTaleb TTserpes K(2023)Security in Cloud-Native Services: A SurveyJournal of Cybersecurity and Privacy10.3390/jcp30400343:4(758-793)Online publication date: 26-Oct-2023
https://doi.org/10.3390/jcp3040034

Recommendations

Deep reinforcement learning for multi-agent interaction
Multi-agent systems research in the United Kingdom

The development of autonomous agents which can interact with other agents to accomplish a given task is a core area of research in artificial intelligence and machine learning. Towards this goal, the Autonomous Agents Research Group develops novel ...
Assured Deep Multi-Agent Reinforcement Learning for Safe Robotic Systems
Agents and Artificial Intelligence
Abstract
Using multi-agent reinforcement learning to find solutions to complex decision-making problems in shared environments has become standard practice in many scenarios. However, this is not the case in safety-critical scenarios, where the ...
A multi-agent reinforcement learning with weighted experience sharing
ICIC'11: Proceedings of the 7th international conference on Advanced Intelligent Computing Theories and Applications: with aspects of artificial intelligence

Reinforcement Learning, also sometimes called learning by rewards and punishments is the problem faced by an agent that must learn behavior through trial-and-error interactions with a dynamic environment [1]. With repeated trials however, it is expected ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

FRAME '23: Proceedings of the 3rd Workshop on Flexible Resource and Application Management on the Edge

August 2023

50 pages

ISBN:9798400701641

DOI:10.1145/3589010

General Chair:
Hanna Kavalionak
ISTI-CNR, Italy
,
Program Chairs:
Ioannis Kontopoulos
Harokopio University, Greece
,
Massimo Coppola
ISTI-CNR, Italy
,
Luca Ferrucci
University of Pisa, Italy

Copyright © 2023 Owner/Author.

This work is licensed under a Creative Commons Attribution International 4.0 License.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 15 August 2023

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Horizon 2020

Conference

HPDC '23

Sponsor:

HPDC '23: The 32nd International Symposium on High-Performance Parallel and Distributed Computing

June 20, 2023

FL, Orlando, USA

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
345
Total Downloads

Downloads (Last 12 months)281
Downloads (Last 6 weeks)47

Reflects downloads up to 26 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Xiao YYang YYu HLiu J(2024)Scalable QoS-Aware Multipath Routing in Hybrid Knowledge-Defined Networking With Multiagent Deep Reinforcement LearningIEEE Transactions on Mobile Computing10.1109/TMC.2024.337919123:11(10628-10646)Online publication date: Nov-2024
https://doi.org/10.1109/TMC.2024.3379191
Benmerar TTheodoropoulos TFevereiro DRosa LRodrigues JTaleb TBarone PGiuliani GTserpes KCordeiro L(2024)Towards establishing intelligent multi-domain edge orchestration for highly distributed immersive services: a virtual touring use caseCluster Computing10.1007/s10586-024-04413-727:4(4223-4253)Online publication date: 26-Apr-2024
https://dl.acm.org/doi/10.1007/s10586-024-04413-7
Theodoropoulos TRosa LBenzaid CGray PMarin EMakris ACordeiro LDiego FSorokin PGirolamo MBarone PTaleb TTserpes K(2023)Security in Cloud-Native Services: A SurveyJournal of Cybersecurity and Privacy10.3390/jcp30400343:4(758-793)Online publication date: 26-Oct-2023
https://doi.org/10.3390/jcp3040034

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents