research-article

Real-time Road Network Optimization with Coordinated Reinforcement Learning

Authors:

Udesh Gunarathna,

Shanika Karunasekera,

Renata Borovica-GajicAuthors Info & Claims

ACM Transactions on Intelligent Systems and Technology, Volume 14, Issue 4

Article No.: 72, Pages 1 - 30

https://doi.org/10.1145/3603379

Published: 21 July 2023 Publication History

Abstract

Dynamic road network optimization has been used for improving traffic flow in an infrequent and localized manner. The development of intelligent systems and technology provides an opportunity to improve the frequency and scale of dynamic road network optimization. However, such improvements are hindered by the high computational complexity of the existing algorithms that generate the optimization plans. We present a novel solution that integrates machine learning and road network optimization. Our solution consists of two complementary parts. The first part is an efficient algorithm that uses reinforcement learning to find the best road network configurations at real-time. The second part is a dynamic routing mechanism, which helps connected vehicles adapt to the change of the road network. Our extensive experimental results demonstrate that the proposed solution can substantially reduce the average travel time in a variety of scenarios, whilst being computationally efficient and hence applicable to real-life situations.

References

[1]

Konstantinos Ampountolas, Joana Alves dos Santos, and Rodrigo Castelan Carlson. 2020. Motorway tidal flow lane control. IEEE Transactions on Intelligent Transportation Systems 21, 4 (2020), 1687–1696.

[2]

I. Arel, C. Liu, T. Urbanik, and A.G. Kohls. 2010. Reinforcement learning-based multi-agent system for network traffic signal control. IET Intelligent Transport Systems 4, 2 (2010), 128–135.

[3]

Craig Boutilier. 1996. Planning, learning and coordination in multiagent decision processes. In Proceedings of the Theoretical Aspects of Rationality and Knowledge. 195–210.

Digital Library

[4]

Henrik Braconier, Mauro Pisu, and Debra Bloch. 2013. The performance of road transport infrastructure and its links to policies. OECD Economics Department Working Papers1016 (2013), 1–87.

[5]

Zhiguang Cao, Hongliang Guo, and Jie Zhang. 2017. A multiagent-based approach for vehicle routing by considering both arriving on time and total travel time. ACM Transactions on Intelligent Systems and Technology 9, 3 (2017), 1–21.

Digital Library

[6]

Chacha Chen, Hua Wei, Nan Xu, Guanjie Zheng, Ming Yang, Yuanhao Xiong, Kai Xu, and Zhenhui Li. 2020. Toward a thousand lights: Decentralized deep reinforcement learning for large-scale traffic signal control. In Proceedings of the AAAI Conference on Artificial Intelligence. 3414–3421.

[7]

Yi-Chang Chiu, Jon Bottom, Michael Mahut, Alexander Paz, Ramachandran Balakrishna, Travis Waller, and Jim Hicks. 2011. Dynamic traffic assignment: A primer. Dynamic Traffic Assignment: A Primer (2011), 1–62. https://www.trb.org/Publications/Blurbs/165620.aspx.

[8]

K. F. Chu, A. Y. S. Lam, and V. O. K. Li. 2020. Dynamic lane reversal routing and scheduling for connected autonomous vehicles. IEEE Transactions on Knowledge and Data Engineering 34, 4 (2020), 1544–1561.

[9]

Mark S. Daskin. 1985. Urban transportation networks: Equilibrium analysis with mathematical programming methods. Transportation Science 19, 4 (1985), 463–466.

Digital Library

[10]

Kakan Chandra Dey, Anjan Rayamajhi, Mashrur Chowdhury, Parth Bhavsar, and James Martin. 2016. Vehicle-to-vehicle (V2V) and vehicle-to-infrastructure (V2I) communication in a heterogeneous wireless network–Performance evaluation. Transportation Research Part C: Emerging Technologies 68 (2016), 168–184.

[11]

Samah El-Tantawy, Baher Abdulhai, and Hossam Abdelgawad. 2013. Multiagent reinforcement learning for integrated network of adaptive traffic signal controllers (MARLIN-ATSC): Methodology and large-scale application on downtown toronto. IEEE Transactions on Intelligent Transportation Systems 14, 3 (2013), 1140–1150.

Digital Library

[12]

L. R. Ford and D. R. Fulkerson. 1958. Constructing maximal dynamic flows from static flows. Operations Research 6, 3 (1958), 419–433.

Digital Library

[13]

Udesh Gunarathna, Hairuo Xie, Egemen Tanin, Shanika Karunasekera, and Renata Borovica-Gajic. 2021. Real-time lane configuration with coordinated reinforcement learning. Machine Learning and Knowledge Discovery in Databases: Applied Data Science Track, Cham, 291–307.

Digital Library

[14]

Pengzhan Guo, Keli Xiao, Zeyang Ye, and Wei Zhu. 2021. Route optimization via environment-aware deep network and reinforcement learning. 12, 6 (2021).

[15]

M. Hausknecht, T. Au, P. Stone, D. Fajardo, and T. Waller. 2011. Dynamic lane reversal in traffic management. In Proceedings of the IEEE Conference on Intelligent Transportation Systems. 1929–1934.

[16]

Mehdi Keyvan-Ekbatani, Anastasios Kouvelas, Ioannis Papamichail, and Markos Papageorgiou. 2012. Exploiting the fundamental diagram of urban networks for feedback-based gating. Transportation Research Part B: Methodological 46, 10 (2012), 1393–1403.

[17]

Sangho Kim, Shashi Shekhar, and Manki Min. 2008. Contraflow transportation network reconfiguration for evacuation route planning. IEEE Transactions on Knowledge and Data Engineering 20, 8 (2008), 1115–1129.

[18]

Ekkehard Köhler, Rolf H. Möhring, and Martin Skutella. 2009. Traffic networks and flows over time. Algorithmics of Large and Complex Networks: Design, Analysis, and Simulation. Lecture Notes in Computer Science 5515 (2009), 166–196.

Digital Library

[19]

Laurence Lambert and Brian Wolshon. 2010. Characterization and comparison of traffic flow on reversible roadways. Journal of Advanced Transportation 44, 2 (2010), 113–122.

[20]

Michael W. Levin and Stephen D. Boyles. 2016. A cell transmission model for dynamic lane reversal with autonomous vehicles. Transportation Research Part C: Emerging Technologies 68 (2016), 126–143.

[21]

Fanyu Meng, Sze Chun Wong, W. Wong, and Y. C. Li. 2017. Estimation of scaling factors for traffic counts based on stationary and mobile sources of data. International Journal of Intelligent Transportation Systems Research 15 (2017), 180–191.

[22]

N. Mitrovic, I. Dakic, and A. Stevanovic. 2020. Combined alternate-direction lane assignment and reservation-based intersection control. IEEE Transactions on Intelligent Transportation Systems 21, 4 (2020), 1779–1789.

[23]

Carlos A. Moncada, Santiago Cardona, and Diego A. Escobar. 2018. Saving travel time as an urban planning instrument. Case study: Manizales, Colombia. Modern Applied Science 12, 6 (2018), 44–57.

[24]

Hoang Nguyen, Chen Cai, and Fang Chen. 2017. Automatic classification of traffic incident’s severity using machine learning approaches. IET Intelligent Transport Systems 11, 10 (2017), 615–623.

[25]

United States Department of Transportation. 2022. TI04: Infrastructure-Provided Trip Planning and Route. Retrieved February 01, 2023 from https://www.arc-it.net/html/servicepackages/sp163.html.

[26]

Srinivas Peeta and Hani S. Mahmassani. 1995. System optimal and user equilibrium time-dependent traffic assignment in congested networks. Annals of Operations Research 60, 1 (1995), 81–113.

[27]

Kotagiri Ramamohanarao, Hairuo Xie, Lars Kulik, Shanika Karunasekera, Egemen Tanin, Rui Zhang, and Eman Bin Khunayn. 2016. SMARTS: Scalable microscopic adaptive road traffic simulator. ACM Transactions on Intelligent Systems and Technology 8, 2 (2016), 22 pages.

[28]

N. R. Ravishankar and M. V. Vijayakumar. 2017. Reinforcement learning algorithms: Survey and classification. Indian Journal of Science and Technology 10, 1 (2017), 1–8.

[29]

Richard S. Sutton and Andrew G. Barto. 1998. Introduction to Reinforcement Learning (1st. ed.). MIT Press.

Digital Library

[30]

David Alexander Tedjopurnomo, Zhifeng Bao, Baihua Zheng, Farhana Choudhury, and A. K. Qin. 2020. A survey on modern deep neural network for traffic prediction: Trends, methods and challenges. IEEE Transactions on Knowledge and Data Engineering(2020).

[31]

E. Uhlemann. 2016. Connected-vehicles applications are emerging [connected vehicles]. IEEE Vehicular Technology Magazine 11, 1 (2016), 25–96.

[32]

Christopher J. C. H. Watkins and Peter Dayan. 1992. Technical note: Q-learning. Machine Learning 8, 4 (1992), 279–292.

Digital Library

[33]

Stephan Winter. 2002. Route specifications with a linear dual graph. In Proceedings of the Advances in Spatial Data Handling. 329–338.

[34]

J. J. Wu, H. J. Sun, Z. Y. Gao, and H. Z. Zhang. 2009. Reversible lane-based traffic network optimization with an advanced traveller information system. Engineering Optimization 41, 1 (2009), 87–97.

[35]

Hongsuk Yi, HeeJin Jung, and Sanghoon Bae. 2017. Deep neural networks for traffic flow prediction. In Proceedings of the 2017 IEEE International Conference on Big Data and Smart Computing. 328–331. DOI:

Cited By

Lozano-Ramírez NSánchez OCarrasco-Beltrán DVidal-Méndez SCastañeda K(2023)Digitalization and Sustainability in Linear Projects Trends: A Bibliometric AnalysisSustainability10.3390/su15221596215:22(15962)Online publication date: 15-Nov-2023
https://doi.org/10.3390/su152215962

Index Terms

Real-time Road Network Optimization with Coordinated Reinforcement Learning
1. Computing methodologies
  1. Machine learning
    1. Learning paradigms
      1. Reinforcement learning
2. Information systems
  1. Information systems applications
    1. Spatial-temporal systems

Recommendations

Automating Coordinated Autonomous Vehicle Control
AAMAS '20: Proceedings of the 19th International Conference on Autonomous Agents and MultiAgent Systems

Recently there has been increasing academic and industry research attention on producing adaptive control systems for autonomous vehicles. To accommodate such autonomous vehicles there have been proposals that current road and highway infrastructure ...
Maneuver-based trajectory planning for highly autonomous vehicles on real road with traffic and driver interaction

This paper presents the design and first test on a simulator of a vehicle trajectory-planning algorithm that adapts to traffic on a lane-structured infrastructure such as highways. The proposed algorithm is designed to run on a fail-safe embedded ...
Real-Time Lane Configuration with Coordinated Reinforcement Learning
Machine Learning and Knowledge Discovery in Databases: Applied Data Science Track
Abstract
Changing lane configuration of roads, based on traffic patterns, is a proven solution for improving traffic throughput. Traditional lane-direction configuration solutions assume pre-known traffic patterns, hence are not suitable for real-world ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Transactions on Intelligent Systems and Technology

ACM Transactions on Intelligent Systems and Technology Volume 14, Issue 4

August 2023

481 pages

ISSN:2157-6904

EISSN:2157-6912

DOI:10.1145/3596215

Editor:
Huan Liu
Arizona State University, USA

Issue’s Table of Contents

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 21 July 2023

Online AM: 10 June 2023

Accepted: 04 May 2023

Revised: 16 March 2023

Received: 25 January 2022

Published in TIST Volume 14, Issue 4

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
271
Total Downloads

Downloads (Last 12 months)195
Downloads (Last 6 weeks)17

Reflects downloads up to 30 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Lozano-Ramírez NSánchez OCarrasco-Beltrán DVidal-Méndez SCastañeda K(2023)Digitalization and Sustainability in Linear Projects Trends: A Bibliometric AnalysisSustainability10.3390/su15221596215:22(15962)Online publication date: 15-Nov-2023
https://doi.org/10.3390/su152215962

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Full Text

View this article in Full Text.

Media

Figures

Other

Tables

View full text|Download PDF

View Issue’s Table of Contents