Article

Real-Time Lane Configuration with Coordinated Reinforcement Learning

Authors:

Udesh Gunarathna,

Shanika Karunasekara,

Renata Borovica-GajicAuthors Info & Claims

Machine Learning and Knowledge Discovery in Databases: Applied Data Science Track: European Conference, ECML PKDD 2020, Ghent, Belgium, September 14–18, 2020, Proceedings, Part IV

Pages 291 - 307

https://doi.org/10.1007/978-3-030-67667-4_18

Published: 14 September 2020 Publication History

Abstract

Changing lane configuration of roads, based on traffic patterns, is a proven solution for improving traffic throughput. Traditional lane-direction configuration solutions assume pre-known traffic patterns, hence are not suitable for real-world applications as they are not able to adapt to changing traffic conditions. We propose a dynamic lane configuration solution for improving traffic flow using a two-layer, multi-agent architecture, named Coordinated Learning-based Lane Allocation (CLLA). At the bottom-layer, a set of reinforcement learning agents find a suitable configuration of lane-directions around individual road intersections. The lane-direction changes proposed by the reinforcement learning agents are then coordinated by the upper level agents to reduce the negative impact of the changes on other parts of the road network. CLLA is the first work that allows city-wide lane configuration while adapting to changing traffic conditions. Our experimental results show that CLLA can reduce the average travel time in congested road networks by 20% compared to an uncoordinated reinforcement learning approach.

References

[1]

Arel I, Liu C, Urbanik T, and Kohls A Reinforcement learning-based multi-agent system for network traffic signal control IET Intel. Transport Syst. 2010 4 2 128-135

[2]

Aslani M, Seipel S, Mesgari MS, and Wiering M Traffic signal optimization through discrete and continuous reinforcement learning with robustness analysis in downtown Tehran Adv. Eng. Inform. 2018 38 639-655

Digital Library

[3]

Boutilier, C.: Planning, learning and coordination in multiagent decision processes. In: Theoretical Aspects of Rationality and Knowledge, pp. 195–210 (1996)

[4]

Chu, K.F., Lam, A.Y.S., Li, V.O.K.: Dynamic lane reversal routing and scheduling for connected autonomous vehicles. In: International Smart Cities Conference, pp. 1–6 (2017)

[5]

El-Tantawy S and Abdulhai B Multi-agent reinforcement learning for integrated network of adaptive traffic signal controllers ITSC 2012 14 3 1140-1150

[6]

Fleischer L and Skutella M Quickest flows over time SIAM J. Comput. 2007 36 6 1600-1630

Digital Library

[7]

Ford LR and Fulkerson DR Constructing maximal dynamic flows from static flows Oper. Res. 1958 6 3 419-433

Digital Library

[8]

Guestrin, C., Lagoudakis, M.G., Parr, R.: Coordinated reinforcement learning. In: International Conference on Machine Learning, pp. 227–234 (2002)

[9]

Hausknecht, M., Au, T., Stone, P., Fajardo, D., Waller, T.: Dynamic lane reversal in traffic management. In: ITSC, pp. 1929–1934 (2011)

[10]

Köhler, E., Möhring, R.H., Skutella, M.: Traffic networks and flows over time, pp. 166–196 (2009)

[11]

Lambert L and Wolshon B Characterization and comparison of traffic flow on reversible roadways J. Adv. Transp. 2010 44 2 113-122

[12]

Levin MW and Boyles SD A cell transmission model for dynamic lane reversal with autonomous vehicles Transp. Res. Part C: Emerg. Technol. 2016 68 126-143

[13]

Mannion P, Duggan J, and Howley E McCluskey TL, Kotsialos A, Müller JP, Klügl F, Rana O, and Schumann R An experimental review of reinforcement learning algorithms for adaptive traffic signal control Autonomic Road Transport Support Systems 2016 Cham Springer 47-66

[14]

Narla SR The evolution of connected vehicle technology: from smart drivers to smart cars to... self-driving cars ITE J. 2013 83 22-26

[15]

Ramamohanarao K et al. SMARTS: scalable microscopic adaptive road traffic simulator ACM Trans. Intell. Syst. Technol. 2016 8 2 1-22

Digital Library

[16]

Ravishankar NR and Vijayakumar MV Reinforcement learning algorithms: survey and classification Indian J. Sci. Technol. 2017 10 1 1-8

[17]

Sutton RS and Barto AG Introduction to Reinforcement Learning 1998 1 Cambridge MIT Press

Digital Library

[18]

Walraven E, Spaan MT, and Bakker B Traffic flow optimization: a reinforcement learning approach Eng. Appl. Artif. Intell. 2016 52 203-212

Digital Library

[19]

Watkins CJ and Dayan P Technical note: Q-learning Mach. Learn. 1992 8 279-292

Digital Library

[20]

Wolshon B and Lambert L Planning and operational practices for reversible roadways ITE J. 2006 76 38-43

[21]

Wu JJ, Sun HJ, Gao ZY, and Zhang HZ Reversible lane-based traffic network optimization with an advanced traveller information system Eng. Optim. 2009 41 1 87-97

[22]

Yau KLA, Qadir J, Khoo HL, Ling MH, and Komisarczuk P A survey on reinforcement learning models and algorithms for traffic signal control ACM Comput. Surv. (CSUR) 2017 50 3 1-38

Digital Library

Cited By

Gunarathna UXie HTanin EKarunasekera SBorovica-Gajic R(2023)Real-time Road Network Optimization with Coordinated Reinforcement LearningACM Transactions on Intelligent Systems and Technology10.1145/360337914:4(1-30)Online publication date: 21-Jul-2023
https://dl.acm.org/doi/10.1145/3603379
Muthugama LXie HTanin EKarunasekera SGunarathna URenz MSarwat M(2022)Concurrent optimization of safety and traffic flow using deep reinforcement learning for autonomous intersection managementProceedings of the 30th International Conference on Advances in Geographic Information Systems10.1145/3557915.3561018(1-12)Online publication date: 1-Nov-2022
https://dl.acm.org/doi/10.1145/3557915.3561018
Muthugama LXie HTanin EKarunasekera S(2022)Real-time road safety optimization through network-level data managementGeoinformatica10.1007/s10707-022-00473-227:3(491-523)Online publication date: 22-Aug-2022
https://dl.acm.org/doi/10.1007/s10707-022-00473-2

Recommendations

Multi-Vehicle Mixed Reality Reinforcement Learning for Autonomous Multi-Lane Driving
AAMAS '20: Proceedings of the 19th International Conference on Autonomous Agents and MultiAgent Systems

Autonomous driving promises to transform road transport. Multi-vehicle and multi-lane scenarios, however, present unique challenges due to constrained navigation and unpredictable vehicle interactions. Learning-based methods-such as deep reinforcement ...
Reward Shaping in Episodic Reinforcement Learning
AAMAS '17: Proceedings of the 16th Conference on Autonomous Agents and MultiAgent Systems

Recent advancements in reinforcement learning confirm that reinforcement learning techniques can solve large scale problems leading to high quality autonomous decision making. It is a matter of time until we will see large scale applications of ...
Reinforcement Learning for Cooperative Overtaking
AAMAS '19: Proceedings of the 18th International Conference on Autonomous Agents and MultiAgent Systems

This paper solves the cooperative overtaking problem in autonomous driving using reinforcement learning techniques. Learning in such a situation is challenging due to vehicular mobility, which renders a continuously changing environment for each ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Guide Proceedings

Machine Learning and Knowledge Discovery in Databases: Applied Data Science Track: European Conference, ECML PKDD 2020, Ghent, Belgium, September 14–18, 2020, Proceedings, Part IV

Sep 2020

611 pages

ISBN:978-3-030-67666-7

DOI:10.1007/978-3-030-67667-4

Editors:
Yuxiao Dong
Microsoft Research, Redmond, WA, USA
,
Dunja Mladenić
Jožef Stefan Institute, Ljubljana, Slovenia
,
Craig Saunders
Amazon Alexa Knowledge, Cambridge, UK

© Springer Nature Switzerland AG 2021.

Publisher

Springer-Verlag

Berlin, Heidelberg

Publication History

Published: 14 September 2020

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 24 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Gunarathna UXie HTanin EKarunasekera SBorovica-Gajic R(2023)Real-time Road Network Optimization with Coordinated Reinforcement LearningACM Transactions on Intelligent Systems and Technology10.1145/360337914:4(1-30)Online publication date: 21-Jul-2023
https://dl.acm.org/doi/10.1145/3603379
Muthugama LXie HTanin EKarunasekera SGunarathna URenz MSarwat M(2022)Concurrent optimization of safety and traffic flow using deep reinforcement learning for autonomous intersection managementProceedings of the 30th International Conference on Advances in Geographic Information Systems10.1145/3557915.3561018(1-12)Online publication date: 1-Nov-2022
https://dl.acm.org/doi/10.1145/3557915.3561018
Muthugama LXie HTanin EKarunasekera S(2022)Real-time road safety optimization through network-level data managementGeoinformatica10.1007/s10707-022-00473-227:3(491-523)Online publication date: 22-Aug-2022
https://dl.acm.org/doi/10.1007/s10707-022-00473-2

View Options

View options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents