Nothing Special   »   [go: up one dir, main page]

skip to main content
10.1145/3678717.3691262acmconferencesArticle/Chapter ViewAbstractPublication PagesgisConference Proceedingsconference-collections
research-article
Free access

Adaptive Dispatching of Mobile Charging Stations using Multi-Agent Graph Convolutional Cooperative-Competitive Reinforcement Learning

Published: 22 November 2024 Publication History

Abstract

Battery electric vehicles (BEV) offer an opportunity to decrease transportation and mobility emissions significantly. The availability of charging station networks and infrastructure is crucial for the proliferation of BEVs. While the expansion of the charging networks is still slow, optimal utilization of the existing infrastructure and dispatching of mobile charging stations can serve as a bypass while more charging stations are built. In this work, we propose a novel multi-agent reinforcement learning - AdapMCS - approach for optimizing the adaptive dispatching of mobile charging stations to maximize the number of served charging requests by a charging station operator while improving the customer experience. By combining graph neural networks with reinforcement learning our approach is able to adapt to dynamic spatio-temporal changes in the demand distribution, for example, during big events such as concerts or fairs. Furthermore, we conduct a thorough evaluation using a publicly available real-world dataset and simulation of dynamic demand distribution changes. The results show that our adaptive dispatching approach is able to deal with the demand shifts and achieve significant gains for both customers, in terms of reducing waiting and charging times, and operators, in terms of increasing their profit.

References

[1]
2023. Open Charge Map. https://openchargemap.org/. Accessed: 2023-10-16.
[2]
Shahab Afshar, Pablo Macedo, Farog Mohamed, and Vahid Disfani. 2021. Mobile charging stations for electric vehicles---A review. Renewable and Sustainable Energy Reviews 152 (2021).
[3]
International Energy Agency. 2022. Global Electric Vehicle Outlook 2022. https://iea.blob.core.windows.net/assets/ad8fb04c-4f75-42fc-973a-6e54c8a4449a/GlobalElectricVehicleOutlook2022.pdf. Accessed: 2023-10-16.
[4]
Lars Buitinck, Gilles Louppe, Mathieu Blondel, Fabian Pedregosa, Andreas Mueller, Olivier Grisel, Vlad Niculae, Peter Prettenhofer, Alexandre Gramfort, Jaques Grobler, Robert Lay ton, Jake VanderPlas, Arnaud Joly, Brian Holt, and Gaël Varoquaux. 2013. API design for machine learning software: experiences from the scikit-learn project. In ECML PKDD Workshop: Languages for Data Mining and Machine Learning. 108--122.
[5]
Ryan G Conrad and Miguel Andres Figliozzi. 2011. The recharging vehicle routing problem. In Proceedings of the 2011 industrial engineering research conference, Vol. 8. IISE Norcross, GA.
[6]
Shaohua Cui, Baozhen Yao, Gang Chen, Chao Zhu, and Bin Yu. 2020. The multi-mode mobile charging service based on electric vehicle spatiotemporal distribution. Energy 198 (2020), 117302.
[7]
Xin Du, Jiahai Wang, and Siyuan Chen. 2023. Multi-Agent Meta-Reinforcement Learning with Coordination and Reward Shaping for Traffic Signal Control. In Advances in Knowledge Discovery and Data Mining, PAKDD 2023. Springer Nature Switzerland, Cham, 349--360.
[8]
Elli. 2022. Charging Conditions 2023. https://www.elli.eco/downloads/en/drive-free.pdf. Accessed: 2023-10-16.
[9]
Fraunhofer-Institut für Solare Energiesysteme ISE. 2023. Stromproduktion und Börsenstrompreise in Portugal 2023. https://www.energy-charts.info/charts/price_spot_market/chart.htm?l=de&c=PT&year=2023&interval=year &year=2023&interval=year &interval=year [Online; accessed 09-November-2023].
[10]
Sven Gronauer and Klaus Diepold. 2022. Multi-agent deep reinforcement learning: a survey. Artificial Intelligence Review (2022), 1--49.
[11]
Dale Hall, Sandra Wappelhorst, Peter Mock, and Nic Lutsey. 2020. European Electric Vehicle Factbook 2019/2020. https://theicct.org/wp-content/uploads/2021/06/EV-EU-Factbook-2020.pdf. Accessed: 2023-10-16.
[12]
Shengyi Huang and Santiago Ontañón. 2022. A Closer Look at Invalid Action Masking in Policy Gradient Algorithms. In Proceedings of the Thirty-Fifth International Florida Artificial Intelligence Research Society Conference, FLAIRS 2022, Hutchinson Island, Jensen Beach, Florida, USA, May 15-18, 2022.
[13]
Soi Jeon and Dae-Hyun Choi. 2021. Optimal energy management framework for truck-mounted mobile charging stations considering power distribution system operating conditions. Sensors 21, 8 (2021), 2798.
[14]
Jiechuan Jiang, Chen Dun, Tiejun Huang, and Zongqing Lu. 2020. Graph Convolutional Reinforcement Learning. In ICLR.
[15]
Thomas N. Kipf and Max Welling. [n. d.]. Semi-Supervised Classification with Graph Convolutional Networks. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings.
[16]
Vijay Konda and John Tsitsiklis. 1999. Actor-critic algorithms. Advances in neural information processing systems 12 (1999).
[17]
Maxime Lenormand, Aleix Bassolas, and José J Ramasco. 2016. Systematic comparison of trip distribution laws and models. Journal of Transport Geography 51 (2016), 158--169.
[18]
Boyu Li, Ting Guo, Yang Wang, Amir H. Gandomi, and Fang Chen. 2021. Adaptive Graph Co-Attention Networks for Traffic Forecasting. In Advances in Knowledge Discovery and Data Mining, PAKDD 2021. Springer International Publishing, Cham, 263--276.
[19]
Yongheng Liang, Hejun Wu, and Haitao Wang. 2022. ASM-PPO: Asynchronous and Scalable Multi-Agent PPO for Cooperative Charging. In Proceedings of the 21st International Conference on Autonomous Agents and Multiagent Systems. 798--806.
[20]
Linfeng Liu, Zhiyuan Xi, Kun Zhu, Ran Wang, and Ekram Hossain. 2022. Mobile charging station placements in internet of electric vehicles: A federated learning approach. IEEE Transactions on Intelligent Transportation Systems 23, 12 (2022), 24561--24577.
[21]
Linfeng Liu, Houqian Zhang, Jia Xu, and Ping Wang. 2024. Providing Active Charging Services: An Assignment Strategy With Profit-Maximizing Heat Maps for Idle Mobile Charging Stations. IEEE Transactions on Mobile Computing 23, 3 (2024), 2139--2152. https://doi.org/10.1109/TMC.2023.3247441
[22]
Ryan Lowe, Yi I Wu, Aviv Tamar, Jean Harb, OpenAI Pieter Abbeel, and Igor Mordatch. 2017. Multi-agent actor-critic for mixed cooperative-competitive environments. Advances in neural information processing systems 30 (2017).
[23]
Luis Moreira-Matias, Michel Ferreira, and Joao Mendes-Moreira. 2015. Taxi Service Trajectory - Prediction Challenge, ECML PKDD. UCI Machine Learning Repository.
[24]
Jianing Ni, Rupeng Liang, and Hao Wu. 2021. Collaborative Mobile Charging Vehicles Placement: A Reinforcement Learning Approach. In 23rd Int Conf on High Performance Computing & Communications; 7th Int Conf on Data Science & Systems; 19th Int Conf on Smart City; 7th Int Conf on Dependability in Sensor, Cloud & Big Data Systems & Application (HPCC/DSS/SmartCity/DependSys). IEEE, 920--926.
[25]
Yaru Niu, Rohan R Paleja, and Matthew C Gombolay. 2021. Multi-Agent Graph-Attention Communication and Teaming. In AAMAS. 964--973.
[26]
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. 2019. PyTorch: An Imperative Style, High-Performance Deep Learning Library. In Advances in Neural Information Processing Systems 32. Curran Associates, Inc., 8024--8035. http://papers.neurips.cc/paper/9015-pytorch-animperative-style-high-performance-deep-learning-library.pdf
[27]
H Qin, X Su, T Ren, and Z Luo. [n. d.]. A review on the electric vehicle routing problems: variants and algorithms. Front. Eng. Manag. 8 (3), 370--389 (2021).
[28]
Ubaid Qureshi, Arnob Ghosh, and Bijaya Ketan Panigrahi. 2022. Scheduling and routing of mobile charging stations with stochastic travel times to service heterogeneous spatiotemporal electric vehicle charging requests with time windows. IEEE Transactions on Industry Applications 58, 5 (2022), 6546--6556.
[29]
Amit Roy, Kashob Kumar Roy, Amin Ahsan Ali, M. Ashraful Amin, and A. K. M. Mahbubur Rahman. 2021. SST-GNN: Simplified Spatio-Temporal Traffic Forecasting Model Using Graph Neural Network. In Advances in Knowledge Discovery and Data Mining. 90--102.
[30]
Franco Scarselli, Marco Gori, Ah Chung Tsoi, Markus Hagenbuchner, and Gabriele Monfardini. 2008. The graph neural network model. IEEE transactions on neural networks 20, 1 (2008), 61--80.
[31]
Richard S Sutton. 2018. Reinforcement learning: An introduction. A Bradford Book (2018).
[32]
Peng Tang, Fang He, Xi Lin, and Meng Li. 2020. Online-to-offline mobile charging system for electric vehicles: Strategic planning and online operation. Transportation Research Part D: Transport and Environment 87 (2020), 102522.
[33]
Leonie von Wahl, Nicolas Tempelmeier, Ashutosh Sao, and Elena Demidova. 2022. Reinforcement Learning-Based Placement of Charging Stations in Urban Road Networks. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (Washington DC, USA) (KDD '22). ACM, New York, NY, USA, 3992--4000.
[34]
Chao Yu, Akash Velu, Eugene Vinitsky, Jiaxuan Gao, Yu Wang, Alexandre Bayen, and Yi Wu. 2022. The surprising effectiveness of ppo in cooperative multi-agent games. Advances in Neural Information Processing Systems 35 (2022), 24611--24624.
[35]
Hengjing Zhang, Bofu Jin, Jian Li, Jingyi Gao, Jiajun Zhao, Mingyao Hou, Guo Yu, and Hengchang Liu. 2019. Optimized scheduling for urban-scale mobile charging vehicle. In 2019 2nd World Symposium on Communication Engineering (WSCE). IEEE, 164--172.
[36]
Weijia Zhang, Hao Liu, Jindong Han, Yong Ge, and Hui Xiong. 2022. Multi-Agent Graph Convolutional Reinforcement Learning for Dynamic Electric Vehicle Charging Pricing. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (Washington DC, USA) (KDD '22). ACM, New York, NY, USA, 2471--2481.
[37]
Yuzhen Zhou, Jincai Huang, Jianmai Shi, Rui Wang, and Kuihua Huang. 2021. The electric vehicle routing problem with partial recharge and vehicle recycling. Complex & Intelligent Systems 7 (2021), 1445--1458.

Index Terms

  1. Adaptive Dispatching of Mobile Charging Stations using Multi-Agent Graph Convolutional Cooperative-Competitive Reinforcement Learning

      Recommendations

      Comments

      Please enable JavaScript to view thecomments powered by Disqus.

      Information & Contributors

      Information

      Published In

      cover image ACM Conferences
      SIGSPATIAL '24: Proceedings of the 32nd ACM International Conference on Advances in Geographic Information Systems
      October 2024
      743 pages
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

      Sponsors

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 22 November 2024

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. Graph Neural Networks
      2. Mobile Charging Stations
      3. Multi-Agent
      4. Reinforcement Learning

      Qualifiers

      • Research-article
      • Research
      • Refereed limited

      Funding Sources

      Conference

      SIGSPATIAL '24
      Sponsor:

      Acceptance Rates

      SIGSPATIAL '24 Paper Acceptance Rate 37 of 122 submissions, 30%;
      Overall Acceptance Rate 257 of 1,238 submissions, 21%

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • 0
        Total Citations
      • 28
        Total Downloads
      • Downloads (Last 12 months)28
      • Downloads (Last 6 weeks)28
      Reflects downloads up to 07 Dec 2024

      Other Metrics

      Citations

      View Options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Login options

      Media

      Figures

      Other

      Tables

      Share

      Share

      Share this Publication link

      Share on social media