Intersec2vec-TSC: Intersection Representation Learning for Large-Scale Traffic Signal Control
Pages 7044 - 7056
Abstract
The intersection network constitutes the basic skeleton of the urban traffic environment, and informative representation of the intersection plays an important role in supporting the wide variety of applications in the intelligent transportation system. In this paper, we propose the Intersection to Vector model, named Intersec2vec, to achieve an accurate, efficient, and low-dimensional representation of each intersection in the large-scale intersection network. It introduces structural and temporal modules with attention mechanisms to specifically represent the evolution of intersection features in the spatiotemporal dimension, ensuring that intersections with stronger correlations have a higher probability of co-occurrence. Furthermore, our proposed Intersec2vec model is integrated into a traffic signal control method based on deep reinforcement learning by supporting more precise sub-area divisions, named Intersec2vec-TSC. For each sub-area, Intersec2vec-TSC adopts a hierarchical structure to design agents, where the upper agent determines the common cycle length based on the overall states of the sub-area, and lower agents jointly train a centralized evaluation network to achieve optimization of green time for each intersection. We conduct the experiment on 108 signalized intersections using real online car-hailing data, and the experimental results show that our proposed method significantly improves the stability of sub-area division and reduces the waiting time of cars during peak hours compared with other comparison methods.
References
[1]
N. I. Zainuddin, S. M. R. Shah, M. Z. Hashim, M. S. Roslam, and L. S. Tey, “Comparison of operational performance before and after improvement: Case study at Pengkalan Weld, Pulau Pinang,” in Proc. AIP Conf., 2018, pp. 020027-1–020027-8.
[2]
E. Namazi, J. Li, and C. Lu, “Intelligent intersection management systems considering autonomous vehicles: A systematic literature review,” IEEE Access, vol. 7, pp. 91946–91965, 2019.
[3]
J. Yu, L. Wang, and X. Gong, “Study on the status evaluation of urban road intersections traffic congestion base on AHP-TOPSIS modal,” Proc.-Social Behav. Sci., vol. 96, pp. 609–616, Nov. 2013.
[4]
Q. Yu, W. Li, D. Yang, and H. Zhang, “Partitioning urban road network based on travel speed correlation,” Int. J. Transp. Sci. Technol., vol. 10, no. 2, pp. 97–109, Jun. 2021.
[5]
I. M. L. N. Jayaweera, “Centrality measures to identify traffic congestion on road networks: A case study of sri Lanka,” IOSR J. Math., vol. 13, no. 2, pp. 13–19, Mar. 2017.
[6]
D. Li and J. Lasenby, “Spatiotemporal attention-based graph convolution network for segment-level traffic prediction,” IEEE Trans. Intell. Transp. Syst., vol. 23, no. 7, pp. 8337–8345, Jul. 2022.
[7]
H. Weiet al., “CoLight: Learning network-level cooperation for traffic signal control,” in Proc. 28th ACM Int. Conf. Inf. Knowl. Manage., Nov. 2019, pp. 1913–1922.
[8]
S. Porta, P. Crucitti, and V. Latora, “The network analysis of urban streets: A primal approach,” Environ. Planning B, Planning Des., vol. 33, no. 5, pp. 705–725, Oct. 2006.
[9]
S. Porta, P. Crucitti, and V. Latora, “The network analysis of urban streets: A dual approach,” Phys. A, Stat. Mech. Appl., vol. 369, no. 2, pp. 853–866, Sep. 2006.
[10]
Y. Zhang, X. Wang, P. Zeng, and X. Chen, “Centrality characteristics of road network patterns of traffic analysis zones,” Transp. Res. Rec., J. Transp. Res. Board, vol. 2256, no. 1, pp. 16–24, Jan. 2011.
[11]
W. Zhang, S. Wang, X. Tian, D. Yu, and Z. Yang, “The backbone of urban street networks: Degree distribution and connectivity characteristics,” Adv. Mech. Eng., vol. 9, no. 11, Nov. 2017, Art. no.
[12]
Y. Yue and A. G.-O. Yeh, “Spatiotemporal traffic-flow dependency and short-term traffic forecasting,” Environ. Planning B, Planning Des., vol. 35, no. 5, pp. 762–771, 2008.
[13]
D. Zhang, J. Yin, X. Zhu, and C. Zhang, “Network representation learning: A survey,” IEEE Trans. Big Data, vol. 6, no. 1, pp. 3–28, Mar. 2020.
[14]
T. Mikolov, K. Chen, G. Corrado, and J. Dean, “Efficient estimation of word representations in vector space,” 2013, arXiv:1301.3781.
[15]
W. L. Hamilton, R. Ying, and J. Leskovec, “Representation learning on graphs: Methods and applications,” 2017, arXiv:1709.05584.
[16]
B. Perozzi, R. Al-Rfou, and S. Skiena, “DeepWalk: Online learning of social representations,” in Proc. 20th ACM SIGKDD Int. Conf. Knowl. Discovery Data Mining, Aug. 2014, pp. 701–710.
[17]
A. Grover and J. Leskovec, “node2vec: Scalable feature learning for networks,” in Proc. 22nd ACM SIGKDD Int. Conf. Knowl. Discovery Data Mining, Aug. 2016, pp. 855–864.
[18]
L. F. R. Ribeiro, P. H. P. Saverese, and D. R. Figueiredo, “struc2vec: Learning node representations from structural identity,” in Proc. 23rd ACM SIGKDD Int. Conf. Knowl. Discovery Data Mining, Aug. 2017, pp. 385–394.
[19]
T. N. Kipf and M. Welling, “Semi-supervised classification with graph convolutional networks,” 2016, arXiv:1609.02907.
[20]
P. Velickovic, G. Cucurull, A. Casanova, A. Romero, P. Liò, and Y. Bengio, “Graph attention networks,” 2017, arXiv:1710.10903.
[21]
A. Pareja, “EvolveGCN: Evolving graph convolutional networks for dynamic graphs,” in Proc. AAAI Conf. Artif. Intell., 2020, vol. 34, no. 4, pp. 5363–5370.
[22]
A. Sankar, Y. Wu, L. Gou, W. Zhang, and H. Yang, “DySAT: Deep neural representation learning on dynamic graphs via self-attention networks,” in Proc. 13th Int. Conf. Web Search Data Mining, Jan. 2020, pp. 519–527.
[23]
Q. Tan, N. Liu, and X. Hu, “Deep representation learning for social network analysis,” Frontiers Big Data, vol. 2, no. 2, pp. 1–10, Apr. 2019.
[24]
W. Wang, T. Tang, F. Xia, Z. Gong, Z. Chen, and H. Liu, “Collaborative filtering with network representation learning for citation recommendation,” IEEE Trans. Big Data, vol. 8, no. 5, pp. 1233–1246, Oct. 2022.
[25]
R. Ietswaart, B. M. Gyori, J. A. Bachman, P. K. Sorger, and L. S. Churchman, “GeneWalk identifies relevant gene functions for a biological context using network representation learning,” Genome Biol., vol. 22, no. 1, pp. 1–35, Dec. 2021.
[26]
K. Liu, S. Gao, P. Qiu, X. Liu, B. Yan, and F. Lu, “Road2Vec: Measuring traffic interactions in urban road system from massive travel routes,” ISPRS Int. J. Geo-Inf., vol. 6, no. 11, p. 321, Oct. 2017.
[27]
D. Xu, C. Wei, P. Peng, Q. Xuan, and H. Guo, “GE-GAN: A novel deep learning framework for road traffic state estimation,” Transp. Res. C, Emerg. Technol., vol. 117, Aug. 2020, Art. no.
[28]
T. S. Jepsen, C. S. Jensen, T. D. Nielsen, and K. Torp, “On network embedding for machine learning on road networks: A case study on the Danish road network,” in Proc. Big Data, Mar. 2018, pp. 3422–3431.
[29]
T. S. Jepsen, C. S. Jensen, and T. D. Nielsen, “Graph convolutional networks for road networks,” in Proc. 27th ACM SIGSPATIAL Int. Conf. Adv. Geographic Inf. Syst., Nov. 2019, pp. 460–463.
[30]
M.-X. Wang, W.-C. Lee, T.-Y. Fu, and G. Yu, “Learning embeddings of intersections on road networks,” in Proc. 27th ACM SIGSPATIAL Int. Conf. Adv. Geographic Inf. Syst., Nov. 2019, pp. 309–318.
[31]
N. Wu, X. W. Zhao, J. Wang, and D. Pan, “Learning effective road network representation with hierarchical graph neural networks,” in Proc. 26th ACM SIGKDD Int. Conf. Knowl. Discovery Data Mining, Aug. 2020, pp. 6–14.
[32]
P. Varaiya, “Max pressure control of a network of signalized intersections,” Transp. Res. C, Emerg. Technol., vol. 36, pp. 177–195, Nov. 2013.
[33]
J. Zeng, J. Hu, and Y. Zhang, “Adaptive traffic signal control with deep recurrent Q-learning,” in Proc. IEEE Intell. Vehicles Symp. (IV), Jun. 2018, pp. 1215–1220.
[34]
J. Gu, Y. Fang, Z. Sheng, and P. Wen, “Double deep Q-network with a dual-agent for traffic signal control,” Appl. Sci., vol. 10, no. 5, p. 1622, Feb. 2020.
[35]
W. Du and S. Ding, “A survey on multi-agent deep reinforcement learning: From the perspective of challenges and applications,” Artif. Intell. Rev., vol. 54, no. 5, pp. 3215–3238, Jun. 2021.
[36]
J. A. Calvo and I. Dusparic, “Heterogeneous multi-agent deep reinforcement learning for traffic lights control,” in Proc. AICS, 2018, pp. 2–13.
[37]
T. Chu, J. Wang, L. Codecà, and Z. Li, “Multi-agent deep reinforcement learning for large-scale traffic signal control,” IEEE Trans. Intell. Transp. Syst., vol. 21, no. 3, pp. 1086–1095, Mar. 2020.
[38]
Z. Zhou, S. Lin, and Y. Xi, “A fast network partition method for large-scale urban traffic networks,” J. Control Theory Appl., vol. 11, no. 3, pp. 359–366, Aug. 2013.
[39]
A. Vaswaniet al., “Attention is all you need,” in Proc. Adv. Neural Inf. Process. Syst., vol. 30, 2017, pp. 5998–6008.
[40]
V. H. Hado, G. Arthur, and S. David, “Deep reinforcement learning with double Q-learning,” in Proc. AAAI Conf. Artif. Intell., Feb. 2016, vol. 30, no. 1, pp. 2094–2100.
[41]
Z. Wang, T. Schaul, M. Hessel, H. Hasselt, M. Lanctot, and N. Freitas, “Dueling network architectures for deep reinforcement learning,” in Proc. Int. Conf. Mach. Learn. (ICML), 2016, pp. 1995–2003.
[42]
R. Lowe, Y. I. Wu, A. Tamar, J. Harb, O. Pieter Abbeel, and I. Mordatch, “Multi-agent actor-critic for mixed cooperative-competitive environments,” in Proc. Adv. Neural Inf. Process. Syst. (NeurIPS), 2017, pp. 6382–6393.
[43]
S. Fujimoto, H. Hoof, and D. Meger, “Addressing function approximation error in actor-critic methods,” in Proc. Int. Conf. Mach. Learn. (ICML), 2018, pp. 1587–1596.
[44]
F. V. Webster, “Traffic signal settings,” Laboratory, London, U.K., Tech. Rep. 39, 1958.
[45]
P. A. Lopezet al., “Microscopic traffic simulation using SUMO,” in Proc. 21st Int. Conf. Intell. Transp. Syst. (ITSC), Nov. 2018, pp. 2575–2582.
[46]
A. Wegener, M. Piórkowski, M. Raya, H. Hellbrück, S. Fischer, and J.-P. Hubaux, “TraCI: An interface for coupling road traffic and network simulators,” in Proc. 11th Commun. Netw. Simulation Symp., 2008, pp. 155–163.
[47]
L. Zhaoet al., “T-GCN: A temporal graph convolutional network for traffic prediction,” IEEE Trans. Intell. Transp. Syst., vol. 21, no. 9, pp. 3848–3858, Sep. 2020.
[48]
H. Weiet al., “PressLight: Learning max pressure control to coordinate traffic signals in arterial network,” in Proc. 25th ACM SIGKDD Int. Conf. Knowl. Discovery Data Mining, Jul. 2019, pp. 1290–1298.
[49]
J. Zenget al., “HALight: Hierarchical deep reinforcement learning for cooperative arterial traffic signal control with cycle strategy,” in Proc. IEEE 25th Int. Conf. Intell. Transp. Syst. (ITSC), Oct. 2022, pp. 479–485.
[50]
Z. Zhou, S. Lin, and Y. Xi, “A dynamic network partition method for heterogenous urban traffic networks,” in Proc. 15th Int. IEEE Conf. Intell. Transp. Syst., Sep. 2012, pp. 820–825.
[51]
M. E. J. Newman, “Modularity and community structure in networks,” Proc. Nat. Acad. Sci. USA, vol. 103, no. 23, pp. 8577–8582, Jun. 2006.
[52]
N. Geroliminis and C. F. Daganzo, “Existence of urban-scale macroscopic fundamental diagrams: Some experimental findings,” Transp. Res. B, Methodol., vol. 42, no. 9, pp. 759–770, Nov. 2008.
[53]
T. Haarnoja, A. Zhou, P. Abbeel, and S. Levine, “Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor,” in Proc. Int. Conf. Mach. Learn. (ICML), 2018, pp. 1861–1870.
Recommendations
LEARNING AND COORDINATION FOR AUTONOMOUS INTERSECTION CONTROL
Future urban road traffic management is an example of a socially relevant problem that can be modeled as a large-scale, open, distributed system, composed of many autonomous interacting agents, which need to be controlled in a decentralized manner. In ...
MARL for Traffic Signal Control in Scenarios with Different Intersection Importance
Distributed Artificial IntelligenceAbstractRecent efforts that applied Multi-Agent Reinforcement Learning (MARL) to the adaptive traffic signal control (ATSC) problem have shown remarkable progress. However, those methods assume that all agents in the cooperative games are isomorphic, ...
Characterization and representation problems for intersection betweennesses
For a set system M=(M"v)"v"@?"V indexed by the elements of a finite set V, the intersection betweenness B(M) induced by M consists of all triples (u,v,w)@?V^3 with M"u@?M"w@?M"v. Similarly, the strict intersection betweenness B"s(M) induced by M ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
1524-9050 © 2023 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See https://www.ieee.org/publications/rights/index.html for more information.
Publisher
IEEE Press
Publication History
Published: 21 December 2023
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 14 Dec 2024