Nothing Special   »   [go: up one dir, main page]

skip to main content
10.1145/3340531.3411940acmconferencesArticle/Chapter ViewAbstractPublication PagescikmConference Proceedingsconference-collections
research-article

ST-GRAT: A Novel Spatio-temporal Graph Attention Networks for Accurately Forecasting Dynamically Changing Road Speed

Published: 19 October 2020 Publication History

Abstract

Predicting road traffic speed is a challenging task due to different types of roads, abrupt speed change and spatial dependencies between roads; it requires the modeling of dynamically changing spatial dependencies among roads and temporal patterns over long input sequences. This paper proposes a novel spatio-temporal graph attention (ST-GRAT) that effectively captures the spatio-temporal dynamics in road networks. The novel aspects of our approach mainly include spatial attention, temporal attention, and spatial sentinel vectors. The spatial attention takes the graph structure information (e.g., distance between roads) and dynamically adjusts spatial correlation based on road states. The temporal attention is responsible for capturing traffic speed changes, and the sentinel vectors allow the model to retrieve new features from spatially correlated nodes or preserve existing features. The experimental results show that ST-GRAT outperforms existing models, especially in difficult conditions where traffic speeds rapidly change (e.g., rush hours). We additionally provide a qualitative study to analyze when and where ST-GRAT tended to make accurate predictions during rush-hour times.

Supplementary Material

MP4 File (3340531.3411940.mp4)
This video presents a novel spatio-temporal graph attention (ST-GRAT) that effectively captures the spatio-temporal dynamics in road networks. The novel aspects of our approach mainly include spatial attention, temporal attention, and spatial sentinel vectors. The spatial attention takes the graph structure information (e.g., distance between roads) and dynamically adjusts spatial correlation based on road states. The temporal attention is responsible for capturing traffic speed changes, and the sentinel vectors allow the model to retrieve new features from spatially correlated nodes or preserve existing features. The experimental results show that ST-GRAT outperforms existing models, especially in difficult conditions where traffic speeds rapidly change (e.g., rush-hours). We additionally provide a qualitative study to analyze when and where ST-GRAT tended to make accurate predictions during rush-hour times.

References

[1]
James Atwood and Donald F. Towsley. 2016. Diffusion-Convolutional Neural Networks. In Proc. the Advances in Neural Information Processing Systems (NIPS).
[2]
Jimmy Ba, Ryan Kiros, and Geoffrey E. Hinton. 2016. Layer Normalization. abs/1607.06450 (2016).
[3]
Samy Bengio, Oriol Vinyals, Navdeep Jaitly, and Noam Shazeer. 2015. Scheduled sampling for sequence prediction with recurrent neural networks. In Proc. the Advances in Neural Information Processing Systems (NIPS).
[4]
Kyunghyun Cho, Dzmitry Bahdanau, Farhadi, and Yoshua Bengio. 2014. Neural Machine Translation by Jointly Learning to Align and Translate. Proc. the International Conference on Learning Representations (ICLR) (2014).
[5]
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proc. the Annual Meeting of the Association for Computational Linguistics (ACL).
[6]
Xavier Glorot and Yoshua Bengio. 2010. Understanding the difficulty of training deep feedforward neural networks. In Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics (Proceedings of Machine Learning Research), Vol. 9. PMLR, 249--256.
[7]
Dan Hendrycks and Kevin Gimpel. 2016. Gaussian Error Linear Units (GELUs). arXiv:1606.08415 (2016).
[8]
Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation 9, 8 (1997), 1735--1780.
[9]
Thomas N Kipf and Max Welling. 2016. Semi-Supervised Classification with Graph Convolutional Networks. Proc. the International Conference on Learning Representations (ICLR) (2016).
[10]
Chunggi Lee, Yeonjun Kim, Seungmin Jin, Dongmin Kim, Ross Maciejewski, David Ebert, and Sungahn Ko. 2019. A Visual Analytics System for Exploring, Monitoring, and Forecasting Road Traffic Congestion. IEEE Transactions on Visualization and Computer (2019).
[11]
Yaguang Li, Rose Yu, Cyrus Shahabi, and Yan Liu. 2018. Diffusion Convolutional Recurrent Neural Network: Data-Driven Traffic Forecasting. Proc. International Conference on Learning Representations (ICLR).
[12]
Zhouhan Lin, Minwei Feng, Cicero Nogueira dos Santos, Mo Yu, Bing Xiang, Bowen Zhou, and Yoshua Bengio. 2017. A structured self-attentive sentence embedding. Proc. International Conference on Learning Representations (ICLR) (2017).
[13]
Jiasen Lu, Caiming Xiong, Devi Parikh, and Richard Socher. 2016. Knowing When to Look: Adaptive Attention via a Visual Sentinel for Image Captioning. Proc. the IEEE conference on computer vision and pattern recognition(CVPR) (2016).
[14]
Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. 2017. Pointer Sentinel Mixture Models. Proc. the International Conference on Learning Representations (ICLR) (2017).
[15]
Zheyi Pan, Yuxuan Liang, Junbo Zhang, Xiuwen Yi, Yingrui Yu, and Yu Zheng. 2018. HyperST-Net: Hypernetworks for Spatio-Temporal Forecasting. Proc. the Association for the Advancement of Artificial ntelligence (AAAI) (2018).
[16]
Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language Models are Unsupervised Multitask Learners. OpenAI Blog (2019).
[17]
Minjoon Seo, Aniruddha Kembhavi, Ali Farhadi, and Hannaneh Hajishirzi. 2016. Bidirectional attention flow for machine comprehension. Proc. the International Conference on Learning Representations (ICLR) (2016).
[18]
Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: A Simple Way to Prevent Neural Networks from Overfitting. Journal of Machine Learning Research (JMLR) (2014).
[19]
Jian Tang, Meng Qu, Mingzhe Wang, Ming Zhang, Jun Yan, and Qiaozhu Mei. 2015. Line: Large-scale information network embedding. In Proc. the International Conference on World Wide Web (WWW). 1067--1077.
[20]
Charles Truong, Laurent Oudre, and Nicolas Vayatis. 2018. ruptures: change point detection in Python. arXiv preprint arXiv:1801.00826 (2018).
[21]
Aäron van den Oord, Sander Dieleman, Heiga Zen, Karen Simonyan, Oriol Vinyals, Alex Graves, Nal Kalchbrenner, Andrew Senior, and Koray Kavukcuoglu. [n.d.]. WaveNet: A Generative Model for Raw Audio. In 9th ISCA Speech Synthesis Workshop. 125--125.
[22]
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention Is All You Need. In Proc. the Advances in Neural Information Processing Systems (NIPS).
[23]
Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lió, and Yoshua Bengio. 2018. Graph Attention Networks. Proc. International Conference on Learning Representations (ICLR) abs/1710.10903 (2018).
[24]
Eleni I Vlahogianni, Matthew G Karlaftis, and John C Golias. 2014. Short-term traffic forecasting: Where we are and where we're going. Transportation Research Part C: Emerging Technologies 43, Part 1 (2014), 3--19.
[25]
Zonghan Wu, Shirui Pan, Guodong Long, Jing Jiang, and Chengqi Zhang. 2019. Graph WaveNet for Deep Spatial-Temporal Graph Modeling. In Proc. the International Joint Conference on Artificial Intelligence (IJCAI).
[26]
Kelvin Xu, Jimmy Ba, Ryan Kiros, Kyunghyun Cho, Aaron C. Courville, Ruslan R. Salakhutdinov, Richard S. Zemel, and Yoshua Bengio. 2015. Show, Attend and Tell: Neural Image Caption Generation with Visual Attention. In Proc. the International Conference on Machine Learning (ICML).
[27]
Bing Yu, Haoteng Yin, and Zhanxing Zhu. 2018. Spatio-temporal Graph Convolutional Networks: A Deep Learning Framework for Traffic Forecasting. In Proc. the International Joint Conference on Artificial Intelligence (IJCAI).
[28]
Jiani Zhang, Xingjian Shi, Junyuan Xie, Hao Ma, Irwin King, and Dit-Yan Yeung. 2018. GaAN: Gated Attention Networks for Learning on Large and Spatiotemporal Graphs. In Proc. the conference on uncertainty in artificial intelligence (UAI).
[29]
Zheng Zhao, Weihai Chen, Xingming Wu, Peter CY Chen, and Jingmeng Liu. 2017. LSTM network: a deep learning approach for short-term traffic forecast. IET Intelligent Transport Systems 11, 2 (2017), 68--75.
[30]
Chuanpan Zheng, Xiaoliang Fan, Cheng Wang, and Jianzhong Qi. 2019. Gman: A graph multi-attention network for traffic prediction. Proc. the Association for the Advancement of Artificial Intelligence (AAAI) (2019).
[31]
Eric Zivot and Jiahui Wang. 2006. Vector autoregressive models for multivariate time series. Modeling Financial Time Series with S-Plus® (2006).

Cited By

View all
  • (2025)Research on Urban Road Traffic Flow Prediction Based on Sa-Dynamic Graph Convolutional Neural NetworkMathematics10.3390/math1303041613:3(416)Online publication date: 27-Jan-2025
  • (2025)A 3D Convolution-Incorporated Dimension Preserved Decomposition Model for Traffic Data PredictionIEEE Transactions on Intelligent Transportation Systems10.1109/TITS.2024.348696326:1(673-690)Online publication date: Jan-2025
  • (2025)An urban road traffic flow prediction method based on multi-information fusionScientific Reports10.1038/s41598-025-88429-y15:1Online publication date: 15-Feb-2025
  • Show More Cited By

Index Terms

  1. ST-GRAT: A Novel Spatio-temporal Graph Attention Networks for Accurately Forecasting Dynamically Changing Road Speed

      Recommendations

      Comments

      Please enable JavaScript to view thecomments powered by Disqus.

      Information & Contributors

      Information

      Published In

      cover image ACM Conferences
      CIKM '20: Proceedings of the 29th ACM International Conference on Information & Knowledge Management
      October 2020
      3619 pages
      ISBN:9781450368599
      DOI:10.1145/3340531
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

      Sponsors

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 19 October 2020

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. attention networks
      2. graph neural networks
      3. spatial-temporal modeling
      4. time-series prediction
      5. traffic prediction

      Qualifiers

      • Research-article

      Funding Sources

      • Artificial Intelligence graduate school support(UNIST)
      • nstitute of Information & communications Technology Planning & Evaluation(IITP)

      Conference

      CIKM '20
      Sponsor:

      Acceptance Rates

      Overall Acceptance Rate 1,861 of 8,427 submissions, 22%

      Upcoming Conference

      CIKM '25

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)189
      • Downloads (Last 6 weeks)14
      Reflects downloads up to 17 Feb 2025

      Other Metrics

      Citations

      Cited By

      View all
      • (2025)Research on Urban Road Traffic Flow Prediction Based on Sa-Dynamic Graph Convolutional Neural NetworkMathematics10.3390/math1303041613:3(416)Online publication date: 27-Jan-2025
      • (2025)A 3D Convolution-Incorporated Dimension Preserved Decomposition Model for Traffic Data PredictionIEEE Transactions on Intelligent Transportation Systems10.1109/TITS.2024.348696326:1(673-690)Online publication date: Jan-2025
      • (2025)An urban road traffic flow prediction method based on multi-information fusionScientific Reports10.1038/s41598-025-88429-y15:1Online publication date: 15-Feb-2025
      • (2025)Forecasting short-term passenger flow via CBGC-SCI: an in-depth comparative study on Shenzhen MetroMachine Learning10.1007/s10994-024-06711-y114:1Online publication date: 14-Jan-2025
      • (2025)Traffic flow prediction based on the RETGCN modelComputing10.1007/s00607-024-01402-x107:1Online publication date: 8-Jan-2025
      • (2024)Spatiotemporal Dynamic Multi-Hop Network for Traffic Flow ForecastingSustainability10.3390/su1614586016:14(5860)Online publication date: 9-Jul-2024
      • (2024)Edge Computing-Enabled Secure Forecasting Nationwide Industry PM2.5 with LLM in the Heterogeneous NetworkElectronics10.3390/electronics1313258113:13(2581)Online publication date: 30-Jun-2024
      • (2024)Local-Global Spatial-Temporal Graph Convolutional Network for Traffic Flow ForecastingElectronics10.3390/electronics1303063613:3(636)Online publication date: 2-Feb-2024
      • (2024)Spatio-Temporal Predictive Modeling Techniques for Different Domains: a SurveyACM Computing Surveys10.1145/369666157:2(1-42)Online publication date: 20-Sep-2024
      • (2024)Multi-Scale Convolution Multi-Graph Attention Neural Networks for Traffic Flow ForecastingProceedings of the 2024 16th International Conference on Machine Learning and Computing10.1145/3651671.3651744(176-184)Online publication date: 2-Feb-2024
      • Show More Cited By

      View Options

      Login options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Figures

      Tables

      Media

      Share

      Share

      Share this Publication link

      Share on social media