Abstract
Traffic flow forecasting has primarily relied on the spatial-temporal models. However, yielding accurate traffic prediction is still challenging due to that the dynamic temporal pattern, intricate spatial dependency and their affluent interaction are difficult to depict. Existing models are often restricted since they can only capture limited-range temporal dependency, shallow spatial dependency, or faint spatial-temporal interaction. In this work, to overcome these limitations, we propose a novel spatial-temporal graph sandwich Transformer (STGST) for traffic flow forecasting. In STGST, we design two temporal Transformers equipped with time encoding and a spatial Transformer equipped with structure and spatial encoding to characterize long-range temporal and deep spatial dependencies, respectively. These two types of Transformers are further structured in a sandwich manner with two temporal Transformers as buns and a spatial Transformer as sliced meat to capture prosperous spatial-temporal interactions. We also assemble a set of such sandwich Transformers together to strengthen the correlations between spatial and temporal domains. Extensive experimental studies are performed on public traffic benchmarks. Promising results demonstrate that the proposed STGST outperforms state-of-the-art baselines.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Bai, L., Yao, L., Kanhere, S.S., Wang, X., Sheng, Q.Z.: STG2Seq: spatial-temporal graph to sequence model for multi-step passenger demand forecasting. In: International Joint Conference on Artificial Intelligence, pp. 1981–1987 (2019)
Bai, L., Yao, L., Li, C., Wang, X., Wang, C.: Adaptive graph convolutional recurrent network for traffic forecasting. In: Advances in Neural Information Processing Systems, vol. 33, pp. 17804–17815 (2020)
Bai, S., Kolter, J.Z., Koltun, V.: An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. arXiv preprint arXiv:1803.01271 (2018)
Chen, C., Petty, K., Skabardonis, A., Varaiya, P., Jia, Z.: Freeway performance measurement system: mining loop detector data. Transp. Res. Rec. 1748(1), 96–102 (2001)
Chen, M., Wei, Z., Huang, Z., Ding, B., Li, Y.: Simple and deep graph convolutional networks. In: International Conference on Machine Learning, pp. 1725–1735 (2020)
Chen, Y., Segovia, I., Gel, Y.R.: Z-gcnets: time zigzags at graph convolutional networks for time series forecasting. In: International Conference on Machine Learning, pp. 1684–1694 (2021)
Chung, J., Gulcehre, C., Cho, K., Bengio, Y.: Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:1412.3555 (2014)
Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: Bert: pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)
Fang, Z., Long, Q., Song, G., Xie, K.: Spatial-temporal graph ode networks for traffic flow forecasting. In: SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 364–373 (2021)
Guo, S., Lin, Y., Feng, N., Song, C., Wan, H.: Attention based spatial-temporal graph convolutional networks for traffic flow forecasting. In: AAAI Conference on Artificial Intelligence, vol. 33, pp. 922–929 (2019)
Huber, P.J.: Robust estimation of a location parameter. In: Breakthroughs in Statistics: Methodology and Distribution, pp. 492–518 (1992)
Lan, S., Ma, Y., Huang, W., Wang, W., Yang, H., Li, P.: DSTAGNN: dynamic spatial-temporal aware graph neural network for traffic flow forecasting. In: International Conference on Machine Learning, pp. 11906–11917 (2022)
Li, M., Zhu, Z.: Spatial-temporal fusion graph neural networks for traffic flow forecasting. In: AAAI Conference on Artificial Intelligence, vol. 35, pp. 4189–4196 (2021)
Li, Q., Han, Z., Wu, X.M.: Deeper insights into graph convolutional networks for semi-supervised learning. In: AAAI Conference on Artificial Intelligence, vol. 32, pp. 3538–3545 (2018)
Li, Y., Yu, R., Shahabi, C., Liu, Y.: Diffusion convolutional recurrent neural network: data-driven traffic forecasting. In: International Conference on Learning Representations (2018)
Min, E., et al.: Transformer for graphs: an overview from architecture perspective. arXiv preprint arXiv:2202.08455 (2022)
Song, C., Lin, Y., Guo, S., Wan, H.: Spatial-temporal synchronous graph convolutional networks: a new framework for spatial-temporal network data forecasting. In: AAAI Conference on Artificial Intelligence, vol. 34, pp. 914–921 (2020)
Sutskever, I., Vinyals, O., Le, Q.V.: Sequence to sequence learning with neural networks. In: Advances in Neural Information Processing Systems, vol. 27, pp. 3104–3112 (2014)
Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, vol. 30, pp. 5998–6008 (2017)
Williams, B.M., Hoel, L.A.: Modeling and forecasting vehicular traffic flow as a seasonal arima process: theoretical basis and empirical results. J. Transp. Eng. 129(6), 664–672 (2003)
Xu, M., et al.: Spatial-temporal transformer networks for traffic flow forecasting. arXiv preprint arXiv:2001.02908 (2020)
Ying, C., et al.: Do transformers really perform badly for graph representation? In: Advances in Neural Information Processing Systems, vol. 34, pp. 28877–28888 (2021)
Yu, B., Yin, H., Zhu, Z.: Spatio-temporal graph convolutional networks: a deep learning framework for traffic forecasting. In: International Joint Conference on Artificial Intelligence, pp. 3634–3640 (2018)
Zheng, C., Fan, X., Wang, C., Qi, J.: GMAN: a graph multi-attention network for traffic prediction. In: AAAI Conference on Artificial Intelligence, vol. 34, pp. 1234–1241 (2020)
Zivot, E., Wang, J.: Vector autoregressive models for multivariate time series. In: Zivot, E., Wang, J. (eds.) Modeling Financial Time Series with S-PLUS®, pp. 385–429. Springer, New York (2006). https://doi.org/10.1007/978-0-387-32348-0_11
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Ethics declarations
Ethics Statement
Our research involves the use of publicly available traffic data to perform traffic flow forecasting. This data was initially collected by the government. We confirm that all data used in our research is obtained in accordance with relevant laws and regulations, and the data does not contain any personal information, such as identifiable information about individuals or vehicles, and therefore the privacy and confidentiality concerns are minimized. Although the data we are using is already publicly available online, we acknowledge the potential for bias to be introduced into research through a variety of factors, including the location and distribution of traffic sensors.
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Fan, Y. et al. (2023). Spatial-Temporal Graph Sandwich Transformer for Traffic Flow Forecasting. In: De Francisci Morales, G., Perlich, C., Ruchansky, N., Kourtellis, N., Baralis, E., Bonchi, F. (eds) Machine Learning and Knowledge Discovery in Databases: Applied Data Science and Demo Track. ECML PKDD 2023. Lecture Notes in Computer Science(), vol 14175. Springer, Cham. https://doi.org/10.1007/978-3-031-43430-3_13
Download citation
DOI: https://doi.org/10.1007/978-3-031-43430-3_13
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-43429-7
Online ISBN: 978-3-031-43430-3
eBook Packages: Computer ScienceComputer Science (R0)