Abstract
Traffic prediction is an essential part of Intelligent Transportation System (ITS). Existing work typically use unimodal traffic data, combining with road network graph or external factors (e.g., weather, POIs) for prediction. However, in real traffic systems multimodal traffic data are collected from one or more co-located sensors, and data of non-target modality are not fully utilized by existing work. To overcome this limitation, we utilize multimodal traffic data to improve target prediction tasks. We propose a novel Spatio-Temporal Multimodal Attention Network (ST-MAN) for traffic prediction. Firstly, we design a cross-modal attention mechanism to learn dynamic inter-modal correlations. Secondly, we propose a compact yet effective multimodal fusion framework to exploit both the inter-modal and intra-modal correlations. Thirdly, a refined spatio-temporal embedding mechanism is designed to feed in more implicit information. Extensive experiments on three real-world datasets show that ST-MAN not only outperforms state-of-the-art methods in all aspects, but also has high computational efficiency. Moreover, the framework is easily generalized to include more data modalities.
L. Li—The majority of this work was done during their internship in iFLYTEK CO., LTD.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Brigham, E.O., Morrow, R.: The fast fourier transform. IEEE Spect. 4(12), 63–70 (1967)
Cui, Z., Ke, R., Wang, Y.: Deep bidirectional and unidirectional LSTM recurrent neural network for network-wide traffic speed prediction. CoRR abs/1801.02143 (2018)
Grover, A., Leskovec, J.: node2vec: scalable feature learning for networks. In: Krishnapuram, B., Shah, M., Smola, A.J., Aggarwal, C.C., Shen, D., Rastogi, R. (eds.) Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2016, pp. 855–864. ACM (2016)
Guo, S., Lin, Y., Feng, N., Song, C., Wan, H.: Attention based spatial-temporal graph convolutional networks for traffic flow forecasting. In: The Thirty-Third AAAI Conference on Artificial Intelligence, 2019, pp. 922–929. AAAI Press (2019)
Han, L., Du, B., Sun, L., Fu, Y., Lv, Y., Xiong, H.: Dynamic and multi-faceted spatio-temporal deep learning for traffic speed forecasting. In: Zhu, F., Ooi, B.C., Miao, C. (eds.) KDD 2021: The 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2021, pp. 547–555. ACM (2021)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, pp. 770–778. IEEE Computer Society (2016)
Huang, J., Luo, K., Cao, L., Wen, Y., Zhong, S.: Learning multiaspect traffic couplings by multirelational graph attention networks for traffic prediction. IEEE Trans. Intell. Transp. Syst. 23(11), 20681–20695 (2022)
Jin, G., Cui, Y., Zeng, L., Tang, H., Feng, Y., Huang, J.: Urban ride-hailing demand prediction with multiple spatio-temporal information fusion network. Transp. Res. Part C: Emerg. Technol. 117, 102665 (2020)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: Bengio, Y., LeCun, Y. (eds.) 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, 7–9 May 2015, Conference Track Proceedings (2015)
Li, M., Zhu, Z.: Spatial-temporal fusion graph neural networks for traffic flow forecasting. In: Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, pp. 4189–4196. AAAI Press (2021)
Li, Y., Yu, R., Shahabi, C., Liu, Y.: Diffusion convolutional recurrent neural network: Data-driven traffic forecasting. In: International Conference on Learning Representations (2018)
Li, Z., et al.: A multi-stream feature fusion approach for traffic prediction. IEEE Trans. Intell. Transp. Syst. 23(2), 1456–1466 (2022)
Song, C., Lin, Y., Guo, S., Wan, H.: Spatial-temporal synchronous graph convolutional networks: a new framework for spatial-temporal network data forecasting. In: The Thirty-Fourth AAAI Conference on Artificial Intelligence, 2020, pp. 914–921 (2020)
Sutskever, I., Vinyals, O., Le, Q.V.: Sequence to sequence learning with neural networks. In: Ghahramani, Z., Welling, M., Cortes, C., Lawrence, N., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems, vol. 27. Curran Associates, Inc. (2014)
Vaswani, A., et al.: Attention is all you need. In: NIPS, pp. 6000–6010 (2017)
Williams, B.M., Hoel, L.A.: Modeling and forecasting vehicular traffic flow as a seasonal arima process: theoretical basis and empirical results. J. Transp. Eng. 129(6), 664–672 (2003)
Wu, Z., Pan, S., Long, G., Jiang, J., Zhang, C.: Graph wavenet for deep spatial-temporal graph modeling. In: IJCAI, pp. 1907–1913 (2019)
Xu, M., et al.: Spatial-temporal transformer networks for traffic flow forecasting. CoRR abs/2001.02908 (2020)
Yu, B., Yin, H., Zhu, Z.: Spatio-temporal graph convolutional networks: a deep learning framework for traffic forecasting. In: IJCAI, pp. 3634–3640 (2018)
Zhang, J., Zheng, Y., Qi, D.: Deep spatio-temporal residual networks for citywide crowd flows prediction. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 31 (2017)
Zhang, X., et al.: Traffic flow forecasting with spatial-temporal graph diffusion network. In: Thirty-Fifth AAAI Conference on Artificial Intelligence, 2021, pp. 15008–15015 (2021)
Zhao, B., Gao, X., Liu, J., Zhao, J., Xu, C.: Spatiotemporal data fusion in graph convolutional networks for traffic prediction. IEEE Access 8, 76632–76641 (2020)
Zheng, C., Fan, X., Wang, C., Qi, J.: GMAN: a graph multi-attention network for traffic prediction. In: The Thirty-Fourth AAAI Conference on Artificial Intelligence, AAAI 2020, pp. 1234–1241. AAAI Press (2020)
Acknowledgment
This work was supported by the National Key R &D Program of China under Grant No. 2018AAA0101200.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
He, R., Li, L., Hua, B., Tong, J., Tan, C. (2023). ST-MAN: Spatio-Temporal Multimodal Attention Network for Traffic Prediction. In: Jin, Z., Jiang, Y., Buchmann, R.A., Bi, Y., Ghiran, AM., Ma, W. (eds) Knowledge Science, Engineering and Management. KSEM 2023. Lecture Notes in Computer Science(), vol 14118. Springer, Cham. https://doi.org/10.1007/978-3-031-40286-9_12
Download citation
DOI: https://doi.org/10.1007/978-3-031-40286-9_12
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-40285-2
Online ISBN: 978-3-031-40286-9
eBook Packages: Computer ScienceComputer Science (R0)