Abstract
Missing traffic data problem has a significant negative impact for data-driven applications in Intelligent Transportation Systems (ITS). However, existing models mainly focus on the imputation results under Missing Completely At Random (MCAR) task, and there is a considerable difference between MCAR with the situation encountered in real life. Furthermore, some existing state-of-the-art models can be vulnerable when dealing with other imputation tasks like block miss imputation. In this paper, we propose a novel deep learning model TINet for missing traffic data imputation problems. TINet uses the self-attention mechanism to dynamically adjust the weight for each entries in the input data. This architecture effectively avoids the limitation of the Fully Connected Network (FCN). Furthermore, TINet uses multi-dimensional embedding for representing data’s spatial-temporal positional information, which alleviates the computation and memory requirements of attention-based model for multi-dimentional data. We evaluate TINet with other baselines on two real-world datasets. Different from the previous work that only employs MCAR for testing, our experiment also tested the performance of models on the Block Miss At Random (BMAR) tasks. The results show that TINet outperforms baseline imputation models for both MCAR and BMAR tasks with different missing rates.
This work is supported by the Stable Support Plan Program of Shenzhen Natural Science Fund No. 20200925155105002, by the General Program of Guangdong Basic and Applied Basic Research Foundation No. 2019A1515011032, and by the Guangdong Provincial Key Laboratory (Grant No. 2020B121201001).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
For PeMS dataset which will be introduced in Sect. 5, we represent \(W_{ij}\) by the distance between \(v_i\) and \(v_j\) as there is no historical travel data statistics.
- 2.
- 3.
- 4.
We don’t set this to be a 100% probability, which somehow rarely happens in practice.
References
Chen, H., Grant-Muller, S., Mussone, L., Montgomery, F.: A study of hybrid neural network approaches and the effects of missing data on traffic forecasting. Neural Comput. Appl. 10(3), 277–286 (2001)
Chen, X., He, Z., Sun, L.: A Bayesian tensor decomposition approach for spatiotemporal traffic data imputation. Transp. Res. Part C Emerg. Technol. 98, 73–84 (2019)
Chen, X., Sun, L.: Bayesian temporal factorization for multidimensional time series prediction. IEEE Trans. Pattern Anal. Mach. Intell. (2021)
Correia, G.M., Niculae, V., Martins, A.F.: Adaptively sparse transformers. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pp. 2174–2184 (2019)
Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pp. 4171–4186 (2019)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: International Conference on Learning Representations, San Diego, CA, USA, 7–9 May (2015)
Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781 (2013)
Perozzi, B., Al-Rfou, R., Skiena, S.: DeepWalk: online learning of social representations. In: ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 701–710 (2014)
Redman, T.C.: If your data is bad, your machine learning tools are useless. Harvard Bus. Rev. 2 (2018)
Sidiropoulos, N.D., De Lathauwer, L., Fu, X., Huang, K., Papalexakis, E.E., Faloutsos, C.: Tensor decomposition for signal processing and machine learning. IEEE Trans. Sig. Process. 65(13), 3551–3582 (2017)
Smith, B.L., Scherer, W.T., Conklin, J.H.: Exploring imputation techniques for missing data in transportation management systems. Transp. Res. Rec. 1836(1), 132–142 (2003)
Vaswani, A., et al.: Attention is all you need. arXiv preprint arXiv:1706.03762 (2017)
Vig, J., Belinkov, Y.: Analyzing the structure of attention in a transformer language model. In: Proceedings of the 2019 ACL Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, pp. 63–76 (2019)
Xu, J.R., Li, X.Y., Shi, H.J.: Short-term traffic flow forecasting model under missing data. J. Comput. Appl. 30(4), 1117–1120 (2010)
Duan, Y., Lv, Y., Kang, W., Zhao, Y.: A deep learning based approach for traffic data imputation. In: IEEE Conference on Intelligent Transportation Systems, pp. 912–917 (2014)
Yoon, J., Jordon, J., Schaar, M.: Gain: missing data imputation using generative adversarial nets. In: International Conference on Machine Learning, pp. 5689–5698. PMLR (2018)
Zaheer, M., et al.: Big bird: transformers for longer sequences. Adv. Neural Inf. Process. Syst. 33, 17283–17297 (2020)
Zhu, L., Yu, F.R., Wang, Y., Ning, B., Tang, T.: Big data analytics in intelligent transportation systems: a survey. IEEE Trans. Intell. Transp. Syst. 20(1), 383–398 (2018)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Song, X., Ye, Y., Yu, J.J.Q. (2021). TINet: Multi-dimensional Traffic Data Imputation via Transformer Network. In: Farkaš, I., Masulli, P., Otte, S., Wermter, S. (eds) Artificial Neural Networks and Machine Learning – ICANN 2021. ICANN 2021. Lecture Notes in Computer Science(), vol 12891. Springer, Cham. https://doi.org/10.1007/978-3-030-86362-3_25
Download citation
DOI: https://doi.org/10.1007/978-3-030-86362-3_25
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-86361-6
Online ISBN: 978-3-030-86362-3
eBook Packages: Computer ScienceComputer Science (R0)