Abstract
Informed learning approaches explicitly integrate prior knowledge into learning systems, which can reduce data needs and increase robustness. However, existing work typically aims to integrate formal scientific knowledge by directly pruning the problem space, which is infeasible for more intuitive world and expert knowledge, or requires specific architecture changes and knowledge representations. We propose a probabilistic informed learning approach to integrate prior world and expert knowledge without these requirements. Our approach repurposes continual learning methods to operationalize Baye’s rule for informed learning and to enable probabilistic and multi-modal predictions. We exemplify our proposal in an application to two state-of-the-art trajectory predictors for autonomous driving. This safety-critical domain is subject to an overwhelming variety of rare scenarios requiring robust and accurate predictions. We evaluate our models on a public benchmark dataset and demonstrate that our approach outperforms non-informed and informed learning baselines. Notably, we can compete with a conventional baseline, even using only half as many observations of the training dataset.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
- 2.
In general, any space-filling heuristic may be used to generate the set \(\mathcal {K}(\epsilon )\), even a data-agnostic one.
- 3.
References
Bahari, M., Nejjar, I., Alahi, A.: Injecting knowledge in data-driven vehicle trajectory predictors. Transp. Res. Part C: Emerg. Technol. 128, 103010 (2021)
Boulton, F.A., Grigore, E.C., Wolff, E.M.: Motion prediction using trajectory sets and self-driving domain knowledge. arXiv preprint https://arxiv.org/abs/2006.04767 (2021)
Caesar, H., et al.: nuscenes: a multimodal dataset for autonomous driving. In: Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020, Seattle, WA, USA (2020)
Chai, Y., Sapp, B., Bansal, M., Anguelov, D.: Multipath: multiple probabilistic anchor trajectory hypotheses for behavior prediction. In: Proceedings of Machine Learning Research (PMLR): 3rd Annual Conference on Robot Learning, CoRL 2019, Osaka, Japan (2019)
Cui, H., et al.: Deep kinematic models for kinematically feasible vehicle trajectory predictions. In: Proceedings of the 2020 IEEE International Conference on Robotics and Automation, ICRA 2020, Paris, France (2020)
De Lange, M., et al.: A continual learning survey: defying forgetting in classification tasks. IEEE Trans. Pattern Anal. Mach. Intell. 44, 3366–3385 (2022)
Gal, Y., Ghahramani, Z.: A theoretically grounded application of dropout in recurrent neural networks. In: Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems, NeurIPS 2016, Barcelona, Spain (2016)
Huang, Y., Du, J., Yang, Z., Zhou, Z., Zhang, L., Chen, H.: A survey on trajectory-prediction methods for autonomous driving. IEEE Trans. Intell. Veh. 7, 652–674 (2022)
Kirkpatrick, J., et al.: Overcoming catastrophic forgetting in neural networks. Proc. Natl. Acad. Sci. PNAS 114, 3521–3526 (2017)
Li, J., Ma, H., Tomizuka, M.: Conditional generative neural system for probabilistic trajectory prediction. In: Proceedings of the 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2019, Macau, SAR, China (2019)
Liu, J., Mao, X., Fang, Y., Zhu, D., Meng, M.Q.H.: A survey on deep-learning approaches for vehicle trajectory prediction in autonomous driving. IEEE Trans. Intell. Veh. (2021)
Loo, N., Swaroop, S., Turner, R.E.: Generalized variational continual learning. In: Proceedings of the 9th International Conference on Learning Representations, ICLR 2021, Virtual Event (2021)
McAllister, R., et al.: Concrete problems for autonomous vehicle safety: advantages of bayesian deep learning. In: Proceedings of the 26th International Joint Conference on Artificial Intelligence, IJCAI 2017, Melbourne, Australia (2017)
Mundt, M., Hong, Y., Pliushch, I., Ramesh, V.: A wholistic view of continual learning with deep neural networks: forgotten lessons and the bridge to active and open world learning. arXiv preprint https://arxiv.org/abs/2009.01797 (2020)
Nguyen, C.V., Li, Y., Bui, T.D., Turner, R.E.: Variational continual learning. In: Proceedings of the 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada (2018)
Parisi, G.I., Kemker, R., Part, J.L., Kanan, C., Wermter, S.: Continual lifelong learning with neural networks: a review. Neural Netw. 113, 54–71 (2019)
Phan-Minh, T., Grigore, E.C., Boulton, F.A., Beijbom, O., Wolff, E.M.: Covernet: multimodal behavior prediction using trajectory sets. In: Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020, Seattle, WA, USA (2020)
von Rueden, L., Houben, S., Cvejoski, K., Bauckhage, C., Piatkowski, N.: Informed pre-training on prior knowledge. arXiv preprint https://arxiv.org/abs/2205.11433 (2022)
Schwarz, J., et al.: Progress & compress: a scalable framework for continual learning. In: Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholm, Sweden (2018)
von Rueden, L., et al.: Informed machine learning - a taxonomy and survey of integrating knowledge into learning systems. IEEE Trans. Knowl. Data Eng. 35, 614–633 (2021)
Wang, E., Cui, H., Yalamanchi, S., Moorthy, M., Djuric, N.: Improving movement predictions of traffic actors in bird’s-eye view models using gans and differentiable trajectory rasterization. In: KDD 2020: Proceedings of the 26th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD 2020, Virtual Event (2020)
Wilson, A.G., Izmailov, P.: Bayesian deep learning and a probabilistic perspective of generalization. In: Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems, NeurIPS 2020, Virtual Event (2020)
Wörmann, J., et al.: Knowledge augmented machine learning with applications in autonomous driving: a survey. arXiv preprint https://arxiv.org/abs/2205.04712 (2022)
Zhan, W., et al.: INTERACTION dataset: an international, adversarial and cooperative motion dataset in interactive driving scenarios with semantic maps. arXiv preprint http://arxiv.org/abs/1910.03088 (2019)
Acknowledgements
The research leading to these results is funded by the German Federal Ministry for Economic Affairs and Climate Action within the project “KI Wissen - Entwicklung von Methoden für die Einbindung von Wissen in maschinelles Lernen". The authors would like to thank the consortium for the successful cooperation.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Schlauch, C., Wirth, C., Klein, N. (2023). Informed Priors for Knowledge Integration in Trajectory Prediction. In: Koutra, D., Plant, C., Gomez Rodriguez, M., Baralis, E., Bonchi, F. (eds) Machine Learning and Knowledge Discovery in Databases: Research Track. ECML PKDD 2023. Lecture Notes in Computer Science(), vol 14173. Springer, Cham. https://doi.org/10.1007/978-3-031-43424-2_24
Download citation
DOI: https://doi.org/10.1007/978-3-031-43424-2_24
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-43423-5
Online ISBN: 978-3-031-43424-2
eBook Packages: Computer ScienceComputer Science (R0)