Abstract
Predicting the maneuvers of surrounding vehicles is imperative for the safe navigation of autonomous vehicles. However, naturalistic driving datasets tend to be highly imbalanced, with a bias towards the “going straight” maneuver. Consequently, learning and accurately predicting turning maneuvers pose significant challenges. In this study, we propose a novel two-stage maneuver learning method that can overcome such strong biases by leveraging two heterogeneous datasets in a complementary manner. In the first training phase, we utilize an intersection-centric dataset characterized by balanced distribution of maneuver classes to learn the representations of each maneuver. Subsequently, in the second training phase, we incorporate an ego-centric driving dataset to account for various geometrical road shapes, by transferring the knowledge of geometric diversity to the maneuver prediction model. To facilitate this, we constructed an in-house intersection-centric trajectory dataset with a well-balanced maneuver distribution. By harnessing the power of heterogeneous datasets, our framework significantly improves maneuver prediction performance, particularly for minority maneuver classes such as turning maneuvers. The dataset is available at https://github.com/KAIST-VDCLab/VDC-Trajectory-Dataset.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Morai simulator. https://www.morai.ai/drive
Azadani, M.N., Boukerche, A.: A novel multimodal vehicle path prediction method based on temporal convolutional networks. IEEE Trans. Intell. Transp. Syst. 23(12), 25384–25395 (2022)
Bock, J., Krajewski, R., Moers, T., Runde, S., Vater, L., Eckstein, L.: The IND dataset: a drone dataset of naturalistic road user trajectories at German intersections. In: 2020 IEEE Intelligent Vehicles Symposium (IV), pp. 1929–1934. IEEE (2020)
Chang, M.F., et al.: Argoverse: 3D tracking and forecasting with rich maps. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8748–8757 (2019)
Chawla, N.V., Bowyer, K.W., Hall, L.O., Kegelmeyer, W.P.: Smote: synthetic minority over-sampling technique. J. Artif. Intell. Res. 16, 321–357 (2002)
Chung, J., Gulcehre, C., Cho, K., Bengio, Y.: Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:1412.3555 (2014)
Cieslak, D.A., Chawla, N.V., Striegel, A.: Combating imbalance in network intrusion datasets. In: GrC, pp. 732–737. Citeseer (2006)
Cui, Y., Song, Y., Sun, C., Howard, A., Belongie, S.: Large scale fine-grained categorization and domain-specific transfer learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4109–4118 (2018)
Deepa, T., Punithavalli, M.: An e-smote technique for feature selection in high-dimensional imbalanced dataset. In: 2011 3rd International Conference on Electronics Computer Technology, vol. 2, pp. 322–324. IEEE (2011)
Drummond, C., Holte, R.C., et al.: C4.5, class imbalance, and cost sensitivity: why under-sampling beats over-sampling. In: Workshop on Learning from Imbalanced Datasets II, vol. 11, pp. 1–8. Citeseer (2003)
Feng, X., Cen, Z., Hu, J., Zhang, Y.: Vehicle trajectory prediction using intention-based conditional variational autoencoder. In: 2019 IEEE Intelligent Transportation Systems Conference (ITSC), pp. 3514–3519. IEEE (2019)
Girase, H., et al.: Loki: long term and key intentions for trajectory prediction. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 9803–9812 (2021)
Goldhammer, M., Köhler, S., Zernetsch, S., Doll, K., Sick, B., Dietmayer, K.: Intentions of vulnerable road users-detection and forecasting by means of machine learning. IEEE Trans. Intell. Transp. Syst. 21(7), 3035–3045 (2019)
He, H., Garcia, E.A.: Learning from imbalanced data. IEEE Trans. Knowl. Data Eng. 21(9), 1263–1284 (2009)
Hu, Y., Li, Y., Huang, H., Lee, J., Yuan, C., Zou, G.: A high-resolution trajectory data driven method for real-time evaluation of traffic safety. Accid. Anal. Prev. 165, 106503 (2022)
Colyar, J.: US highway 101 dataset. Federal Highway Administration (FHWA), Technical report FHWA-HRT-07-030 (2007)
Jeon, H., Choi, J., Kum, D.: Scale-net: scalable vehicle trajectory prediction network under random number of interacting vehicles via edge-enhanced graph convolutional neural network. In: 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 2095–2102. IEEE (2020)
Jeon, H., Kim, S., Lee, K., Kang, D., Choi, J., Kum, D.: Are reactions to ego vehicles predictable without data?: a semi-supervised approach. IEEE Trans. Intell. Transp. Syst. 24(6), 6477–6490 (2023)
Kim, S., Jeon, H., Choi, J.W., Kum, D.: Diverse multiple trajectory prediction using a two-stage prediction network trained with lane loss. IEEE Robot. Autom. Lett. 8(4), 2038–2045 (2022)
Kozerawski, J., Sharan, M., Yu, R.: Taming the long tail of deep probabilistic forecasting. arXiv preprint arXiv:2202.13418 (2022)
Liang, M., et al.: Learning lane graph representations for motion forecasting. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12347, pp. 541–556. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58536-5_32
Van der Maaten, L., Hinton, G.: Visualizing data using t-SNE. J. Mach. Learn. Res. 9(11) (2008)
Makansi, O., Çiçek, Ö., Marrakchi, Y., Brox, T.: On exposing the challenging long tail in future prediction of traffic actors. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 13147–13157 (2021)
Marron, J.S., Wand, M.P.: Exact mean integrated squared error. Ann. Stat. 20(2), 712–736 (1992)
Mirus, F., Stewart, T.C., Conradt, J.: The importance of balanced data sets: analyzing a vehicle trajectory prediction model based on neural networks and distributed representations. In: 2020 International Joint Conference on Neural Networks (IJCNN), pp. 1–8. IEEE (2020)
Oord, A.v.d., Li, Y., Vinyals, O.: Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748 (2018)
Park, D., Ryu, H., Yang, Y., Cho, J., Kim, J., Yoon, K.J.: Leveraging future relationship reasoning for vehicle trajectory prediction. arXiv preprint arXiv:2305.14715 (2023)
Peng, J., Bu, X., Sun, M., Zhang, Z., Tan, T., Yan, J.: Large-scale object detection in the wild from imbalanced multi-labels. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9709–9718 (2020)
Robicquet, A., Sadeghian, A., Alahi, A., Savarese, S.: Learning social etiquette: human trajectory prediction in crowded scenes. In: European Conference on Computer Vision (ECCV), vol. 2 (2020)
Shangguan, Q., Fu, T., Wang, J., Fu, L., et al.: A proactive lane-changing risk prediction framework considering driving intention recognition and different lane-changing patterns. Accid. Anal. Prev. 164, 106500 (2022)
Shen, L., Lin, Z., Huang, Q.: Relay backpropagation for effective learning of deep convolutional neural networks. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9911, pp. 467–482. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46478-7_29
Strigel, E., Meissner, D., Seeliger, F., Wilking, B., Dietmayer, K.: The KO-per intersection laserscanner and video dataset. In: 17th International IEEE Conference on Intelligent Transportation Systems (ITSC), pp. 1900–1901. IEEE (2014)
Varadarajan, B., et al.: Multipath++: efficient information fusion and trajectory aggregation for behavior prediction. In: 2022 International Conference on Robotics and Automation (ICRA), pp. 7814–7821. IEEE (2022)
Wang, J., Xu, M., Wang, H., Zhang, J.: Classification of imbalanced data by using the smote algorithm and locally linear embedding. In: 2006 8th International Conference on Signal Processing, vol. 3. IEEE (2006)
Wang, K., Xue, Q., Lu, J.J.: Risky driver recognition with class imbalance data and automated machine learning framework. Int. J. Environ. Res. Public Health 18(14), 7534 (2021)
Westny, T., Frisk, E., Olofsson, B.: Vehicle behavior prediction and generalization using imbalanced learning techniques. In: 2021 IEEE International Intelligent Transportation Systems Conference (ITSC), pp. 2003–2010. IEEE (2021)
Yang, D., Li, L., Redmill, K., Özgüner, Ü.: Top-view trajectories: a pedestrian dataset of vehicle-crowd interaction from controlled experiments and crowded campus. In: 2019 IEEE Intelligent Vehicles Symposium (IV), pp. 899–904. IEEE (2019)
Zhou, Z., Ye, L., Wang, J., Wu, K., Lu, K.: HIVT: hierarchical vector transformer for multi-agent motion prediction. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8823–8833 (2022)
Acknowledgements
This work was supported by Autonomous Driving Development Center, AVP Division, Hyundai Motor Company and the National Research Foundation of Korea (NRF) funded by the Korea government (MSIT) under Grants 2022R1A2C200494413.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2025 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Jeon, H., Kim, S., Syamil, A.R., Kim, J., Kum, D. (2025). Beyond the Data Imbalance: Employing the Heterogeneous Datasets for Vehicle Maneuver Prediction. In: Leonardis, A., Ricci, E., Roth, S., Russakovsky, O., Sattler, T., Varol, G. (eds) Computer Vision – ECCV 2024. ECCV 2024. Lecture Notes in Computer Science, vol 15114. Springer, Cham. https://doi.org/10.1007/978-3-031-72992-8_3
Download citation
DOI: https://doi.org/10.1007/978-3-031-72992-8_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-72991-1
Online ISBN: 978-3-031-72992-8
eBook Packages: Computer ScienceComputer Science (R0)