Nothing Special   »   [go: up one dir, main page]
Skip to main content
Springer Nature Link
Log in

Domain Generalization of 3D Object Detection by Density-Resampling

  • Conference paper
  • First Online:
Computer Vision – ECCV 2024 (ECCV 2024)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 15122))

Included in the following conference series:

  • 23 Accesses

Abstract

Point-cloud-based 3D object detection suffers from performance degradation when encountering data with novel domain gaps. To tackle it, single-domain generalization (SDG) aims to generalize the detection model trained in a limited single source domain to perform robustly on unexplored domains. Through analysis of errors and missed detections in 3D point clouds, it has become evident that challenges predominantly arise from variations in point cloud density, especially the sparsity of point cloud data. Thus, in this paper, we propose an SDG method centered around the theme of point cloud density resampling, which involves using data augmentation to simulate point clouds of different densities and developing a novel point cloud densification algorithm to enhance the detection accuracy of low-density point clouds. Specifically, our physical-aware density-resampling data augmentation (PDDA) is the first to consider the physical constraints on point density distribution in data augmentation, leading to a more realistic simulation of variation in cloud density. In systematic design, an auxiliary self-supervised point cloud densification task is incorporated into the detection framework, forming a basis for test-time model update. By manipulating point cloud density, our method not only increases the model’s adaptability to point clouds of different densities but also allows the self-supervised densification algorithm to serve as a metric for assessing the model’s understanding of the environment and semantic information. This, in turn, enables a test-time adjustment of the model to better adapt to varying domains. Extensive cross-dataset experiments covering “Car”, “Pedestrian”, and “Cyclist” detections demonstrate our method outperforms state-of-the-art SDG methods and even overpass unsupervised domain adaptation methods under some circumstances. The code is released at https://github.com/xingyu-group/3D-Density-Resampling-SDG.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 64.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 79.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Arnold, E., Al-Jarrah, O.Y., Dianati, M., Fallah, S., Oxtoby, D., Mouzakitis, A.: A survey on 3D object detection methods for autonomous driving applications. IEEE Trans. Intell. Transp. Syst. 20(10), 3782–3795 (2019)

    Article  Google Scholar 

  2. Balaji, Y., Sankaranarayanan, S., Chellappa, R.: MetaReg: towards domain generalization using meta-regularization. In: Advances in Neural Information Processing Systems. vol. 31 (2018)

    Google Scholar 

  3. Boyce, S.J., Pollatsek, A., Rayner, K.: Effect of background information on object identification. J. Exp. Psychol. Hum. Percept. Perform. 15(3), 556 (1989)

    Article  Google Scholar 

  4. Caesar, H., et al.: nuScenes: a multimodal dataset for autonomous driving. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11621–11631 (2020)

    Google Scholar 

  5. Chen, D., Wang, D., Darrell, T., Ebrahimi, S.: Contrastive test-time adaptation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 295–305 (2022)

    Google Scholar 

  6. Chen, L., Zhang, Y., Song, Y., Shan, Y., Liu, L.: Improved test-time adaptation for domain generalization. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 24172–24182 (2023)

    Google Scholar 

  7. Chen, W., Lin, L., Yang, S., Xie, D., Pu, S., Zhuang, Y.: Self-supervised noisy label learning for source-free unsupervised domain adaptation. In: 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 10185–10192. IEEE (2022)

    Google Scholar 

  8. Choi, J., Song, Y., Kwak, N.: Part-aware data augmentation for 3D object detection in point cloud. In: 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 3391–3397. IEEE (2021)

    Google Scholar 

  9. Deng, J., Shi, S., Li, P., Zhou, W., Zhang, Y., Li, H.: Voxel R-CNN: towards high performance voxel-based 3D object detection. In: Proceedings of the AAAI Conference on Artificial Intelligence. vol. 35, pp. 1201–1209 (2021)

    Google Scholar 

  10. Du, L., et al.: Associate-3DDET: perceptual-to-conceptual association for 3D point cloud object detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13329–13338 (2020)

    Google Scholar 

  11. Geiger, A., Lenz, P., Stiller, C., Urtasun, R.: Vision meets robotics: the KITTI dataset. Int. J. Robot. Res. 32(11), 1231–1237 (2013)

    Article  Google Scholar 

  12. Geyer, J., et al.: A2D2: Audi autonomous driving dataset. arXiv preprint arXiv:2004.06320 (2020)

  13. Hahner, M., et al.: Lidar snowfall simulation for robust 3D object detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 16364–16374 (2022)

    Google Scholar 

  14. Hahner, M., Sakaridis, C., Dai, D., Van Gool, L.: Fog simulation on real lidar point clouds for 3D object detection in adverse weather. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 15283–15292 (2021)

    Google Scholar 

  15. Hammer, M., Hebel, M., Laurenzis, M., Arens, M.: Lidar-based detection and tracking of small UAVs. In: Emerging Imaging and Sensing Technologies for Security and Defence III; and Unmanned Sensors, Systems, and Countermeasures. vol. 10799, pp. 177–185. SPIE (2018)

    Google Scholar 

  16. Hu, J.S., Kuai, T., Waslander, S.L.: Point density-aware voxels for lidar 3D object detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8469–8478 (2022)

    Google Scholar 

  17. Hu, Q., Liu, D., Hu, W.: Density-insensitive unsupervised domain adaption on 3D object detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 17556–17566 (2023)

    Google Scholar 

  18. Iwasawa, Y., Matsuo, Y.: Test-time classifier adjustment module for model-agnostic domain generalization. Adv. Neural. Inf. Process. Syst. 34, 2427–2440 (2021)

    Google Scholar 

  19. Kilic, V., Hegde, D., Sindagi, V., Cooper, A.B., Foster, M.A., Patel, V.M.: Lidar light scattering augmentation (lISA): physics-based simulation of adverse weather conditions for 3D object detection. arXiv preprint arXiv:2107.07004 (2021)

  20. Lang, A.H., Vora, S., Caesar, H., Zhou, L., Yang, J., Beijbom, O.: PointPillars: fast encoders for object detection from point clouds. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12697–12705 (2019)

    Google Scholar 

  21. Lehner, A., et al.: 3D-VField: adversarial augmentation of point clouds for domain generalization in 3D object detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 17295–17304 (2022)

    Google Scholar 

  22. Li, D., Yang, Y., Song, Y.Z., Hospedales, T.: Learning to generalize: Meta-learning for domain generalization. In: Proceedings of the AAAI Conference on Artificial Intelligence. vol. 32 (2018)

    Google Scholar 

  23. Liang, J., He, R., Tan, T.: A comprehensive survey on test-time adaptation under distribution shifts. arXiv preprint arXiv:2303.15361 (2023)

  24. Liang, J., Hu, D., Feng, J.: Do we really need to access the source data? Source hypothesis transfer for unsupervised domain adaptation. In: International Conference on Machine Learning, pp. 6028–6039. PMLR (2020)

    Google Scholar 

  25. Liu, Q., Chen, C., Dou, Q., Heng, P.A.: Single-domain generalization in medical image segmentation via test-time adaptation from shape dictionary. In: Proceedings of the AAAI Conference on Artificial Intelligence. vol. 36, pp. 1756–1764 (2022)

    Google Scholar 

  26. Mao, J., et al.: Voxel transformer for 3D object detection. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 3164–3173 (2021)

    Google Scholar 

  27. Mirza, M.J., et al.: MATE: masked autoencoders are online 3D test-time learners. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 16709–16718 (2023)

    Google Scholar 

  28. Motiian, S., Piccirilli, M., Adjeroh, D.A., Doretto, G.: Unified deep supervised domain adaptation and generalization. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 5715–5725 (2017)

    Google Scholar 

  29. Muandet, K., Balduzzi, D., Schölkopf, B.: Domain generalization via invariant feature representation. In: International Conference on Machine Learning, pp. 10–18. PMLR (2013)

    Google Scholar 

  30. Ouyang, C., et al.: Causality-inspired single-source domain generalization for medical image segmentation. IEEE Trans. Med. Imaging 42(4), 1095–1106 (2022)

    Article  Google Scholar 

  31. Qi, C.R., Yi, L., Su, H., Guibas, L.J.: PointNet++: deep hierarchical feature learning on point sets in a metric space. In: Advances in Neural Information Processing Systems. vol. 30 (2017)

    Google Scholar 

  32. Qian, R., Lai, X., Li, X.: 3D object detection for autonomous driving: a survey. Pattern Recogn. 130, 108796 (2022)

    Article  Google Scholar 

  33. Shi, S., et al.: PV-RCNN: point-voxel feature set abstraction for 3D object detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10529–10538 (2020)

    Google Scholar 

  34. Shi, S., et al.: PV-RCNN++: point-voxel feature set abstraction with local vector representation for 3D object detection. Int. J. Comput. Vision 131(2), 531–551 (2023)

    Article  Google Scholar 

  35. Shi, S., Wang, X., Li, H.: PointRCNN: 3D object proposal generation and detection from point cloud. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 770–779 (2019)

    Google Scholar 

  36. Shi, W., Rajkumar, R.: Point-GNN: graph neural network for 3D object detection in a point cloud. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1711–1719 (2020)

    Google Scholar 

  37. Shi, Z., Chen, Z., Xu, Z., Yang, W., Yu, Z., Huang, L.: Shape prior guided attack: sparser perturbations on 3D point clouds. In: Proceedings of the AAAI Conference on Artificial Intelligence. vol. 36, pp. 8277–8285 (2022)

    Google Scholar 

  38. Soum-Fontez, L., Deschaud, J.E., Goulette, F.: MDT3D: multi-dataset training for lidar 3D object detection generalization. arXiv preprint arXiv:2308.01000 (2023)

  39. Sun, P., et al.: Scalability in perception for autonomous driving: waymo open dataset. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2446–2454 (2020)

    Google Scholar 

  40. Team, O.D.: OpenPCDet: an open-source toolbox for 3D object detection from point clouds. https://github.com/open-mmlab/OpenPCDet (2020)

  41. Vidit, V., Engilberge, M., Salzmann, M.: Clip the gap: a single domain generalization approach for object detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3219–3229 (2023)

    Google Scholar 

  42. Volpi, R., Murino, V.: Addressing model vulnerability to distributional shifts over image transformation sets. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 7980–7989 (2019)

    Google Scholar 

  43. Volpi, R., Namkoong, H., Sener, O., Duchi, J.C., Murino, V., Savarese, S.: Generalizing to unseen domains via adversarial data augmentation. In: Advances in Neural Information Processing Systems. vol. 31 (2018)

    Google Scholar 

  44. Wang, D., Shelhamer, E., Liu, S., Olshausen, B., Darrell, T.: Tent: Fully test-time adaptation by entropy minimization. arXiv preprint arXiv:2006.10726 (2020)

  45. Wang, Y., et al.: Train in Germany, test in the USA: making 3D object detectors generalize. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11713–11723 (2020)

    Google Scholar 

  46. Wang, Z., Luo, Y., Qiu, R., Huang, Z., Baktashmotlagh, M.: Learning to diversify for single domain generalization. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 834–843 (2021)

    Google Scholar 

  47. Wu, A., Deng, C.: Single-domain generalized object detection in urban scene via cyclic-disentangled self-distillation. In: Proceedings of the IEEE/CVF Conference on computer vision and pattern recognition, pp. 847–856 (2022)

    Google Scholar 

  48. Wu, G., Cao, T., Liu, B., Chen, X., Ren, Y.: Towards universal lidar-based 3D object detection by multi-domain knowledge transfer. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 8669–8678 (2023)

    Google Scholar 

  49. Xiao, P., et al.: PandaSet: advanced sensor suite dataset for autonomous driving. In: 2021 IEEE International Intelligent Transportation Systems Conference (ITSC), pp. 3095–3101. IEEE (2021)

    Google Scholar 

  50. Xu, Q., Zhong, Y., Neumann, U.: Behind the curtain: learning occluded shapes for 3D object detection. In: Proceedings of the AAAI Conference on Artificial Intelligence. vol. 36, pp. 2893–2901 (2022)

    Google Scholar 

  51. Yan, Y., Mao, Y., Li, B.: SECOND: sparsely embedded convolutional detection. Sensors 18(10), 3337 (2018)

    Article  Google Scholar 

  52. Yang, J., Shi, S., Wang, Z., Li, H., Qi, X.: ST3D++: denoised self-training for unsupervised domain adaptation on 3D object detection. arXiv preprint arXiv:2108.06682 (2021)

  53. Yang, J., Shi, S., Wang, Z., Li, H., Qi, X.: ST3D: self-training for unsupervised domain adaptation on 3D object detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10368–10378 (2021)

    Google Scholar 

  54. Yang, Q., et al.: Low-dose CT image denoising using a generative adversarial network with Wasserstein distance and perceptual loss. IEEE Trans. Med. Imaging 37(6), 1348–1357 (2018)

    Article  Google Scholar 

  55. Ye, Y., Chen, H., Zhang, C., Hao, X., Zhang, Z.: SARPNET: shape attention regional proposal network for LiDAR-based 3D object detection. Neurocomputing 379, 53–63 (2020)

    Article  Google Scholar 

  56. Zhang, B., Yuan, J., Shi, B., Chen, T., Li, Y., Qiao, Y.: Uni3D: a unified baseline for multi-dataset 3D object detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9253–9262 (2023)

    Google Scholar 

  57. Zheng, Y., Duan, Y., Lu, J., Zhou, J., Tian, Q.: HyperDet3D: learning a scene-conditioned 3D object detector. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5585–5594 (2022)

    Google Scholar 

  58. Zhou, K., Liu, Z., Qiao, Y., Xiang, T., Loy, C.C.: Domain generalization: a survey. IEEE Trans. Pattern Anal. Machine Intell. 45(4), 4396–4415 (2022)

    Google Scholar 

Download references

Acknowledgments

This work is supported in part by Canada CIFAR AI Chairs Program, the Natural Sciences and Engineering Research Council of Canada, Alberta Innovates, as well as JST-Mirai Program Grant No.JPMJMI20B8, JSPS KAKENHI Grant No.JP21H04877, No.JP23H03372, and No.JP24K02920.

Author information

Authors and Affiliations

  1. University of Alberta, Edmonton, Canada

    Shuangzhi Li, Lei Ma & Xingyu Li

  2. The University of Tokyo, Tokyo, Japan

    Lei Ma

Authors
  1. Shuangzhi Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lei Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xingyu Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xingyu Li .

Editor information

Editors and Affiliations

  1. University of Birmingham, Birmingham, UK

    Aleš Leonardis

  2. University of Trento, Trento, Italy

    Elisa Ricci

  3. Technical University of Darmstadt, Darmstadt, Germany

    Stefan Roth

  4. Princeton University, Princeton, NJ, USA

    Olga Russakovsky

  5. Czech Technical University in Prague, Prague, Czech Republic

    Torsten Sattler

  6. École des Ponts ParisTech, Marne-la-Vallée, France

    Gül Varol

1 Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 3375 KB)

Rights and permissions

Reprints and permissions

Copyright information

© 2025 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Li, S., Ma, L., Li, X. (2025). Domain Generalization of 3D Object Detection by Density-Resampling. In: Leonardis, A., Ricci, E., Roth, S., Russakovsky, O., Sattler, T., Varol, G. (eds) Computer Vision – ECCV 2024. ECCV 2024. Lecture Notes in Computer Science, vol 15122. Springer, Cham. https://doi.org/10.1007/978-3-031-73039-9_26

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-73039-9_26

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-73038-2

  • Online ISBN: 978-3-031-73039-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation