research-article

On the Robustness of 3D Object Detectors

Authors:

Fatima Albreiki,

Sultan Abu Ghazal,

Hisham Cholakkal,

Fahad KhanAuthors Info & Claims

MMAsia '22: Proceedings of the 4th ACM International Conference on Multimedia in Asia

Article No.: 22, Pages 1 - 7

https://doi.org/10.1145/3551626.3564956

Published: 13 December 2022 Publication History

Abstract

In recent years, significant progress has been achieved for 3D object detection on point clouds thanks to the advances in 3D data collection and deep learning techniques. Nevertheless, 3D scenes exhibit a lot of variations and are prone to sensor inaccuracies as well as information loss during pre-processing. Thus, it is crucial to design techniques that are robust against these variations. This requires a detailed analysis and understanding of the effect of such variations. This work aims to analyze and benchmark popular point-based 3D object detectors against several data corruptions. To the best of our knowledge, we are the first to investigate the robustness of point-based 3D object detectors. To this end, we design and evaluate corruptions that involve data addition, reduction, and alteration. We further study the robustness of different modules against local and global variations. Our experimental results reveal several intriguing findings. For instance, we show that methods that integrate Transformers at a patch or object level lead to increased robustness, compared to using Transformers at the point level. The code is available at https://github.com/sultanabughazal/robustness3d.

Supplementary Material

PDF File (a22-albreiki-supp.pdf)

Supplemental material.

Download
543.70 KB

References

[1]

Xiaozhi Chen, Huimin Ma, Ji Wan, Bo Li, and Tian Xia. 2017. Multi-view 3d object detection network for autonomous driving. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition. 1907--1915.

[2]

Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schiele. 2016. The cityscapes dataset for semantic urban scene understanding. In Proceedings of the IEEE conference on computer vision and pattern recognition. 3213--3223.

[3]

Angela Dai, Angel X. Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, and Matthias Nießner. 2017. ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes. In Proc. Computer Vision and Pattern Recognition (CVPR), IEEE.

[4]

Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. 2009. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition. Ieee, 248--255.

[5]

Mark Everingham, Luc Van Gool, Christopher KI Williams, John Winn, and Andrew Zisserman. 2010. The pascal visual object classes (voc) challenge. International journal of computer vision 88, 2 (2010), 303--338.

Digital Library

[6]

Dan Hendrycks, Steven Basart, Norman Mu, Saurav Kadavath, Frank Wang, Evan Dorundo, Rahul Desai, Tyler Zhu, Samyak Parajuli, Mike Guo, Dawn Song, Jacob Steinhardt, and Justin Gilmer. 2021. The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization. ICCV (2021).

[7]

Dan Hendrycks and Thomas Dietterich. 2019. Benchmarking Neural Network Robustness to Common Corruptions and Perturbations. Proceedings of the International Conference on Learning Representations (2019).

[8]

Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. 2014. Microsoft coco: Common objects in context. In European conference on computer vision. Springer, 740--755.

[9]

Ze Liu, Zheng Zhang, Yue Cao, Han Hu, and Xin Tong. 2021. Group-Free 3D Object Detection via Transformers. 2021 IEEE/CVF International Conference on Computer Vision (ICCV) (2021), 2929--2938.

[10]

Claudio Michaelis, Benjamin Mitzkus, Robert Geirhos, Evgenia Rusak, Oliver Bringmann, Alexander S. Ecker, Matthias Bethge, and Wieland Brendel. 2019. Benchmarking Robustness in Object Detection: Autonomous Driving when Winter is Coming. arXiv preprint arXiv:1907.07484 (2019).

[11]

Muhammad Jehanzeb Mirza, Cornelius Buerkle, Julio Jarquin, Michael Opitz, Fabian Oboril, Kay-Ulrich Scholl, and Horst Bischof. 2021. Robustness of Object Detectors in Degrading Weather Conditions. 2021 IEEE International Intelligent Transportation Systems Conference (ITSC) (2021), 2719--2724.

[12]

Ishan Misra, Rohit Girdhar, and Armand Joulin. 2021. An End-to-End Transformer Model for 3D Object Detection. 2021 IEEE/CVF International Conference on Computer Vision (ICCV) (2021), 2886--2897.

[13]

Charles R Qi, Or Litany, Kaiming He, and Leonidas J Guibas. 2019. Deep Hough Voting for 3D Object Detection in Point Clouds. In Proceedings of the IEEE International Conference on Computer Vision.

[14]

Charles R. Qi, Hao Su, Kaichun Mo, and Leonidas J. Guibas. 2017. PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[15]

Charles R. Qi, Li Yi, Hao Su, and Leonidas J. Guibas. 2017. PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space (NIPS'17). Curran Associates Inc., Red Hook, NY, USA, 5105--5114.

[16]

Xie Qian, Lai Yu-kun, Wu Jing, Wang Zhoutao, Zhang Yiming, Xu Kai, and Wang Jun. 2020. MLCVNet: Multi-Level Context VoteNet for 3D Object Detection. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[17]

Jiawei Ren, Liang Pan, and Ziwei Liu. 2022. Benchmarking and Analyzing Point Cloud Classification under Corruptions. CoRR abs/2202.03377 (2022). arXiv:2202.03377 https://arxiv.org/abs/2202.03377

[18]

Shaoshuai Shi, Chaoxu Guo, Li Jiang, Zhe Wang, Jianping Shi, Xiaogang Wang, and Hongsheng Li. 2020. Pv-rcnn: Point-voxel feature set abstraction for 3d object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 10529--10538.

[19]

Jiachen Sun, Qingzhao Zhang, Bhavya Kailkhura, Zhiding Yu, Chaowei Xiao, and Z. Morley Mao. 2022. Benchmarking Robustness of 3D Point Cloud Recognition Against Common Corruptions. CoRR abs/2201.12296 (2022). arXiv:2201.12296 https://arxiv.org/abs/2201.12296

[20]

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is All You Need. In Proceedings of the 31st International Conference on Neural Information Processing Systems (Long Beach, California, USA) (NIPS'17). Curran Associates Inc., Red Hook, NY, USA, 6000--6010.

Digital Library

[21]

Yue Wang, Alireza Fathi, Abhijit Kundu, David A Ross, Caroline Pantofaru, Tom Funkhouser, and Justin Solomon. 2020. Pillar-based object detection for autonomous driving. In European Conference on Computer Vision. Springer, 18--34.

Digital Library

[22]

Yue Wang, Yongbin Sun, Ziwei Liu, Sanjay E Sarma, Michael M Bronstein, and Justin M Solomon. 2019. Dynamic graph cnn for learning on point clouds. Acm Transactions On Graphics (tog) 38, 5 (2019), 1--12.

Digital Library

[23]

Zhirong Wu, Shuran Song, Aditya Khosla, Fisher Yu, Linguang Zhang, Xiaoou Tang, and Jianxiong Xiao. 2015. 3d shapenets: A deep representation for volumetric shapes. In Proceedings of the IEEE conference on computer vision and pattern recognition. 1912--1920.

[24]

Yan Yan, Yuxing Mao, and Bo Li. 2018. Second: Sparsely embedded convolutional detection. Sensors 18, 10 (2018), 3337.

[25]

Bin Yang, Wenjie Luo, and Raquel Urtasun. 2018. Pixor: Real-time 3d object detection from point clouds. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition. 7652--7660.

[26]

Yin Zhou and Oncel Tuzel. 2018. Voxelnet: End-to-end learning for point cloud based 3d object detection. In Proceedings of the IEEE conference on computer vision and pattern recognition. 4490--4499.

Cited By

Puy GGidaris SBoulch ASiméoni OSautier CPérez PBursuc AMarlet R(2024)Three Pillars Improving Vision Foundation Model Distillation for Lidar2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52733.2024.02033(21519-21529)Online publication date: 16-Jun-2024
https://doi.org/10.1109/CVPR52733.2024.02033
Kong LLiu YLi XChen RZhang WRen JPan LChen KLiu Z(2023)Robo3D: Towards Robust and Reliable 3D Perception against Corruptions2023 IEEE/CVF International Conference on Computer Vision (ICCV)10.1109/ICCV51070.2023.01830(19937-19949)Online publication date: 1-Oct-2023
https://doi.org/10.1109/ICCV51070.2023.01830
Zhu ZZhang YChen HDong YZhao SDing WZhong JZheng S(2023)Understanding the Robustness of 3D Object Detection with Bird'View Representations in Autonomous Driving2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52729.2023.02069(21600-21610)Online publication date: Jun-2023
https://doi.org/10.1109/CVPR52729.2023.02069

Index Terms

On the Robustness of 3D Object Detectors
1. Computing methodologies

Index terms have been assigned to the content through auto-classification.

Recommendations

A Comprehensive Study of the Robustness for LiDAR-Based 3D Object Detectors Against Adversarial Attacks
Abstract
Recent years have witnessed significant advancements in deep learning-based 3D object detection, leading to its widespread adoption in numerous applications. As 3D object detectors become increasingly crucial for security-critical tasks, it is ...
Benchmarking Object Detection Robustness against Real-World Corruptions
Abstract
With the rapid recent development, deep learning based object detection techniques have been applied to various real-world software systems, especially in safety-critical applications like autonomous driving. However, few studies are conducted to ...
Trimming stability selection increases variable selection robustness
Abstract
Contamination can severely distort an estimator unless the estimation procedure is suitably robust. This is a well-known issue and has been addressed in Robust Statistics, however, the relation of contamination and distorted variable selection has ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

MMAsia '22: Proceedings of the 4th ACM International Conference on Multimedia in Asia

December 2022

296 pages

ISBN:9781450394789

DOI:10.1145/3551626

Conference Chair:
Shuqiang Jiang
CASROLE@GENERAL CHAIR
,
General Chairs:
Kiyoharu Aizawa
The University of Tokyo
,
Phoebe Chen
La Trobe
,
Keiji Yanai
The University of Electro-Communications

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 13 December 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Mohamed Bin Zayed University of Artificial Intelligence, Abu Dhabi, UAE

Conference

MMAsia '22

Sponsor:

SIGMM

MMAsia '22: ACM Multimedia Asia

December 13 - 16, 2022

Tokyo, Japan

Acceptance Rates

Overall Acceptance Rate 59 of 204 submissions, 29%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
84
Total Downloads

Downloads (Last 12 months)24
Downloads (Last 6 weeks)1

Reflects downloads up to 19 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Puy GGidaris SBoulch ASiméoni OSautier CPérez PBursuc AMarlet R(2024)Three Pillars Improving Vision Foundation Model Distillation for Lidar2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52733.2024.02033(21519-21529)Online publication date: 16-Jun-2024
https://doi.org/10.1109/CVPR52733.2024.02033
Kong LLiu YLi XChen RZhang WRen JPan LChen KLiu Z(2023)Robo3D: Towards Robust and Reliable 3D Perception against Corruptions2023 IEEE/CVF International Conference on Computer Vision (ICCV)10.1109/ICCV51070.2023.01830(19937-19949)Online publication date: 1-Oct-2023
https://doi.org/10.1109/ICCV51070.2023.01830
Zhu ZZhang YChen HDong YZhao SDing WZhong JZheng S(2023)Understanding the Robustness of 3D Object Detection with Bird'View Representations in Autonomous Driving2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52729.2023.02069(21600-21610)Online publication date: Jun-2023
https://doi.org/10.1109/CVPR52729.2023.02069

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents