Robust extrinsic symmetry estimation in 3D point clouds
Abstract
Detecting the reflection symmetry plane of an object represented by a 3D point cloud is a fundamental problem in 3D computer vision and geometry processing due to its various applications, such as compression, object detection, robotic grasping, 3D surface reconstruction, etc. Several approaches exist to solve this problem for clean 3D point clouds. However, it is a challenging problem to solve in the presence of outliers and missing parts. The existing methods try to overcome this challenge primarily by voting-based techniques but do not work efficiently. In this work, we proposed a statistical estimator-based approach for the plane of reflection symmetry that is robust to outliers and missing parts. We pose the problem of finding the optimal estimator for the reflection symmetry as an optimization problem on a 2-sphere that quickly converges to the global solution for an approximate initialization. We further adapt the heat kernel signature for symmetry invariant matching of mirror symmetric points. This approach helps us to decouple the chicken-and-egg problem of finding the optimal symmetry plane and correspondences between the reflective symmetric points. The proposed approach achieves comparable mean ground-truth error and 4.5% increment in the F-score as compared to the state-of-the-art approaches on the benchmark dataset.
References
[1]
Abbasi A, Kalkan S, and Sahillioğlu Y Deep 3d semantic scene extrapolation Vis. Comput. 2019 35 271-279
[2]
Absil PA, Mahony R, and Sepulchre R Optimization Algorithms on Matrix Manifolds 2009 Princeton Princeton University Press
[3]
Basu A, Harris IR, Hjort NL, and Jones MC Robust and efficient estimation by minimising a density power divergence Biometrika 1998 85 3 549-559
[4]
Belkin, M., Sun, J., Wang, Y.: Constructing Laplace operator from point clouds in Rd. In: Proceedings of the Twentieth Annual ACM-SIAM Symposium on Discrete Algorithms, pp. 1031–1040. SIAM (2009)
[5]
Berner, A., Bokeloh, M., Wand, M., Schilling, A., Seidel, H.P.: A graph-based approach to symmetry detection. In: Volume Graphics, vol. 40, pp. 1–8 (2008)
[6]
Boumal N, Mishra B, Absil PA, and Sepulchre R Manopt, a matlab toolbox for optimization on manifolds J. Mach. Learn. Res. 2014 15 1 1455-1459
[7]
Chang, A.X., Funkhouser, T., Guibas, L., Hanrahan, P., Huang, Q., Li, Z., Savarese, S., Savva, M., Song, S., Su, H., et al.: Shapenet: an information-rich 3d model repository. arXiv preprint arXiv:1512.03012 (2015)
[8]
Cicconet, M., Hildebrand, D.G., Elliott, H.: Finding mirror symmetry via registration and optimal symmetric pairwise assignment of curves: algorithm and results. In: IEEE International Conference on Computer Vision Workshop (ICCVW), pp. 1759–1763. IEEE (2017)
[9]
Cohen, A., Zach, C., Sinha, S.N., Pollefeys, M.: Discovering and exploiting 3d symmetries in structure from motion. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1514–1521. IEEE (2012)
[10]
Combès, B., Hennessy, R., Waddington, J., Roberts, N., Prima, S.: Automatic symmetry plane estimation of bilateral objects in point clouds. In: IEEE Conference on Computer Vision and Pattern Recognition, 2008. CVPR 2008, pp. 1–8. IEEE (2008)
[11]
Cosmo, L., Rodolà, E., Bronstein, M.M., Torsello, A., Cremers, D., Sahillioglu, Y.: Shrec’16: partial matching of deformable shapes. In: Proceedings of the 3DOR, vol. 2, No. 9, p. 12 (2016)
[12]
Ecins, A., Fermuller, C., Aloimonos, Y.: Detecting reflectional symmetries in 3d data through symmetrical fitting. In: Proceedings of the IEEE International Conference on Computer Vision Workshops, pp. 1779–1783 (2017)
[13]
Fletcher R Practical Methods of Optimization 2000 Hoboken Wiley
[14]
Funk, C., Liu, Y.: Symmetry recaptcha. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5165–5174 (2016)
[15]
Funk, C., Lee, S., Oswald, M.R., Tsogkas, S., Shen, W., Cohen, A., Dickinson, S., Liu, Y.: 2017 ICCV challenge: detecting symmetry in the wild. In: Proceedings of the IEEE International Conference on Computer Vision Workshops, pp. 1692–1701 (2017)
[16]
Gao L, Zhang L-X, Meng H-Y, Ren Y-H, Lai Y-K, and Kobbelt L PRS-Net: Planar reflective symmetry detection net for 3d models IEEE Trans. Vis. Comput. Gr. 2020 27 6 3007-3018
[17]
Gnutti A, Guerrini F, and Leonardi R Combining appearance and gradient information for image symmetry detection IEEE Trans. Image Process. 2021 30 5708-5723
[18]
Hauagge, D.C., Snavely, N.: Image matching using local symmetry features. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 206–213. IEEE (2012)
[19]
Hauberg, S., Feragen, A., Black, M.J.: Grassmann averages for scalable robust PCA. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 3810–3817 (2014)
[20]
Hodan, T., Barath, D., Matas, J.: Epos: estimating 6d pose of objects with symmetries. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 11703–11712 (2020)
[21]
Hruda, L., Kolingerová, I., Lávička, M.: Plane space representation in context of mode-based symmetry plane detection. In: International Conference on Computational Science, pp. 509–523. Springer (2020)
[22]
Hruda L, Kolingerová I, and Váša L Robust, fast and flexible symmetry plane detection based on differentiable symmetry measure Vis. Comput. 2021 38 1-17
[23]
Hu L and Kneip L Globally optimal point set registration by joint symmetry plane fitting J. Math. Imaging Vis. 2021 63 1-19
[24]
Jian B and Vemuri BC Robust point set registration using Gaussian mixture models IEEE Trans. Pattern Anal. Mach. Intell. 2010 33 8 1633-1645
[25]
Kazhdan, M., Chazelle, B., Dobkin, D., Finkelstein, A., Funkhouser, T.: A reflective symmetry descriptor. In: European Conference on Computer Vision, pp. 642–656. Springer (2002)
[26]
Kim, V.G., Lipman, Y., Chen, X., Funkhouser, T.: Möbius transformations for global intrinsic symmetry analysis. In: Computer Graphics Forum, vol. 29, pp. 1689–1700. Wiley Online Library (2010)
[27]
Korman, S., Litman, R., Avidan, S., Bronstein, A.: Probably approximately symmetric: fast rigid symmetry detection with global guarantees. In: Computer Graphics Forum, vol. 34, pp 2–13. Wiley Online Library (2015)
[28]
Köser, K., Zach, C., Pollefeys, M.: Dense 3d reconstruction of symmetric scenes from a single image. In: Pattern Recognition, pp. 266–275. Springer (2011)
[29]
Lasowski, R., Tevs, A., Seidel, H.P., Wand, M.: A probabilistic framework for partial intrinsic symmetries in geometric data. In: 2009 IEEE 12th International Conference on Computer Vision, pp. 963–970. IEEE (2009)
[30]
Li B, Johan H, Ye Y, and Yijuan L Efficient 3d reflection symmetry detection: a view-based approach Gr. Models 2016 83 2-14
[31]
Li, Q., Hu, L., Liu, S., Yang, D., Liu, X.: Anisotropic spectral manifold wavelet descriptor. In: Computer Graphics Forum, vol. 40, pp. 81–96. Wiley Online Library (2021)
[32]
Li, R.W., Zhang, L.X., Li, C., Lai, Y.K., Gao, L.: E3sym: Leveraging e (3) invariance for unsupervised 3d planar reflective symmetry detection. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 14543–14553 (2023)
[33]
Lipman, Y., Chen, X., Daubechies, I., Funkhouser, T.: Symmetry factored embedding and distance. In: ACM Transactions on Graphics (TOG), vol. 29, p. 103. ACM (2010)
[34]
Liu, T., Kim, V.G., Funkhouser, T.: Finding surface correspondences using symmetry axis curves. In: Computer Graphics Forum, vol. 31, pp. 1607–1616. Wiley Online Library (2012)
[35]
Liu Y, Prabhakaran B, and Guo X Point-based manifold harmonics IEEE Trans. Vis. Comput. Gr. 2012 18 10 1693-1703
[36]
Liu Y, Hel-Or H, Kaplan CS, and Van Gool L Computational symmetry in computer vision and computer graphics Found. Trends Comput. Gr. Vis. 2010 5 1–2 1-195
[37]
Lukáč M, Sýkora D, Sunkavalli K, Shechtman E, Jamriška O, Carr N, and Pajdla T Nautilus: recovering regional symmetry transformations for image editing ACM Trans. Gr. 2017 36 4 1-11
[38]
Ma, J., Zhao, J., Tian, J., Tu, Z., Yuille, A.L.: Robust estimation of nonrigid transformation for point set registration. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2147–2154 (2013)
[39]
Mancas, M., Gosselin, B., Macq, B.: Fast and automatic tumoral area localisation using symmetry. In: Proceedings.(ICASSP’05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005, vol. 2, pp. 725–728. IEEE (2005)
[40]
Martinet A, Soler C, Holzschuch N, and Sillion FX Accurate detection of symmetries in 3d shapes ACM Trans. Gr. 2006 25 2 439-464
[41]
Melzi S, Ren J, Rodolà E, Sharma A, Wonka P, and Ovsjanikov M Zoomout: spectral upsampling for efficient shape correspondence ACM Trans. Gr. 2019 38 6 1-14
[42]
Miglani, A., Roy, S.D., Chaudhury, S., Srivastava, J.B.: Symmetry based 3d reconstruction of repeated cylinders. In: 2013 Fourth National Conference on Computer Vision, Pattern Recognition, Image Processing and Graphics (NCVPRIPG), pp. 1–4. IEEE (2013)
[43]
Mitra, N.J., Guibas, L.J., Pauly, M.: Partial and approximate symmetry detection for 3d geometry. In: ACM Transactions on Graphics (TOG), vol. 25, pp. 560–568. ACM (2006)
[44]
Mitra, N.J., Pauly, M., Wand, M., Ceylan, D.: Symmetry in 3d geometry: extraction and applications. In: Computer Graphics Forum, vol. 32, pp. 1–23. Wiley Online Library (2013)
[45]
Mitra, S., Liu, Y.: Local facial asymmetry for expression classification. In: Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004, vol. 2, pp. II–889. IEEE (2004)
[46]
Nagar, R., Raman, S.: Fast and accurate intrinsic symmetry detection. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 417–434 (2018)
[47]
Nagar R and Raman S Detecting approximate reflection symmetry in a point set using optimization on manifold IEEE Trans. Signal Process. 2019 67 6 1582-1595
[48]
Nagar R and Raman S 3dsymm: robust and accurate 3d reflection symmetry detection Pattern Recognit. 2020 107 107483
[49]
Ovsjanikov, M., Sun, J., Guibas, L.: Global intrinsic symmetries of shapes. In: Computer Graphics Forum, vol. 27, pp. 1341–1348. Wiley Online Library (2008)
[50]
Podolak J, Shilane P, Golovinskiy A, Rusinkiewicz S, and Funkhouser T A planar-reflective symmetry transform for 3d shapes ACM Trans. Gr. 2006 25 3 549-559
[51]
Qiao, Y.L., Gao, L., Liu, S.Z., Liu, L., Lai, Y.K., Chen, X.: Learning-based real-time detection of intrinsic reflectional symmetry. arXiv preprint arXiv:1911.00189 (2019)
[52]
Ren J, Melzi S, Ovsjanikov M, and Wonka P Maptree: recovering multiple solutions in the space of maps ACM Trans. Gr. 2020 39 6 1-17
[53]
Sahillioğlu Y Recent advances in shape correspondence Vis. Comput. 2020 36 8 1705-1721
[54]
Sahillioğlu, Y., Yemez, Y.: Coarse-to-fine isometric shape correspondence by tracking symmetric flips. In: Computer Graphics Forum, vol. 32, pp. 177–189. Wiley Online Library (2013)
[55]
Seo, A., Shim, W., Cho, M.: Learning to discover reflection symmetry via polar matching convolution. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 1285–1294 (2021)
[56]
Sharp, N., Crane, K.: A Laplacian for nonmanifold triangle meshes. In: Computer Graphics Forum, vol. 39, pp. 69–80. Wiley Online Library (2020)
[57]
Shi Y, Huang J, Zhang H, Xin X, Rusinkiewicz S, and Kai X SymmetryNet: learning to predict reflectional and rotational symmetries of 3d shapes from single-view RGB-d images ACM Trans. Gr. 2020 39 6 1-14
[58]
Shi Y, Xin X, Xi J, Xiaochang H, Dewen H, and Kai X Learning to detect 3d symmetry from single-view RGB-d images with weak supervision IEEE Trans. Pattern Anal. Mach. Intell. 2022 45 4 4882-4896
[59]
Shi, Z., Alliez, P., Desbrun, M., Bao, H., Huang, J.: Symmetry and orbit detection via lie-algebra voting. In: Computer Graphics Forum, vol. 35, pp. 217–227. Wiley Online Library (2016)
[60]
Sipiran, I., Gregor, R., Schreck, T.: Approximate symmetry detection in partial 3d meshes. In: Computer Graphics Forum, vol. 33, pp. 131–140. Wiley Online Library (2014)
[61]
Sipiran, I., Romanengo, C., Falcidieno, B., Biasotti, S., Arvanitis, G., Chen, C., Fotis, V., He, J., Lv, X., Moustakas, K., et al.: Shrec 2023: detection of symmetries on 3d point clouds representing simple shapes. In: Eurographics Workshop on 3D Object Retrieval, pp. 17–237. The Eurographics Association (2023)
[62]
Speciale, P., Oswald, M.R., Cohen, A., Pollefeys, M.: A symmetry prior for convex variational 3d reconstruction. In: European Conference on Computer Vision, pp. 313–328. Springer (2016)
[63]
Sun, J., Ovsjanikov, M., Guibas, L.: A concise and provably informative multi-scale signature based on heat diffusion. In: Computer Graphics Forum, vol. 28, pp. 1383–1392. Wiley Online Library (2009)
[64]
Sung M, Kim VG, Angst R, and Guibas L Data-driven structural priors for shape completion ACM Trans. Gr. 2015 34 6 1-11
[65]
Tan VYF and Févotte C Automatic relevance determination in nonnegative matrix factorization with the/spl beta/-divergence IEEE Trans. Pattern Anal. Mach. Intell. 2012 35 7 1592-1605
[66]
Thomas DM and Natarajan V Multiscale symmetry detection in scalar fields by clustering contours IEEE Trans. Vis. Comput. Gr. 2014 20 12 2427-2436
[67]
Thrun, S., Wegbreit, B.: Shape from symmetry. In: Tenth IEEE International Conference on Computer Vision, 2005. ICCV 2005, vol. 2, pp. 1824–1831. IEEE (2005)
[68]
Tyler CW Human Symmetry Perception and Its Computational Analysis 2003 London Psychology Press
[69]
Van Kaick, O., Zhang, H., Hamarneh, G., Cohen-Or, D.: A survey on shape correspondence. In: Computer Graphics Forum, vol. 30, pp. 1681–1707. Wiley Online Library (2011)
[70]
Wang, H., Huang, H.: Group representation of global intrinsic symmetries. In: Computer Graphics Forum, vol. 36, pp. 51–61. Wiley Online Library (2017)
[71]
Wang, W., Ma, J., Xu, P., Chu, Y.,: Intrinsic symmetry detection on 3d models with skeleton-guided combination of extrinsic symmetries. In: Computer Graphics Forum, vol. 38, pp. 617–628. Wiley Online Library (2019)
[72]
Wang, Y., Guo, J., Yan, D.M., Wang, K., Zhang, X.: A robust local spectral descriptor for matching non-rigid shapes with incompatible shape structures. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6231–6240 (2019)
[73]
Wang Y, Ren J, Yan D-M, Guo J, Zhang X, and Wonka P MGCN: descriptor learning using multiscale GCNs ACM Trans. Gr. 2020 39 4 122-1
[74]
Wu, S., Rupprecht, C., Vedaldi, A.: Unsupervised learning of probably symmetric deformable 3d objects from images in the wild. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1–10 (2020)
[75]
Xu K, Zhang H, Jiang W, Dyer R, Cheng Z, Liu L, and Chen B Multi-scale partial intrinsic symmetry detection ACM Trans. Gr. 2012 31 6 181
[76]
Xu, K., Zhang, H., Tagliasacchi, A., Liu, L., Li, G., Meng, M., Xiong, Y.: Partial intrinsic reflectional symmetry of 3d shapes. In: ACM SIGGRAPH Asia 2009 papers, pp. 1–10 (2009)
[77]
Zabrodsky H, Peleg S, and Avnir D Symmetry as a continuous feature IEEE Trans. Pattern Anal. Mach. Intell. 1995 17 12 1154-1166
[78]
Zhang, Z., Dong, B., Li, T., Heide, F., Peers, P., Yin, B., Yang, X.: Single depth-image 3d reflection symmetry and shape prediction. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 8896–8906 (2023)
[79]
Zhong, Y.: Intrinsic shape signatures: A shape descriptor for 3d object recognition. In: 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops, pp. 689–696. IEEE (2009)
[80]
Zhou, Y., Liu, S., Ma, Y.: Nerd: neural 3d reflection symmetry detector. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 15940–15949 (2021)
Index Terms
- Robust extrinsic symmetry estimation in 3D point clouds
Index terms have been assigned to the content through auto-classification.
Recommendations
Symmetry group detection of point clouds in 3D via a decomposition method
AbstractAnalyzing the symmetries present in point clouds, which represent sets of 3D coordinates, is important for understanding their underlying structure and facilitating various applications. In this paper, we propose a novel decomposition-based ...
Graphical abstractHighlights- We present a theoretical framework to assess exact point clouds symmetries via decomposition-based method.
- The exact symmetry group of the 3D point cloud is derived from the symmetries of simpler components.
- The method can be ...
Skeleton-based intrinsic symmetry detection on point clouds
We present a skeleton-based algorithm for intrinsic symmetry detection on imperfect 3D point cloud data. The data imperfections such as noise and incompleteness make it difficult to reliably compute geodesic distances, which play essential roles in ...
Efficient 3D reflection symmetry detection
Propose a novel and efficient view-based symmetry detection algorithm.Demonstrate better accuracy and efficiency compared with two latest approaches.Dataset-level evaluation and two related applications validate good robustness and flexibility. Display ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
© The Author(s), under exclusive licence to Springer-Verlag GmbH Germany, part of Springer Nature 2024.
Publisher
Springer-Verlag
Berlin, Heidelberg
Publication History
Published: 15 March 2024
Accepted: 10 February 2024
Author Tags
Author Tag
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 18 Feb 2025