research-article

MGCN: descriptor learning using multiscale GCNs

Authors:

Xiaopeng Zhang,

Peter WonkaAuthors Info & Claims

ACM Transactions on Graphics (TOG), Volume 39, Issue 4

Article No.: 122, Pages 122:1 - 122:15

https://doi.org/10.1145/3386569.3392443

Published: 12 August 2020 Publication History

Abstract

We propose a novel framework for computing descriptors for characterizing points on three-dimensional surfaces. First, we present a new non-learned feature that uses graph wavelets to decompose the Dirichlet energy on a surface. We call this new feature Wavelet Energy Decomposition Signature (WEDS). Second, we propose a new Multiscale Graph Convolutional Network (MGCN) to transform a non-learned feature to a more discriminative descriptor. Our results show that the new descriptor WEDS is more discriminative than the current state-of-the-art non-learned descriptors and that the combination of WEDS and MGCN is better than the state-of-the-art learned descriptors. An important design criterion for our descriptor is the robustness to different surface discretizations including triangulations with varying numbers of vertices. Our results demonstrate that previous graph convolutional networks significantly overfit to a particular resolution or even a particular triangulation, but MGCN generalizes well to different surface discretizations. In addition, MGCN is compatible with previous descriptors and it can also be used to improve the performance of other descriptors, such as the heat kernel signature, the wave kernel signature, or the local point signature.

Supplemental Material

MP4 File

Presentation video

Transcript for: Presentation video

ZIP File

Supplemental files.

Download
35.58 KB

References

[1]

Dragomir Anguelov, Praveen Srinivasan, Daphne Koller, Sebastian Thrun, Jim Rodgers, and James Davis. 2005. SCAPE: shape completion and animation of people. ACM Trans. on Graphics (Proc. SIGGRAPH) 24, 3 (2005), 408--416.

Digital Library

[2]

Mathieu Aubry, Ulrich Schlickewei, and Daniel Cremers. 2011. The wave kernel signature: A quantum mechanical approach to shape analysis. In IEEE International Conference on Computer Vision Workshops. 1626--1633.

[3]

Silvia Biasotti, Andrea Cerri, Alex Bronstein, and Michael Bronstein. 2016. Recent trends, applications, and perspectives in 3D shape similarity assessment. Computer Graphics Forum 35, 6 (2016), 87--119.

Digital Library

[4]

Federica Bogo, Javier Romero, Matthew Loper, and Michael J Black. 2014. FAUST: Dataset and evaluation for 3D mesh registration. In IEEE Computer Vision and Pattern Recognition (CVPR). 3794--3801.

[5]

Davide Boscaini, Jonathan Masci, Simone Melzi, Michael M. Bronstein, Umberto Castellani, and Pierre Vandergheynst. 2015. Learning class-specific descriptors for deformable shapes using localized spectral convolutional networks. Computer Graphics Forum 34, 5 (2015), 13--23.

[6]

Davide Boscaini, Jonathan Masci, Emanuele Rodolà, Michael M Bronstein, and Daniel Cremers. 2016. Anisotropic diffusion descriptors. Computer Graphics Forum 35, 2 (2016), 431--441.

[7]

Michael M Bronstein, Joan Bruna, Yann LeCun, Arthur Szlam, and Pierre Vandergheynst. 2017. Geometric deep learning: going beyond euclidean data. IEEE Signal Processing Magazine 34, 4 (2017), 18--42.

[8]

Michael M Bronstein and Iasonas Kokkinos. 2010. Scale-invariant heat kernel signatures for non-rigid shape recognition. In IEEE Computer Vision and Pattern Recognition (CVPR). 1704--1711.

[9]

Joan Bruna, Wojciech Zaremba, Arthur Szlam, and Yann Lecun. 2014. Spectral networks and locally connected networks on graphs. In International Conference on Learning Representations.

[10]

Michaël Defferrard, Xavier Bresson, and Pierre Vandergheynst. 2016. Convolutional neural networks on graphs with fast localized spectral filtering. In Advances in Neural Information Processing Systems. 3844--3852.

[11]

Haowen Deng, Tolga Birdal, and Slobodan Ilic. 2018. Ppfnet: Global context aware local features for robust 3D point matching. In IEEE Computer Vision and Pattern Recognition (CVPR). 195--205.

[12]

Nicolas Donati, Abhishek Sharma, and Maks Ovsjanikov. 2020. Deep Geometric Functional Maps: Robust Feature Learning for Shape Correspondence. arXiv preprint arXiv:2003.14286 (2020).

[13]

Matthias Fey, Jan Eric Lenssen, Frank Weichert, and Heinrich Müller. 2018. SplineCNN: Fast geometric deep learning with continuous B-spline kernels. In IEEE Computer Vision and Pattern Recognition (CVPR). 869--877.

[14]

Andrea Frome, Daniel Huber, Ravi Kolluri, Thomas Bülow, and Jitendra Malik. 2004. Recognizing objects in range data using regional point descriptors. In European Conference on Computer Vision (ECCV). 224--237.

[15]

Shunwang Gong, Lei Chen, Michael Bronstein, and Stefanos Zafeiriou. 2019. Spiral-Net++: A Fast and Highly Efficient Mesh Convolution Operator. In IEEE International Conference on Computer Vision Workshops. 4141--4148.

[16]

Jianwei Guo, Hanyu Wang, Zhanglin Cheng, Xiaopeng Zhang, and Dong-Ming Yan. 2020. Learning local shape descriptors for computing non-rigid dense correspondence. Computational Visual Media 6, 1 (2020), 95--112.

[17]

Yulan Guo, Ferdous Sohel, Mohammed Bennamoun, Min Lu, and Jianwei Wan. 2013. Rotational projection statistics for 3D local surface description and object recognition. Int. Journal of Computer Vision 105, 1 (2013), 63--86.

[18]

Oshri Halimi, Or Litany, Emanuele Rodola, Alex M Bronstein, and Ron Kimmel. 2019. Unsupervised learning of dense shape correspondence. In IEEE Computer Vision and Pattern Recognition (CVPR). 4370--4379.

[19]

David K Hammond, Pierre Vandergheynst, and Rémi Gribonval. 2011. Wavelets on graphs via spectral graph theory. Applied and Computational Harmonic Analysis 30, 2 (2011), 129--150.

[20]

Rana Hanocka, Amir Hertz, Noa Fish, Raja Giryes, Shachar Fleishman, and Daniel Cohen-Or. 2019. MeshCNN: a network with an edge. ACM Trans. on Graphics (Proc. SIGGRAPH) 38, 4 (2019), 90:1--90:12.

Digital Library

[21]

Haibin Huang, Evangelos Kalogerakis, Siddhartha Chaudhuri, Duygu Ceylan, Vladimir G Kim, and Ersin Yumer. 2018. Learning Local Shape Descriptors from Part Correspondences with Multiview Convolutional Networks. ACM Transactions on Graphics 37, 1 (2018), 6:1--6:14.

Digital Library

[22]

A. E. Johnson and M. Hebert. 1999. Using spin images for efficient object recognition in cluttered 3D scenes. IEEE Trans. on Pattern Analysis and Machine Intelligence 21, 5 (1999), 433--449.

Digital Library

[23]

Marc Khoury, Qian-Yi Zhou, and Vladlen Koltun. 2017. Learning compact geometric features. In IEEE Computer Vision and Pattern Recognition (CVPR). 153--161.

[24]

Vladimir G Kim, Yaron Lipman, and Thomas Funkhouser. 2011. Blended intrinsic maps. ACM Trans. on Graphics (Proc. SIGGRAPH) 30, 4 (2011), 79:1--79:12.

Digital Library

[25]

Thomas N. Kipf and Max Welling. 2017. Semi-Supervised Classification with Graph Convolutional Networks. In ICLR Posters.

[26]

Isaak Lim, Alexander Dielen, Marcel Campen, and Leif Kobbelt. 2018. A Simple Approach to Intrinsic Correspondence Learning on Unstructured 3D Meshes. In European Conference on Computer Vision Workshops . 349--362.

[27]

Or Litany, Tal Remez, Emanuele Rodolà, Alexander M Bronstein, and Michael M Bronstein. 2017. Deep Functional Maps: Structured Prediction for Dense Shape Correspondence. In IEEE International Conference on Computer Vision (ICCV). 5660--5668.

[28]

Roee Litman and Alexander M Bronstein. 2014. Learning spectral descriptors for deformable shape correspondence. IEEE Trans. on Pattern Analysis and Machine Intelligence 36, 1 (2014), 171--180.

Digital Library

[29]

Jonathan Masci, Davide Boscaini, Michael Bronstein, and Pierre Vandergheynst. 2015. Geodesic convolutional neural networks on riemannian manifolds. In IEEE International Conference on Computer Vision Workshops. 37--45.

Digital Library

[30]

Majid Masoumi and A Ben Hamza. 2017. Spectral shape classification: A deep learning approach. Journal of Visual Communication and Image Representation 43 (2017), 198--211.

Digital Library

[31]

Simone Melzi, Maks Ovsjanikov, Giorgio Roffo, Marco Cristani, and Umberto Castellani. 2018. Discrete time evolution process descriptor for shape analysis and matching. ACM Transactions on Graphics 37, 1 (2018), 4:1--4:18.

Digital Library

[32]

Simone Melzi, Jing Ren, Emanuele Rodolà, Abhishek Sharma, Peter Wonka, and Maks Ovsjanikov. 2019. Zoomout: Spectral upsampling for efficient shape correspondence. ACM Trans. on Graphics (Proc. SIGGRAPH Asia) 38, 6 (2019), 155:1--155:14.

[33]

Anastasiia Mishchuk, Dmytro Mishkin, Filip Radenovic, and Jiri Matas. 2017. Working hard to know your neighbor's margins: Local descriptor learning loss. In Advances in Neural Information Processing Systems. 4826--4837.

[34]

Maks Ovsjanikov, Mirela Ben-Chen, Justin Solomon, Adrian Butscher, and Leonidas Guibas. 2012. Functional maps: a flexible representation of maps between shapes. ACM Trans. on Graphics (Proc. SIGGRAPH) 31, 4 (2012), 30:1--30:11.

Digital Library

[35]

Martin Reuter, Franz-Erich Wolter, and Niklas Peinecke. 2006. Laplace-Beltrami spectra as 'Shape-DNA' of surfaces and solids. Computer-Aided Design 38, 4 (2006), 342--366.

Digital Library

[36]

Raif M Rustamov. 2007. Laplace-Beltrami eigenfunctions for deformation invariant shape representation. In Proc. of Symp. of Geometry Processing. 225--233.

[37]

Olga Sorkine and Marc Alexa. 2007. As-rigid-as-possible surface modeling. In Proc. of Symp. of Geometry Processing. 109--116.

[38]

Ljubiša Stanković and Ervin Sejdić. 2019. Vertex-frequency analysis of graph signals. Springer.

[39]

Jian Sun, Maks Ovsjanikov, and Leonidas J. Guibas. 2010. A Concise and Provably Informative Multi-Scale Signature Based on Heat Diffusion. Computer Graphics Forum 28, 5 (2010), 1383--1392.

[40]

Federico Tombari, Samuele Salti, and Luigi Di Stefano. 2010. Unique signatures of histograms for local surface description. In European Conference on Computer Vision (ECCV). 356--369.

[41]

Nitika Verma, Edmond Boyer, and Jakob Verbeek. 2018. Feastnet: Feature-steered graph convolutions for 3d shape analysis. In IEEE Computer Vision and Pattern Recognition (CVPR). 2598--2606.

[42]

Hanyu Wang, Jianwei Guo, Dong-Ming Yan, Weize Quan, and Xiaopeng Zhang. 2018. Learning 3D Keypoint Descriptors for Non-rigid Shape Matching. In European Conference on Computer Vision (ECCV). 3--20.

Digital Library

[43]

Yiqun Wang, Jianwei Guo, Dong-Ming Yan, Kai Wang, and Xiaopeng Zhang. 2019a. A Robust Local Spectral Descriptor for Matching Non-Rigid Shapes With Incompatible Shape Structures. In IEEE Computer Vision and Pattern Recognition (CVPR). 6231--6240.

[44]

Yue Wang, Yongbin Sun, Ziwei Liu, Sanjay E Sarma, Michael M Bronstein, and Justin M Solomon. 2019b. Dynamic graph cnn for learning on point clouds. ACM Transactions on Graphics 38, 5 (2019), 146:1--146:12.

Digital Library

[45]

Yiqun Wang, Dong-Ming Yan, Xiaohan Liu, Chengcheng Tang, Jianwei Guo, Xiaopeng Zhang, and Peter Wonka. 2019c. Isotropic surface remeshing without large and small angles. IEEE Trans. on Vis. and Comp. Graphics 25, 7 (2019), 2430--2442.

[46]

Lingyu Wei, Qixing Huang, Duygu Ceylan, Etienne Vouga, and Hao Li. 2016. Dense human body correspondences using convolutional networks. In IEEE Computer Vision and Pattern Recognition (CVPR). 1544--1553.

[47]

Bingbing Xu, Huawei Shen, Qi Cao, Yunqi Qiu, and Xueqi Cheng. 2019. Graph Wavelet Neural Network. In ICLR (Poster).

[48]

Dong-Ming Yan, Guanbo Bao, Xiaopeng Zhang, and Peter Wonka. 2014. Low-resolution remeshing using the localized restricted Voronoi diagram. IEEE Trans. on Vis. and Comp. Graphics 20, 10 (2014), 1418--1427.

[49]

Andrei Zaharescu, Edmond Boyer, Kiran Varanasi, and Radu Horaud. 2009. Surface feature detection and description with applications to mesh matching. In IEEE Computer Vision and Pattern Recognition (CVPR). 373--380.

[50]

Andy Zeng, Shuran Song, Matthias Nießner, Matthew Fisher, Jianxiong Xiao, and Thomas Funkhouser. 2017. 3DMatch: Learning local geometric descriptors from rgb-d reconstructions. In IEEE Computer Vision and Pattern Recognition (CVPR). 199--208.

[51]

Silvia Zuffi, Angjoo Kanazawa, David Jacobs, and Michael J. Black. 2017. 3D Menagerie: Modeling the 3D Shape and Pose of Animals. In IEEE Computer Vision and Pattern Recognition (CVPR). 6365--6373.

Cited By

Liu SWang HYan DLi QLuo FTeng ZLiu X(2025)Spectral Descriptors for 3D Deformable Shape Matching: A Comparative SurveyIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2024.336808331:3(1677-1697)Online publication date: Mar-2025
https://doi.org/10.1109/TVCG.2024.3368083
Nagar R(2025)Robust extrinsic symmetry estimation in 3D point cloudsThe Visual Computer: International Journal of Computer Graphics10.1007/s00371-024-03313-641:1(115-128)Online publication date: 1-Jan-2025
https://dl.acm.org/doi/10.1007/s00371-024-03313-6
Hu LWang HYang XXu HLiao Y(2024)AFSMWD: A Descriptor Flexibly Encoding Multiscale and Oriented Shape FeaturesMathematics10.3390/math1218294612:18(2946)Online publication date: 22-Sep-2024
https://doi.org/10.3390/math12182946
Show More Cited By

Index Terms

MGCN: descriptor learning using multiscale GCNs
1. Computing methodologies
  1. Computer graphics
    1. Shape modeling
      1. Shape analysis

Recommendations

A Wavelet Energy Decomposition Signature for Robust Non-Rigid Shape Matching
SA '19: SIGGRAPH Asia 2019 Posters

We present a novel local shape descriptor, named wavelet energy decomposition signature (WEDS) for robustly matching non-rigid 3D shapes with different resolutions. The local shape descriptors are generated by decomposing Dirichlet energy on the input ...
The multi angular descriptor (MAD): a binary and gray images descriptor for shape recognition
HIP '13: Proceedings of the 2nd International Workshop on Historical Document Imaging and Processing

In this paper, we present the Multi Angular Descriptor (MAD), a new shape descriptor for shape based object recognition and image retrieval. In the binary case, the MAD descriptor captures the angular view to multi resolution rings from each contour ...
Invariant curvature-based Fourier shape descriptors

Shape descriptors have demonstrated encouraging potential for retrieving images based on image content, and a number of them have been reported in the literature. Nevertheless, most of the reported descriptors are still face accuracy and computational ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Transactions on Graphics

ACM Transactions on Graphics Volume 39, Issue 4

August 2020

1732 pages

ISSN:0730-0301

EISSN:1557-7368

DOI:10.1145/3386569

Editor:
Szymon Rusinkiewicz
Princeton University

Issue’s Table of Contents

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 August 2020

Published in TOG Volume 39, Issue 4

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Beijing Natural Science Foundation
Shenzhen Basic Research Program
the National Natural Science Foundation of China
Alibaba Group through Alibaba Innovative Research Program
the CCF-Tencent Open Research Fund
National Key R&D Program of China
KAUST OSR Award

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

21
Total Citations
View Citations
477
Total Downloads

Downloads (Last 12 months)45
Downloads (Last 6 weeks)2

Reflects downloads up to 19 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Liu SWang HYan DLi QLuo FTeng ZLiu X(2025)Spectral Descriptors for 3D Deformable Shape Matching: A Comparative SurveyIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2024.336808331:3(1677-1697)Online publication date: Mar-2025
https://doi.org/10.1109/TVCG.2024.3368083
Nagar R(2025)Robust extrinsic symmetry estimation in 3D point cloudsThe Visual Computer: International Journal of Computer Graphics10.1007/s00371-024-03313-641:1(115-128)Online publication date: 1-Jan-2025
https://dl.acm.org/doi/10.1007/s00371-024-03313-6
Hu LWang HYang XXu HLiao Y(2024)AFSMWD: A Descriptor Flexibly Encoding Multiscale and Oriented Shape FeaturesMathematics10.3390/math1218294612:18(2946)Online publication date: 22-Sep-2024
https://doi.org/10.3390/math12182946
Lian YPei SChen MHua J(2024)Relation Constrained Capsule Graph Neural Networks for Non-rigid Shape CorrespondenceACM Transactions on Intelligent Systems and Technology10.1145/3688851Online publication date: 16-Aug-2024
https://dl.acm.org/doi/10.1145/3688851
Dong XZhao CSong XZhang LLiu YWu JXu YXu NLiu JYu HYang KZhou X(2024)PresRecST: a novel herbal prescription recommendation algorithm for real-world patients with integration of syndrome differentiation and treatment planningJournal of the American Medical Informatics Association10.1093/jamia/ocae06631:6(1268-1279)Online publication date: 10-Apr-2024
https://doi.org/10.1093/jamia/ocae066
Lazzaro DMorigi SZuzolo P(2024)Learning intrinsic shape representations via spectral mesh convolutionsNeurocomputing10.1016/j.neucom.2024.128152598:COnline publication date: 14-Sep-2024
https://dl.acm.org/doi/10.1016/j.neucom.2024.128152
Gao JWang PDong QZeng QXin SZhang C(2024)D3FormerComputer Aided Geometric Design10.1016/j.cagd.2024.102300111:COnline publication date: 18-Jul-2024
https://dl.acm.org/doi/10.1016/j.cagd.2024.102300
Liu SXu HYan DHu LLiu XLi Q(2023)WTFM Layer: An Effective Map Extractor for Unsupervised Shape CorrespondenceComputer Graphics Forum10.1111/cgf.1465641:7(51-61)Online publication date: 20-Mar-2023
https://doi.org/10.1111/cgf.14656
Farazi MZhu WYang ZWang Y(2023)Anisotropic Multi-Scale Graph Convolutional Network for Dense Shape Correspondence2023 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)10.1109/WACV56688.2023.00316(3145-3154)Online publication date: Jan-2023
https://doi.org/10.1109/WACV56688.2023.00316
Attaiki SLi LOvsjanikov M(2023)Generalizable Local Feature Pre-training for Deformable Shape Analysis2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52729.2023.01312(13650-13661)Online publication date: Jun-2023
https://doi.org/10.1109/CVPR52729.2023.01312
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Issue’s Table of Contents