Nothing Special   »   [go: up one dir, main page]

skip to main content
research-article

ALIGNet: Partial-Shape Agnostic Alignment via Unsupervised Learning

Published: 14 December 2018 Publication History

Abstract

The process of aligning a pair of shapes is a fundamental operation in computer graphics. Traditional approaches rely heavily on matching corresponding points or features to guide the alignment, a paradigm that falters when significant shape portions are missing. These techniques generally do not incorporate prior knowledge about expected shape characteristics, which can help compensate for any misleading cues left by inaccuracies exhibited in the input shapes. We present an approach based on a deep neural network, leveraging shape datasets to learn a shape-aware prior for source-to-target alignment that is robust to shape incompleteness. In the absence of ground truth alignments for supervision, we train a network on the task of shape alignment using incomplete shapes generated from full shapes for self-supervision. Our network, called ALIGNet, is trained to warp complete source shapes to incomplete targets, as if the target shapes were complete, thus essentially rendering the alignment partial-shape agnostic. We aim for the network to develop specialized expertise over the common characteristics of the shapes in each dataset, thereby achieving a higher-level understanding of the expected shape space to which a local approach would be oblivious. We constrain ALIGNet through an anisotropic total variation identity regularization to promote piecewise smooth deformation fields, facilitating both partial-shape agnosticism and post-deformation applications. We demonstrate that ALIGNet learns to align geometrically distinct shapes and is able to infer plausible mappings even when the target shape is significantly incomplete. We show that our network learns the common expected characteristics of shape collections without over-fitting or memorization, enabling it to produce plausible deformations on unseen data during test time.

References

[1]
Dror Aiger, Niloy J. Mitra, and Daniel Cohen-Or. 2008. 4-points congruent sets for robust pairwise surface registration. In ACM SIGGRAPH 2008 Paper (SIGGRAPH’08). ACM, 85:1--85:10.
[2]
S. Belongie, J. Malik, and J. Puzicha. 2002. Shape matching and object recognition using shape contexts. IEEE Trans. Pattern Anal. Mach. Intell. 24, 4 (Apr. 2002), 509--522.
[3]
P. J. Besl and N. D. McKay. 1992. A method for registration of 3-D shapes. IEEE Trans. Pattern Anal. Mach. Intell. 14, 2 (Feb. 1992), 239--256.
[4]
Fred L. Bookstein. 1989. Principal warps: Thin-plate splines and the decomposition of deformations. IEEE Trans. Pattern Anal. Mach. Intell. 11, 6 (1989), 567--585.
[5]
Angel X. Chang, Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan, Qixing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, Jianxiong Xiao, Li Yi, and Fisher Yu. 2015. ShapeNet: An Information-Rich 3D Model Repository. Technical Report. Stanford University/Princeton University /Toyota Technological Institute at Chicago.
[6]
Christopher B. Choy, JunYoung Gwak, Silvio Savarese, and Manmohan Chandraker. 2016. Universal correspondence network. In Advances in Neural Information Processing Systems. 2414--2422.
[7]
Philipp Fischer, Alexey Dosovitskiy, Eddy Ilg, Philip Häusser, Caner Hazirbas, Vladimir Golkov, Patrick van der Smagt, Daniel Cremers, and Thomas Brox. 2015. FlowNet: Learning optical flow with convolutional networks. In Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV’15). IEEE Computer Society, 2758--2766.
[8]
Martin A. Fischler and Robert C. Bolles. 1981. Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24, 6 (1981), 381--395.
[9]
Ravi Garg, Vijay Kumar BG, Gustavo Carneiro, and Ian Reid. 2016. Unsupervised cnn for single view depth estimation: Geometry to the rescue. In European Conference on Computer Vision. Springer, 740--756.
[10]
Max Jaderberg, Karen Simonyan, Andrew Zisserman, and Koray Kavukcuoglu. 2015. Spatial transformer networks. In Proceedings of the 28th International Conference on Neural Information Processing Systems (NIPS’15). MIT Press, Cambridge, MA, 2017--2025.
[11]
B. Jian and B. C. Vemuri. 2011. Robust point set registration using gaussian mixture models. IEEE Trans. Pattern Anal. Mach. Intell. 33, 8 (Aug. 2011), 1633--1645.
[12]
Justin Johnson, Alexandre Alahi, and Li Fei-Fei. 2016. Perceptual losses for real-time style transfer and super-resolution. In European Conference on Computer Vision. Springer, 694--711.
[13]
Angjoo Kanazawa, David W. Jacobs, and Manmohan Chandraker. 2016. Warpnet: Weakly supervised matching for single-view reconstruction. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 3253--3261.
[14]
Longin Jan Latecki and Rolf Lakamper. 2000. Shape similarity measure based on correspondence of visual parts. IEEE Trans. Pattern Anal. Mach. Intell. 22, 10 (2000), 1185--1190.
[15]
Jun Li, Yongjun Chen, Lei Cai, Ian Davidson, and Shuiwang Ji. 2017. Dense transformer networks. arXiv Preprint arXiv:1705.08881 (2017).
[16]
Haibin Ling and David W. Jacobs. 2007. Shape classification using the inner-distance. IEEE Trans. Pattern Anal. Mach. Intell. 29, 2 (2007), 286--299.
[17]
Or Litany, Emanuele Rodolà, Alex M. Bronstein, and Michael M. Bronstein. 2017. Fully spectral partial shape matching. In Computer Graphics Forum, Vol. 36. Wiley Online Library, 247--258.
[18]
Jiayi Ma, Ji Zhao, Jinwen Tian, Alan L. Yuille, and Zhuowen Tu. 2014. Robust point matching via vector field consensus. IEEE Trans. Image Process. 23, 4 (2014), 1706--1721.
[19]
Jiayi Ma, Ji Zhao, and Alan L. Yuille. 2016. Non-rigid point set registration by preserving global and local structures. IEEE Trans. Image Process. 25, 1 (2016), 53--64.
[20]
G. Mori and J. Malik. 2003. Recognizing objects in adversarial clutter: Breaking a visual CAPTCHA. In Proceedings of the 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Vol. 1. I--134--I--141 vol.1.
[21]
A. Myronenko and X. Song. 2010. Point set registration: Coherent point drift. IEEE Trans. Pattern Anal. Mach. Intell. 32, 12 (Dec. 2010), 2262--2275.
[22]
S. Rusinkiewicz and M. Levoy. 2001. Efficient variants of the ICP algorithm. In Proceedings of the 3rd International Conference on 3-D Digital Imaging and Modeling. 145--152.
[23]
Thomas W. Sederberg and Scott R. Parry. 1986. Free-form deformation of solid geometric models. ACM SIGGRAPH Comput. Graph. 20, 4 (Aug. 1986), 151--160.
[24]
Amit Shaked and Lior Wolf. 2016. Improved stereo matching with constant highway networks and reflective confidence learning. arXiv Preprint arXiv:1701.00165 (2016).
[25]
Edgar Simo-Serra, Eduard Trulls, Luis Ferraz, Iasonas Kokkinos, Pascal Fua, and Francesc Moreno-Noguer. 2015. Discriminative learning of deep convolutional feature point descriptors. In Proceedings of the IEEE International Conference on Computer Vision. 118--126.
[26]
Ayush Tewari, Michael Zollhöfer, Hyeongwoo Kim, Pablo Garrido, Florian Bernard, Patrick Perez, and Christian Theobalt. 2017. Mofa: Model-based deep convolutional face autoencoder for unsupervised monocular reconstruction. In Proceedings of the IEEE International Conference on Computer Vision (ICCV’17), Vol. 2.
[27]
A. Thayananthan, B. Stenger, P. H. S. Torr, and R. Cipolla. 2003. Shape context and chamfer matching in cluttered scenes. In Proceedings of the 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’03). IEEE Computer Society, Los Alamitos, CA, 127--133.
[28]
Bin Fan Yurun Tian and Fuchao Wu. 2017. L2-net: Deep learning of discriminative patch descriptor in euclidean space. In Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR’17).
[29]
Yanghai Tsin and Takeo Kanade. 2004. A Correlation-Based Approach to Robust Point Set Registration. Springer, 558--569.
[30]
Yunhai Wang, Shmulik Asafi, Oliver van Kaick, Hao Zhang, Daniel Cohen-Or, and Baoquan Chen. 2012. Active co-analysis of a set of shapes. ACM Trans. Graph. 31, 6 (2012), 165.
[31]
M. Ersin Yumer and Niloy J. Mitra. 2016. Learning semantic deformation flows with 3d convolutional networks. In European Conference on Computer Vision. Springer, 294--311.
[32]
Jure Zbontar and Yann LeCun. 2016. Stereo matching by training a convolutional neural network to compare image patches. J. Mach. Learn. Res. 17, 1--32 (2016), 2.
[33]
Andy Zeng, Shuran Song, Matthias Nießner, Matthew Fisher, Jianxiong Xiao, and Thomas Funkhouser. 2017. 3DMatch: Learning local geometric descriptors from RGB-D reconstructions. In CVPR.
[34]
Yefeng Zheng and D. Doermann. 2006. Robust point matching for nonrigid shapes by preserving local neighborhood structures. IEEE Trans. Pattern Anal. Mach. Intell. 28, 4 (Apr. 2006), 643--649.

Cited By

View all
  • (2024)Enhancing the Aesthetics of 3D Shapes via Reference-based EditingACM Transactions on Graphics10.1145/368795443:6(1-15)Online publication date: 19-Nov-2024
  • (2024)MeshPointNet: 3D Surface Classification Using Graph Neural Networks and Conformal Predictions on Mesh-Based RepresentationsJournal of Mechanical Design10.1115/1.4064673146:5Online publication date: 18-Mar-2024
  • (2024)Laplacian2Mesh: Laplacian-Based Mesh UnderstandingIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.325904430:7(4349-4361)Online publication date: Jul-2024
  • Show More Cited By

Recommendations

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Transactions on Graphics
ACM Transactions on Graphics  Volume 38, Issue 1
February 2019
176 pages
ISSN:0730-0301
EISSN:1557-7368
DOI:10.1145/3300145
Issue’s Table of Contents
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 14 December 2018
Accepted: 01 July 2018
Revised: 01 July 2018
Received: 01 September 2017
Published in TOG Volume 38, Issue 1

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Deep learning
  2. self-supervised learning
  3. shape deformation

Qualifiers

  • Research-article
  • Research
  • Refereed

Funding Sources

  • Israel Science Foundation as part of the ISF-NSFC joint program
  • ISF

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)24
  • Downloads (Last 6 weeks)3
Reflects downloads up to 15 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Enhancing the Aesthetics of 3D Shapes via Reference-based EditingACM Transactions on Graphics10.1145/368795443:6(1-15)Online publication date: 19-Nov-2024
  • (2024)MeshPointNet: 3D Surface Classification Using Graph Neural Networks and Conformal Predictions on Mesh-Based RepresentationsJournal of Mechanical Design10.1115/1.4064673146:5Online publication date: 18-Mar-2024
  • (2024)Laplacian2Mesh: Laplacian-Based Mesh UnderstandingIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.325904430:7(4349-4361)Online publication date: Jul-2024
  • (2024)3-D Model Extraction Network Based on RFM-Constrained Deformation Inference and Self-Similar Convolution for Satellite Stereo ImagesIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing10.1109/JSTARS.2024.341989617(11877-11885)Online publication date: 2024
  • (2024)MDSSN: An end-to-end deep network on triangle mesh parameterizationKnowledge-Based Systems10.1016/j.knosys.2023.111177284(111177)Online publication date: Jan-2024
  • (2024)MeshCL: Towards robust 3D mesh analysis via contrastive learningAdvanced Engineering Informatics10.1016/j.aei.2024.10243960(102439)Online publication date: Apr-2024
  • (2024)Shape embedding and retrieval in multi-flow deformationComputational Visual Media10.1007/s41095-022-0315-310:3(439-451)Online publication date: 8-Feb-2024
  • (2024)FusionDeformer: text-guided mesh deformation using diffusion modelsThe Visual Computer: International Journal of Computer Graphics10.1007/s00371-024-03463-740:7(4701-4712)Online publication date: 1-Jul-2024
  • (2024)Unified Shape Analysis and Synthesis via Deformable Voxel GridsComputer Vision, Imaging and Computer Graphics Theory and Applications10.1007/978-3-031-66743-5_2(24-47)Online publication date: 22-Aug-2024
  • (2023)OptCtrlPoints: Finding the Optimal Control Points for Biharmonic 3D Shape DeformationComputer Graphics Forum10.1111/cgf.1496342:7Online publication date: 5-Nov-2023
  • Show More Cited By

View Options

Login options

Full Access

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media