Abstract
This paper extends the investigation and application of Deformable Voxel Grids (DVGs) into a unified framework for 3D shape analysis and synthesis. DVGs consist in a grid that approximates a shape’s silhouette via energy-minimization. This provides an improved embedding space over a regular voxel grid as it aligns with the geometry of the shape, and subsequently allows for the deformation of the shape by manipulating the DVG’s control points. We demonstrate how DVGs directly and naturally serve for an array of applications: correspondences, style transfer, shape retrieval, and PCA deformations. We further address the challenge of morphing non-parametric shapes, an ill-posed problem because of the trade-off between plausibility and smoothness, particularly under large topology changes. Thanks to DVGs, we extract a shape content descriptor, and propose a similarity metric adapted to the extracted content and a formulation of morphings as minimal paths in a graph. Our approach leverages the strengths and interpretability of DVGs while achieving morphing capabilities comparable to those provided by neural networks. Throughout the course of our study, we conducted qualitative and quantitative analyses on the robustness and quality of our proposed methods, and we provide valuable insights into the effective manual intervention that can enhance quality, given the interpretability of each component in our method. In conclusion, this work elucidates the wide-ranging implications and potential of DVGs in 3D shape comparison, processing, and morphing, paving the way for future research and applications in the field.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
Note however that our application is different from the typical style transfer found in the literature.
- 2.
As for the consistent alignment of shapes, this could be performed as a pre-processing step.
- 3.
Say we optimized V up to its fourth resolution level, \(r = 3\), into an \(8 \times 8 \times 8\) grid. All subsequent subdivisions will not be optimized, but can still be used to refine the resolution of the voxelization.
References
Achlioptas, P., Diamanti, O., Mitliagkas, I., Guibas, L.J.: Learning representations and generative models for 3D point clouds (2017)
Allen, B., Curless, B., Curless, B., Popović, Z.: The space of human body shapes: reconstruction and parameterization from range scans. ACM Trans. Graph. 22(3), 587–594 (2003)
Barreira, N., Penedo, M.G., Mariño, C., Ansia, F.M.: Topological active volumes. In: Petkov, N., Westenberg, M.A. (eds.) CAIP 2003. LNCS, vol. 2756, pp. 337–344. Springer, Heidelberg (2003). https://doi.org/10.1007/978-3-540-45179-2_42
Besl, P.J., McKay, N.D.: A method for registration of 3-D shapes. IEEE Trans. Pattern Anal. Mach. Intell. 14(2), 239–256 (1992)
Chang, A.X., et al.: ShapeNet: an information-rich 3D model repository. Technical report. arXiv:1512.03012 [cs.GR], Stanford University — Princeton University—Toyota Technological Institute at Chicago (2015)
Cohen, L.D.: On active contour models and balloons. CVGIP Image Underst. 53(2), 211–218 (1991)
Dubrovina, A., Xia, F., Achlioptas, P., Shalah, M., Groscot, R., Guibas, L.J.: Composite shape modeling via latent space factorization. In: The IEEE International Conference on Computer Vision (ICCV), October 2019
Fan, H., Su, H., Guibas, L.J.: A point set generation network for 3D object reconstruction from a single image. CoRR abs/1612.00603 (2016)
Fish, N., Averkiou, M., van Kaick, O., Sorkine-Hornung, O., Cohen-Or, D., Mitra, N.J.: Meta-representation of shape families. ACM Trans. Graph. 33(4), 34:1–34:11 (2014)
Gao, L., Lai, Y.K., Huang, Q., Hu, S.: A data-driven approach to realistic shape morphing. Comput. Graph. Forum 32 (2013). https://doi.org/10.1111/cgf.12065
Gkioxari, G., Malik, J., Johnson, J.: Mesh R-CNN (2019)
Goodfellow, I.J., et al.: Generative adversarial networks (2014)
Groscot, R., Cohen, L., Guibas, L.: Shape part transfer via semantic latent space factorization. In: Nielsen, F., Barbaresco, F. (eds.) GSI 2019. LNCS, vol. 11712, pp. 511–519. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-26980-7_53
Groscot, R., Cohen, L.D.: Deformable voxel grids for shape comparisons. In: Jiang, X., Tao, W., Zeng, D., Xie, Y. (eds.) Fourteenth International Conference on Digital Image Processing (ICDIP 2022), vol. 12342, p. 123423G. International Society for Optics and Photonics, SPIE (2022). https://doi.org/10.1117/12.2645961
Groscot, R., Cohen, L.D.: Shape morphing as a minimal path in the graph of cubified shapes. In: de Sousa, A.A., Bashford-Rogers, T., Bouatouch, K. (eds.) Proceedings of the 18th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications, VISIGRAPP 2023, Volume 1: GRAPP, Lisbon, Portugal, 19–21 February 2023, pp. 98–109. Scitepress (2023). https://doi.org/10.5220/0011680200003417
Groscot, R.: Separable 3D shape representations for shape processing. Ph.D. thesis (2021). http://www.theses.fr/2021UPSLD018, thèse de doctorat dirigée par Cohen, Laurent David Mathématiques appliquées Université Paris sciences et lettres 2021
Groueix, T., Fisher, M., Kim, V.G., Russell, B., Aubry, M.: AtlasNet: a Papier-Mâché approach to learning 3D surface generation. In: CVPR 2018, Salt Lake City, United States, June 2018
Haibin, H., Kalogerakis, E., Marlin, B.: Analysis and synthesis of 3D shape families via deep-learned generative models of surfaces. Comput. Graph. Forum 34 (2015)
Hanocka, R., Fish, N., Wang, Z., Giryes, R., Fleishman, S., Cohen-Or, D.: ALIGNet: partial-shape agnostic alignment via unsupervised learning (2018)
Hao, Z., Averbuch-Elor, H., Snavely, N., Belongie, S.: DualSDF: semantic shape manipulation using a two-level representation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2020
Hart, J.: Sphere tracing: a geometric method for the antialiased ray tracing of implicit surfaces. Vis. Comput. 12 (1995)
Hemalatha, R., Thamizhvani, T., Dhivya, A.J.A., Joseph, J.E., Babu, B., Chandrasekaran, R.: Active contour based segmentation techniques for medical image analysis. In: Koprowski, R. (ed.) Medical and Biological Image Analysis, chap. 2. IntechOpen, Rijeka (2018)
Kalogerakis, E., Chaudhuri, S., Koller, D., Koltun, V.: A probabilistic model for component-based shape synthesis. ACM Trans. Graph. 31(4), 55:1–55:11 (2012)
Kass, M., Witkin, A., Terzopoulos, D.: Snakes: active contour models. Int. J. Comput. Vis. 1(4), 321–331 (1988). https://doi.org/10.1007/BF00133570
Kingma, D.P., Welling, M.: Auto-Encoding Variational Bayes. arXiv e-prints arXiv:1312.6114, December 2013
Kleineberg, M., Fey, M., Weichert, F.: Adversarial generation of continuous implicit shape representations (2020)
Kurenkov, A., et al.: DeformNet: free-form deformation network for 3D shape reconstruction from a single image (2017)
Li, J., Xu, K., Chaudhuri, S., Yumer, E., Zhang, H., Guibas, L.J.: GRASS: generative recursive autoencoders for shape structures. ACM Trans. Graph. (Proc. SIGGRAPH 2017) (2017)
Lorensen, W.E., Cline, H.E.: Marching cubes: a high resolution 3d surface construction algorithm. In: SIGGRAPH 1987, New York, NY, USA (1987)
van der Maaten, L., Hinton, G.E.: Visualizing high-dimensional data using t-SNE. J. Mach. Learn. Res. 9, 2579–2605 (2008)
Niemeyer, M., Mescheder, L., Oechsle, M., Geiger, A.: Occupancy flow: 4D reconstruction by learning particle dynamics. In: International Conference on Computer Vision, October 2019
Ovsjanikov, M., Ben-Chen, M., Solomon, J., Butscher, A., Guibas, L.: Functional maps: a flexible representation of maps between shapes. ACM Trans. Graph. 31(4) (2012)
Park, E., Yang, J., Yumer, E., Ceylan, D., Berg, A.C.: Transformation-grounded image generation network for novel 3D view synthesis. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 2017
Park, J.J., Florence, P., Straub, J., Newcombe, R., Lovegrove, S.: DeepSDF: learning continuous signed distance functions for shape representation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2019
Qi, C.R., Su, H., Mo, K., Guibas, L.J.: PointNet: deep learning on point sets for 3D classification and segmentation (2016)
Qi, C.R., Yi, L., Su, H., Guibas, L.J.: PointNet++: deep hierarchical feature learning on point sets in a metric space. CoRR abs/1706.02413 (2017)
Sederberg, T.W., Parry, S.R.: Free-form deformation of solid geometric models. SIGGRAPH Comput. Graph. 20(4), 151–160 (1986). https://doi.org/10.1145/15886.15903
Shin, D., Fowlkes, C.C., Hoiem, D.: Pixels, voxels, and views: a study of shape representations for single view 3D object shape prediction. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3061–3069 (2018). https://doi.org/10.1109/CVPR.2018.00323
Sun, J., Ovsjanikov, M., Guibas, L.: A concise and provably informative multi-scale signature based on heat diffusion. Comput. Graph. Forum 28, 1383–1392 (2009)
Tatarchenko, M., Dosovitskiy, A., Brox, T.: Octree generating networks: efficient convolutional architectures for high-resolution 3D outputs. CoRR abs/1703.09438 (2017)
Tulsiani, S., Su, H., Guibas, L.J., Efros, A.A., Malik, J.: Learning shape abstractions by assembling volumetric primitives. In: Computer Vision and Pattern Regognition (CVPR) (2017)
Wang, N., Zhang, Y., Li, Z., Fu, Y., Liu, W., Jiang, Y.-G.: Pixel2Mesh: generating 3D mesh models from single RGB images. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11215, pp. 55–71. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01252-6_4
Wu, J., Zhang, C., Xue, T., Freeman, W.T., Tenenbaum, J.B.: Learning a probabilistic latent space of object shapes via 3D generative-adversarial modeling. In: Proceedings of the 30th International Conference on Neural Information Processing Systems, NIPS 2016, pp. 82–90. Curran Associates Inc., USA (2016)
Zheng, Z., Yu, T., Dai, Q., Liu, Y.: Deep implicit templates for 3D shape representation (2020)
Acknowledgements
This work was funded in part by the French government under management of Agence Nationale de la Recherche as part of the “Investissements d’avenir” program, reference ANR-19-P3IA-0001 (PRAIRIE 3IA Institute).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Groscot, R., Cohen, L.D. (2024). Unified Shape Analysis and Synthesis via Deformable Voxel Grids. In: de Sousa, A.A., et al. Computer Vision, Imaging and Computer Graphics Theory and Applications. VISIGRAPP 2023. Communications in Computer and Information Science, vol 2103. Springer, Cham. https://doi.org/10.1007/978-3-031-66743-5_2
Download citation
DOI: https://doi.org/10.1007/978-3-031-66743-5_2
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-66742-8
Online ISBN: 978-3-031-66743-5
eBook Packages: Computer ScienceComputer Science (R0)