MeshFeat: Multi-resolution Features for Neural Fields on Meshes
Pages 268 - 285
Abstract
Parametric feature grid encodings have gained significant attention as an encoding approach for neural fields since they allow for much smaller MLPs, which significantly decreases the inference time of the models. In this work, we propose MeshFeat, a parametric feature encoding tailored to meshes, for which we adapt the idea of multi-resolution feature grids from Euclidean space. We start from the structure provided by the given vertex topology and use a mesh simplification algorithm to construct a multi-resolution feature representation directly on the mesh. The approach allows the usage of small MLPs for neural fields on meshes, and we show a significant speed-up compared to previous representations while maintaining comparable reconstruction quality for texture reconstruction and BRDF representation. Given its intrinsic coupling to the vertices, the method is particularly well-suited for representations on deforming meshes, making it a good fit for object animation.
References
[1]
Baatz, H., Granskog, J., Papas, M., Rousselle, F., Novák, J.: Nerf-tex: neural reflectance field textures. In: Computer Graphics Forum (2022)
[2]
Basri, R., Galun, M., Geifman, A., Jacobs, D.W., Kasten, Y., Kritchman, S.: Frequency bias in neural networks for input of non-uniform density. In: ICML (2020)
[3]
Burley, B., Studios, W.D.A.: Physically-based shading at disney. In: ACM SIGGRAPH (2012)
[4]
Chabra R et al. Vedaldi A, Bischof H, Brox T, Frahm J-M, et al. Deep local shapes: learning local SDF priors for detailed 3D reconstruction Computer Vision – ECCV 2020 2020 Cham Springer 608-625
[5]
Chen, Z., Zhang, H.: Learning implicit fields for generative shape modeling. In: CVPR (2019).
[6]
Fridovich-Keil, S., Yu, A., Tancik, M., Chen, Q., Recht, B., Kanazawa, A.: Plenoxels: radiance fields without neural networks. In: CVPR (2022)
[7]
Garland, M., Heckbert, P.S.: Surface simplification using quadric error metrics. In: SIGGRAPH (1997).
[8]
Garland, M., Heckbert, P.S.: Simplifying surfaces with color and texture using quadric error metrics. In: IEEE Visualization Conference (1998).
[9]
Grattarola, D., Vandergheynst, P.: Generalised implicit neural representations. In: NeurIPS (2022)
[10]
Gropp, A., Yariv, L., Haim, N., Atzmon, M., Lipman, Y.: Implicit geometric regularization for learning shapes. In: ICML (2020)
[11]
Hanocka R, Hertz A, Fish N, Giryes R, Fleishman S, and Cohen-Or D Meshcnn: a network with an edge ACM TOG 2019
[12]
Hertz, A., Perel, O., Giryes, R., Sorkine-Hornung, O., Cohen-Or, D.: SAPE: spatially-adaptive progressive encoding for neural optimization. In: NeurIPS (2021)
[13]
Hertz A, Perel O, Giryes R, Sorkine-Hornung O, and Cohen-Or D Mesh draping: parametrization-free neural mesh transfer Comput. Graph. Forum 2023
[14]
Hwang, I., Kim, H., Kim, Y.M.: Text2scene: text-driven indoor scene stylization with part-aware details. In: CVPR (2023).
[15]
Jiang, C.M., Sud, A., Makadia, A., Huang, J., Nießner, M., Funkhouser, T.: Local implicit grid representations for 3D scenes. In: CVPR (2020)
[16]
Jiang Z, Schneider T, Zorin D, and Panozzo D Bijective projection in a shell ACM TOG 2020
[17]
Kim, H., Jang, Y., Lee, J., Ahn, S.: Hybrid neural representations for spherical data. CoRR (2024).
[18]
Koestler L, Grittner D, Möller M, Cremers D, and Lähner Z Avidan S, Brostow G, Cissé M, Farinella GM, and Hassner T Intrinsic neural fields: learning functions on manifolds ECCV 2022 2022 Cham Springer 622-639
[19]
Lee, A.W.F., Sweldens, W., Schröder, P., Cowsar, L.C., Dobkin, D.P.: MAPS: multiresolution adaptive parameterization of surfaces. In: SIGGRAPH (1998).
[20]
Li, M., Zhou, Z., Wu, Z., Shi, B., Diao, C., Tan, P.: Multi-view photometric stereo: a robust solution and benchmark dataset for spatially varying isotropic materials. IEEE TIP (2020).
[21]
Li, Z., et al.: Neuralangelo: high-fidelity neural surface reconstruction. In: CVPR (2023)
[22]
Lin, S., Zhou, B., Zheng, Z., Zhang, H., Liu, Y.: Leveraging intrinsic properties for non-rigid garment alignment. In: ICCV (2023).
[23]
Liu HD, Zhang JE, Ben-Chen M, and Jacobson A Surface multigrid via intrinsic prolongation ACM TOG 2021
[24]
Liu, L., Gu, J., Lin, K.Z., Chua, T.S., Theobalt, C.: Neural sparse voxel fields. In: NeurIPS (2020)
[25]
Mehta, I., Gharbi, M., Barnes, C., Shechtman, E., Ramamoorthi, R., Chandraker, M.: Modulated periodic activations for generalizable local functional representations. In: ICCV (2021)
[26]
Mescheder, L., Oechsle, M., Niemeyer, M., Nowozin, S., Geiger, A.: Occupancy networks: learning 3D reconstruction in function space. In: CVPR (2019)
[27]
Michel, O., Bar-On, R., Liu, R., Benaim, S., Hanocka, R.: Text2mesh: text-driven neural stylization for meshes. In: CVPR (2022).
[28]
Mildenhall, B., Hedman, P., Martin-Brualla, R., Srinivasan, P.P., Barron, J.T.: Nerf in the dark: high dynamic range view synthesis from noisy raw images. In: CVPR (2022).
[29]
Mildenhall, B., Srinivasan, P.P., Tancik, M., Barron, J.T., Ramamoorthi, R., Ng, R.: Nerf: representing scenes as neural radiance fields for view synthesis. In: ECCV (2020)
[30]
Müller T, Evans A, Schied C, and Keller A Instant neural graphics primitives with a multiresolution hash encoding ACM TOG 2022
[31]
Oechsle, M., Mescheder, L.M., Niemeyer, M., Strauss, T., Geiger, A.: Texture fields: learning texture representations in function space. In: ICCV (2019).
[32]
Park, J.J., Florence, P., Straub, J., Newcombe, R., Lovegrove, S.: Deepsdf: learning continuous signed distance functions for shape representation. In: CVPR (2019)
[33]
Peng S, Niemeyer M, Mescheder L, Pollefeys M, and Geiger A Vedaldi A, Bischof H, Brox T, and Frahm J-M Convolutional occupancy networks Computer Vision – ECCV 2020 2020 Cham Springer 523-540
[34]
Rahaman, N., et al.: On the spectral bias of neural networks. In: ICML (2019)
[35]
Rahimi, A., Recht, B.: Random features for large-scale kernel machines. In: NIPS (2007)
[36]
Sharp, N., Crane, K.: A laplacian for nonmanifold triangle meshes. Comput. Graph. Forum (2020)
[37]
Sitzmann, V., Martel, J.N.P., Bergman, A.W., Lindell, D.B., Wetzstein, G.: Implicit neural representations with periodic activation functions. In: NeurIPS (2020)
[38]
Sumner RW and Popović J Deformation transfer for triangle meshes ACM TOG 2004 23 3 399-405
[39]
Sun, C., Sun, M., Chen, H.: Direct voxel grid optimization: super-fast convergence for radiance fields reconstruction. In: CVPR (2022)
[40]
Takikawa, T., et al.: Neural geometric level of detail: real-time rendering with implicit 3D shapes. In: CVPR (2021)
[41]
Tancik, M., et al.: Fourier features let networks learn high frequency functions in low dimensional domains. In: NeurIPS (2020)
[42]
Tang, D., et al.: Real-time compression and streaming of 4D performances. In: ACM TOG (2018)
[43]
Vaswani, A., et al.: Attention is all you need. In: NIPS (2017)
[44]
Walker, T., Mariotti, O., Vaxman, A., Bilen, H.: Explicit neural surfaces: learning continuous geometry with deformation fields. CoRR (2023).
[45]
Wang, P., Liu, L., Liu, Y., Theobalt, C., Komura, T., Wang, W.: Neus: learning neural implicit surfaces by volume rendering for multi-view reconstruction. In: NeurIPS (2021)
[46]
Wang Z, Bovik AC, Sheikh HR, and Simoncelli EP Image quality assessment: from error visibility to structural similarity IEEE TIP 2004
[47]
Xiang, F., Xu, Z., Hasan, M., Hold-Geoffroy, Y., Sunkavalli, K., Su, H.: Neutex: neural texture mapping for volumetric neural rendering. In: CVPR (2021).
[48]
Yang B et al. Avidan S, Brostow G, Cissé M, Farinella GM, Hassner T, et al. Neumesh: learning disentangled neural mesh-based implicit field for geometry and texture editing ECCV 2022 2022 Cham Springer 597-614
[49]
Yariv, L., Gu, J., Kasten, Y., Lipman, Y.: Volume rendering of neural implicit surfaces. In: NeurIPS (2021)
[50]
Zhang, R., Isola, P., Efros, A.A., Shechtman, E., Wang, O.: The unreasonable effectiveness of deep features as a perceptual metric. In: CVPR (2018).
Index Terms
- MeshFeat: Multi-resolution Features for Neural Fields on Meshes
Index terms have been assigned to the content through auto-classification.
Recommendations
Surface interpolation of meshes by geometric subdivision
Subdivision surfaces are generated by repeated approximation or interpolation from initial control meshes. In this paper, two new non-linear subdivision schemes, face based subdivision scheme and normal based subdivision scheme, are introduced for ...
Normal meshes
SIGGRAPH '00: Proceedings of the 27th annual conference on Computer graphics and interactive techniquesNormal meshes are new fundamental surface descriptions inspired by differential geometry. A normal mesh is a multiresolution mesh where each level can be written as a normal offset from a coarser version. Hence the mesh can be stored with a single float ...
Surface Interpolation of Meshes with Shape Optimization
GMP '04: Proceedings of the Geometric Modeling and Processing 2004Subdivision surface is an efficient geometric modelingtool that is generated by repeated approximation or interpolationfrom an initial control mesh. For each time of subdivisionthe old mesh will be split and the vertexes of the meshwill be refined for a ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
Sep 2024
570 pages
ISBN:978-3-031-73396-3
DOI:10.1007/978-3-031-73397-0
© The Author(s), under exclusive license to Springer Nature Switzerland AG 2025.
Publisher
Springer-Verlag
Berlin, Heidelberg
Publication History
Published: 03 November 2024
Author Tags
Qualifiers
- Article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 25 Nov 2024
Other Metrics
Citations
View Options
View options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in