Multi-view Normal Estimation – Application to Slanted Plane-Sweeping
Authors: 
Lilian Calvet, Nicolas Maignan, Baptiste Brument, Jean Mélou, Silvia Tozza, Jean-Denis Durou, Yvain QuéauAuthors Info & Claims
Pages 704 - 716
Abstract
In this paper, we show how to estimate the normals of a 3D surface from a minimum of two views, assuming that the poses of a calibrated camera are perfectly known. For each pair of image points, the normal at the corresponding 3D point is expressed in function of the local gradients of the grey level, whatever the type of image formation (orthogonal or perspective projection). As an application, this allows us to fully estimate the inter-image homography, which not only depends on the relative pose between views, but also on the local orientation of the surface. Hence, the photo-consistency between patches from two images, which is the basis of the so-called “plane-sweeping” method, is improved. Experiments on synthetic and real data validate our approach.
References
[1]
Agarwal S et al. Building Rome in a day Commun. ACM 2011 54 10 105-112
[2]
Bartoli A, Gérard Y, Chadebecq F, Collins T, and Pizarro D Shape-from-template PAMI 2015 37 10 2099-2118
[3]
Birchfield, S., Tomasi, C.: Multiway cut for stereo and motion with slanted surfaces. In: ICCV (1999)
[4]
Burt, P.J., Wixson, L.E., Salgian, G.: Electronically directed “focal” stereo. In: ICCV (1995)
[5]
Chen, W., et al.: Learning to predict 3D objects with an interpolation-based differentiable renderer. In: NeurIPS (2019)
[6]
Chen, W., et al.: DIB-R++: learning to predict lighting and material with a hybrid differentiable renderer. In: NeurIPS (2021)
[7]
Furukawa Y and Hernàndez C Multi-view stereo: a tutorial Found. Trends Comput. Graph. Vis. 2013 9 1–2 1-148
[8]
Furukawa Y and Ponce J Accurate, dense, and robust multiview stereopsis PAMI 2010 32 8 1362-1376
[9]
Galliani, S., Lasinger, K., Schindler, K.: Massively parallel multiview stereopsis by surface normal diffusion. In: ICCV, pp. 873–881 (2015)
[10]
Gallup, D., Frahm, J., Mordohai, P., Yang, Q., Pollefeys, M.: Real-time plane-sweeping stereo with multiple sweeping directions. In: CVPR (2007)
[11]
Gårding J and Lindeberg T Direct computation of shape cues using scale-adapted spatial derivative operators IJCV 1996 17 2 163-191
[12]
Goesele, M., Curless, B., Seitz, S.M.: Multi-view stereo revisited. In: CVPR, vol. 2, pp. 2402–2409 (2006)
[13]
Griwodz, C., et al.: AliceVision Meshroom: an open-source 3D reconstruction pipeline. In: ACM Multimedia Systems Conference, pp. 241–247 (2021)
[14]
Hernández C and Schmitt F Silhouette and stereo fusion for 3D object modeling CVIU 2004 96 3 367-392
[15]
Jiang, Y., Ji, D., Han, Z., Zwicker, M.: SDFDiff: differentiable rendering of signed distance fields for 3D shape optimization. In: CVPR (2020)
[16]
Kazhdan MM and Hoppe H Screened poisson surface reconstruction ACM Trans. Graph. 2013 32 3 1-13
[17]
Liao, Y., Donné, S., Geiger, A.: Deep marching cubes: learning explicit surface representations. In: CVPR (2018)
[18]
Lindeberg T and Gårding J Eklundh J-O Shape-adapted smoothing in estimation of 3-D depth cues from affine distortions of local 2-D brightness structure Computer Vision — ECCV ’94 1994 Heidelberg Springer 389-400
[19]
Liu, H.D., Williams, F., Jacobson, A., Fidler, S., Litany, O.: Learning smooth neural functions via Lipschitz regularization. In: SIGGRAPH (2022)
[20]
Liu, J., et al.: PlaneMVS: 3D plane reconstruction from multi-view stereo. In: CVPR (2022)
[21]
Liu, S., Chen, W., Li, T., Li, H.: Soft rasterizer: a differentiable renderer for image-based 3D reasoning. In: ICCV (2019)
[22]
Mildenhall B, Srinivasan PP, Tancik M, Barron JT, Ramamoorthi R, and Ng R NeRF: representing scenes as neural radiance fields for view synthesis Commun. ACM 2022 65 1 99-106
[23]
Munkberg, J., et al.: Extracting triangular 3D models, materials, and lighting from images. In: CVPR (2022)
[24]
Niemeyer, M., Mescheder, L.M., Oechsle, M., Geiger, A.: Differentiable volumetric rendering: learning implicit 3D representations without 3D supervision. In: CVPR (2020)
[25]
Schönberger JL, Zheng E, Frahm J-M, and Pollefeys M Leibe B, Matas J, Sebe N, and Welling M Pixelwise view selection for unstructured multi-view stereo Computer Vision – ECCV 2016 2016 Cham Springer 501-518
[26]
Shen, T., Gao, J., Yin, K., Liu, M., Fidler, S.: Deep marching tetrahedra: a hybrid representation for high-resolution 3D shape synthesis. In: NeurIPS (2021)
[27]
Strecha, C., Fransens, R., Gool, L.V.: Wide-baseline stereo from multiple views: a probabilistic account. In: CVPR (2004)
[28]
Zabulis, X., Daniilidis, K.: Multi-camera reconstruction based on surface normal estimation and best viewpoint selection. In: 3DPVT (2004)
Recommendations
Accurate Camera Calibration from Multi-View Stereo and Bundle Adjustment
The advent of high-resolution digital cameras and sophisticated multi-view stereo algorithms offers the promise of unprecedented geometric fidelity in image-based modeling tasks, but it also puts unprecedented demands on camera calibration to fulfill ...
Accurate Multi-view Stereopsis Fusing DAISY Descriptor and Scaled-Neighbourhood Patches
PCM 2016: 17th Pacific-Rim Conference on Advances in Multimedia Information Processing - Volume 9916In this paper, we present an efficient patch-based multi-view stereo reconstruction approach, which is designed to reconstruct accurate, dense 3D models on high-resolution image sets. Wide-baseline matching becomes more challenging due to large ...
Stereo fusion
A stereo fusion system that combines binocular and refractive stereo is presented.Our stereo fusion outperforms traditional binocular and refractive stereo.An efficient calibration method for refractive stereo is proposed. Display Omitted The performance ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
May 2023
766 pages
ISBN:978-3-031-31974-7
DOI:10.1007/978-3-031-31975-4
© The Author(s), under exclusive license to Springer Nature Switzerland AG 2023.
Publisher
Springer-Verlag
Berlin, Heidelberg
Publication History
Published: 21 May 2023
Author Tags
Qualifiers
- Article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 24 Nov 2024
Other Metrics
Citations
View Options
View options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in