research-article

Mirror Surface Reconstruction from a Single Image

Authors:

Richard Hartley,

Mathieu SalzmannAuthors Info & Claims

IEEE Transactions on Pattern Analysis and Machine Intelligence, Volume 37, Issue 4

Pages 760 - 773

https://doi.org/10.1109/TPAMI.2014.2353622

Published: 01 April 2015 Publication History

Abstract

This paper tackles the problem of reconstructing the shape of a smooth mirror surface from a single image. In particular, we consider the case where the camera is observing the reflection of a static reference target in the unknown mirror. We first study the reconstruction problem given dense correspondences between 3D points on the reference target and image locations. In such conditions, our differential geometry analysis provides a theoretical proof that the shape of the mirror surface can be recovered if the pose of the reference target is known. We then relax our assumptions by considering the case where only sparse correspondences are available. In this scenario, we formulate reconstruction as an optimization problem, which can be solved using a nonlinear least-squares method. We demonstrate the effectiveness of our method on both synthetic and real images. We then provide a theoretical analysis of the potential degenerate cases with and without prior knowledge of the pose of the reference target. Finally we show that our theory can be similarly applied to the reconstruction of the surface of transparent object.

References

[1]

Y. Adato, Y. Vasilyev, O. Ben-Shahar, and T. Zickler, “ Toward a theory of shape from specular flow,” in Proc. IEEE Int. Conf. Comput. Vis., 2007, pp. 1–8.

[2]

J. Balzer, S. Höfer, and J. Beyerer, “ Multiview specular stereo reconstruction of large mirror surfaces,” in Proc. Comput. Vis. Pattern Recog., 2011, pp. 2537–2544.

[3]

T. Bonfort and P. Sturm, “Voxel carving for specular surfaces,” in Proc. IEEE Int. Conf. Comput. Vis., 2003, pp. 591–596.

[4]

T. Bonfort, P. Sturm, and P. Gargallo, “General Specular Surface Triangulation,” in Proc. Asian Conf. Comput. Vis., 2006, pp. 872–881.

[5]

G. D. Canas, Y. Vasilyev, Y. Adato, T. Zickler, S. Gortler, and O. Ben-Shahar, “A linear formulation of shape from specular flow,” in Proc. IEEE Int. Conf. Comput. Vis., 2009, pp. 191–198.

[6]

N. L. Earl and A. Coddington, Theory of Ordinary Differential Equations. New York, NY, USA: McGraw-Hill, 1955.

[7]

D. F. M. Endre Süli, An Introduction to Numerical Analysis. Cambridge, United Kingdom: Cambridge Univ. Press, 2003.

[8]

T. A. Fleming, R. W. and E. H. Adelson, “ Specular reflections and the perception of shape,” J. Vis., vol. 4, pp. 798–820, 2004.

[9]

M. A. Halstead, B. A. Barsky, S. A. Klein, and R. B. Mandell, “Reconstructing curved surfaces from specular reflection patterns using spline surface fitting of normals,” in Proc. 23rd Annu. Conf. Comput. Graph. Interact. Techniques, 1996, pp. 335– 342.

[10]

K. N. Kutulakos and E. Steger, “A theory of refractive and specular 3d shape by light-path triangulation,” Int. J. Comput. Vis., vol. 76, no. 1, pp. 13 –29, 2008.

Digital Library

[11]

M. Liu, K.-Y. K. Wong, Z. Dai, and Z. Chen, “Pose estimation from reflections for specular surface recovery,” in Proc. IEEE Int. Conf. Comput. Vis., 2011, pp. 579–586.

[12]

M. Liu, R. Hartley, and M. Salzmann, “Mirror Surface Reconstruction from a Single Image,” in Proc. Comput. Vis. Pattern Recog., 2013, pp. 129–136.

[13]

D. Nehab, T. Weyrich, and S. Rusinkiewicz, “Dense 3d reconstruction from specularity consistency,” in Proc. Comput. Vis. Pattern Recog. , 2008, pp. 1–8.

[14]

M. Oren and S. K. Nayar, “A theory of specular surface geometry,” Int. J. Comput. Vis. , vol. 24, no. 2, pp. 105–124, 1997.

Digital Library

[15]

S. Roth and M. J. Black, “Specular flow and the recovery of surface structure,” in Proc. Comput. Vis. Pattern Recog., 2006, pp. 1869–1876.

[16]

S. Rozenfeld, I. Shimshoni, and M. Lindenbaum, “Dense mirroring surface recovery from 1d homographies and sparse correspondences,” in Proc. Comput. Vis. Pattern Recog., 2007, pp. 1–8.

[17]

A. C. Sankaranarayanan, A. Veeraraghavan, O. Tuzel, and A. Agrawal, “Specular surface reconstruction from sparse reflection correspondences,” in Proc. Comput. Vis. Pattern Recogn., 2010, pp. 1245–1252.

[18]

S. Savarese, M. Chen, and P. Perona, “Local shape from mirror reflections,” Int. J. Comput. Vis., vol. 64, no. 1, pp. 31–67, 2005.

Digital Library

[19]

M. Tarini, H. P. A. Lensch, M. Goesele, and H.-P. Seidel, “3d acquisition of mirroring objects using striped patterns,” Graph. Model., vol. 67, no. 4, pp. 233–259, 2005.

Digital Library

[20]

Y. Vasilyev, T. Zickler, S. J. Gortler, and O. Ben -Shahar, “Shape from specular flow: Is one flow enough?” in Proc. Comput. Vis. Pattern Recog., 2011, pp. 2561–2568.

[21]

A. Zisserman, P. Giblin, and A. Blake, “The information available to a moving observer from specularities,” J. Image Vis. Comput. , vol. 7, no. 1, pp. 38–42, 1989.

Digital Library

[22]

S. Baker and Shree K. Nayar, “Single Viewpoint Catadioptric Cameras,” in Panoramic Vision: Sensors, Theory, Applications. New York, NY, USA: Springer, 2001.

[23]

R. Swaminathan, M. D. Grossberg, and S. K. Nayar, “ Non-Single Viewpoint Catadioptric Cameras: Geometry and Analysis,” Int. J. Comput. Vis., vol. 66, no. 3, pp. 211–229, 2006.

Digital Library

[24]

Peter F. Sturm and Thomas Bonfort, “How to compute the pose of an object without a direct view? ” in Proc. Asian Conf. Comput. Vis., 2006, pp. 21–31.

[25]

R. I. Hartley and A. Zisserman, Multiple View Geometry in Computer Vision. Cambridge, United Kingdom: Cambridge Univ. Press, 2004

[26]

V. Chari and P. Sturm, “A theory of refractive photo-light-path triangulation,” in Proc. Comput. Vis. Pattern Recog., 2013, pp. 1438–1445.

Cited By

Whelan TGoesele MLovegrove SStraub JGreen SSzeliski RButterfield SVerma SNewcombe R(2018)Reconstructing scenes with mirror and glass surfacesACM Transactions on Graphics10.1145/3197517.320131937:4(1-11)Online publication date: 30-Jul-2018
https://dl.acm.org/doi/10.1145/3197517.3201319

Index Terms

Mirror Surface Reconstruction from a Single Image
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems
        Reconstruction
  2. Computer graphics
    1. Shape modeling
2. Theory of computation
  1. Randomness, geometry and discrete structures
    1. Computational geometry

Index terms have been assigned to the content through auto-classification.

Recommendations

Mirror Surface Reconstruction from a Single Image
CVPR '13: Proceedings of the 2013 IEEE Conference on Computer Vision and Pattern Recognition

This paper tackles the problem of reconstructing the shape of a smooth mirror surface from a single image. In particular, we consider the case where the camera is observing the reflection of a static reference target in the unknown mirror. We first study ...
Single Image Based Reconstruction of High Field-Like MR Images
Medical Image Computing and Computer Assisted Intervention – MICCAI 2019
Abstract
Unavailability of high field magnetic resonance (MR) scanners has encouraged the development of post-processing algorithms to reconstruct 7T-like MR images from the given 3T MR images. Such algorithms have recently been developed with promising ...
Recovering Three-Dimensional Shape from a Single Image of Curved Objects

An algorithm to recover three-dimensional shape, i.e., surface orientation and relative depth from a single segmented image is presented. It is assumed that the scene is composed of opaque regular solid objects bounded by piecewise smooth surfaces with ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image IEEE Transactions on Pattern Analysis and Machine Intelligence

IEEE Transactions on Pattern Analysis and Machine Intelligence Volume 37, Issue 4

April 2015

208 pages

ISSN:0162-8828

Issue’s Table of Contents

Copyright © 2014.

Publisher

IEEE Computer Society

United States

Publication History

Published: 01 April 2015

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 26 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Whelan TGoesele MLovegrove SStraub JGreen SSzeliski RButterfield SVerma SNewcombe R(2018)Reconstructing scenes with mirror and glass surfacesACM Transactions on Graphics10.1145/3197517.320131937:4(1-11)Online publication date: 30-Jul-2018
https://dl.acm.org/doi/10.1145/3197517.3201319

View Options

View options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents