Abstract
The goal of most image based rendering systems can be stated as follows: given a set of pictures taken from various vantage points, synthesize the image that would be obtained from a novel viewpoint. In this paper we present a novel approach to view synthesis which hinges on the observation that human viewers tend to be quite sensitive to the motion of features in the image corresponding to intensity discontinuities or edges. Our system focuses its efforts on recovering the 3D position of these features so that their motions can be synthesized correctly. In the current implementation these feature points are recovered from image sequences by employing the epipolar plane image (EPI) analysis techniques proposed by Bolles, Baker, and Marimont. The output of this procedure resembles the output of an edge extraction system where the edgels are augmented with accurate depth information. This method has the advantage of producing accurate depth estimates for most of the salient features in the scene including those corresponding to occluding contours. We will demonstrate that it is possible to produce compelling novel views based on this information.
The paper will also describe a principled approach to reasoning about the 3D structure of the scene based on the quasi-sparse features returned by the EPI analysis. This analysis allows us to correctly reproduce occlusion and disocclusion effects in the synthetic views without requiring dense correspondences. Importantly, the technique could also be used to analyze and refine the 3-D results returned by range finders, stereo systems or structure from motion algorithms. Results obtained by applying the proposed techniques to actual image data sets are presented.
Chapter PDF
Similar content being viewed by others
References
R.C. Bolles, H.H. Baker, and D.H. Marimont. Epipolar-plane image anlysis: An approach to determining structure from motion. International Journal of Computer Vision, 1(1):7–55, 1987.
S. E. Chen and L. Williams. View interpolation from image synthesis. In SIGGRAPH, pages 279–288, August 1993.
Brian Curless and Marc Levoy. A volumetric method for building complex models from range images. In Proceedings of SIGGRAPH 96. In Computer Graphics Proceedings, Annual Conference Series, pages 31–43, New Orleans, LA, August 4–9 1996. ACM SIGGRAPH.
Y. Genc and J. Ponce. Parameterized image varieties: A novel approach to the analysis and synthesis of image sequences. In International Conference on Computer Vision, pages 11–16, January 1998.
Steven J. Gortler, Radek Grzeszczuk, Richard Szeliski, and Michael Cohen. The lumigraph. In Proceedings of SIGGRAPH 96. In Computer Graphics Proceedings, Annual Conference Series, pages 31–43, New Orleans, LA, August 4–9 1996. ACM SIGGRAPH.
V. Hlavac, A. Leonardis, and T. Werner. Automatic selection of reference views for image-based scene representations. In European Conference on Computer Vision, pages 526–535, 1996.
T. Kanade, P.W. Rander, and J. P. Narayanan. Virtualized reality: Constructing virtual worlds from real scenes. IEEE Multimedia, 4(1):34–47, 1997.
S. Laveau and O.D Faugeras. 3-d scene representation as a collection of images. In International Conference on Pattern Recognition, pages 689–691, 1994.
Marc Levoy and Pat Hanrahan. Light field rendering. In Proceedings of SIGGRAPH 96. In Computer Graphics Proceedings, Annual Conference Series, pages 31–43, New Orleans, LA, August 4–9 1996. ACM SIGGRAPH.
Maxime Lhuiller and Long Quan. Image interpolation by joint view triangulation. In Proc. IEEE Conf. on Vision and Patt. Recog., pages 139–145, 1999.
Maxime Lhuiller and Long Quan. Edge-constrained joint view triangulation for image interpolation. In Proc. IEEE Conf. on Vision and Patt. Recog., pages 218–224, 2000.
Ko Nishino, Yoichi Sato, and Katsui Ikeuchi. Eigen-texture method: Appearance compression based on 3d model. In Proc. IEEE Conf. on Vision and Patt. Recog., pages 618–624, 1999.
M. Pollefeys, L Van Gool, and M. Proesmans. Euclidean 3d reconstruction from image sequences with variable focal lengths. In European Conference on Computer Vision, pages 31–42, 1996.
W. Press, B. Flannery, S. Teukolsky, and W. Vetterling. Numerical Recipes in C. Cambridge University Press, 1988.
Y. Sato, M.D. Wheeler, and K. Ikeuchi. Object shape and reflectance modeling from observation. In Proceedings of SIGGRAPH 97. In Computer Graphics Proceedings, Annual Conference Series, pages 379–387. ACM SIGGRAPH, August 1997.
Steven Seitz and Charles R. Dyer. View morphing. In Proceedings of SIGGRAPH 96. In Computer Graphics Proceedings, Annual Conference Series, pages 31–43, New Orleans, LA, August 4–9 1996. ACM SIGGRAPH.
J.A. Sethian. Level Set Methods; Evolving Interfaces in Geometry, Fluid Mechanics, Computer Vision and Material Sciences. Cambridge University Press, 1996.
Jonathan Shade, Steven Gortler, Li wei He, and Richard Szeliski. Layered depth images. In Proceedings of SIGGRAPH 98. In Computer Graphics Proceedings, Annual Conference Series, pages 231–242. ACM SIGGRAPH, August 1998.
Heung-Yeung Shum and Li-Wei He. Rendering with concentric mosaics. In SIGGRAPH, pages 299–306, August 1999.
Ioannis Stamos and Peter Allen. 3-d model construction using range and image data. In IEEE Conference on Computer Vision and Pattern Recogniton, 2000.
Hai Tao and Harpreet Sawhney. Global matching criterion and color segmentation based stereo. In Workshop on the Applications of Computer Vision, pages 246–253, December 2000.
Hai Tao, Harpreet Sawhney, and Rakesh Kumar. A global matching framework for stereo computation. In International Conference on Computer Vision, pages 532–539, 2001.
T. Werner, R.D. Hersch, and V. Hlavac. Rendering real-world objects using view interpolation. In International Conference on Computer Vision, pages 957–962, 1995.
Masanobu Yamamoto. The Image Sequence Analysis of Three-Dimensional Dynamic Scenes. PhD thesis, Electrotechnical Laboratory, Agency of Industrial Science and Technology, May 1988.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2002 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Jelinek, D., Taylor, C.J. (2002). View Synthesis with Occlusion Reasoning Using Quasi-Sparse Feature Correspondences. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds) Computer Vision — ECCV 2002. ECCV 2002. Lecture Notes in Computer Science, vol 2351. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-47967-8_31
Download citation
DOI: https://doi.org/10.1007/3-540-47967-8_31
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-43744-4
Online ISBN: 978-3-540-47967-3
eBook Packages: Springer Book Archive