Article

Open access

A signal-processing framework for inverse rendering

Authors:

Ravi Ramamoorthi,

Pat HanrahanAuthors Info & Claims

SIGGRAPH '01: Proceedings of the 28th annual conference on Computer graphics and interactive techniques

Pages 117 - 128

https://doi.org/10.1145/383259.383271

Published: 01 August 2001 Publication History

PDF eReader

Abstract

Realism in computer-generated images requires accurate input models for lighting, textures and BRDFs. One of the best ways of obtaining high-quality data is through measurements of scene attributes from real photographs by inverse rendering. However, inverse rendering methods have been largely limited to settings with highly controlled lighting. One of the reasons for this is the lack of a coherent mathematical framework for inverse rendering under general illumination conditions. Our main contribution is the introduction of a signal-processing framework which describes the reflected light field as a convolution of the lighting and BRDF, and expresses it mathematically as a product of spherical harmonic coefficients of the BRDF and the lighting. Inverse rendering can then be viewed as deconvolution. We apply this theory to a variety of problems in inverse rendering, explaining a number of previous empirical results. We will show why certain problems are ill-posed or numerically ill-conditioned, and why other problems are more amenable to solution. The theory developed here also leads to new practical representations and algorithms. For instance, we present a method to factor the lighting and BRDF from a small number of views, i.e. to estimate both simultaneously when neither is known.

References

[1]

R. Basri and D. Jacobs. Lambertian reflectance and linear subspaces. In International Conference on Computer Vision, 2001.

Crossref

Google Scholar

[2]

R. Bastos, K. Hoff, W. Wynn, and A. Lastra. Increased photorealism for interactive architectural walkthroughs. In I3D 99, pages 183-190, 1999.

Digital Library

Google Scholar

[3]

B. Cabral, N. Max, and R. Springmeyer. Bidirectional reflection functions from surface bump maps. In SIGGRAPH 87, pages 273-281, 1987.

Digital Library

Google Scholar

[4]

B. Curless and M. Levoy. A volumetric method for building complex models from range images. In SIGGRAPH 96, pages 303-312, 1996.

Digital Library

Google Scholar

[5]

K. Dana, B. Ginneken, S. Nayar, and J. Koenderink. Reflectance and texture of real-world surfaces. ACM Transactions on Graphics, 18(1):1-34, January 1999.

Digital Library

Google Scholar

[6]

P. Debevec, T. Hawkins, C. Tchou, H.P. Duiker, W. Sarokin, and M. Sagar. Acquiring the reflectance field of a human face. In SIGGRAPH 00, pages 145-156.

Digital Library

Google Scholar

[7]

R. Dror, E. Adelson, and A. Willsky. Estimating surface reflectance properties from images under unknown illumination. In SPIE Photonics West: Human Vision and Electronic Imaging VI, 2001.

Crossref

Google Scholar

[8]

M. D'Zmura. Computational Models of Visual Processing, chapter Shading Ambiguity: Reflectance and Illumination, pages 187-207. MIT Press, 1991.

Google Scholar

[9]

Numerical Algorithms Group. NAG C Library Manual, Mark 5. 1999.

Google Scholar

[10]

K. Ikeuchi and K. Sato. Determining reflectance properties of an object using range and brightness images. PAMI, 13(11):1139-1153, 1991.

Digital Library

Google Scholar

[11]

T. Inui, Y. Tanabe, and Y. Onodera. Group theory and its applications in physics. Springer Verlag, 1990.

Crossref

Google Scholar

[12]

K. F. Karner, H. Mayer, and M. Gervautz. An image based measurement system for anisotropic reflection. Computer Graphics Forum, 15(3):119-128, 1996.

Crossref

Google Scholar

[13]

G.J. Klinker, S.A. Shafer, and T. Kanade. The measurement of highlights in color images. IJCV, 2(1):7-32, 1988.

Crossref

Google Scholar

[14]

J. J. Koenderink and A. J. van Doorn. Phenomenological description of bidirectional surface reflection. JOSA A, 15(11):2903-2912, 1998.

Crossref

Google Scholar

[15]

E. Land and J. McCann. Lightness and retinex theory. Journal of the Optical Society of America, 61(1):1-11, 1971.

Crossref

Google Scholar

[16]

M. Levoy, K. Pulli, B. Curless, S. Rusinkiewicz, D. Koller, L. Pereira, M. Ginzton, S. Anderson, J. Davis, J. Ginsberg, J. Shade, and D. Fulk. The digital michelangelo project: 3D scanning of large statues. In SIGGRAPH 00, pages 131-144.

Digital Library

Google Scholar

[17]

R.C. Love. Surface Reflection Model Estimation from Naturally Illuminated Image Sequences. PhD thesis, Leeds, 1997.

Google Scholar

[18]

R. Lu, J. Koenderink, and A. Kappers. Optical properties (bidirectional reflection distribution functions) of velvet. Applied Optics, 37(25):5974-5984, 1998.

Crossref

Google Scholar

[19]

T.M. MacRobert. Spherical harmonics; an elementary treatise on harmonic functions, with applications. Dover Publications, 1948.

Google Scholar

[20]

S.R. Marschner and D.P. Greenberg. Inverse lighting for photography. In Fifth Color Imaging Conference, pages 262-265, 1997.

Crossref

Google Scholar

[21]

S.R. Marschner, S.H. Westin, E.P.F. Lafortune, and K.E. Torrance. Image-Based BRDF measurement. Applied Optics, 39(16):2592-2600, 2000.

Crossref

Google Scholar

[22]

N. McCormick. Inverse radiative transfer problems: a review. Nuclear Science and Engineering, 112:185-198, 1992.

Crossref

Google Scholar

[23]

G. Miller and C. Hoffman. Illumination and reflection maps: Simulated objects in simulated and real environments. SIGGRAPH 84 Advanced Computer Graphics Animation seminar notes, 1984.

Google Scholar

[24]

F. E. Nicodemus, J. C. Richmond, J. J. Hsia, I. W. Ginsberg, and T. Limperis. Geometric Considerations and Nomenclature for Reflectance. National Bureau of Standards (US), 1977.

Google Scholar

[25]

J. Nimeroff, E. Simoncelli, and J. Dorsey. Efficient re-rendering of naturally illuminated environments. In EGWR 94, pages 359-373.

Google Scholar

[26]

R.W. Preisendorfer. Hydrologic Optics. US Dept Commerce, 1976.

Google Scholar

[27]

R. Ramamoorthi and P. Hanrahan. Analysis of planar light fields from homogeneous convex curved surfaces under distant illumination. In SPIE Photonics West: Human Vision and Electronic Imaging VI, 2001.

Crossref

Google Scholar

[28]

R. Ramamoorthi and P. Hanrahan. An efficient representation for irradiance environment maps. In SIGGRAPH 01, 2001.

Digital Library

Google Scholar

[29]

R. Ramamoorthi and P. Hanrahan. On the relationship between radiance and irradiance: Determining the illumination from images of a convex lambertian object. To appear, Journal of the Optical Society of America A, 2001.

Crossref

Google Scholar

[30]

I. Sato, Y. Sato, and K. Ikeuchi. Illumination distribution from brightness in shadows: adaptive estimation of illumination distribution with unknown reflectance properties in shadow regions. In ICCV 99, pages 875 - 882, 1999.

Google Scholar

[31]

Y. Sato and K. Ikeuchi. Reflectance analysis under solar illumination. Technical Report CMU-CS-94-221, Carnegie Mellon University, 1994.

Crossref

Google Scholar

[32]

Y. Sato, M. D. Wheeler, and K. Ikeuchi. Object shape and reflectance modeling from observation. In SIGGRAPH 97, pages 379-388, 1997.

Digital Library

Google Scholar

[33]

F. X. Sillion, J. Arvo, S. H. Westin, and D. Greenberg. A global illumination solution for general reflectance distributions. In SIGGRAPH 91, pages 187-196.

Digital Library

Google Scholar

[34]

S.R.Marschner. Inverse Rendering for Computer Graphics. PhD thesis, Cornell, 1998.

Digital Library

Google Scholar

[35]

S. Tominaga and N. Tanaka. Estimating reflection parameters from a single color image. IEEE Computer Graphics & Applications, 20(5):58-66, 2000.

Digital Library

Google Scholar

[36]

K. E. Torrance and E. M. Sparrow. Theory for off-specular reflection from roughened surfaces. JOSA, 57(9):1105-1114, 1967.

Crossref

Google Scholar

[37]

G. J. Ward. Measuring and modeling anisotropic reflection. In SIGGRAPH 92, pages 265-272, 1992.

Digital Library

Google Scholar

[38]

Y. Yu, P. Debevec, J. Malik, and T. Hawkins. Inverse global illumination: Recovering reflectance models of real scenes from photographs. In SIGGRAPH 99, pages 215-224, 1999.

Digital Library

Google Scholar

[39]

Y. Yu and J. Malik. Recovering photometric properties of architectural scenes from photographs. In SIGGRAPH 98, pages 207-218, 1998.

Digital Library

Google Scholar

[40]

Z. Zhang. A flexible new technique for camera calibration. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(11):1330-1334, 2000.

Digital Library

Google Scholar

Cited By

View all

Yi SKim DNa JTong XKim M(2024)Spin-Weighted Spherical Harmonics for Polarized Light TransportACM Transactions on Graphics10.1145/365813943:4(1-24)Online publication date: 19-Jul-2024
https://dl.acm.org/doi/10.1145/3658139
Huang YCao YLai YShan YGao L(2024)NeRF-Texture: Synthesizing Neural Radiance Field TexturesIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2024.338219846:9(5986-6000)Online publication date: Sep-2024
https://doi.org/10.1109/TPAMI.2024.3382198
Juven AAumeunier MMarot J(2024)Generative Models and Simulation to Assess Uncertainties for Tokamak Infrared Thermography2024 IEEE 34th International Workshop on Machine Learning for Signal Processing (MLSP)10.1109/MLSP58920.2024.10734728(1-6)Online publication date: 22-Sep-2024
https://doi.org/10.1109/MLSP58920.2024.10734728
Show More Cited By

Index Terms

A signal-processing framework for inverse rendering
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
  2. Computer graphics
    1. Animation

Recommendations

An efficient representation for irradiance environment maps
SIGGRAPH '01: Proceedings of the 28th annual conference on Computer graphics and interactive techniques

We consider the rendering of diffuse objects under distant illumination, as specified by an environment map. Using an analytic expression for the irradiance in terms of spherical harmonic coefficients of the lighting, we show that one needs to compute ...
A Signal-Processing Framework for Inverse Rendering
Seminal Graphics Papers: Pushing the Boundaries, Volume 2

Realism in computer-generated images requires accurate input models for lighting, textures and BRDFs. One of the best ways of obtaining high-quality data is through measurements of scene attributes from real photographs by inverse rendering. However, ...
A signal-processing framework for reflection

We present a signal-processing framework for analyzing the reflected light field from a homogeneous convex curved surface under distant illumination. This analysis is of theoretical interest in both graphics and vision and is also of practical ...

Reviews

Reviewer: Thomas W. Crockett

In computer graphics, rendering is the process of generating an image from an abstract description of a scene. In this paper, the authors address the inverse rendering problem, in which properties of a scene are deduced from a series of images. The focus in this case is on extracting illumination parameters and surface properties from known geometry under complex lighting conditions. To guide their efforts, the authors first develop a theoretical framework in which inverse rendering can be understood mathematically as a deconvolution operation. This theory is then used to determine what quantities can and cannot be recovered successfully by inverse rendering, and under what circumstances problems are well-posed or ill-conditioned. Although the analysis is based on a fairly restrictive set of assumptions (known geometry, convex isotropic surfaces, no interreflections), it leads to algorithms that can be applied to a wider range of problems. The successes and limitations of the method are demonstrated by comparing photographs of real scenes to renderings under known and unknown lighting conditions. The paper is well organized, well written, and amply illustrated. It is also heavily mathematical in content, and will therefore be of most interest to readers with a firm grounding in the mathematics of illumination and signal processing. The paper represents a fine example of the scientific method in computer science, using thorough analysis to guide the development of practical algorithms, which are verified through careful experimentation. The results obtained are compelling enough to suggest that this paper will become one of the classics in the field of inverse rendering. Online Computing Reviews Service

Access critical reviews of Computing literature here

Become a reviewer for Computing Reviews.

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

SIGGRAPH '01: Proceedings of the 28th annual conference on Computer graphics and interactive techniques

August 2001

600 pages

ISBN:158113374X

DOI:10.1145/383259

Chairman:
Lynn Pocock
New York Inst. of Technology

Seminal Graphics Papers: Pushing the Boundaries, Volume 2
August 2023
893 pages
ISBN:9798400708978
DOI:10.1145/3596711
Editor:
Mary C. Whitton
Department of Computer Science, UNC Chapel Hill, USA

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 August 2001

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Badges

Seminal Paper

Author Tags

Qualifiers

Article

Conference

SIGGRAPH01

Sponsor:

SIGGRAPH

SIGGRAPH01: The 28th International Conference on Computer Graphics and Interactive Techniques

Acceptance Rates

SIGGRAPH '01 Paper Acceptance Rate 65 of 300 submissions, 22%;

Overall Acceptance Rate 1,822 of 8,601 submissions, 21%

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

371
Total Citations
View Citations
2,362
Total Downloads

Downloads (Last 12 months)297
Downloads (Last 6 weeks)41

Reflects downloads up to 25 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Yi SKim DNa JTong XKim M(2024)Spin-Weighted Spherical Harmonics for Polarized Light TransportACM Transactions on Graphics10.1145/365813943:4(1-24)Online publication date: 19-Jul-2024
https://dl.acm.org/doi/10.1145/3658139
Huang YCao YLai YShan YGao L(2024)NeRF-Texture: Synthesizing Neural Radiance Field TexturesIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2024.338219846:9(5986-6000)Online publication date: Sep-2024
https://doi.org/10.1109/TPAMI.2024.3382198
Juven AAumeunier MMarot J(2024)Generative Models and Simulation to Assess Uncertainties for Tokamak Infrared Thermography2024 IEEE 34th International Workshop on Machine Learning for Signal Processing (MLSP)10.1109/MLSP58920.2024.10734728(1-6)Online publication date: 22-Sep-2024
https://doi.org/10.1109/MLSP58920.2024.10734728
Zuo ZLi YZhang TMo F(2024)A NeRF-Based Color Consistency Correction Method for Remote Sensing ImagesIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing10.1109/JSTARS.2024.337480817(6805-6817)Online publication date: 2024
https://doi.org/10.1109/JSTARS.2024.3374808
Enyo YNishino K(2024)Diffusion Reflectance Map: Single-Image Stochastic Inverse Rendering of Illumination and Reflectance2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52733.2024.01128(11873-11883)Online publication date: 16-Jun-2024
https://doi.org/10.1109/CVPR52733.2024.01128
Fei FTang JTan PShi B(2024)VMINer: Versatile Multi-view Inverse Rendering with Near-and Far-field Light Sources2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52733.2024.01121(11800-11809)Online publication date: 16-Jun-2024
https://doi.org/10.1109/CVPR52733.2024.01121
Xu FLi RWu SXu YWong H(2024)Text-Conditional Attribute Alignment Across Latent Spaces for 3D Controllable Face Image Synthesis2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52733.2024.00876(9172-9181)Online publication date: 16-Jun-2024
https://doi.org/10.1109/CVPR52733.2024.00876
Dib AGustavo Hafemann LGot EAnderson TFadaeinejad ACruz RCarbonneau M(2024)MoSAR: Monocular Semi-Supervised Model for Avatar Reconstruction using Differentiable Shading2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52733.2024.00174(1770-1780)Online publication date: 16-Jun-2024
https://doi.org/10.1109/CVPR52733.2024.00174
Han YLyu JXu F(2024)High-Quality Facial Geometry and Appearance Capture at Home2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52733.2024.00073(697-707)Online publication date: 16-Jun-2024
https://doi.org/10.1109/CVPR52733.2024.00073
Yang DYang QJiang WChen JShao ZLiu Q(2024)Video-driven speaker-listener generation based on Transformer and neural rendererMultimedia Tools and Applications10.1007/s11042-024-18291-z83:27(70501-70522)Online publication date: 29-Jan-2024
https://doi.org/10.1007/s11042-024-18291-z
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

An efficient representation for irradiance environment maps

A Signal-Processing Framework for Inverse Rendering

A signal-processing framework for reflection

Reviews

Access critical reviews of Computing literature here