article

Multiscale shape and detail enhancement from multi-light image collections

Authors:

Maneesh Agrawala,

Szymon RusinkiewiczAuthors Info & Claims

ACM Transactions on Graphics (TOG), Volume 26, Issue 3

Pages 51 - es

https://doi.org/10.1145/1276377.1276441

Published: 29 July 2007 Publication History

Abstract

We present a new image-based technique for enhancing the shape and surface details of an object. The input to our system is a small set of photographs taken from a fixed viewpoint, but under varying lighting conditions. For each image we compute a multiscale decomposition based on the bilateral filter and then reconstruct an enhanced image that combines detail information at each scale across all the input images. Our approach does not require any information about light source positions, or camera calibration, and can produce good results with 3 to 5 input images. In addition our system provides a few high-level parameters for controlling the amount of enhancement and does not require pixel-level user input. We show that the bilateral filter is a good choice for our multiscale algorithm because it avoids the halo artifacts commonly associated with the traditional Laplacian image pyramid. We also develop a new scheme for computing our multiscale bilateral decomposition that is simple to implement, fast O(N² log N) and accurate.

Supplementary Material

JPG File (pps051.jpg)

Download
14.89 KB

MP4 File (pps051.mp4)

Download
36.49 MB

References

[1]

Agarwala, A., Dontcheva, M., Agrawala, M., Drucker, S., Colburn, A., Curless, B., Salesin, D., AND Cohen, M. 2004. Interactive digital photomontage. ACM Trans. Graph. (Proc. SIGGRAPH) 23, 3, 294--302.

Digital Library

[2]

Akers, D., Losasso, F., Klingner, J., Agrawala, M., Rick, J., and Hanrahan, P. 2003. Conveying shape and features with image-based relighting. In Proc. IEEE Visualization, 349--254.

Digital Library

[3]

Bae, S., Paris, S., and Durand, F. 2006. Two-scale tone management for photographic look. ACM Trans. Graph. (Proc. SIGGRAPH) 25, 3, 637--645.

Digital Library

[4]

Barash, D., and Comaniciu, D. 2004. A common framework for nonlinear diffusion, adaptive smoothing, bilateral filtering and mean shift. Journal of Image and Vision Computing 22, 1, 73--81.

[5]

Burt, P. J., and Adelson, E. H. 1983. The Laplacian pyramid as a compact image code. IEEE Trans. on Comm. 31, 4, 532--540.

[6]

Burt, P., and Kolczynski, R. 1993. Enhanced image capture through fusion. Proc. ICCV, 173--182.

[7]

Burt, P. J. 1981. Fast filter transforms for image processing. Computer Graphics, Image Processing 6, 20--51.

[8]

Choudhury, P., and Tumblin, J. 2003. The trilateral filter for high contrast images and meshes. In Proc. Eurographics Symposium on Rendering, 186--196.

Digital Library

[9]

Debevec, P., Hawkins, T., Tchou, C., Duiker, H.-P., Sarokin, W., and Sagar, M. 2000. Acquiring the reflectance field of a human face. In Proc. SIGGRAPH, 145--156.

Digital Library

[10]

Durand, F., and Dorsey, J. 2002. Fast bilateral filtering for the display of high-dynamic-range images. In Proc. SIGGRAPH, 257--266.

Digital Library

[11]

Eisemann, E., and Durand, F. 2004. Flash photography enhancement via intrinsic relighting. ACM Trans. Graph. (Proc. SIGGRAPH) 23, 3, 673--678.

Digital Library

[12]

Fattal, R., Lischinski, D., and Werman, M. 2002. Gradient domain high dynamic range compression. In Proc. SIGGRAPH, 249--256.

Digital Library

[13]

Finalyson, G. D., Hordley, S. D., Lu, C., and Drew, M. S. 2006. On the removal of shadows from images. IEEE Trans. Pattern Anal. Mach. Intell. 28, 1, 59--68.

Digital Library

[14]

Holschneider, M., Kronland-Martinet, R., Morlet, J., and Tchamitchian, P. 1989. A real-time algorithms for signal analysis with the help of the wavelet transform. Wavelets, Time-Frequency Methods and Phase Space, 286--297.

[15]

Khan, E. A., Reinhard, E., Fleming, R. W., and Bülthoff, H. H. 2006. Image-based material editing. ACM Trans. Graph. (Proc. SIGGRAPH) 25, 3, 654--663.

Digital Library

[16]

Li, H., Manjunath, B., and Mitra, S. 1994. Multi-sensor image fusion using the wavelet transform. Proc. ICIP 1, 51--55.

[17]

Li, Y., Sharan, L., and Adelson, E. H. 2005. Compressing and companding high dynamic range images with subband architectures. ACM Trans. Graph. (Proc. SIGGRAPH) 24, 3, 836--844.

Digital Library

[18]

Mallat, S. 1998. A Wavelet Tour of Signal Processing. Academic Press.

Digital Library

[19]

Malzbender, T., Gelb, D., and Wolters, H. 2001. Polynomial texture maps. In Proc. SIGGRAPH, 519--528.

Digital Library

[20]

Masselus, V., Dutré, P., and Anrys, F. 2002. The free-form light stage. In Proc. Eurographics Symposium on Rendering, 247--256.

Digital Library

[21]

Mohan, A., Tumblin, J., Bodenheimer, B., Grimm, C., and Bailey, R. 2005. Table-top computed lighting for practical digital photography. In Eurographics Symposium on Rendering, 165--172.

Digital Library

[22]

Mudge, M., Voutaz, J.-P., Schroer, C., and Lum, M. 2005. Reflection transformation imaging and virtual representations of coins from the Hospice of the Grand St. Bernard. In Proc. VAST.

[23]

Mudge, M., Malzbender, T., Schroer, C., and Lum, M. 2006. New reflection transformation imaging methods for rock art and multiple-viewpoint display. In Proc. VAST.

[24]

Ogden, J., Adelson, E., Bergen, J., and Burt, P. 1985. Pyramid-based computer graphics. RCA Engineer 30, 5, 4--15.

[25]

Oh, B. M., Chen, M., Dorsey, J., and Durand, F. 2001. Image-based modeling and photo editing. In Proc. SIGGRAPH, 433--442.

Digital Library

[26]

Paris, S., and Durand, F. 2006. A fast approximation of the bilateral filter using a signal processing approach. Tech. Rep. MIT-CSAIL-TR-2006-073.

[27]

Pattanaik, S. N., Ferwerda, J. A., Fairchild, M. D., and Greenberg, D. P. 1998. A multiscale model of adaptation and spatial vision for realistic image display. In Proc. SIGGRAPH, 287--298.

Digital Library

[28]

Petschnigg, G., Szeliski, R., Agrawala, M., Cohen, M., Hoppe, H., and Toyama, K. 2004. Digital photography with flash and no-flash image pairs. ACM Trans. Graph. (Proc. SIGGRAPH) 23, 3, 664--672.

Digital Library

[29]

Pham, T., and van Vliet, L. 2005. Separable Bilateral Filtering for Fast Video Preprocessing. Proc. ICME, 454--457.

[30]

Raskar, R., Ilie, A., and Yu, J. 2004. Image fusion for context enhancement and video surrealism. Proc. NPAR, 85--152.

Digital Library

[31]

Raskar, R., Tan, K.-H., Feris, R., Yu, J., and Turk, M. 2004. Non-photorealistic camera: depth edge detection and stylized rendering using multi-flash imaging. ACM Trans. Graph. (Proc. SIGGRAPH) 23, 3, 679--688.

Digital Library

[32]

Reinhard, E., Stark, M., Shirley, P., and Ferwerda, J. 2002. Photographic tone reproduction for digital images. In Proc. SIGGRAPH, 267--276.

Digital Library

[33]

Rusinkiewicz, S., Burns, M., and DeCarlo, D. 2006. Exaggerated shading for depicting shape and detail. ACM Trans. Graph. (Proc. SIGGRAPH) 25, 3, 1199--1205.

Digital Library

[34]

Tomasi, C., and Manduchi, R. 1998. Bilateral filtering for gray and color images. In Proc. ICCV, 839--846.

Digital Library

[35]

Tumblin, J., and Turk, G. 1999. LCIS: A boundary hierarchy for detail-preserving contrast reduction. In Proc. SIGGRAPH, 83--90.

Digital Library

[36]

Weiss, Y. 2001. Deriving intrinsic images from image sequences. In ICCV, 68--75.

[37]

Weiss, B. 2006. Fast median and bilateral filtering. ACM Trans. Graph. (Proc. SIGGRAPH) 25, 3, 519--526.

Digital Library

[38]

Winnemöller, H., Mohan, A., Tumblin, J., and Gooch, B. 2005. Light waving: Estimating light positions from photographs alone. Computer Graphics Forum 24, 3, 433--438.

[39]

Winnemöller, H., Olsen, S. C., and Gooch, B. 2006. Realtime video abstraction. ACM Trans. Graph. (Proc. SIGGRAPH) 25, 3, 1221--1226.

Digital Library

[40]

Woodham, R. 1980. Photometric method for determining surface orientation from multiple images. Optical Engineering 19, 1, 139--144.

Cited By

Xie AZhang AMei G(2024)An FPGA-Based Hardware Architecture of Gaussian-Adaptive Bilateral Filter for Real-Time Image DenoisingIEEE Access10.1109/ACCESS.2024.344399912(115277-115285)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3443999
Shang WLiu GWang YWang JMa Y(2024)A non-convex low-rank image decomposition model via unsupervised networkSignal Processing10.1016/j.sigpro.2024.109572223(109572)Online publication date: Oct-2024
https://doi.org/10.1016/j.sigpro.2024.109572
Yang YMa WGao WZeng LPan Z(2024)Hyperbolic tangent penalty function for edge-preserving image filteringDigital Signal Processing10.1016/j.dsp.2024.104591153(104591)Online publication date: Oct-2024
https://doi.org/10.1016/j.dsp.2024.104591
Show More Cited By

Index Terms

Multiscale shape and detail enhancement from multi-light image collections
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision representations
        Shape representations
  2. Computer graphics
    1. Image manipulation
    2. Shape modeling
      1. Parametric curve and surface models
      2. Volumetric models
2. Theory of computation
  1. Randomness, geometry and discrete structures
    1. Computational geometry

Recommendations

Multiscale shape and detail enhancement from multi-light image collections
SIGGRAPH '07: ACM SIGGRAPH 2007 papers

We present a new image-based technique for enhancing the shape and surface details of an object. The input to our system is a small set of photographs taken from a fixed viewpoint, but under varying lighting conditions. For each image we compute a ...
Improving Shape Depiction under Arbitrary Rendering

Based on the observation that shading conveys shape information through intensity gradients, we present a new technique called Radiance Scaling that modifies the classical shading equations to offer versatile shape depiction functionalities. It works by ...
Radiance Scaling for versatile surface enhancement
I3D '10: Proceedings of the 2010 ACM SIGGRAPH symposium on Interactive 3D Graphics and Games

We present a novel technique called Radiance Scaling for the depiction of surface shape through shading. It adjusts reflected light intensities in a way dependent on both surface curvature and material characteristics. As a result, diffuse shading or ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Transactions on Graphics

ACM Transactions on Graphics Volume 26, Issue 3

July 2007

976 pages

ISSN:0730-0301

EISSN:1557-7368

DOI:10.1145/1276377

Issue’s Table of Contents

Copyright © 2007 ACM.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 29 July 2007

Published in TOG Volume 26, Issue 3

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

240
Total Citations
View Citations
2,041
Total Downloads

Downloads (Last 12 months)10
Downloads (Last 6 weeks)1

Reflects downloads up to 26 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Xie AZhang AMei G(2024)An FPGA-Based Hardware Architecture of Gaussian-Adaptive Bilateral Filter for Real-Time Image DenoisingIEEE Access10.1109/ACCESS.2024.344399912(115277-115285)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3443999
Shang WLiu GWang YWang JMa Y(2024)A non-convex low-rank image decomposition model via unsupervised networkSignal Processing10.1016/j.sigpro.2024.109572223(109572)Online publication date: Oct-2024
https://doi.org/10.1016/j.sigpro.2024.109572
Yang YMa WGao WZeng LPan Z(2024)Hyperbolic tangent penalty function for edge-preserving image filteringDigital Signal Processing10.1016/j.dsp.2024.104591153(104591)Online publication date: Oct-2024
https://doi.org/10.1016/j.dsp.2024.104591
Pradhan KPatra S(2024)A semantic edge-aware parameter efficient image filtering techniqueComputers & Graphics10.1016/j.cag.2024.104068124(104068)Online publication date: Nov-2024
https://doi.org/10.1016/j.cag.2024.104068
Jia HYin QLu M(2024)Steering Kernel Weighted Guided Image Filtering with Gradient ConstraintComputers & Graphics10.1016/j.cag.2024.103908119(103908)Online publication date: Apr-2024
https://doi.org/10.1016/j.cag.2024.103908
Long JZhang C(2024)Image smoothing combining edge-consistency with region-piecewise flattingComputers & Graphics10.1016/j.cag.2023.12.002118(90-101)Online publication date: Feb-2024
https://doi.org/10.1016/j.cag.2023.12.002
Liu WZhang PQin HHuang XYang JNg M(2024)Fast Global Image Smoothing via Quasi Weighted Least SquaresInternational Journal of Computer Vision10.1007/s11263-024-02105-8Online publication date: 13-Jul-2024
https://doi.org/10.1007/s11263-024-02105-8
Lewis DChatoux HMansouri A(2024)SFF-RTI: an active multi-light approach to shape from focusThe Visual Computer: International Journal of Computer Graphics10.1007/s00371-023-02902-140:3(2067-2079)Online publication date: 1-Mar-2024
https://dl.acm.org/doi/10.1007/s00371-023-02902-1
Zhu DWang WXue XXie HCheng GWang F(2024)Structure-preserving image smoothing via contrastive learningThe Visual Computer: International Journal of Computer Graphics10.1007/s00371-023-02897-940:8(5139-5153)Online publication date: 1-Aug-2024
https://dl.acm.org/doi/10.1007/s00371-023-02897-9
Gušić MSokić E(2024)Flexible and Transportable Illumination Control System for Machine Vision ApplicationsNew Technologies, Development and Application VII10.1007/978-3-031-66271-3_7(58-65)Online publication date: 28-Jul-2024
https://doi.org/10.1007/978-3-031-66271-3_7
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents