article

Removing photography artifacts using gradient projection and flash-exposure sampling

Authors:

Shree K. Nayar,

Yuanzhen LiAuthors Info & Claims

ACM Transactions on Graphics (TOG), Volume 24, Issue 3

Pages 828 - 835

https://doi.org/10.1145/1073204.1073269

Published: 01 July 2005 Publication History

Abstract

Flash images are known to suffer from several problems: saturation of nearby objects, poor illumination of distant objects, reflections of objects strongly lit by the flash and strong highlights due to the reflection of flash itself by glossy surfaces. We propose to use a flash and no-flash (ambient) image pair to produce better flash images. We present a novel gradient projection scheme based on a gradient coherence model that allows removal of reflections and highlights from flash images. We also present a brightness-ratio based algorithm that allows us to compensate for the falloff in the flash image brightness due to depth. In several practical scenarios, the quality of flash/no-flash images may be limited in terms of dynamic range. In such cases, we advocate using several images taken under different flash intensities and exposures. We analyze the flash intensity-exposure space and propose a method for adaptively sampling this space so as to minimize the number of captured images for any given scene. We present several experimental results that demonstrate the ability of our algorithms to produce improved flash images.

Supplementary Material

MP4 File (pps052.mp4)

Download
42.15 MB

References

[1]

Agarwala, A., Dontcheva, M., Agrawala, M., Drucker, S., Colburn, A., Curless, B., Salesin, D., and Cohen, M. 2004. Interactive digital photomontage. ACM Transactions on Graphics 23, 3 (Aug.), 294--302.

Digital Library

[2]

Canon. http://www.cps.canon-europe.com/kb/.

[3]

Chen, H., Belhumeur, P., and Jacobs, D. 2000. In search of illumination invariants. In Proc. of IEEE Conf. on Computer Vision and Pattern Recognition, vol. 1, 254--261.

[4]

Debevec, P. E., and Malik, J. 1997. Recovering high dynamic range radiance maps from photographs. In Proc. of the 24th annual conference on Computer graphics and interactive techniques, 369--378.

Digital Library

[5]

Eisemann, E., and Durand, F. 2004. Flash photography enhancement via intrinsic relighting. ACM Transactions on Graphics 23, 3 (Aug.). 673--678.

Digital Library

[6]

Farid, H., and Adelson, E. H. 1999. Separating reflections and lighting using independent components analysis. In Proc. of IEEE Conf. on Computer Vision and Pattern Recognition, vol. 1, 1262--1267.

[7]

Fattal, R., Lischinski, D., and Werman, M. 2002. Gradient domain high dynamic range compression. ACM Transactions on Graphics 21, 3, 249--256.

Digital Library

[8]

Finlayson, G., Hordley, S., and Drew, M. 2002. Removing shadows from images. In Proc. of European Conf. on Computer Vision, vol. 4, 823--836.

Digital Library

[9]

Finlayson, G. D., Drew, M. S., and Lu, C. 2004. Intrinsic images by entropy minimization. In Proc. of European Conf. on Computer Vision, vol. 3, 582--595.

[10]

Grossberg, M. D., and Nayar, S. K. 2003. High dynamic range from multiple images: which exposures to combine? In Proc. of ICCV Workshop on Color and Photometric Methods in Computer Vision.

[11]

Jahne, B. 1993. Spatio-temporal image processing, theory and scientific applications, vol. 751 of Lecture Notes in Computer Vision. Springer-Verlag.

Digital Library

[12]

Kodak. 2001. CCD image sensor noise sources. Application note MPT/PS-0233.

[13]

Levin, A., Zomet, A., and Weiss, Y. 2004. Separating reflections from a single image using local features. In Proc. of IEEE Conf. on Computer Vision and Pattern Recognition, vol. 1, 306--313.

[14]

Lichtenauer, J., Reinders, M., and Hendriks, E. 2004. Influence of the observation likelihood function on particle filtering performance in tracking applications. In Sixth IEEE Int'l. Conf. on Automatic Face and Gesture Recognition, 767--772.

Digital Library

[15]

Mann, S., and Picard, R. W. 1995. Being undigital with digital cameras: extending dynamic range by combining differently exposed pictures. In Proc. of IS&T 48th annual conference, 422--428.

[16]

Nayar, S. K., and Mitsunaga, T. 2000. High dynamic range imaging: spatially varying pixel exposures. In Proc. of IEEE Conf. on Computer Vision and Pattern Recognition, vol. 1, 472--479.

[17]

Nayar, S. K., Fang, X.-S., and Boult, T. 1997. Separation of reflection components using color and polarization. Int'l. Journal of Computer Vision 21, 3 (Feb.), 163--186.

Digital Library

[18]

Perez, P., Gangnet, M., and Blake, A. 2003. Poisson image editing. ACM Transactions on Graphics 22, 3, 313--318.

Digital Library

[19]

Petschnigg, G., Agrawala, M., Hoppe, H., Szeliski, R., Cohen, M., and Toyama, K. 2004. Digital photography with flash and no-flash image pairs. ACM Transactions on Graphics 23, 3 (Aug.), 664--672.

Digital Library

[20]

Press, W. H., Teukolsky, S., Vetterling, W. T., and Flannery, B. P. 1992. Numerical recipes in C: the art of scientific computing. Pearson Education.

Digital Library

[21]

Raskar, R., Tan, K., Feris, R., Yu, J., and Turk, M. 2004. Non-photorealistic camera: depth edge detection and stylized rendering using multi-flash imaging. ACM Transactions on Graphics 23, 3, 679--688.

Digital Library

[22]

Raskar, R., Ilie, A., and Yu, J. 2004. Image fusion for context enhancement and video surrealism. In Proc. of NPAR, 85--95.

Digital Library

[23]

Schechner, Y. Y., Kiryati, N., and Basri, R. 2000. Separation of transparent layers using focus. Int'l. Journal of Computer Vision 39, 1 (Aug.), 25--39.

Digital Library

[24]

Sun, J., Jia, J., Tang, C.-K., and Shum, H.-Y. 2004. Poisson matting. ACM Transactions on Graphics 23, 3, 315--321.

Digital Library

[25]

Szeliski, R., Avidan, S., and Anandan, P. 2000. Layer extraction from multiple images containing reflections and transparency. In Proc. of IEEE Conf. on Computer Vision and Pattern Recognition, vol. 1, 246--243.

Cited By

Fong TYan W(2025)Signs on glasses: LiDAR data voids, hotspot effect, and reflection artifactsAutomation in Construction10.1016/j.autcon.2024.105877169(105877)Online publication date: Jan-2025
https://doi.org/10.1016/j.autcon.2024.105877
Lu ZWang WGuo TWang F(2025)Single-image reflection removal via self-supervised diffusion modelsThe Journal of Supercomputing10.1007/s11227-024-06837-981:1Online publication date: 1-Jan-2025
https://dl.acm.org/doi/10.1007/s11227-024-06837-9
Shi ZLi YZhang F(2024)Reflection Removal via Recurrent Learning Guided by Physics Prior and Focal Perceptual LossIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2024.340393234:10_Part_2(10152-10165)Online publication date: 1-Oct-2024
https://dl.acm.org/doi/10.1109/TCSVT.2024.3403932
Show More Cited By

Index Terms

Removing photography artifacts using gradient projection and flash-exposure sampling
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems
      2. Computer vision tasks
        Scene understanding
  2. Computer graphics
    1. Image manipulation
2. Theory of computation
  1. Randomness, geometry and discrete structures
    1. Random projections and metric embeddings

Recommendations

Removing photography artifacts using gradient projection and flash-exposure sampling
SIGGRAPH '05: ACM SIGGRAPH 2005 Papers

Flash images are known to suffer from several problems: saturation of nearby objects, poor illumination of distant objects, reflections of objects strongly lit by the flash and strong highlights due to the reflection of flash itself by glossy surfaces. ...
Siamese Dense Network for Reflection Removal with Flash and No-Flash Image Pairs
Abstract
This work addresses the reflection removal with flash and no-flash image pairs to separate reflection from transmission. When objects are covered by glass, the no-flash image usually contains reflection, and thus flash is used to enhance ...
Adaptive Page Packing and Storing Method for PCM-Flash Hybrid Memory Structure
HPCC-CSS-ICESS '15: Proceedings of the 2015 IEEE 17th International Conference on High Performance Computing and Communications, 2015 IEEE 7th International Symposium on Cyberspace Safety and Security, and 2015 IEEE 12th International Conf on Embedded Software and Systems

This paper presents an advanced PCM-Flash hybrid memory structure for the integrated memory-disk (IMD) structure merging the conventional main memory and disk storage into a single memory layer. The proposed structure can enhance overall write access ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Transactions on Graphics

ACM Transactions on Graphics Volume 24, Issue 3

July 2005

826 pages

ISSN:0730-0301

EISSN:1557-7368

DOI:10.1145/1073204

Issue’s Table of Contents

Copyright © 2005 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 July 2005

Published in TOG Volume 24, Issue 3

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

195
Total Citations
View Citations
2,193
Total Downloads

Downloads (Last 12 months)15
Downloads (Last 6 weeks)4

Reflects downloads up to 03 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Fong TYan W(2025)Signs on glasses: LiDAR data voids, hotspot effect, and reflection artifactsAutomation in Construction10.1016/j.autcon.2024.105877169(105877)Online publication date: Jan-2025
https://doi.org/10.1016/j.autcon.2024.105877
Lu ZWang WGuo TWang F(2025)Single-image reflection removal via self-supervised diffusion modelsThe Journal of Supercomputing10.1007/s11227-024-06837-981:1Online publication date: 1-Jan-2025
https://dl.acm.org/doi/10.1007/s11227-024-06837-9
Shi ZLi YZhang F(2024)Reflection Removal via Recurrent Learning Guided by Physics Prior and Focal Perceptual LossIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2024.340393234:10_Part_2(10152-10165)Online publication date: 1-Oct-2024
https://dl.acm.org/doi/10.1109/TCSVT.2024.3403932
Zhou Junzhuo 周Chen Minglin 陈Feng Ruqi 封Zong Yuan 纵Hao Jia 郝Yu Yiting 虞(2023)基于偏振信息的车窗透反混叠图像解耦研究Laser & Optoelectronics Progress10.3788/LOP22264360:3(0312023)Online publication date: 2023
https://doi.org/10.3788/LOP222643
Wan RShi BLi HHong YDuan LKot A(2023)Benchmarking Single-Image Reflection Removal AlgorithmsIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2022.316856045:2(1424-1441)Online publication date: 1-Feb-2023
https://doi.org/10.1109/TPAMI.2022.3168560
Hu QGuo X(2023)Single Image Reflection Separation via Component Synergy2023 IEEE/CVF International Conference on Computer Vision (ICCV)10.1109/ICCV51070.2023.01208(13092-13101)Online publication date: 1-Oct-2023
https://doi.org/10.1109/ICCV51070.2023.01208
Maralan SCareaga CAksoy Y(2023)Computational Flash Photography through Intrinsics2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52729.2023.01598(16654-16662)Online publication date: Jun-2023
https://doi.org/10.1109/CVPR52729.2023.01598
Petruzza SSumma BGooch ALaney CGoulden TSchreiner JCallahan SPascucci V(2023)Interactive Visualization and Portable Image Blending of Massive Aerial Image Mosaics2023 IEEE International Conference on Big Data (BigData)10.1109/BigData59044.2023.10386428(3365-3370)Online publication date: 15-Dec-2023
https://doi.org/10.1109/BigData59044.2023.10386428
Li YLiu MYi YLi QRen DZuo W(2023)Two-stage single image reflection removal with reflection-aware guidanceApplied Intelligence10.1007/s10489-022-04391-653:16(19433-19448)Online publication date: 7-Mar-2023
https://doi.org/10.1007/s10489-022-04391-6
Zhang HLiu TLiu YYuan Z(2023)Csf: global–local shading orders for intrinsic image decompositionMachine Vision and Applications10.1007/s00138-023-01485-035:1Online publication date: 27-Nov-2023
https://dl.acm.org/doi/10.1007/s00138-023-01485-0
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Issue’s Table of Contents