Article

Video enhancement using per-pixel virtual exposures

Authors:

Eric P. Bennett,

Leonard McMillanAuthors Info & Claims

SIGGRAPH '05: ACM SIGGRAPH 2005 Papers

Pages 845 - 852

https://doi.org/10.1145/1186822.1073272

Published: 01 July 2005 Publication History

Abstract

We enhance underexposed, low dynamic range videos by adaptively and independently varying the exposure at each photoreceptor in a post-process. This virtual exposure is a dynamic function of both the spatial neighborhood and temporal history at each pixel. Temporal integration enables us to expand the image's dynamic range while simultaneously reducing noise. Our non-linear exposure variation and denoising filters smoothly transition from temporal to spatial for moving scene elements. Our virtual exposure framework also supports temporally coherent per frame tone mapping. Our system outputs restored video sequences with significantly reduced noise, increased exposure time of dark pixels, intact motion, and improved details.

Supplementary Material

JPG File (pps054.jpg)

Download
14.90 KB

MP4 File (pps054.mp4)

Download
48.14 MB

References

[1]

Acosta-Serafini, P. M., Masaki, I., and Sodini, C. G. 2004, Predictive Multiple Sampling Algorithm with Overlapping Integration Intervals for Linear Wide Dynamic Range Integrating Image Sensors. IEEE Transactions on Intelligent Transportation Systems, 5, 1, 33--41.

Digital Library

[2]

Barash, D. 2002. A Fundamental Relationship Between Bilateral Filtering, Adaptive Smoothing, and the Nonlinear Diffusion Equation. Transactions on Pattern Matching and Machine Learning, 24, 6, 844--847.

Digital Library

[3]

Bennett E. P. and McMillan, L. 2003. Proscenium: A Framework for Spatio-Temporal Video Editing. In Proceedings of ACM Multimedia 2003, 177--183.

Digital Library

[4]

Bidermann, W., El Gamal, A., Ewedemi, S., Reyneri, J., Tian, H., Wile, D., and Yang, D., 2003. A .18μm High Dynamic Range NTSC/PAL Imaging System-on-Chip with Embedded DRAM Frame Buffer, In Proceedings of the IEEE International Solid-State Circuits Conference, 212--213.

[5]

Boomgaard R. V. D., and Weijer, J. V. D. 2002. On the Equivalence of Local-Mode Finding, Robust Estimation and Mean Shift Analysis As Used In Early Vision Tasks. In Proceedings of the International Conference on Pattern Recognition, 927--930.

Digital Library

[6]

Choudhury, P. and Tumblin, J. 2003. The Trilateral Filter for High Contrast Images and Meshes. In Proceedings of the Eurographics Symposium on Rendering 2003. 1--11.

Digital Library

[7]

Cohen, M., Colburn, A., and Drucker, S. 2003. Image Stacks. Microsoft Research Technical Report, MSR-TR-2003-40.

[8]

Debevec, P. E. and Malik, J. 1997, Recovering High Dynamic Range Radiance Maps from Photographs. In Proceedings of ACM SIGGRAPH 1997. ACM SIGGRAPH / Addison Wesley, Computer Graphics Proceedings, Annual Conference Series, 369--378.

Digital Library

[9]

Drago, F., Myszkowski, K., Annen, T., and Chiba, N. 2003. Adaptive Logarithmic Mapping for Displaying High Contrast Scenes. In Proceedings of EUROGRAPHICS 2003, 22, 3, 419--426.

[10]

Dubois, E. and Sabri, S., 1984. Noise Reduction in Image Sequences Using Motion-Compensated Temporal Filtering, IEEE Transactions on Communications, 32, 7, 826--831.

[11]

Durand, F. and Dorsey, J. 2002. Fast Bilateral Filtering for the Display of High-Dynamic Range Images. ACM Transactions on Graphics, 21, 3, 257--266.

Digital Library

[12]

Eisemann, E. and Durand, F. 2004. Flash Photography Enhancement via Intrinsic Relighting. ACM Transactions on Graphics, 23, 3, 670--675.

Digital Library

[13]

Fattal, R., Lischinski, D., and Werman, M. 2002. Gradient Domain High Dynamic Range Compression. ACM Transactions on Graphics, 21, 3, 249--256.

Digital Library

[14]

Francis, J. J. and Jager, G. D. 2003. The Bilateral Median Filter. In Proceedings of the 14th Symposium of the Pattern Recognition Association of South Africa.

[15]

Jobson, D. J., Rahman, Z.-U., and Woodell, G. A. 1997. A Multiscale Retinex for Bridging the Gap Between Color Images and the Human Observation of Scenes. IEEE Transactions on Image Processing, 6, 7, 965--976.

Digital Library

[16]

Jostschulte, K., Amer, A., Schu, M., and Scroder, H., 1998. Perception Adaptive Temporal TV-Noise Reduction Using Contour Preserving Prefilter Techniques, IEEE Transactions on Consumer Electronics, 44, 3 (August), 1091--1096.

Digital Library

[17]

Kang, S. B., Uyttendaele, M., Winder, S., and Szeliski, R. 2003. High Dynamic Range Video, ACM Transactions on Graphics, 22, 3, 319--325.

Digital Library

[18]

Lee, S. H. and Kang, M. G. 1998. Spatio-Temporal Video Filtering Algorithm based on 3-D Anisotropic Diffusion Equation. In Proceedings of the International Conference on Image Processing, 98, 2, 447--450.

[19]

Liu, X., and El Gamal, A., 2003. Synthesis of High Dynamic Range Motion Blur Free Image From Multiple Captures. IEEE Transactions on Circuits and Systems, Fundamental Theory and Applications, 50, 4, 530--539.

[20]

Nayar, S. and Branzoi, V. 2003. Adaptive Dynamic Range Imaging: Optical Control of Pixel Exposures over Space and Time. In Proceedings of the International Conference on Computer Vision, 1--8.

Digital Library

[21]

Nayar, S. and Branzoi, V. 2004. Programmable Imaging Using a Digital Micromirror Array. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 436--443.

[22]

Pattanaik, S. N., Tumblin, J., Yee, H. and Greenberg, D. 2000. Time Dependent Visual Adaptation for Fast Realistic Image Display. In Proceedings of ACM SIGGRAPH 2000, ACM SIGGRAPH / Addison Wesley, Computer Graphics Proceedings, 47--54.

Digital Library

[23]

Perona, P. and Malik, J. 1990. Scale-Space and Edge Detection Using Anisotropic Diffusion. IEEE Transactions of Pattern Matching and Machine Intelligence, 12, 7, 629--639.

Digital Library

[24]

Petschnigg, G., Agrawala, M., Hoppe, H., Szeliski, R., Cohen, M. F., and Toyama, K. 2004. Digital Photograph with Flash and No-Flash Pairs. ACM Transactions on Graphics, 23, 3, 661--669.

Digital Library

[25]

Raskar, R., Ilie, A., and Yu, J. 2004. Image Fusion for Context Enhancement and Video Surrealism. In Proceedings of the International Symposium on Non-Photorealistic Animation and Rendering, 85--94.

Digital Library

[26]

Reibel, Y., Jung, M., Bouhifd. M., Cunin, B., and Draman, C. 2003. CCD or CMOS Camera Noise Characteristics. In Proceedings of the European Physical Journal of Applied Physics, 75--80.

[27]

Sand, P. and Teller, S. 2004. Video Matching. ACM Transactions on Graphics, 23, 3, 592--599.

Digital Library

[28]

Stockham, T. G. 1972. Image Processing in the Context of a Visual Model, In Proceedings of the IEEE, 60, 828--842.

[29]

Tomasi, C. and Manduchi, R. 1998. Bilateral Filtering for Gray and Color Images. In Proceedings of the International Conference on Computer Vision, 836--846.

Digital Library

[30]

Tumblin, J. and Rushmeier, H. E. 1993. Tone Reproduction for Realistic Images. IEEE Computer Graphics and Applications, 13,6,42--48.

Digital Library

[31]

Tumblin, J. and Turk, G. 1999. LCIS: A boundary hierarchy for detail preserving contrast reductions. In Proceedings of SIGGRAPH 1999,83--90.

Digital Library

[32]

Ward, G. 1991. Real Pixels. Graphics Gems II. Academic Press. 80--83.

[33]

Yee, H., Pattanaik, S, and Greenberg, D. P. 2001. Spatio-Temporal Sensitivity and Visual Attention for Efficient Rendering of Dynamic Environments. ACM Transactions on Graphics, 20, 1, 39--65.

Digital Library

Cited By

Yao ZSu JFan GGan MChen C(2024)GACA: A Gradient-Aware and Contrastive-Adaptive Learning Framework for Low-Light Image EnhancementIEEE Transactions on Instrumentation and Measurement10.1109/TIM.2024.335328573(1-14)Online publication date: 2024
https://doi.org/10.1109/TIM.2024.3353285
Abdullah Almujally NMehmood Qureshi AAlazeb ARahman HSadiq TAlonazi MAlgarni AJalal A(2024)A Novel Framework for Vehicle Detection and Tracking in Night Ware Surveillance SystemsIEEE Access10.1109/ACCESS.2024.341726712(88075-88085)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3417267
Buades ALisani J(2024)Efficient Recurrent Real Video RestorationDigital Signal Processing10.1016/j.dsp.2024.104851(104851)Online publication date: Nov-2024
https://doi.org/10.1016/j.dsp.2024.104851
Show More Cited By

Index Terms

Video enhancement using per-pixel virtual exposures
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
  2. Computer graphics
    1. Image manipulation
    2. Rendering

Recommendations

Video enhancement using per-pixel virtual exposures

We enhance underexposed, low dynamic range videos by adaptively and independently varying the exposure at each photoreceptor in a post-process. This virtual exposure is a dynamic function of both the spatial neighborhood and temporal history at each ...
Multispectral Bilateral Video Fusion

We present a technique for enhancing underexposed visible-spectrum video by fusing it with simultaneously captured video from sensors in nonvisible spectra, such as Short Wave IR or Near IR. Although IR sensors can accurately capture video in low-light ...
Modified Double Bilateral Filter for Sharpness Enhancement and Noise Removal
ACE '10: Proceedings of the 2010 International Conference on Advances in Computer Engineering

This paper propose a double bilateral filter that extends the classical bilateral filtering for image noise removal. A new median-metric weighting function is introduced by incorporating a 3 3 median filter into a second bilateral filter. A variety of ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGGRAPH '05: ACM SIGGRAPH 2005 Papers

July 2005

826 pages

ISBN:9781450378253

DOI:10.1145/1186822

Editor:
Markus Gross
Eidgenössische Technische Hochschule Zürich

Copyright © 2005 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGGRAPH: ACM Special Interest Group on Computer Graphics and Interactive Techniques

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 July 2005

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

SIGGRAPH05

Sponsor:

SIGGRAPH

SIGGRAPH05: Special Interest Group on Computer Graphics and Interactive Techniques Conference

July 31 - August 4, 2005

California, Los Angeles

Acceptance Rates

SIGGRAPH '05 Paper Acceptance Rate 98 of 461 submissions, 21%;

Overall Acceptance Rate 1,822 of 8,601 submissions, 21%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

53
Total Citations
View Citations
3,534
Total Downloads

Downloads (Last 12 months)40
Downloads (Last 6 weeks)6

Reflects downloads up to 22 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Yao ZSu JFan GGan MChen C(2024)GACA: A Gradient-Aware and Contrastive-Adaptive Learning Framework for Low-Light Image EnhancementIEEE Transactions on Instrumentation and Measurement10.1109/TIM.2024.335328573(1-14)Online publication date: 2024
https://doi.org/10.1109/TIM.2024.3353285
Abdullah Almujally NMehmood Qureshi AAlazeb ARahman HSadiq TAlonazi MAlgarni AJalal A(2024)A Novel Framework for Vehicle Detection and Tracking in Night Ware Surveillance SystemsIEEE Access10.1109/ACCESS.2024.341726712(88075-88085)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3417267
Buades ALisani J(2024)Efficient Recurrent Real Video RestorationDigital Signal Processing10.1016/j.dsp.2024.104851(104851)Online publication date: Nov-2024
https://doi.org/10.1016/j.dsp.2024.104851
Wang SWu GWang JZhu QShi YYin B(2024)SBC-Net: semantic-guided brightness curve estimation network for low-light image enhancementThe Visual Computer10.1007/s00371-024-03636-4Online publication date: 19-Oct-2024
https://doi.org/10.1007/s00371-024-03636-4
Zhu LYang WChen BZhu HNi ZMao QWang S(2024)Unrolled Decomposed Unpaired Learning for Controllable Low-Light Video EnhancementComputer Vision – ECCV 202410.1007/978-3-031-73337-6_19(329-347)Online publication date: 31-Oct-2024
https://doi.org/10.1007/978-3-031-73337-6_19
Bose SNawale SKhut DKolekar M(2023)LumiNet: Multispatial Attention Generative Adversarial Network for Backlit Image EnhancementIEEE Transactions on Instrumentation and Measurement10.1109/TIM.2023.331738472(1-14)Online publication date: 2023
https://doi.org/10.1109/TIM.2023.3317384
Wang JLiu XZhang YHu J(2023)TDS: Two Diffusion Streams for Low-Light Image Enhancement2023 6th International Conference on Software Engineering and Computer Science (CSECS)10.1109/CSECS60003.2023.10428430(1-7)Online publication date: 22-Dec-2023
https://doi.org/10.1109/CSECS60003.2023.10428430
Fan GFan BGan MChen GChen C(2022)Multiscale Low-Light Image Enhancement Network With Illumination ConstraintIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2022.318688032:11(7403-7417)Online publication date: Nov-2022
https://doi.org/10.1109/TCSVT.2022.3186880
Susa JMilitante SAcoba AEvangelista RLacatan LLaviňa CTanguilig B(2022)A Machine Vision-Based Person Detection Under Low-Illuminance Conditions Using High Dynamic Range Imagery for Visual Surveillance System2022 Third International Conference on Smart Technologies in Computing, Electrical and Electronics (ICSTCEE)10.1109/ICSTCEE56972.2022.10099673(1-6)Online publication date: 16-Dec-2022
https://doi.org/10.1109/ICSTCEE56972.2022.10099673
Buades ALisani J(2020)Enhancement of noisy and compressed videos by optical flow and non-local denoisingIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2019.2911877(1-1)Online publication date: 2020
https://doi.org/10.1109/TCSVT.2019.2911877
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents