Article

Image fusion for context enhancement and video surrealism

Authors:

Jingyi YuAuthors Info & Claims

NPAR '04: Proceedings of the 3rd international symposium on Non-photorealistic animation and rendering

Pages 85 - 152

https://doi.org/10.1145/987657.987671

Published: 07 June 2004 Publication History

Abstract

We present a class of image fusion techniques to automatically combine images of a scene captured under different illumination. Beyond providing digital tools for artists for creating surrealist images and videos, the methods can also be used for practical applications. For example, the non-realistic appearance can be used to enhance the context of nighttime traffic videos so that they are easier to understand. The context is automatically captured from a fixed camera and inserted from a day-time image (of the same scene). Our approach is based on a gradient domain technique that preserves important local perceptual cues while avoiding traditional problems such as aliasing, ghosting and haloing. We presents several results in generating surrealistic videos and in increasing the information density of low quality nighttime videos.

References

[1]

BOSTONWEBCAMS, 2003. (Observe the cameras at night). http://www.boston.com/traffic/cameras/artery.htm.

[2]

BRAUN, M. 1992. Picturing Time. University of Chicago.

[3]

CHUANG, Y., CURLESS, B., SALESIN, D., AND R., S. 2001. A Bayesian Approach to Digital Matting. In Proceedings of CVPR, vol. 2, 264--271.

[4]

COHEN, M., 2003. Image Stacks. Presentation at MIT, Cambridge, USA, October 2003.

[5]

DECARLO, D., AND SANTELLA, A. 2002. Stylization and Abstraction of Photographs. In Proc. Siggraph 02, ACM Press.

Digital Library

[6]

DICARLO, J., AND WANDELL, B. 2000. Rendering High Dynamic Range Images. In Proceedings of SPIE: Image Sensors, vol. 3965, 392--401.

[7]

DURAND, F., AND DORSEY, J. 2002. Fast Bilateral Filtering for High-Dynamic-Range Images. In Proceedings of SIGGRAPH 2002, ACM SIGGRAPH, 257--266.

Digital Library

[8]

ESSA, I., 2002. Graphical Display of Motion-Capture System using a Shape-time Style Rendering.

[9]

FATTAL, R., LISCHINSKI, D., AND WERMAN, M. 2002. Gradient Domain High Dynamic Range Compression. In Proceedings of SIGGRAPH 2002, ACM SIGGRAPH, 249--256.

Digital Library

[10]

FELS, S., AND MASE, K. 1999. Interactive video cubism. In Workshop on New Paradigms in Information Visualization and Manipulation, ACM Press, 78--82.

Digital Library

[11]

FINLAYSON, G., HORDLEY, S., AND DREW, M. 2002. Removing Shadows from Images. In Proceedings of ECCV, vol. 4, 823--836.

Digital Library

[12]

FREEMAN, B., AND ZHANG, H. 2003. Shapetime photography. In Proceedings of CVPR, vol. 2, 264--271.

[13]

GLEICHER, M., HECK, R., AND WALLICK, M. 2002. A framework for virtual videography. In Smart Graphics.

Digital Library

[14]

HAEBERLI, P., 1994. A Multifocus Method for Controlling Depth of Field. Availavle at: http://www.sgi.com/grafica/depth/index.html.

[15]

HERTZMANN, A., AND PERLIN, K. 2000. Painterly Rendering for Video and Interaction. In Proceedings of NPAR 2000, Symposium on Non-Photorealistic Animation and Rendering (Annecy, France, June 2000), ACM, 7--12.

Digital Library

[16]

KLEIN, A. W., GRANT, T., FINKELSTEIN, A., AND COHEN, M. F. 2002. Video Mosaics. In Proceedings of NPAR 2002, International Symposium on Non Photorealistic Animation and Rendering (Annecy, France, June2002).

Digital Library

[17]

KLEIN, A. W., SLOAN, P.-P. J., FINKELSTEIN, A., AND COHEN, M. F. 2002. Stylized Video Cubes. In ACM SIGGRAPH Symposium on Computer Animation, 15--22.

Digital Library

[18]

LITWINOWICZ, P. 1997. Processing Images and Video for an Impressionist Effect. In Proceedings of SIGGRAPH'97 (Los Angeles, Aug. 1997), T. Whitted, Ed., Computer Graphics Proceedings, Annual Conference Series, ACM SIGGRAPH, 407--414.

Digital Library

[19]

MEIER, B. J. 1996. Painterly Rendering for Animation. In Proceedings of SIGGRAPH'96 (New Orleans, Aug. 1996), H. Rushmeier, Ed., Computer Graphics Proceedings, Annual Conference Series, ACM SIGGRAPH, 477--484.

Digital Library

[20]

MERRIAM-WEBSTER. 2001. Collegiate Dictionary.

[21]

MUYBRIDGE, E. 1985. Horses and other animals in motion. Dover.

[22]

PÉREZ, P., GANGNET, M., AND BLAKE, A. 2003. Poisson image editing. In Proceedings of SIGGRAPH 2003, 313--318.

Digital Library

[23]

PRESS, W. H., TEUKOLSKY, S., VETTERLING, W. T., AND FLANNERY, B. P. 1992. Numerical Recipes in C: The Art of Scientific Computing. Pearson Education.

Digital Library

[24]

RADEMACHER, P., AND BISHOP, G. 1998. Multiple-Center-of-Projection Images. In Proceedings of SIGGRAPH 98 (Orlando, July 1998), M. Cohen, Ed., Annual Conference Series, ACM SIGGRAPH, 199--206.

Digital Library

[25]

RASKAR, R., YU, J., AND ILIE, A., 2003. Stylized Images using Variable Illumination. Unpublished, January 2003.

[26]

RASKAR, R., YU, J., AND ILIE, A., 2003. Video Surveillance with NPR Image Fusion. http://www.merl.com/projects/NPRfusion/, March 2003.

[27]

REINHARD, E., STARK, M., SHIRLEY, P., AND FERWERDA, J. 2002. Photographic Tone Reproduction for Images. In Proceedings of SIGGRAPH 2002, ACM SIGGRAPH, 267--276.

Digital Library

[28]

SMITH, S., AND BRADY, J. 1997. SUSAN - a new approach to low level image processing. Int. Journal of Computer Vision 23, 1, 45--78.

Digital Library

[29]

SOCOLINSKY, D., AND WOLFF, L. 1999. A New Visualization Paradigm for Multispectral Imagery and Data Fusion. In Proceedings of IEEE CVPR, 319--324.

[30]

TOYAMA, K., KRUMM, J., BRUMITT, B., AND MEYERS, B. 1999. Wallflower: Principles and Practice of Background Maintenance. In ICCV, 255--261.

[31]

WEISS, Y. 2001. Deriving Intrinsic Images From Image Sequences. In Proceedings of ICCV, vol. 2, 68--75.

Cited By

Zhang YYuan JYan C(2024)Enhanced Seamless Video Fusion: A Convolutional Pyramid-Based 3D Integration AlgorithmSensors10.3390/s2406185224:6(1852)Online publication date: 14-Mar-2024
https://doi.org/10.3390/s24061852
Agarwala ADontcheva MAgrawala MDrucker SColburn ACurless BSalesin DCohen M(2023)Interactive Digital PhotomontageSeminal Graphics Papers: Pushing the Boundaries, Volume 210.1145/3596711.3596775(599-607)Online publication date: 1-Aug-2023
https://dl.acm.org/doi/10.1145/3596711.3596775
Liu SLiu S(2023)Low-Exposure Image and Video EnhancementImage and Video Color Editing10.1007/978-3-031-26030-8_7(65-75)Online publication date: 21-Mar-2023
https://doi.org/10.1007/978-3-031-26030-8_7
Show More Cited By

Image fusion for context enhancement and video surrealism
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision tasks
  2. Computer graphics

Recommendations

Image fusion for context enhancement and video surrealism
SIGGRAPH '05: ACM SIGGRAPH 2005 Courses

We present a class of image fusion techniques to automatically combine images of a scene captured under different illumination. Beyond providing digital tools for artists for creating surrealist images and videos, the methods can also be used for ...
Low Illumination Color Image Enhancement Based on Improved Retinex Theory
EITCE '20: Proceedings of the 2020 4th International Conference on Electronic Information Technology and Computer Engineering

In view of the defects of the classical Retinex image enhancement algorithm, such as poor texture detail retention, halo and over enhancement, and abrupt tone, this paper proposes a low illumination color image enhancement algorithm based on Gabor ...
Deep unsupervised endoscopic image enhancement based on multi-image fusion
Highlights
- For the first time,the advantages of Gamma derived image, CLAHE derived image and LIME derived image arecombinedwith a deep learning networktoobtainenhanced endoscopic images with superior image quality.
- An unsupervised derived images ...
Abstract Background and objective
A deep unsupervised endoscopic image enhancement method is proposed based on multi-image fusion to achieve high quality endoscope images from poorly illuminated, low contrast and color deviated images through an ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

NPAR '04: Proceedings of the 3rd international symposium on Non-photorealistic animation and rendering

June 2004

150 pages

ISBN:1581138873

DOI:10.1145/987657

Conference Chairs:
Aaron Hertzmann
University of Toronto, Canada
,
Craig Kaplan
University of Waterloo, Canada
,
Editor:
Stephen N. Spencer
The University of Washington

Copyright © 2004 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGGRAPH: ACM Special Interest Group on Computer Graphics and Interactive Techniques
Annecy Animation Festival

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 June 2004

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

NPAR04

Sponsor:

SIGGRAPH

NPAR04: The 3rd International Symposium on Non-Photorealistic Animation

June 7 - 9, 2004

Annecy, France

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

96
Total Citations
View Citations
1,131
Total Downloads

Downloads (Last 12 months)2
Downloads (Last 6 weeks)0

Reflects downloads up to 16 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Zhang YYuan JYan C(2024)Enhanced Seamless Video Fusion: A Convolutional Pyramid-Based 3D Integration AlgorithmSensors10.3390/s2406185224:6(1852)Online publication date: 14-Mar-2024
https://doi.org/10.3390/s24061852
Agarwala ADontcheva MAgrawala MDrucker SColburn ACurless BSalesin DCohen M(2023)Interactive Digital PhotomontageSeminal Graphics Papers: Pushing the Boundaries, Volume 210.1145/3596711.3596775(599-607)Online publication date: 1-Aug-2023
https://dl.acm.org/doi/10.1145/3596711.3596775
Liu SLiu S(2023)Low-Exposure Image and Video EnhancementImage and Video Color Editing10.1007/978-3-031-26030-8_7(65-75)Online publication date: 21-Mar-2023
https://doi.org/10.1007/978-3-031-26030-8_7
Liu SLiu S(2023)IntroductionImage and Video Color Editing10.1007/978-3-031-26030-8_1(1-8)Online publication date: 21-Mar-2023
https://doi.org/10.1007/978-3-031-26030-8_1
Saad NDahalan ASaudi A(2022)Modified Poisson compositing technique on skewed gridAIMS Mathematics10.3934/math.20221247:2(2176-2194)Online publication date: 2022
https://doi.org/10.3934/math.2022124
Wang JSilva DKosinka JTelea AHashimoto RRoerdink J(2022)Interactive image manipulation using morphological trees and spline-based skeletonsComputers & Graphics10.1016/j.cag.2022.09.002108(61-73)Online publication date: Nov-2022
https://doi.org/10.1016/j.cag.2022.09.002
Zhu ZMeng YKong DZhang XGuo YZhao Y(2021)To See in the Dark: N2DGAN for Background Modeling in Nighttime SceneIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2020.298787431:2(492-502)Online publication date: Feb-2021
https://doi.org/10.1109/TCSVT.2020.2987874
Zhu YXu CFeng BFang S(2021)Research on Endoscopic Image Enhancement Algorithm Based on Contrast Fusion2021 IEEE 6th International Conference on Signal and Image Processing (ICSIP)10.1109/ICSIP52628.2021.9689033(210-215)Online publication date: 22-Oct-2021
https://doi.org/10.1109/ICSIP52628.2021.9689033
Yang BSun YLi Y(2020)Image fusion with structural saliency measure and content adaptive consistency verificationJournal of Electronic Imaging10.1117/1.JEI.29.1.01301429:01(1)Online publication date: 4-Feb-2020
https://doi.org/10.1117/1.JEI.29.1.013014
Alenezi F(2020)Bayesian Estimation of Global and Local Variances for Improved Perceptual Quality of Fused Medical Images2020 International Conference on Intelligent Engineering and Management (ICIEM)10.1109/ICIEM48762.2020.9160127(7-11)Online publication date: Jun-2020
https://doi.org/10.1109/ICIEM48762.2020.9160127
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents