article

Real-time video abstraction

Authors:

Holger Winnemöller,

Bruce GoochAuthors Info & Claims

ACM Transactions on Graphics (TOG), Volume 25, Issue 3

Pages 1221 - 1226

https://doi.org/10.1145/1141911.1142018

Published: 01 July 2006 Publication History

Abstract

We present an automatic, real-time video and image abstraction framework that abstracts imagery by modifying the contrast of visually important features, namely luminance and color opponency. We reduce contrast in low-contrast regions using an approximation to anisotropic diffusion, and artificially increase contrast in higher contrast regions with difference-of-Gaussian edges. The abstraction step is extensible and allows for artistic or data-driven control. Abstracted images can optionally be stylized using soft color quantization to create cartoon-like effects with good temporal coherence. Our framework design is highly parallel, allowing for a GPU-based, real-time implementation. We evaluate the effectiveness of our abstraction framework with a user-study and find that participants are faster at naming abstracted faces of known persons compared to photographs. Participants are also better at remembering abstracted images of arbitrary scenes in a memory task.

Supplementary Material

JPG File (p1221-winnemoller-high.jpg)

Download
15.89 KB

JPG File (p1221-winnemoller-low.jpg)

Download
13.96 KB

High Resolution (p1221-winnemoller-high.mov)

Download
90.25 MB

Low Resolution (p1221-winnemoller-low.mov)

Download
30.59 MB

References

[1]

Arad, N., and Gotsman, C. 1999. Enhancement by image-dependent warping. IEEE Trans. on Image Processing 8, 9, 1063--1074.

Digital Library

[2]

Barash, D., and Comaniciu, D. 2004. A common framework for non-linear diffusion, adaptive smoothing, bilateral filtering and mean shift. Image and Video Computing 22, 1, 73--81.

[3]

Boomgaard, R. V. D., and de Weijer, J. V. 2002. On the equivalence of local-mode finding, robust estimation and mean-shift analysis as used in early vision tasks. 16th Internat. Conf. on Pattern Recog. 3, 927--390.

Digital Library

[4]

Canny, J. F. 1986. A computational approach to edge detection. IEEE Trans. on Pattern Analysis and Machine Intelligence 8, 769--798.

Digital Library

[5]

Collomosse, J. P., Rowntree, D., and Hall, P. M. 2005. Stroke surfaces: Temporally coherent artistic animations from video. IEEE Trans. on Visualization and Computer Graphics 11, 5, 540--549.

Digital Library

[6]

DeCarlo, D., and Santella, A. 2002. Stylization and abstraction of photographs. ACM Trans. Graph. 21, 3, 769--776.

Digital Library

[7]

Elder, J. H. 1999. Are edges incomplete? Internat. Journal of Computer Vision 34, 2-3, 97--122.

Digital Library

[8]

Fischer, J., Bartz, D., and Strasser, W. 2005. Stylized Augmented Reality for Improved Immersion. In Proc. of IEEE VR, 195--202.

Digital Library

[9]

Gooch, B., Reinhard, E., and Gooch, A. 2004. Human facial illustrations: Creation and psychophysical evaluation. ACM Trans. Graph. 23, 1, 27--44.

Digital Library

[10]

Hertzmann, A. 2001. Paint by relaxation. In CGI '01:Computer Graphics Internat. 2001, 47--54.

Digital Library

[11]

Itti, L., and Koch, C. 2001. Computational modeling of visual attention. Nature Reviews Neuroscience 2, 3, 194--203.

[12]

Loviscach, J. 1999. Scharfzeichner: Klare bilddetails durch verformung. Computer Technik 22, 236ff.

[13]

Marr, D., and Hildreth, E. C. 1980. Theory of edge detection. Proc. Royal Soc. London, Bio. Sci. 207, 187--217.

[14]

Palmer, S. E. 1999. Vision Science: Photons to Phenomenology. The MIT Press.

[15]

Perona, P., and Malik, J. 1991. Scale-space and edge detection using anisotropic diffusion. IEEE Trans. on Pattern Analysis and Machine Intelligence 12, 7.

Digital Library

[16]

Pham, T. Q., and Vliet, L. J. V. 2005. Separable bilateral filtering for fast video preprocessing. In IEEE Internat. Conf. on Multimedia & Expo, CD1-4.

[17]

Privitera, C. M., and Stark, L. W. 2000. Algorithms for defining visual regions-of-interest: Comparison with eye fixations. IEEE Trans. on Pattern Analysis and Machine Intelligence 22, 9, 970--982.

Digital Library

[18]

Raskar, R., Tan, K.-H., Feris, R., Yu, J., and Turk, M. 2004. Non-photorealistic camera: depth edge detection and stylized rendering using multi-flash imaging. ACM Trans. Graph. 23, 3, 679--688.

Digital Library

[19]

Saito, T., and Takahashi, T. 1990. Comprehensible rendering of 3-D shapes. In Proc. of ACM SIGGRAPH 90, 197--206.

Digital Library

[20]

Santella, A., and DeCarlo, D. 2004. Visual interest and NPR: an evaluation and manifesto. In Proc. of NPAR '04, 71--78.

Digital Library

[21]

Stevenage, S. V. 1995. Can caricatures really produce distinctiveness effects? British Journal of Psychology 86, 127--146.

[22]

Tomasi, C., and Manduchi, R. 1998. Bilateral filtering for gray and color images. In Proceedings of ICCV '98, 839.

Digital Library

[23]

Wang, J., Xu, Y., Shum, H.-Y., and Cohen, M. F. 2004. Video tooning. ACM Trans. Graph. 23, 3, 574--583.

Digital Library

[24]

Winkenbach, G., and Salesin, D. H. 1994. Computer-generated pen-and-ink illustration. In Proc. of ACM SIGGRAPH 94, 91--100.

Digital Library

[25]

Wyszecki, G., and Styles, W. 1982. Color Science: Concepts and Methods, Quantitative Data and Formulae. Wiley, New York, NY.

Cited By

Cui YZainol A(2024)Traditional art design expression based on embedded system developmentPeerJ Computer Science10.7717/peerj-cs.205510(e2055)Online publication date: 28-Jun-2024
https://doi.org/10.7717/peerj-cs.2055
Wang LYang XZhang J(2024)Fast Coherent Video Style Transfer via Flow Errors ReductionApplied Sciences10.3390/app1406263014:6(2630)Online publication date: 21-Mar-2024
https://doi.org/10.3390/app14062630
West RMukherjee S(2024)Stylized Rendering as a Function of ExpectationACM Transactions on Graphics10.1145/365816143:4(1-19)Online publication date: 19-Jul-2024
https://dl.acm.org/doi/10.1145/3658161
Show More Cited By

Index Terms

Real-time video abstraction
1. Computing methodologies
  1. Computer graphics
    1. Image manipulation
    2. Rendering

Recommendations

Real-time video abstraction
SIGGRAPH '06: ACM SIGGRAPH 2006 Papers

We present an automatic, real-time video and image abstraction framework that abstracts imagery by modifying the contrast of visually important features, namely luminance and color opponency. We reduce contrast in low-contrast regions using an ...
Real-time saliency-aware video abstraction

Existing real-time automatic video abstraction systems rely on local contrast only for identifying perceptually important information and abstract imagery by reducing contrast in low-contrast regions while artificially increasing contrast in higher ...
Active lighting for video conferencing

In consumer video conferencing, lighting conditions are usually not ideal thus the image qualities are poor. Lighting affects image quality on two aspects: brightness and skin tone. While there has been much research on improving the brightness of the ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Transactions on Graphics

ACM Transactions on Graphics Volume 25, Issue 3

July 2006

742 pages

ISSN:0730-0301

EISSN:1557-7368

DOI:10.1145/1141911

Issue’s Table of Contents

Copyright © 2006 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 July 2006

Published in TOG Volume 25, Issue 3

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

385
Total Citations
View Citations
5,859
Total Downloads

Downloads (Last 12 months)86
Downloads (Last 6 weeks)7

Reflects downloads up to 26 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Cui YZainol A(2024)Traditional art design expression based on embedded system developmentPeerJ Computer Science10.7717/peerj-cs.205510(e2055)Online publication date: 28-Jun-2024
https://doi.org/10.7717/peerj-cs.2055
Wang LYang XZhang J(2024)Fast Coherent Video Style Transfer via Flow Errors ReductionApplied Sciences10.3390/app1406263014:6(2630)Online publication date: 21-Mar-2024
https://doi.org/10.3390/app14062630
West RMukherjee S(2024)Stylized Rendering as a Function of ExpectationACM Transactions on Graphics10.1145/365816143:4(1-19)Online publication date: 19-Jul-2024
https://dl.acm.org/doi/10.1145/3658161
Zheng YCun XXia MPun C(2024)Sketch Video SynthesisComputer Graphics Forum10.1111/cgf.1504443:2Online publication date: 30-Apr-2024
https://doi.org/10.1111/cgf.15044
Yang YTang LYan TZeng LShen XZhan Y(2024)Parameterized $L_{0}$ Image Smoothing With Unsupervised LearningIEEE Transactions on Emerging Topics in Computational Intelligence10.1109/TETCI.2024.33590608:2(1938-1951)Online publication date: Apr-2024
https://doi.org/10.1109/TETCI.2024.3359060
Sekhar S(2024)Enhancing Anime Avatar Generation and Image Transfer with GANs2024 IEEE International Conference on Interdisciplinary Approaches in Technology and Management for Social Innovation (IATMSI)10.1109/IATMSI60426.2024.10503489(1-6)Online publication date: 14-Mar-2024
https://doi.org/10.1109/IATMSI60426.2024.10503489
Jingxin GXiao LRunPing H(2024)Research of Image Arbitrary Style Transfer based on Contrastive Learning2024 IEEE 7th Advanced Information Technology, Electronic and Automation Control Conference (IAEAC)10.1109/IAEAC59436.2024.10504039(69-74)Online publication date: 15-Mar-2024
https://doi.org/10.1109/IAEAC59436.2024.10504039
Long JZhang C(2024)Image smoothing combining edge-consistency with region-piecewise flattingComputers and Graphics10.1016/j.cag.2023.12.002118:C(90-101)Online publication date: 1-Feb-2024
https://dl.acm.org/doi/10.1016/j.cag.2023.12.002
Le TDinh PVu VGiang N(2024)A new approach to medical image fusion based on the improved Extended difference-of-Gaussians combined with the Coati optimization algorithmBiomedical Signal Processing and Control10.1016/j.bspc.2024.10617593(106175)Online publication date: Jul-2024
https://doi.org/10.1016/j.bspc.2024.106175
Spagnolo FCorsonello PFrustaci FPerri S(2024)Approximate bilateral filters for real-time and low-energy imaging applications on FPGAsThe Journal of Supercomputing10.1007/s11227-024-06084-y80:11(15894-15916)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1007/s11227-024-06084-y
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents