Nothing Special   »   [go: up one dir, main page]
Skip to main content
Springer Nature Link
Log in

Image stylization with enhanced structure on GPU

  • Research Paper
  • Special Focus
  • Published:
Science China Information Sciences Aims and scope Submit manuscript

Abstract

This paper presents a graphics processing unit (GPU) based stylization approach that preserves the fine structure between the original and the stylized images using gradient optimization. Existing abstraction and painterly stylization methods focused on contrast manipulation only, and thus the detailed salient structures of the input images are always destroyed when performing the current stylization techniques because of limitations like unavoidable salience information loss caused by contrast abstraction. We propose an image structure map to naturally model the fine structure existing in the original images. Gradient-based structure tangent generation and tangent-guided image morphology are used to construct the structure map. The image structure map, unlike an edge map, not only systematically models the boundary information within the imagery but also accentuates the underlying inner structure detail for further stylization. We facilitate the final stylization via parallel bilateral grid and structure-aware stylizing optimization on a GPU-CUDA platform in real time. In multiple experiments, the proposed method consistently demonstrates efficient and high quality image stylization performance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Lu J W, Sander P V, Finkelstein A. Interactive painterly stylization of images, videos and 3D animations. In: Proc ACM SIGGRAPH Sym Interactive 3D Graphics and Games (I3D’10), 2010. 127–134

  2. Bousseau A, Neyret F, Thollot J, et al. Video watercolorization using bidirectional texture advection. In: Proc SIGGRAPH, 2007

  3. DeCarlo D, Santella A. Stylization and abstraction of photographs. In: Proc SIGGRAPH, 2002

  4. Hertzmann A. Paint by relaxation. In: Proc Computer Graphics International (CGI’01), 2001. 47–54

  5. Zhao M T, Zhu S C. Sisley the abstract painter. In: Proc ACM Sym Non-Photorealistic Animation and Rendering (NPAR’10), 2010. 99–107

  6. Kang H, Lee S Y, Chui C K. Flow-based image abstraction. IEEE Trans Vis Comput Graph, 2009, 15: 62–76

    Article  Google Scholar 

  7. Winnemöller H, Olsen S C, Gooch B. Real-time video abstraction. In: Proc SIGGRAPH, 2006

  8. Zhao H L, Mao X Y, Jin X G, et al. Real-time saliency-aware video abstraction. The Visual Computer, 2009, 25: 973–984

    Article  Google Scholar 

  9. Kang H, Lee S Y, Chui C K. Coherent line drawing. In: Proc ACM Sym Non-Photorealistic Animation and Rendering (NPAR’10), 2010. 43–50

  10. Wen F, Luan Q, Liang L, et al. Color sketch generation. In: Proc ACM Sym Non-Photorealistic Animation and Rendering (NPAR’06), 2006. 47–54

  11. Prendeville B. Realism in 20th Century Painting. New York: Thames & Hudson, 2000

    Google Scholar 

  12. Stremmel K, Grosenick U. Realism. Koln: Taschen, 2004

    Google Scholar 

  13. Hertzmann A. Non-photorealistic rendering and the science of art. In: Proc ACM Sym Non-Photorealistic Animation and Rendering (NPAR’10), 2010. 147–157

  14. Hertzmann A. Painterly rendering with curved brush strokes of multiple sizes. In: Proc SIGGRAPH, 1998. 453–460

  15. Hertzmann A, Jacobs C E, Oliver N, et al. Image Analogies. In: Proc SIGGRAPH, 2001. 327–340

  16. Bousseau A, Kaplan M, Thollot J, et al. Interactive watercolor rendering with temporal coherence and abstraction. In: Proc ACM Sym Non-Photorealistic Animation and Rendering (NPAR’06), 2006. 141–149

  17. Kyprianidis J E, Kang H. Image and video abstraction by coherence-enhancing filtering. In: Computer Graphics Forum (Eurographics’11), 2011

  18. Kyprianidis J E, Kang H, Döllner J. Image and video abstraction by anisotropic Kuwahara filtering. In: Computer Graphics Forum (Pacific Graphics’09), 2009, 28. 1955–1963

    Article  Google Scholar 

  19. Zhao H L, Jin X G, Shen J B, et al. Real-time feature-aware video abstraction. The Visual Computer, 2008, 24: 727–734

    Article  Google Scholar 

  20. Huang H, Zhang L, Fu T N. Video painting via motion layer manipulation. Computer Graphics Forum, 2010, 29: 2055–2064

    Article  Google Scholar 

  21. Huang H, Fu T N, Li C F. Painterly rendering with content-dependent natural paint strokes. The Visual Computer, 2011, 27: 861–871

    Article  Google Scholar 

  22. Zhang S H, Li X Y, Hu S M, et al. Online video stream abstraction and stylization. IEEE Trans Multimedia, 2011, 13: 1286–1294

    Article  Google Scholar 

  23. Hachisuka T, Jensen H W. Parallel progressive photon mapping on GPUs. In: Proc SIGGRAPH Asia Sketches, 2010

  24. Bertal M, Fort P, Sanchez-Crespo D. Real-time, accurate depth of field using anisotropic diffusion and programmable graphics cards. In: Proc 3DPVT, 2004. 767–773

  25. Kazhdan M, Hoppe H. Streaming multigrid for gradient-domain operations on large images. In: Proc SIGGRAPH, 2008

  26. Orzan A, Bousseau A, Winnemöller H, et al. Diffusion curves: a vector representation for smooth-shaded images. In: Proc SIGGRAPH, 2008

  27. Mavridis P, Papaioannou G. High quality elliptical texture filtering on GPU. In: Proc ACM SIGGRAPH Sym Interactive 3D Graphics and Games (I3D’11), 2011. 23–30

  28. Krähenbühl P, Lang M, Hornung A, et al. A system for retargeting of streaming video. In: Proc SIGGRAPH Asia, 2009

  29. McCann J, Pollard N S. Real-time gradient-domain painting. In: Proc SIGGRAPH, 2008

  30. Paris S, Durand F. A fast approximation of the bilateral filter using a signal processing approach. In: Proc European Conference on Computer Vision (ECCV’06), 2006. 24–52

  31. Chen J, Paris S, Durand F. Real-time edge-aware image processing with the bilateral grid. In: Proc SIGGRAPH, 2007

  32. Cheng M M, Zhang G X, Mitra N J, et al. Global contrast based salient region detection. In: Proc IEEE Conference on Computer Vision and Pattern Recognition (CVPR’11), 2011. 409–416

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, The Chinese University of Hong Kong, Hong Kong, China

    Ping Li, HanQiu Sun & Bin Sheng

  2. Department of Computer Science and Engineering, Shanghai Jiao Tong University, Shanghai, 200240, China

    Bin Sheng

  3. School of Computer Science and Technology, Beijing Institute of Technology, Beijing, 100081, China

    JianBing Shen

Authors
  1. Ping Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. HanQiu Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Bin Sheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. JianBing Shen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Ping Li, HanQiu Sun or Bin Sheng.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Li, P., Sun, H., Sheng, B. et al. Image stylization with enhanced structure on GPU. Sci. China Inf. Sci. 55, 1093–1105 (2012). https://doi.org/10.1007/s11432-012-4558-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11432-012-4558-y

Keywords

Search

Navigation