Abstract
Chinese traditional painting is one of the most historical artworks in the world. It is very popular in Eastern and Southeast Asia due to being aesthetically appealing. Compared with western artistic painting, it is usually more visually abstract and textureless. Recently, neural network based style transfer methods have shown promising and appealing results which are mainly focused on western painting. It remains a challenging problem to preserve abstraction in neural style transfer. In this paper, we present a Neural Abstract Style Transfer method for Chinese traditional painting. It learns to preserve abstraction and other style jointly end-to-end via a novel MXDoG-guided filter (Modified version of the eXtended Difference-of-Gaussians) and three fully differentiable loss terms. To the best of our knowledge, there is little work study on neural style transfer of Chinese traditional painting. To promote research on this direction, we collect a new dataset with diverse photo-realistic images and Chinese traditional paintings (The dataset will be released at https://github.com/lbsswu/Chinese_style_transfer.). In experiments, the proposed method shows more appealing stylized results in transferring the style of Chinese traditional painting than state-of-the-art neural style transfer methods.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Artisto (2016). http://artisto.my.com/
Baxter, B.: Dab: interactive haptic painting with 3D virtual brushes. In: ACM SIGGRAPH 2001 Video Review on Animation Theater Program, p. 10 (2001)
Becattini, F., Ferracani, A., Landucci, L., Pezzatini, D., Uricchio, T., Del Bimbo, A.: Imaging Novecento. A mobile app for automatic recognition of artworks and transfer of artistic styles (2016)
Gatys, L.A., Ecker, A.S., Bethge, M.: Image style transfer using convolutional neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2016)
Gatys, L.A., Ecker, A.S., Bethge, M., Hertzmann, A., Shechtman, E.: Controlling perceptual factors in neural style transfer. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, July 2017
Huang, H., et al.: Real-time neural style transfer for videos. In: CVPR (2017)
Johnson, J., Alahi, A., Fei-Fei, L.: Perceptual losses for real-time style transfer and super-resolution. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016, Part II. LNCS, vol. 9906, pp. 694–711. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46475-6_43
Joshi, B.J., Stewart, K., Shapiro, D.: Bringing impressionism to life with neural style transfer in come swim. CoRR abs/1701.04928 (2017)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. CoRR abs/1412.6980 (2014)
Lecun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86, 2278–2324 (1998)
Lee, J.: Simulating oriental black-ink painting. IEEE Comput. Graph. Appl. 19(3), 74–81 (1999)
Li, C., Wand, M.: Combining Markov random fields and convolutional neural networks for image synthesis. CoRR abs/1601.04589 (2016)
Li, C., Wand, M.: Precomputed real-time texture synthesis with Markovian generative adversarial networks. CoRR abs/1604.04382 (2016)
Li, S., Xu, X., Nie, L., Chua, T.S.: Laplacian-steered neural style transfer. In: Proceedings of the ACM Multimedia Conference (MM) (2017)
Lin, D., Wang, Y., Xu, G., Li, J., Fu, K.: Transform a simple sketch to a Chinese painting by a multiscale deep neural network. Algorithms 11(1), 4 (2018)
Lin, T., et al.: Microsoft COCO: common objects in context. CoRR abs/1405.0312 (2014)
Liu, Y., Qin, Z., Luo, Z., Wang, H.: Auto-painter: cartoon image generation from sketch by using conditional generative adversarial networks. CoRR abs/1705.01908 (2017)
Luan, F., Paris, S., Shechtman, E., Bala, K.: Deep photo style transfer (2017). arXiv preprint: arXiv:1703.07511
Marr, D., Hildreth, E.: Theory of edge detection. Proc. R. Soc. Lond. Ser. B 207, 187–217 (1980)
Nikulin, Y., Novak, R.: Exploring the neural algorithm of artistic style. CoRR abs/1602.07188 (2016)
Novak, R., Nikulin, Y.: Improving the neural algorithm of artistic style. CoRR abs/1605.04603 (2016)
Paszke, A., et al.: Automatic differentiation in PyTorch (2017)
Prisma (2016). http://prisma-ai.com/
Russakovsky, O., et al.: ImageNet large scale visual recognition challenge. Int. J. Comput. Vis. (IJCV) 115(3), 211–252 (2015)
Selim, A., Elgharib, M., Doyle, L.: Painting style transfer for head portraits using convolutional neural networks. ACM Trans. Graph. 35(4), 129:1–129:18 (2016)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. CoRR abs/1409.1556 (2014)
Strassmann, S.: Hairy brushes. In: Conference on Computer Graphics and Interactive Techniques, pp. 225–232 (1986)
Ulyanov, D., Lebedev, V., Vedaldi, A., Lempitsky, V.S.: Texture networks: Feed-forward synthesis of textures and stylized images. CoRR abs/1603.03417 (2016)
Ulyanov, D., Vedaldi, A., Lempitsky, V.S.: Instance normalization: the missing ingredient for fast stylization. CoRR abs/1607.08022 (2016)
Way, D.L., Lin, Y.R., Shih, Z.C.: The synthesis of trees in Chinese landscape painting using silhouette and texture strokes. J. WSCG 10, 499–506 (2002)
Winnemöller, H., Kyprianidis, J.E., Olsen, S.C.: XDoG: an extended difference-of-Gaussians compendium including advanced image stylization. Comput. Graph. 36(6), 740–753 (2012)
Xu, S., Xu, Y., Kang, S.B., Salesin, D.H., Pan, Y., Shum, H.Y.: Animating Chinese paintings through stroke-based decomposition. ACM Trans. Graph. 25(2), 239–267 (2006)
Zhang, S.H., Chen, T., Zhang, Y.F., Hu, S.M., Martin, R.: Video-based running water animation in Chinese painting style. Sci. China Inf. Sci. 52(2), 162–171 (2009)
Zhao, H., Li, H., Cheng, L.: Synthesizing filamentary structured images with GANs. CoRR abs/1706.02185 (2017)
Zhu, J.Y., Park, T., Isola, P., Efros, A.A.: Unpaired image-to-image translation using cycle-consistent adversarial networks (2017). arXiv preprint: arXiv:1703.10593
Acknowledgement
T. Wu is supported by ARO award W911NF1810295, ARO DURIP award W911NF1810209 and NSF IIS 1822477.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Li, B., Xiong, C., Wu, T., Zhou, Y., Zhang, L., Chu, R. (2019). Neural Abstract Style Transfer for Chinese Traditional Painting. In: Jawahar, C., Li, H., Mori, G., Schindler, K. (eds) Computer Vision – ACCV 2018. ACCV 2018. Lecture Notes in Computer Science(), vol 11362. Springer, Cham. https://doi.org/10.1007/978-3-030-20890-5_14
Download citation
DOI: https://doi.org/10.1007/978-3-030-20890-5_14
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-20889-9
Online ISBN: 978-3-030-20890-5
eBook Packages: Computer ScienceComputer Science (R0)