Destylization of text with decorative elements
Article No.: 14, Pages 1 - 7
Abstract
Style text with decorative elements has a strong visual sense, and enriches our daily work, study and life. However, it introduces new challenges to text detection and recognition. In this study, we propose a text destylized framework, that can transform the stylized texts with decorative elements into a type that is easily distinguishable by a detection or recognition model. We arranged and integrate an existing stylistic text data set to train the destylized network. The new destylized data set contains English letters and Chinese characters. The proposed approach enables a framework to handle both Chinese characters and English letters without the need for additional networks. Experiments show that the method is superior to the state-of-the-art style-related models.
Supplementary Material
Supplemental files.
- Download
- 918.67 KB
References
[1]
S. Azadi, M. Fisher, V. Kim, Z. Wang, E. Shechtman, and T. Darrell. 2018. Multi-content GAN for Few-Shot Font Style Transfer. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 7564--7573.
[2]
Alex J Champandard. 2016. Semantic style transfer and turning two-bit doodles into fine artworks. arXiv preprint arXiv:1603.01768 (2016).
[3]
Tao Chen, Ming-Ming Cheng, Ping Tan, Ariel Shamir, and Shi-Min Hu. 2009. Sketch2photo: Internet image montage. ACM Transactions on Graphics 28, 5 (2009), 1--10.
[4]
Yunjey Choi, Minje Choi, Munyoung Kim, Jung-Woo Ha, Sunghun Kim, and Jaegul Choo. 2018. StarGAN: Unified generative adversarial networks for multi-domain image-to-image translation. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 8789--8797.
[5]
Yingying Deng, Fan Tang, Weiming Dong, Haibin Huang, Chongyang Ma, and Changsheng Xu. 2021. Arbitrary Video Style Transfer via Multi-Channel Correlation. In Thirty-Fifth AAAI Conference on Artificial Intelligence (AAAI).
[6]
Yingying Deng, Fan Tang, Weiming Dong, Wen Sun, Feiyue Huang, and Changsheng Xu. 2020. Arbitrary Style Transfer via Multi-Adaptation Network. In Proceedings of the 28th ACM International Conference on Multimedia (Seattle, WA, USA). Association for Computing Machinery, New York, NY, USA, 2719--2727.
[7]
Lars Doyle, Forest Anderson, Ehren Choy, and David Mould. 2019. Automated pebble mosaic stylization of images. Computational Visual Media 5, 1 (2019), 33--44.
[8]
Iddo Drori, Daniel Cohen-Or, and Hezy Yeshurun. 2003. Example-based style synthesis. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), Vol. 2. IEEE, II--143.
[9]
Alexei A Efros and William T Freeman. 2001. Image quilting for texture synthesis and transfer. In Proceedings of the 28th annual conference on Computer graphics and interactive techniques. 341--346.
[10]
Michael Elad and Peyman Milanfar. 2017. Style transfer via texture synthesis. IEEE Transactions on Image Processing 26, 5 (2017), 2338--2351.
[11]
Oriel Frigo, Neus Sabater, Julie Delon, and Pierre Hellier. 2016. Split and match: Example-based adaptive patch sampling for unsupervised style transfer. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 553--561.
[12]
L. A. Gatys, A. S. Ecker, and M. Bethge. 2016. Image Style Transfer Using Convolutional Neural Networks. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2414--2423.
[13]
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative adversarial nets. In Advances in Neural Information Processing Systems (NIPS). 2672--2680.
[14]
Ishaan Gulrajani, Faruk Ahmed, Martin Arjovsky, Vincent Dumoulin, and Aaron C. Courville. 2017. Improved Training of Wasserstein GANs. In Advances in Neural Information Processing Systems (NIPS). 5767--5777.
[15]
Hideaki Hayashi, Kohtaro Abe, and Seiichi Uchida. [n.d.]. GlyphGAN: Style-Consistent Font Generation Based on Generative Adversarial Networks. 186 ([n. d.]).
[16]
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 770--778.
[17]
Aaron Hertzmann, Charles E Jacobs, Nuria Oliver, Brian Curless, and David H. Salesin. 2001. Image analogies. In Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques. 327--340.
[18]
Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A. Efros. 2017. Image-to-image translation with conditional adversarial networks. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 1125--1134.
[19]
Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen. 2017. Progressive Growing of GANs for Improved Quality, Stability, and Variation.
[20]
Zhouhui Lian, Bo Zhao, Xudong Chen, and Jianguo Xiao. 2019. EasyFont: A Style Learning-Based System to Easily Build Your Large-Scale Handwriting Fonts. ACM Transactions on Graphics 38, 1 (2019), 6:1--6:18.
[21]
Ming Yu Liu, Xun Huang, Arun Mallya, Tero Karras, Timo Aila, Jaakko Lehtinen, and Jan Kautz. 2019. Few-Shot Unsupervised Image-to-Image Translation. In IEEE/CVF International Conference on Computer Vision (ICCV). 10550--10559.
[22]
Jingwan Lu, Fisher Yu, Adam Finkelstein, and Stephen DiVerdi. 2012. Helping-Hand: Example-Based Stroke Stylization. ACM Transactions on Graphics 31, 4, Article 46 (July 2012), 10 pages.
[23]
Rui Qian, Robby T Tan, Wenhan Yang, Jiajun Su, and Jiaying Liu. 2018. Attentive generative adversarial network for raindrop removal from a single image. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2482--2491.
[24]
Alec Radford, Luke Metz, and Soumith Chintala. 2016. Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks. In International Conference on Learning Representations (ICLR).
[25]
Shuai, Yang, Jiaying, Liu, Wenhan, Zongming, and Guo. 2018. Context-Aware Text-Based Binary Image Stylization and Synthesis. IEEE Transactions on Image Processing 28, 2 (Feb. 2018), 952--964.
[26]
Dmitry Ulyanov, Andrea Vedaldi, and Victor Lempitsky. 2016. Instance normalization: The missing ingredient for fast stylization. arXiv preprint arXiv:1607.08022 (2016).
[27]
H. Wang, Y. Li, Y. Wang, H. Hu, and M. H. Yang. 2020. Collaborative Distillation for Ultra-Resolution Universal Style Transfer. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 1857--1866.
[28]
Wenjing Wang, Jiaying Liu, Shuai Yang, and Zongming Guo. 2019. Typography with Decor: Intelligent text style transfer. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 5889--5897.
[29]
Shuai Yang, Jiaying Liu, Zhouhui Lian, and Zongming Guo. 2017. Awesome Typography: Statistics-Based Text Effects Transfer. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2886--2895.
[30]
Shuai Yang, Jiaying Liu, Wenjing Wang, and Zongming Guo. 2019. TET-GAN: Text Effects Transfer via Stylization and Destylization. In Thirty-Third AAAI Conference on Artificial Intelligence (AAAI). 1238--1245.
[31]
Wenhan Yang, Robby T Tan, Jiashi Feng, Jiaying Liu, Zongming Guo, and Shuicheng Yan. 2017. Deep joint rain detection and removal from a single image. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 1357--1366.
[32]
Richard Zhang, Phillip Isola, and Alexei A. Efros. 2016. Colorful image colorization. In European Conference on Computer Vision (ECCV). Springer, 649--666.
[33]
Rui Zhang, Mingkun Yang, Xiang Bai, Baoguang Shi, and Minghui Liao. 2019. ICDAR 2019 Robust Reading Challenge on Reading Chinese Text on Signboard. In 2019 International Conference on Document Analysis and Recognition (ICDAR).
[34]
Richard Zhang, Jun-Yan Zhu, Phillip Isola, Xinyang Geng, Angela S. Lin, Tianhe Yu, and Alexei A. Efros. 2017. Real-Time User-Guided Image Colorization with Learned Deep Priors. ACM Transactions on Graphics 36, 4, Article 119 (July 2017), 11 pages.
[35]
Y. Zhang, C. Fang, Y. Wang, Z. Wang, Z. Lin, Y. Fu, and J. Yang. 2019. Multimodal Style Transfer via Graph Cuts. In IEEE/CVF International Conference on Computer Vision (ICCV). 5942--5950.
[36]
Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. 2017. Unpaired image-to-image translation using cycle-consistent adversarial networks. In IEEE International Conference on Computer Vision (ICCV). 2223--2232.
Index Terms
- Destylization of text with decorative elements
Recommendations
Detecting Text Areas and Decorative Elements in Ancient Manuscripts
ICFHR '10: Proceedings of the 2010 12th International Conference on Frontiers in Handwriting RecognitionAn approach for the detection of decorative elements – such as initials and headlines – and text regions, focused on ancient manuscripts, is presented. Due to their age, ancient manuscripts suffer from degradation and staining as well as ink is faded-...
Font Generation and Keypoint Ranking for Stroke Order of Chinese Characters by Deep Neural Networks
AbstractDetermining the stroke order of a Chinese character image is challenging, because there is no explicit representation for image to sequence learning. This paper investigates the approach in Chinese character generation given just a few image ...
Decorative Character Recognition by Graph Matching
A practical optical character reader is required to deal with not only common fonts but also complex designed fonts. However, recognizing various kinds of decorative character images is still a challenging problem in the field of document image ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
March 2021
512 pages
ISBN:9781450383080
DOI:10.1145/3444685
- General Chairs:
- Tat-Seng Chua,
- Jingdong Wang,
- Qi Tian,
- Program Chairs:
- Cathal Gurrin,
- Jia Jia,
- Hanwang Zhang,
- Qianru Sun
Copyright © 2021 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 03 May 2021
Check for updates
Author Tags
Qualifiers
- Research-article
Funding Sources
- National Natural Science Foundation of China
- National Key R&D Program of China
Conference
MMAsia '20
Sponsor:
Acceptance Rates
Overall Acceptance Rate 59 of 204 submissions, 29%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 309Total Downloads
- Downloads (Last 12 months)92
- Downloads (Last 6 weeks)8
Reflects downloads up to 13 Nov 2024
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in