Nothing Special   »   [go: up one dir, main page]
Skip to main content
Springer Nature Link
Log in

RoDAL: style generation in robot calligraphy with deep adversarial learning

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Generative art has drawn increased attention in recent AI applications. Traditional approaches of robot calligraphy have faced challenges in achieving style consistency, line smoothness and high-quality structural uniformity. To address the limitation of existing methods, we propose a dual generator framework based on deep adversarial networks for robotic calligraphy reproduction. The proposed model utilizes a encoder-decoder module as one generator for style learning and a robot arm as the other generator for motion learning to optimize the networks and obtain the best robot calligraphy works. Based on the enhanced datasets, multiple evaluation metrics including coverage rate, structural similarity index measure, intersection over union and Turing test are employed to perform the experimental validation. The evaluations demonstrate that the proposed method is highly effective and applicable in robot calligraphy and achieves state-of-the-art results with the average structural similarity index measure 75.91% , coverage rate 70.25%, and intersection over union 80.68%, which provides a paradigm for evaluation in the field of art.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data Availability

The data that support the findings of this study are available from the corresponding author upon reasonable request.

References

  1. Kobayashi R, Katsura S (2022) A generative model of calligraphy based on image and human motion. Precis Eng 77:340–348

    Article  Google Scholar 

  2. Yang L-J, Xu T-C, Li X-S, Wu E-H (2014) Feature-oriented writing process reproduction of chinese calligraphic artwork. In: SIGGRAPH Asia 2014 technical briefs, pp 1–4

  3. Chen X, Xu C, Yang X, Song L, Tao D (2018) Gated-gan: Adversarial gated networks for multi-collection style transfer. IEEE Trans Image Process 28(2):546–560

    Article  MathSciNet  Google Scholar 

  4. Wang W-Y, Hsu M-J, Yu L-A, Chien Y-H, Hsu C-C (2020) Deep learning-based hypothesis generation model and its application on virtual chinese calligraphy-writing robot. IEEE Access 8:87243–87251

    Article  Google Scholar 

  5. Wang X, Yang Y, Wang W, Zhou Y, Yin Y, Gong Z (2023) Generative adversarial networks based motion learning towards robotic calligraphy synthesis. CAAI transactions on intelligence technology

  6. Liang D-t, Liang D, Xing S-m, Li P, Wu X-c (2020) A robot calligraphy writing method based on style transferring algorithm and similarity evaluation. Intell Serv Robot 13:137–146

    Article  Google Scholar 

  7. Wen Q, Li S, Han B, Yuan Y (2021) Zigan: Fine-grained chinese calligraphy font generation via a few-shot style transfer approach. In: Proceedings of the 29th ACM international conference on multimedia, pp 621–629

  8. Han K, You W, Deng H, Sun L, Song J, Hu Z, Yi H (2024) Lant: finding experts for digital calligraphy character restoration. Multimed Tools Appl 1–24

  9. Dai G, Zhang Y, Wang Q, Du Q, Yu Z, Liu Z, Huang S (2023) Disentangling writer and character styles for handwriting generation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 5977–5986

  10. Jian M, Dong J, Gong M, Yu H, Nie L, Yin Y, Lam K-M (2019) Learning the traditional art of chinese calligraphy via three-dimensional reconstruction and assessment. IEEE Trans Multimed 22(4):970–979

    Article  Google Scholar 

  11. Chao F, Huang Y, Zhang X, Shang C, Yang L, Zhou C, Hu H, Lin C-M (2017) A robot calligraphy system: From simple to complex writing by human gestures. Eng Appl Artif Intell 59:1–14

    Article  Google Scholar 

  12. Chao F, Huang Y, Lin C-M, Yang L, Hu H, Zhou C (2018) Use of automatic chinese character decomposition and human gestures for chinese calligraphy robots. IEEE Trans Hum-Mach Syst 49(1):47–58

    Article  Google Scholar 

  13. Ma Z, Su J (2016) Aesthetics evaluation for robotic chinese calligraphy. IEEE Trans Cogn Dev Syst 9(1):80–90

    Article  Google Scholar 

  14. Wu R, Chao F, Zhou C, Chang X, Yang L, Shang C, Zhang Z, Shen Q (2023) Internal model control structure inspired robotic calligraphy system. IEEE Trans Ind Inf

  15. Gao X, Zhou C, Chao F, Yang L, Lin C-M, Xu T, Shang C, Shen Q (2019) A data-driven robotic chinese calligraphy system using convolutional auto-encoder and differential evolution. Knowl-Based Syst 182:104802

    Article  Google Scholar 

  16. Hsu M-J, Yeh P-C, Chien Y-H, Lu C-K, Wang W-Y, Hsu C-CJ (2023) A chinese calligraphy-writing robotic system based on image-to-action translations and a hypothesis generation net. IEEE Access 11:25801–25816

    Article  Google Scholar 

  17. Li X, Yang C, Xu S, Ou Y (2024) Image-based anti-interference robotic chinese character writing system. Robotica 1–15

  18. Guo D, Ye L, Yan G (2023) Ccd-bsmg: Composite-curve-dilation-based brush stroke model generator for robotic chinese calligraphy. IEEE Access 11:129722–129732

    Article  Google Scholar 

  19. Zhang Y, Chen L, Chen H, Chu J, Chang B, Wang Y, Sun G (2023) Visual analysis of inscriptions in the tang dynasty: a case study on the calligraphy style of wang xizhi. Vis Intell 1(1):1–15

    Article  Google Scholar 

  20. Dong-mei G, Hua-song M (2022) Survey of calligraphy robot. Control Decis 37(7):1665–1674

    Google Scholar 

  21. Zhang J (2019) A survey of digital calligraphy. Sci Sin Inf 49:143–158

    Article  Google Scholar 

  22. Li Q, Guo Z, Chao F, Chang X, Yang L, Lin C-M, Shang C, Shen Q (2022) Solving robotic trajectory sequential writing problem via learning character’s structural and sequential information. IEEE Trans Cybern

  23. Liu Y, Binti Khalid F, Wang C, Binti Mustaffa MR, Bin Azman A (2024) An end-to-end chinese font generation network with stroke semantics and deformable attention skip-connection. Expert Syst Appl 237:121407

    Article  Google Scholar 

  24. Li L, Xiao J, Shi H, Wang W, Shao J, Liu A-A, Yang Y, Chen L (2023) Label semantic knowledge distillation for unbiased scene graph generation. IEEE Trans Circ Syst Video Technol

  25. Wu J, Fan H, Li Z, Liu G-H, Lin S (2023) Information transfer in semi-supervised semantic segmentation. IEEE Trans Circ Syst Video Technol

  26. Yang Z, Chu T, Lin X, Gao E, Liu D, Yang J, Wang C (2023) Eliminating contextual prior bias for semantic image editing via dual-cycle diffusion. IEEE Trans Circ Syst Video Technol

  27. Sun Y, Xu Y (2013) A calligraphy robot–callibot: Design, analysis and applications. In: 2013 IEEE international conference on robotics and biomimetics (ROBIO), pp 185–190. IEEE

  28. Zhang X, Li Y, Zhang Z, Konno K, Hu S (2019) Intelligent chinese calligraphy beautification from handwritten characters for robotic writing. Vis Comput 35(6):1193–1205

    Article  Google Scholar 

  29. Zhang J, Bi H, Chen Y, Zhang Q, Fu Z, Li Y, Li Z (2020) Smartso: Chinese character and stroke order recognition with smartwatch. IEEE Trans Mob Comput 20(7):2490–2504

    Article  Google Scholar 

  30. Yan G, Guo D, Min H (2022) Robot calligraphy based on footprint model and brush trajectory extraction. In: International conference on cognitive systems and signal processing, pp 437–451. Springer

  31. Yang Y, Chen W, Zhou L, Zheng B, Xiao W, Huang Y, Sun Z (2021) A hybrid control framework teaching robot to write chinese characters: From image to handwriting. In: 2021 IEEE 17th International conference on automation science and engineering (CASE), pp 1161–1166. IEEE

  32. Ridge B, Ude A, Morimoto J et al (2019) Learning to write anywhere with spatial transformer image-to-motion encoder-decoder networks. In: 2019 International conference on robotics and automation (ICRA), pp 2111– 2117. IEEE

  33. Guo D, Liu Z, Li R (2023) Regraphgan: A graph generative adversarial network model for dynamic network anomaly detection. Neural Netw 166:273–285

    Article  Google Scholar 

  34. Tabata AN, Zimmer A, dos Santos Coelho L, Mariani VC (2023) Analyzing carla’s performance for 2d object detection and monocular depth estimation based on deep learning approaches. Expert Syst Appl 227:120200

  35. Song X, Qin J, Ren Q, Zheng J (2023) Igan: A collaborative filtering model based on improved generative adversarial networks for recommendation

  36. Ravaee H, Manshaei MH, Safayani M, Sartakhti JS (2024) Intelligent phenotype-detection and gene expression profile generation with generative adversarial networks. J Theor Biol 577:111636

    Article  Google Scholar 

  37. Chu M, Wang J, Fan Z, Yang M, Xu C, Ma Y, Tao Z, Wu D (2023) A multidomain generative adversarial network for hoarse-to-normal voice conversion. J Voice

  38. Yu L, Zhang W, Wang J, Yu Y (2017) Seqgan: Sequence generative adversarial nets with policy gradient. In: Proceedings of the AAAI conference on artificial intelligence, vol 31

  39. Chao F, Lv J, Zhou D, Yang L, Lin C-M, Shang C, Zhou C (2018) Generative adversarial nets in robotic chinese calligraphy. In: 2018 IEEE international conference on robotics and automation (ICRA), pp 1104– 1110. IEEE

  40. Cao J (2021) Hierarchical-based calligraphy style transfer. World Sci Res J 7(5):430–439

    Google Scholar 

Download references

Acknowledgements

This work was supported by the Science and Technology Development Fund, Macau SAR (0068/2020/AGJ, 001/2024/SKL), GDST (2019B090909005, 2020B1212030003, 2023A0505030013), Nansha District (2023ZD001), MYRG2022-00192-FST, MYRG-GRG2023-00186-FST-UMDF. Special thanks to Professor Xiaoshan Li for the valuable guidance and Dr. Peng Liu for the selfless assistance.

Author information

Authors and Affiliations

  1. Department of Computer and Information Science, University of Macau, Macau, 999078, China

    Xiaoming Wang & Zhiguo Gong

Authors
  1. Xiaoming Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhiguo Gong
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Conceptualization: Zhiguo Gong and Xiaoming Wang proposed the initial idea; Methodology: Xiaoming Wang and Zhiguo Gong contributed with ideas and discussion of critical points; Conducting experiments: Xiaoming Wang; Writing - original draft preparation: Xiaoming Wang; Writing - review and editing: Zhiguo Gong, Xiaoming Wang; Supervision: Zhiguo Gong.

Corresponding author

Correspondence to Xiaoming Wang.

Ethics declarations

Conflict of interests

The authors declare no conflict of interests.

Ethical and informed consent for data used

Informed consent was obtained from all the participants prior to the publication of this study.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, X., Gong, Z. RoDAL: style generation in robot calligraphy with deep adversarial learning. Appl Intell 54, 7913–7923 (2024). https://doi.org/10.1007/s10489-024-05597-6

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-024-05597-6

Keywords

Search

Navigation