Abstract
Code summary (CS) produces natural language descriptions based on code snippets, while code generation (CG) produces code snippets based on natural language. Since both tasks are intended to model the relationship between natural language and code snippets, recent research has combined these tasks to improve their performance. The existing approach either relies on LSTM for dual training, which makes it impossible to address the issue of long-distance dependency or has an imbalance in the performance between code generation and code summary. In this paper, an end-to-end model based on Transformer is proposed to handle these problems. We propose two new regularization terms to not only constrain the duality of the two models by explicitly utilizing the probability correlation between CS and CG but also promote alignment between CS and CG models. Based on this, we propose a dual-learning algorithm for CS and CG. Experiments on real Java and Python datasets demonstrated that our model significantly improved the results of CS and CG tasks, surpassing the performance of existing models.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Barbella, M., Tortora, G.: Rouge metric evaluation for text summarization techniques. Available at SSRN 4120317 (2022)
Brown, T., et al.: Language models are few-shot learners. In: Advances in Neural Information Processing Systems, vol. 33, pp. 1877–1901 (2020)
Caprio, M., Sale, Y., Hüllermeier, E., Lee, I.: A novel Bayes’ theorem for upper probabilities. In: Cuzzolin, F., Sultana, M. (eds.) Epi UAI 2023. LNCS, vol. 14523, pp. 1–12. Springer, Cham (2023). https://doi.org/10.1007/978-3-031-57963-9_1
Dong, L., Lapata, M.: Language to logical form with neural attention. arXiv preprint arXiv:1601.01280 (2016)
Elshamy, R., Abu-Elnasr, O., Elhoseny, M., Elmougy, S.: Improving the efficiency of RMSProp optimizer by utilizing Nestrove in deep learning. Sci. Rep. 13(1), 8814 (2023)
Freitag, M., Grangier, D., Caswell, I.: Bleu might be guilty but references are not innocent. arXiv abs/2004.06063 (2020). https://api.semanticscholar.org/CorpusID:215744964
Huang, S., Zhou, X., Chin, S.: Application of Seq2Seq models on code correction. Front. Artif. Intell. 4, 590215 (2021)
Iyer, S., Konstas, I., Cheung, A., Zettlemoyer, L.: Summarizing source code using a neural attention model. In: 54th Annual Meeting of the Association for Computational Linguistics 2016, pp. 2073–2083. Association for Computational Linguistics (2016)
Kaptchuk, G., Jois, T.M., Green, M., Rubin, A.D.: Meteor: cryptographically secure steganography for realistic distributions. In: Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security, pp. 1529–1548 (2021)
Li, B., Yan, M., Xia, X., Hu, X., Li, G., Lo, D.: Deepcommenter: a deep code comment generation tool with hybrid lexical and syntactical information. In: Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, pp. 1571–1575 (2020)
Sutter, T., Daunhawer, I., Vogt, J.: Multimodal generative learning utilizing jensen-shannon-divergence. In: Advances in Neural Information Processing Systems, vol. 33, pp. 6100–6110 (2020)
Wang, J., Hao, S., Shan, J., Song, X.: Visual language–let the product say what you want. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp. 23841–23843 (2024)
Wang, J., Shan, J., Santos, O.E., Bao, J.: High quality error-tolerant phrase mining on text corpus. Expert Syst. Appl. 171, 114557 (2021)
Wang, Y., Wang, W., Joty, S., Hoi, S.C.: Codet5: identifier-aware unified pre-trained encoder-decoder models for code understanding and generation. arXiv preprint arXiv:2109.00859 (2021)
Wei, B., Li, G., Xia, X., Fu, Z., Jin, Z.: Code generation as a dual task of code summarization. In: Advances in Neural Information Processing Systems, vol. 32 (2019)
Xia, Y., Qin, T., Chen, W., Bian, J., Yu, N., Liu, T.Y.: Dual supervised learning. In: International Conference on Machine Learning, pp. 3789–3798. PMLR (2017)
Zhai, H., Cao, X., Sun, P., Shen, D., Nie, T., Kou, Y.: Rule-enhanced evolutional dual graph convolutional network for temporal knowledge graph link prediction. In: Yuan, L., Yang, S., Li, R., Kanoulas, E., Zhao, X. (eds.) WISA 2023. LNCS, vol. 14094, pp. 64–75. Springer, Cham (2023). https://doi.org/10.1007/978-981-99-6222-8_6
Acknowledgments
This work is partly supported by the National Natural Science Foundation of China (Nos. 61702346 and 61702345), Basic Scientific Research Project of Liaoning Provincial Department of Education (No. JYTMS20231226), Ministry of Education industry-university cooperative education project (No. 231002108104009) and China Machinery Industry Education Association 2024 annual project on the integration of industry and education (No. ZJJX24CY008).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Wang, J., Cao, L., Shan, J., Song, X., Jiang, J. (2024). Dual Learning Model of Code Summary and Generation Based on Transformer. In: Jin, C., Yang, S., Shang, X., Wang, H., Zhang, Y. (eds) Web Information Systems and Applications. WISA 2024. Lecture Notes in Computer Science, vol 14883. Springer, Singapore. https://doi.org/10.1007/978-981-97-7707-5_4
Download citation
DOI: https://doi.org/10.1007/978-981-97-7707-5_4
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-97-7706-8
Online ISBN: 978-981-97-7707-5
eBook Packages: Computer ScienceComputer Science (R0)