research-article

Improving automatic source code summarization via deep reinforcement learning

Authors:

Philip S. YuAuthors Info & Claims

ASE '18: Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering

Pages 397 - 407

https://doi.org/10.1145/3238147.3238206

Published: 03 September 2018 Publication History

Abstract

Code summarization provides a high level natural language description of the function performed by code, as it can benefit the software maintenance, code categorization and retrieval. To the best of our knowledge, most state-of-the-art approaches follow an encoder-decoder framework which encodes the code into a hidden space and then decode it into natural language space, suffering from two major drawbacks: a) Their encoders only consider the sequential content of code, ignoring the tree structure which is also critical for the task of code summarization; b) Their decoders are typically trained to predict the next word by maximizing the likelihood of next ground-truth word with previous ground-truth word given. However, it is expected to generate the entire sequence from scratch at test time. This discrepancy can cause an exposure bias issue, making the learnt decoder suboptimal. In this paper, we incorporate an abstract syntax tree structure as well as sequential content of code snippets into a deep reinforcement learning framework (i.e., actor-critic network). The actor network provides the confidence of predicting the next word according to current state. On the other hand, the critic network evaluates the reward value of all possible extensions of the current state and can provide global guidance for explorations. We employ an advantage reward composed of BLEU metric to train both networks. Comprehensive experiments on a real-world dataset show the effectiveness of our proposed model when compared with some state-of-the-art methods.

References

[1]

A. V Aho, R. Sethi, and J. D Ullman. 1986. Compilers, Principles, Techniques. Addison Wesley 7, 8 (1986), 9.

Digital Library

[2]

M. Allamanis, H. Peng, and C. Sutton. 2016. A convolutional attention network for extreme summarization of source code. In International Conference on Machine Learning. 2091–2100.

[3]

M. Allamanis, D. Tarlow, A. Gordon, and Y. Wei. 2015. Bimodal modelling of source code and natural language. In International Conference on Machine Learning. 2123–2132.

Digital Library

[4]

D. Bahdanau, K. Cho, and Y. Bengio. 2014. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473 (2014).

[5]

S. Banerjee and A. Lavie. 2005. METEOR: An automatic metric for MT evaluation with improved correlation with human judgments. In Proceedings of the acl workshop on intrinsic and extrinsic evaluation measures for machine translation and/or summarization, Vol. 29. 65–72.

Digital Library

[6]

A. V. M. Barone and R. Sennrich. 2017. A parallel corpus of Python functions and documentation strings for automated code documentation and code generation. arXiv preprint arXiv:1707.02275 (2017).

[7]

I. D. Baxter, A. Yahin, L. Moura, M. Sant’Anna, and L. Bier. 1998. Clone detection using abstract syntax trees. In Software Maintenance, 1998. Proceedings., International Conference on. IEEE, 368–377.

Digital Library

[8]

H. K. Dam, T. Tran, and T. Pham. 2016. A deep language model for software code. arXiv preprint arXiv:1608.02715 (2016).

[9]

S. C. B. de Souza, N. Anquetil, and K. M. de Oliveira. 2005. A study of the documentation essential to software maintenance. In Proceedings of the 23rd annual international conference on Design of communication: documenting & designing for pervasive information. ACM, 68–75.

Digital Library

[10]

J. Duchi, E. Hazan, and Y. Singer. 2011. Adaptive subgradient methods for online learning and stochastic optimization. Journal of Machine Learning Research 12, Jul (2011), 2121–2159.

Digital Library

[11]

A. Eriguchi, K. Hashimoto, and Y. Tsuruoka. 2016. Tree-to-sequence attentional neural machine translation. arXiv preprint arXiv:1603.06075 (2016).

[12]

X. Gu, H. Zhang, D. Zhang, and S. Kim. 2016. Deep API learning. In Proceedings of the 2016 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering. ACM, 631–642.

Digital Library

[13]

T. Haije, B. O. K. Intelligentie, E. Gavves, and H. Heuer. 2016. Automatic Comment Generation using a Neural Translation Model. (2016).

[14]

S. Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation 9, 8 (1997), 1735–1780.

Digital Library

[15]

S. Iyer, I. Konstas, A. Cheung, and L. Zettlemoyer. 2016. Summarizing Source Code using a Neural Attention Model. In ACL (1).

[16]

Mira Kajko-Mattsson. 2005. A survey of documentation practice within corrective maintenance. Empirical Software Engineering 10, 1 (2005), 31–55.

Digital Library

[17]

Y. Keneshloo, T. Shi, C. K Reddy, and N. Ramakrishnan. 2018. Deep Reinforcement Learning For Sequence to Sequence Models. arXiv preprint arXiv:1805.09461 (2018).

[18]

M. Kilickaya, A. Erdem, N. Ikizler-Cinbis, and E. Erdem. 2016. Re-evaluating automatic metrics for image captioning. arXiv preprint arXiv:1612.07600 (2016).

[19]

V. R. Konda and J. N. Tsitsiklis. 2000. Actor-critic algorithms. In Advances in neural information processing systems. 1008–1014.

Digital Library

[20]

J. Li, W. Monroe, A. Ritter, M. Galley, J. Gao, and D. Jurafsky. 2016. Deep reinforcement learning for dialogue generation. arXiv preprint arXiv:1606.01541 (2016).

[21]

Y. Li, D. Tarlow, M. Brockschmidt, and R. Zemel. 2015. Gated graph sequence neural networks. arXiv preprint arXiv:1511.05493 (2015).

[22]

B. P. Lientz and E. B. Swanson. 1980. Software maintenance management. (1980).

Digital Library

[23]

C. Y. Lin. 2004. Rouge: A package for automatic evaluation of summaries. Text Summarization Branches Out (2004).

[24]

W. Ling, E. Grefenstette, K. M. Hermann, T. Kočisk`y, A. Senior, F. Wang, and P. Blunsom. 2016. Latent predictor networks for code generation. arXiv preprint arXiv:1603.06744 (2016).

[25]

C. Maddison and D. Tarlow. 2014. Structured generative models of natural source code. In International Conference on Machine Learning. 649–657.

Digital Library

[26]

R. C Martin. 2009. Clean code: a handbook of agile software craftsmanship. Pearson Education.

Digital Library

[27]

A. Mnih and Y. W. Teh. 2012. A fast and simple algorithm for training neural probabilistic language models. arXiv preprint arXiv:1206.6426 (2012).

Digital Library

[28]

V. Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G. Bellemare, A. Graves, M. Riedmiller, A. K. Fidjeland, G. Ostrovski, et al. 2015. Human-level control through deep reinforcement learning. Nature 518, 7540 (2015), 529–533.

[29]

L. Mou, G. Li, L. Zhang, T. Wang, and Z. Jin. 2016. Convolutional Neural Networks over Tree Structures for Programming Language Processing. In AAAI, Vol. 2. 4.

Digital Library

[30]

D. Movshovitz-Attias and W. W. Cohen. 2013. Natural language models for predicting programming comments. (2013).

[31]

A. T. Nguyen and T. N. Nguyen. 2017. Automatic categorization with deep neural network for open-source Java projects. In Proceedings of the 39th International Conference on Software Engineering Companion. IEEE Press, 164–166.

Digital Library

[32]

L. Nie, H. Jiang, Z. Ren, Z. Sun, and X. Li. 2016. Query expansion based on crowd knowledge for code search. IEEE Transactions on Services Computing 9, 5 (2016), 771–783.

[33]

Y. Oda, H. Fudaba, G. Neubig, H. Hata, S. Sakti, T. Toda, and S. Nakamura. 2015. Learning to generate pseudo-code from source code using statistical machine translation (t). In Automated Software Engineering (ASE), 2015 30th IEEE/ACM International Conference on. IEEE, 574–584.

[34]

K. Papineni, S. Roukos, T. Ward, and W. J. Zhu. 2002. BLEU: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting on association for computational linguistics. Association for Computational Linguistics, 311–318.

Digital Library

[35]

E. Parisotto, A. Mohamed, R. Singh, L. Li, D. Zhou, and P. Kohli. 2016. Neurosymbolic program synthesis. arXiv preprint arXiv:1611.01855 (2016).

[36]

C. Piech, J. Huang, A. Nguyen, M. Phulsuksombati, M. Sahami, and L. Guibas. 2015. Learning program embeddings to propagate feedback on student code. arXiv preprint arXiv:1505.05969 (2015).

Digital Library

[37]

M. Ranzato, S. Chopra, M. Auli, and W. Zaremba. 2015. Sequence level training with recurrent neural networks. arXiv preprint arXiv:1511.06732 (2015).

[38]

Z. Ren, X. Wang, N. Zhang, X. Lv, and L. J. Li. 2017. Deep Reinforcement Learning-Based Image Captioning with Embedding Reward. In Computer Vision and Pattern Recognition (CVPR), 2017 IEEE Conference on. IEEE, 1151–1159.

[39]

R. Rosenfeld. 2000. Two decades of statistical language modeling: Where do we go from here? Proc. IEEE 88, 8 (2000), 1270–1278.

[40]

J. Schulman, P. Moritz, S. Levine, M. Jordan, and P. Abbeel. 2015. Highdimensional continuous control using generalized advantage estimation. arXiv preprint arXiv:1506.02438 (2015).

[41]

D. Silver, A. Huang, C. J. Maddison, A. Guez, L. Sifre, G. Van Den Driessche, J. Schrittwieser, I. Antonoglou, V. Panneershelvam, M. Lanctot, and S. Dieleman. 2016. Mastering the game of Go with deep neural networks and tree search. Nature 529, 7587 (2016), 484–489.

[42]

G. Sridhara, E. Hill, D. Muppaneni, L. Pollock, and K. Vijay-Shanker. 2010. Towards automatically generating summary comments for java methods. In Proceedings of the IEEE/ACM international conference on Automated software engineering. ACM, 43–52.

Digital Library

[43]

I. Sutskever, O. Vinyals, and Q. V. Le. 2014. Sequence to sequence learning with neural networks. In Advances in neural information processing systems. 3104– 3112.

Digital Library

[44]

R. S Sutton and A. G Barto. 1998. Introduction to reinforcement learning. Vol. 135. MIT press Cambridge.

Digital Library

[45]

R. S Sutton, D. A McAllester, S. P Singh, and Y. Mansour. 2000. Policy gradient methods for reinforcement learning with function approximation. In Advances in neural information processing systems. 1057–1063.

Digital Library

[46]

K. S. Tai, R. Socher, and C. D. Manning. 2015. Improved semantic representations from tree-structured long short-term memory networks. arXiv preprint arXiv:1503.00075 (2015).

[47]

R. Vedantam, C. Lawrence Zitnick, and D. Parikh. 2015. Cider: Consensus-based image description evaluation. In Proceedings of the IEEE conference on computer vision and pattern recognition. 4566–4575.

[48]

C. J. Watkins and P. Dayan. 1992. Q-learning. Machine learning 8, 3-4 (1992), 279–292.

Digital Library

[49]

H. H. Wei and M. Li. 2017. Supervised Deep Features for Software Functional Clone Detection by Exploiting Lexical and Syntactical Information in Source Code. (2017).

[50]

R. J Williams. 1992. Simple statistical gradient-following algorithms for connectionist reinforcement learning. In Reinforcement Learning. Springer, 5–32.

[51]

D. Yang, A. Hussain, and C. V. Lopes. 2016. From query to usable code: An analysis of stack overflow code snippets. In Mining Software Repositories (MSR), 2016 IEEE/ACM 13th Working Conference on. IEEE, 391–401.

Digital Library

[52]

X. Zhang and M. Lapata. 2017. Sentence Simplification with Deep Reinforcement Learning. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. 584–594.

[53]

Y. Zhu, R. Mottaghi, E. Kolve, J. J. Lim, A Gupta, L. Fei-Fei, and A. Farhadi. 2017. Target-driven visual navigation in indoor scenes using deep reinforcement learning. In Robotics and Automation (ICRA), 2017 IEEE International Conference on. IEEE, 3357–3364.

Cited By

Hatipoğlu ABilgin T(2024)DOĞAL DİL METİNLERİNDEN PROGRAMLAMA DİLİ KODU OLUŞTURMA ÇALIŞMALARI: BİR DERLEME ÇALIŞMASIİstanbul Ticaret Üniversitesi Fen Bilimleri Dergisi10.55071/ticaretfbd.1354040Online publication date: 21-Mar-2024
https://doi.org/10.55071/ticaretfbd.1354040
Shi YYin YYu MChu L(2024)CogCol: Code Graph-Based Contrastive Learning Model for Code SummarizationElectronics10.3390/electronics1310181613:10(1816)Online publication date: 8-May-2024
https://doi.org/10.3390/electronics13101816
Zhang QJin DWang YGong Y(2024)Statement-Grained Hierarchy Enhanced Code SummarizationElectronics10.3390/electronics1304076513:4(765)Online publication date: 15-Feb-2024
https://doi.org/10.3390/electronics13040765
Show More Cited By

Index Terms

Improving automatic source code summarization via deep reinforcement learning
1. Computing methodologies
  1. Artificial intelligence
    1. Natural language processing
      1. Natural language generation
2. Software and its engineering
  1. Software creation and management
    1. Software post-development issues
      1. Documentation

Recommendations

An Extractive-and-Abstractive Framework for Source Code Summarization
(Source) Code summarization aims to automatically generate summaries/comments for given code snippets in the form of natural language. Such summaries play a key role in helping developers understand and maintain source code. Existing code summarization ...
Summarizing source code with hierarchical code representation
Abstract Context
Code summarization aims to automatically generate natural language descriptions for code, and has become a rapidly expanding research area. Data-driven code summarization models based on neural networks have proliferated in recent few ...
Active Learning for Low-Resource Project-Specific Code Summarization
Knowledge Science, Engineering and Management
Abstract
Code summarization aims to condense source code into concise and efficient summaries, crucial for enhancing code comprehension and maintainability. Deep learning and transfer learning techniques have significantly advanced state-of-the-art code ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

ASE '18: Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering

September 2018

955 pages

ISBN:9781450359375

DOI:10.1145/3238147

General Chair:
Marianne Huchard
University of Montpellier, France
,
Program Chairs:
Christian Kästner
Carnegie Mellon University, USA
,
Gordon Fraser
University of Passau, Germany

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGAI: ACM Special Interest Group on Artificial Intelligence
CNRS: Centre National De La Rechercue Scientifique
SIGSOFT: ACM Special Interest Group on Software Engineering
IEEE-CS: Computer Society

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 03 September 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

ASE '18

Sponsor:

SIGAI
CNRS
SIGSOFT
IEEE-CS

ASE '18: 33rd ACM/IEEE International Conference on Automated Software Engineering

September 3 - 7, 2018

Montpellier, France

Acceptance Rates

Overall Acceptance Rate 82 of 337 submissions, 24%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

259
Total Citations
View Citations
2,266
Total Downloads

Downloads (Last 12 months)205
Downloads (Last 6 weeks)34

Reflects downloads up to 23 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Hatipoğlu ABilgin T(2024)DOĞAL DİL METİNLERİNDEN PROGRAMLAMA DİLİ KODU OLUŞTURMA ÇALIŞMALARI: BİR DERLEME ÇALIŞMASIİstanbul Ticaret Üniversitesi Fen Bilimleri Dergisi10.55071/ticaretfbd.1354040Online publication date: 21-Mar-2024
https://doi.org/10.55071/ticaretfbd.1354040
Shi YYin YYu MChu L(2024)CogCol: Code Graph-Based Contrastive Learning Model for Code SummarizationElectronics10.3390/electronics1310181613:10(1816)Online publication date: 8-May-2024
https://doi.org/10.3390/electronics13101816
Zhang QJin DWang YGong Y(2024)Statement-Grained Hierarchy Enhanced Code SummarizationElectronics10.3390/electronics1304076513:4(765)Online publication date: 15-Feb-2024
https://doi.org/10.3390/electronics13040765
Hou XZhao YLiu YYang ZWang KLi LLuo XLo DGrundy JWang H(2024)Large Language Models for Software Engineering: A Systematic Literature ReviewACM Transactions on Software Engineering and Methodology10.1145/3695988Online publication date: 20-Sep-2024
https://doi.org/10.1145/3695988
Wan YBi ZHe YZhang JZhang HSui YXu GJin HYu P(2024)Deep Learning for Code Intelligence: Survey, Benchmark and ToolkitACM Computing Surveys10.1145/366459756:12(1-41)Online publication date: 18-May-2024
https://dl.acm.org/doi/10.1145/3664597
Phan HJannesari A(2024)Leveraging Statistical Machine Translation for Code SearchProceedings of the 28th International Conference on Evaluation and Assessment in Software Engineering10.1145/3661167.3661233(191-200)Online publication date: 18-Jun-2024
https://dl.acm.org/doi/10.1145/3661167.3661233
Ding XHuang YChen XBian J(2024)Adversarial Attack and Robustness Improvement on Code SummarizationProceedings of the 28th International Conference on Evaluation and Assessment in Software Engineering10.1145/3661167.3661173(17-27)Online publication date: 18-Jun-2024
https://dl.acm.org/doi/10.1145/3661167.3661173
Ding XPeng RChen XHuang YBian JZheng Z(2024)Do Code Summarization Models Process Too Much Information? Function Signature May Be All That Is NeededACM Transactions on Software Engineering and Methodology10.1145/365215633:6(1-35)Online publication date: 27-Jun-2024
https://dl.acm.org/doi/10.1145/3652156
Huang HZhao JChristakis MPradel M(2024)Multi-modal Learning for WebAssembly Reverse EngineeringProceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis10.1145/3650212.3652141(453-465)Online publication date: 11-Sep-2024
https://dl.acm.org/doi/10.1145/3650212.3652141
Raj RCosta DSpinellis DConstantinou EBacchelli A(2024)The role of library versions in Developer-ChatGPT conversationsProceedings of the 21st International Conference on Mining Software Repositories10.1145/3643991.3645075(172-176)Online publication date: 15-Apr-2024
https://dl.acm.org/doi/10.1145/3643991.3645075
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents