research-article

An empirical study of deep transfer learning-based program repair for Kotlin projects

Authors:

Youngkyoung Kim,

Eunseok LeeAuthors Info & Claims

ESEC/FSE 2022: Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering

Pages 1441 - 1452

https://doi.org/10.1145/3540250.3558967

Published: 09 November 2022 Publication History

Abstract

Deep learning-based automated program repair (DL-APR) can automatically fix software bugs and has received significant attention in the industry because of its potential to significantly reduce software development and maintenance costs. The Samsung mobile experience (MX) team is currently switching from Java to Kotlin projects. This study reviews the application of DL-APR, which automatically fixes defects that arise during this switching process; however, the shortage of Kotlin defect-fixing datasets in Samsung MX team precludes us from fully utilizing the power of deep learning. Therefore, strategies are needed to effectively reuse the pretrained DL-APR model. This demand can be met using the Kotlin defect-fixing datasets constructed from industrial and open-source repositories, and transfer learning. This study aims to validate the performance of the pretrained DL-APR model in fixing defects in the Samsung Kotlin projects, then improve its performance by applying transfer learning. We show that transfer learning with open source and industrial Kotlin defect-fixing datasets can improve the defect-fixing performance of the existing DL-APR by 307%. Furthermore, we confirmed that the performance was improved by 532% compared with the baseline DL-APR model as a result of transferring the knowledge of an industrial (non-defect) bug-fixing dataset. We also discovered that the embedded vectors and overlapping code tokens of the code-change pairs are valuable features for selecting useful knowledge transfer instances by improving the performance of APR models by up to 696%. Our study demonstrates the possibility of applying transfer learning to practitioners who review the application of DL-APR to industrial software.

References

[1]

2022. CircleCI. https://circleci.com/ Accessed: 2022-05-19

[2]

2022. Jenkins. https://www.jenkins.io/ Accessed: 2022-05-19

[3]

2022. NLTK Tokenizer. https://www.nltk.org/api/nltk.tokenize.html Accessed: 2022-05-19

[4]

Gunnar Applelid. 2021. Evaluating template-based automatic program repair in industry.

[5]

Johannes Bader, Andrew Scott, Michael Pradel, and Satish Chandra. 2019. Getafix: Learning to fix bugs automatically. Proceedings of the ACM on Programming Languages, 3, OOPSLA (2019), 1–27. https://doi.org/10.1145/3360585

Digital Library

[6]

Berkay Berabi, Jingxuan He, Veselin Raychev, and Martin Vechev. 2021. Tfix: Learning to fix coding errors with a text-to-text transformer. In International Conference on Machine Learning. 780–791.

[7]

Antônio Carvalho, Welder Luz, Diego Marcílio, Rodrigo Bonifácio, Gustavo Pinto, and Edna Dias Canedo. 2020. C-3PR: A Bot for Fixing Static Analysis Violations via Pull Requests. 2020 IEEE 27th International Conference on Software Analysis, Evolution and Reengineering (SANER), 00 (2020), 161–171. https://doi.org/10.1109/saner48275.2020.9054842

[8]

Zimin Chen, Steve Kommrusch, Michele Tufano, Louis-Noël Pouchet, Denys Poshyvanyk, and Martin Monperrus. 2019. Sequencer: Sequence-to-sequence learning for end-to-end program repair. IEEE Transactions on Software Engineering, 47, 9 (2019), 1943–1959. https://doi.org/10.1109/TSE.2019.2940179

[9]

Ozren Dabic, Emad Aghajani, and Gabriele Bavota. 2021. Sampling Projects in GitHub for MSR Studies. In 18th IEEE/ACM International Conference on Mining Software Repositories, MSR 2021. IEEE, 560–564. https://doi.org/10.1109/MSR52588.2021.00074

[10]

Khashayar Etemadi, Nicolas Harrand, Simon Larsen, Haris Adzemovic, Henry Luong Phu, Ashutosh Verma, Fernanda Madeiral, Douglas Wikstrom, and Martin Monperrus. 2021. Sorald: Automatic Patch Suggestions for SonarQube Static Analysis Violations. arXiv preprint arXiv:2103.12033, https://doi.org/10.48550/arXiv.2103.12033

[11]

Open JS Foundation. 2022. ESLint - Pluggable JavaScript liner. https://eslint.org/ Accessed: 2022-05-11

[12]

Javier García-Munoz, Marisol García-Valls, and Julio Escribano-Barreno. 2016. Improved metrics handling in SonarQube for software quality monitoring. In Distributed Computing and Artificial Intelligence, 13th International Conference. 463–470. https://doi.org/10.1007/978-3-319-40162-1_50

[13]

github. 2022. GitHub. https://github.com/ Accessed: 2022-05-19

[14]

Rahul Gupta, Soham Pal, Aditya Kanade, and Shirish Shevade. 2017. Deepfix: Fixing common c language errors by deep learning. In Thirty-First AAAI Conference on Artificial Intelligence. https://doi.org/10.1609/aaai.v31i1.10742

[15]

Vincent J Hellendoorn and Premkumar Devanbu. 2017. Are deep neural networks the best choice for modeling source code? In Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering. 763–773. https://doi.org/10.1145/3106237.3106290

Digital Library

[16]

Tzu Ming Harry Hsu, Wei Yu Chen, Cheng-An Hou, Yao-Hung Hubert Tsai, Yi-Ren Yeh, and Yu-Chiang Frank Wang. 2015. Unsupervised domain adaptation with imbalanced cross-domain data. In Proceedings of the IEEE International Conference on Computer Vision. 4121–4129. https://doi.org/10.1109/ICCV.2015.469

Digital Library

[17]

Anil K Jain, M Narasimha Murty, and Patrick J Flynn. 1999. Data clustering: a review. ACM computing surveys (CSUR), 31, 3 (1999), 264–323. https://doi.org/10.1145/331499.331504

Digital Library

[18]

JetBrain. 2022. Kotlin Coding Style Guide. https://developer.android.com/kotlin/style-guide Accessed: 2022-05-19

[19]

JetBrain. 2022. Kotlin Programming Language. https://developer.android.com/kotlin/first Accessed: 2022-05-11

[20]

Nan Jiang, Thibaud Lutellier, and Lin Tan. 2021. CURE: Code-aware neural machine translation for automatic program repair. In 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE). 1161–1173. https://doi.org/10.1109/ICSE43902.2021.00107

Digital Library

[21]

Heechul Jung, Jeongwoo Ju, Minju Jung, and Junmo Kim. 2016. Less-forgetting learning in deep neural networks. arXiv preprint arXiv:1607.00122, https://doi.org/10.48550/arXiv.1607.00122

[22]

George Karypis. 2002. CLUTO-a clustering toolkit. MINNESOTA UNIV MINNEAPOLIS DEPT OF COMPUTER SCIENCE. https://hdl.handle.net/11299/215521

[23]

Jindae Kim and Sunghun Kim. 2019. Automatic patch generation with context-based change application. Empirical Software Engineering, 24, 6 (2019), 4071–4106. issn:1382-3256 https://doi.org/10.1007/s10664-019-09742-5

[24]

Misoo Kim, Youngkyoung Kim, and Eunseok Lee. 2021. A Novel Automatic Query Expansion with Word Embedding for IR-based Bug Localization. In 2021 IEEE 32nd International Symposium on Software Reliability Engineering (ISSRE). 276–287. https://doi.org/10.1109/ISSRE52982.2021.00038

[25]

Serkan Kirbas, Etienne Windels, Olayori McBello, Kevin Kells, Matthew Pagano, Rafal Szalanski, Vesna Nowack, Emily Rowan Winter, Steve Counsell, and David Bowes. 2021. On the introduction of automatic program repair in Bloomberg. IEEE Software, 38, 4 (2021), 43–51. https://doi.org/10.1109/MS.2021.3071086

Digital Library

[26]

Gary T Leavens, Alessandro Garcia, Corina S Păsăreanu, Junhee Lee, Seongjoon Hong, and Hakjoo Oh. 2018. MemFix: static analysis-based repair of memory deallocation errors for C. Proceedings of the 2018 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, 10, 95–106. https://doi.org/10.1145/3236024.3236079

Digital Library

[27]

Valentina Lenarduzzi, Francesco Lomio, Heikki Huttunen, and Davide Taibi. 2020. Are sonarqube rules inducing bugs? In 2020 IEEE 27th International Conference on Software Analysis, Evolution and Reengineering (SANER). 501–511. https://doi.org/10.1109/SANER48275.2020.9054821

[28]

Yi Li, Shaohua Wang, and Tien N Nguyen. 2020. Dlfix: Context-based code transformation learning for automated program repair. In Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering. 602–614. https://doi.org/10.1145/3377811.3380345

Digital Library

[29]

Kui Liu, Anil Koyuncu, Dongsun Kim, and Tegawendé F Bissyandé. 2019. TBar: Revisiting template-based automated program repair. In Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis. 31–42. https://doi.org/10.1145/3293882.3330577

Digital Library

[30]

Kui Liu, Anil Koyuncu, Dongsun Kim, and Tegawende F. Bissyandè. 2019. AVATAR: Fixing Semantic Bugs with Fix Patterns of Static Analysis Violations. 2019 IEEE 26th International Conference on Software Analysis, Evolution and Reengineering (SANER), 00 (2019), 2, 1–12. https://doi.org/10.1109/SANER.2019.8667970

[31]

Thibaud Lutellier, Hung Viet Pham, Lawrence Pang, Yitong Li, Moshi Wei, and Lin Tan. 2020. Coconut: combining context-aware neural translation models using ensemble for program repair. In Proceedings of the 29th ACM SIGSOFT international symposium on software testing and analysis. 101–114. https://doi.org/10.1145/3395363.3397369

Digital Library

[32]

Diego Marcilio, Rodrigo Bonifácio, Eduardo Monteiro, Edna Canedo, Welder Luz, and Gustavo Pinto. 2019. Are static analysis violations really fixed? a closer look at realistic usage of sonarqube. In 2019 IEEE/ACM 27th International Conference on Program Comprehension (ICPC). 209–219. https://doi.org/10.1109/ICPC.2019.00040

Digital Library

[33]

Diego Marcilio, Carlo A. Furia, Rodrigo Bonifácio, and Gustavo Pinto. 2020. SpongeBugs: Automatically generating fix suggestions in response to static code analysis warnings. Journal of Systems and Software, 168 (2020), 10, 110671. issn:0164-1212 https://doi.org/10.1016/j.jss.2020.110671

[34]

Alexandru Marginean, Johannes Bader, Satish Chandra, Mark Harman, Yue Jia, Ke Mao, Alexander Mols, and Andrew Scott. 2019. Sapfix: Automated end-to-end repair at scale. In 2019 IEEE/ACM 41st International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP). 269–278. https://doi.org/10.1109/ICSE-SEIP.2019.00039

Digital Library

[35]

Maurizio Martignano, Andraes Jung, and T Ihmann. 2015. Source code analysis of flight software using a SonarQube based code quality platform. Ada User Journal, 36, 2 (2015), 99.

[36]

Matias Martinez and Bruno Gois Mateus. 2020. Why did developers migrate Android applications from Java to Kotlin? arXiv preprint arXiv:2003.12730, https://doi.org/10.48550/arXiv.2003.12730

[37]

Ali Mesbah, Andrew Rice, Emily Johnston, Nick Glorioso, and Edward Aftandilian. 2019. Deepdelta: learning to repair compilation errors. In Proceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering. 925–936. https://doi.org/10.1145/3338906.3340455

Digital Library

[38]

Keigo Naitou, Akito Tanikado, Shinsuke Matsumoto, Yoshiki Higo, Shinji Kusumoto, Hiroyuki Kirinuki, Toshiyuki Kurabayashi, and Haruto Tanno. 2018. Toward introducing automated program repair techniques to industrial software development. In 2018 IEEE/ACM 26th International Conference on Program Comprehension (ICPC). 332–3323. https://doi.org/10.1145/3196321.3196358

Digital Library

[39]

Kunihiro Noda, Yusuke Nemoto, Keisuke Hotta, Hideo Tanida, and Shinji Kikuchi. 2020. Experience report: How effective is automated program repair for industrial software? In 2020 IEEE 27th International Conference on Software Analysis, Evolution and Reengineering (SANER). 612–616. https://doi.org/10.1109/SANER48275.2020.9054829

[40]

Andrei V Novikov. 2019. PyClustering: Data mining library. Journal of Open Source Software, 4, 36 (2019), 1230. https://doi.org/10.21105/joss.01230

[41]

Dan Pelleg and Andrew W Moore. 2000. X-means: Extending k-means with efficient estimation of the number of clusters. In Icml. 1, 727–734.

Digital Library

[42]

PMD Open Source Project. 2022. PMD. https://pmd.github.io/ Accessed: 2022-05-11

[43]

Payam Refaeilzadeh, Lei Tang, and Huan Liu. 2009. Cross-validation. Encyclopedia of database systems, 5 (2009), 532–538.

[44]

Gregg Rothermel, Doo-Hwan Bae, Hiroaki Yoshida, Rohan Bavishi, Keisuke Hotta, Yusuke Nemoto, Mukul R Prasad, and Shinji Kikuchi. 2020. Phoenix: a tool for automated data-driven synthesis of repairs for static analysis violations. Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering: Companion Proceedings, 53–56. https://doi.org/10.1145/3377812.3382150

Digital Library

[45]

SonarSource SA. 2022. SonarQube. https://www.sonarqube.org/ Accessed: 2022-05-11

[46]

Nyyti Saarimaki, Maria Teresa Baldassarre, Valentina Lenarduzzi, and Simone Romano. 2019. On the accuracy of sonarqube technical debt remediation time. In 2019 45th Euromicro Conference on Software Engineering and Advanced Applications (SEAA). 317–324. https://doi.org/10.1109/SEAA.2019.00055

[47]

SpotBugs. 2022. SpotBugs. https://spotbugs.github.io/ Accessed: 2022-05-11

[48]

Aparna Growth Strategist. 2019. Kotlin vs Java: Know which is the best programming language for Android App Development in 2021? https://www.mobileappdaily.com/kotlin-vs-java

[49]

Chuanqi Tan, Fuchun Sun, Tao Kong, Wenchang Zhang, Chao Yang, and Chunfang Liu. 2018. A survey on deep transfer learning. In International conference on artificial neural networks. 270–279. https://doi.org/10.1007/978-3-030-01424-7_27

[50]

Haoye Tian, Kui Liu, Abdoul Kader Kaboré, Anil Koyuncu, Li Li, Jacques Klein, and Tegawendé F Bissyandé. 2020. Evaluating representation learning of code changes for predicting patch correctness in program repair. In 2020 35th IEEE/ACM International Conference on Automated Software Engineering (ASE). 981–992. https://doi.org/10.1145/3324884.3416532

Digital Library

[51]

Shangwen Wang, Ming Wen, Bo Lin, Hongjun Wu, Yihao Qin, Deqing Zou, Xiaoguang Mao, and Hai Jin. 2020. Automated patch correctness assessment: How far are we? In Proceedings of the 35th IEEE/ACM International Conference on Automated Software Engineering. 968–980. https://doi.org/10.1145/3324884.3416590

Digital Library

[52]

Ming Wen, Junjie Chen, Rongxin Wu, Dan Hao, and Shing-Chi Cheung. 2018. Context-Aware Patch Generation for Better Automated Program Repair. ICSE, 5, https://doi.org/10.1145/3180155.3180233

Digital Library

[53]

Deheng Yang, Kui Liu, Dongsun Kim, Anil Koyuncu, Kisub Kim, Haoye Tian, Yan Lei, Xiaoguang Mao, Jacques Klein, and Tegawendé F Bissyandé. 2021. Where were the repair ingredients for Defects4j bugs? Empirical Software Engineering, 26, 6 (2021), 1–33. https://doi.org/10.1007/s10664-021-10003-7

Digital Library

[54]

Michihiro Yasunaga and Percy Liang. 2020. Graph-based, self-supervised program repair from diagnostic feedback. In International Conference on Machine Learning. 10799–10808.

[55]

Qihao Zhu, Zeyu Sun, Yuan-an Xiao, Wenjie Zhang, Kang Yuan, Yingfei Xiong, and Lu Zhang. 2021. A syntax-guided edit decoder for neural program repair. In Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering. 341–353. https://doi.org/10.1145/3468264.3468544

Digital Library

[56]

Fuzhen Zhuang, Zhiyuan Qi, Keyu Duan, Dongbo Xi, Yongchun Zhu, Hengshu Zhu, Hui Xiong, and Qing He. 2020. A comprehensive survey on transfer learning. Proc. IEEE, 109, 1 (2020), 43–76. https://doi.org/10.1109/JPROC.2020.3004555

Cited By

Huang KXu ZYang SSun HLi XYan ZZhang Y(2024)Evolving Paradigms in Automated Program Repair: Taxonomy, Challenges, and OpportunitiesACM Computing Surveys10.1145/369645057:2(1-43)Online publication date: 10-Oct-2024
https://dl.acm.org/doi/10.1145/3696450
Zhang QFang CMa YSun WChen Z(2023)A Survey of Learning-based Automated Program RepairACM Transactions on Software Engineering and Methodology10.1145/363197433:2(1-69)Online publication date: 23-Dec-2023
https://dl.acm.org/doi/10.1145/3631974
Ahmed HBoshchenko AKhan NKnyajev DGarifollina DScoccia GMartinez MMalavolta I(2023)Evolution of Kotlin Apps in terms of Energy Consumption: An Exploratory Study2023 International Conference on ICT for Sustainability (ICT4S)10.1109/ICT4S58814.2023.00014(46-56)Online publication date: 5-Jun-2023
https://doi.org/10.1109/ICT4S58814.2023.00014
Show More Cited By

Index Terms

An empirical study of deep transfer learning-based program repair for Kotlin projects
1. Computing methodologies
2. Human-centered computing

Index terms have been assigned to the content through auto-classification.

Recommendations

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

ESEC/FSE 2022: Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering

November 2022

1822 pages

ISBN:9781450394130

DOI:10.1145/3540250

General Chair:
Abhik Roychoudhury
National University of Singapore, Singapore
,
Program Chairs:
Cristian Cadar
Imperial College London, UK
,
Miryung Kim
University of California at Los Angeles, USA

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 09 November 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Samsung electronics
National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT)

Conference

ESEC/FSE '22

Sponsor:

ESEC/FSE '22: 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering

November 14 - 18, 2022

Singapore, Singapore

Acceptance Rates

Overall Acceptance Rate 112 of 543 submissions, 21%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
387
Total Downloads

Downloads (Last 12 months)126
Downloads (Last 6 weeks)9

Reflects downloads up to 22 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Huang KXu ZYang SSun HLi XYan ZZhang Y(2024)Evolving Paradigms in Automated Program Repair: Taxonomy, Challenges, and OpportunitiesACM Computing Surveys10.1145/369645057:2(1-43)Online publication date: 10-Oct-2024
https://dl.acm.org/doi/10.1145/3696450
Zhang QFang CMa YSun WChen Z(2023)A Survey of Learning-based Automated Program RepairACM Transactions on Software Engineering and Methodology10.1145/363197433:2(1-69)Online publication date: 23-Dec-2023
https://dl.acm.org/doi/10.1145/3631974
Ahmed HBoshchenko AKhan NKnyajev DGarifollina DScoccia GMartinez MMalavolta I(2023)Evolution of Kotlin Apps in terms of Energy Consumption: An Exploratory Study2023 International Conference on ICT for Sustainability (ICT4S)10.1109/ICT4S58814.2023.00014(46-56)Online publication date: 5-Jun-2023
https://doi.org/10.1109/ICT4S58814.2023.00014
Heo JJeong HChoi DLee EHaiduc SSchneider J(2023)REFERENT: Transformer-Based Feedback Generation Using Assignment Information for Programming CourseProceedings of the 45th International Conference on Software Engineering: Software Engineering Education and Training10.1109/ICSE-SEET58685.2023.00035(308-313)Online publication date: 17-May-2023
https://dl.acm.org/doi/10.1109/ICSE-SEET58685.2023.00035
Heo JJeong HLee E(2022)Patch It If You Can: Increasing the Efficiency of Patch Generation Using ContextElectronics10.3390/electronics1201017912:1(179)Online publication date: 30-Dec-2022
https://doi.org/10.3390/electronics12010179

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents