Utilizing source code syntax patterns to detect bug inducing commits using machine learning models
Authors: 
Md Nadim, Banani RoyAuthors Info & Claims
Pages 775 - 807
Abstract
Detecting Bug Inducing Commit (BIC) or Just in Time (JIT) defect prediction using Machine Learning (ML) based models requires tabulated feature values extracted from the source code or historical maintenance data of a software system. Existing studies have utilized meta-data from source code repositories (we named them GitHub Statistics or GS), n-gram-based source code text processing, and developer’s information (e.g., the experience of a developer) as the feature values in ML-based bug detection models. However, these feature values do not represent the source code syntax styles or patterns that a developer might prefer over available valid alternatives provided by programming languages. This investigation proposed a method to extract features from its source code syntax patterns to represent software commits and investigate whether they are helpful in detecting bug proneness in software systems. We utilize six manually and two automatically labeled datasets from eight open-source software projects written in Java, C++, and Python programming languages. Our datasets contain 642 manually labeled and 4014 automatically labeled buggy and non-buggy commits from six and two subject systems, respectively. The subject systems contain a diverse number of revisions, and they are from various application domains. Our investigation shows the inclusion of the proposed features increases the performance of detecting buggy and non-buggy software commits using five different machine learning classification models. Our proposed features also perform better in detecting buggy commits using the Deep Belief Network generated features and classification model. This investigation also implemented a state-of-the-art tool to compare the explainability of predicted buggy commits using our proposed and traditional features and found that our proposed features provide better reasoning about buggy commit detection compared to the traditional features. The continuation of this study can lead us to enhance software effectiveness by identifying, minimizing, and fixing software bugs during its maintenance and evolution.
References
[1]
albertbup. (2017). A python implementation of deep belief networks built upon numpy and tensorflow with scikit-learn compatibility. https://github.com/albertbup/deep-belief-network
[2]
Asaduzzaman, M., Roy, C. K., & Schneider, K. A. (2011). Viscad: Flexible code clone analysis support for nicad. In: Proceedings of the 5th International Workshop on Software Clones (IWSC’11). Association for Computing Machinery, New York, NY, USA, pp 77–78.
[3]
Asaduzzaman, M., Bullock, M. C., Roy, C. K., et al. (2012). Bug introducing changes: A case study with android. In: Proceedings of the 9th IEEE Working Conference on Mining Software Repositories (MSR’12), pp 116–119.
[4]
Asaduzzaman, M., Ahasanuzzaman, M., Roy, C. K., et al. (2016). How developers use exception handling in java? In: Proceedings of the IEEE/ACM 13th Working Conference on Mining Software Repositories (MSR), pp 516–519.
[5]
Aversano, L., Cerulo, L., & DelGrosso, C. (2007). Learning from bug-introducing changes to prevent fault prone code. In: Proceedings of the 9th International Workshop on Principles of Software Evolution: In Conjunction with the 6th ESEC/FSE Joint Meeting (IWPSE’07), pp 19–26.
[6]
Bavota, G., DeCarluccio, B., DeLucia, A., et al. (2012). When does a refactoring induce bugs? an empirical study. In: Proceedings of the IEEE 12th International Working Conference on Source Code Analysis and Manipulation (SCAM’12), pp 104–113.
[7]
Bernardi, M. L., Canfora, G., Di Lucca, G. A., et al. (2012). Do developers introduce bugs when they do not communicate? the case of eclipse and mozilla. In: Proceedings of the 16th European Conference on Software Maintenance and Reengineering (CSMR’12), pp 139–148.
[8]
Borg, M., Svensson, O., Berg, K., et al. (2019). Szz unleashed: an open implementation of the szz algorithm - featuring example usage in a study of just-in-time bug prediction for the jenkins project. In: Proceedings of the 3rd ACM SIGSOFT International Workshop on Machine Learning Techniques for Software Quality Evaluation (MaLTeSQuE’19).
[9]
Brownlee, J. (2017). A gentle introduction to the bag-of-words model. https://machinelearningmastery.com/gentle-introduction-bag-words-model/. [Online; Accessed 28 Sept 2021].
[10]
Canfora, G., Ceccarelli, M., Cerulo, L., et al. (2011). How long does a bug survive? an empirical study. In: Proceedings of the 18th Working Conference on Reverse Engineering (WCRE’11), pp 191–200.
[11]
Casalnuovo, C., Lee, K., Wang, H., et al. (2019). Do people prefer “natural” code? CoRR.
[12]
Cavnar W and Trenkle J N-gram-based text categorization. Ann Arbor MI 1994 48113 2 161-175
[13]
Cordy, J. R., & Roy, C. K. (2011). The nicad clone detector. In: Proceedings of the IEEE International Conference on Program Comprehension (ICPC’11), pp 219–220.
[14]
da Costa DA, McIntosh S, Shang W, et al. A framework for evaluating the results of the szz approach for identifying bug-introducing changes IEEE Transactions on Software Engineering 2017 43 7 641-657
[15]
Davies S, Roper M, and Wood M Comparing text-based and dependence-based approaches for determining the origins of bugs Journal of Software: Evolution and Process 2014 26 1 107-139
[16]
Developers.Google. (2020a). Classification: Precision and Recall - Machine Learning Crash Course. https://developers.google.com/machine-learning/crash-course/classification/precision-and-recall. Accessed 26 Aug 2021.
[17]
Developers.Google. (2020b). Classification: ROC Curve and AUC - Machine Learning Crash Course. https://developers.google.com/machine-learning/crash-course/classification/roc-and-auc. Accessed 26 Aug 2021.
[18]
Ell, J. (2013). Identifying failure inducing developer pairs within developer networks. In: Proceedings of the 35th International Conference on Software Engineering (ICSE’13), pp 1471–1473.
[19]
Eyolfson, J., Tan, L., & Lam, P. (2011). Do time of day and developer experience affect commit bugginess? In: Proceedings of the 8th Working Conference on Mining Software Repositories (MSR’11), pp 153–162.
[20]
Fukushima, T., Kamei, Y., McIntosh, S., et al. (2014). An empirical study of just-in-time defect prediction using cross-project models. In: Proceedings of the 11th Working Conference on Mining Software Repositories (MSR’14), pp 172–181.
[21]
Goues, C. L., Pradel, M., & Roychoudhury, A. (2019). Automated program repair. Communications of the ACM, p. 56–65.
[22]
Gu, Z., Barr, E. T., Hamilton, D. J., et al. (2010). Has the bug really been fixed? In: Proceedings of the 32nd ACM/IEEE International Conference on Software Engineering (ICSE’10), pp 55–64.
[23]
Hall T, Beecham S, Bowes D, et al. A systematic literature review on fault prediction performance in software engineering IEEE Transactions on Software Engineering 2012 38 6 1276-1304
[24]
Hinton, G. E. (2007). Learning multiple layers of representation. Trends in Cognitive Sciences, 11(10), 428–434.
[25]
Hinton GE and Salakhutdinov R Reducing the dimensionality of data with neural networks Science 2006 313 5786 504-507
[26]
Hinton, G. E., Osindero, S., & Teh, Y. W. (2006). A fast learning algorithm for deep belief nets. Neural Computation, 18(7), 1527–1554.
[27]
Hoang, T., Khanh Dam, H., Kamei, Y., et al. (2019). Deepjit: An end-to-end deep learning framework for just-in-time defect prediction. In: Proceedings of the IEEE/ACM 16th International Conference on Mining Software Repositories (MSR’19), pp 34–45.
[28]
Hoang, T., Kang, H. J., Lo, D., et al. (2020). Cc2vec: Distributed representations of code changes. In: Proceedings of the IEEE/ACM 42nd International Conference on Software Engineering (ICSE’20), pp 518–529.
[29]
Jeffrey, D., Feng, M., Neelam, G., et al. (2009). Bugfix: A learning-based tool to assist developers in fixing bugs. In: Proceedings of the IEEE 17th International Conference on Program Comprehension, pp 70–79.
[30]
Jiang, J., Xiong, Y., Zhang, H, et al. (2018). Shaping program repair space with existing patches and similar code. In: Proceedings of the 27th ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA’18), p 298–309.
[31]
Jimenez, M., Maxime, C., LeTraon, Y., et al. (2018). On the impact of tokenizer and parameters on n-gram based code analysis. In: 2018 IEEE International Conference on Software Maintenance and Evolution (ICSME), pp 437–448.
[32]
Kamei, Y., Shihab, E., Adams, B., et al. (2013). A large-scale empirical study of just-in-time quality assurance. IEEE Transactions on Software Engineering, pp. 757–773.
[33]
Kamei, Y., Fukushima, T., Mcintosh, S., et al. (2016). Studying just-in-time defect prediction using cross-project models. Empirical Software Engineering, p 2072–2106.
[34]
Kim, D., Nam, J., Song, J., et al. (2013). Automatic patch generation learned from human-written patches. In: Proceedings of the 2013 International Conference on Software Engineering (ICSE ’13), p 802–811.
[35]
Kim, S., & Whitehead Jr, E. J. (2006). How long did it take to fix bugs? In: Proceedings of the International Workshop on Mining Software Repositories (MSR’06), pp 173 – 174.
[36]
Kim, S., Pan, K., Whitehead, E. E. J. (2006a). Memories of bug fixes. In: Proceedings of the 14th ACM SIGSOFT International Symposium on Foundations of Software Engineering. Association for Computing Machinery, New York, NY, USA, SIGSOFT ’06/FSE-14, p 35–45. https://doi.org/10.1145/1181775.1181781
[37]
Kim, S., Zimmermann, T., Pan, K., et al. (2006b). Automatic identification of bug-introducing changes. In: Proceedings of the 21st IEEE/ACM International Conference on Automated Software Engineering (ASE’06), pp 81–90.
[38]
Kim, S., Zimmermann, T., Whitehead Jr, E. J., et al. (2007). Predicting faults from cached history. In: Proceedings of the 29th International Conference on Software Engineering (ICSE’07), pp 489–498.
[39]
Kim S, Whitehead EJ Jr, and Zhang Y Classifying software changes: Clean or buggy? IEEE Transactions on Software Engineering 2008 34 2 181-196
[40]
Kirch, W., (ed). (2008). Pearson’s Correlation Coefficient, Springer Netherlands, Dordrecht, pp 1090–1091.
[41]
Le, Q., & Mikolov, T. (2014). Distributed representations of sentences and documents. In: Proceedings of the 31st International Conference on International Conference on Machine Learning, 32(ICML’14), II–1188–II–1196.
[42]
Li, K., Xiang, Z., Chen, T., et al. (2020a). Understanding the automated parameter optimization on transfer learning for cross-project defect prediction: An empirical study. In: Proceedings of the IEEE/ACM 42nd International Conference on Software Engineering (ICSE’20), pp 566–577.
[43]
Li, Y., Wang, S., Nguyen, T. N. (2020b). Dlfix: Context-based code transformation learning for automated program repair. In: Proceedings of the IEEE/ACM 42nd International Conference on Software Engineering (ICSE’20), pp 602–614.
[44]
Liu, K., Koyuncu, A., Kim, D., et al. (2019). Tbar: Revisiting template-based automated program repair. In: Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA’19), p 31–42.
[45]
Martinez, M., & Monperrus, M. (2015). Mining software repair models for reasoning on the search space of automated program fixing. Empirical Software Engineering, p 176–205.
[46]
Martinez, M., Weimer, W., & Monperrus, M. (2014). Do the fix ingredients already exist? an empirical inquiry into the redundancy assumptions of program repair approaches. In: Companion Proceedings of the 36th International Conference on Software Engineering, p 492–495.
[47]
Mizuno O and Hata H A metric to detect fault-prone software modules using text filtering International Journal of Reliability and Safety 2013 7 1 17-31
[48]
Nadim, M. (2020). Investigating the techniques to detect and reduce bug inducing commits during change operations in software systems. Master’s thesis, University of Saskatchewan, Saskatoon, Canada, https://harvest.usask.ca/handle/10388/13125
[49]
Nadim, M., Mondal, M., & Roy, C. K. (2020). Evaluating performance of clone detection tools in detecting cloned cochange candidates. In: Proceedings of the 14th International Workshop on Software Clones (IWSC’20), pp 15–21.
[50]
Nayrolles, M., & Hamou-Lhadj, A. (2018). Clever: Combining code metrics with clone detection for just-in-time fault prevention and resolution in large industrial projects. In: Proceedings of the IEEE/ACM 15th International Conference on Mining Software Repositories (MSR’18), pp 153–164.
[51]
Pedregosa F, Varoquaux G, Gramfort A, et al. Scikit-learn: Machine learning in Python Journal of Machine Learning Research 2011 12 2825-2830
[52]
Pei, Y., Furia, C. A., Nordio, M., et al. (2014). Automatic program repair by fixing contracts. In: Proceedings of Fundamental Approaches to Software Engineering, pp 246–260.
[53]
Pornprasit, C., & Tantithamthavorn, C. (2021). Jitline: A simpler, better, faster, finer-grained just-in-time defect prediction. In: Proceedings of the International Conference on Mining Software Repositories (MSR), p To Appear.
[54]
Pornprasit, C., Tantithamthavorn, C., Jiarpakdee, J., et al. (2021). Pyexplainer: Explaining the predictions of just-in-time defect models. In: 2021 36th IEEE/ACM International Conference on Automated Software Engineering (ASE), pp 407–418.
[55]
Rahman, M. M., & Roy, C. K. (2018). Improving ir-based bug localization with context-aware query reformulation. In: Proceedings of the 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE’18). Association for Computing Machinery, p 621–632.
[56]
Rosen, C., Grawi, B., & Shihab, E. (2015). Commit guru: Analytics and risk prediction of software commits. In: Proceedings of the 10th Joint Meeting on Foundations of Software Engineering (ESEC/FSE’15), pp 966–969.
[57]
Rosner B, Glynn RJ, and Lee MLT The wilcoxon signed rank test for paired comparisons of clustered data Biometrics 2006 62 1 185-192
[58]
Shivaji S, James Whitehead E, Akella R, et al. Reducing features to improve code change-based bug prediction IEEE Transactions on Software Engineering 2013 39 4 552-569
[59]
Śliwerski J, Zimmermann T, and Zeller A Hatari: Raising risk awareness ACM SIGSOFT Software Engineering Notes 2005 30 5 107-110
[60]
Śliwerski J, Zimmermann T, and Zeller A When do changes induce fixes? ACM SIGSOFT Software Engineering Notes 2005 30 4 1-5
[61]
Spearman Rank Correlation Coefficient. (2008). Springer New York, New York, NY, pp 502–505.
[62]
Tabassum, S., Minku, L. L., Feng, D., et al. (2020). An investigation of cross-project learning in online just-in-time software defect prediction. In: Proceedings of the IEEE/ACM 42nd International Conference on Software Engineering (ICSE’20), pp 554–565.
[63]
Tan, M., Tan, L., Dara, S., et al. (2015). Online defect prediction for imbalanced data. In: Proceedings of the 37th International Conference on Software Engineering (ICSE’15), pp 99–108.
[64]
Taunk, K., De, S., Verma, S., et al. (2019). A brief review of nearest neighbor algorithm for learning and classification. In: 2019 International Conference on Intelligent Computing and Control Systems (ICCS), pp 1255–1260.
[65]
Vieira, R., da Silva, A., Rocha, L., et al. (2019). From reports to bug-fix commits: A 10 years dataset of bug-fixing activity from 55 apache’s open source projects. In: Proceedings of the Fifteenth International Conference on Predictive Models and Data Analytics in Software Engineering. Association for Computing Machinery, New York, NY, USA, PROMISE’19, p 80–89. https://doi.org/10.1145/3345629.3345639
[66]
Virtanen P, Gommers R, Oliphant T, et al. SciPy 1.0: Fundamental Algorithms for Scientific Computing in Python Nature Methods 2020 17 261-272
[67]
Wen, M., Wu, R., & Cheung, S. C. (2016). Locus: Locating bugs from software changes. In: Proceedings of the 31st IEEE/ACM International Conference on Automated Software Engineering (ASE’16), pp 262–273.
[68]
Wen, M., Wu, R., Liu, Y., et al. (2019). Exploring and exploiting the correlations between bug-inducing and bug-fixing commits. In: Proceedings of the 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE’19), pp 326–337.
[69]
Wen, M., Liu, Y., & Cheung, S. C. (2020). Boosting automated program repair with bug-inducing commits. In: Proceedings of the IEEE/ACM 42nd International Conference on Software Engineering: New Ideas and Emerging Results (ICSE-NIER’20), pp 77–80.
[70]
Wilcoxon, F. (1945). Individual comparisons by ranking methods. Biometrics Bulletin, 1(6), 80–83. http://www.jstor.org/stable/3001968
[71]
Wu R, Wen M, Cheung SC, et al. Changelocator: locate crash-inducing changes based on crash reports Empirical Software Engineering 2018 23 5 2866-2900
[72]
Xin, Q., & Reiss, S. P. (2019). Better code search and reuse for better program repair. In: Proceedings of the 6th International Workshop on Genetic Improvement (GI ’19), p 10–17.
[73]
Yang, X., Lo, D., Xia, X., et al. (2015). Deep learning for just-in-time defect prediction. In: Proceedings of the IEEE International Conference on Software Quality, Reliability and Security (QRS’15), pp 17–26.
[74]
Yin, Z., Yuan, D., Zhou, Y., et al. (2011). How do fixes become bugs? In: Proceedings of the 19th ACM SIGSOFT Symposium and the 13th European Conference on Foundations of Software Engineering (ESEC/FSE ’11), pp 26–36.
[75]
Yue, R., Meng, N., & Wang, Q. (2017). A characterization study of repeated bug fixes. In: 2017 IEEE International Conference on Software Maintenance and Evolution (ICSME), pp 422–432.
[76]
Zeng, Z., Zhang, Y., Zhang, H., et al. (2021). Deep just-in-time defect prediction: How far are we? In: Proceedings of the 30th ACM SIGSOFT International Symposium on Software Testing and Analysis. Association for Computing Machinery, New York, NY, USA, ISSTA 2021, p 427–438.
[77]
Zhao R and Mao K Fuzzy bag-of-words model for document representation IEEE Transactions on Fuzzy Systems 2018 26 2 794-804
[78]
Zibran, M. F., & Roy, C. K. (2012). Ide-based real-time focused search for near-miss clones. In: Proceedings of the 27th Annual ACM Symposium on Applied Computing, New York, NY, USA, SAC 2012, pp 1235–1242.
Recommendations
Detect Related Bugs from Source Code Using Bug Information
COMPSAC '10: Proceedings of the 2010 IEEE 34th Annual Computer Software and Applications ConferenceOpen source projects often maintain open bug repositories during development and maintenance, and the reporters often point out straightly or implicitly the reasons why bugs occur when they submit them. The comments about a bug are very valuable for ...
Reducing the search space of bug inducing commits using failure coverage
ESEC/FSE 2021: Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software EngineeringKnowing how exactly a bug has been introduced into the code can help developers debug the bug efficiently. However, techniques currently used to retrieve Bug Inducing Commits (BICs) from the repository timeline are limited in their accuracy. Automated ...
Fonte: Finding Bug Inducing Commits from Failures
ICSE '23: Proceedings of the 45th International Conference on Software EngineeringA Bug Inducing Commit (BIC) is a commit that introduces a software bug into the codebase. Knowing the relevant BIC for a given bug can provide valuable information for debugging as well as bug triaging. However, existing BIC identification techniques ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
© The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature 2022. Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
Publisher
Kluwer Academic Publishers
United States
Publication History
Published: 31 December 2022
Accepted: 04 December 2022
Author Tags
Qualifiers
- Research-article
Funding Sources
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 19 Feb 2025