The Impact Of Bug Localization Based on Crash Report Mining: A Developers' Perspective
Authors: 
Marcos Medeiros, Uira Kulesza, Roberta Coelho, Rodrigo Bonifacio, 
Christoph Treude, Eiji Adachi BarbosaAuthors Info & Claims
Pages 13 - 24
Abstract
Developers often use crash reports to understand the root cause of bugs. However, locating the buggy source code snippet from such information is a challenging task, mainly when the log database contains many crash reports. To mitigate this issue, recent research has proposed and evaluated approaches for grouping crash report data and using stack trace information to locate bugs. The effectiveness of such approaches has been evaluated by mainly comparing the candidate buggy code snippets with the actual changed code in bug-fix commits---which happens in the context of retrospective repository mining studies. Therefore, the existing literature still lacks discussing the use of such approaches in the daily life of a software company, which could explain the developers' perceptions on the use of these approaches. In this paper, we report our experience of using an approach for grouping crash reports and finding buggy code on a weekly basis for 18 months, within three development teams in a software company. We grouped over 750,000 crash reports, opened over 130 issues, and collected feedback from 18 developers and team leaders. Among other results, we observe that the amount of system logs related to a crash report group is not the only criteria developers use to choose a candidate bug to be analyzed. Instead, other factors were considered, such as the need to deliver customer-prioritized features and the difficulty of solving complex crash reports (e.g., architectural debts), to cite some. The approach investigated in this study correctly suggested the buggy file most of the time---the approach's precision was around 80%. In this study, the developers also shared their perspectives on the usefulness of the suspicious files and methods extracted from crash reports to fix related bugs.
References
[1]
L. An and F. Khomh. 2015. Challenges and Issues of Mining Crash Reports. In 2015 IEEE 1st International Workshop on Software Analytics (SWAN). 5--8.
[2]
Le An and Foutse Khomh. 2015. Challenges and issues of mining crash reports. In 2015 IEEE 1st International Workshop on Software Analytics (SWAN). IEEE, 5--8.
[3]
Thomas Ball, Mayur Naik, and Sriram K Rajamani. 2003. From symptom to cause: localizing errors in counterexample traces. In ACM SIGPLAN Notices, Vol. 38. ACM, 97--105.
[4]
Nicolas Bettenburg, Sascha Just, Adrian Schröter, Cathrin Weiss, Rahul Premraj, and Thomas Zimmermann. 2008. What makes a good bug report?. In Proceedings of the 16th ACM SIGSOFT International Symposium on Foundations of software engineering. 308--318.
[5]
Daniel Cukier. 2013. DevOps patterns to scale web applications using cloud services. In Proceedings of the 2013 companion publication for conference on Systems, programming, & applications: software for humanity - SPLASH '13. ACM Press, New York, New York, USA, 143--152.
[6]
Yingnong Dang, Rongxin Wu, Hongyu Zhang, Dongmei Zhang, and Peter Nobel. 2012. ReBucket: a method for clustering duplicate crash reports based on call stack similarity. In 2012 34th International Conference on Software Engineering (ICSE). IEEE, 1084--1093.
[7]
Mona Erfani Joorabchi, Mehdi Mirzaaghaei, and Ali Mesbah. 2014. Works for me! characterizing non-reproducible bug reports. In Proceedings of the 11th working conference on mining software repositories. 62--71.
[8]
Yongfeng Gu, Jifeng Xuan, Hongyu Zhang, Lanxin Zhang, Qingna Fan, Xiaoyuan Xie, and Tieyun Qian. 2019. Does the fault reside in a stack trace? Assisting crash localization by predicting crashing fault residence. Journal of Systems and Software 148 (2019), 88--104.
[9]
Darryl Jarman, Jeffrey Berry, Riley Smith, Ferdian Thung, and David Lo. 2021. Legion: Massively composing rankers for improved bug localization at adobe. IEEE Transactions on Software Engineering 48, 8 (2021), 3010--3024.
[10]
James A Jones and Mary Jean Harrold. 2005. Empirical evaluation of the tarantula automatic fault-localization technique. In Proceedings of the 20th IEEE/ACM international Conference on Automated software engineering. ACM, 273--282.
[11]
James A Jones, Mary Jean Harrold, and John Stasko. 2002. Visualization of test information to assist fault localization. In Proceedings of the 24th International Conference on Software Engineering. ICSE 2002. IEEE, 467--477.
[12]
Foutse Khomh, Brian Chan, Ying Zou, and Ahmed E Hassan. 2011. An entropy evaluation approach for triaging field crashes: A case study of mozilla firefox. In 2011 18th Working Conference on Reverse Engineering. IEEE, 261--270.
[13]
Richard Kilgore and Craig Chase. 1997. Re-execution of distributed programs to detect bugs hidden by racing messages. In Proceedings of the Thirtieth Hawaii International Conference on System Sciences, Vol. 1. IEEE, 423--432.
[14]
Dongsun Kim, Xinming Wang, Sunghun Kim, Andreas Zeller, Shing-Chi Cheung, and Sooyong Park. 2011. Which crashes should i fix first?: Predicting top crashes at an early stage to prioritize debugging efforts. IEEE Transactions on Software Engineering 37, 3 (2011), 430--447.
[15]
Kinshuman Kinshumann, Kirk Glerum, Steve Greenberg, Gabriel Aul, Vince Orgovan, Greg Nichols, David Grant, Gretchen Loihle, and Galen Hunt. 2011. Debugging in the (very) large: ten years of implementation and experience. Commun. ACM 54, 7 (2011), 111--116.
[16]
Ken Kovash. 2010. Dramatic Stability Improvements in Firefox. https://blog.mozilla.org/metrics/2010/04/08/dramatic-stability-improvements-in-firefox/
[17]
Wei Li, Qingan Li, Yunlong Ming, Weijiao Dai, Shi Ying, and Mengting Yuan. 2022. An empirical study of the effectiveness of IR-based bug localization for large-scale industrial projects. Empirical Software Engineering 27, 2 (2022), 47.
[18]
Marcos Medeiros, Uirá Kulesza, Rodrigo Bonifácio, Eiji Adachi, and Roberta Coelho. 2020. Improving Bug Localization by Mining Crash Reports: An Industrial Study. In 2020 IEEE International Conference on Software Maintenance and Evolution (ICSME). 766--775.
[19]
Syeda Nessa, Muhammad Abedin, W Eric Wong, Latifur Khan, and Yu Qi. 2008. Software fault localization using N-gram analysis. In International Conference on Wireless Algorithms, Systems, and Applications. Springer, 548--559.
[20]
Andy Podgurski, David Leon, Patrick Francis, Wes Masri, Melinda Minch, Jiayang Sun, and Bin Wang. 2003. Automated support for classifying software failure reports. In 25th International Conference on Software Engineering, 2003. Proceedings. IEEE, 465--475.
[21]
Mohammad Masudur Rahman, Foutse Khomh, and Marco Castelluccio. 2020. Why are Some Bugs Non-Reproducible?:-An Empirical Investigation using Data Fusion-. In 2020 IEEE international conference on software maintenance and evolution (ICSME). IEEE, 605--616.
[22]
Adrian Schroter, Adrian Schröter, Nicolas Bettenburg, and Rahul Premraj. 2010. Do stack traces help developers fix bugs?. In 2010 7th IEEE Working Conference on Mining Software Repositories (MSR 2010). IEEE, 118--121.
[23]
Sina Shamshiri, René Just, José Miguel Rojas, Gordon Fraser, Phil McMinn, and Andrea Arcuri. 2015. Do automatically generated unit tests find real faults? an empirical study of effectiveness and challenges (t). In 2015 30th IEEE/ACM International Conference on Automated Software Engineering (ASE). IEEE, 201--211.
[24]
Laura Thomson. 2012. Socorro: Mozilla's Crash Reporting System. https://blog.mozilla.org/webdev/2010/05/19/socorro-mozilla-crash-reports/
[25]
Shaohua Wang, Foutse Khomh, and Ying Zou. 2013. Improving bug localization using correlations in crash reports. In 2013 10th Working Conference on Mining Software Repositories (MSR). IEEE, 247--256.
[26]
Shaohua Wang, Foutse Khomh, and Ying Zou. 2016. Improving bug management using correlations in crash reports. Empirical Software Engineering 21, 2 (apr 2016), 337--367.
[27]
Chu-Pan Wong, Yingfei Xiong, Hongyu Zhang, Dan Hao, Lu Zhang, and Hong Mei. 2014. Boosting bug-report-oriented fault localization with segmentation and stack-trace analysis. In 2014 IEEE International Conference on Software Maintenance and Evolution. IEEE, 181--190.
[28]
Rongxin Wu, Ming Wen, Shing-Chi Cheung, and Hongyu Zhang. 2018. Change-Locator: locate crash-inducing changes based on crash reports. Empirical Software Engineering 23, 5 (2018), 2866--2900.
[29]
Rongxin Wu, Hongyu Zhang, Shing-Chi Cheung, and Sunghun Kim. 2014. CrashLocator: locating crashing faults based on crash stacks. In Proceedings of the 2014 International Symposium on Software Testing and Analysis - ISSTA 2014. ACM Press, New York, New York, USA, 204--214.
Index Terms
- The Impact Of Bug Localization Based on Crash Report Mining: A Developers' Perspective
Recommendations
Improving bug management using correlations in crash reports
Nowadays, many software organizations rely on automatic problem reporting tools to collect crash reports directly from users' environments. These crash reports are later grouped together into crash types. Usually, developers prioritize crash types based ...
Bug localization via searching crowd-contributed code
Internetware '14: Proceedings of the 6th Asia-Pacific Symposium on InternetwareBug localization, i.e., locating bugs in code snippets, is a frequent task in software development. Although static bug-finding tools are available to reduce manual effort in bug localization, these tools typically detect bugs with known project-...
Considering dependencies between bug reports to improve bugs triage
AbstractSoftware development teams need to deal with several open reports of critical bugs to be addressed urgently and simultaneously. The management of these bugs is a complex problem due to the limited resources and the deadlines-pressure. Most of the ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
April 2024
480 pages
ISBN:9798400705014
DOI:10.1145/3639477
- Co-chairs:
- Ana Paiva,
- Rui Abreu,
- Maurício Aniche,
- Nachiappan Nagappan,
- Program Co-chairs:
- Abhik Roychoudhury,
- Margaret Storey
Copyright © 2024 Copyright is held by the owner/author(s). Publication rights licensed to ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
In-Cooperation
- Faculty of Engineering of University of Porto
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 31 May 2024
Check for updates
Author Tags
Qualifiers
- Research-article
Funding Sources
- CNPq
- CAPES
- FACEPE
- PRONEX
Conference
ICSE-SEIP '24
Sponsor:
ICSE-SEIP '24: 46th International Conference on Software Engineering: Software Engineering in Practice
April 14 - 20, 2024
Lisbon, Portugal
Upcoming Conference
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 35Total Downloads
- Downloads (Last 12 months)35
- Downloads (Last 6 weeks)8
Reflects downloads up to 14 Nov 2024
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in