Abstract
Because debugging is a notoriously expensive activity, numerous automated debugging techniques have been proposed in the literature. In the last ten years, statistical fault localization emerged as the most popular approach to automated debugging. One problem with statistical fault localization techniques is that they tend to make strong assumptions on how developers behave during debugging. These assumptions are often unrealistic, which considerably limits the practical applicability and effectiveness of these techniques. To mitigate this issue, we propose Swift, an iterative user-driven technique designed to support developers during debugging. Swift (1) leverages statistical fault localization to identify suspicious methods, (2) generates high-level queries to the developer about the correctness of specific executions of the most suspicious methods, (3) uses the feedback from the developer to improve the localization results, and (4) repeats this cycle until the fault has been localized. Our empirical evaluation of Swift, performed on 26 faults in 5 programs, produced promising results; on average, Swift required less than 10 user queries to identify the fault. Most importantly, these queries were only about input/output relationships for specific executions of the methods, which developers should be able to answer quickly and without having to look at the code. We believe that Swift is a first important step towards defining fault localization techniques that account for the presence of humans in the loop and are practically applicable.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
SAEG - Software Analysis and Experimentation Group (at Universidade de São Paulo (USP), Brazil). https://github.com/saeg/experiments/tree/master/jaguar-2015
SIR Repository. http://sir.unl.edu/portal/index.php
Abreu, R., Zoeteweij, P., van Gemund, A.J.C.: An evaluation of similarity coefficients for software fault localization. In: Proceedings of the 12th Pacific Rim International Symposium on Dependable Computing, PRDC 2006, pp. 39–46. IEEE Computer Society, Washington (2006)
Ball, T., Naik, M., Rajamani, S.K.: From symptom to cause: localizing errors in counterexample traces. In: POPL, pp. 97–105 (2003)
Bandyopadhyay, A., Ghosh, S.: Tester feedback driven fault localization. In: Proceedings of the 2012 IEEE Fifth International Conference on Software Testing, Verification and Validation, ICST 2012, pp. 41–50. IEEE Computer Society, Washington (2012)
Chandra, S., Torlak, E., Barman, S., Bodik, R.: Angelic debugging. In: ICSE, pp. 121–130 (2011)
Cleve, H., Zeller, A.: Locating causes of program failures. In: ICSE, pp. 342–351 (2005)
Demsky, B., Ernst, M.D., Guo, P.J., McCamant, S., Perkins, J.H., Rinard, M.: Inference and enforcement of data structure consistency specifications. In: ISSTA, pp. 233–244 (2006)
Dillig, I., Dillig, T., Aiken, A.: Automated error diagnosis using abductive inference. In: PLDI, pp. 181–192 (2012)
Gong, L., Lo, D., Jiang, L., Zhang, H.: Interactive fault localization leveraging simple user feedback. In: 2012 28th IEEE International Conference on Software Maintenance (ICSM), pp. 67–76, September 2012
Hao, D., Zhang, L., Xie, T., Mei, H., Sun, J.-S.: Interactive fault localization using test information. J. Comput. Sci. Technol. 24(5), 962–974 (2009)
Hao, D., Zhang, L., Zhang, L., Sun, J., Mei, H.: Vida: Visual interactive debugging. In: 2009 IEEE 31st International Conference on Software Engineering, pp. 583–586, May 2009
Jones, J.A., Bowring, J.F., Harrold, M.J.: Debugging in parallel. In: Proceedings of the 2007 International Symposium on Software Testing and Analysis, ISSTA 2007, pp. 16–26. ACM, New York (2007)
Jones, J.A., Harrold, M.J., Stasko, J.: Visualization of test information to assist fault localization. In: ICSE, pp. 467–477 (2002)
Jose, M., Majumdar, R.: Cause clue clauses: error localization using maximum satisfiability. In: PLDI, pp. 437–446 (2011)
Just, R., Jalali, D., Ernst, M.D.: Defects4J: a database of existing faults to enable controlled testing studies for Java programs. In: Proceedings of the International Symposium on Software Testing and Analysis (ISSTA), San Jose, CA, USA, pp. 437–440, July 23–25 (2014)
Ko, A.J., Myers, B.A.: Designing the whyline: a debugging interface for asking questions about program behavior. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI 2004, pp. 151–158. ACM, New York (2004)
Liblit, B., Naik, M., Zheng, A.X., Aiken, A., Jordan, M.I.: Scalable statistical bug isolation. In: PLDI, pp. 15–26 (2005)
Mangal, R., Zhang, X., Nori, A.V., Naik, M.: A user-guided approach to program analysis. In: ESEC/FSE, pp. 462–473 (2015)
Naish, L., Lee, H.J., Ramamohanarao, K.: A model for spectra-based software diagnosis. ACM Trans. Softw. Eng. Methodol. 11:20(3), 1–11:32 (2011)
Parnin, C., Orso, A.: Are automated debugging techniques actually helping programmers? In: ISSTA, pp. 199–209 (2011)
Renieris, M., Reiss, S.P.: Fault localization with nearest neighbor queries. In: ASE, pp. 30–39 (2003)
Shapiro, E.Y.: Algorithmic Program DeBugging. MIT Press, Cambridge (1983)
Stotts, P.D., Lindsey, M., Antley, A.: An informal formal method for systematic junit test case generation. In: Proceedings of the Second XP Universe and First Agile Universe Conference on Extreme Programming and Agile Methods - XP/Agile Universe 2002, pp. 131–143 (2002)
Vessey, I.: Expertise in debugging computer programs: An analysis of the content of verbal protocols. IEEE Trans. Syst. Man Cybern. B Cybern. 16(5), 621–637 (1986)
Wang, X., Cheung, S.C., Chan, W.K., Zhang, Z.: Taming coincidental correctness: coverage refinement with context patterns to improve fault localization. In: Proceedings of the 31st International Conference on Software Engineering, ICSE 2009, pp. 45–55. IEEE Computer Society , Washington (2009)
Zhang, X., Gupta, N., Gupta, R.: Locating faults through automated predicate switching. In: ICSE, pp. 272–281 (2006)
Zheng, A.X., Jordan, M.I., Liblit, B., Naik, M., Aiken, A.: Statistical debugging: simultaneous identification of multiple bugs. In: ICML, pp. 1105–1112 (2006)
Acknowledgments
Mayur Naik was engaged in early discussions about this work. Higor Amario de Souza shared the code. This work was partially supported by CNPq grants 457756/2014-4 and 203981/2014-6, by NSF grants CCF1320783 and CCF1161821, and by funding from Google, IBM Research, and Microsoft Research.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing AG
About this paper
Cite this paper
Li, X., d’Amorim, M., Orso, A. (2016). Iterative User-Driven Fault Localization. In: Bloem, R., Arbel, E. (eds) Hardware and Software: Verification and Testing. HVC 2016. Lecture Notes in Computer Science(), vol 10028. Springer, Cham. https://doi.org/10.1007/978-3-319-49052-6_6
Download citation
DOI: https://doi.org/10.1007/978-3-319-49052-6_6
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-49051-9
Online ISBN: 978-3-319-49052-6
eBook Packages: Computer ScienceComputer Science (R0)