research-article

Minimizing reproduction of software failures

Authors:

Andreas ZellerAuthors Info & Claims

ISSTA '11: Proceedings of the 2011 International Symposium on Software Testing and Analysis

Pages 221 - 231

https://doi.org/10.1145/2001420.2001447

Published: 17 July 2011 Publication History

Abstract

A program fails. What now? Taking a single failing run, we record and minimize the interaction between objects to the set of calls relevant for the failure. The result is a minimal unit test that faithfully reproduces the failure at will: "Out of these 14,628 calls, only 2 are required". In a study of 17 real-life bugs, our JINSI prototype reduced the search space to 13.7% of the dynamic slice or 0.22% of the source code, with only 1--12 calls left to examine.

References

[1]

S. Artzi, J. Dolby, F. Tip, and M. Pistoia. Directed test generation for effective fault localization. In ISSTA '10, pages 49--59. ACM, 2010.

Digital Library

[2]

S. Artzi, S. Kim, and M. D. Ernst. ReCrash: Making software failures reproducible by preserving object states. In ECOOP '08, pages 542--565, Berlin, Heidelberg, 2008. Springer-Verlag.

Digital Library

[3]

M. Burger and A. Zeller. Replaying and isolating failing multi-object interactions. In WODA '08, pages 71--77, New York, NY, USA, 2008. ACM.

Digital Library

[4]

J. Clause and A. Orso. A technique for enabling and supporting debugging of field failures. In ICSE '07, pages 261--270, Washington, DC, USA, 2007. IEEE Computer Society.

Digital Library

[5]

H. Cleve and A. Zeller. Locating causes of program failures. In ICSE '05, pages 342--351, New York, NY, USA, 2005. ACM.

Digital Library

[6]

S. Colebourne. Joda Time---Java date and time API. http://joda-time.sourceforge.net/.

[7]

B. L. Computer, B. Liblit, A. Aiken, M. Naik, and A. X. Zheng. Scalable statistical bug isolation. In PLDI '05, pages 15--26. ACM Press, 2005.

Digital Library

[8]

C. Csallner and Y. Smaragdakis. Check 'n' crash: combining static checking and testing. In ICSE '05, pages 422--431, New York, NY, USA, 2005. ACM.

Digital Library

[9]

C. Csallner and T. Xie. DSD-crasher: A hybrid analysis tool for bug finding. In ISSTA '06, pages 245--254. ACM, 2006.

Digital Library

[10]

V. Dallmeier, C. Lindig, and A. Zeller. Lightweight defect localization for Java. In ECOOP '05, number 3586 in Lecture Notes in Computer Science, pages 528--550. Springer, July 2005.

Digital Library

[11]

V. Dallmeier, A. Zeller, and B. Meyer. Generating fixes from object behavior anomalies. In ASE '09, Auckland, New Zealand, November 2009.

Digital Library

[12]

H. Do, S. Elbaum, and G. Rothermel. Supporting controlled experimentation with testing techniques: An infrastructure and its potential impact. Empirical Softw. Engg., 10(4):405--435, 2005.

Digital Library

[13]

M. Doliner. Cobertura. http://cobertura.sourceforge.net/.

[14]

A. Z. Ee, A. X. Zheng, M. I. Jordan, B. Liblit, and A. Aiken. Statistical debugging of sampled programs. In Advances in Neural Information Processing Systems 16, pages 9--11. MIT Press, 2003.

[15]

S. Elbaum, H. N. Chin, M. B. Dwyer, and J. Dokulil. Carving differential unit test cases from system test cases. In SIGSOFT '06/FSE-14, pages 253--264, New York, NY, USA, 2006. ACM.

Digital Library

[16]

S. Elbaum, H. N. Chin, M. B. Dwyer, and M. Jorde. Carving and replaying differential unit test cases from system test cases. IEEE Trans. Softw. Eng., 35(1):29--45, 2009.

Digital Library

[17]

N. Gupta, H. He, X. Zhang, and R. Gupta. Locating faulty code using failure-inducing chops. In ASE '05, pages 263--272, New York, NY, USA, 2005. ACM.

Digital Library

[18]

C. Hammacher. Java slicer. http://www.st.cs.uni-saarland.de/javaslicer/.

[19]

J. A. Jones and M. J. Harrold. Empirical evaluation of the Tarantula automatic fault-localization technique. In ASE '05, pages 273--282, New York, NY, USA, 2005. ACM.

Digital Library

[20]

J. A. Jones, M. J. Harrold, and J. Stasko. Visualization of test information to assist fault localization. In ICSE '02, pages 467--477, New York, NY, USA, 2002. ACM.

Digital Library

[21]

Y. Lei and J. H. Andrews. Minimization of randomized unit test cases. In ISSRE '05, pages 267--276, Washington, DC, USA, 2005. IEEE Computer Society.

Digital Library

[22]

A. Leitner, M. Oriol, A. Zeller, I. Ciupa, and B. Meyer. Efficient unit test case minimization. In ASE '07, pages 417--420, New York, NY, USA, 2007. ACM.

Digital Library

[23]

B. McWhirter. JAXEN --- Universal Java XPath engine. http://jaxen.codehaus.org/.

[24]

A. Orso, S. Joshi, M. Burger, and A. Zeller. Isolating relevant component interactions with JINSI. In WODA '06, pages 3--10, New York, NY, USA, 2006. ACM.

Digital Library

[25]

A. Orso and B. Kennedy. Selective Capture and Replay of Program Executions. In WODA '05, pages 29--35, St. Louis, MO, USA, may 2005.

Digital Library

[26]

D. Saff, S. Artzi, J. H. Perkins, and M. D. Ernst. Automatic test factoring for Java. In ASE '05, pages 114--123, New York, NY, USA, 2005. ACM.

Digital Library

[27]

J. Steven, P. Chandra, B. Fleck, and A. Podgurski. jRapture: A capture/replay tool for observation-based testing. In ISSTA '00, pages 158--167, New York, NY, USA, 2000. ACM.

Digital Library

[28]

F. Tip. A survey of program slicing techniques. Journal of programming languages, 3(3):121--189, 1995.

[29]

D. A. Wheeler. SLOCCount --- count source lines of code (SLOC). http://www.dwheeler.com/sloccount/.

[30]

A. Zeller. Yesterday, my program worked. Today, it does not. Why? In ESEC/FSE '99, pages 253--267, London, UK, 1999. Springer-Verlag.

Digital Library

[31]

A. Zeller. Isolating cause-effect chains from computer programs. In SIGSOFT '02/FSE-10, pages 1--10, New York, NY, USA, November 2002. ACM Press.

Digital Library

[32]

A. Zeller and R. Hildebrandt. Simplifying and isolating failure-inducing input. IEEE Trans. Softw. Eng., 28(2):183--200, 2002.

Digital Library

[33]

Z. Zhang, W. K. Chan, T. H. Tse, B. Jiang, and X. Wang. Capturing propagation of infected program states. In ESEC/FSE '09, pages 43--52, New York, NY, USA, 2009. ACM.

Digital Library

Cited By

Jiang MTang CZhang XZhao YDing Z(2024)DDImage: an image reduction based approach for automatically explaining black-box classifiersEmpirical Software Engineering10.1007/s10664-024-10505-029:5Online publication date: 30-Jul-2024
https://doi.org/10.1007/s10664-024-10505-0
Jiang MTang CZhang XZhao YDing Z(2023)Automated Image Reduction for Explaining Black-box Classifiers2023 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER)10.1109/SANER56733.2023.00042(367-378)Online publication date: Mar-2023
https://doi.org/10.1109/SANER56733.2023.00042
Wong WGao RLi YAbreu RWotawa FLi D(2023)Software Fault Localization: an Overview of Research, Techniques, and ToolsHandbook of Software Fault Localization10.1002/9781119880929.ch1(1-117)Online publication date: 21-Apr-2023
https://doi.org/10.1002/9781119880929.ch1
Show More Cited By

Recommendations

Locating causes of program failures
ICSE '05: Proceedings of the 27th international conference on Software engineering

Which is the defect that causes a software failure? By comparing the program states of a failing and a passing run, we can identify the state differences that cause the failure. However, these state differences can occur all over the program run. ...
Replaying and isolating failing multi-object interactions
WODA '08: Proceedings of the 2008 international workshop on dynamic analysis: held in conjunction with the ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA 2008)

When a program fails, there are typically multiple objects that contribute to the failure. Our JINSI tool automatically captures the failure-causing interaction between objects and isolates a sequence of calls that all are relevant for reproducing the ...
Automated atomicity-violation fixing
PLDI '11

Fixing software bugs has always been an important and time-consuming process in software development. Fixing concurrency bugs has become especially critical in the multicore era. However, fixing concurrency bugs is challenging, in part due to non-...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

ISSTA '11: Proceedings of the 2011 International Symposium on Software Testing and Analysis

July 2011

394 pages

ISBN:9781450305624

DOI:10.1145/2001420

General Chair:
Matthew Dwyer
University of Nebraska
,
Program Chair:
Frank Tip
IBM T.J. Watson Research Center

Copyright © 2011 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGSOFT: ACM Special Interest Group on Software Engineering

In-Cooperation

SIGPLAN: ACM Special Interest Group on Programming Languages

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 July 2011

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Deutsche Forschungsgemeinschaft

Conference

ISSTA '11

Sponsor:

SIGSOFT

ISSTA '11: International Symposium on Software Testing and Analysis

July 17 - 21, 2011

Ontario, Toronto, Canada

Acceptance Rates

Overall Acceptance Rate 58 of 213 submissions, 27%

Upcoming Conference

ISSTA '25

Sponsor:
sigsoft

34th ACM SIGSOFT International Symposium on Software Testing and Analysis

June 25 - 28, 2025

Trondheim , Norway

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

41
Total Citations
View Citations
515
Total Downloads

Downloads (Last 12 months)16
Downloads (Last 6 weeks)1

Reflects downloads up to 01 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Jiang MTang CZhang XZhao YDing Z(2024)DDImage: an image reduction based approach for automatically explaining black-box classifiersEmpirical Software Engineering10.1007/s10664-024-10505-029:5Online publication date: 30-Jul-2024
https://doi.org/10.1007/s10664-024-10505-0
Jiang MTang CZhang XZhao YDing Z(2023)Automated Image Reduction for Explaining Black-box Classifiers2023 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER)10.1109/SANER56733.2023.00042(367-378)Online publication date: Mar-2023
https://doi.org/10.1109/SANER56733.2023.00042
Wong WGao RLi YAbreu RWotawa FLi D(2023)Software Fault Localization: an Overview of Research, Techniques, and ToolsHandbook of Software Fault Localization10.1002/9781119880929.ch1(1-117)Online publication date: 21-Apr-2023
https://doi.org/10.1002/9781119880929.ch1
Bársony DKiss ÁMarín BSaadatmand M(2022)OpenGL API call trace reduction with the minimizing Delta debugging algorithmProceedings of the 13th International Workshop on Automating Test Case Design, Selection and Evaluation10.1145/3548659.3561308(53-56)Online publication date: 7-Nov-2022
https://dl.acm.org/doi/10.1145/3548659.3561308
Zhou XPeng XXie TSun JJi CLi WDing D(2022)Delta Debugging Microservice Systems with Parallel OptimizationIEEE Transactions on Services Computing10.1109/TSC.2019.291982315:1(16-29)Online publication date: 1-Jan-2022
https://doi.org/10.1109/TSC.2019.2919823
Soremekun EKirschner LBöhme MZeller A(2021)Locating faults with program slicing: an empirical analysisEmpirical Software Engineering10.1007/s10664-020-09931-726:3Online publication date: 1-Apr-2021
https://doi.org/10.1007/s10664-020-09931-7
Kiss A(2020)Generalizing the Split Factor of the Minimizing Delta Debugging AlgorithmIEEE Access10.1109/ACCESS.2020.30430278(219837-219846)Online publication date: 2020
https://doi.org/10.1109/ACCESS.2020.3043027
Zhou XPeng XXie TSun JLi WJi CDing DHuchard MKästner CFraser G(2018)Delta debugging microservice systemsProceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering10.1145/3238147.3240730(802-807)Online publication date: 3-Sep-2018
https://dl.acm.org/doi/10.1145/3238147.3240730
Wang JDou WGao CGao YWei J(2018)Context-Based Event Trace Reduction in Client-Side JavaScript Applications2018 IEEE 11th International Conference on Software Testing, Verification and Validation (ICST)10.1109/ICST.2018.00022(127-138)Online publication date: Apr-2018
https://doi.org/10.1109/ICST.2018.00022
Herfert SPatra JPradel MRosu GDi Penta MNguyen T(2017)Automatically reducing tree-structured test inputsProceedings of the 32nd IEEE/ACM International Conference on Automated Software Engineering10.5555/3155562.3155669(861-871)Online publication date: 30-Oct-2017
https://dl.acm.org/doi/10.5555/3155562.3155669
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten