research-article

An empirical study on the usage of testability information to fault localization in software

Authors:

Alberto Gonzalez-Sanchez,

Hans-Gerhard Gross,

Arjan J. C. van GemundAuthors Info & Claims

SAC '11: Proceedings of the 2011 ACM Symposium on Applied Computing

Pages 1398 - 1403

https://doi.org/10.1145/1982185.1982489

Published: 21 March 2011 Publication History

Abstract

When failures occur during software testing, automated software fault localization helps to diagnose their root causes and identify the defective statements of a program to support debugging. Diagnosis is carried out by selecting test cases in such way that their pass or fail information will narrow down the set of fault candidates, and, eventually, pinpoint the root cause. An essential in gredient of effective and efficient fault localization is knowledge about the false negative rate of tests, which is related to the rate at which defective statements of a program will exhibit failures. In current fault localization processes, false negative rates are either ignored completely, or merely estimated a posteriori as part of the diagnosis. In this paper, we study the reduction in diagnosis effort when false negative rates are known a priori. We deduce this information from testability, following the propagation-infection-execution (PIE) approach. Experiments with real programs suggest significant improvement in the diagnosis process, both in the single and the multiple-fault cases. When compared to the next-best technique, PIE-based false negative rate information yields a fault localization effort reduction of up to 80% for systems with only one fault, and up to 60% for systems with multiple faults.

References

[1]

R. Abreu, P. Zoeteweij, and A. van Gemund. An observation-based model for fault localization. In Proc. WODA'08.

Digital Library

[2]

R. Abreu, P. Zoeteweij, and A. van Gemund. On the accuracy of spectrum-based fault localization. In Proc. TAIC PART'07.

Digital Library

[3]

R. Abreu, P. Zoeteweij, and A. J. van Gemund. A new bayesian approach to multiple intermittent fault diagnosis. In Proc. IJCAI'09.

Digital Library

[4]

R. Abreu, P. Zoeteweij, and A. J. van Gemund. Spectrum-based multiple fault localization. In Proc. ASE'09.

Digital Library

[5]

M. Bruntink and A. van Deursen. An empirical study into class testability. J. Syst. Softw., 79(9): 1219--1232, 2006.

Digital Library

[6]

J. de Kleer. Diagnosing multiple persistent and intermittent faults. In Proc. IJCAI'09.

Digital Library

[7]

R. S. Freedman. Testability of software components. IEEE TSE, 17(6): 553--564, 1991.

Digital Library

[8]

M. Hutchins, H. Foster, T. Goradia, and T. Ostrand. Experiments of the effectiveness of dataflow- and controlflow-based test adequacy criteria. In Proc. ICSE'94.

Digital Library

[9]

T. Janssen, R. Abreu, and A. J. C. van Gemund. Zoltar: A toolset for automatic fault localization. In Proc. ASE'09 - Tool Demonstrations.

Digital Library

[10]

B. Jiang, Z. Zhang, W. Chan, and T. Tse. Adaptive random test case prioritization. In Proc. ASE'09.

Digital Library

[11]

J. A. Jones, M. J. Harrold, and J. Stasko. Visualization of test information to assist fault localization. In Proc. ICSE'02.

Digital Library

[12]

B. Liblit. Cooperative debugging with five hundred million test cases. In Proc. ISSTA'08.

Digital Library

[13]

C. Liu, X. Yan, L. Fei, J. Han, and S. P. Midkiff. Sober: Statistical model-based bug localization. In Proc. ESEC/FSE-13.

Digital Library

[14]

A. J. Offutt and J. H. Hayes. A semantic model of program faults. SIGSOFT Soft. Eng. Notes, 21(3): 195--200, 1996.

Digital Library

[15]

A. J. Offutt, A. Lee, G. Rothermel, R. H. Untch, and C. Zapf. An experimental determination of sufficient mutant operators. ACM TOSEM, 5: 99--118, April 1996.

Digital Library

[16]

M. Renieris and S. P. Reiss. Fault localization with nearest neighbor queries. In Proc. ASE'03.

[17]

G. Rothermel, R. H. Untch, C. Chu, and M. J. Harrold. Prioritizing test cases for regression testing. IEEE TSE, 27(10), 2001.

Digital Library

[18]

J. M. Voas. Pie: A dynamic failure-based technique. IEEE TSE, 18(8): 717--727, 1992.

Digital Library

[19]

J. M. Voas and K. W. Miller. Semantic metrics for software testability. Journal of Systems and Software, 20(3): 207--216, 1993.

Digital Library

[20]

J. M. Voas and K. W. Miller. Software testability: The new verification. IEEE Software, 12(3): 17--28, 1995.

Digital Library

[21]

W. Wong, T. Wei, Y. Qi, and L. Zhao. A crosstab-based statistical method for effective fault localization. In Proc. ICST'08.

Digital Library

[22]

M. R. Woodward and Z. A. Al-Khanjari. Testability, fault size and the domain-to-range ratio: An eternal triangle. SIGSOFT Soft. Eng. Notes, 25(5): 168--172, 2000.

Digital Library

[23]

L. Zhao. A new approach for software testability analysis. In ICSE '06.

Digital Library

Cited By

Wang DGalster MMorales-Trujillo M(2024)A systematic mapping study of bug reproduction and localizationInformation and Software Technology10.1016/j.infsof.2023.107338165:COnline publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1016/j.infsof.2023.107338
Wong WGao RLi YAbreu RWotawa FLi D(2023)Software Fault Localization: an Overview of Research, Techniques, and ToolsHandbook of Software Fault Localization10.1002/9781119880929.ch1(1-117)Online publication date: 21-Apr-2023
https://doi.org/10.1002/9781119880929.ch1
Wong WGao RLi YAbreu RWotawa F(2016)A Survey on Software Fault LocalizationIEEE Transactions on Software Engineering10.1109/TSE.2016.252136842:8(707-740)Online publication date: 1-Aug-2016
https://dl.acm.org/doi/10.1109/TSE.2016.2521368
Show More Cited By

Recommendations

An observation-based model for fault localization
WODA '08: Proceedings of the 2008 international workshop on dynamic analysis: held in conjunction with the ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA 2008)

Automatic techniques for helping developers in finding the root causes of software failures are extremely important in the development cycle of software. In this paper we study a dynamic modeling approach to fault localization, which is based on logic ...
Fault Diagnosis on Multiple Fault Models by Using Pass/Fail Information

In general, we do not know which fault model can explain the cause of the faulty values at the primary outputs in a circuit under test before starting diagnosis. Moreover, under Built-In Self Test (BIST) environment, it is difficult to know which ...
Spectrum-Based Multiple Fault Localization
ASE '09: Proceedings of the 24th IEEE/ACM International Conference on Automated Software Engineering

Fault diagnosis approaches can generally be categorized into spectrum-based fault localization (SFL, correlating failures with abstractions of program traces), and model-based diagnosis (MBD, logic reasoning over a behavioral model). Although MBD ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

SAC '11: Proceedings of the 2011 ACM Symposium on Applied Computing

March 2011

1868 pages

ISBN:9781450301138

DOI:10.1145/1982185

Conference Chairs:
William Chu
Tunghai University, TaiChung, Taiwan
,
W. Eric Wong
University of Texas at Dallas, Richardson, Texas
,
Program Chairs:
Mathew J. Palakal
Indiana University Purdue University, Indianapolis
,
Chih-Cheng Hung
Southern Polytechnic State University, Marietta

Copyright © 2011 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGAPP: ACM Special Interest Group on Applied Computing

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 21 March 2011

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article

Conference

SAC'11

Sponsor:

SIGAPP

SAC'11: The 2011 ACM Symposium on Applied Computing

March 21 - 24, 2011

TaiChung, Taiwan

Acceptance Rates

Overall Acceptance Rate 1,650 of 6,669 submissions, 25%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
220
Total Downloads

Downloads (Last 12 months)1
Downloads (Last 6 weeks)0

Reflects downloads up to 24 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Wang DGalster MMorales-Trujillo M(2024)A systematic mapping study of bug reproduction and localizationInformation and Software Technology10.1016/j.infsof.2023.107338165:COnline publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1016/j.infsof.2023.107338
Wong WGao RLi YAbreu RWotawa FLi D(2023)Software Fault Localization: an Overview of Research, Techniques, and ToolsHandbook of Software Fault Localization10.1002/9781119880929.ch1(1-117)Online publication date: 21-Apr-2023
https://doi.org/10.1002/9781119880929.ch1
Wong WGao RLi YAbreu RWotawa F(2016)A Survey on Software Fault LocalizationIEEE Transactions on Software Engineering10.1109/TSE.2016.252136842:8(707-740)Online publication date: 1-Aug-2016
https://dl.acm.org/doi/10.1109/TSE.2016.2521368
Gupta SAbreu Rde Kleer Jvan Gemund A(2014)Automatic systems diagnosis without behavioral models2014 IEEE Aerospace Conference10.1109/AERO.2014.6836252(1-8)Online publication date: Mar-2014
https://doi.org/10.1109/AERO.2014.6836252
Gupta Svan Gemund AAbreu R(2011)Probabilistic Error Propagation Modeling in Logic CircuitsProceedings of the 2011 IEEE Fourth International Conference on Software Testing, Verification and Validation Workshops10.1109/ICSTW.2011.40(617-623)Online publication date: 21-Mar-2011
https://dl.acm.org/doi/10.1109/ICSTW.2011.40
Gonzalez-Sanchez AAbreu RGross Hvan Gemund A(2011)Prioritizing tests for fault localization through ambiguity group reductionProceedings of the 26th IEEE/ACM International Conference on Automated Software Engineering10.1109/ASE.2011.6100153(83-92)Online publication date: 6-Nov-2011
https://dl.acm.org/doi/10.1109/ASE.2011.6100153

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents