research-article

Diagnosing new faults using mutants and prior faults (NIER track)

Authors:

Syed Shariyar Murtaza,

Nazim Madhavji,

Mechelle Gittens,

Zude LiAuthors Info & Claims

ICSE '11: Proceedings of the 33rd International Conference on Software Engineering

Pages 960 - 963

https://doi.org/10.1145/1985793.1985959

Published: 21 May 2011 Publication History

Abstract

Literature indicates that 20% of a program's code is responsible for 80% of the faults, and 50-90% of the field failures are rediscoveries of previous faults. Despite this, identification of faulty code can consume 30-40% time of error correction. Previous fault-discovery techniques focusing on field failures either require many pass-fail traces, discover only crashing failures, or identify faulty "files" (which are of large granularity) as origin of the source code. In our earlier work (the F007 approach), we identify faulty "functions" (which are of small granularity) in a field trace by using earlier resolved traces of the same release, which limits it to the known faulty functions. This paper overcomes this limitation by proposing a new "strategy" to identify new and old faulty functions using F007. This strategy uses failed traces of mutants (artificial faults) and failed traces of prior releases to identify faulty functions in the traces of succeeding release. Our results on two UNIX utilities (i.e., Flex and Gzip) show that faulty functions in the traces of the majority (60-85%) of failures of a new software release can be identified by reviewing only 20% of the code. If compared against prior techniques then this is a notable improvement in terms of contextual knowledge required and accuracy in the discovery of finer-grain fault origin.

References

[1]

Andrews, J., H.; Briand, L.,C.; Labiche, Y.; Namin, A., S.; "Using Mutation Analysis for Assessing and Comparing Testing Coverage Criteria." IEEE Trans. on Softw. Eng., Vol.32, No.8, Aug. 2006, pp.608--624.

Digital Library

[2]

Basili, V., R.; Condon, S., E.; El Emam, K.; Hendrick, R., B.; and Melo, W.; "Characterizing and Modeling the Cost of Rework in a Library of Reusable Software Components." Proc. 19th Intl. Conf. on Softw. Eng. (ICSE), ACM Press, Boston, USA, May 1997, pp. 282--291.

Digital Library

[3]

Brodie et al.; "Quickly Finding Known Software Problems via Automated Symptom Matching." Proc. 2nd Int'l Conf. on Autonomic Computing, IEEE CS, Seattle, USA, June 2005, pp. 101--110.

Digital Library

[4]

Chilimbi et al.; "HOLMES: Effective Statistical Debugging via Efficient Path Profiling." Proc. 31st Intl. Conf. on Softw. Eng., IEEE CS, Canada, May 2009, pp. 34--44.

Digital Library

[5]

Dallmeier, V.; Lindig, C.; Zeller, A.; "Lightweight Defect Localization for Java", ECOOP 05 - Object Oriented programming, Lecture Notes in Computer Science, Springer, Glasgow, UK, Aug. 2005, pp 528--550.

Digital Library

[6]

Do, H.; Elbaum, S., G.; and Rothermel, G.; "Supporting Controlled Experimentation with Testing Techniques: An Infrastructure and its Potential Impact." Empirical Softw. Eng., Vol. 10, Springer, Oct. 2005, pp. 405--435.

Digital Library

[7]

Etrace (Tracing Tool): http://ndevilla.free.fr/etrace/; Mar. 2008.

[8]

Gittens, M.; Kim, Y.; and Godwin, D.; "The Vital Few Versus the Trivial Many: Examining the Pareto Principle for Software." Proc. 29th Int'l Computer Softw. and Appl. Conf., IEEE CS, Edinburgh, Scotland, July 2005, pp. 179--185.

Digital Library

[9]

Haran et al.; "Techniques for Classifying Executions of Deployed Software to Support Software Engineering Tasks. IEEE Trans. on Softw. Eng., Vol. 33, No.5, May 2007, pp.287--304.

Digital Library

[10]

Lee, I.; and Iyer, R.; "Diagnosing Rediscovered Problems Using Symptoms." IEEE Trans. on Softw. Eng., Vol. 26, No. 2, Feb. 2000, pp.113--127.

Digital Library

[11]

Liu, C.; and Han, J.; "Failure Proximity: A Fault Localization-based Approach." Proc. of the 14th SIGSOFT Symp. on Foundations of Softw. Eng., ACM, Portland, USA, Nov. 2006, pp.45--56.

Digital Library

[12]

Lounis, H.; Ait-Mehedine, L.; "Machine Learning Techniques for Software Product Quality Assessment." Proc. Conf. on Quality Software (QSIC), IEEE CS, Germany, Sep. 2004, pp. 102--109.

Digital Library

[13]

Murtaza, S. S.; Gittens, M.; Li, Z., Madhavji, N. H.; "F007: Finding Rediscovered Faults from the Field using Function-level Failed Traces of Software in the Field." Proc. conf. of centre of advanced studies on collaborative research: meetings of minds (CASCON), ACM Press, Ontario, Canada, Oct. 2010, pp. 57--71.

Digital Library

[14]

Offutt, A. J.; and Untch, R. H.; "Mutation 2000: Uniting the Orthogonal," in Mutation Testing for the New Century, Wong W. E., Ed., USA: Kluwer Academic Publishers, 2001, pp. 34--44.

Digital Library

[15]

Podgurski, A.; Leon, D.; Francis, P.; Masri, W.; Minch, M.; Sun, J.; & Wang., B.; "Automated Support for Classifying Software Failure Reports". Proc. Intl. Conf. on Softw. Eng., IEEE CS, Portland, USA, May 2003, pp. 465--475.

Digital Library

[16]

Proprietary workshop on large commercial software, Sep. 2008.

[17]

Witten, I. H.; and Frank, E.; Data Mining: Practical Machine Learning Tools and Techniques, USA, Morgan Kaufmann, 2005.

Digital Library

Cited By

Yang YXia XLo DBi TGrundy JYang X(2022)Predictive Models in Software Engineering: Challenges and OpportunitiesACM Transactions on Software Engineering and Methodology10.1145/350350931:3(1-72)Online publication date: 9-Apr-2022
https://dl.acm.org/doi/10.1145/3503509
Zhu QPanichella AZaidman A(2018)A systematic literature review of how mutation testing supports quality assurance processesSoftware Testing, Verification and Reliability10.1002/stvr.167528:6Online publication date: 16-Jul-2018
https://doi.org/10.1002/stvr.1675
Martino CSarkar SGanesan RKalbarczyk ZIyer R(2017)Analysis and Diagnosis of SLA Violations in a Production SaaS CloudIEEE Transactions on Reliability10.1109/TR.2016.263503366:1(54-75)Online publication date: Mar-2017
https://doi.org/10.1109/TR.2016.2635033
Show More Cited By

Index Terms

Diagnosing new faults using mutants and prior faults (NIER track)
1. Software and its engineering
  1. Software creation and management
    1. Software development techniques
      1. Error handling and recovery
    2. Software verification and validation
      1. Process validation
        Traceability
      2. Software defect analysis
        Software testing and debugging

Recommendations

Using Dummy Bridging Faults to Define Reduced Sets of Target Faults

To address the large numbers of bridging faults in a circuit, several approaches have been proposed for the selection of subsets of bridging faults as targets for test generation. A different approach that can be viewed as a fault collapsing method ...
On undetectable faults and fault diagnosis

The presence of an undetectable fault u_i may modify the response of a detectable fault d_j to a test set used for fault diagnosis. This may impact the accuracy of fault diagnosis based on the responses of single faults. Many state-of-the-art diagnosis ...
Adaptive Bayesian Diagnosis of Intermittent Faults

With increasing transient error rates, distinguishing intermittent and transient faults is especially challenging. In addition to particle strikes relatively high transient error rates are observed in architectures for opportunistic computing and in ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

ICSE '11: Proceedings of the 33rd International Conference on Software Engineering

May 2011

1258 pages

ISBN:9781450304450

DOI:10.1145/1985793

General Chair:
Richard N. Taylor
UC Irvine, USA
,
Program Chairs:
Harald Gall
University of Zurich, Switzerland
,
Nenad Medvidović
University of Southern California, USA

Copyright © 2011 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 21 May 2011

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

ICSE11

Sponsor:

SIGSOFT

ICSE11: International Conference on Software Engineering

May 21 - 28, 2011

HI, Waikiki, Honolulu, USA

Acceptance Rates

Overall Acceptance Rate 276 of 1,856 submissions, 15%

Upcoming Conference

ICSE 2025

2025 IEEE/ACM 46th International Conference on Software Engineering

April 26 - May 3, 2025

Ottawa , ON , Canada

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
191
Total Downloads

Downloads (Last 12 months)1
Downloads (Last 6 weeks)0

Reflects downloads up to 22 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Yang YXia XLo DBi TGrundy JYang X(2022)Predictive Models in Software Engineering: Challenges and OpportunitiesACM Transactions on Software Engineering and Methodology10.1145/350350931:3(1-72)Online publication date: 9-Apr-2022
https://dl.acm.org/doi/10.1145/3503509
Zhu QPanichella AZaidman A(2018)A systematic literature review of how mutation testing supports quality assurance processesSoftware Testing, Verification and Reliability10.1002/stvr.167528:6Online publication date: 16-Jul-2018
https://doi.org/10.1002/stvr.1675
Martino CSarkar SGanesan RKalbarczyk ZIyer R(2017)Analysis and Diagnosis of SLA Violations in a Production SaaS CloudIEEE Transactions on Reliability10.1109/TR.2016.263503366:1(54-75)Online publication date: Mar-2017
https://doi.org/10.1109/TR.2016.2635033
Murtaza SHamou-Lhadj AMadhavji NGittens M(2014)An empirical study on the use of mutant traces for diagnosis of faults in deployed systemsJournal of Systems and Software10.5555/2747013.274713390:C(29-44)Online publication date: 1-Apr-2014
https://dl.acm.org/doi/10.5555/2747013.2747133
Di Martino CKalbarczyk ZIyer RGoel GSarkar SGanesan RJalote PBriand LHoek A(2014)Characterization of operational failures from a business data processing SaaS platformCompanion Proceedings of the 36th International Conference on Software Engineering10.1145/2591062.2591172(195-204)Online publication date: 31-May-2014
https://dl.acm.org/doi/10.1145/2591062.2591172
Martino CChen DGoel GGanesan RKalbarczyk ZIyer R(2014)Analysis and Diagnosis of SLA Violations in a Production SaaS CloudProceedings of the 2014 IEEE 25th International Symposium on Software Reliability Engineering10.1109/ISSRE.2014.26(178-188)Online publication date: 3-Nov-2014
https://dl.acm.org/doi/10.1109/ISSRE.2014.26

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents