article

Free access

A methodology for controlling the size of a test suite

Editor: W. Richards Adrion Authors:

M. Jean Harrold,

Rajiv Gupta,

Mary Lou SoffaAuthors Info & Claims

ACM Transactions on Software Engineering and Methodology (TOSEM), Volume 2, Issue 3

Pages 270 - 285

https://doi.org/10.1145/152388.152391

Published: 01 July 1993 Publication History

PDF eReader

Abstract

This paper presents a technique to select a representative set of test cases from a test suite that provides the same coverage as the entire test suite. This selection is performed by identifying, and then eliminating, the redundant and obsolete test cases in the test suite. The representative set replaces the original test suite and thus, potentially produces a smaller test suite. The representative set can also be used to identify those test cases that should be rerun to test the program after it has been changed. Our technique is independent of the testing methodology and only requires an association between a testing requirement and the test cases that satisfy the requirement. We illustrate the technique using the data flow testing methodology. The reduction that is possible with our technique is illustrated by experimental results.

References

[1]

~AHO, A. V., SETm, R., XND ULLMAN, J. D Compilers, Princzples~ Techmques, and Tool, ~Addison-Wesley, Reading~ Mass. 1986. ~ACM Transaetmns on Software Engineering and Methodology, Vol 2, No 3, July 1993

Crossref

Google Scholar

[2]

~BE~ZER, B. Software Testzng Techniques. Van Nostrand Reinhold, New York, 1990.

Crossref

Google Scholar

[3]

~FRANKL, P. G., AND WEYUKER, }~. J. An applicable family of data flow testing criteria. IEEE ~Trans. Softw. Eng. SE-14, 10 (Oct. 1988), 1483-1498.

Crossref

Google Scholar

[4]

~GAREY, M. R., AND JOHNSON, n.S. Computers and ~ntractabdity, A Guide to the Theory of ~NP-Completeness, V. Klee, Ed. Freeman, New York, 1979.

Crossref

Google Scholar

[5]

~GUPTA, R. A reconfigurable LIW architecture and its compiler. Tech. Rep. 87-3. Dept. ~Computer Science, Univ. Pittsburgh, Pittsburgh, Pa., 1987.

Google Scholar

[6]

~GUPTA, R., AND SOFFA, M.L. Compile-time techniques for improving scalar access perfor- ~mance in parallel memories. IEEE Trans. Parallel and Distruibuted Systems 2, 2 (Apr. ~1991), 138-148.

Crossref

Google Scholar

[7]

~HARROLD, M. J. An approach to incremental testing. Tech. Rep. 89-1. Ph.D dissertation. ~Dept. Computer Science, Univ. Pittsburgh, Pittsburgh, Pa., 1989.

Crossref

Google Scholar

[8]

~HARROLD, M. J., A~D KOLTE, P. Combat: A compiler-based data flow testing system. In ~Proceedings of Pacific Northwest Quality Assurance (Oct.). Lawrence and Craig, 1992, ~311 323.

Google Scholar

[9]

~HARROLD, M. J., AND SOFFA, M. L. Computation of interprocedural definition and use ~dependencies. In Proceedings of IEEE Computer Society 1990 International Conference on ~Computer Languages, (New Orleans, La., Mar.). IEEE, 1990, 297-306.

Google Scholar

[10]

~HARROLD, M. J., AND SOFFA, M.L. An incremental data flow testing tool. In Proceedings of ~the 6th International Conference on Testing Computer Software (Washington, D.C., May). ~USPDI, 1989.

Google Scholar

[11]

~HARROLD, M. J., AND SOFFA, M.L. An incremental approach to unit testing during mainte- ~nance. In Proceedings of the Conference on Software Ma~nteaance 1988 (Oct.). IEEE, New ~York, 1988, 362-367.

Google Scholar

[12]

~LASKL J. W., AND KOREL, B. A data flow oriented program testing strategy. IEEE Trans. ~Softw. Eng. SE-9, 3 (May 1983), 347-354.

Google Scholar

[13]

~LEUNC, H. K., AND WHITE, L A cost model to compare regression test strategies. In ~Proceedings of Conference on Software Maintenace 1991 (Oct.). IEEE, New York, 1991, ~201 208.

Google Scholar

[14]

~LEUN6, H. K. L., AND WHITE, L. A study of regression testing. In Proceedings of the 6th ~International Conference on Testing Computer Software (Washington, D.C., May). USPDI, ~1989.

Google Scholar

[15]

~NTAFOS, S. C. An evaluation of required element testing strategies. In Proceedings of the ~7th International Conference on Software Engineering (Mar.). IEEE, New York, 1984, ~250-256.

Crossref

Google Scholar

[16]

~OSTRAND, T. J., AND WEYUKER, E.J. Using data flow analysis for regression testing. In ~Proceedtngs of the 6th Annual Pacific Northwest Software Quality Conference (Sept.). IEEE, ~New York, 1988, 58-71.

Google Scholar

[17]

~RAPPS, S., AND WEYUKER, E.J. Selecting software test data using data flow information. ~IEEE Trans. Softw. Eng. SE-11, 4 (Apr. 1985), 367-375.

Crossref

Google Scholar

[18]

~TAHA, A. M., THEBUT, S. M., AND LIu, S.S. An approach to software fault localization and ~revalidation-based on incremental data flow analysis. In Proceedings of COMPSAC 89 ~(Sept.). IEEE, New York, 1989, 527-534.

Google Scholar

Cited By

View all

Zamli KKader MMekeng A(2024)A Population Division based Multi-Task Sine Cosine Algorithm for Test Redundancy Reduction OptimizationProceedings of the 2024 10th International Conference on Computer Technology Applications10.1145/3674558.3674571(94-102)Online publication date: 15-May-2024
https://dl.acm.org/doi/10.1145/3674558.3674571
Haas RNömmer RJuergens EApel S(2024)Optimization of Automated and Manual Software Tests in Industrial Practice: A Survey and Historical AnalysisIEEE Transactions on Software Engineering10.1109/TSE.2024.341819150:8(2005-2020)Online publication date: 1-Aug-2024
https://dl.acm.org/doi/10.1109/TSE.2024.3418191
Sebastian ANaseem HCatal C(2024)Unsupervised Machine Learning Approaches for Test Suite ReductionApplied Artificial Intelligence10.1080/08839514.2024.232233638:1Online publication date: 4-Mar-2024
https://doi.org/10.1080/08839514.2024.2322336
Show More Cited By

Index Terms

A methodology for controlling the size of a test suite
1. Software and its engineering
  1. Software creation and management
  2. Software notations and tools
    1. Software configuration management and version control systems

Recommendations

Empirically evaluating Greedy-based test suite reduction methods at different levels of test suite complexity

Test suite reduction is an important approach that decreases the cost of regression testing. A test suite reduction technique operates based on the relationship between the test cases in the regression test suite and the test requirements in the program ...
Probability Model-Based Test Suite Reduction

The paper presents a test suite reduction approach based on probability models in regression testing. First, we model the process of software testing by using the joint probability of both test cases and program faults, and propose two probability ...
Improving Fault Detection Capability by Selectively Retaining Test Cases during Test Suite Reduction

Software testing is a critical part of software development. As new test cases are generated over time due to software modifications, test suite sizes may grow significantly. Because of time and resource constraints for testing, test suite minimization ...

Reviews

Reviewer: Michael Marcotty

Typically, the many modifications that a program undergoes during its lifetime require the development of new test cases, which are usually added to the test suite used for regression testing after modification. Unless special steps are taken, the test suite grows and regression testing becomes time-consuming. This paper presents a technique that helps in the management of software test suites by identifying a representative set of test cases from a suite that provides the same coverage as the entire test suite. This selection is performed by identifying and then eliminating the redundant and obsolete test cases in the test suite. The representative set replaces the original test suite. The representative set can also be used to identify those test cases that should be rerun to test the program after it has been changed. The authors discuss their algorithm in detail and illustrate its use in data flow testing methodology. They also give experimental results that show that, even for small programs, a significant reduction in the test suite may be realized.

Access critical reviews of Computing literature here

Become a reviewer for Computing Reviews.

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Transactions on Software Engineering and Methodology

ACM Transactions on Software Engineering and Methodology Volume 2, Issue 3

July 1993

101 pages

ISSN:1049-331X

EISSN:1557-7392

DOI:10.1145/152388

Editor:
W. Richards Adrion
Univ. of Massachusetts, Amherst, MA

Issue’s Table of Contents

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 July 1993

Published in TOSEM Volume 2, Issue 3

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

522
Total Citations
View Citations
2,719
Total Downloads

Downloads (Last 12 months)253
Downloads (Last 6 weeks)43

Reflects downloads up to 27 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Zamli KKader MMekeng A(2024)A Population Division based Multi-Task Sine Cosine Algorithm for Test Redundancy Reduction OptimizationProceedings of the 2024 10th International Conference on Computer Technology Applications10.1145/3674558.3674571(94-102)Online publication date: 15-May-2024
https://dl.acm.org/doi/10.1145/3674558.3674571
Haas RNömmer RJuergens EApel S(2024)Optimization of Automated and Manual Software Tests in Industrial Practice: A Survey and Historical AnalysisIEEE Transactions on Software Engineering10.1109/TSE.2024.341819150:8(2005-2020)Online publication date: 1-Aug-2024
https://dl.acm.org/doi/10.1109/TSE.2024.3418191
Sebastian ANaseem HCatal C(2024)Unsupervised Machine Learning Approaches for Test Suite ReductionApplied Artificial Intelligence10.1080/08839514.2024.232233638:1Online publication date: 4-Mar-2024
https://doi.org/10.1080/08839514.2024.2322336
Habib AKhan SHussain SIbrahim NNisa HYousafzai A(2024)A similarity-based multi-objective test optimization technique using search algorithmSystems and Soft Computing10.1016/j.sasc.2024.2001646(200164)Online publication date: Dec-2024
https://doi.org/10.1016/j.sasc.2024.200164
Wang ZYu MFeng YZhang W(2024)Do we need high-order mutation in fault-based Boolean-specification testing?Journal of Systems and Software10.1016/j.jss.2023.111933210:COnline publication date: 1-Apr-2024
https://dl.acm.org/doi/10.1016/j.jss.2023.111933
Wu ZWang ZChen JYou HYan MWang L(2024)Stratified random sampling for neural network test input selectionInformation and Software Technology10.1016/j.infsof.2023.107331165(107331)Online publication date: Jan-2024
https://doi.org/10.1016/j.infsof.2023.107331
Alenzi AAlhumud WBryce R(2024)Element-Based Test Suite Reduction for SARSA-Generated Test SuitesITNG 2024: 21st International Conference on Information Technology-New Generations10.1007/978-3-031-56599-1_50(387-395)Online publication date: 11-Mar-2024
https://doi.org/10.1007/978-3-031-56599-1_50
Koitz‐Hristov RSterner TStracke LWotawa F(2024)On the suitability of checked coverage and genetic parameter tuning in test suite reductionJournal of Software: Evolution and Process10.1002/smr.265636:8Online publication date: 5-Aug-2024
https://dl.acm.org/doi/10.1002/smr.2656
Raju SGopalan STowfek SSukumar AKhafaga DAlkahtani HAlahmadi T(2023)Test Case Selection through Novel Methodologies for Software Application DevelopmentsSymmetry10.3390/sym1510195915:10(1959)Online publication date: 23-Oct-2023
https://doi.org/10.3390/sym15101959
Liu YNie PGuo AGligoric MLegunsen OJust RFraser G(2023)Extracting Inline Tests from Unit TestsProceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis10.1145/3597926.3598149(1458-1470)Online publication date: 12-Jul-2023
https://dl.acm.org/doi/10.1145/3597926.3598149
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Abstract

References

Cited By

Index Terms

Recommendations

Empirically evaluating Greedy-based test suite reduction methods at different levels of test suite complexity

Probability Model-Based Test Suite Reduction

Improving Fault Detection Capability by Selectively Retaining Test Cases during Test Suite Reduction

Reviews

Access critical reviews of Computing literature here

Comments

Information

Published In

Publisher

Publication History

Permissions

Check for updates

Author Tags

Qualifiers

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

View options

PDF

eReader

Login options

Full Access

Figures

Other

Share

Share this Publication link

Share on social media

Affiliations