Nothing Special   »   [go: up one dir, main page]
Skip to main content
Springer Nature Link
Log in

Combinatorial Test Set Prioritization Using Data Flow Techniques

  • Research Article - Computer Engineering and Computer Science
  • Published:
Arabian Journal for Science and Engineering Aims and scope Submit manuscript

Abstract

Software testing is an expensive and important part of the software development process. One of the effective and cost- efficient test generation techniques is combinatorial testing, which identifies interaction faults that arise due to faulty combinations of few input parameters. However, for a large-sized system, it is practically impossible to execute the complete t-way test set due to time or resources constraints. As a result, only a portion of the test set can be executed leading to possible loss of fault detection capability. Prioritizing the test set helps in improving the fault detection capability. In this paper, prioritization of combinatorial test set is proposed using data flow technique. Different methods of prioritization criterion are proposed, and their effectiveness is compared by taking two software subjects. The effectiveness is compared by measuring percentage of pairs covered, percentage of weight covered and rate of fault detection. Computational results indicate that prioritized t-way test set achieves better fault detection rate as compared to unordered t-way test set.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Myers, G.J.; Sandler, C.; Badgett, T.: The Art of Software Testing. John Wiley & Sons, Hoboken (2011)

    Google Scholar 

  2. Cohen, D.M.; Dalal, S.R.; Parelius, J.; Patton, G.C.: The combinatorial design approach to automatic test generation. IEEE Softw. 5, 83–88 (1996)

    Article  Google Scholar 

  3. Petke, J.; Cohen, M.B.; Harman, M.; Yoo, S.: Practical combinatorial interaction testing: empirical findings on efficiency and early fault detection. Softw. Eng. IEEE Trans. 41(9), 901–924 (2015)

    Article  Google Scholar 

  4. Sabharwal, S.; Aggarwal, M.: Test set generation for pairwise testing using genetic algorithms. J. Inf. Process. Syst. doi:10.3745/JIPS.04.0019

  5. Zamli, K.Z.; Klaib, M.F.; Younis, M.I.; Isa, N.A.M.; Abdullah, R.: Design and implementation of a t-way test data generation strategy with automated execution tool support. Inf. Sci. 181(9), 1741–1758 (2011)

    Article  Google Scholar 

  6. Bryce, R.C.; Sampath, S.; Pedersen, J.B.; Manchester, S.: Test suite prioritization by cost-based combinatorial interaction coverage. Int. J. Syst. Assur. Eng. Manag. 2(2), 126–134 (2011)

    Article  Google Scholar 

  7. Kuhn, D.R.; Kacker, R.N.; Lei, Y.: Practical combinatorial testing. NIST Special Publication 800–142, Gaithersburg (2010)

  8. Cohen, D.M.; Dalal, S.R.; Fredman, M.L.; Patton, G.C.: The AETG system: an approach to testing based on combinatorial design. IEEE Trans. Softw. Eng. 23(7), 437–444 (1997)

    Article  Google Scholar 

  9. Henard, C.; Papadakis, M.; Perrouin, G.; Klein, J.; Heymans, P.; Le Traon, Y.: Bypassing the combinatorial explosion: using similarity to generate and prioritize t-wise test configurations for software product lines. Softw. Eng. IEEE Trans. 40(7), 650–670 (2014)

    Article  Google Scholar 

  10. Wang, Y.; Wu, H.; Sheng, Z.: A prioritized test generation method for pair-wise testing. TELKOMNIKA Indones. J. Electr. Eng. 11(1), 136–143 (2013)

    Google Scholar 

  11. Mathur, A.P.: Foundations of Software Testing, 2nd edn. Pearson Education, Noida, India (2008)

  12. Bryce, R.C.; Colbourn, C.J.: Test prioritization for pairwise interaction coverage. ACM SIGSOFT Softw. Eng. Notes 30(4), 1–7 (2005)

    Article  Google Scholar 

  13. Bryce, R.C.; Sampath, S.; Memon, A.M.: Developing a single model and test prioritization strategies for event-driven software. IEEE Trans. Softw. Eng. 37(1), 48–64 (2011)

    Article  Google Scholar 

  14. Qu, X.; Cohen, M.B.; Rothermel, G.: Configuration-aware regression testing: an empirical study of sampling and prioritization. In: Proceedings of the 2008 International Symposium on Software Testing and Analysis 2008. ACM, pp. 75–86

  15. Henard, C.; Papadakis, M.; Harman, M.; Jia, Y.; Le Traon, Y.: Comparing white-box and black-box test prioritization. In: Proceedings of the 38th International Conference on Software Engineering 2016. ACM, pp. 523–534

  16. Qu, X.; Cohen, M.B.; Woolf, K.M.: Combinatorial interaction regression testing: a study of test case generation and prioritization. In: International Conference on Software Maintenance, 2007. ICSM 2007. IEEE 2007. IEEE, pp. 255–264

  17. Qu, X.; Cohen, M.B.: A study in prioritization for higher strength combinatorial testing. In: IEEE Sixth International Conference on Software Testing, Verification and Validation Workshops, 2013 (ICSTW), 2013 . IEEE, pp. 285–294

  18. Said, S.K.; Othman, R.R.; Zamli, K.Z.: Prioritizing interaction test suite for t-way testing. In: 5th Malaysian Conference in Software Engineering (MySEC), 2011. IEEE, pp. 292–297

  19. Sampath, S.; Bryce, R.C.; Jain, S.; Manchester, S.: A tool for combination-based prioritization and reduction of user-session-based test suites. In: 27th IEEE International Conference on 2011 Software Maintenance (ICSM), 2011 . IEEE, pp. 574–577

  20. Bryce, R.C.; Colbourn, C.J.: Prioritized interaction testing for pair-wise coverage with seeding and constraints. Inf. Softw. Technol. 48(10), 960–970 (2006)

    Article  Google Scholar 

  21. Chen, X.; Gu, Q.; Zhang, X.; Chen, D.: Building prioritized pairwise interaction test suites with ant colony optimization. In: 9th International Conference on Quality Software, 2009. QSIC’09. IEEE, pp. 347–352

  22. Ferrer, J.; Kruse, P.M.; Chicano, F.; Alba, E.: Evolutionary algorithm for prioritized pairwise test data generation. In: Proceedings of the 14th Annual Conference on Genetic and Evolutionary Computation 2012. ACM, pp. 1213-1220

  23. Sabharwal, S.; Aggarwal, M.: A novel approach for deriving interactions for combinatorial testing. Eng. Sci. Technol. Int. J. 20(1), 59–71 (2017)

  24. NetBeans: TikiOne JaCoCoverage - plugin detail. http://plugins.netbeans.org/plugin/48570/tikione-jacocoverage (2013)

  25. Flores, P.; Cheon, Y.: Pwisegen: Generating test cases for pairwise testing using genetic algorithms. In: IEEE International Conference on Computer Science and Automation Engineering (CSAE), 2011. IEEE, pp. 747–752

  26. Sabharwal, S.; Aggarwal, M.: Variable strength interaction test set generation using multi objective genetic algorithms. In: 2015 International Conference on Advances in Computing, Communications and Informatics (ICACCI), 2015. IEEE, pp. 2049–2053

  27. Varshney, S.; Mehrotra, M.: Search-based test data generator for data-flow dependencies using dominance concepts, branch distance and elitism. Arab. J. Sci. Eng. 41(3), 853–881 (2016)

  28. Papadakis, M.; Henard, C.; Harman, M.; Jia, Y.; Le Traon, Y.: Threats to the validity of mutation-based test assessment. In: Proceedings of the 25th International Symposium on Software Testing and Analysis 2016. ACM, pp. 354–365

Download references

Author information

Authors and Affiliations

  1. Department of Computer Engineering, NSIT, Delhi, India

    Manuj Aggarwal & Sangeeta Sabharwal

Authors
  1. Manuj Aggarwal
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sangeeta Sabharwal
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Manuj Aggarwal.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Aggarwal, M., Sabharwal, S. Combinatorial Test Set Prioritization Using Data Flow Techniques. Arab J Sci Eng 43, 483–497 (2018). https://doi.org/10.1007/s13369-017-2631-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13369-017-2631-y

Keywords

Search

Navigation