research-article

Industrial evaluation of pairwise SPL testing with MoSo-PoLiTe

Authors:

Michaela Steffens,

Sebastian Oster,

Thomas FogdalAuthors Info & Claims

VaMoS '12: Proceedings of the 6th International Workshop on Variability Modeling of Software-Intensive Systems

Pages 55 - 62

https://doi.org/10.1145/2110147.2110154

Published: 25 January 2012 Publication History

Abstract

Testing Software Product Lines is a very challenging task due to variability. Frequently, approaches such as combinatorial testing are used to generate representative sets of products for testing purposes instead of testing each individual product of the SPL under test. In this contribution we present the results of applying the MoSo-PoLiTe framework at Danfoss Power Electronics A/S to calculate a representative set of product configurations for black box testing purposes. Within this evaluation we use MoSo-PoLiTe's pairwise configuration selection component on the basis of a feature model. This component implements a heuristics finding a minimal subset of configurations covering 100% T-wise feature interaction. According to the best of our knowledge, this is the first publication providing industrial results about pairwise SPL testing.

References

[1]

B. Beizer, Software testing techniques (2nd ed.). New York, NY, USA: Van Nostrand Reinhold Co., 1990.

Digital Library

[2]

A. Bertolino and S. Gnesi, "Use case-based testing of product lines," SIGSOFT Software Engineering Notes, vol. 28, no. 5, pp. 355--358, 2003.

Digital Library

[3]

H. Cichos, S. Oster, M. Lochau, and A. Schürr, "Model-based Coverage-Driven Test Suite Generation for Software Product Lines," in Proceedings of the ACM/IEEE 14th International Conference on Model Driven Engineering Languages and Systems, ser. Lecture Notes in Computer Science (LNCS), vol. 6981. Heidelberg: Springer Verlag, 2011, pp. 425--439.

Digital Library

[4]

M. Cohen, M. Dwyer, and J. Shi, "Interaction testing of highly-configurable systems in the presence of constraints," in Proceeding of the International Symposium on Software Testing and Analysis, 2007, pp. 129--139.

Digital Library

[5]

Danfoss A/S, "Devision Power Electronics," http://www.danfoss.com, Danfoss A/S, DK-6430 Nordborg, Denmark.

[6]

Danfoss Drives A/S, Facts worth knowing about frequency converters, 1st edition, 3rd issue 1998.

[7]

M. Ellims, D. Ince, M. Petre, W. Hall, and M. Ellims, "Aetg vs. man: an assessment of the effectiveness of combinatorial test data generation," 2007.

[8]

E. Engström, "Regression test selection and product line system testing," in Proceedings of the 2010 Third International Conference on Software Testing, Verification and Validation. Washington, DC, USA: IEEE Computer Society, 2010, pp. 512--515.

Digital Library

[9]

E. Engström, P. Runeson, and M. Skoglund, "A Systematic Review on Regression Test Selection Techniques," Information & Software Technology, vol. 52, no. 1, pp. 14--30, 2010.

Digital Library

[10]

V. N. D. J. W. Erlich and J. Zevallos, "a statistical assessment of some software testing strategies and application of experimental design techniques," Statistica Sinica, vol. 8, no. 1, pp. 165--184, 1998.

[11]

H. P. Jepsen and D. Beuche, "Running a software product line: standing still is going backwards," in Proceedings of the 13th International Software Product Line Conference, ser. SPLC '09. Pittsburgh, PA, USA: Carnegie Mellon University, 2009, pp. 101--110.

Digital Library

[12]

H. P. Jepsen, J. G. Dall, and D. Beuche, "Minimally invasive migration to software product lines," Software Product Line Conference, International, vol. 0, pp. 203--211, 2007.

Digital Library

[13]

C. H. P. Kim, D. S. Batory, and S. Khurshid, "Reducing Combinatorics in Testing Product Lines," in Proceeding of the 10th International Conference on Aspect-Oriented Software Development. ACM, 2011, pp. 57--68.

Digital Library

[14]

D. R. Kuhn and M. J. Reilly, "An investigation of the applicability of design of experiments to software testing," in Proceedings of the 27th Annual NASA Goddard Software Engineering Workshop (SEW-27'02), ser. SEW '02. Washington, DC, USA: IEEE Computer Society, 2002, pp. 91--.

Digital Library

[15]

D. R. Kuhn, D. R. Wallace, and A. M. Gallo, "Software fault interactions and implications for software testing," IEEE Transactions on SE, vol. 30, pp. 418--421, 2004.

Digital Library

[16]

F. J. v. d. Linden, K. Schmid, and E. Rommes, Software Product Lines in Action: The Best Industrial Practice in Product Line Engineering. Secaucus, NJ, USA: Springer-Verlag New York, Inc., 2007.

Digital Library

[17]

M. Lochau, S. Oster, U. Goltz, and A. Schürr, "Model-based Pairwise Testing for Feature Interaction Coverage in Software Product Line Engineering," Software Quality Journal - Special issue on Quality Engineering for Software Product Lines, 2011.

Digital Library

[18]

J. D. McGregor, "Testing a software product line," Tech. Rep. CMU/SEI-2001-TR-022, 2001.

[19]

E. M. Olimpiew, "Model-Based Testing for Software Product Lines," Ph.D. dissertation, George Mason University, 2008.

Digital Library

[20]

S. Oster, F. Markert, and P. Ritter, "Automated Incremental Pairwise Testing of Software Product Lines," in Proceedings of the 14th International Software Product Line Conference, 2010.

Digital Library

[21]

S. Oster, A. Wübbeke, G. Engels, and A. Schürr, "Model-Based Software Product Lines Testing Survey," in Model-based Testing for Embedded Systems, J. Zander, I. Schieferdecker, and P. Mosterman, Eds. CRC Press/Taylor&Francis, 2010.

[22]

S. Oster, I. Zorcic, F. Markert, and M. Lochau, "MoSo-PoLiTe - Tool Support for Pairwise and Model-Based Software Product Line Testing," in 5th International Workshop on Variability Modelling of Software-Intensive Systems (VAMOS), Namur, Belgium, ser. ACM International Conference Proceedings Series, K. Czarnecki and U. Eisenecker, Eds. New York: ACM Press, 2011, pp. 79--82.

Digital Library

[23]

G. Perrouin, S. Sen, J. Klein, B. Baudry, and Y. L. Traon, "Automated and scalable t-wise test case generation strategies for software product lines," in Proceedings of the third International Conference on Software Testing, Verification and Validation, 2010, pp. 459--468.

Digital Library

[24]

pure::systems. {Online}. Available: http://www.pure-systems.com

[25]

A. Reuys, E. Kamsties, K. Pohl, and S. Reis, "Model-based System Testing of Software Product Families," in Advanced Information Systems Engineering, 17th International Conference, CAiSE 2005, 2005, pp. 519--534.

Digital Library

[26]

K. Scheidemann, "Verifying families of system configurations," PhD Thesis, TU Munich 2007.

[27]

B. Stevens and E. Mendelsohn, "Efficient software testing protocols," in Conference of the Centre for Advanced Studies on Collaborative research. IBM Press, 1998.

Digital Library

[28]

A. Tevanlinna, J. Taina, and R. Kauppinen, "Product family testing: a survey," ACM SIGSOFT Software Engineering Notes., vol. 29, pp. 12--12, 2004.

Digital Library

[29]

D. R. Wallace and D. R. Kuhn, "Failure modes in medical device software: an analysis of 15 years of recall data," in ACS/IEEE International Conference on Computer Systems and Applications, 2001, pp. 301--311.

Cited By

Agh HAzamnouri AWagner S(2024)Software product line testing: a systematic literature reviewEmpirical Software Engineering10.1007/s10664-024-10516-x29:6Online publication date: 2-Sep-2024
https://dl.acm.org/doi/10.1007/s10664-024-10516-x
Singh MSrivastava SGarg S(2023)Sustainable Technology Accessing the Software Product Line (SPL) via Model-Based MutationOperational Research for Renewable Energy and Sustainable Environments10.4018/978-1-6684-9130-0.ch012(267-278)Online publication date: 15-Dec-2023
https://doi.org/10.4018/978-1-6684-9130-0.ch012
Damasceno CMousavi MSimao A(2021)Learning by sampling: learning behavioral family models from software product linesEmpirical Software Engineering10.1007/s10664-020-09912-w26:1Online publication date: 1-Jan-2021
https://dl.acm.org/doi/10.1007/s10664-020-09912-w
Show More Cited By

Index Terms

Industrial evaluation of pairwise SPL testing with MoSo-PoLiTe
1. Software and its engineering
  1. Software creation and management
    1. Software development techniques
      1. Reusability
    2. Software verification and validation
      1. Software defect analysis
        Software testing and debugging
  2. Software notations and tools
    1. Software libraries and repositories

Recommendations

MoSo-PoLiTe: tool support for pairwise and model-based software product line testing
VaMoS '11: Proceedings of the 5th International Workshop on Variability Modeling of Software-Intensive Systems

Testing Software Product Lines is a very challenging task and approaches like combinatorial testing and model-based testing are frequently used to reduce the effort of testing Software Product Lines and to reuse test artifacts. In this contribution we ...
Using feature model knowledge to speed up the generation of covering arrays
VaMoS '13: Proceedings of the 7th International Workshop on Variability Modelling of Software-Intensive Systems

Combinatorial Interaction Testing has shown great potential for effectively testing Software Product Lines (SPLs). An important part of this type of testing is determining a subset of SPL products in which interaction errors are more likely to occur. ...
Pairwise feature-interaction testing for SPLs: potentials and limitations
SPLC '11: Proceedings of the 15th International Software Product Line Conference, Volume 2

A fundamental problem of testing Software Product Lines (SPLs) is that variability enables the production of a large number of instances and it is difficult to construct and run test cases even for SPLs with a small number of variable features. ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

VaMoS '12: Proceedings of the 6th International Workshop on Variability Modeling of Software-Intensive Systems

January 2012

193 pages

ISBN:9781450310581

DOI:10.1145/2110147

General Chair:
Ulrich W. Eisenecker
University of Leipzig, Germany
,
Program Chairs:
Sven Apel
University of Passau, Germany
,
Stefania Gnesi
ISTI-CNR, Italy

Copyright © 2012 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 January 2012

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

VaMoS '12

VaMoS '12: Variability Modeling of Software-intensive Systems

January 25 - 27, 2012

Leipzig, Germany

Acceptance Rates

Overall Acceptance Rate 66 of 147 submissions, 45%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

18
Total Citations
View Citations
251
Total Downloads

Downloads (Last 12 months)4
Downloads (Last 6 weeks)0

Reflects downloads up to 12 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Agh HAzamnouri AWagner S(2024)Software product line testing: a systematic literature reviewEmpirical Software Engineering10.1007/s10664-024-10516-x29:6Online publication date: 2-Sep-2024
https://dl.acm.org/doi/10.1007/s10664-024-10516-x
Singh MSrivastava SGarg S(2023)Sustainable Technology Accessing the Software Product Line (SPL) via Model-Based MutationOperational Research for Renewable Energy and Sustainable Environments10.4018/978-1-6684-9130-0.ch012(267-278)Online publication date: 15-Dec-2023
https://doi.org/10.4018/978-1-6684-9130-0.ch012
Damasceno CMousavi MSimao A(2021)Learning by sampling: learning behavioral family models from software product linesEmpirical Software Engineering10.1007/s10664-020-09912-w26:1Online publication date: 1-Jan-2021
https://dl.acm.org/doi/10.1007/s10664-020-09912-w
Ferreira FDiniz JSilva CFigueiredo EWeyns DPerrouin GWeyns D(2019)Testing Tools for Configurable Software SystemsProceedings of the 13th International Workshop on Variability Modelling of Software-Intensive Systems10.1145/3302333.3302344(1-10)Online publication date: 6-Feb-2019
https://dl.acm.org/doi/10.1145/3302333.3302344
Qian YZhang CWang F(2018)Selecting Products for High-strength T-wise Testing of Software Product Line by Multi-Objective Method2018 IEEE International Conference on Progress in Informatics and Computing (PIC)10.1109/PIC.2018.8706270(370-378)Online publication date: Dec-2018
https://doi.org/10.1109/PIC.2018.8706270
Echeverría J(2017)Research on Augmenting the MDD Process with Variability ModelingACM SIGSOFT Software Engineering Notes10.1145/3011286.301130241:6(1-6)Online publication date: 5-Jan-2017
https://dl.acm.org/doi/10.1145/3011286.3011302
Al-Hajjaji MMeinicke JKrieter SSchröter RThüm TLeich TSaake G(2016)Tool demo: testing configurable systems with FeatureIDEACM SIGPLAN Notices10.1145/3093335.299325452:3(173-177)Online publication date: 20-Oct-2016
https://dl.acm.org/doi/10.1145/3093335.2993254
Al-Hajjaji MMeinicke JKrieter SSchröter RThüm TLeich TSaake GFischer BSchaefer I(2016)Tool demo: testing configurable systems with FeatureIDEProceedings of the 2016 ACM SIGPLAN International Conference on Generative Programming: Concepts and Experiences10.1145/2993236.2993254(173-177)Online publication date: 20-Oct-2016
https://dl.acm.org/doi/10.1145/2993236.2993254
Echeverría JPérez FAbellanas APanach JCetina CPastor ÓGenero MJedlitschka AJørgensen MScanniello GSampath SCaivano DPort D(2016)Evaluating Bug-Fixing in Software Product LinesProceedings of the 10th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement10.1145/2961111.2962635(1-6)Online publication date: 8-Sep-2016
https://dl.acm.org/doi/10.1145/2961111.2962635
Devroey XPerrouin GLegay ASchobbens PHeymans P(2016)Search-based Similarity-driven Behavioural SPL TestingProceedings of the 10th International Workshop on Variability Modelling of Software-Intensive Systems10.1145/2866614.2866627(89-96)Online publication date: 27-Jan-2016
https://dl.acm.org/doi/10.1145/2866614.2866627
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents