Software product line testing: a systematic literature review
Abstract
A Software Product Line (SPL) is a software development paradigm in which a family of software products shares a set of core assets. Testing has a vital role in both single-system development and SPL development in identifying potential faults by examining the behavior of a product or products, but it is especially challenging in SPL. There have been many research contributions in the SPL testing field; therefore, assessing the current state of research and practice is necessary to understand the progress in testing practices and to identify the gap between required techniques and existing approaches. This paper aims to survey existing research on SPL testing to provide researchers and practitioners with up-to-date evidence and issues that enable further development of the field. To this end, we conducted a Systematic Literature Review (SLR) with seven research questions in which we identified and analyzed 118 studies dating from 2003 to 2022. The results indicate that the literature proposes many techniques for specific aspects (e.g., controlling cost/effort in SPL testing); however, other elements (e.g., regression testing and non-functional testing) still need to be covered by existing research. Furthermore, most approaches are evaluated by only one empirical method, most of which are academic evaluations. This may jeopardize the adoption of approaches in industry. The results of this study can help identify gaps in SPL testing since specific points of SPL Engineering still need to be addressed entirely.
References
[1]
Aduni Sulaiman R, Jawawi DN, Halim SA (2019) Derivation of test cases for model-based testing of software product line with hybrid heuristic approach. In: IRICT’19, pp 199–208.
[2]
Akbari Z, Khoshnevis S, and Mohsenzadeh M A method for prioritizing integration testing in software product lines based on feature model Int J Softw Eng Knowl Eng 2017 27 04 575-600
[3]
Akimoto H, Isogami Y, Kitamura T, Noda N, Kishi T (2019) A prioritization method for SPL pairwise testing based on user profiles. In: APSEC’19, pp 118–125.
[4]
Al-Dallal J and Sorenson PG Testing software assets of framework-based product families during application engineering stage J Softw 2008 3 5 11-25
[5]
Al-Hajjaji M, Krieter S, Thüm T, Lochau M, and Saake G IncLing: efficient product-line testing using incremental pairwise sampling ACM SIGPLAN Not 2016 52 3 144-155
[6]
Al-Hajjaji M, Krüger J, Schulze S, Leich T, Saake G (2017a) Efficient product-line testing using cluster-based product prioritization. In: AST’17, pp 16–22.
[7]
Al-Hajjaji M, Lity S, Lachmann R, Thüm T, Schaefer I, Saake G (2017b) Delta-oriented product prioritization for similarity-based product-line testing. In: VACE’17, pp 34–40.
[8]
Al-Hajjaji M, Thüm T, Lochau M, Meinicke J, and Saake G Effective product-line testing using similarity-based product prioritization Softw Syst Model 2019 18 1 499-521
[9]
AL-Msie’deen RF, Seriai A, Huchard M, Urtado C, Vauttier S, Salman HE (2013) Feature location in a collection of software product variants using formal concept analysis. In: ICSR’13, pp 302–307.
[10]
Alves V, Niu N, Alves C, and Valença G Requirements engineering for software product lines: a systematic literature review Inf Softw Technol 2010 52 8 806-820
[11]
Alves Pereira J, Acher M, Martin H, Jézéquel JM (2020) Sampling effect on performance prediction of configurable systems: A case study. In: ICPE’20, pp 277–288.
[12]
Ammann P and Offutt J Introduction to software testing 2008 Cambridge Cambridge University Press
[13]
Ampatzoglou A, Bibi S, Avgeriou P, Verbeek M, and Chatzigeorgiou A Identifying, categorizing and mitigating threats to validity in software engineering secondary studies Inf Softw Technol 2019 106 201-230
[14]
Aoyama Y, Kuroiwa T, Kushiro N (2021) Executable test case generation from specifications written in natural language and test execution environment. In: CCNC’21, pp 1–6.
[15]
Apel S, Batory D, Kästner C, and Saake G Feature-oriented software product lines: concepts and implementation 2013 Berlin Springer
[16]
Araújo IL, Santos IS, Filho JB, Andrade RM, Neto PS (2017) Generating test cases and procedures from use cases in dynamic software product lines. In: SAC’17, pp 1296–1301.
[17]
Arrieta A, Sagardui G, Etxeberria L (2015) Test control algorithms for the validation of cyber-physical systems product lines. In: SPLC’15, pp 273–282.
[18]
Arrieta A, Wang S, Sagardui G, Etxeberria L (2016) Search-based test case selection of cyber-physical system product lines for simulation-based validation. In: SPLC’16, pp 297–306.
[19]
Arrieta A, Wang S, Sagardui G, and Etxeberria L Search-based test case prioritization for simulation-based testing of cyber-physical system product lines J Syst Softw 2019 149 1-34
[20]
Baller H, Lity S, Lochau M, Schaefer I (2014) Multi-objective test suite optimization for incremental product family testing. In: ICST’14, pp 303–312.
[21]
Belli F, Tuglular T, Ufuktepe E (2021) Heterogeneous modeling and testing of software product lines. In: QRS-C’21, pp 1079–1088.
[22]
Beohar H and Mousavi MR Input–output conformance testing for software product lines J Log Algebr Methods Program 2016 85 6 1131-1153
[23]
Bharathi M, Sangeetha V (2018) Weighted rank ant colony metaheuristics optimization-based test suite reduction in combinatorial testing for improving software quality. In: ICICCS’18, pp 525–534.
[24]
Bucaioni A, Di Silvestro F, Singh I, Saadatmand M, and Muccini H Model-based generation of test scripts across product variants: an experience report from the railway industry J Softw Evol Process 2022 34 11 e2498
[25]
Bürdek J, Lochau M, Bauregger S, Holzer A, Rhein AV, Apel S, Beyer D (2015) Facilitating reuse in multi-goal test-suite generation for software product lines. In: FASE’15, pp 84–99.
[26]
Cabral I, Cohen MB, Rothermel G (2010) Improving the testing and testability of software product lines. In: SPLC’10, pp 241–255.
[27]
Calvagna A, Gargantini A, Vavassori P (2013) Combinatorial testing for feature models using CitLab. In: ICSTW’13, pp 338–347.
[28]
Carlsson M, Gotlieb A, Marijan D (2016) Software product line test suite reduction with constraint optimization. In: ICSOFT’16, pp 68–87.
[29]
Catal C and Mishra D Test case prioritization: a systematic mapping study Softw Qual J 2012 21 3 445-478
[30]
Chen L and Babar MA A systematic review of evaluation of variability management approaches in software product lines Inf Softw Technol 2011 53 4 344-362
[31]
Clarke D, Helvensteijn M, and Schaefer I Abstract delta modeling ACM SIGPLAN Not 2010 46 2 13-22
[32]
Clements P and Northrop L Software product lines: practices and patterns 2002 Boston Addison-Wesley
[33]
Cmyrev A and Reissing R Efficient and effective testing of automotive software product lines Appl Sci Eng Prog 2014 7 2 53-57
[34]
Cohen MB, Dwyer MB, Shi J (2006) Coverage and adequacy in software product line testing. In: ROSATEA’06, pp 53–63.
[35]
Cohen MB, Dwyer MB, Shi J (2007) Interaction testing of highly-configurable systems in the presence of constraints. In: ISSTA’07, pp 129–139.
[36]
Cruzes DS and Dybä T Research synthesis in software engineering: a tertiary study Inf Softw Technol 2011 53 5 440-455
[37]
Czarnecki K and Eisenecker UW Generative programming: methods, tools and applications 2000 New York Addison-Wesley
[38]
Da Mota Silveira Neto PA, do, Carmo Machado I, McGregor JD, De Almeida ES, de Lemos Meira SR (2011) A systematic mapping study of software product lines testing. Inf Softw Technol 53(5):407–423.
[39]
Denger C, Kolb R (2006) Testing and inspecting reusable product line components: First empirical results. In: ISESE’06, pp 184–193.
[40]
Devroey X, Perrouin G, Legay A, Schobbens PY, Heymans P (2017) Dissimilar test case selection for behavioural software product line testing, In: SPLC’17, pp 1–9
[41]
do Carmo Machado I, McGregor JD, Cavalcanti YC, and De Almeida ES On strategies for testing software product lines: a systematic literature review Inf Softw Technol 2014 56 10 1183-1199
[42]
do Nascimento Ferreira T, Kuk JN, Pozo A, Vergilio SR (2016) Product selection based on upper confidence bound MOEA/D-DRA for testing software product lines. In: CEC’16, pp 4135–4142.
[43]
Dominka S, Mandl M, Dübner M, Ertl D (2018) Using combinatorial testing for distributed automotive features: Applying combinatorial testing for automated feature-interaction-testing. In: CCWC’18, pp 490–495.
[44]
Drave I, Hillemacher S, Greifenberg T, Kriebel S, Kusmenko E, Markthaler M, Orth P, Salman KS, Richenhagen J, Rumpe B, and Schulze C SMArDT modeling for automotive software testing Softw Pract Exp 2019 49 2 301-328
[45]
Dybå T and Dingsøyr T Empirical studies of agile software development: a systematic review Inf Softw Technol 2008 50 9–10 833-859
[46]
Ebert R, Jolianis J, Kriebel S, Markthaler M, Pruenster B, Rumpe B, Salman KS (2019) Applying product line testing for the electric drive system. In: SPLC’19, pp 14–24.
[47]
Engström E and Runeson P Software product line testing–A systematic mapping study Inf Softw Technol 2011 53 1 2-13
[48]
Ensan F, Bagheri E, Gašević D (2012) Evolutionary search-based test generation for software product line feature models. In: CAiSE’12, pp 613–628.
[49]
Ergun B, Gebizli CŞ, Sözer H (2017) FORMAT: A tool for adapting test models based on feature models. In: COMPSAC’17, pp 66–71.
[50]
Ferreira JM, Vergilio SR, Quináia MA (2013) A mutation approach to feature testing of software product lines. In: SEKE’13, pp 231–237
[51]
Ferreira JM, Vergilio SR, and Quinaia MA Software product line testing based on feature model mutation Int J Softw Eng Knowl Eng 2017 27 05 817-839
[52]
Ferreira TN, Vergilio SR, and de Souza JT Incorporating user preferences in search-based software engineering: a systematic mapping study Inf Softw Technol 2017 90 55-69
[53]
Ferrer J, Chicano F, Alba E (2017) Hybrid algorithms based on integer programming for the search of prioritized test data in software product lines. In: EvoCOP’17, pp 3–19.
[54]
Ferrer J, Chicano F, and Ortega-Toro JA CMSA algorithm for solving the prioritized pairwise test data generation problem in software product lines J Heuristics 2021 27 1 229-249
[55]
Fischer S, Linsbauer L, Egyed A, Lopez-Herrejon RE (2018) Predicting higher order structural feature interactions in variable systems. In: ICSME’18, pp 252–263.
[56]
Fomel S Object management group: UML profile for schedulability, performance and time specification OMG Doc 2002 2 03 1-101
[57]
Fragal VH, Simao A, Endo AT, Mousavi MR (2017) Reducing the concretization effort in FSM-based testing of software product lines. In: ICSTW’17, pp 329–336.
[58]
Fragal VH, Simao A, Mousavi MR, and Turker UC Extending HSI test generation method for software product lines Comput J 2019 62 1 109-129
[59]
Galindo JA, Alférez M, Acher M, Baudry B, Benavides D (2014) A variability-based testing approach for synthesizing video sequences. In: ISSTA’14, pp 293–303.
[60]
Galindo JA, Turner H, Benavides D, and White J Testing variability-intensive systems using automated analysis: an application to Android Softw Qual J 2016 24 2 365-405
[61]
Gebizli CS, Sözer H (2016) Model-based software product line testing by coupling feature models with hierarchical markov chain usage models. In: QRS-C’16, pp 278–283.
[62]
Ghanam Y, Andreychuk D, Maurer F (2010) Reactive variability management in agile software development. In: 2010 Agile Conference, pp 27–34.
[63]
Hajri I, Goknil A, Pastore F, and Briand LC Automating system test case classification and prioritization for use case-driven testing in product lines Empir Softw Eng 2020 25 5 3711-3769
[64]
Henard C, Papadakis M, Perrouin G, Klein J, Traon YL (2013) Multi-objective test generation for software product lines. In: SPLC’13, pp 62–71.
[65]
Henard C, Papadakis M, Perrouin G, Klein J, Heymans P, and Le Traon Y Bypassing the combinatorial explosion: using similarity to generate and prioritize t-wise test configurations for software product lines IEEE Trans Softw Eng 2014 40 7 650-670
[66]
Henard C, Papadakis M, Traon YL (2014b) Mutation-based generation of software product line test configurations. In: SSBSE’14, pp 92–106.
[67]
Hentze M, Pett T, Sundermann C, Krieter S, Thüm T, Schaefer I (2022) Generic Solution-Space Sampling for Multi-domain Product Lines. In: GPCE’22, pp 135–147.
[68]
Hervieu A, Baudry B, Gotlieb A (2011) PACOGEN: Automatic generation of pairwise test configurations from feature models. In: ISSRE’11, pp 120–129.
[69]
Hervieu A, Marijan D, Gotlieb A, and Baudry B Practical minimization of pairwise-covering test configurations using constraint programming Inf Softw Technol 2016 71 129-146
[70]
Hierons RM, Li M, Liu X, Parejo JA, Segura S, and Yao X Many-objective test suite generation for software product lines ACM Trans Softw Eng Methodol 2020 29 1 1-46
[71]
Ibias A, Llana L, Núñez M (2022) Using ant colony optimisation to select features having associated costs. In: ICTSS’22, pp 106–122.
[72]
Jackson D (2012) Software abstractions: logic, language, and analysis. MIT Press
[73]
Jakubovski Filho HL, Ferreira TN, Vergilio SR (2018) Incorporating user preferences in a software product line testing hyper-heuristic approach. In: CEC’18, pp 1–8.
[74]
Jakubovski Filho HL, Ferreira TN, and Vergilio SR Preference based multi-objective algorithms applied to the variability testing of software product lines J Syst Softw 2019 151 194-209
[75]
Jaring M, Krikhaar RL, Bosch J (2008) Modeling variability and testability interaction in software product line engineering. In: ICCBSS’08, pp 120–129.
[76]
Johansen MF, Haugen Ø, Fleurey F (2011) Properties of realistic feature models make combinatorial testing of product lines feasible. In: MODELS’11, pp 638–652.
[77]
Jorgensen PC (2013) Software testing: a craftsman’s approach. Auerbach Publications
[78]
Jung P, Kang S, and Lee J Automated code-based test selection for software product line regression testing J Syst Softw 2019 158 110419
[79]
Jung P, Kang S, and Lee J Efficient regression testing of software product lines by reducing redundant test executions Appl Sci 2020 10 23 8686
[80]
Jung P, Kang S, and Lee J Reducing redundant test executions in software product line testing—A case study Electronics 2022 11 7 1165
[81]
Käkölä T and Dueñas JC Research issues in software product lines—Engineering and management 2006 Heidelberg Springer
[82]
Kang S, Baek H, Kim J, Lee J (2015) Systematic software product line test case derivation for test data reuse. In: COMPSAC’15, pp 433–440.
[83]
Kim CH, Batory DS, Khurshid S (2011) Reducing combinatorics in testing product lines. In: AOSD’11, pp 57–68.
[84]
Kim CH, Khurshid S, Batory D (2012) Shared execution for efficiently testing product lines. In: ISSRE’12, pp 221–230.
[85]
Kim CH, Marinov D, Khurshid S, Batory D, Souto S, Barros P, d’Amorim M (2013) SPLat: Lightweight dynamic analysis for reducing combinatorics in testing configurable systems. In: ESEC/FSE’13, pp 257–267.
[86]
Kitchenham B, Charters S (2007) Guidelines for performing systematic literature reviews in software engineering. Technical Report, Keele University and Durham University
[87]
Kitchenham B, Budgen D, Brereton P (2016) Evidence-based software engineering and systematic reviews. CRC Press
[88]
Krieter S, Thüm T, Schulze S, Saake G, Leich T (2020) YASA: Yet another sampling algorithm. In: VaMoS’20, pp 1–10.
[89]
Kuhn DR, Wallace DR, and Gallo AM Software fault interactions and implications for software testing IEEE Trans Softw Eng 2004 30 6 418-421
[90]
Kumar S (2016) Test case prioritization techniques for software product line: A survey. In: ICCCA, pp 884–889.
[91]
Lachmann R, Lity S, Lischke S, Beddig S, Schulze S, Schaefer I (2015) Delta-oriented test case prioritization for integration testing of software product lines. In: SPLC’15, pp 81–90.
[92]
Lachmann R, Lity S, Al-Hajjaji M, Fürchtegott F, Schaefer I (2016) Fine-grained test case prioritization for integration testing of delta-oriented software product lines. In: FOSD’16, pp 1–10.
[93]
Lachmann R, Beddig S, Lity S, Schulze S, Schaefer I (2017) Risk-based integration testing of software product lines. In: VaMoS’17, pp 52–59.
[94]
Lackner H, Thomas M, Wartenberg F, Weißleder S (2014) Model-based test design of product lines: Raising test design to the product line level. In: ICST’14, pp 51–60.
[95]
Lamancha BP, Polo M, and Piattini M PROW: a pairwise algorithm with constraints, order and weight J Syst Softw 2015 99 1-19
[96]
Lee J and Hwang S Combinatorial test design using design-time decisions for variability Int J Softw Eng Knowl Eng 2019 29 08 1141-1158
[97]
Li Z, Harman M, and Hierons RM Search algorithms for regression test case prioritization IEEE Trans Softw Eng 2007 33 4 225-237
[98]
Li X, Wong WE, Gao R, Hu L, and Hosono S Genetic algorithm-based test generation for software product line with the integration of fault localization techniques Empir Softw Eng 2018 23 1 1-51
[99]
Lima JA, Mendonça WD, Vergilio SR, Assunção WK (2020) Learning-based prioritization of test cases in continuous integration of highly-configurable software. In: SPLC’20, pp 1–11.
[100]
Lity S, Nieke M, Thüm T, and Schaefer I Retest test selection for product-line regression testing of variants and versions of variants J Syst Softw 2019 147 46-63
[101]
Lochau M, Oster S, Goltz U, and Schürr A Model-based pairwise testing for feature interaction coverage in software product line engineering Softw Qual J 2012 20 3 567-604
[102]
Lochau M, Schaefer I, Kamischke J, Lity S (2012b) Incremental model-based testing of delta-oriented software product lines. In: TAP’12, pp 67–82.
[103]
Lochau M, Lity S, Lachmann R, Schaefer I, and Goltz U Delta-oriented model-based integration testing of large-scale systems J Syst Softw 2014 91 63-84
[104]
Lopez-Herrejon RE, Javier Ferrer J, Chicano F, Haslinger EN, Egyed A, Alba E (2014) A parallel evolutionary algorithm for prioritized pairwise testing of software product lines. In: GECCO’14, pp 1255–1262.
[105]
Lopez-Herrejon RE, Fischer S, Ramler R, Egyed A (2015) A first systematic mapping study on combinatorial interaction testing for software product lines. In: ICSTW’15, pp 1–10.
[106]
Luo L Software testing techniques. Institute for Software Research International Carnegie Mellon University Pittsburgh PA 2001 15232 19 1-19
[107]
Luo G, Petrenko A, Bochmann GV (1995) Selecting test sequences for partially-specified nondeterministic finite state machines. In: IFIP WG, pp 95–110.
[108]
Luthmann L, Gerecht T, Stephan A, Bürdek J, and Lochau M Minimum/maximum delay testing of product lines with unbounded parametric real-time constraints J Syst Softw 2019 149 535-553
[109]
Luthmann L, Gerecht T, and Lochau M Sampling strategies for product lines with unbounded parametric real-time constraints Int J Softw Tools Technol Transf 2019 21 6 613-633
[110]
Marijan D, Gotlieb A, Sen S, Hervieu A (2013) Practical pairwise testing for software product lines. In: SPLC’13, pp 227–235.
[111]
Marijan D, Liaaen M, Gotlieb A, Sen S, Ieva C (2017) Titan: Test suite optimization for highly configurable software. In: ICST’17, pp 524–531.
[112]
Markiegi U, Arrieta A, Sagardui G, Etxeberria L (2017) Search-based product line fault detection allocating test cases iteratively. In: SPLC’17, pp 123–132.
[113]
Markiegi U, Arrieta A, Etxeberria L, Sagardui G (2019) Test case selection using structural coverage in software product lines for time-budget constrained scenarios. In: SAC’19, pp 2362–2371.
[114]
Matnei Filho RA and Vergilio SR A multi-objective test data generation approach for mutation testing of feature models J Softw Eng Res Dev 2016 4 1 1-29
[115]
Mayring P (2014) Qualitative Content Analysis: Theoretical Foundation, Basic Procedures and Software Solution. Klagenfurt. Available at Social Science Open Access Repository (SSOAR) https://nbn-resolving.de/urn:nbn:de:0168-ssoar-395173 (accessed 04 June 2024)
[116]
McGregor JD (2001) Testing a software product line. Technical Report, Carnegie Mellon University
[117]
Mendes E, Wohlin C, Felizardo K, and Kalinowski M When to update systematic literature reviews in software engineering J Syst Softw 2020 167 110607
[118]
Mishra S (2006) Specification based software product line testing: A case study. In: CS&P’06, pp 243–254
[119]
Nebut C, Pickin S, Le Traon Y, Jézéquel JM (2003) Automated requirements-based generation of test cases for product families. In: ASE’03, pp 263–266.
[120]
Nebut C, Traon YL, and Jézéquel JM Käköla T and Duenas JC System testing of product lines: from requirements to test cases Software Product lines 2006 Berlin, Heidelberg Springer 447-477
[121]
Neto PA, do Carmo Machado I, Cavalcanti YC, De Almeida ES, Garcia VC, de Lemos Meira SR (2010) A regression testing approach for software product lines architectures. In: SBCARS’10, pp 41–50.
[122]
Nguyen QL (2009) Non-functional requirements analysis modeling for software product lines. In: ICSE’09, pp 56–61.
[123]
Northrop L, Clements P, Bachmann F, Bergey J, Chastek G, Cohen S, Donohoe P, Jones L, Krut R, Little R (2007) A framework for software product line practice, version 5.0. Technical report, Carnegie Mellon University
[124]
Olimpiew EM, Gomaa H (2009) Reusable model-based testing. In: ICSR’09, pp 76–85.
[125]
Oster S, Markert F, Ritter P (2010) Automated incremental pairwise testing of software product lines. In: SPLC’10, pp 196–210.
[126]
Parejo JA, Sánchez AB, Segura S, Ruiz-Cortés A, Lopez-Herrejon RE, and Egyed A Multi-objective test case prioritization in highly configurable systems: a case study J Syst Softw 2016 122 287-310
[127]
Parra C, Giral L, Infante A, Cortés C (2012) Extractive SPL adoption using multi-level variability modeling. In: SPLC’12, pp 99–106.
[128]
Patel S, Gupta P, Shah V (2013) Combinatorial interaction testing with multi-perspective feature models. In: ICSTW’13, pp 321–330.
[129]
Pérez B, Polo M, Piatini M (2009) Software product line testing-A systematic review. In: ICSOFT’09, pp 1–8
[130]
Perrouin G, Sen S, Klein J, Baudry B, Le Traon Y (2010) Automated and scalable t-wise test case generation strategies for software product lines. In: ICST’10, pp 459–468.
[131]
Petry KL, OliveiraJr E, and Zorzo AF Model-based testing of software product lines: mapping study and research roadmap J Syst Softw 2020 167 110608
[132]
Pett T, Eichhorn D, Schaefer I (2020) Risk-based compatibility analysis in automotive systems engineering. In: MODELS’20, pp 1–10.
[133]
Pohl K and Metzger A Software product line testing Commun ACM 2006 49 12 78-81
[134]
Pohl K, Böckle G, and Van Der Linden F Software product line engineering: foundations, principles, and techniques 2005 Heidelberg Springer Berlin
[135]
Qian Y, Zhang C, Wang F (2018) Selecting products for high-strength t-wise testing of software product line by multi-objective method. In: PIC’18, pp 370–378.
[136]
Reis S, Metzger A, Pohl K (2006) A reuse technique for performance testing of software product lines. In: SPLiT’06, pp 5–10
[137]
Reis S, Metzger A, Pohl K (2007) Integration testing in software product line engineering: a model-based technique. In: FASE’07, pp 321–335.
[138]
Reuling D, Bürdek J, Rotärmel S, Lochau M, Kelter U (2015) Fault-based product-line testing: Effective sample generation based on feature-diagram mutation. In: SPLC’15, pp 131–140.
[139]
Reuys A, Kamsties E, Pohl K, Reis S (2005) Model-based system testing of software product families. In: CAiSE’05, pp 519–534.
[140]
Reuys A, Reis S, Kamsties E, Pohl K (2006) The scented method for testing software product lines. In: SPLC’06, pp 479–520.
[141]
Rocha L, Machado I, Almeida E, Kästner C, Nadi S (2020) A semi-automated iterative process for detecting feature interactions. In: SBES’20, pp 778–787.
[142]
Roggenbach M CSP-CASL—A new integration of process algebra and algebraic specification Theor Comput Sci 2006 354 1 42-71
[143]
Rubin J and Chechik M Reinhartz-Berger I, Sturm A, Clark T, Cohen S, and Bettin J A survey of feature location techniques Domain Engineering 2013 Berlin, Heidelberg Springer 29-58
[144]
Safdar SA, Yue T, and Ali S Recommending faulty configurations for interacting systems under test using multi-objective search ACM Trans Softw Eng Methodol 2021 30 4 1-36
[145]
Saini A, Rajkumar, Kumar S (2022) Software Product Line Testing—A Proposal of Distance-Based Approach. In: AISE’20, pp 187–198.
[146]
Sánchez AB, Segura S, Ruiz-Cortés A (2014) A comparison of test case prioritization criteria for software product lines. In: ICST’14, pp 41–50.
[147]
Schaefer I, Bettini L, Bono V, Damiani F, Tanzarella N (2010) Delta-oriented programming of software product lines. In: SPLC’10, pp 77–91.
[148]
Scheidemann KD (2006) Optimizing the selection of representative configurations in verification of evolving product lines of distributed embedded systems. In: SPLC’06, pp 75–84.
[149]
Shi J, Cohen MB, Dwyer MB (2012) Integration testing of software product lines using compositional symbolic execution. In: FASE’12, pp 270–284.
[150]
Sjoberg DI, Dyba T, Jorgensen M (2007) The future of empirical methods in software engineering research. In: FOSE’07, pp 358–378.
[151]
Soe NT, Wild N, Tanachutiwat S, Lichter H (2022) Design and implementation of a test automation framework for configurable devices. In: APIT’22, pp 200–207.
[152]
Souto S and d’Amorim M Time-space efficient regression testing for configurable systems J Syst Softw 2018 137 733-746
[153]
Souto S, d’Amorim M, Gheyi R (2017) Balancing soundness and efficiency for practical testing of configurable systems. In: ICSE’17, pp 632–642.
[154]
Steffens M, Oster S, Lochau M, Fogdal T (2012) Industrial evaluation of pairwise SPL testing with MoSo-PoLiTe. In: VaMoS’12, pp 55–62.
[155]
Stricker V, Metzger A, Pohl K (2010) Avoiding redundant testing in application engineering. In: SPLC’10, pp 226–240.
[156]
Strickler A, Lima JA, Vergilio SR, and Pozo AT Deriving products for variability test of feature models with a hyper-heuristic approach Appl Soft Comput 2016 49 1232-1242
[157]
Tevanlinna A, Taina J, and Kauppinen R Product family testing: a survey ACM SIGSOFT Softw Eng Notes 2004 29 2 12-12
[158]
Tuglular T, Coşkun DE (2021) Behavior-driven development of software product lines. In: DSA’21, pp 230–239.
[159]
Tuglular T, Beyazıt M, Öztürk D (2019) Featured event sequence graphs for model-based incremental testing of software product lines. In: COMPSAC’19, pp 197–202.
[160]
Uzuncaova E, Khurshid S, and Batory D Incremental test generation for software product lines IEEE Trans Softw Eng 2010 36 3 309-322
[161]
Varshosaz M, Al-Hajjaji M, Thüm T, Runge T, Mousavi MR, Schaefer I (2018) A classification of product sampling for software product lines. In: SPLC’18, pp 1–13.
[162]
Vazirani VV Approximation algorithms 2001 Berlin Springer
[163]
Vidács L, Horváth F, Mihalicza J, Vancsics B, Beszédes Á (2015) Supporting software product line testing by optimizing code configuration coverage. In: ICSTW’15, pp 1–7.
[164]
Vidal Silva C, Galindo Duarte JÁ, Benavides Cuevas DF (2020) Functional testing of conflict detection and diagnosis tools in feature model configuration: a test suite design. In: ConfWS’20, pp 17–24
[165]
Wang S, Buchmann D, Ali S, Gotlieb A, Pradhan D, Liaaen M (2014) Multi-objective test prioritization in software product line testing: an industrial case study. In: SPLC’14, pp 32–41.
[166]
Wang S, Ali S, and Gotlieb A Cost-effective test suite minimization in product lines using search techniques J Syst Softw 2015 103 370-391
[167]
Wang S, Ali S, Gotlieb A, and Liaaen M Automated product line test case selection: industrial case study and controlled experiment Softw Syst Model 2017 16 2 417-441
[168]
Webster J and Watson RT Analyzing the past to prepare for the future: writing a literature review MIS Q 2002 26 2 xiii-xxiii
[169]
Weiss DM (2008) The product line hall of fame. In: SPLC’08, pp 39.
[170]
Wohlin C, Höst M, and Henningsson K Empirical research methods in software engineering Empirical methods and studies in Software Engineering-experiences 2003 Berlin, Heidelberg Springer 7-23
[171]
Xiang Y, Huang H, Zhou Y, Li S, Luo C, Lin Q, Yang X (2022) Search-based diverse sampling from real-world software product lines. In: ICSE’22, pp 1945–1957.
[172]
Xu Z, Cohen MB, Motycka W, Rothermel G (2013) Continuous test suite augmentation in software product lines. In: SPLC’13, pp 52–61.
[173]
Yan L, Hu W, Han L (2019) Optimize SPL test cases with adaptive simulated annealing genetic algorithm. In: ACM TURC’19, pp 1–7.
[174]
Yu L, Duan F, Lei Y, Kacker RN, Kuhn DR (2014) Combinatorial test generation for software product lines using minimum invalid tuples. In: HASE’14, pp 65–72.
[175]
Zhang L, Tian JH, Jiang J, Liu YJ, Pu MY, and Yue T Empirical research in software engineering-A literature survey JCST 2018 33 5 876-899
Index Terms
- Software product line testing: a systematic literature review
Index terms have been assigned to the content through auto-classification.
Recommendations
Agile product line engineering—a systematic literature review
Software Product Line Engineering (SPLE) demands upfront long-term investment in (i) designing a common set of core-assets and (ii) managing variability across the products from the same family. When anticipated changes in these core-assets have been ...
On strategies for testing software product lines: A systematic literature review
Context: Testing plays an important role in the quality assurance process for software product line engineering. There are many opportunities for economies of scope and scale in the testing activities, but techniques that can take advantage of these ...
A systematic mapping study of software product lines testing
ContextIn software development, Testing is an important mechanism both to identify defects and assure that completed products work as specified. This is a common practice in single-system development, and continues to hold in Software Product Lines (SPL)...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
© The Author(s) 2024.
Publisher
Kluwer Academic Publishers
United States
Publication History
Published: 02 September 2024
Accepted: 19 June 2024
Author Tags
Qualifiers
- Research-article
Funding Sources
- Technische Universität München (1025)
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 12 Nov 2024
Other Metrics
Citations
View Options
View options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in