research-article

Public Access

Test-case prioritization for configuration testing

Authors:

Runxiang Cheng,

Lingming Zhang,

Tianyin XuAuthors Info & Claims

ISSTA 2021: Proceedings of the 30th ACM SIGSOFT International Symposium on Software Testing and Analysis

Pages 452 - 465

https://doi.org/10.1145/3460319.3464810

Published: 11 July 2021 Publication History

Abstract

Configuration changes are among the dominant causes of failures of large-scale software system deployment. Given the velocity of configuration changes, typically at the scale of hundreds to thousands of times daily in modern cloud systems, checking these configuration changes is critical to prevent failures due to misconfigurations. Recent work has proposed configuration testing, Ctest, a technique that tests configuration changes together with the code that uses the changed configurations. Ctest can automatically generate a large number of ctests that can effectively detect misconfigurations, including those that are hard to detect by traditional techniques. However, running ctests can take a long time to detect misconfigurations. Inspired by traditional test-case prioritization (TCP) that aims to reorder test executions to speed up detection of regression code faults, we propose to apply TCP to reorder ctests to speed up detection of misconfigurations. We extensively evaluate a total of 84 traditional and novel ctest-specific TCP techniques. The experimental results on five widely used cloud projects demonstrate that TCP can substantially speed up misconfiguration detection. Our study provides guidelines for applying TCP to configuration testing in practice.

References

[1]

Mona Attariyan, Michael Chow, and Jason Flinn. 2012. X-ray: Automating Root-Cause Diagnosis of Performance Anomalies in Production Software. In OSDI.

Digital Library

[2]

Mona Attariyan and Jason Flinn. 2010. Automating Configuration Troubleshooting with Dynamic Information Flow Analysis. In OSDI.

[3]

Luiz André Barroso, Urs Hölzle, and Parthasarathy Ranganathan. 2018. The Datacenter as a Computer: Designing Warehouse-Scale Machines. Morgan and Claypool Publishers. https://doi.org/10.2200/S00874ED3V01Y201809CAC046

[4]

Salman Baset, Sahil Suneja, Nilton Bila, Ozan Tuncer, and Canturk Isci. 2017. Usable Declarative Configuration Specification and Validation for Applications, Systems, and Cloud. In Middleware. https://doi.org/10.1145/3154448.3154453

Digital Library

[5]

Jonathan Bell, Owolabi Legunsen, Michael Hilton, Lamyaa Eloussi, Tifany Yung, and Darko Marinov. 2018. DeFlaker: Automatically Detecting Flaky Tests. In ICSE. https://doi.org/10.1145/3180155.3180164

Digital Library

[6]

Junjie Chen, Yiling Lou, Lingming Zhang, Jianyi Zhou, Xiaoleng Wang, Dan Hao, and Lu Zhang. 2018. Optimizing Test Prioritization via Test Distribution Analysis. In ESEC/FSE. https://doi.org/10.1145/3236024.3236053

Digital Library

[7]

Qingrong Chen, Teng Wang, Owolabi Legunsen, Shanshan Li, and Tianyin Xu. 2020. Understanding and Discovering Software Configuration Dependencies in Cloud and Datacenter Systems. In ESEC/FSE. https://doi.org/10.1145/3368089.3409727

Digital Library

[8]

2020. Docker Hub. https://www.docker.com/products/docker-hub

[9]

Sebastian Elbaum, Alexey Malishevsky, and Gregg Rothermel. 2001. Incorporating Varying Test Costs and Fault Severities into Test Case Prioritization. In ICSE. https://doi.org/10.1109/ICSE.2001.919106

[10]

Sebastian Elbaum, Gregg Rothermel, and John Penix. 2014. Techniques for Improving Regression Testing in Continuous Integration Development Environments. In FSE. https://doi.org/10.1145/2635868.2635910

Digital Library

[11]

Michael G. Epitropakis, Shin Yoo, Mark Harman, and Edmund K Burke. 2015. Empirical Evaluation of Pareto Efficient Multi-Objective Regression Test Case Prioritisation. In ISSTA. https://doi.org/10.1145/2771783.2771788

Digital Library

[12]

Ali Ghanbari, Samuel Benton, and Lingming Zhang. 2019. Practical Program Repair via Bytecode Mutation. In ISSTA. https://doi.org/10.1145/3293882.3330559

Digital Library

[13]

Alberto Gonzalez-Sanchez, Eric Piel, Hans-Gerhard Gross, and Arjan JC van Gemund. 2010. Prioritizing Tests for Software Fault Localization. In QSIC. https://doi.org/10.1109/QSIC.2010.28

Digital Library

[14]

Haryadi S. Gunawi, Mingzhe Hao, Riza O. Suminto, Agung Laksono, Anang D. Satria, Jeffry Adityatama, and Kurnia J. Eliazar. 2016. Why Does the Cloud Stop Computing? Lessons from Hundreds of Service Outages. In SoCC. https://doi.org/10.1145/2987550.2987583

Digital Library

[15]

Christopher Henard, Mike Papadakis, Mark Harman, Yue Jia, and Yves Le Traon. 2016. Comparing White-Box and Black-Box Test Prioritization. In ICSE. https://doi.org/10.1145/2884781.2884791

Digital Library

[16]

Christopher Henard, Mike Papadakis, Gilles Perrouin, Jacques Klein, Patrick Heymans, and Yves Le Traon. 2014. Bypassing the Combinatorial Explosion: Using Similarity to Generate and Prioritize T-Wise Test Configurations for Software Product Lines. TSE, 40, 7 (2014), https://doi.org/10.1109/TSE.2014.2327020

Digital Library

[17]

Peng Huang, William J. Bolosky, Abhishek Sigh, and Yuanyuan Zhou. 2015. ConfValley: A Systematic Configuration Validation Framework for Cloud Services. In EuroSys. https://doi.org/10.1145/2741948.2741963

Digital Library

[18]

Qiang Huang, Helen J. Wang, and Nikita Borisov. 2005. Privacy-Preserving Friends Troubleshooting Network. In NDSS.

[19]

2020. JavaParser. https://javaparser.org/about.html

[20]

Bo Jiang, Zhenyu Zhang, Wing Kwong Chan, and TH Tse. 2009. Adaptive Random Test Case Prioritization. In ASE. https://doi.org/10.1109/ASE.2009.77

Digital Library

[21]

Robert Johnson. 2010. More Details on Today’s Outage. http://www.facebook.com/note.php?note_id=431441338919

[22]

Lorenzo Keller, Prasang Upadhyaya, and George Candea. 2008. ConfErr: A Tool for Assessing Resilience to Human Configuration Errors. In DSN. https://doi.org/10.1109/DSN.2008.4630084

[23]

Stuart Kendrick. 2012. What Takes Us Down? USENIX ;login:, 37, 5 (2012).

[24]

Chang Hwan Peter Kim, Darko Marinov, Sarfraz Khurshid, Don Batory, Sabrina Souto, Paulo Barros, and Marcelo D’Amorim. 2013. SPLat: Lightweight Dynamic Analysis for Reducing Combinatorics in Testing Configurable Systems. In ESEC/FSE. https://doi.org/10.1145/2491411.2491459

Digital Library

[25]

Jung-Min Kim and Adam Porter. 2002. A History-Based Test Prioritization Technique for Regression Testing in Resource Constrained Environments. In ICSE. https://doi.org/10.1145/581339.581357

Digital Library

[26]

Zheng Li, Mark Harman, and Robert M Hierons. 2007. Search Algorithms for Regression Test Case Prioritization. TSE, 33, 4 (2007), https://doi.org/10.1109/TSE.2007.38

Digital Library

[27]

Yiling Lou, Samuel Benton, Dan Hao, Lu Zhang, and Lingming Zhang. 2021. How Does Regression Test Selection Affect Program Repair? An Extensive Study on 2 Million Patches. arXiv:2105.07311.

[28]

Yafeng Lu, Yiling Lou, Shiyang Cheng, Lingming Zhang, Dan Hao, Yangfan Zhou, and Lu Zhang. 2016. How Does Regression Test Prioritization Perform in Real-World Software Evolution? In ICSE. https://doi.org/10.1145/2884781.2884874

Digital Library

[29]

Qi Luo, Kevin Moran, Lingming Zhang, and Denys Poshyvanyk. 2018. How do Static and Dynamic Test Case Prioritization Techniques Perform on Modern Software Systems? An Extensive Study on GitHub Projects. TSE, 45, 11 (2018), https://doi.org/10.1109/TSE.2018.2822270

[30]

Alexey G. Malishevsky, Joseph R Ruthruff, Gregg Rothermel, and Sebastian Elbaum. 2006. Cost-Cognizant Test Case Prioritization. TR-UNL-CSE-2006-0004, University of Nebraska-Lincoln.

[31]

Ben Maurer. 2015. Fail at Scale: Reliability in the Face of Rapid Change. CACM, 58, 11 (2015), https://doi.org/10.1145/2838344.2839461

Digital Library

[32]

Flávio Medeiros, Christian Kästner, Márcio Ribeiro, Rohit Gheyi, and Sven Apel. 2016. A Comparison of 10 Sampling Algorithms for Configurable Systems. In ICSE. https://doi.org/10.1145/2884781.2884793

Digital Library

[33]

Sonu Mehta, Ranjita Bhagwan, Rahul Kumar, B. Ashok, Chetan Bansal, Chandra Maddila, Christian Bird, Sumit Asthana, and Aditya Kumar. 2020. Rex: Preventing Bugs and Misconfiguration in Large Services using Correlated Change Analysis. In NSDI.

[34]

Mukelabai Mukelabai, Damir Nešić, Salome Maro, Thorsten Berger, and Jan-Philipp Steghöfer. 2018. Tackling Combinatorial Explosion: A Study of Industrial Needs and Practices for Analyzing Highly Configurable Systems. In ASE. https://doi.org/10.1145/3238147.3238201

Digital Library

[35]

Kiran Nagaraja, Fábio Oliveira, Ricardo Bianchini, Richard P. Martin, and Thu D. Nguyen. 2004. Understanding and Dealing with Operator Mistakes in Internet Services. In OSDI.

[36]

2020. openctest. https://github.com/xlab-uiuc/openctest

[37]

David Oppenheimer, Archana Ganapathi, and David A. Patterson. 2003. Why Do Internet Services Fail, and What Can Be Done About It? In USITS.

[38]

Noam Palatin, Arie Leizarowitz, Assaf Schuster, and Ran Wolff. 2006. Mining for Misconfigured Machines in Grid Systems. In KDD. https://doi.org/10.1145/1150402.1150488

Digital Library

[39]

Marek Parfianowicz and Grzegorz Lewandowski. 2017–2018. OpenClover. https://openclover.org

[40]

David Paterson, José Campos, Rui Abreu, Gregory M. Kapfhammer, Gordon Fraser, and Phil McMinn. 2019. An Empirical Study on the Use of Defect Prediction for Test Case Prioritization. In ICST. https://doi.org/10.1109/ICST.2019.00041

[41]

Qianyang Peng, August Shi, and Lingming Zhang. 2020. Empirically Revisiting and Enhancing IR-Based Test-Case Prioritization. In ISSTA. https://doi.org/10.1145/3395363.3397383

Digital Library

[42]

Rahul Potharaju, Joseph Chan, Luhui Hu, Cristina Nita-Rotaru, Mingshi Wang, Liyuan Zhang, and Navendu Jain. 2015. ConfSeer: Leveraging Customer Support Knowledge Bases for Automated Misconfiguration Detection. In VLDB. https://doi.org/10.14778/2824032.2824079

Digital Library

[43]

Yuhua Qi, Xiaoguang Mao, and Yan Lei. 2013. Efficient Automated Program Repair Through Fault-Recorded Testing Prioritization. In ICSM. https://doi.org/10.1109/ICSM.2013.29

Digital Library

[44]

Xiao Qu, Myra B. Cohen, and Gregg Rothermel. 2008. Configuration-Aware Regression Testing: An Empirical Study of Sampling and Prioritization. In ISSTA. https://doi.org/10.1145/1390630.1390641

Digital Library

[45]

Ariel Rabkin and Randy Katz. 2011. Precomputing Possible Configuration Error Diagnosis. In ASE. https://doi.org/10.1109/ASE.2011.6100053

Digital Library

[46]

Ariel Rabkin and Randy Katz. 2013. How Hadoop Clusters Break. IEEE Software, 30, 4 (2013), https://doi.org/10.1109/MS.2012.73

Digital Library

[47]

Stephen E. Robertson, Steve Walker, and Micheline Hancock-Beaulieu. 2000. Experimentation as a Way of Life: Okapi at TREC. Inf. Process. Manag., 36, 1 (2000), https://doi.org/10.1016/S0306-4573(99)00046-1

Digital Library

[48]

Gregg Rothermel, Roland H. Untch, Chengyun Chu, and Mary Jean Harrold. 1999. Test Case Prioritization: An Empirical Study. In ICSM. https://doi.org/10.1109/ICSM.1999.792604

[49]

Gregg Rothermel, Roland H. Untch, Chengyun Chu, and Mary Jean Harrold. 2001. Prioritizing Test Cases for Regression Testing. TSE, 27, 10 (2001), https://doi.org/10.1145/347324.348910

Digital Library

[50]

David Saff and Michael D. Ernst. 2003. Reducing Wasted Development Time via Continuous Testing. In ISSRE. https://doi.org/10.1109/ISSRE.2003.1251050

[51]

Ripon K. Saha, Lingming Zhang, Sarfraz Khurshid, and Dewayne E. Perry. 2015. An Information Retrieval Approach for Regression Test Prioritization Based on Program Changes. In ICSE. https://doi.org/10.1109/ICSE.2015.47

[52]

Gerard Salton and Christopher Buckley. 1988. Term-Weighting Approaches in Automatic Text Retrieval. Inf. Process. Manag., 24, 5 (1988), https://doi.org/10.1016/0306-4573(88)90021-0

Digital Library

[53]

Mark Santolucito, Ennan Zhai, Rahul Dhodapkar, Aaron Shim, and Ruzica Piskac. 2017. Synthesizing Configuration File Specifications with Association Rule Learning. In OOPSLA. https://doi.org/10.1145/3133888

Digital Library

[54]

Mark Santolucito, Ennan Zhai, and Ruzica Piskac. 2016. Probabilistic Automated Language Learning for Configuration Files. In CAV. https://doi.org/10.1007/978-3-319-41540-6_5

[55]

Alex Sherman, Phil Lisiecki, Andy Berkheimer, and Joel Wein. 2005. ACMS: Akamai Configuration Management System. In NSDI.

[56]

Jonathan Shieber. 2019. Facebook Blames a Server Configuration Change for Yesterday’s Outage. https://techcrunch.com/2019/03/14/facebook-blames-a-misconfigured-server-for-yesterdays-outage

[57]

Hema Srikanth, Myra B Cohen, and Xiao Qu. 2009. Reducing Field Failures in System Configurable Software: Cost-Based Prioritization. In ISSRE. https://doi.org/10.1109/ISSRE.2009.26

Digital Library

[58]

M. Stone. 1974. Cross-Validatory Choice and Assessment of Statistical Predictions. Journal of the Royal Statistical Society. Series B (Methodological), 36, 2 (1974), https://doi.org/10.1111/j.2517-6161.1974.tb00994.x

[59]

Xudong Sun, Runxiang Cheng, Jianyan Chen, Elaine Ang, Owolabi Legunsen, and Tianyin Xu. 2020. Testing Configuration Changes in Context to Prevent Production Failures. In OSDI.

[60]

Chunqiang Tang, Thawan Kooburat, Pradeep Venkatachalam, Akshay Chander, Zhe Wen, Aravind Narayanan, Patrick Dowell, and Robert Karl. 2015. Holistic Configuration Management at Facebook. In SOSP. https://doi.org/10.1145/2815400.2815401

Digital Library

[61]

John W. Tukey. 1949. Comparing Individual Means in the Analysis of Variance. Biometrics, 5, 2 (1949), https://doi.org/10.2307/3001913

[62]

Ozan Tuncer, Nilton Bila, Canturk Isci, and Ayse K. Coskun. 2018. ConfEx: An Analytics Framework for Text-Based Software Configurations in the Cloud. IBM Research. https://doi.org/10.1109/DSN-W.2018.00019

[63]

Helen J. Wang, Yih-Chun Hu, Chun Yuan, Zheng Zhang, and Yi-Min Wang. 2004. Friends Troubleshooting Network: Towards Privacy-Preserving, Automatic Troubleshooting. In IPTPS. https://doi.org/10.1007/978-3-540-30183-7_18

Digital Library

[64]

Helen J. Wang, John C. Platt, Yu Chen, Ruyun Zhang, and Yi-Min Wang. 2004. Automatic Misconfiguration Troubleshooting with PeerPressure. In OSDI.

[65]

Andrew Whitaker, Richard S. Cox, and Steven D. Gribble. 2004. Configuration Debugging as Search: Finding the Needle in the Haystack. In OSDI.

[66]

Chengcheng Xiang, Haochen Huang, Andrew Yoo, Yuanyuan Zhou, and Shankar Pasupathy. 2020. PracExtractor: Extracting Configuration Good Practices from Manuals to Detect Server Misconfigurations. In USENIX ATC.

[67]

Chengcheng Xiang, Yudong Wu, Bingyu Shen, Mingyao Shen, Haochen Huang, Tianyin Xu, Yuanyuan Zhou, Cindy Moore, Xinxin Jin, and Tianwei Sheng. 2019. Towards Continuous Access Control Validation and Forensics. In CCS. https://doi.org/10.1145/3319535.3363191

Digital Library

[68]

Tianyin Xu, Long Jin, Xuepeng Fan, Yuanyuan Zhou, Shankar Pasupathy, and Rukma Talwadker. 2015. Hey, You Have Given Me Too Many Knobs! Understanding and Dealing with Over-Designed Configuration in System Software. In ESEC/FSE. https://doi.org/10.1145/2786805.2786852

Digital Library

[69]

Tianyin Xu, Xinxin Jin, Peng Huang, Yuanyuan Zhou, Shan Lu, Long Jin, and Shankar Pasupathy. 2016. Early Detection of Configuration Errors to Reduce Failure Damage. In OSDI.

[70]

Tianyin Xu and Owolabi Legunsen. 2019. Configuration Testing: Testing Configuration Values as Code and with Code. arXiv:1905.12195.

[71]

Tianyin Xu and Darko Marinov. 2018. Mining Container Image Repositories for Software Configurations and Beyond. In ICSE. https://doi.org/10.1145/3183399.3183403

Digital Library

[72]

Tianyin Xu, Jiaqi Zhang, Peng Huang, Jing Zheng, Tianwei Sheng, Ding Yuan, Yuanyuan Zhou, and Shankar Pasupathy. 2013. Do Not Blame Users for Misconfigurations. In SOSP. https://doi.org/10.1145/2517349.2522727

Digital Library

[73]

Tianyin Xu and Yuanyuan Zhou. 2015. Systems Approaches to Tackling Configuration Errors: A Survey. ACM Comput. Surv., 47, 4 (2015), https://doi.org/10.1145/2791577

Digital Library

[74]

Zuoning Yin, Xiao Ma, Jing Zheng, Yuanyuan Zhou, Lakshmi N. Bairavasundaram, and Shankar Pasupathy. 2011. An Empirical Study on Configuration Errors in Commercial and Open Source Systems. In SOSP.

[75]

Shin Yoo and Mark Harman. 2012. Regression Testing Minimisation, Selection and Prioritisation: A Survey. STVR, 22, 2 (2012), https://doi.org/10.1002/stvr.430

[76]

Ding Yuan, Yinglian Xie, Rina Panigrahy, Junfeng Yang, Chad Verbowski, and Arunvijay Kumar. 2011. Context-based Online Configuration Error Detection. In USENIX ATC.

[77]

Andreas Zeller. 1999. Yesterday, my program worked. Today, it does not. Why? In ESEC/FSE. https://doi.org/10.1145/318774.318946

Digital Library

[78]

Jiaqi Zhang, Lakshmi Renganarayana, Xiaolan Zhang, Niyu Ge, Vasanth Bala, Tianyin Xu, and Yuanyuan Zhou. 2014. EnCore: Exploiting System Environment and Correlation Information for Misconfiguration Detection. In ASPLOS. https://doi.org/10.1145/2644865.2541983

Digital Library

[79]

Lingming Zhang, Dan Hao, Lu Zhang, Gregg Rothermel, and Hong Mei. 2013. Bridging the Gap between the Total and Additional Test-Case Prioritization Strategies. In ICSE. https://doi.org/10.1109/ICSE.2013.6606565

[80]

Lu Zhang, Shan-Shan Hou, Chao Guo, Tao Xie, and Hong Mei. 2009. Time-Aware Test-Case Prioritization using Integer Linear Programming. In ISSTA. https://doi.org/10.1145/1572272.1572297

Digital Library

[81]

Lingming Zhang, Darko Marinov, and Sarfraz Khurshid. 2013. Faster Mutation Testing Inspired by Test Prioritization and Reduction. In ISSTA. https://doi.org/10.1145/2483760.2483782

Digital Library

[82]

Sai Zhang and Michael D. Ernst. 2013. Automated Diagnosis of Software Configuration Errors. In ICSE. https://doi.org/10.1109/ICSE.2013.6606577

[83]

Sai Zhang and Michael D. Ernst. 2014. Which Configuration Option Should I Change? In ICSE. https://doi.org/10.1145/2568225.2568251

Digital Library

[84]

Yuanliang Zhang, Haochen He, Owolabi Legunsen, Shanshan Li, Wei Dong, and Tianyin Xu. 2021. An Evolutionary Study of Configuration Design and Implementation in Cloud Systems. In ICSE. https://doi.org/10.1109/ICSE43902.2021.00029

Digital Library

Cited By

Ahmmed JCohen MGazzillo PFilkov VRay BZhou M(2024)Towards Automated Configuration DocumentationProceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering10.1145/3691620.3695311(2256-2261)Online publication date: 27-Oct-2024
https://dl.acm.org/doi/10.1145/3691620.3695311
Wang SLian XLi QMarinov DXu Td'Amorim M(2024)Ctest4J: A Practical Configuration Testing Framework for JavaCompanion Proceedings of the 32nd ACM International Conference on the Foundations of Software Engineering10.1145/3663529.3663799(562-566)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3663529.3663799
Yıldıran NOh JLawall JGazzillo P(2024)Maximizing Patch Coverage for Testing of Highly-Configurable Software without Exploding Build TimesProceedings of the ACM on Software Engineering10.1145/36437461:FSE(427-449)Online publication date: 12-Jul-2024
https://dl.acm.org/doi/10.1145/3643746
Show More Cited By

Index Terms

Test-case prioritization for configuration testing
1. Software and its engineering
  1. Software creation and management
    1. Software verification and validation
      1. Software defect analysis
        Software testing and debugging
  2. Software organization and properties
    1. Extra-functional properties
      1. Software reliability

Recommendations

Revisiting Test-Case Prioritization on Long-Running Test Suites
ISSTA 2024: Proceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis

The prolonged continuous integration (CI) runs are affecting timely feedback to software developers. Test-case prioritization (TCP) aims to expose faults sooner by reordering tests such that the ones more likely to fail are run earlier. TCP is thus ...
Faster mutation testing inspired by test prioritization and reduction
ISSTA 2013: Proceedings of the 2013 International Symposium on Software Testing and Analysis

Mutation testing is a well-known but costly approach for determining test adequacy. The central idea behind the approach is to generate mutants, which are small syntactic transformations of the program under test, and then to measure for a given test ...
Experimental Comparison of Code-Based and Model-Based Test Prioritization
ICSTW '09: Proceedings of the IEEE International Conference on Software Testing, Verification, and Validation Workshops

During regression testing, a modified system needs to beretested using the existing test suite. Since test suites may be very large, developers are interested in detecting faults in the system as early as possible. Test prioritization orders test cases ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

ISSTA 2021: Proceedings of the 30th ACM SIGSOFT International Symposium on Software Testing and Analysis

July 2021

685 pages

ISBN:9781450384599

DOI:10.1145/3460319

General Chair:
Cristian Cadar
Imperial College London, UK
,
Program Chair:
Xiangyu Zhang
Purdue University, USA

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGSOFT: ACM Special Interest Group on Software Engineering

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 11 July 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Badges

Author Tags

Qualifiers

Research-article

Funding Sources

National Science Foundation

Conference

ISSTA '21

Sponsor:

SIGSOFT

ISSTA '21: 30th ACM SIGSOFT International Symposium on Software Testing and Analysis

July 11 - 17, 2021

Virtual, Denmark

Acceptance Rates

Overall Acceptance Rate 58 of 213 submissions, 27%

Upcoming Conference

ISSTA '25

Sponsor:
sigsoft

34th ACM SIGSOFT International Symposium on Software Testing and Analysis

June 25 - 28, 2025

Trondheim , Norway

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

15
Total Citations
View Citations
1,132
Total Downloads

Downloads (Last 12 months)327
Downloads (Last 6 weeks)39

Reflects downloads up to 22 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Ahmmed JCohen MGazzillo PFilkov VRay BZhou M(2024)Towards Automated Configuration DocumentationProceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering10.1145/3691620.3695311(2256-2261)Online publication date: 27-Oct-2024
https://dl.acm.org/doi/10.1145/3691620.3695311
Wang SLian XLi QMarinov DXu Td'Amorim M(2024)Ctest4J: A Practical Configuration Testing Framework for JavaCompanion Proceedings of the 32nd ACM International Conference on the Foundations of Software Engineering10.1145/3663529.3663799(562-566)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3663529.3663799
Yıldıran NOh JLawall JGazzillo P(2024)Maximizing Patch Coverage for Testing of Highly-Configurable Software without Exploding Build TimesProceedings of the ACM on Software Engineering10.1145/36437461:FSE(427-449)Online publication date: 12-Jul-2024
https://dl.acm.org/doi/10.1145/3643746
Yu GChen PHe ZYan QLuo YLi FZheng Z(2024)ChangeRCA: Finding Root Causes from Software Changes in Large Online SystemsProceedings of the ACM on Software Engineering10.1145/36437281:FSE(24-46)Online publication date: 12-Jul-2024
https://dl.acm.org/doi/10.1145/3643728
Yang YWang WLi ZZhang LPan C(2024)Security Development Lifecycle-Based Adaptive Reward Mechanism for Reinforcement Learning in Continuous Integration Testing OptimizationInternational Journal of Software Engineering and Knowledge Engineering10.1142/S021819402450024434:09(1457-1483)Online publication date: 7-Jun-2024
https://doi.org/10.1142/S0218194024500244
Haas RNömmer RJuergens EApel S(2024)Optimization of Automated and Manual Software Tests in Industrial Practice: A Survey and Historical AnalysisIEEE Transactions on Software Engineering10.1109/TSE.2024.341819150:8(2005-2020)Online publication date: Aug-2024
https://doi.org/10.1109/TSE.2024.3418191
Zhang YYu XLiu JZhang LLi YTan Y(2024)CCE&D: A Configuration Failure Prevention Method for Autonomous Driving SystemsInformation Security and Privacy10.1007/978-981-97-5101-3_16(288-302)Online publication date: 15-Jul-2024
https://doi.org/10.1007/978-981-97-5101-3_16
Wang SLian XMarinov DXu TGrundy JPollock LPenta M(2023)Test Selection for Unified Regression TestingProceedings of the 45th International Conference on Software Engineering10.1109/ICSE48619.2023.00145(1687-1699)Online publication date: 14-May-2023
https://dl.acm.org/doi/10.1109/ICSE48619.2023.00145
Wang TJia ZLi SZheng SYu YXu EPeng SLiao XGrundy JPollock LPenta M(2023)Understanding and Detecting On-the-Fly Configuration BugsProceedings of the 45th International Conference on Software Engineering10.1109/ICSE48619.2023.00062(628-639)Online publication date: 14-May-2023
https://dl.acm.org/doi/10.1109/ICSE48619.2023.00062
Luo FXue MLi JLi KLi SSun GYu H(2023)Configuration Error Localization Based on Delta Debugging2023 International Conference on Blockchain Technology and Information Security (ICBCTIS)10.1109/ICBCTIS59921.2023.00011(27-32)Online publication date: Jun-2023
https://doi.org/10.1109/ICBCTIS59921.2023.00011
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents