Nothing Special   »   [go: up one dir, main page]

skip to main content
10.1145/3551349.3556899acmotherconferencesArticle/Chapter ViewAbstractPublication PagesaseConference Proceedingsconference-collections
research-article
Open access

Scalable Sampling of Highly-Configurable Systems: Generating Random Instances of the Linux Kernel

Published: 05 January 2023 Publication History

Abstract

Software systems are becoming increasingly configurable. A paradigmatic example is the Linux kernel, which can be adjusted for a tremendous variety of hardware devices, from mobile phones to supercomputers, thanks to the thousands of configurable features it supports. In principle, many relevant problems on configurable systems, such as completing a partial configuration to get the system instance that consumes the least energy or optimizes any other quality attribute, could be solved through exhaustive analysis of all configurations. However, configuration spaces are typically colossal and cannot be entirely computed in practice. Alternatively, configuration samples can be analyzed to approximate the answers. Generating those samples is not trivial since features usually have inter-dependencies that constrain the configuration space. Therefore, getting a single valid configuration by chance is extremely unlikely. As a result, advanced samplers are being proposed to generate random samples at a reasonable computational cost. However, to date, no sampler can deal with highly configurable complex systems, such as the Linux kernel. This paper proposes a new sampler that does scale for those systems, based on an original theoretical approach called extensible logic groups. The sampler is compared against five other approaches. Results show our tool to be the fastest and most scalable one.

References

[1]
Dimitris Achlioptas, Zayd S. Hammoudeh, and Panos Theodoropoulos. 2018. Fast Sampling of Perfectly Uniform Satisfying Assignments. In 21st International Conference on Theory and Applications of Satisfiability Testing (SAT). Oxford, UK, 135–147.
[2]
Thorsten Berger, Divya Nair, Ralf Rublack, Joanne M. Atlee, Krzysztof Czarnecki, and Andrzej Wasowski. 2014. Three Cases of Feature-Based Variability Modeling in Industry. In Model-Driven Engineering Languages and Systems International Conference (MODELS). Valencia, Spain, 302–319.
[3]
Thorsten Berger, Steven She, R. Lotufo, Andrzej Wasowski, and Krzysztof Czarnecki. 2013. A Study of Variability Models and Languages in the Systems Software Domain. IEEE Transactions on Software Engineering 39, 12 (2013), 1611–1640.
[4]
Thorsten Berger, Jan-Philipp Steghöfer, Tewfik Ziadi, Jacques Robin, and Jabier Martinez. 2020. The state of adoption and the challenges of systematic variability management in industry. Empirical Software Engineering 25, 3 (2020), 1755–1797.
[5]
Armin Biere, Marijn Heule, Hans van Maaren, and Toby Walsh. 2009. Handbook of Satisfiability: Volume 185 Frontiers in Artificial Intelligence and Applications. IOS Press.
[6]
Randal E. Bryant. 1986. Graph-Based Algorithms for Boolean Function Manipulation. IEEE Trans. Comput. C-35, 8 (1986), 677–691.
[7]
Supratik Chakraborty, Daniel J. Fremont, Kuldeep S. Meel, Sanjit A. Seshia, and Moshe Y. Vardi. 2015. On Parallel Scalable Uniform SAT Witness Generation. In 21st International Conference on Tools and Algorithms for the Construction and Analysis of Systems (TACAS). London, UK, 304–319.
[8]
Supratik Chakraborty, Kuldeep S. Meel, and Moshe Y. Vardi. 2013. A Scalable and Nearly Uniform Generator of SAT Witnesses. In 25th International Conference on Computer Aided Verification (CAV). Saint Petersburg, Russia, 608–623.
[9]
Supratik Chakraborty, Kuldeep S. Meel, and Moshe Y. Vardi. 2014. Balancing Scalability and Uniformity in SAT Witness Generator. In 51st Annual Design Automation Conference (DAC). San Francisco, CA, USA, 1–6.
[10]
Martin Davis, George Logemann, and Donald Loveland. 1962. A Machine Program for Theorem-proving. Commun. ACM 5, 7 (1962), 394–397.
[11]
Rafael Dutra, Kevin Laeufer, Jonathan Bachrach, and Koushik Sen. 2018. Efficient Sampling of SAT Solutions for Testing. In 40th International Conference on Software Engineering (ICSE). Gothenburg, Sweden, 549–559.
[12]
David Fernandez-Amoros, Ruben Heradio, Christoph Mayr-Dorn, and Alexander Egyed. 2019. A Kconfig translation to logic with one-way validation system. In 23rd International Systems and Software Product Line Conference (SPLC). Paris, France, 303–308.
[13]
Michael Forbes, James Lawrence, Yu Lei, Raghu Kacker, and D. Kuhn. 2008. Refining the In-Parameter-Order Strategy for Constructing Covering Arrays. Journal of Research of the National Institute of Standards and Technology 113, 5(2008), 287–297.
[14]
Axel Halin, Alexandre Nuttinck, Mathieu Acher, Xavier Devroey, Gilles Perrouin, and Benoit Baudry. 2019. Test them all, is it worth it? Assessing configuration sampling on the JHipster Web development stack. Empirical Software Engineering 24, 2 (2019), 674–717.
[15]
Ruben Heradio, David Fernandez-Amoros, JoséA. Galindo, David Benavides, and Don Batory. 2022. Uniform and scalable sampling of highly configurable systems. Empirical Software Engineering 27, 2 (2022), 44.
[16]
Ruben Heradio, David Fernandez-Amoros, Jose A. Galindo, and David Benavides. 2020. Uniform and scalable SAT-sampling for configurable systems. In 24th Systems and Software Product Line Conference (SPLC). Montréal, Canada, 1–11.
[17]
Jose Miguel Horcas, José A. Galindo, Ruben Heradio, David Fernandez-Amoros, and David Benavides. 2021. Monte Carlo tree search for feature model analyses: a general framework for decision-making. In 25th ACM International Systems and Software Product Line Conference, Vol. A. Leicester, UK, 190–201.
[18]
Alexander Ivrii, Sharad Malik, Kuldeep S. Meel, and Moshe Y. Vardi. 2016. On computing minimal independent support and its applications to sampling and counting. Constraints 21, 1 (2016), 41–58.
[19]
Kyo C. Kang, Sholom G. Cohen, James A. Hess, William E. Novak, and A. Spencer Peterson. 1990. Feature-oriented domain analysis (FODA) feasibility study. Technical Report CMU/SEI-90-TR-021. Software Engineering Institute.
[20]
Nathan Kitchen and Andreas Kuehlmann. 2007. Stimulus generation for constrained random simulation. In IEEE/ACM International Conference on Computer-Aided Design (ICCAD). San Jose, CA, USA, 258–265.
[21]
Donald E. Knuth. 2009. The Art of Computer Programming, Volume 4, Fascicle 1: Bitwise Tricks & Techniques; Binary Decision Diagrams. Addison-Wesley Professional.
[22]
Sergiy Kolesnikov, Norbert Siegmund, Christian Kästner, Alexander Grebhahn, and Sven Apel. 2019. Tradeoffs in modeling performance of highly configurable software systems. Software & Systems Modeling 18, 3 (2019), 2265–2283.
[23]
Sebastian Krieter. 2019. Enabling Efficient Automated Configuration Generation and Management. In 23rd International Systems and Software Product Line Conference (SPLC). Paris, France, 215–221.
[24]
Sebastian Krieter, Thomas Thüm, Sandro Schulze, Gunter Saake, and Thomas Leich. 2020. YASA: Yet Another Sampling Algorithm. In 14th International Working Conference on Variability Modelling of Software-Intensive System (VaMoS). Magdeburg, Germany, 1–10.
[25]
Yu Lei, Raghu Kacker, D. Kuhn, Vadim Okun, and James Lawrence. 2008. IPOG/IPOG-D: Efficient Test Generation for Multi-way Combinatorial Testing. Software Testing Verification & Reliability 18, 3(2008), 125–148.
[26]
Yu Lei, Raghu Kacker, D. Richard Kuhn, Vadim Okun, and James Lawrence. 2007. IPOG: A General Strategy for T-Way Software Testing. In 14th Annual IEEE International Conference and Workshops on the Engineering of Computer-Based Systems (ECBS’07). 549–556.
[27]
Daniel-Jesus Muñoz, Jeho Oh, Mónica Pinto, Lidia Fuentes, and Don Batory. 2019. Uniform Random Sampling Product Configurations of Feature Models That Have Numerical Features. In 23rd International Systems and Software Product Line Conference (SPLC). Paris, France, 289–301.
[28]
Jeho Oh, Don Batory, Margaret Myers, and Norbert Siegmund. 2017. Finding Near-optimal Configurations in Product Lines by Random Sampling. In 11th Joint Meeting on Foundations of Software Engineering (ESEC/FSE). Paderborn, Germany, 61–71.
[29]
Jeho Oh, Don S. Batory, Marijn J. H. Heule, Margaret Myers, and Paul Gazzillo. 2019. Uniform Sampling from Kconfig Feature Models. Technical Report TR-19-02. Department of Computer Science. The University of Texas at Austin.
[30]
Jeho Oh, Paul Gazillo, Don Batory, Marijn J. H. Huele, and Margaret Meyers. 2020. Scalable Uniform Sampling for Real-World Software Product Lines. Technical Report TR-20-01. University of Texas at Austin.
[31]
Jeho Oh, Paul Gazzillo, and Don Batory. 2019. t-wise Coverage by Uniform Sampling. In 23rd International Systems and Software Product Line Conference (SPLC). Paris, France, 84–87.
[32]
Jeho Oh, Margaret Myers, and Don Batory. 2016. Finding Product Line Configurations with High Performance by Random Sampling. Technical Report TR-16-22. Department of Computer Science, University of Texas at Austin.
[33]
Tobias Pett, Thomas Thüm, Tobias Runge, Sebastian Krieter, Malte Lochau, and Ina Schaefer. 2019. Product Sampling for Product Lines: The Scalability Challenge. In 23rd International Systems and Software Product Line Conference (SPLC). Paris, France, 78–83.
[34]
Quentin Plazar, Mathieu Acher, Gilles Perrouin, Xavier Devroey, and Maxime Cordy. 2019. Uniform Sampling of SAT Solutions for Configurable Systems: Are We There Yet?. In 12th IEEE Conference on Software Testing, Validation and Verification (ICST). Xian, China, China, 240–251.
[35]
Subhajit Roy, Awanish Pandey, Brendan Dolan-Gavitt, and Yu Hu. 2018. Bug Synthesis: Challenging Bug-Finding Tools with Deep Faults. In 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE). Lake Buena Vista, Florida, USA, 224–234.
[36]
Shubham Sharma, Rahul Gupta, Subhajit Roy, and Kuldeep S. Meel. 2018. Knowledge Compilation meets Uniform Sampling. In 22nd International Conference on Logic for Programming, Artificial Intelligence and Reasoning (LPAR). Awassa, Ethiopia, 620–636.
[37]
Norbert Siegmund, Alexander Grebhahn, Sven Apel, and Christian Kästner. 2015. Performance-influence Models for Highly Configurable Systems. In 10th Joint Meeting on Foundations of Software Engineering (ESEC/FSE). Bergamo, Italy, 284–294.
[38]
Chico Sundermann, Thomas Thüm, and Ina Schaefer. 2020. Evaluating #SAT Solvers on Industrial Feature Models. In 14th International Working Conference on Variability Modelling of Software-Intensive Systems (VaMoS). Magdeburg, Germany.
[39]
Thomas Thüm. 2020. A BDD for Linux? The Knowledge Compilation Challenge for Variability. In 24th ACM Conference on Systems and Software Product Lines (SPLC)(SPLC ’20). Montreal, Quebec, Canada, 1–6.
[40]
Marc Thurley. 2006. sharpSAT - Counting Models with Advanced Component Caching and Implicit BCP. In 9th International Conference on Theory and Applications of Satisfiability Testing (SAT). Seattle, WA, USA, 424–429.
[41]
Gregory S. Tseitin. 1983. On the Complexity of Derivation in Propositional Calculus. Springer Berlin Heidelberg, 466–483.
[42]
Mahsa Varshosaz, Mustafa Al-Hajjaji, Thomas Thüm, Tobias Runge, Mohammad Reza Mousavi, and Ina Schaefer. 2018. A Classification of Product Sampling for Software Product Lines. In 22md International Systems and Software Product Line Conference (SPLC). Gothenburg, Sweden, 1–13.
[43]
Jun Yuan, Ken Albin, Adnan Aziz, and Carl Pixley. 2002. Simplifying Boolean Constraint Solving for Random Simulation-Vector Generation. In IEEE/ACM International Conference on Computer Aided Design (ICCAD). San Jose, CA, USA, 123–127.
[44]
Jun Yuan, Ken Albin, Adnan Aziz, and Carl Pixley. 2002. Simplifying Constraint Solving in Random Simulation Generation. In 11th IEEE/ACM International Workshop on Logic & Synthesis (IWLS). New Orleans, Louisiana, USA, 185–190.
[45]
Jun Yuan, Kurt Shultz, and Carl Pixley. 1999. Modeling Design Constraints and Biasing in Simulation Using BDDs. In International Conference on Computer-Aided Design (ICCAD). San Jose, CA, USA, 584–589.

Cited By

View all
  • (2024)Pragmatic Random Sampling of the Linux Kernel: Enhancing the Randomness and Correctness of the conf ToolProceedings of the 28th ACM International Systems and Software Product Line Conference10.1145/3646548.3672586(24-35)Online publication date: 2-Sep-2024
  • (2023)Finding Near-optimal Configurations in Colossal Spaces with Statistical GuaranteesACM Transactions on Software Engineering and Methodology10.1145/361166333:1(1-36)Online publication date: 23-Nov-2023

Index Terms

  1. Scalable Sampling of Highly-Configurable Systems: Generating Random Instances of the Linux Kernel

    Recommendations

    Comments

    Please enable JavaScript to view thecomments powered by Disqus.

    Information & Contributors

    Information

    Published In

    cover image ACM Other conferences
    ASE '22: Proceedings of the 37th IEEE/ACM International Conference on Automated Software Engineering
    October 2022
    2006 pages
    ISBN:9781450394758
    DOI:10.1145/3551349
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 05 January 2023

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. Kconfig
    2. SAT
    3. binary decision diagrams
    4. configurable systems
    5. random sampling
    6. software product lines
    7. variability modeling

    Qualifiers

    • Research-article
    • Research
    • Refereed limited

    Funding Sources

    • Universidad Nacional De Educación a Distancia

    Conference

    ASE '22

    Acceptance Rates

    Overall Acceptance Rate 82 of 337 submissions, 24%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)153
    • Downloads (Last 6 weeks)14
    Reflects downloads up to 20 Sep 2024

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)Pragmatic Random Sampling of the Linux Kernel: Enhancing the Randomness and Correctness of the conf ToolProceedings of the 28th ACM International Systems and Software Product Line Conference10.1145/3646548.3672586(24-35)Online publication date: 2-Sep-2024
    • (2023)Finding Near-optimal Configurations in Colossal Spaces with Statistical GuaranteesACM Transactions on Software Engineering and Methodology10.1145/361166333:1(1-36)Online publication date: 23-Nov-2023

    View Options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    HTML Format

    View this article in HTML Format.

    HTML Format

    Get Access

    Login options

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media