research-article

Configuration coverage in the analysis of large-scale system software

Authors:

Julio SinceroAuthors Info & Claims

ACM SIGOPS Operating Systems Review, Volume 45, Issue 3

Pages 10 - 14

https://doi.org/10.1145/2094091.2094095

Published: 11 January 2012 Publication History

Get Access

Abstract

System software, especially operating systems, tends to be highly configurable. Like every complex piece of software, a considerable amount of bugs in the implementation has to be expected. In order to improve the general code quality, tools for static analysis provide means to check for source code defects without having to run actual test cases on real hardware. Still, for proper type checking a specific configuration is required so that all header include paths are available and all types are properly resolved.

In order to find as many bugs as possible, usually a "full configuration" is used for the check. However, mainly because of alternative blocks in form of #else-blocks, a single configuration is insufficient to achieve full coverage. In this paper, we present a metric for configuration coverage (CC) and explain the challenges for (properly) calculating it. Furthermore, we present an efficient approach for determining a sufficiently small set of configurations that achieve (nearly) full coverage and evaluate it on a recent Linux kernel version.

References

[1]

Bessey, A., Block, K., Chelf, B., Chou, A., Fulton, B., Hallem, S., Henri-Gros, C., Kamsky, A., McPeak, S., and Engler, D. A few billion lines of code later: using static analysis to find bugs in the real world. CACM 53 (February 2010), pp. 66--75.

Digital Library

Google Scholar

[2]

Engler, D., Chen, D. Y., Hallem, S., Chou, A., and Chelf, B. Bugs as deviant behavior: a general approach to inferring errors in systems code. In 18th ACM Symp. on OS Principles (SOSP'01), ACM, pp. 57--72.

Digital Library

Google Scholar

[3]

Kästner, C., Giarrusso, P. G., Rendel, T., Erdweg, S., Ostermann, K., and Berger, T. Variability-aware parsing in the presence of lexical macros and conditional compilation. In 26th ACM Conf. on OOP, Systems, Languages, and Applications (OOPSLA '11), ACM.

Digital Library

Google Scholar

[4]

Kremenek, T., Twohey, P., Back, G., Ng, A., and Engler, D. From uncertainty to belief: inferring the specification within. In 7th Symp. on OS Design and Implementation (OSDI '06), USENIX, pp. 161--176.

Digital Library

Google Scholar

[5]

Liebig, J., Apel, S., Lengauer, C., Kästner, C., and Schulze, M. An analysis of the variability in forty preprocessor-based software product lines. In 32nd Int. Conf. on Software Engineering (ICSE '10), ACM.

Digital Library

Google Scholar

[6]

Oster, S., Markert, F., and Ritter, P. Automated incremental pairwise testing of software product lines. In 14th Software Product Line Conf. (SPLC '10) (2010), vol. 6287 of LNCS, Springer, pp. 196--210.

Digital Library

Google Scholar

[7]

Palix, N., Thomas, G., Saha, S., Calvés, C., Lawall, J. L., and Muller, G. Faults in Linux: Ten years later. In 16th Int. Conf. on Arch. Support for Programming Languages and Operating Systems (ASPLOS '11) (2011), ACM, pp. 305--318.

Digital Library

Google Scholar

[8]

Spencer, H., and Collyer, G. #ifdef considered harmful, or portability experience with C News. In 1992 USENIX ATC, USENIX.

Google Scholar

[9]

Spinellis, D. A tale of four kernels. In 30th Int. Conf. on Software Engineering (ICSE '08), W. Schafer, M. B. Dwyer, and V. Gruhn, Eds., ACM, pp. 381--390.

Digital Library

Google Scholar

[10]

Tan, L., Yuan, D., Krishna, G., and Zhou, Y. /*icomment: Bugs or bad comments?*/. In 21st ACM Symp. on OS Principles (SOSP '07), ACM, pp. 145--158.

Digital Library

Google Scholar

[11]

Tartler, R., Lohmann, D., Sincero, J., and Schröder-Preikschat, W. Feature consistency in compile-timeconfigurable system software: Facing the Linux 10,000 feature problem. In ACM SIGOPS/EuroSys Eur. Conf. on Computer Systems 2011 (EuroSys '11) (Apr. 2011), ACM, pp. 47--60.

Digital Library

Google Scholar

[12]

Urban, M., Lohmann, D., and Spinczyk, O. The aspect-oriented design of the puma c/c++ parser framework. In 9th Int. Conf. on Aspect-Oriented Software Development (AOSD '10), ACM, pp. 217--221.

Digital Library

Google Scholar

[13]

Zengler, C., and Küchlin, W. Encoding the Linux kernel configuration in propositional logic. In Proceedings of the 19th European Conference on Artificial Intelligence (ECAI 2010) Workshop on Configuration 2010 (2010), L. Hotz and A. Haselböck, Eds., pp. 51--56.

Google Scholar

Cited By

View all

Hentze MPett TSundermann CKrieter SThüm TSchaefer IScholz BKameyama Y(2022)Generic Solution-Space Sampling for Multi-domain Product LinesProceedings of the 21st ACM SIGPLAN International Conference on Generative Programming: Concepts and Experiences10.1145/3564719.3568695(135-147)Online publication date: 29-Nov-2022
https://dl.acm.org/doi/10.1145/3564719.3568695
Nieke MSampaio GThüm TSeidl CTeixeira LSchaefer I(2021)Guiding the evolution of product-line configurationsSoftware and Systems Modeling (SoSyM)10.1007/s10270-021-00906-w21:1(225-247)Online publication date: 4-Jul-2021
https://dl.acm.org/doi/10.1007/s10270-021-00906-w
Ruland SLuthmann LBürdek JLity SThüm TLochau MRibeiro M(2020)Measuring effectiveness of sample-based product-line testingACM SIGPLAN Notices10.1145/3393934.327813053:9(119-133)Online publication date: 7-Apr-2020
https://dl.acm.org/doi/10.1145/3393934.3278130
Show More Cited By

Index Terms

Configuration coverage in the analysis of large-scale system software
1. General and reference
  1. Cross-computing tools and techniques
    1. Performance
2. Software and its engineering
  1. Software creation and management
    1. Designing software
  2. Software organization and properties
    1. Contextual software domains
      1. Operating systems
    2. Extra-functional properties
      1. Software performance

Recommendations

Configuration coverage in the analysis of large-scale system software
PLOS '11: Proceedings of the 6th Workshop on Programming Languages and Operating Systems

System software, especially operating systems, tends to be highly configurable. Like every complex piece of software, a considerable amount of bugs in the implementation has to be expected. In order to improve the general code quality, tools for static ...
Revealing and repairing configuration inconsistencies in large-scale system software

System software typically offers a large amount of compile-time options and variability. A good example is the Linux kernel, which provides more than 10,000 configurable features, growing rapidly. This allows users to tailor it with respect to a broad ...
Large scale Linux configuration with LCFG
ALS'00: Proceedings of the 4th annual Linux Showcase & Conference - Volume 4

This paper describes the automatic installation and configuration system currently being used to manage several hundred Linux machines in the Division of Informatics at Edinburgh University. This is a development of the LCFG system which has been used ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM SIGOPS Operating Systems Review

ACM SIGOPS Operating Systems Review Volume 45, Issue 3

December 2011

94 pages

ISSN:0163-5980

DOI:10.1145/2094091

Issue’s Table of Contents

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 11 January 2012

Published in SIGOPS Volume 45, Issue 3

Check for updates

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

30
Total Citations
View Citations
160
Total Downloads

Downloads (Last 12 months)3
Downloads (Last 6 weeks)0

Reflects downloads up to 20 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Hentze MPett TSundermann CKrieter SThüm TSchaefer IScholz BKameyama Y(2022)Generic Solution-Space Sampling for Multi-domain Product LinesProceedings of the 21st ACM SIGPLAN International Conference on Generative Programming: Concepts and Experiences10.1145/3564719.3568695(135-147)Online publication date: 29-Nov-2022
https://dl.acm.org/doi/10.1145/3564719.3568695
Nieke MSampaio GThüm TSeidl CTeixeira LSchaefer I(2021)Guiding the evolution of product-line configurationsSoftware and Systems Modeling (SoSyM)10.1007/s10270-021-00906-w21:1(225-247)Online publication date: 4-Jul-2021
https://dl.acm.org/doi/10.1007/s10270-021-00906-w
Ruland SLuthmann LBürdek JLity SThüm TLochau MRibeiro M(2020)Measuring effectiveness of sample-based product-line testingACM SIGPLAN Notices10.1145/3393934.327813053:9(119-133)Online publication date: 7-Apr-2020
https://dl.acm.org/doi/10.1145/3393934.3278130
Lienhardt MDamiani FJohnsen EMauro JRothermel GBae D(2020)Lazy product discovery in huge configuration spacesProceedings of the ACM/IEEE 42nd International Conference on Software Engineering10.1145/3377811.3380372(1509-1521)Online publication date: 27-Jun-2020
https://dl.acm.org/doi/10.1145/3377811.3380372
Kuo HWilliams DKoller RMohan SBilas AMagoutis KMarkatos EKostic DSeltzer M(2020)A Linux in unikernel clothingProceedings of the Fifteenth European Conference on Computer Systems10.1145/3342195.3387526(1-15)Online publication date: 15-Apr-2020
https://dl.acm.org/doi/10.1145/3342195.3387526
Kaltenecker CGrebhahn ASiegmund NApel S(2020)The Interplay of Sampling and Machine Learning for Software Performance PredictionIEEE Software10.1109/MS.2020.298702437:4(58-66)Online publication date: 18-Jun-2020
https://dl.acm.org/doi/10.1109/MS.2020.2987024
Nguyen SNguyen HTran NTran HNguyen TZimmermann TLawall JMarinov D(2019)Feature-interaction aware configuration prioritization for configurable codeProceedings of the 34th IEEE/ACM International Conference on Automated Software Engineering10.1109/ASE.2019.00053(489-501)Online publication date: 10-Nov-2019
https://dl.acm.org/doi/10.1109/ASE.2019.00053
Kolesnikov SSiegmund NKästner CApel S(2019)On the relation of control-flow and performance feature interactionsEmpirical Software Engineering10.1007/s10664-019-09705-w24:4(2410-2437)Online publication date: 1-Aug-2019
https://dl.acm.org/doi/10.1007/s10664-019-09705-w
Rigger MMarr SKell SLeopoldseder DMössenböck H(2018)An Analysis of x86-64 Inline Assembly in C ProgramsACM SIGPLAN Notices10.1145/3296975.318641853:3(84-99)Online publication date: 25-Mar-2018
https://dl.acm.org/doi/10.1145/3296975.3186418
Rhein ALiebig JJanker AKästner CApel S(2018)Variability-Aware Static Analysis at ScaleACM Transactions on Software Engineering and Methodology10.1145/328098627:4(1-33)Online publication date: 16-Nov-2018
https://dl.acm.org/doi/10.1145/3280986
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

Configuration coverage in the analysis of large-scale system software

Revealing and repairing configuration inconsistencies in large-scale system software

Large scale Linux configuration with LCFG