research-article

Faults in linux: ten years later

Authors:

Christophe Calvès,

Gilles MullerAuthors Info & Claims

ACM SIGARCH Computer Architecture News, Volume 39, Issue 1

Pages 305 - 318

https://doi.org/10.1145/1961295.1950401

Published: 05 March 2011 Publication History

Abstract

In 2001, Chou et al. published a study of faults found by applying a static analyzer to Linux versions 1.0 through 2.4.1. A major result of their work was that the drivers directory contained up to 7 times more of certain kinds of faults than other directories. This result inspired a number of development and research efforts on improving the reliability of driver code. Today Linux is used in a much wider range of environments, provides a much wider range of services, and has adopted a new development and release model. What has been the impact of these changes on code quality? Are drivers still a major problem?

To answer these questions, we have transported the experiments of Chou et al. to Linux versions 2.6.0 to 2.6.33, released between late 2003 and early 2010. We find that Linux has more than doubled in size during this period, but that the number of faults per line of code has been decreasing. And, even though drivers still accounts for a large part of the kernel code and contains the most faults, its fault rate is now below that of other directories, such as arch (HAL) and fs (file systems). These results can guide further development and research efforts. To enable others to continually update these results as Linux evolves, we define our experimental protocol and make our checkers and results available in a public archive.

References

[1]

A. Aiken, S. Bugrara, I. Dillig, T. Dillig, B. Hackett, and P. Hawkins. An overview of the Saturn project. In Proceedings of the 7th ACM SIGPLAN-SIGSOFT Workshop on Program Analysis for Software Tools and Engineering, PASTE'07, pages 43--48, San Diego, CA, June 2007.

Digital Library

[2]

J. Brunel, D. Doligez, R. R. Hansen, J. Lawall, and G. Muller. A foundation for flow-based program matching using temporal logic and model checking. In The 36th annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, pages 114--126, Savannah, GA, USA, Jan. 2009.

Digital Library

[3]

How is the department of homeland security involved?, 2009. http://scan.coverity.com/faq.html#how-department-homeland-security-in%volved.

[4]

Checkpatch. http://www.codemonkey.org.uk/projects/checkpatch/.

[5]

A. Chou, J. Yang, B. Chelf, S. Hallem, and D. Engler. An empirical study of operating systems errors. In Proceedings of the 18th ACM Symposium on Operating System Principles, pages 73--88, Banff, Canada, Oct. 2001.

Digital Library

[6]

Comedi: Linux Control and Mesurement Device Interface. http://www.comedi.org/.

[7]

J. Corbet. The age of kernel code in various subsystems, Feb. 2010. http://lwn.net/Articles/374622/.

[8]

J. Corbet. How old is our kernel?, Feb. 2010. http://lwn.net/Articles/374574/.

[9]

Static source code analysis, static analysis, software quality tools by Coverity Inc. http://www.coverity.com/, 2008.

[10]

A. Depoutovitch and M. Stumm. Otherworld -- giving applications a chance to survive OS kernel crashes. In ACM EuroSys, pages 181--194, Paris, France, Apr. 2010.

Digital Library

[11]

Fedora project, 2010. http://fedoraproject.org/.

[12]

J. N. Herder, H. Bos, B. Gras, P. Homburg, and A. S. Tanenbaum. Fault isolation for device drivers. In 2009 IEEE/IFIP International Conference on Dependable Systems and Networks (DSN), pages 33--42, Estoril, Portugal, June 2009.

[13]

IEEE std 982.2--1988 IEEE guide for the use of IEEE standard dictionary of measures to produce reliable software, 1988.

[14]

A. Israeli and D. G. Feitelson. The Linux kernel as a case study in software evolution. Journal of Systems and Software, 83(3):485--501, 2010.

Digital Library

[15]

J. L. Lawall, J. Brunel, R. R. Hansen, H. Stuart, G. Muller, and N. Palix. WYSIWIB: A declarative approach to finding protocols and bugs in Linux code. In The 39th Annual IEEE/IFIP International Conference on Dependable Systems and Networks, pages 43--52, Estoril, Portugal, June 2009.

[16]

Z. Li and Y. Zhou. PR-Miner: automatically extracting implicit programming rules and detecting violations in large software code. In Proceedings of the 10th European Software Engineering Conference held jointly with 13th ACM SIGSOFT International Symposium on Foundations of Software Engineering, pages 306--315, Lisbon, Portugal, Sept. 2005.

Digital Library

[17]

Lkml: The Linux kernel mailing list. http://lkml.org/.

[18]

S. Lu, S. Park, E. Seo, and Y. Zhou. Learning from mistakes: a comprehensive study on real world concurrency bug characteristics. In Architectural Support for Programming Languages and Operating Systems (ASPLOS), pages 329--339, Seattle, WA, USA, Mar. 2008.

Digital Library

[19]

T. J. McCabe. A complexity measure. IEEE Transactions on Software Engineering, 2(4):308--320, July 1976.

Digital Library

[20]

P. E. McKenney and J. Walpole. Introducing technology into the Linux kernel: a case study. ACM SIGOPS Operating Systems Review, 42(5):4--17, 2008.

Digital Library

[21]

J. C. Munson and S. G. Elbaum. Code churn: A measure for estimating the impact of code change. In International Conference Software Maintenance (ICSM), pages 24--31, 1998.

Digital Library

[22]

N. Nagappan and T. Ball. Use of relative code churn measures to predict system defect density. In 27th International Conference on Software Engineering (ICSE), pages 284--292, St. Louis, Missouri, USA, May 2005.

Digital Library

[23]

P. Oman and J. Hagemeister. Construction and testing of polynomials predicting software maintainability. Journal of Systems and Software, 24(3):251--266, 1994.

Digital Library

[24]

Y. Padioleau, J. Lawall, R. R. Hansen, and G. Muller. Documenting and automating collateral evolutions in Linux device drivers. In EuroSys 2008, pages 247--260, Glasgow, Scotland, Mar. 2008.

Digital Library

[25]

N. Palix, J. Lawall, and G. Muller. Tracking code patterns over multiple software versions with Herodotos. In Proc. of the ACM International Conference on Aspect-Oriented Software Development, AOSD'10, pages 169--180, Rennes and Saint Malo, France, Mar. 2010.

Digital Library

[26]

N. Palix, J. L. Lawall, and G. Muller. Herodotos: A tool to expose bugs' lives. Research report RR-6984, INRIA, July 2009.

[27]

N. Palix, S. Saha, G. Thomas, C. Calvès, J. Lawall, and G. Muller. Database of Faults in Linux: Ten Years Later, Aug. 2010. http://hal.inria.fr/docs/00/50/92/56/ANNEX/10years.sql.pg_dump.

[28]

N. Palix, S. Saha, G. Thomas, C. Calvès, J. Lawall, and G. Muller. Faults in Linux: Ten years later. Research report RR-7357, INRIA, Aug. 2010.

[29]

N. Palix, S. Saha, G. Thomas, C. Calvès, J. Lawall, and G. Muller. Website of Faults in Linux: Ten Years Later, Dec. 2010. http://faultlinux.lip6.fr/.

[30]

E. S. Raymond. The Cathedral and the Bazaar: Musings on Linux and Open Source by an Accidental Revolutionary. O'Reilly & Associates, Inc., 2001.

Digital Library

[31]

D. Searls. Sparse, Linus & the Lunatics, Nov. 2004. Available at http://www.linuxjournal.com/article/7272.

[32]

X. Song, H. Chen, and B. Zang. Why software hangs and what can be done with it. In International Conference on Dependable Systems and Networks (DSN 2010), Chicago, IL, USA, June 2010.

[33]

Sparse. https://sparse.wiki.kernel.org/index.php/Main_Page.

[34]

B. Spencer. Local kernel exploit in /dev/net/tun. http://grsecurity.net/ spender/cheddar_bay.tgz.

[35]

M. M. Swift, M. Annamalai, B. N. Bershad, and H. M. Levy. Recovering device drivers. ACM Transactions on Computer Systems, 24(4):333--360, 2006.

Digital Library

[36]

The Kernel Janitors. Smatch, the source matcher, 2010. Available at http://smatch.sourceforge.net.

[37]

Ubuntu, 2010. http://www.ubuntu.com/.

[38]

D. Wheeler. Flawfinder home page. Web page: http://www.dwheeler.com/flawfinder/, Oct. 2006.

[39]

D. A. Wheeler. SLOCCount. http://www.dwheeler.com/sloccount/.

Cited By

Qian MLuo JGe YSun CGe XHuang W(2021)Semantic-based false alarm detection approach via machine learning2021 IEEE 21st International Conference on Software Quality, Reliability and Security Companion (QRS-C)10.1109/QRS-C55045.2021.00019(60-66)Online publication date: Dec-2021
https://doi.org/10.1109/QRS-C55045.2021.00019
Li DLi LKim DBissyandé TLo DLe Traon Y(2019)Watch out for this commit! A study of influential software changesJournal of Software: Evolution and Process10.1002/smr.2181Online publication date: 19-Aug-2019
https://doi.org/10.1002/smr.2181
Abal IMelo JStănciulescu ŞBrabrand CRibeiro MWąsowski A(2018)Variability Bugs in Highly Configurable SystemsACM Transactions on Software Engineering and Methodology10.1145/314911926:3(1-34)Online publication date: 12-Jan-2018
https://dl.acm.org/doi/10.1145/3149119
Show More Cited By

Index Terms

Faults in linux: ten years later
1. General and reference
  1. Cross-computing tools and techniques
    1. Reliability
2. Software and its engineering
  1. Software organization and properties
    1. Extra-functional properties
      1. Software reliability

Recommendations

Faults in linux: ten years later
ASPLOS XVI: Proceedings of the sixteenth international conference on Architectural support for programming languages and operating systems

In 2001, Chou et al. published a study of faults found by applying a static analyzer to Linux versions 1.0 through 2.4.1. A major result of their work was that the drivers directory contained up to 7 times more of certain kinds of faults than other ...
Faults in linux: ten years later
ASPLOS '11

In 2001, Chou et al. published a study of faults found by applying a static analyzer to Linux versions 1.0 through 2.4.1. A major result of their work was that the drivers directory contained up to 7 times more of certain kinds of faults than other ...
Faults in Linux 2.6

In August 2011, Linux entered its third decade. Ten years before, Chou et al. published a study of faults found by applying a static analyzer to Linux versions 1.0 through 2.4.1. A major result of their work was that the drivers directory contained up ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM SIGARCH Computer Architecture News

ACM SIGARCH Computer Architecture News Volume 39, Issue 1

ASPLOS '11

March 2011

407 pages

ISSN:0163-5964

DOI:10.1145/1961295

Issue’s Table of Contents

ASPLOS XVI: Proceedings of the sixteenth international conference on Architectural support for programming languages and operating systems
March 2011
432 pages
ISBN:9781450302661
DOI:10.1145/1950365
General Chair:
Rajiv Gupta
University of California, Riverside
,
Program Chair:
Todd C. Mowry
Carnegie Mellon University

Copyright © 2011 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 05 March 2011

Published in SIGARCH Volume 39, Issue 1

Check for updates

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

180
Total Citations
View Citations
2,051
Total Downloads

Downloads (Last 12 months)110
Downloads (Last 6 weeks)13

Reflects downloads up to 20 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Qian MLuo JGe YSun CGe XHuang W(2021)Semantic-based false alarm detection approach via machine learning2021 IEEE 21st International Conference on Software Quality, Reliability and Security Companion (QRS-C)10.1109/QRS-C55045.2021.00019(60-66)Online publication date: Dec-2021
https://doi.org/10.1109/QRS-C55045.2021.00019
Li DLi LKim DBissyandé TLo DLe Traon Y(2019)Watch out for this commit! A study of influential software changesJournal of Software: Evolution and Process10.1002/smr.2181Online publication date: 19-Aug-2019
https://doi.org/10.1002/smr.2181
Abal IMelo JStănciulescu ŞBrabrand CRibeiro MWąsowski A(2018)Variability Bugs in Highly Configurable SystemsACM Transactions on Software Engineering and Methodology10.1145/314911926:3(1-34)Online publication date: 12-Jan-2018
https://dl.acm.org/doi/10.1145/3149119
Li YDolan-Gavitt BWeber SCappos JSilva DFord B(2017)Lock-in-PopProceedings of the 2017 USENIX Conference on Usenix Annual Technical Conference10.5555/3154690.3154692(1-13)Online publication date: 12-Jul-2017
https://dl.acm.org/doi/10.5555/3154690.3154692
Wang KHussain AZuo ZXu GAmiri Sani A(2017)GraspanACM SIGPLAN Notices10.1145/3093336.303774452:4(389-404)Online publication date: 4-Apr-2017
https://dl.acm.org/doi/10.1145/3093336.3037744
Wang KHussain AZuo ZXu GAmiri Sani AChen YTemam OCarter J(2017)GraspanProceedings of the Twenty-Second International Conference on Architectural Support for Programming Languages and Operating Systems10.1145/3037697.3037744(389-404)Online publication date: 4-Apr-2017
https://dl.acm.org/doi/10.1145/3037697.3037744
Roy PMahapatra GDey K(2017)An Efficient Particle Swarm Optimization-Based Neural Network Approach for Software Reliability AssessmentInternational Journal of Reliability, Quality and Safety Engineering10.1142/S021853931750019X24:04(1750019)Online publication date: Aug-2017
https://doi.org/10.1142/S021853931750019X
Ahmed IJensen CGroce AMcKenney P(2017)Applying Mutation Analysis on Kernel Test Suites: An Experience Report2017 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW)10.1109/ICSTW.2017.26(110-115)Online publication date: Mar-2017
https://doi.org/10.1109/ICSTW.2017.26
Kwon YDunn ALee MHofmann OXu YWitchel E(2016)SegoACM SIGARCH Computer Architecture News10.1145/2980024.287237244:2(277-290)Online publication date: 25-Mar-2016
https://dl.acm.org/doi/10.1145/2980024.2872372
Wildani AMiller E(2016)Can We Group Storage? Statistical Techniques to Identify Predictive Groupings in Storage System AccessesACM Transactions on Storage10.1145/273804212:2(1-33)Online publication date: 1-Feb-2016
https://dl.acm.org/doi/10.1145/2738042
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents