Nothing Special   »   [go: up one dir, main page]

skip to main content
research-article

Security Smells in Ansible and Chef Scripts: A Replication Study

Published: 20 January 2021 Publication History

Abstract

Context: Security smells are recurring coding patterns that are indicative of security weakness and require further inspection. As infrastructure as code (IaC) scripts, such as Ansible and Chef scripts, are used to provision cloud-based servers and systems at scale, security smells in IaC scripts could be used to enable malicious users to exploit vulnerabilities in the provisioned systems. Goal: The goal of this article is to help practitioners avoid insecure coding practices while developing infrastructure as code scripts through an empirical study of security smells in Ansible and Chef scripts. Methodology: We conduct a replication study where we apply qualitative analysis with 1,956 IaC scripts to identify security smells for IaC scripts written in two languages: Ansible and Chef. We construct a static analysis tool called Security Linter for Ansible and Chef scripts (SLAC) to automatically identify security smells in 50,323 scripts collected from 813 open source software repositories. We also submit bug reports for 1,000 randomly selected smell occurrences. Results: We identify two security smells not reported in prior work: missing default in case statement and no integrity check. By applying SLAC we identify 46,600 occurrences of security smells that include 7,849 hard-coded passwords. We observe agreement for 65 of the responded 94 bug reports, which suggests the relevance of security smells for Ansible and Chef scripts amongst practitioners. Conclusion: We observe security smells to be prevalent in Ansible and Chef scripts, similarly to that of the Puppet scripts. We recommend practitioners to rigorously inspect the presence of the identified security smells in Ansible and Chef scripts using (i) code review, and (ii) static analysis tools.

References

[1]
Y. Acar, M. Backes, S. Fahl, S. Garfinkel, D. Kim, M. L. Mazurek, and C. Stransky. 2017. Comparing the usability of cryptographic APIs. In Proceedings of the 2017 IEEE Symposium on Security and Privacy (SP’17). 154--171.
[2]
Amritanshu Agrawal, Akond Rahman, Rahul Krishna, Alexander Sobran, and Tim Menzies. 2018. We don’t need another hero?: The impact of ”heroes” on software development. In Proceedings of the 40th International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP’18). ACM, New York, NY, 245--253.
[3]
Ansible. 2019. NASA: Increasing Cloud Efficiency with Ansible and Ansible Tower. Technical Report. Ansible.
[4]
Ansible. 2020. Ansible Project. Retrieved April 25, 2020 from https://docs.ansible.com/.
[5]
Elaine Barker. 2016. Guideline for Using Cryptographic Standards in the Federal Government: Cryptographic Mechanisms. Technical Report. National Institute of Standards and Technology, Gaithersburg, Maryland.
[6]
Raj Chandra Bose. 1939. On the construction of balanced incomplete block designs. Ann. Eugen. 9, 4 (1939), 353--399.
[7]
Amiangshu Bosu, Jeffrey C. Carver, Munawar Hafiz, Patrick Hilley, and Derek Janni. 2014. Identifying the characteristics of vulnerable code changes: An empirical study. In Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering (FSE’14). ACM, New York, NY, 257--268.
[8]
Sven Bugiel, Stefan Nurnberger, Thomas Poppelmann, Ahmad-Reza Sadeghi, and Thomas Schneider. 2011. Amazon IA: When elasticity snaps back. In Proceedings of the 18th ACM Conference on Computer and Communications Security (CCS’11). ACM, New York, NY, 389--400.
[9]
Chef. 2018. Sitemap-Chef Docs. Retrieved July 4, 2019 from https://docs.chef.io/.
[10]
B. Chen and Z. M. Jiang. 2017. Characterizing and detecting anti-patterns in the logging code. In Proceedings of the 2017 IEEE/ACM 39th International Conference on Software Engineering (ICSE’17). 71--81.
[11]
Jacob Cohen. 1960. A coefficient of agreement for nominal scales. Educ. Psychol. Meas. 20, 1 (1960), 37--46.
[12]
Bert den Boer and Antoon Bosselaers. 1994. Collisions for the compression function of MD5. In Proceedings of the Workshop on the Theory and Application of Cryptographic Techniques on Advances in Cryptology (EUROCRYPT’93). Springer-Verlag, Secaucus, NJ, 293--304. http://dl.acm.org/citation.cfm?id=188307.188356.
[13]
Albert Endres and H. Dieter Rombach. 2003. A Handbook of Software and Systems Engineering: Empirical Observations, Laws, and Theories. Pearson Education.
[14]
Martin Fowler and Kent Beck. 1999. Refactoring: Improving the Design of Existing Code. Addison-Wesley Professional.
[15]
Oliver Hanappi, Waldemar Hummer, and Schahram Dustdar. 2016. Asserting reliable convergence for configuration management scripts. SIGPLAN Not. 51, 10 (October 2016), 328--343.
[16]
Jez Humble and David Farley. 2010. Continuous Delivery: Reliable Software Releases Through Build, Test, and Deployment Automation (1st ed.). Addison-Wesley Professional.
[17]
Waldemar Hummer, Florian Rosenberg, Fábio Oliveira, and Tamar Eilam. 2013. Testing idempotence for infrastructure as code. In Proceedings of the Middleware 2013, David Eyers and Karsten Schwan (Eds.). Springer, 368--388.
[18]
Yujuan Jiang and Bram Adams. 2015. Co-evolution of infrastructure and source code: An empirical study. In Proceedings of the 12th Working Conference on Mining Software Repositories (MSR’15). IEEE Press, Piscataway, NJ, 45--55. http://dl.acm.org/citation.cfm?id=2820518.2820527.
[19]
Natalia Juristo and Omar S. Gómez. 2010. Replication of software engineering experiments. In Empirical Software Engineering and Verification. Springer, 60--88.
[20]
John C. Kelly, Joseph S. Sherif, and Jonathan Hops. 1992. An analysis of defect densities found during software inspections. J. Syst. Softw. 17, 2 (1992), 111--117.
[21]
Jonathan L. Krein and Charles D. Knutson. 2010. A case for replication: Synthesizing research methodologies in software engineering. In Proceedings of the 1st International Workshop on Replication in Empirical Software Engineering Research.
[22]
Rahul Krishna, Amritanshu Agrawal, Akond Rahman, Alexander Sobran, and Tim Menzies. 2018. What is the connection between issues, bugs, and enhancements?: Lessons learned from 800+ software projects. In Proceedings of the 40th International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP’18). ACM, New York, NY, 306--315.
[23]
Puppet Labs. 2018. Borsa Istanbul: Improving Efficiency and Reducing Costs to Manage a Growing Infrastructure. Technical Report. Puppet.
[24]
J. Richard Landis and Gary G. Koch. 1977. The measurement of observer agreement for categorical data. Biometrics 33, 1 (1977), 159--174.
[25]
Mike Leone. 2016. The Economic Benefits of Puppet Enterprise. Technical Report. ESG.
[26]
Michael Meli, Matthew R. McNiece, and Bradley Reaves. 2019. How bad can it git? Characterizing secret leakage in public GitHub repositories. In Proceedings of the 26th Annual Network and Distributed System Security Symposium, (NDSS’19).
[27]
MITRE. 2018. CWE-Common Weakness Enumeration. Retrieved July 2, 2019 from https://cwe.mitre.org/index.html.
[28]
Nuthan Munaiah, Steven Kroh, Craig Cabrey, and Meiyappan Nagappan. 2017. Curating GitHub for engineered software projects. Empir. Softw. Eng. 22, 6 (2017), 1--35.
[29]
Pars Mutaf. 1999. Defending against a Denial-of-Service Attack on TCP. In Recent Advances in Intrusion Detection.
[30]
National Institute of Standards and Technology. 2014. Security and Privacy Controls for Federal Information Systems and Organizations. Retrieved July 4, 2019 from https://www.nist.gov/publications/security-and-privacy-controls-federal-information-systems-and-organizations-including-0.
[31]
Laboratory of Cryptography and System Security (CrySyS). 2012. sKyWIper (a.k.a. Flame a.k.a. Flamer): A Complex Malware for Targeted Attacks. Technical Report. Laboratory of Cryptography and System Security, Budapest, Hungary.
[32]
Puppet. 2018. Ambit Energy’s Competitive Advantage? It’s Really a DevOps Software Company. Technical Report. Puppet.
[33]
Akond Rahman, Amritanshu Agrawal, Rahul Krishna, and Alexander Sobran. 2018. Characterizing the influence of continuous integration: Empirical results from 250+ open source and proprietary projects. In Proceedings of the 4th ACM SIGSOFT International Workshop on Software Analytics (SWAN’18). ACM, New York, NY, 8--14.
[34]
Akond Rahman, Effat Farhana, Chris Parnin, and Laurie Williams. 2020. Gang of eight: A defect taxonomy for infrastructure as code scripts. In Proceedings of the 42nd International Conference on Software Engineering (ICSE’20).
[35]
Akond Rahman, Effat Farhana, and Laurie Williams. 2020. The ‘as code’ activities: Development anti-patterns for infrastructure as code. Empirical Softw. Engg. 25, 5 (2020), 43.
[36]
Akond Rahman, Rezvan Mahdavi-Hezaveh, and Laurie Williams. 2018. A systematic mapping study of infrastructure as code research. Inf. Softw. Technol. 108, 4 (2018).
[37]
Akond Rahman, Chris Parnin, and Laurie Williams. 2019. The seven sins: Security smells in infrastructure as code scripts. In Proceedings of the 41st International Conference on Software Engineering (ICSE’19). IEEE Press, Piscataway, NJ, 164--175.
[38]
Akond Rahman, Asif Partho, David Meder, and Laurie Williams. 2017. Which factors influence practitioners’ usage of build automation tools? In Proceedings of the 3rd International Workshop on Rapid Continuous Software Engineering (RCoSE’17). IEEE Press, Piscataway, NJ, 20--26.
[39]
Akond Rahman, M. Rahman, Chris Parnin, and Laurie Williams. 2020. Dataset for security smells for ansible and chef scripts used in DevOps.
[40]
A. Rahman and L. Williams. 2018. Characterizing defective configuration scripts used for continuous deployment. In Proceedings of the 2018 IEEE 11th International Conference on Software Testing, Verification and Validation (ICST’18). 34--45.
[41]
Akond Rahman and Laurie Williams. 2019. Source code properties of defective infrastructure as code scripts. Inf. Softw. Technol. 112, 11 (2019).
[42]
Eric Rescorla. 2000. Http over tls. (2000). http://dret.net/biblio/reference/rfc2818.
[43]
Johnny Saldana. 2015. The Coding Manual for Qualitative Researchers. Sage.
[44]
J. H. Saltzer and M. D. Schroeder. 1975. The protection of information in computer systems. Proc. IEEE 63, 9 (September 1975), 1278--1308.
[45]
Stefan Schmidt. 2009. Shall we really do it again? The powerful concept of replication is neglected in the social sciences. Rev. Gen. Psychol. 13, 2 (2009), 90--100.
[46]
Julian Schwarz. 2017. Code Smell Detection in Infrastructure as Code. Retrieved July 2, 2019 from https://www.swc.rwth-aachen.de/thesis/code-smell-detection-infrastructure-code/.
[47]
J. Schwarz, A. Steffens, and H. Lichter. 2018. Code smells in infrastructure as code. In Proceedings of the 2018 11th International Conference on the Quality of Information and Communications Technology (QUATIC’18). 220--228.
[48]
Tushar Sharma, Marios Fragkoulis, and Diomidis Spinellis. 2016. Does your configuration code smell? In Proceedings of the 13th International Conference on Mining Software Repositories (MSR’16). ACM, New York, NY, 189--200.
[49]
Jacek Sliwerski, Thomas Zimmermann, and Andreas Zeller. 2005. When do changes induce fixes? In Proceedings of the 2005 International Workshop on Mining Software Repositories (MSR’05). ACM, New York, NY, 1--5.
[50]
Margaret-Anne Storey, Jody Ryall, R. Ian Bull, Del Myers, and Janice Singer. 2008. TODO or to bug: Exploring how task annotations play a role in the work practices of software developers. In Proceedings of the 30th International Conference on Software Engineering (ICSE’08). ACM, New York, NY, 251--260.
[51]
Lin Tan, Ding Yuan, Gopal Krishna, and Yuanyuan Zhou. 2007. Icomment: Bugs or bad comments? In Proceedings of 21st ACM SIGOPS Symposium on Operating Systems Principles (SOSP’07). ACM, New York, NY, 145--158.
[52]
Eduard van der Bent, Jurriaan Hage, Joost Visser, and Georgios Gousios. 2018. How good is your puppet? An empirically defined and validated quality model for puppet. In Proceedings of the 2018 IEEE 25th International Conference on Software Analysis, Evolution and Reengineering (SANER’18). 164--174.
[53]
Xiaoyun Wang and Hongbo Yu. 2005. How to break MD5 and other hash functions. In Proceedings of the 24th Annual International Conference on Theory and Applications of Cryptographic Techniques (EUROCRYPT’05). Springer-Verlag, Berlin, 19--35.
[54]
Claes Wohlin, Per Runeson, Martin Hst, Magnus C. Ohlsson, Bjrn Regnell, and Anders Wessln. 2012. Experimentation in Software Engineering. Springer, Berlin.
[55]
Yevgeniy Brikman. 2016. Why we use Terraform and not Chef, Puppet, Ansible, SaltStack, or CloudFormation. Retrieved April 24, 2020 from https://blog.gruntwork.io/why-we-use-terraform-and-not-chef-puppet-ansible-saltstack-or-cloudformation-7989dad2865c.
[56]
Tatu Ylonen and Chris Lonvick. 2006. The secure shell (SSH) protocol architecture. (2006). https://tools.ietf.org/html/rfc4251.

Cited By

View all
  • (2024)Evaluating the Quality of Open Source Ansible Playbooks: An Executability PerspectiveProceedings of the 4th International Workshop on Software Engineering and AI for Data Quality in Cyber-Physical Systems/Internet of Things10.1145/3663530.3665019(2-5)Online publication date: 15-Jul-2024
  • (2024)Fine-Grained Just-In-Time Defect Prediction at the Block Level in Infrastructure-as-Code (IaC)Proceedings of the 21st International Conference on Mining Software Repositories10.1145/3643991.3644934(100-112)Online publication date: 15-Apr-2024
  • (2024)The PIPr Dataset of Public Infrastructure as Code ProgramsProceedings of the 21st International Conference on Mining Software Repositories10.1145/3643991.3644888(498-503)Online publication date: 15-Apr-2024
  • Show More Cited By

Index Terms

  1. Security Smells in Ansible and Chef Scripts: A Replication Study

    Recommendations

    Comments

    Please enable JavaScript to view thecomments powered by Disqus.

    Information & Contributors

    Information

    Published In

    cover image ACM Transactions on Software Engineering and Methodology
    ACM Transactions on Software Engineering and Methodology  Volume 30, Issue 1
    Continuous Special Section: AI and SE
    January 2021
    444 pages
    ISSN:1049-331X
    EISSN:1557-7392
    DOI:10.1145/3446626
    • Editor:
    • Mauro Pezzè
    Issue’s Table of Contents
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 20 January 2021
    Accepted: 01 June 2020
    Revised: 01 May 2020
    Received: 01 February 2020
    Published in TOSEM Volume 30, Issue 1

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. Ansible
    2. chef
    3. configuration as code
    4. configuration scripts
    5. devops
    6. devsecops
    7. empirical study
    8. infrastructure as code
    9. insecure coding
    10. security
    11. smell
    12. static analysis

    Qualifiers

    • Research-article
    • Research
    • Refereed

    Funding Sources

    • NSA Science of Security Lablet

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)302
    • Downloads (Last 6 weeks)27
    Reflects downloads up to 19 Sep 2024

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)Evaluating the Quality of Open Source Ansible Playbooks: An Executability PerspectiveProceedings of the 4th International Workshop on Software Engineering and AI for Data Quality in Cyber-Physical Systems/Internet of Things10.1145/3663530.3665019(2-5)Online publication date: 15-Jul-2024
    • (2024)Fine-Grained Just-In-Time Defect Prediction at the Block Level in Infrastructure-as-Code (IaC)Proceedings of the 21st International Conference on Mining Software Repositories10.1145/3643991.3644934(100-112)Online publication date: 15-Apr-2024
    • (2024)The PIPr Dataset of Public Infrastructure as Code ProgramsProceedings of the 21st International Conference on Mining Software Repositories10.1145/3643991.3644888(498-503)Online publication date: 15-Apr-2024
    • (2024)TerraMetrics: An Open Source Tool for Infrastructure-as-Code (IaC) Quality Metrics in TerraformProceedings of the 32nd IEEE/ACM International Conference on Program Comprehension10.1145/3643916.3644439(450-454)Online publication date: 15-Apr-2024
    • (2024)Automated Infrastructure as Code Program TestingIEEE Transactions on Software Engineering10.1109/TSE.2024.339307050:6(1585-1599)Online publication date: 1-May-2024
    • (2024)Test-suite-guided discovery of least privilege for cloud infrastructure as codeAutomated Software Engineering10.1007/s10515-024-00420-531:1Online publication date: 5-Mar-2024
    • (2024)Exploring the Feasibility of ChatGPT for Improving the Quality of Ansible Scripts in Edge-Cloud Infrastructures Through Code RecommendationCurrent Trends in Web Engineering10.1007/978-3-031-50385-6_7(75-83)Online publication date: 4-Jan-2024
    • (2023)Scaling Automated Programming Assessment SystemsElectronics10.3390/electronics1204094212:4(942)Online publication date: 13-Feb-2023
    • (2023)Empirical evidence on technical challenges when adopting continuous practicesProceedings of the XXXVII Brazilian Symposium on Software Engineering10.1145/3613372.3613390(11-20)Online publication date: 25-Sep-2023
    • (2023)Secrets Revealed in Container Images: An Internet-wide Study on Occurrence and ImpactProceedings of the 2023 ACM Asia Conference on Computer and Communications Security10.1145/3579856.3590329(797-811)Online publication date: 10-Jul-2023
    • Show More Cited By

    View Options

    Get Access

    Login options

    Full Access

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    HTML Format

    View this article in HTML Format.

    HTML Format

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media