The ‘as code’ activities: development anti-patterns for infrastructure as code

• Adams B, McIntosh S (2016) Modern release engineering in a nutshell – why researchers should care. In: 2016 IEEE 23Rd international conference on software analysis, evolution, and reengineering (SANER). https://doi.org/10.1109/SANER.2016.108, vol 5, pp 78–90

• Alali A, Kagdi H, Maletic JI (2008) What’s a typical commit? a characterization of open source software repositories. In: 2008 16th IEEE international conference on program comprehension, pp 182–191. https://doi.org/10.1109/ICPC.2008.24

• Anderson S, Allen P, Peckham S, Goodwin N (2008) Asking the right questions: scoping studies in the commissioning of research on the organisation and delivery of health services. Health Res Policy Syst 6(1):7

  Google Scholar 

• Ansible (2019) Nasa: Increasing cloud efficiency with ansible and ansible tower. Tech. Rep., Ansible, https://www.ansible.com/hubfs/pdf/Ansible-Case-Study-NASA.pdf?hsLang=en-us

• Arisholm E, Briand LC, Johannessen EB (2010) A systematic and comprehensive investigation of methods to build and evaluate fault prediction models. J Syst Software 83(1):2–17. https://doi.org/10.1016/j.jss.2009.06.055. sI: Top Scholars

  Article  Google Scholar 

• Arksey H, O’Malley L (2005) Scoping studies: towards a methodological framework. Int J Soc Res Methodol 8(1):19–32. https://doi.org/10.1080/1364557032000119616

  Google Scholar 

• Bird C, Nagappan N, Murphy B, Gall H, Devanbu P (2011) Don’t touch my code!: examining the effects of ownership on software quality. In: Proceedings of the 19th ACM SIGSOFT symposium and the 13th european conference on foundations of software engineering, ACM, New York, NY, USA, ESEC/FSE ’11, pp 4–14. https://doi.org/10.1145/2025113.2025119

• Breiman L (2001) Random forests. Mach Learn 45(1):5–32. https://doi.org/10.1023/A:1010933404324

  MATH  Google Scholar 

• Breiman L et al (1984) Classification and Regression Trees, 1st. Chapman & Hall, New York. http://www.crcpress.com/catalog/C4841.htm

  MATH  Google Scholar 

• Bright J (2017) Slalom’s approach to breaking down silos between devops and security teams. https://blog.chef.io/2017/08/16/slaloms-approach-to-breaking-down-silos-between-devops-and-security/. [Online; Accessed 18-Feb-2019]

• Brooks FP Jr (1995) The Mythical Man-month (Anniversary Ed.) Addison-Wesley Longman Publishing Co., Inc, Boston

  Google Scholar 

• Brown WH, Malveau RC, McCormick HWS, Mowbray TJ (1998) Antipatterns: Refactoring Software, Architectures, and Projects in Crisis, 1st. John Wiley & Sons, Inc., New York

  Google Scholar 

• Businge J, Kawuma S, Bainomugisha E, Khomh F, Nabaasa E (2017) Code authorship and fault-proneness of open-source android applications: an empirical study. In: Proceedings of the 13th international conference on predictive models and data analytics in software engineering, ACM, New York, NY, USA, PROMISE, pp 33–42. https://doi.org/10.1145/3127005.3127009

• C SN, Menzies T (2019) Assessing developer beliefs: a reply to “perceptions, expectations, and challenges in defect prediction”. arXiv:1904.05794

• Chen B, Jiang ZMJ (2017) Characterizing and detecting anti-patterns in the logging code. In: Proceedings of the 39th international conference on software engineering, IEEE Press, ICSE ’17, pp 71–81. https://doi.org/10.1109/ICSE.2017.15

• Cliff N (1993) Dominance statistics: ordinal analyses to answer ordinal questions. Psychol Bull 114(3):494–509

  Google Scholar 

• Cohen J (1960) A coefficient of agreement for nominal scales. Educ Psychol Meas 20(1):37–46. https://doi.org/10.1177/001316446002000104

  Google Scholar 

• Commons W (2017) Incident documentation/20170118-Labs. https://wikitech.wikimedia.org/wiki/Incident_documentation/20170118-Labs [Online; accessed 27-Jan-2019]

• Cramer D, Howitt DL (2004) The Sage dictionary of statistics: a practical resource for students in the social sciences. Sage

• Devanbu P, Zimmermann T, Bird C (2016) Belief and evidence in empirical software engineering. In: Proceedings of the 38th international conference on software engineering, ACM, New York, NY, USA, ICSE ’16, pp 108–119. https://doi.org/10.1145/2884781.2884812

• Easterbrook S, Singer J, Storey MA, Damian D (2008) Selecting empirical methods for software engineering research. Springer London, London, pp 285–311

  Google Scholar 

• Elberzhager F, Kremer S, Münch J, Assmann D (2012) Guiding testing activities by predicting defect-prone parts using product and inspection metrics. In: 2012 38th Euromicro conference on software engineering and advanced applications, pp 406–413. https://doi.org/10.1109/SEAA.2012.30

• Freedman D (2005) Statistical models : theory and practice. Cambridge University Press, Cambridge

  MATH  Google Scholar 

• Fu W, Menzies T, Shen X (2016) Tuning for software analytics: is it really necessary?. Inf Softw Technol 76:135–146. http://www.sciencedirect.com/science/article/pii/S0950584916300738

  Google Scholar 

• Garousi V, Küçük B (2018) Smells in software test code: a survey of knowledge in industry and academia. J Syst Software 138:52–81. http://www.sciencedirect.com/science/article/pii/S0164121217303060

  Google Scholar 

• Ghotra B, McIntosh S, Hassan AE (2015) Revisiting the impact of classification techniques on the performance of defect prediction models. In: Proceedings of the 37th international conference on software engineering - volume 1. IEEE Press, Piscataway, pp 789–800. http://dl.acm.org/citation.cfm?id=2818754.2818850

• Hall T, Beecham S, Bowes D, Gray D, Counsell S (2012) A systematic literature review on fault prediction performance in software engineering. IEEE T Software Eng 38(6):1276–1304. https://doi.org/10.1109/TSE.2011.103

  Google Scholar 

• Hassan AE (2009) Predicting faults using the complexity of code changes. In: Proceedings of the 31st international conference on software engineering, IEEE computer society, Washington, DC, USA, ICSE ’09, pp 78–88. https://doi.org/10.1109/ICSE.2009.5070510

• Hersher R (2017) Incident documentation/20170118-Labs. https://www.npr.org/sections/thetwo-way/2017/03/03/518322734/amazon-and-the-150-million-typo, [Online; accessed 27-Jan-2019]

• Hove SE, Anda B (2005) Experiences from conducting semi-structured interviews in empirical software engineering research. In: 11th IEEE International software metrics symposium (METRICS’05), pp 10 pp.–23 https://doi.org/10.1109/METRICS.2005.24

• Huberty CJ, Olejnik S (2006) Applied MANOVA and discriminant analysis, vol 498. John Wiley & Sons, New York

  MATH  Google Scholar 

• Hudak P (1998) Modular domain specific languages and tools. In: Proceedings. Fifth international conference on software reuse (Cat. No.98TB100203), pp 134–142. https://doi.org/10.1109/ICSR.1998.685738

• Humble J, Farley D (2010) Continuous delivery: reliable software releases through build, test, and deployment automation, 1st. Addison-Wesley Professional, Boston

  Google Scholar 

• IEEE (2010) Ieee standard classification for software anomalies. IEEE Std 1044-2009 (Revision of IEEE Std 1044-1993) pp 1–23. https://doi.org/10.1109/IEEESTD.2010.5399061

• Jiang Y, Adams B (2015) Co-evolution of infrastructure and source code: an empirical study. In: Proceedings of the 12th working conference on mining software repositories, ieee press, Piscataway, NJ, USA, MSR ’15, pp 45–55. http://dl.acm.org/citation.cfm?id=2820518.2820527

• Kitchenham BA, Pfleeger SL (2008) Personal opinion surveys. Springer London, London, pp 63–92. https://doi.org/10.1007/978-1-84800-044-5_3

  Google Scholar 

• Labs P (2018) Puppet documentation. https://docs.puppet.com/, [Online; accessed 08-Aug-2018]

• Landis JR, Koch GG (1977) The measurement of observer agreement for categorical data. Biometrics 33(1):159–174. http://www.jstor.org/stable/2529310

  MATH  Google Scholar 

• Leone M (2016) The economic benefits of puppet enterprise. Tech. rep., ESG. https://puppet.com/resources/analyst-report/the-economic-benefits-puppet-enterprise

• MacLeod L, Greiler M, Storey M, Bird C, Czerwonka J (2018) Code reviewing in the trenches: challenges and best practices. IEEE Softw 35 (4):34–42. https://doi.org/10.1109/MS.2017.265100500

  Google Scholar 

• Mann HB, Whitney DR (1947) On a test of whether one of two random variables is stochastically larger than the other. Ann Math Statist 18(1):50–60. http://www.jstor.org/stable/2236101

  MathSciNet  MATH  Google Scholar 

• Martin RC (2011) The clean coder: a code of conduct for professional programmers. Pearson Education

• McCune JT, Jeffrey (2011) Pro Puppet, 1st edn. Apress. https://doi.org/10.1007/978-1-4302-3058-8. https://www.springer.com/gp/book/9781430230571

• Meneely A, Williams L (2009) Secure open source collaboration: an empirical study of linus’ law. In: Proceedings of the 16th ACM conference on computer and communications security, ACM, New York, NY, USA, CCS ’09, pp 453–462. https://doi.org/10.1145/1653662.1653717

• Meneely A, Smith B, Williams L (2013) Validating software metrics: a spectrum of philosophies. ACM Trans Softw Eng Methodol 21(4):24:1–24:28. https://doi.org/10.1145/2377656.2377661

  Google Scholar 

• Menzies T, Greenwald J, Frank A (2007) Data mining static code attributes to learn defect predictors. IEEE T Software Eng 33(1):2–13. https://doi.org/10.1109/TSE.2007.256941

  Google Scholar 

• Morris K (2016) Infrastructure as code: managing servers in the cloud. “ O’Reilly Media, Inc.”

• Munaiah N, Kroh S, Cabrey C, Nagappan M (2017) Curating github for engineered software projects. Empirical Software Engineering pp 1–35. https://doi.org/10.1007/s10664-017-9512-6

• Munn Z, Peters MD, Stern C, Tufanaru C, McArthur A, Aromataris E (2018) Systematic review or scoping review? guidance for authors when choosing between a systematic or scoping review approach. BMC Med Res Methodol 18 (1):143

  Google Scholar 

• Oktaba P (2015) Keep your commits small. https://dzone.com/articles/keep-your-commits-small, [Online; accessed 08-Feb-2019]

• Ostrand TJ, Weyuker EJ, Bell RM (2004) Where the bugs are. In: Proceedings of the 2004 ACM SIGSOFT international symposium on software testing and analysis, ACM, New York, NY, USA, ISSTA ’04, pp 86–96. https://doi.org/10.1145/1007512.1007524

• Pinzger M, Nagappan N, Murphy B (2008) Can developer-module networks predict failures?. In: Proceedings of the 16th ACM SIGSOFT international symposium on foundations of software engineering, ACM, New York, NY, USA, SIGSOFT ’08/FSE-16, pp 2–12. https://doi.org/10.1145/1453101.1453105

• Rahman A, Williams L (2018) Characterizing defective configuration scripts used for continuous deployment. In: 2018 IEEE 11th International conference on software testing, verification and validation (ICST), pp 34–45. https://doi.org/10.1109/ICST.2018.00014

• Rahman A, Williams L (2019) Source code properties of defective infrastructure as code scripts. Information and Software Technology. https://doi.org/10.1016/j.infsof.2019.04.013, http://www.sciencedirect.com/science/article/pii/S0950584919300965

• Rahman A, Partho A, Morrison P, Williams L (2018) What questions do programmers ask about configuration as code?. In: Proceedings of the 4th international workshop on rapid continuous software engineering, ACM, New York, NY, USA, RCoSE ’18, pp 16–22. https://doi.org/10.1145/3194760.3194769

• Rahman A, Parnin C, Williams L (2019) The seven sins: Security smells in infrastructure as code scripts. In: Proceedings of the 41st international conference on software engineering, IEEE Press, Piscataway, NJ, USA, ICSE ’19, pp 164–175. https://doi.org/10.1109/ICSE.2019.00033

• Rahman A, Farhana E, Parnin C, Williams L (2020) Gang of eight: a defect taxonomy for infrastructure as code scripts. In: Proceedings of the 42nd international conference on software engineering, ICSE ’20, to appear

• Rahman F, Devanbu P (2013a) How, and why, process metrics are better. In: Proceedings of the 2013 international conference on software engineering, IEEE Press, Piscataway, NJ, USA, ICSE ’13, pp 432–441. http://dl.acm.org/citation.cfm?id=2486788.2486846

• Rahman F, Devanbu P (2013b) How, and why, process metrics are better. In: Proceedings of the 2013 international conference on software engineering, IEEE press, Piscataway, NJ, USA, ICSE ’13, pp 432–441. http://dl.acm.org/citation.cfm?id=2486788.2486846

• Rigby PC, German DM, Storey MA (2008) Open source software peer review practices: a case study of the apache server. In: Proceedings of the 30th international conference on software engineering, ACM, New York, NY, USA, ICSE ’08, pp 541–550. https://doi.org/10.1145/1368088.1368162

• Romano J, Kromrey J, Coraggio J, Skowronek J (2006) Appropriate statistics for ordinal level data: Should we really be using t-test and Cohen’sd for evaluating group differences on the NSSE and other surveys?. In: Annual meeting of the florida association of institutional research, pp 1–3

• Saldana J (2015) The coding manual for qualitative researchers. Sage

• Shambaugh R, Weiss A, Guha A (2016) Rehearsal: a configuration verification tool for puppet. SIGPLAN Not 51(6):416–430. https://doi.org/10.1145/2980983.2908083

  Google Scholar 

• Sharma T, Fragkoulis M, Spinellis D (2016) Does your configuration code smell?. In: Proceedings of the 13th international conference on mining software repositories, ACM, New York, NY, USA, MSR ’16, pp 189–200. https://doi.org/10.1145/2901739.2901761

• Shihab E, Jiang ZM, Adams B, Hassan AE, Bowerman R (2011) Prioritizing the creation of unit tests in legacy software systems. Software Pract Exper 41(10):1027–1048. https://doi.org/10.1002/spe.1053

  Google Scholar 

• Smith E, Loftin R, Murphy-Hill E, Bird C, Zimmermann T (2013) Improving developer participation rates in surveys. In: 2013 6th International workshop on cooperative and human aspects of software engineering (CHASE), pp 89–92. https://doi.org/10.1109/CHASE.2013.6614738

• Sullivan GM, Feinn R (2012) Using effect size-or why the p value is not enough. J Grad Med Educ 4(3):279–282. https://doi.org/10.4300/JGME-D-12-00156.1

  Google Scholar 

• Tan PN, Steinbach M, Kumar V (2005) Introduction to data mining, 1st. Addison-Wesley Longman Publishing Co., Inc., Boston

  Google Scholar 

• Tantithamthavorn C, McIntosh S, Hassan AE, Matsumoto K (2016) Automated parameter optimization of classification techniques for defect prediction models. In: Proceedings of the 38th international conference on software engineering, ACM, New York, NY, USA, ICSE ’16, pp 321–332 https://doi.org/10.1145/2884781.2884857

• Tantithamthavorn C, McIntosh S, Hassan AE, Matsumoto K (2017) An empirical comparison of model validation techniques for defect prediction models. IEEE Trans Softw Eng 43(1):1–18. https://doi.org/10.1109/TSE.2016.2584050

  Google Scholar 

• Tosun A, Bener A, Turhan B, Menzies T (2010) Practical considerations in deploying statistical methods for defect prediction: a case study within the turkish telecommunications industry. Inf Softw Technol 52(11):1242–1257. https://doi.org/10.1016/j.infsof.2010.06.006

  Google Scholar 

• Tufano M, Bavota G, Poshyvanyk D, Di Penta M, Oliveto R, De Lucia A (2017) An empirical study on developer-related factors characterizing fix-inducing commits. J Softw Evol Proc 29(1):e1797. https://onlinelibrary.wiley.com/doi/abs/10.1002/smr.1797

  Google Scholar 

• Turhan B, Kocak G, Bener A (2009) Data mining source code for locating software bugs: a case study in telecommunication industry. Expert Syst Appl 36(6):9986–9990. https://doi.org/10.1016/j.eswa.2008.12.028. http://www.sciencedirect.com/science/article/pii/S0957417408009275

  Google Scholar 

• Turnbull J (2007) Pulling strings with puppet: automated system administration done right. Apress

• van der Bent E, Hage J, Visser J, Gousios G (2018) How good is your puppet? an empirically defined and validated quality model for puppet. In: 2018 IEEE 25th international conference on software analysis, evolution and reengineering (SANER), pp 164–174. https://doi.org/10.1109/SANER.2018.8330206

• Van Wyk E, Krishnan L, Bodin D, Schwerdfeger A (2007) Attribute grammar-based language extensions for java. In: Proceedings of the 21st european conference on object-oriented programming, springer-verlag, Berlin, Heidelberg, ECOOP’07, pp 575–599. http://dl.acm.org/citation.cfm?id=2394758.2394796

• Voelter M (2013) DSL engineering: designing implementing and using domain-specific languages. CreateSpace Independent Publishing Platform, USA

  Google Scholar 

• Weinberg GM (1992) Quality software management (vol. 1): systems thinking. Dorset House Publishing Co., Inc., New York

  MATH  Google Scholar 

• Weiss A, Guha A, Brun Y (2017) Tortoise: Interactive system configuration repair. In: Proceedings of the 32Nd IEEE/ACM international conference on automated software engineering, IEEE press, Piscataway, NJ, USA, ASE 2017, pp 625–636. http://dl.acm.org/citation.cfm?id=3155562.3155641

Download references