Nothing Special   »   [go: up one dir, main page]

skip to main content
article

An empirical study of architecting for continuous delivery and deployment

Published: 01 June 2019 Publication History

Abstract

Recently, many software organizations have been adopting Continuous Delivery and Continuous Deployment (CD) practices to develop and deliver quality software more frequently and reliably. Whilst an increasing amount of the literature covers different aspects of CD, little is known about the role of software architecture in CD and how an application should be (re-) architected to enable and support CD. We have conducted a mixed-methods empirical study that collected data through in-depth, semi-structured interviews with 21 industrial practitioners from 19 organizations, and a survey of 91 professional software practitioners. Based on a systematic and rigorous analysis of the gathered qualitative and quantitative data, we present a conceptual framework to support the process of (re-) architecting for CD. We provide evidence-based insights about practicing CD within monolithic systems and characterize the principle of "small and independent deployment units" as an alternative to the monoliths. Our framework supplements the architecting process in a CD context through introducing the quality attributes (e.g., resilience) that require more attention and demonstrating the strategies (e.g., prioritizing operations concerns) to design operations-friendly architectures. We discuss the key insights (e.g., monoliths and CD are not intrinsically oxymoronic) gained from our study and draw implications for research and practice.

References

[1]
2015 State of DevOps Report (2015) Available at:goo.gl/oJ2Tvi [Last accessed: 5 October 2015].
[2]
Adams B, McIntosh S (2016) Modern release engineering in a nutshell - why researchers should care. Presented at the IEEE 23rd International Conference on Software Analysis, Evolution, and Reengineering (SANER), 14-18 March, 2016.
[3]
Adrian F (1996) Response bias, social desirability and dissimulation. Personal Individ Differ 7(3):385-400.
[4]
Andre van Hoorn PJ, Leitner P, Weber I (2017) Report from GI-Dagstuhl seminar 16394: software performance engineering in the DevOps world. Available at: https://arxiv.org/abs/1709.08951.
[5]
Arun G (2015) Microservices, monoliths, and NoOps, Available at: goo.gl/zou2x3 [Last accessed: 8 November 2016].
[6]
Balalaie A, Heydarnoori A, Jamshidi P (2016) Microservices architecture enables DevOps: migration to a cloudnative architecture. IEEE Softw 33(3):42-52.
[7]
Bass L (2017) The software architect and DevOps. IEEE Softw 35(1):8-10.
[8]
Bass L, Jeffery R, Wada H, Weber I, Liming Z (2013) Eliciting operations requirements for applications. In 1st International Workshop on Release Engineering (RELENG), pp. 5-8.
[9]
Bass L, Weber I, Zhu L (2015) DevOps: a software architect's perspective. Addison-Wesley Professional.
[10]
Beijer P, de Klerk T (2010) IT Architecture - Essential Practice for IT Business Solutions. Lulu.com.
[11]
Bellomo S, Ernst N, Nord R, Kazman R (2014) Toward design decisions to enable deployability: empirical study of three projects reaching for the continuous delivery holy grail. Presented at the 44th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN), 23-26 June 2014.
[12]
Bosch J (2016) Speed, data, and ecosystems: the future of software engineering. Software, IEEE 33(1):82-88.
[13]
Brandolini A (2013) Introducing event storming, Available at: goo.gl/GMzzDv [Last accessed: 8 July 2017].
[14]
Brandtner M, Giger E, Gall H (2015) SQA-mashup: a mashup framework for continuous integration. Inf Softw Technol 65:97-113.
[15]
Braun V, Clarke V (2006) Using thematic analysis in psychology. Qual Res Psychol 3(2):77-101.
[16]
Brown A (2015) What's the best team structure for DevOps success? Available at: goo.gl/3Z11og [Last accessed: 13 September 2017].
[17]
Capilla R, Jansen A, Tang A, Avgeriou P, Babar MA (2016) 10 years of software architecture knowledge management: practice and future. J Syst Softw 116:191-205.
[18]
Chen L (2015a) Continuous delivery: huge benefits, but challenges too. IEEE Softw 32(2):50-54.
[19]
Chen L (2015b) Towards architecting for continuous delivery. In 12th Working IEEE/IFIP Conference on Software Architecture (WICSA), pp. 131-134.
[20]
Chris R. (2014). Pattern: monolithic architecture, Available at: goo.gl/royZ7i [Last accessed: 4 November 2016].
[21]
Cito J, Leitner P, Fritz T, Gall HC (2015) The making of cloud applications: an empirical study on software development for the cloud. In 10th Joint Meeting on Foundations of Software Engineering, Bergamo, Italy, pp. 393-403: ACM.
[22]
Claps GG, Berntsson Svensson R, Aurum A (2015) On the journey to continuous deployment: technical and social challenges along the way. Inf Softw Technol 57:21-31.
[23]
Conway ME (1968) How do committees invent? Datamation 14(5).
[24]
Cruzes DS, Dyba T (2011) Recommended steps for thematic synthesis in software engineering. In International Symposium on Empirical Software Engineering and Measurement (ESEM), pp. 275-284.
[25]
Debbiche A, Dienér M, Berntsson Svensson R (2014) Challenges when adopting continuous integration: a case study. Cham, pp. 17-32: Springer International Publishing.
[26]
Dooley PM (2015) The intersection of DevOps and ITIL, Available at: goo.gl/tqg2hD [Last accessed: 14 June 2017]. Global Knowledge.
[27]
Dragoni N et al (2017) Microservices: yesterday, today, and tomorrow. In: Mazzara M, Meyer B (eds) Present and ulterior software engineering. Springer International Publishing, Cham, pp 195-216.
[28]
Easterbrook S, Singer J, Storey M-A, Damian D (2008) Selecting empirical methods for software engineering research. In: Shull F, Singer J, Sjøberg DIK (eds) Guide to advanced empirical software engineering. Springer London, London, pp 285-311.
[29]
Elbaum S, Rothermel G, Penix J (2014) Techniques for improving regression testing in continuous integration development environments. In 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering, Hong Kong, China, pp. 235-245: ACM.
[30]
Erder M, Pureur P (2015) Continuous architecture: sustainable architecture in an agile and cloud-centric world. Morgan Kaufmann.
[31]
Ernst N, Klein J, Mathew G, Menzies T (2017) Using stakeholder preferences to make better architecture decisions. In 2017 IEEE International Conference on Software Architecture Workshops (ICSAW), pp. 133-136.
[32]
Evans E (2004) Domain-driven design: tackling complexity in the heart of softwareT. Addison-Wesley Professional.
[33]
Fitzgerald B, Stol K-J (2017) Continuous software engineering: a roadmap and agenda. J Syst Softw 123.
[34]
Ford N (2016) Architecture is abstract until operationalized, Available at: goo.gl/HorpbH [Last accessed: 21 February 2016].
[35]
Ford N, Parsons R (2016) Microservices as an Evolutionary Architecture, Available at: goo.gl/aysZvA [Last accessed: 20 March 2016].
[36]
Fowler M (2013) Continuous Delivery, Available at: goo.gl/gB8sTj [Last accessed: 21 October 2015].
[37]
Fowler M (2015a) Continuous Integration, Available at: goo.gl/5EhHR7 [Last accessed: 21 October 2015].
[38]
Fowler M (2015b) Microservice Premium, Available at: goo.gl/3WVKsn [Last accessed: 31 October 2016].
[39]
Fowler Jr FJ (2013) Survey research methods. Sage publications.
[40]
Gabhart K (2014) Resilient IT Through DevOps, Available at: https://puppet.com/blog/resilient-it-through-devops/ [Last accessed: 1 July 2017].
[41]
Garousi V, Felderer M, Mäntylä MV (2016) The need for multivocal literature reviews in software engineering: complementing systematic literature reviews with grey literature. In 20th International Conference on Evaluation and Assessment in Software Engineering, Limerick, Ireland, pp. 1-6, 2916008: ACM.
[42]
Gibson S (2016) Monoliths are bad design... and you know it, Available at: goo.gl/xVEbSE [Last accessed: 4 March 2016].
[43]
Gitlevich V, Evans E (2015) What is Domain-driven design? Available at: goo.gl/S3zMSR [Last accessed: 21 June 2016].
[44]
Goodman LA (1961) Snowball sampling. Ann Math Stat 32(1):148-170.
[45]
Gousios G, Storey M-A, Bacchelli A (2016) Work practices and challenges in pull-based development: the contributor's perspective. In 38th International Conference on Software Engineering, Austin, Texas, pp. 285-296: ACM.
[46]
Hasselbring W, Steinacker G (2017) Microservice architectures for scalability, agility and reliability in e-commerce. In 2017 IEEE International Conference on Software Architecture Workshops (ICSAW), pp. 243-246.
[47]
Hilton M, Nelson N, Tunnell T, Marinov D, Dig D (2017) Trade-offs in continuous integration: assurance, security, and flexibility. Presented at the Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering, Paderborn, Germany.
[48]
Hohpe G, Ozkaya I, Zdun U, Zimmermann O (2016) The software architect's role in the digital age. IEEE Softw 33(6):30-39.
[49]
Hove SE, Anda B (2005) Experiences from conducting semi-structured interviews in empirical software engineering research. In 11th IEEE International Software Metrics Symposium, p. 23, 1092163: IEEE Computer Society.
[50]
Humble J (2011) Organize software delivery around outcomes, not roles: continuous delivery and cross-functional teams, Available at: goo.gl/MnFtJN [Last accessed: 10 August 2016].
[51]
Humble J (2007) Continuous delivery vs continuous deployment, Available at: goo.gl/qE1JoM [Last accessed: 1 March 2016].
[52]
Humble J, Farley D (2010) Continuous delivery: reliable software releases through build, test, and deployment automation, 1st ed. Addison-Wesley Professional.
[53]
ISO/IEC/IEEE Systems and software engineering - Architecture description (2011) ISO/IEC/IEEE 42010: 2011(E) (Revision of ISO/IEC 42010:2007 and IEEE Std 1471-2000), pp. 1-46.
[54]
Jiang B, Zhang Z, Chan WK, Tse TH, Chen TY (2012) How well does test case prioritization integrate with statistical fault localization? Information and Software Technology, vol. 54, no. 7, pp. 739-758, 2012/07/01/2012.
[55]
Jong Md, Deursen Av, Cleve A (2017) Zero-downtime SQL database schema evolution for continuous deployment. In 39th International Conference on Software Engineering: Software Engineering in Practice Track, Buenos Aires, Argentina, pp. 143-152: IEEE Press.
[56]
Kim EH, Na JC, Ryoo SM (2009) Test automation framework for implementing continuous integration. In Sixth International Conference on Information Technology: New Generations, pp. 784-789.
[57]
Kitchenham BA, Pfleeger SL (2008) Personal opinion surveys. In: Shull F, Singer J, Sjøberg DIK (eds) Guide to advanced empirical software engineering. Springer London, London, pp 63-92.
[58]
Kitchenham B, Pickard L, Pfleeger SL (1995) Case studies for method and tool evaluation. IEEE Softw 12(4):52-62.
[59]
Laukkanen E, Lehtinen TOA, Itkonen J, Paasivaara M, Lassenius C (2016) Bottom-up adoption of continuous delivery in a stage-gate managed software organization. In 10th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement, Ciudad Real, Spain, pp. 1-10: ACM.
[60]
Laukkanen E, Itkonen J, Lassenius C (2017) Problems, causes and solutions when adopting continuous delivery--a systematic literature review. Inf Softw Technol 82:55-79.
[61]
Leppanen M et al (2015) The highways and country roads to continuous deployment. IEEE Softw 32(2):64-72.
[62]
Lewis J, Fowler M (2010) Microservices: a definition of this new architectural term, Available at: goo.gl/me6tp5 [Last accessed: 05 January 2016].
[63]
Luke E, Prince S (2016) No one agrees how to define CI or CD. Available at: goo.gl/Z8Qonq [Last accessed: 1 August 2016].
[64]
Mäkinen S et al (2016) Improving the delivery cycle: a multiple-case study of the toolchains in Finnish software intensive enterprises. Inf Softw Technol 80:175-194.
[65]
Manotas I, et al (2016) An empirical study of practitioners' perspectives on green software engineering. In 38th International Conference on Software Engineering, Austin, Texas, pp. 237-248: ACM.
[66]
Mäntylä M, Adams B, Khomh F, Engström E, Petersen K (2015) On rapid releases and software testing: a case study and a semi-systematic literature review," (in English). Empir Softw Eng 20(5):1384-1425.
[67]
Mårtensson T, Ståhl D, Bosch J (2017) Continuous integration impediments in large-scale industry projects. In 2017 IEEE International Conference on Software Architecture (ICSA), pp. 169-178.
[68]
Meade AW, Craig SB (2012) Identifying careless responses in survey data. Psychol Methods 17(3):437.
[69]
Meho LI (2006) E-mail interviewing in qualitative research: a methodological discussion: research articles. J Am Soc Inf Sci Technol 57(10):1284-1295.
[70]
Murphy-Hill E, Zimmermann T, Bird C, Nagappan N (2015) The design space of bug fixes and how developers navigate it. IEEE Trans Softw Eng 41(1):65-81.
[71]
Newman S (2015) Building microservices. O'Reilly Media, Inc.
[72]
Northrop L (2015) Trends and new directions in software architecture, Available at: goo.gl/ZAnkQp.
[73]
Palinkas LA, Horwitz SM, Green CA, Wisdom JP, Duan N, Hoagwood K (2015) Purposeful sampling for qualitative data collection and analysis in mixed method implementation research. Adm Policy Ment Health Ment Health Serv Res 42(5):533-544.
[74]
Pauw Td (2017) Feature Branching is Evil, Available at: https://speakerdeck.com/tdpauw/xp2017-feature-branching-is-evil/ [Last accessed: 27 May 2017].
[75]
Prewer L (2015) Smoothing the continuous delivery path - a tale of two teams, Available at: goo.gl/1oqjsP [Last accessed: 2 October 2016].
[76]
Prince S (2016) The product managers' guide to continuous delivery and DevOps, Available at: goo.gl/D8mGkH [Last accessed: 2 November 2016].
[77]
Rahman MT, Querel LP, Rigby PC, Adams B (2016) Feature toggles: practitioner practices and a case study. In 2016 IEEE/ACM 13th Working Conference on Mining Software Repositories (MSR), pp. 201-211.
[78]
Rodríguez P et al (2017) Continuous deployment of software intensive products and services: a systematic mapping study. J Syst Softw 123:263-291.
[79]
Savor T, Douglas M, Gentili M, Williams L, Beck K, Stumm M (2016) Continuous deployment at Facebook and OANDA. In 38th International Conference on Software Engineering Companion, Austin, Texas, pp. 21-30: ACM.
[80]
Schauenberg D (2014) Development, deployment and collaboration at Etsy, Available at: goo.gl/umGTM2 [Last accessed: 1 September 2017].
[81]
Schermann G, Cito J, Leitner P, Zdun U, Gall H (2016) An empirical study on principles and practices of continuous delivery and deployment. PeerJ Preprints 4:e1889v1.
[82]
Seaman CB (1999) Qualitative methods in empirical studies of software engineering. IEEE Trans Softw Eng 25(4):557-572.
[83]
Self-Contained Systems: Assembling Software from Independent Systems (2014) Available at: http://scsarchitecture.org/ [Last accessed: 1 June 2017].
[84]
Shahin M, Babar MA, Zhu L (2016) The intersection of continuous deployment and architecting process: practitioners' perspectives. In ACM/IEEE International Symposium on Empirical Software Engineering and Measurement, Ciudad Real, Spain, pp. 1-10: ACM.
[85]
Shahin M, Babar MA, Zhu L (2017a) Continuous integration, delivery and deployment: a systematic review on approaches, tools, challenges and practices. IEEE Access 5:3909-3943.
[86]
Shahin M, Babar MA, Zahedi M, Zhu L (2017b) Beyond continuous delivery: an empirical investigation of continuous deployment challenges. In 11th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM), Toronto, Canada: IEEE.
[87]
Shahin M, Zahedi M, Babar MA, Zhu L (2017c) Adopting continuous delivery and deployment: impacts on team structures, collaboration and responsibilities. In 21st International Conference on Evaluation and Assessment in Software Engineering, Karlskrona, Sweden, pp. 384-393: ACM.
[88]
Skelton M (2016) How to break apart a monolithic system safely without destroying your team, Available at: goo.gl/pqBVm2 [Last accessed: 4 November 2016].
[89]
Skelton M, O'Dell C (2016) Continuous delivery with windows and. NET. O'Reilly.
[90]
Sokhan B (2016) Domain driven design for services architecture, Available at: goo.gl/ftCLnR [Last accessed: 10 January 2016].
[91]
Ståhl D, Bosch J (2014) Modeling continuous integration practice differences in industry software development. J Syst Softw 87:48-59.
[92]
Suneja S et al (2017) Safe inspection of live virtual machines. In 13th ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments, Xi'an, China, pp. 97-111, 3050766: ACM.
[93]
Thiele A (2014) Continuous delivery: an easy must-have for agile development, Available at: goo.gl/ymgCSq [Last accessed: 10 July 2016].
[94]
Vishal N (2015) Architecting for continuous delivery, Available at: goo.gl/zWA5kT [Last accessed: 15 March 2016].
[95]
Wallgren A (2015) Continuous delivery of microservices: patterns and processes, Available at: goo.gl/Yk6ddH [Last accessed: 10 February 2018].
[96]
Waterman MG (2014) Reconciling agility and architecture: a theory of agile architecture. PhD Thesis, Victoria University of Wellington.
[97]
What Team Structure is Right for DevOps to Flourish (2017) Available at: goo.gl/KM6N3p [Last accessed: 24 September 2017].
[98]
Wnuk K (2017) Involving relevant stakeholders into the decision process about software components. In 2017 IEEE International Conference on Software Architecture Workshops (ICSAW), pp. 129-132.
[99]
Woods E (2016) Operational: the forgotten architectural view. IEEE Softw 33(3):20-23.
[100]
Yaniv Y (2014) Closing the gap between database continuous delivery and code continuous delivery, Available at: goo.gl/mERZcV [Last accessed: 21 August 2016].

Cited By

View all
  • (2024)Paving a Path for a Combined Family of Feature Toggle and Configuration Option ResearchACM Transactions on Software Engineering and Methodology10.1145/367255533:7(1-27)Online publication date: 14-Jun-2024
  • (2024)On the Understandability of MLOps System ArchitecturesIEEE Transactions on Software Engineering10.1109/TSE.2024.336748850:5(1015-1039)Online publication date: 20-Feb-2024
  • (2024)The never-ending story–How companies transition to and sustain continuous software engineering practicesJournal of Systems and Software10.1016/j.jss.2024.112056213:COnline publication date: 1-Jul-2024
  • Show More Cited By

Index Terms

  1. An empirical study of architecting for continuous delivery and deployment
    Index terms have been assigned to the content through auto-classification.

    Recommendations

    Comments

    Please enable JavaScript to view thecomments powered by Disqus.

    Information & Contributors

    Information

    Published In

    cover image Empirical Software Engineering
    Empirical Software Engineering  Volume 24, Issue 3
    Jun 2019
    592 pages

    Publisher

    Kluwer Academic Publishers

    United States

    Publication History

    Published: 01 June 2019

    Author Tags

    1. Continuous delivery
    2. Continuous deployment
    3. DevOps
    4. Empirical study
    5. Software architecture

    Qualifiers

    • Article

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)0
    • Downloads (Last 6 weeks)0
    Reflects downloads up to 20 Nov 2024

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)Paving a Path for a Combined Family of Feature Toggle and Configuration Option ResearchACM Transactions on Software Engineering and Methodology10.1145/367255533:7(1-27)Online publication date: 14-Jun-2024
    • (2024)On the Understandability of MLOps System ArchitecturesIEEE Transactions on Software Engineering10.1109/TSE.2024.336748850:5(1015-1039)Online publication date: 20-Feb-2024
    • (2024)The never-ending story–How companies transition to and sustain continuous software engineering practicesJournal of Systems and Software10.1016/j.jss.2024.112056213:COnline publication date: 1-Jul-2024
    • (2023)From DevOps to DevSecOps is not enough. CyberDevOps: an extreme shifting-left architecture to bring cybersecurity within software security lifecycle pipelineSoftware Quality Journal10.1007/s11219-023-09619-331:2(619-654)Online publication date: 26-Apr-2023
    • (2023)DevOpsJournal of Software: Evolution and Process10.1002/smr.253435:12Online publication date: 20-Jan-2023
    • (2023)A qualitative study of architectural design issues in DevOpsJournal of Software: Evolution and Process10.1002/smr.237935:5Online publication date: 25-Apr-2023
    • (2022)Towards a theory of shared understanding of non-functional requirements in continuous software engineeringProceedings of the ACM/IEEE 44th International Conference on Software Engineering: Companion Proceedings10.1145/3510454.3517069(300-304)Online publication date: 21-May-2022
    • (2022)Towards cost-benefit evaluation for continuous software engineering activitiesEmpirical Software Engineering10.1007/s10664-022-10191-w27:6Online publication date: 16-Aug-2022
    • (2022)A comprehensive overview of software product management challengesEmpirical Software Engineering10.1007/s10664-022-10134-527:5Online publication date: 1-Sep-2022
    • (2021)Iter8Proceedings of the ACM Symposium on Cloud Computing10.1145/3472883.3486984(289-304)Online publication date: 1-Nov-2021
    • Show More Cited By

    View Options

    View options

    Login options

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media