research-article

Releasing fast and slow: an exploratory case study at ING

Authors:

Ayushi Rastogi,

Hennie Huijgens,

Arie van Deursen,

Georgios GousiosAuthors Info & Claims

ESEC/FSE 2019: Proceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering

Pages 785 - 795

https://doi.org/10.1145/3338906.3338978

Published: 12 August 2019 Publication History

Abstract

The appeal of delivering new features faster has led many software projects to adopt rapid releases. However, it is not well understood what the effects of this practice are. This paper presents an exploratory case study of rapid releases at ING, a large banking company that develops software solutions in-house, to characterize rapid releases. Since 2011, ING has shifted to a rapid release model. This switch has resulted in a mixed environment of 611 teams releasing relatively fast and slow. We followed a mixed-methods approach in which we conducted a survey with 461 participants and corroborated their perceptions with 2 years of code quality data and 1 year of release delay data. Our research shows that: rapid releases are more commonly delayed than their non-rapid counterparts, however, rapid releases have shorter delays; rapid releases can be beneficial in terms of reviewing and user-perceived quality; rapidly released software tends to have a higher code churn, a higher test coverage and a lower average complexity; challenges in rapid releases are related to managing dependencies and certain code aspects, e.g., design debt.

References

[1]

Chandrasekar Subramaniam, Ravi Sen, and Matthew L Nelson. Determinants of open source software project success: A longitudinal study. Decision Support Systems, 46(2):576–585, 2009.

Digital Library

[2]

Mika V Mäntylä, Bram Adams, Foutse Khomh, Emelie Engström, and Kai Petersen. On rapid releases and software testing: a case study and a semi-systematic literature review. Empirical Software Engineering, 20(5):1384–1425, 2015.

Digital Library

[3]

Foutse Khomh, Tejinder Dhaliwal, Ying Zou, and Bram Adams. Do faster releases improve software quality?: an empirical case study of mozilla firefox. In Proceedings of the 9th IEEE Working Conference on Mining Software Repositories, pages 179–188. IEEE Press, 2012.

Digital Library

[4]

Bram Adams and Shane McIntosh. Modern release engineering in a nutshell– why researchers should care. In Software Analysis, Evolution, and Reengineering (SANER), 2016 IEEE 23rd International Conference on, volume 5, pages 78–90. IEEE, 2016.

[5]

H. Kniberg. Spotify engineering culture. Available at https:// labs.spotify.com/ 2014/ 03/ 27/ spotify-engineering-culture-part-1, 2014.

[6]

C Rossi. Moving to mobile: The challenges of moving from web to mobile releases. Keynote at RELENG, 2014.

[7]

Kent Beck and Erich Gamma. Extreme programming explained: embrace change. addison-wesley professional, 2000.

Digital Library

[8]

Daniel Alencar da Costa, Shane McIntosh, Uirá Kulesza, and Ahmed E Hassan. The impact of switching to a rapid release cycle on the integration delay of addressed issues-an empirical study of the mozilla firefox project. In Mining Software Repositories (MSR), 2016 IEEE/ACM 13th Working Conference on, pages 374–385. IEEE, 2016.

Digital Library

[9]

Michele Tufano, Fabio Palomba, Gabriele Bavota, Rocco Oliveto, Massimiliano Di Penta, Andrea De Lucia, and Denys Poshyvanyk. When and why your code starts to smell bad. In Proceedings of the 37th International Conference on Software Engineering-Volume 1, pages 403–414. IEEE Press, 2015.

Digital Library

[10]

Julia Rubin and Martin Rinard. The challenges of staying together while moving fast: An exploratory study. In Software Engineering (ICSE), 2016 IEEE/ACM 38th International Conference on, pages 982–993. IEEE, 2016.

Digital Library

[11]

ING. 2017 Annual ING Group N.V. Available at https:// www.ing.com/ Aboutus/ Annual-reporting-suite/ Annual-Report/ 2017-Annual-Report-Empoweringpeople.htm, 2018.

[12]

Carmine Vassallo, Fiorella Zampetti, Daniele Romano, Moritz Beller, Annibale Panichella, Massimiliano Di Penta, and Andy Zaidman. Continuous delivery practices in a large financial organization. In Software Maintenance and Evolution (ICSME), 2016 IEEE International Conference on, pages 519–528. IEEE, 2016.

[13]

Per Runeson and Martin Höst. Guidelines for conducting and reporting case study research in software engineering. Empirical software engineering, 14(2):131, 2009.

Digital Library

[14]

Carolyn B. Seaman. Qualitative methods in empirical studies of software engineering. IEEE Transactions on software engineering, 25(4):557–572, 1999.

Digital Library

[15]

John W Creswell and J David Creswell. Research design: Qualitative, quantitative, and mixed methods approaches. Sage publications, 2017.

[16]

Colin Robson. Real world research: A resource for social scientists and practitionerresearchers. Wiley-Blackwell, 2002.

[17]

Uwe Flick. An introduction to qualitative research. Sage Publications Limited, 2018.

[18]

Juliet M Corbin and Anselm Strauss. Grounded theory research: Procedures, canons, and evaluative criteria. Qualitative sociology, 13(1):3–21, 1990.

[19]

Zengyang Li, Paris Avgeriou, and Peng Liang. A systematic mapping study on technical debt and its management. Journal of Systems and Software, 101:193–220, 2015.

Digital Library

[20]

Thomas J McCabe. A complexity measure. IEEE Transactions on software Engineering, 2(4):308–320, 1976.

Digital Library

[21]

Robert J Eisenberg. A threshold based approach to technical debt. ACM SIGSOFT Software Engineering Notes, 37(2):1–6, 2012.

Digital Library

[22]

Henry B Mann and Donald R Whitney. On a test of whether one of two random variables is stochastically larger than the other. The annals of mathematical statistics, pages 50–60, 1947.

[23]

Samuel Sanford Shapiro and Martin B Wilk. An analysis of variance test for normality (complete samples). Biometrika, 52(3/4):591–611, 1965.

[24]

C Bonferroni. Teoria statistica delle classi e calcolo delle probabilita. Pubblicazioni del R Istituto Superiore di Scienze Economiche e Commericiali di Firenze, 8:3–62, 1936.

[25]

Norman Cliff. Dominance statistics: Ordinal analyses to answer ordinal questions. Psychological bulletin, 114(3):494, 1993.

[26]

Noureddine Kerzazi and Foutse Khomh. Factors impacting rapid releases: an industrial case study. In Proceedings of the 8th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement, page 61. ACM, 2014.

Digital Library

[27]

Stephany Bellomo, Robert L Nord, and Ipek Ozkaya. A study of enabling factors for rapid fielding: combined practices to balance speed and stability. In Proceedings of the 2013 International Conference on Software Engineering, pages 982–991. IEEE Press, 2013.

Digital Library

[28]

Sandy Clark, Michael Collis, Matt Blaze, and Jonathan M Smith. Moving targets: Security and rapid-release in firefox. In Proceedings of the 2014 ACM SIGSAC Conference on Computer and Communications Security, pages 1256–1266. ACM, 2014.

Digital Library

[29]

Alexandre Decan, Tom Mens, and Philippe Grosjean. An empirical comparison of dependency network evolution in seven software packaging ecosystems. Empirical Software Engineering, pages 1–36, 2018.

Digital Library

[30]

Alexandre Decan, Tom Mens, and Maëlick Claes. An empirical comparison of dependency issues in oss packaging ecosystems. In 2017 IEEE 24th International Conference on Software Analysis, Evolution and Reengineering (SANER), pages 2–12. IEEE, 2017.

[31]

Joseph Hejderup, Arie van Deursen, and Georgios Gousios. Software ecosystem call graph for dependency management. In Proceedings of the 40th International Conference on Software Engineering: New Ideas and Emerging Results, pages 101– 104. ACM, 2018.

Digital Library

[32]

Hadi Hemmati, Zhihan Fang, and Mika V Mantyla. Prioritizing manual test cases in traditional and rapid release environments. In Software Testing, Verification and Validation (ICST), 2015 IEEE 8th International Conference on, pages 1–10. IEEE, 2015.

[33]

Martin Michlmayr, Brian Fitzgerald, and Klaas-Jan Stol. Why and how should open source projects adopt time-based releases? IEEE Software, 32(2):55–63, 2015.

Digital Library

[34]

Kai Petersen and Claes Wohlin. The effect of moving from a plan-driven to an incremental software development approach with agile practices. Empirical Software Engineering, 15(6):654–693, 2010.

Digital Library

[35]

Günther Ruhe and Des Greer. Quantitative studies in software release planning under risk and resource constraints. In Empirical Software Engineering, 2003. ISESE 2003. Proceedings. 2003 International Symposium on, pages 262–270. IEEE, 2003.

Digital Library

[36]

Rashmi Popli and Naresh Chauhan. Cost and effort estimation in agile software development. In Optimization, Reliabilty, and Information Technology (ICROIT), 2014 International Conference on, pages 57–61. IEEE, 2014.

[37]

Adrian Furnham. Response bias, social desirability and dissimulation. Personality and individual differences, 7(3):385–400, 1986.

[38]

Andrew Begel and Nachiappan Nagappan. Usage and perceptions of agile software development in an industrial context: An exploratory study. In Empirical Software Engineering and Measurement, 2007. ESEM 2007. First International Symposium on, pages 255–264. IEEE, 2007.

Digital Library

[39]

Jingyue Li, Nils B Moe, and Tore Dybå. Transition from a plan-driven process to scrum: a longitudinal case study on software quality. In Proceedings of the 2010 ACM-IEEE international symposium on empirical software engineering and measurement, page 13. ACM, 2010.

Digital Library

[40]

Matthias Marschall. Transforming a six month release cycle to continuous flow. In Agile Conference (AGILE), 2007, pages 395–400. IEEE, 2007.

Digital Library

[41]

Daniel Alencar da Costa, Surafel Lemma Abebe, Shane McIntosh, Uirá Kulesza, and Ahmed E Hassan. An empirical study of delays in the integration of addressed issues. In Software Maintenance and Evolution (ICSME), 2014 IEEE International Conference on, pages 281–290. IEEE, 2014.

Digital Library

[42]

Zadia Codabux and Byron Williams. Managing technical debt: An industrial case study. In Proceedings of the 4th International Workshop on Managing Technical Debt, pages 8–15. IEEE Press, 2013.

Digital Library

[43]

Richard Torkar, Pau Minoves, and Janina Garrigós. Adopting free/libre/open source software practices, techniques and methods for industrial use. Journal of the AIS, 12(1), 2011.

[44]

Foutse Khomh, Bram Adams, Tejinder Dhaliwal, and Ying Zou. Understanding the impact of rapid releases on software quality. Empirical Software Engineering, 20(2):336–373, 2015.

Digital Library

[45]

Adam Porter, Cemal Yilmaz, Atif M Memon, Arvind S Krishna, Douglas C Schmidt, and Aniruddha Gokhale. Techniques and processes for improving the quality and performance of open-source software. Software Process: Improvement and Practice, 11(2):163–176, 2006.

Cited By

Kanita N(2024)Modèles explicatifs de la production de la dette technique dans les contextes Agile et DevOps : une revue de la littérature basée sur la méthode BIBGTSystèmes d'information & management10.3917/sim.234.0007Volume 28:4(7-41)Online publication date: 30-Apr-2024
https://doi.org/10.3917/sim.234.0007
Rastogi A(2024)Roads Ahead to Diversity and Inclusion by Software EngineeringEquity, Diversity, and Inclusion in Software Engineering10.1007/978-1-4842-9651-6_1(3-16)Online publication date: 21-Sep-2024
https://doi.org/10.1007/978-1-4842-9651-6_1
Kula EGreuter Evan Deursen AGousios GChandra SBlincoe KTonella P(2023)Dynamic Prediction of Delays in Software Projects using Delay Patterns and Bayesian ModelingProceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering10.1145/3611643.3616328(1012-1023)Online publication date: 30-Nov-2023
https://dl.acm.org/doi/10.1145/3611643.3616328
Show More Cited By

Index Terms

Releasing fast and slow: an exploratory case study at ING
1. Software and its engineering
  1. Software creation and management
    1. Software development process management

Recommendations

Factors impacting rapid releases: an industrial case study
ESEM '14: Proceedings of the 8th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement

Context: Software release teams try to reduce the time needed for the transit of features or bug fixes from the development environment to the production, crossing all the quality gates. However, little is known about the factors that influence the time-...
Releasing sooner or later: an optimization approach and its case study evaluation
RELENG '13: Proceedings of the 1st International Workshop on Release Engineering

Decisions about the release date need to balance between the degree of readiness (quality) of the product and the potential competitive advantage and added value of (early) delivery. Based on an existing optimization approach for solving the maximum ...
An Exploratory Study of Software Evolution and Quality: Before, during and after a Transfer
ICGSE '12: Proceedings of the 2012 IEEE Seventh International Conference on Global Software Engineering

In the light of globalization it is not uncommon that different teams from different locations get involved in the development of a software product during its evolution. However, empirical evidence that demonstrates the effect of changing team members ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

ESEC/FSE 2019: Proceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering

August 2019

1264 pages

ISBN:9781450355728

DOI:10.1145/3338906

General Chairs:
Marlon Dumas
University of Tartu, Estonia
,
Dietmar Pfahl
University of Tartu, Estonia
,
Program Chairs:
Sven Apel
Saarland University, Germany
,
Alessandra Russo
Imperial College, UK

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGSOFT: ACM Special Interest Group on Software Engineering

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 August 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

ESEC/FSE '19

Sponsor:

SIGSOFT

ESEC/FSE '19: 27th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering

August 26 - 30, 2019

Tallinn, Estonia

Acceptance Rates

Overall Acceptance Rate 112 of 543 submissions, 21%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

8
Total Citations
View Citations
320
Total Downloads

Downloads (Last 12 months)20
Downloads (Last 6 weeks)3

Reflects downloads up to 13 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Kanita N(2024)Modèles explicatifs de la production de la dette technique dans les contextes Agile et DevOps : une revue de la littérature basée sur la méthode BIBGTSystèmes d'information & management10.3917/sim.234.0007Volume 28:4(7-41)Online publication date: 30-Apr-2024
https://doi.org/10.3917/sim.234.0007
Rastogi A(2024)Roads Ahead to Diversity and Inclusion by Software EngineeringEquity, Diversity, and Inclusion in Software Engineering10.1007/978-1-4842-9651-6_1(3-16)Online publication date: 21-Sep-2024
https://doi.org/10.1007/978-1-4842-9651-6_1
Kula EGreuter Evan Deursen AGousios GChandra SBlincoe KTonella P(2023)Dynamic Prediction of Delays in Software Projects using Delay Patterns and Bayesian ModelingProceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering10.1145/3611643.3616328(1012-1023)Online publication date: 30-Nov-2023
https://dl.acm.org/doi/10.1145/3611643.3616328
Scanlon TMorales J(2022)Revelations from an Agile and DevSecOps Transformation in a Large Organization: An Experiential Case StudyProceedings of the International Conference on Software and System Processes and International Conference on Global Software Engineering10.1145/3529320.3529329(77-81)Online publication date: 19-May-2022
https://dl.acm.org/doi/10.1145/3529320.3529329
Anderson KVisser DMannen JJiang Yvan Deursen AHarman MMiller H(2022)Challenges in applying continuous experimentationProceedings of the 44th International Conference on Software Engineering: Software Engineering in Practice10.1145/3510457.3513052(107-114)Online publication date: 21-May-2022
https://dl.acm.org/doi/10.1145/3510457.3513052
Anderson KVisser DMannen JJiang Yvan Deursen A(2022)Challenges in Applying Continuous Experimentation: A Practitioners' Perspective2022 IEEE/ACM 44th International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP)10.1109/ICSE-SEIP55303.2022.9793934(107-114)Online publication date: May-2022
https://doi.org/10.1109/ICSE-SEIP55303.2022.9793934
Nourry OKashiwa YKamei YUbayashi N(2021)Does shortening the release cycle affect refactoring activities: A case study of the JDT Core, Platform SWT, and UI projectsInformation and Software Technology10.1016/j.infsof.2021.106623(106623)Online publication date: May-2021
https://doi.org/10.1016/j.infsof.2021.106623
Cardoso TSantos AChanin RSales A(2021)Communication of Changes in Continuous Software DevelopmentSoftware Business10.1007/978-3-030-67292-8_7(86-101)Online publication date: 22-Jan-2021
https://doi.org/10.1007/978-3-030-67292-8_7
Lunde BColomo-Palacios R(2020)Continuous practices and technical debt: a systematic literature review2020 20th International Conference on Computational Science and Its Applications (ICCSA)10.1109/ICCSA50381.2020.00018(40-44)Online publication date: Jul-2020
https://doi.org/10.1109/ICCSA50381.2020.00018

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten