Understanding the Implications of Changes to Build Systems
Abstract
The maintenance of build systems imposes a considerable overhead on software development. Since automated quality assurance methods are rarely applied to build specifications, the importance of the role peer code review plays in the maintenance of build systems is amplified. Yet prior work shows that the review process for build systems suffers from a lack of build experts and effective tooling.
To support the understanding of changes to build specifications (a key stage in the review process), we propose BCIA---an approach to summarize the impact of changes to build specifications across the build configuration space. BCIA traverses the paths through which data and control flow in the prior and updated versions of the build system to generate an Impact Knowledge Graph (IKG), which describes the impact of the change across the build configuration space. We develop BuiScout---a prototype implementation of BCIA for CMake-based build systems. We use BuiScout to evaluate our approach through an empirical study of 10,000 change sets that we mine from ten large open-source projects that span a total of 28 development years. Our findings indicate that BuiScout can detect an impact that propagates to unmodified parts of the build system in 77.37% of the studied change sets. These changes impact a median of 14 unmodified commands, with a median of 95.55% of the impacted commands per change set appearing in unmodified files. Our study suggests that dedicated approaches, such as BCIA, have the potential to alleviate the challenges developers face when assessing the impact of changes to build systems.
References
[1]
Bram Adams, Kris De Schutter, Herman Tromp, and Wolfgang De Meuter. The Evolution of the Linux Build System. Electronic Communications of the EASST (ECEASST), 8(0), 2008.
[2]
Shane McIntosh, Bram Adams, and Ahmed E. Hassan. The Evolution of ANT Build Systems. In the Proceedings of the Working Conference on Mining Software Repositories (MSR), pages 42--51, 2010.
[3]
Shane McIntosh, Bram Adams, and Ahmed E. Hassan. The Evolution of Java Build Systems. Empirical Software Engineering (EMSE), 17(4):578--608, 2012.
[4]
Shane McIntosh, Bram Adams, Thanh H.D. Nguyen, Yasutaka Kamei, and Ahmed E. Hassan. An Empirical Study of Build Maintenance Effort. In the Proceedings of the International Conference on Software Engineering (ICSE), pages 141--150, 2011.
[5]
Shane McIntosh, Meiyappan Nagappan, Bram Adams, Audris Mockus, and Ahmed E. Hassan. A Large-Scale Empirical Study of the Relationship between Build Technology and Build Maintenance. Empirical Software Engineering (EMSE), 20(6):1587--1633, 2015.
[6]
Nachiappan Nagappan and Thomas Ball. Using Software Dependencies and Churn Metrics to Predict Field Failures: An Empirical Case Study. In the Proceedings of the International Symposium on Empirical Software Engineering and Measurement (ESEM), pages 364--373, 2007.
[7]
J. David Morgenthaler, Misha Gridnev, Raluca Sauciuc, and Sanjay Bhansali. Searching for Build Debt: Experiences Managing Technical Debt at Google. In the Proceedings of the International Workshop on Managing Technical Debt (MTD), pages 1--6, 2012.
[8]
Ying Zhang, Yanyan Jiang, Chang Xu, Xiaoxing Ma, and Ping Yu. ABC: Accelerated Building of C/C++ Projects. In the Proceedings of the Asia-Pacific Software Engineering Conference (APSEC), pages 182--189, 2015.
[9]
Hyunmin Seo, Caitlin Sadowski, Sebastian Elbaum, Edward Aftandilian, and Robert Bowdidge. Programmers' Build Errors: A Case Study (at Google). In the Proceedings of the International Conference on Software Engineering (ICSE), pages 724--734, 2014.
[10]
Lorin Hochstein and Yang Jiao. The Cost of the Build Tax in Scientific Software. In the Proceedings of the International Symposium on Empirical Software Engineering and Measurement (ESEM), pages 384--387, 2011.
[11]
Sarah Nadi and Ric Holt. Make It or Break It: Mining Anomalies from Linux Kbuild. In the Proceedings of the Working Conference on Reverse Engineering (WCRE), pages 315--324, 2011.
[12]
Mahtab Nejati, Mahmoud Alfadel, and Shane McIntosh. Code Review of Build System Specifications: Prevalence, Purposes, Patterns, and Perceptions. In the Proceedings of the International Conference on Software Engineering (ICSE), pages 1213--1224, 2023.
[13]
Shaun Phillips, Thomas Zimmermann, and Christian Bird. Understanding and Improving Software Build Teams. In the Proceedings of the International Conference on Software Engineering (ICSE), pages 735--744, 2014.
[14]
Yuan Huang, Nan Jia, Xiangping Chen, Kai Hong, and Zibin Zheng. Salient-Class Location: Help Developers Understand Code Change in Code Review. In the Proceedings of the ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE), pages 770--774, 2018.
[15]
Quinn Hanam, Ali Mesbah, and Reid Holmes. Aiding Code Change Understanding with Semantic Change Impact Analysis. In the Proceedings of the International Conference on Software Maintenance and Evolution (ICSME), pages 202--212, 2019.
[16]
Fernanda M. Delfim, Lilian P. Scatalon, Jorge M. Prates, and Rogério E. Garcia. Visual Approach for Change Impact Analysis: A Controlled Experiment. In the Proceedings of the International Conference on Information Technology - New Generations (ITNG), pages 391--396, 2015.
[17]
Siyuan Jiang, Collin McMillan, and Raul Santelices. Do Programmers Do Change Impact Analysis in Debugging? Empirical Software Engineering (EMSE), (2): 631--669, 2017.
[18]
Mehran Meidani, Maxime Lamothe, and Shane McIntosh. Assessing the Exposure of Software Changes: The DiPiDi Approach. Empirical Software Engineering (EMSE), 28(2):41, 2023.
[19]
Ruiyin Wen, David Gilbert, Michael G. Roche, and Shane McIntosh. BLIMP Tracer: Integrating Build Impact Analysis with Code Review. In the Proceedings of the International Conference on Software Maintenance and Evolution (ICSME), pages 685--694, 2018.
[20]
Nikhil Parasaram, Earl T. Barr, and Sergey Mechtaev. Rete: Learning Namespace Representation for Program Repair. In the Proceedings of the International Conference on Software Engineering (ICSE), pages 1264--1276, 2023.
[21]
Shane McIntosh, Meiyappan Nagappan, Bram Adams, Audris Mockus, and Ahmed E. Hassan. A Large-Scale Empirical Study of the Relationship between Build Technology and Build Maintenance. Empirical Software Engineering (EMSE), 20(6):1587--1633, 2015.
[22]
Ahmed Tamrawi, Hoan Anh Nguyen, Hung Viet Nguyen, and Tien N. Nguyen. Build Code Analysis with Symbolic Evaluation. In the Proceedings of the International Conference on Software Engineering (ICSE), pages 650--660, 2012.
[23]
Jafar Al-Kofahi, Hung Viet Nguyen, and Tien N. Nguyen. Fault Localization for Build Code Errors in Makefiles. In the Companion Proceedings of the International Conference on Software Engineering (ICSE-Companion), pages 600--601, 2014.
[24]
Cor-Paul Bezemer, Shane McIntosh, Bram Adams, Daniel M German, and Ahmed E Hassan. An Empirical Study of Unspecified Dependencies in Make-based Build Systems. Empirical Software Engineering (EMSE), 22(6):3117--3148, 2017.
[25]
Ying Wang, Ming Wen, Zhenwei Liu, Rongxin Wu, Rui Wang, Bo Yang, Hai Yu, Zhiliang Zhu, and Shing-Chi Cheung. Do the Dependency Conflicts in My Project Matter? In the Proceedings of the ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE), pages 319--330, 2018.
[26]
Ying Wang, Ming Wen, Rongxin Wu, Zhenwei Liu, Shin Hwei Tan, Zhiliang Zhu, Hai Yu, and Shing-Chi Cheung. Could I Have a Stack Trace to Examine the Dependency Conflict Issue? In the Proceedings of the International Conference on Software Engineering (ICSE), pages 572--583, 2019.
[27]
Christian Macho, Fabian Oraze, and Martin Pinzger. DValidator: An Approach for Validating Dependencies in Build Configurations. Journal of Systems and Software (JSS), 209:111916, 2024.
[28]
Christian Macho, Shane McIntosh, and Martin Pinzger. Automatically Repairing Dependency-related Build Breakage. In the Proceedings of the International Conference on Software Analysis, Evolution and Reengineering (SANER), pages 106--117, 2018.
[29]
Foyzul Hassan and Xiaoyin Wang. Hirebuild: An Automatic Approach to History-Driven Repair of Build Scripts. In the Proceedings of the International Conference on Software Engineering (ICSE), pages 1078--1089, 2018.
[30]
Yiling Lou, Junjie Chen, Lingming Zhang, Dan Hao, and Lu Zhang. History-Driven Build Failure Fixing: How Far Are We? In the Proceedings of the International Symposium on Software Testing and Analysis (ISSTA), pages 43--54, 2019.
[31]
Thodoris Sotiropoulos, Stefanos Chaliasos, Dimitris Mitropoulos, and Diomidis Spinellis. A Model for Detecting Faults in Build Specifications. the Proceedings of the ACM on Programming Languages, 4(OOPSLA):1--30, 2020.
[32]
Gang Fan, Chengpeng Wang, Rongxin Wu, Xiao Xiao, Qingkai Shi, and Charles Zhang. Escaping Dependency Hell: Finding Build Dependency Errors with the Unified Dependency Graph. In the Proceedings of the International Symposium on Software Testing and Analysis (ISSTA), pages 463--474, 2020.
[33]
Bram Adams, Herman Tromp, Kris de Schutter, and Wolfgang de Meuter. Design Recovery and Maintenance of Build Systems. In the Proceedings of the International Conference on Software Maintenance (ICSM), pages 114--123, 2007.
[34]
Carlene Lebeuf, Elena Voyloshnikova, Kim Herzig, and Margaret-Anne Storey. Understanding, Debugging, and Optimizing Distributed Software Builds: A Design Study. In the Proceedings of the International Conference on Software Maintenance and Evolution (ICSME), pages 496--507, 2018.
[35]
Ryan Hardt and Ethan V. Munson. An Empirical Evaluation of Ant Build Maintenance Using Formiga. In the Proceedings of the International Conference on Software Maintenance and Evolution (ICSME), pages 201--210, 2015.
[36]
Jim Buffenbarger. Adding Automatic Dependency Processing to Makefile-Based Build Systems with Amake. In the Proceedings of the International Workshop on Release Engineering (RELENG), pages 1--4, 2013.
[37]
Carmine Vassallo, Sebastian Proksch, Timothy Zemp, and Harald C. Gall. Every Build You Break: Developer-Oriented Assistance for Build Failure Resolution. Empirical Software Engineering (EMSE), 25(3):2218--2257, 2020.
[38]
Christian Macho, Stefanie Beyer, Shane McIntosh, and Martin Pinzger. The Nature of Build Changes: An Empirical Study of Maven-Based Build Systems. Empirical Software Engineering (EMSE), 26(3):32, 2021.
[39]
Bohner. Impact analysis in the software change process: a year 2000 perspective. In the Proceedings of the International Conference on Software Maintenance (ICSM), pages 42--51, 1996.
[40]
Bixin Li, Xiaobing Sun, Hareton Leung, and Sai Zhang. A Survey of Code-Based Change Impact Analysis Techniques. Software Testing, Verification and Reliability (STVR), 23(8):613--646, 2013.
[41]
Zijian Jiang, Ye Wang, Hao Zhong, and Na Meng. Automatic Method Change Suggestion to Complement Multi-Entity Edits. Journal of Systems and Software (JSS), 159:110441, 2020.
[42]
Zijian Jiang, Hao Zhong, and Na Meng. Investigating and Recommending Co-Changed Entities for JavaScript Programs. Journal of Systems and Software (JSS), 180:111027, 2021.
[43]
Daihong Zhou, Yijian Wu, Xin Peng, Jiyue Zhang, and Ziliang Li. Revealing Code Change Propagation Channels by Evolution History Mining. Journal of Systems and Software (JSS), 208:111912, 2024.
[44]
Binish Tanveer, Anna Maria Vollmer, and Ulf Martin Engel. Utilizing Change Impact Analysis for Effort Estimation in Agile Development. In the Proceedings of the Euromicro Conference on Software Engineering and Advanced Applications (SEAA), pages 430--434, 2017.
[45]
Binish Tanveer, Anna Maria Vollmer, Stefan Braun, and Nauman bin Ali. An Evaluation of Effort Estimation Supported by Change Impact Analysis in Agile Software Development. Journal of Software: Evolution and Process (JSEP), 31(5): e2165, 2019.
[46]
Xiaoxia Ren, Fenil Shah, Frank Tip, Barbara G. Ryder, and Ophelia Chesley. Chianti: A Tool for Change Impact Analysis of Java Programs. In the Proceedings of the ACM SIGPLAN Conference on Object-Oriented Programming, Systems, Languages, and Applications (OOPSLA), pages 432--448, 2004.
[47]
Ye Wang, Na Meng, and Hao Zhong. An Empirical Study of Multi-entity Changes in Real Bug Fixes. In the Proceedings of the International Conference on Software Maintenance and Evolution (ICSME), pages 287--298, 2018.
[48]
Maria Kretsou, Elvira-Maria Arvanitou, Apostolos Ampatzoglou, Ignatios Deligiannis, and Vassilis C. Gerogiannis. Change impact analysis: A systematic mapping study. Journal of Systems and Software (JSS), 174:110892, 2021.
[49]
Leon Moonen, David Binkley, and Sydney Pugh. On Adaptive Change Recommendation. Journal of Systems and Software (JSS), 164:110550, 2020.
[50]
Manishankar Mondal, Banani Roy, Chanchal K. Roy, and Kevin A. Schneider. Associating Code Clones with Association Rules for Change Impact Analysis. In the Proceedings of the International Conference on Software Analysis, Evolution and Reengineering (SANER), pages 93--103, 2020.
[51]
Peng Dai, Yawen Wang, Dahai Jin, Yunzhan Gong, and Wenjin Yang. An improving approach to analyzing change impact of C programs. Journal of Computer Communications (ComputCommun), 182:60--71, 2022.
[52]
Shane Mcintosh, Bram Adams, Meiyappan Nagappan, and Ahmed E. Hassan. Mining Co-Change Information to Understand When Build Changes Are Necessary. In the Proceedings of the International Conference on Software Maintenance and Evolution (ICSME), pages 241--250, 2014.
[53]
Christian Macho, Shane McIntosh, and Martin Pinzger. Predicting Build Co-changes with Source Code Change and Commit Categories. In the Proceedings of the International Conference on Software Analysis, Evolution, and Reengineering (SANER), pages 541--551, 2016.
[54]
Xin Xia, David Lo, Shane McIntosh, Emad Shihab, and Ahmed E. Hassan. Cross-Project Build Co-Change Prediction. In the Proceedings of the International Conference on Software Analysis, Evolution, and Reengineering (SANER), pages 311--320, 2015.
[55]
Jafar M. Al-Kofahi, Hung Viet Nguyen, Anh Tuan Nguyen, Tung Thanh Nguyen, and Tien N. Nguyen. Detecting Semantic Changes in Makefile Build Code. In The Proceedings of the International Conference on Software Maintenance (ICSM), pages 150--159, 2012.
[56]
Shruti Raghavan, Rosanne Rohana, David Leon, Andy Podgurski, and Vinay Augustine. Dex: A Semantic-Graph Differencing Tool for Studying Changes in Large Code Bases. In the Proceedings of the International Conference on Software Maintenance (ICSM), pages 188--197, 2004.
[57]
Jean-Rémy Falleri, Floréal Morandat, Xavier Blanc, Matias Martinez, and Martin Monperrus. Fine-Grained and Accurate Source Code Differencing. In the Proceedings of the International Conference on Automated Software Engineering (ASE), pages 313--324, 2014.
[58]
Beat Fluri, Michael Wursch, Martin PInzger, and Harald Gall. Change Distilling: Tree Differencing for Fine-Grained Source Code Change Extraction. IEEE Transactions on Software Engineering (TSE), 33(11):725--743, 2007.
[59]
Georg Dotzler and Michael Philippsen. Move-Optimized Source Code Tree Differencing. In the Proceedings of the International Conference on Automated Software Engineering (ASE), pages 660--671, 2016.
[60]
Veit Frick, Thomas Grassauer, Fabian Beck, and Martin Pinzger. Generating Accurate and Compact Edit Scripts Using Tree Differencing. In the Proceedings of the International Conference on Software Maintenance and Evolution (ICSME), pages 264--274, 2018.
[61]
Kaifeng Huang, Bihuan Chen, Xin Peng, Daihong Zhou, Ying Wang, Yang Liu, and Wenyun Zhao. ClDiff: generating concise linked code differences. In the Proceedings of the International Conference on Automated Software Engineering (ASE), page 679--690, 2018.
[62]
Aidan Hogan, Eva Blomqvist, Michael Cochez, Claudia D'amato, Gerard De Melo, Claudio Gutierrez, Sabrina Kirrane, José Emilio Labra Gayo, Roberto Navigli, Sebastian Neumaier, Axel-Cyrille Ngonga Ngomo, Axel Polleres, Sabbir M. Rashid, Anisa Rula, Lukas Schmelzeisen, Juan Sequeda, Steffen Staab, and Antoine Zimmermann. Knowledge Graphs. ACM Computing Surveys, 54(4):1--37, 2021.
[63]
Davide Spadini, Maurício Aniche, Margaret-Anne Storey, Magiel Bruntink, and Alberto Bacchelli. When Testing Meets Code Review: Why and How Developers Review Tests. In the Proceedings of the International Conference on Software Engineering (ICSE), pages 677--687, 2018.
[64]
Yuxing Ma, Tapajit Dey, Chris Bogart, Sadika Amreen, Marat Valiev, Adam Tutko, David Kennard, Russell Zaretzki, and Audris Mockus. World of code: enabling a research workflow for mining and analyzing the universe of open source VCS data. Empirical Software Engineering (EMSE), 26(22):1--42, 2021.
[65]
Audris Mockus and David M Weiss. Predicting risk of software changes. Bell Labs Technical Journal, 5(2):169--180, 2000.
[66]
Jerrold H. Zar. Significance Testing of the Spearman Rank Correlation Coefficient. Journal of the American Statistical Association (JASA), 67(339):578--580, 1972.
[67]
William S. Cleveland. Robust Locally Weighted Regression and Smoothing Scatterplots. Journal of the American Statistical Association (JASA), 74(368): 829--836, 1979.
[68]
KimHao Nguyen, ThanhVu Nguyen, and Quoc-Sang Phan. Analyzing the CMake Build System. In the Proceedings of the International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP), pages 27--28, 2022.
Index Terms
- Understanding the Implications of Changes to Build Systems
Index terms have been assigned to the content through auto-classification.
Recommendations
Extracting build changes with BuildDiff
MSR '17: Proceedings of the 14th International Conference on Mining Software RepositoriesBuild systems are an essential part of modern software engineering projects. As software projects change continuously, it is crucial to understand how the build system changes because neglecting its maintenance can lead to expensive build breakage. ...
A Large-Scale Empirical Study of the Relationship between Build Technology and Build Maintenance
Build systems specify how source code is translated into deliverables. They require continual maintenance as the system they build evolves. This build maintenance can become so burdensome that projects switch build technologies, potentially having to ...
On the benefits and limits of incremental build of software configurations: an exploratory study
ICSE '22: Proceedings of the 44th International Conference on Software EngineeringSoftware projects use build systems to automate the compilation, testing, and continuous deployment of their software products. As software becomes increasingly configurable, the build of multiple configurations is a pressing need, but expensive and ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
October 2024
2587 pages
ISBN:9798400712487
DOI:10.1145/3691620
- General Chair:
- Vladimir Filkov,
- Program Co-chairs:
- Baishakhi Ray,
- Minghui Zhou
Copyright © 2024 Copyright is held by the owner/author(s). Publication rights licensed to ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 27 October 2024
Check for updates
Badges
Author Tags
Qualifiers
- Research-article
Conference
ASE '24: 39th IEEE/ACM International Conference on Automated Software Engineering
October 27 - November 1, 2024
CA, Sacramento, USA
Acceptance Rates
Overall Acceptance Rate 82 of 337 submissions, 24%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 23Total Downloads
- Downloads (Last 12 months)23
- Downloads (Last 6 weeks)23
Reflects downloads up to 20 Nov 2024
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in