research-article

Investigating the Effectiveness of Peer Code Review in Distributed Software Development

Authors:

Eduardo Witter dos Santos,

Ingrid NunesAuthors Info & Claims

SBES '17: Proceedings of the XXXI Brazilian Symposium on Software Engineering

Pages 84 - 93

https://doi.org/10.1145/3131151.3131161

Published: 20 September 2017 Publication History

Abstract

Code review is a potential means of improving software quality. To be effective, it depends on different factors, and many have been investigated in the literature to identify the scenarios in which it adds quality to the final code. However, factors associated with distributed software development, which is becoming increasingly common, have been little explored. Geographic distance can impose additional challenges to the reviewing process. We thus in this paper present the results of a quantitative study of the effectiveness of code review in a distributed software project involving 201 members. We investigate factors that can potentially influence the outcomes of peer code review. Our results show that a high number of changed lines of code tends to increase the review duration with a reduced number of messages, while the number of involved teams, locations, and participant reviewers generally improve reviewer contributions, but with a severe penalty to the duration.

References

[1]

Alberto Bacchelli and Christian Bird. 2013. Expectations, Outcomes, and Challenges of Modern Code Review. In ICSE 2013. IEEE Press, 712--721.

Digital Library

[2]

Vipin Balachandran. 2013. Reducing human effort and improving quality in peer code reviews using automatic static analysis and reviewer recommendation. In ICSE 2013. IEEE, 931--940.

Digital Library

[3]

V R Basili, R W Selby, and D H Hutchens. 1986. Experimentation in Software Engineering. IEEE Trans. Softw. Eng. 12, 7 (July 1986), 733--743.

Digital Library

[4]

Olga Baysal, Oleksii Kononenko, Reid Holmes, and Michael W Godfrey. 2016. Investigating technical and non-technical factors influencing modern code review. Empirical Software Engineering 21, 3 (2016), 932--959.

Digital Library

[5]

Moritz Beller, Alberto Bacchelli, Andy Zaidman, and Elmar Juergens. 2014. Modern Code Reviews in Open-source Projects: Which Problems Do They Fix?. In MSR 2014. ACM, 202--211.

Digital Library

[6]

Amiangshu Bosu, Michaela Greiler, and Christian Bird. 2015. Characteristics of useful code reviews: An empirical study at microsoft. In MSR 2015. IEEE, 146--156.

Digital Library

[7]

Jacek Czerwonka, Michaela Greiler, and Jack Tilford. 2015. Code Reviews Do Not Find Bugs: How the Current Code Review Best Practice Slows Us Down. In ICSE 2015. IEEE Press, 27--28.

Digital Library

[8]

Michael E Fagan. 1986. Advances in software inspections. IEEE Transactions on Software Engineering 12, 1 (1986), 744--751.

Digital Library

[9]

Andre L Ferreira, Ricardo J Machado, Jose G Silva, Rui F Batista, Lino Costa, and Mark C Paulk. 2010. An approach to improving software inspections performance. In ICSM 2010. IEEE, 1--8.

Digital Library

[10]

Google. 2017. Gerrit Code Review. (2017). https://www.gerritcodereview.com

[11]

Google. 2017. Gerrit Code Review v2.11.2, Queries. (2017). https://gerrit-documentation.storage.googleapis.com/Documentation/2.12.2/cmd-query.html

[12]

Google. 2017. Gerrit Plugin: Reviewers by Blame. (2017). https://gerrit-review.googlesource.com/Documentation/config-plugins.html#reviewers-by-blame

[13]

Christopher D. Hundhausen, Anukrati Agrawal, and Pawan Agarwal. 2013. Talking About Code: Integrating Pedagogical Code Reviews into Early Computing Courses. Trans. Comput. Educ. 13, 3, Article 14 (Aug. 2013), 28 pages.

Digital Library

[14]

Internet Engineering Task Force (IETF). 2017. RFC 6020: YANG - A Data Modeling Language for the Network Configuration Protocol (NETCONF). (2017). http://tools.ietf.org/html/rfc6020

[15]

C. F. Kemerer and M. C. Paulk. 2009. The Impact of Design and Code Reviews on Software Quality: An Empirical Study Based on PSP Data. IEEE Transactions on Software Engineering 35, 4 (July 2009), 534--550.

Digital Library

[16]

Sami Kollanus and Jussi Koskinen. 2009. Survey of software inspection research. The Open Software Engineering Journal 3, 1 (2009), 15--34.

[17]

S. McIntosh, Y. Kamei, B. Adams, and A. Hassan. 2014. The Impact of Code Review Coverage and Code Review Participation on Software Quality: A Case Study of the Qt, VTK, and ITK Projects. In MSR 2014. ACM, 192--201.

Digital Library

[18]

Bertrand Meyer. 2008. Design and Code Reviews in the Age of the Internet. Commun. ACM 51, 9 (Sept. 2008), 66--71.

Digital Library

[19]

Mohammad Masudur Rahman, Chanchal K Roy, Jesse Redl, and Jason A Collins. 2016. CORRECT: Code reviewer recommendation at GitHub for Vendasta technologies. In ASE 2016. IEEE, 792--797.

Digital Library

[20]

Bikram Sengupta, Satish Chandra, and Vibha Sinha. 2006. A Research Agenda for Distributed Software Development. In ICSE 2006. ACM, 731--740.

Digital Library

[21]

Junji Shimagaki, Yasutaka Kamei, Shane McIntosh, Ahmed E. Hassan, and Naoyasu Ubayashi. 2016. A Study of the Quality-impacting Practices of Modern Code Review at Sony Mobile. In ICSE 2016. ACM, 212--221.

Digital Library

[22]

Patanamon Thongtanunam, Raula Gaikovina Kula, Ana Erika Camargo Cruz, Norihiro Yoshida, and Hajimu Iida. 2014. Improving code review effectiveness through reviewer recommendations. In CHASE 2014. ACM, 119--122.

Digital Library

[23]

Patanamon Thongtanunam, Shane Mcintosh, Ahmed E. Hassan, and Hajimu Iida. 2015. Investigating Code Review Practices in Defective Files: An Empirical Study of the Qt System. In MSR 2015. IEEE Press, 168--179.

Digital Library

[24]

Patanamon Thongtanunam, Shane Mcintosh, Ahmed E Hassan, and Hajimu Iida. 2016. Review participation in modern code review. Empirical Software Engineering (2016), 1--50.

Digital Library

[25]

Patanamon Thongtanunam, Shane Mcintosh, Ahmed E. Hassan, and Hajimu Iida. 2016. Revisiting Code Ownership and Its Relationship with Software Quality in the Scope of Modern Code Review. In ICSE 2016. ACM, 1039--1050.

Digital Library

[26]

Patanamon Thongtanunam, Chakkrit Tantithamthavorn, Raula Gaikovina Kula, Norihiro Yoshida, Hajimu Iida, and Ken-ichi Matsumoto. 2015. Who should review my code? A file location-based code-reviewer recommendation approach for modern code review. In SANER 2015. IEEE, 141--150.

[27]

Giovanni Viviani and Gail C Murphy. 2016. Removing stagnation from modern code review. In SPLASH 2016. ACM, 43--44.

Digital Library

[28]

Xin Xia, David Lo, Xinyu Wang, and Xiaohu Yang. 2015. Who should review this change?: Putting text and file location analyses together for more accurate recommendations. In ICSME 2015. IEEE, 261--270.

Digital Library

[29]

Xin Yang. 2014. Social network analysis in open source software peer review. In FSE 2014. ACM, 820--822.

Digital Library

[30]

Motahareh Bahrami Zanjani, Huzefa Kagdi, and Christian Bird. 2016. Automatically recommending peer reviewers in modern code review. IEEE Transactions on Software Engineering 42, 6 (2016), 530--543.

Digital Library

Cited By

Kree LHelmke RWinter E(2024)Using Semgrep OSS to Find OWASP Top 10 Weaknesses in PHP Applications: A Case StudyDetection of Intrusions and Malware, and Vulnerability Assessment10.1007/978-3-031-64171-8_4(64-83)Online publication date: 9-Jul-2024
https://doi.org/10.1007/978-3-031-64171-8_4
Silva DPabasara WSangkavi SL.G.A.T.D WW.M.K.H WS.A R(2023)The Effectiveness of Code Reviews on Improving Software Quality: An Empirical StudyInternational Journal of Recent Technology and Engineering (IJRTE)10.35940/ijrte.B7666.071222312:2(1-10)Online publication date: 30-Jul-2023
https://doi.org/10.35940/ijrte.B7666.0712223
Badampudi DUnterkalmsteiner MBritto R(2023)Modern Code Reviews—Survey of Literature and PracticeACM Transactions on Software Engineering and Methodology10.1145/358500432:4(1-61)Online publication date: 26-May-2023
https://dl.acm.org/doi/10.1145/3585004
Show More Cited By

Index Terms

Investigating the Effectiveness of Peer Code Review in Distributed Software Development
1. General and reference
  1. Cross-computing tools and techniques
    1. Empirical studies
2. Software and its engineering
  1. Software creation and management
    1. Collaboration in software development
      1. Programming teams
    2. Software verification and validation
      1. Empirical software validation

Recommendations

Agile vs. structured distributed software development: A case study

In globally distributed software development, does it matter being agile rather than structured? To answer this question, this paper presents an extensive case study that compares agile (Scrum, XP, etc.) vs. structured (RUP, waterfall) processes to ...
Success Factors of Distributed Software Development Projects in Israel
ECBS 2021: 7th Conference on the Engineering of Computer Based Systems

Geographically Distributed Development (GDD) has now become a prominent software development paradigm. Software companies are increasingly adopting geographically distributed development approaches to produce high quality software [1]. geographically ...
Are you smelling it? Investigating how similar developers detect code smells

ContextA code smell indicates a poor implementation choice that often worsens software quality. Thus, code smell detection is an elementary technique to identify refactoring opportunities in software systems. Unfortunately, there is limited knowledge on ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

SBES '17: Proceedings of the XXXI Brazilian Symposium on Software Engineering

September 2017

409 pages

ISBN:9781450353267

DOI:10.1145/3131151

Program Chairs:
José Carlos Maldonado
ICMC-USP
,
Fabiano Cutigi Ferrari
UFSCar
,
Uirá Kulesza
DIMAp/UFRN
,
Tayana Uchôa Conte
UFAM

Copyright © 2017 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

In-Cooperation

SBC: Brazilian Computer Society
CNPq: Conselho Nacional de Desenvolvimento Cientifico e Tecn
CAPES: Brazilian Higher Education Funding Council

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 20 September 2017

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

Coordenação de Aperfeiçoamento de Pessoal de Nível Superior
Alexander von Humboldt-Stiftung
Conselho Nacional de Desenvolvimento Científico e Tecnológico

Conference

SBES'17

SBES'17: 31st Brazilian Symposium on Software Engineering

September 20 - 22, 2017

CE, Fortaleza, Brazil

Acceptance Rates

SBES '17 Paper Acceptance Rate 42 of 134 submissions, 31%;

Overall Acceptance Rate 147 of 427 submissions, 34%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

14
Total Citations
View Citations
363
Total Downloads

Downloads (Last 12 months)21
Downloads (Last 6 weeks)3

Reflects downloads up to 01 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Kree LHelmke RWinter E(2024)Using Semgrep OSS to Find OWASP Top 10 Weaknesses in PHP Applications: A Case StudyDetection of Intrusions and Malware, and Vulnerability Assessment10.1007/978-3-031-64171-8_4(64-83)Online publication date: 9-Jul-2024
https://doi.org/10.1007/978-3-031-64171-8_4
Silva DPabasara WSangkavi SL.G.A.T.D WW.M.K.H WS.A R(2023)The Effectiveness of Code Reviews on Improving Software Quality: An Empirical StudyInternational Journal of Recent Technology and Engineering (IJRTE)10.35940/ijrte.B7666.071222312:2(1-10)Online publication date: 30-Jul-2023
https://doi.org/10.35940/ijrte.B7666.0712223
Badampudi DUnterkalmsteiner MBritto R(2023)Modern Code Reviews—Survey of Literature and PracticeACM Transactions on Software Engineering and Methodology10.1145/358500432:4(1-61)Online publication date: 26-May-2023
https://dl.acm.org/doi/10.1145/3585004
Kudrjavets GRastogi A(2023)Does code review speed matter for practitioners?Empirical Software Engineering10.1007/s10664-023-10401-z29:1Online publication date: 22-Nov-2023
https://dl.acm.org/doi/10.1007/s10664-023-10401-z
Jureczko MKajda ŁGórecki P(2021)Code review effectiveness: an empirical study on selected factors influenceIET Software10.1049/iet-sen.2020.013414:7(794-805)Online publication date: 18-Feb-2021
https://doi.org/10.1049/iet-sen.2020.0134
Kaur HSaini M(2021)Identification of Key Parameters Contributing to Technical Debt in Software Using Rank-Based OptimizationSustainable Communication Networks and Application10.1007/978-981-15-8677-4_35(425-438)Online publication date: 26-Jan-2021
https://doi.org/10.1007/978-981-15-8677-4_35
Fatima NNazir SChuprat S(2019)Individual, Social and Personnel Factors Influencing Modern Code Review Process2019 IEEE Conference on Open Systems (ICOS)10.1109/ICOS47562.2019.8975708(40-45)Online publication date: Nov-2019
https://doi.org/10.1109/ICOS47562.2019.8975708
Nazir SFatima NChuprat S(2019)Does Project Associated Situational Factors have Impact on Sustainability of Modern Code Review Workforce?2019 IEEE 6th International Conference on Engineering Technologies and Applied Sciences (ICETAS)10.1109/ICETAS48360.2019.9117541(1-5)Online publication date: Dec-2019
https://doi.org/10.1109/ICETAS48360.2019.9117541
Fatima NNazir SChuprat S(2019)Knowledge sharing, a key sustainable practice is on risk: An insight from Modern Code Review2019 IEEE 6th International Conference on Engineering Technologies and Applied Sciences (ICETAS)10.1109/ICETAS48360.2019.9117444(1-6)Online publication date: Dec-2019
https://doi.org/10.1109/ICETAS48360.2019.9117444
Nazir SFatima NChuprat S(2019)Situational factors affecting Software Engineers Sustainability: A Vision of Modern Code Review2019 IEEE 6th International Conference on Engineering Technologies and Applied Sciences (ICETAS)10.1109/ICETAS48360.2019.9117366(1-6)Online publication date: Dec-2019
https://doi.org/10.1109/ICETAS48360.2019.9117366
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents