Will they like this? evaluating code contributions with language models

VJ Hellendoorn, PT Devanbu… - 2015 IEEE/ACM 12th …, 2015 - ieeexplore.ieee.org
2015 IEEE/ACM 12th Working Conference on Mining Software Repositories, 2015ieeexplore.ieee.org
Popular open-source software projects receive and review contributions from a diverse array
of developers, many of whom have little to no prior involvement with the project. A recent
survey reported that reviewers consider conformance to the project's code style to be one of
the top priorities when evaluating code contributions on Github. We propose to quantitatively
evaluate the existence and effects of this phenomenon. To this aim we use language
models, which were shown to accurately capture stylistic aspects of code. We find that …
Popular open-source software projects receive and review contributions from a diverse array of developers, many of whom have little to no prior involvement with the project. A recent survey reported that reviewers consider conformance to the project's code style to be one of the top priorities when evaluating code contributions on Github. We propose to quantitatively evaluate the existence and effects of this phenomenon. To this aim we use language models, which were shown to accurately capture stylistic aspects of code. We find that rejected change sets do contain code significantly less similar to the project than accepted ones, furthermore, the less similar change sets are more likely to be subject to thorough review. Armed with these results we further investigate whether new contributors learn to conform to the project style and find that experience is positively correlated with conformance to the project's code style.
ieeexplore.ieee.org