Understanding Developers’ Discussions and Perceptions on Non-functional Requirements: The Case of the Spring Ecosystem
Authors: 
Anderson Oliveira, 
João Correia, 
Wesley K. G. Assunção, 
Juliana Alves Pereira, Rafael de Mello, Daniel Coutinho, Caio Barbosa, Paulo Libório, Alessandro GarciaAuthors Info & Claims
Article No.: 24, Pages 517 - 538
Abstract
Non-Functional Requirements (NFRs) should be defined in the early stages of the software development process, driving developers to make important design decisions. Neglecting NFRs may lead developers to create systems that are difficult to maintain and do not meet users expectations. Despite their importance, the discussion of NFRs is often ad-hoc and scattered through multiple sources, limiting developers' awareness of NFRs. In that scenario, Pull Request (PR) discussions provide a centralized platform for comprehensive NFR discussions. However, existing studies do not explore this important source of information in open-source software development, which developers widely use to discuss software requirements. In this study, we report an investigation of NFR discussions in PRs of repositories of the Spring ecosystem. We collected, manually curated, and analyzed PR discussions addressing four categories of NFRs: maintainability, security, performance, and robustness. We observed that discussions surrounding these PRs tend to address the introduction of a code change or explain some anomaly regarding a particular NFR. Also, we found that more than 77% of the discussions related to NFRs are triggered in the PR title and/or description, indicating that developers are often provided with information regarding NFRs straightway. To gain additional knowledge from these NFR discussions, our study also analyzed the characteristics and activities of developers who actually discuss and fix NFR issues. In particular, we performed an in-depth analysis of 63 developers that stood out in collaborating with the mapped PRs. To complement this analysis, we conducted a survey with 44 developers to gather their perceptions on NFR discussions. By observing how developers approach NFRs and participate in discussions, we documented the best practices and strategies newcomers can adopt to address NFRs effectively. We also provided a curated dataset of 1,533 PR discussions classified with NFR presence.
References
[1]
Hervé Abdi and Lynne J Williams. 2010. Principal component analysis. Wiley interdisciplinary reviews: computational statistics, 2, 4 (2010), 433–459. https://doi.org/10.1002/wics.1246
[2]
David Ameller, Claudia Ayala, Jordi Cabot, and Xavier Franch. 2012. How do software architects consider non-functional requirements: An exploratory study. In 2012 20th IEEE international requirements engineering conference (RE). 41–50. https://doi.org/10.1109/RE.2012.6345838
[3]
Cody Baker, Lin Deng, Suranjan Chakraborty, and Josh Dehlinger. 2019. Automatic Multi-class Non-Functional Software Requirements Classification Using Neural Networks. In 2019 IEEE 43rd Annual Computer Software and Applications Conference (COMPSAC). 2, 610–615. https://doi.org/10.1109/COMPSAC.2019.10275
[4]
Caio Barbosa, Anderson Uchôa, Filipe Falcao, Daniel Coutinho, Hyago Brito, Guilherme Amaral, Alessandro Garcia, Baldoino Fonseca, Marcio Ribeiro, Vinicius Soares, and Leonardo Sousa. 2020. Revealing the Social Aspects of Design Decay: A Retrospective Study of Pull Requests. In 34th SBES. 1–10. https://doi.org/10.1145/3422392.3422443
[5]
Tanmay Bhowmik and Anh Quoc Do. 2019. Refinement and resolution of just-in-time requirements in open source software and a closer look into non-functional requirements. Journal of Industrial Information Integration, 14 (2019), 24–33. https://doi.org/10.1016/j.jii.2018.03.001
[6]
Manal Binkhonain and Liping Zhao. 2019. A review of machine learning algorithms for identification and classification of non-functional requirements. Expert Systems with Applications: X, 1 (2019), 100001. https://doi.org/10.1016/j.eswax.2019.100001
[7]
Cristina Rosa Camacho, Sabrina Marczak, and Daniela S Cruzes. 2016. Agile team members perceptions on non-functional testing: influencing factors from an empirical study. In 2016 11th international conference on availability, reliability and security (ARES). 582–589. https://doi.org/10.1109/ARES.2016.98
[8]
Agustin Casamayor, Daniela Godoy, and Marcelo Campo. 2010. Identification of non-functional requirements in textual specifications: A semi-supervised learning approach. Information and Software Technology, 52, 4 (2010), 436–445. https://doi.org/10.1016/j.infsof.2009.10.010
[9]
Ranit Chatterjee, Abdul Ahmed, Preethu Rose Anish, Brijendra Suman, Prashant Lawhatre, and Smita Ghaisas. 2021. A Pipeline for Automating Labeling to Prediction in Classification of NFRs. In 2021 IEEE 29th International Requirements Engineering Conference (RE). 323–323. https://doi.org/10.1109/RE51729.2021.00036
[10]
Lawrence Chung and Brian A. Nixon. 1995. Dealing with Non-Functional Requirements: Three Experimental Studies of a Process-Oriented Approach. In 17th International Conference on Software Engineering (ICSE ’95). Association for Computing Machinery, New York, NY, USA. 25–37. isbn:0897917081 https://doi.org/10.1145/225014.225017
[11]
Lawrence Chung, Brian A Nixon, Eric Yu, and John Mylopoulos. 2012. Non-functional requirements in software engineering. 5, Springer Science & Business Media. https://doi.org/10.1007/978-1-4615-5269-7
[12]
Jane Cleland-Huang, Raffaella Settimi, Xuchang Zou, and Peter Solc. 2007. Automated classification of non-functional requirements. Requirements engineering, 12, 2 (2007), 103–120. https://doi.org/10.1007/s00766-007-0045-1
[13]
Juliet Corbin and Anselm Strauss. 2014. Basics of qualitative research: Techniques and procedures for developing grounded theory. Sage publications.
[14]
Iuliana Cosmina, Rob Harrop, Chris Schaefer, Clarence Ho, Iuliana Cosmina, Rob Harrop, Chris Schaefer, and Clarence Ho. 2017. Introducing Spring. Pro Spring 5: An In-Depth Guide to the Spring Framework and Its Tools, 1–18.
[15]
Sergio Cozzetti B de Souza, Nicolas Anquetil, and Káthia M de Oliveira. 2005. A study of the documentation essential to software maintenance. In Proceedings of the 23rd annual international conference on Design of communication: documenting & designing for pervasive information. 68–75. https://doi.org/10.1145/1085313.1085331
[16]
Github. 2019. Optimize CacheKey handling in SpringIterableConfigurationPropertySource by dreis2211 · Pull Request #16717 · spring-projects/spring-boot · GitHub. https://github.com/spring-projects/spring-boot/pull/16717 (Accessed on 10/21/2022)
[17]
Github. 2020. Avoid unnecessary sorting overhead by dreis2211 · Pull Request #24617 · spring-projects/spring-framework · GitHub. https://github.com/spring-projects/spring-framework/pull/24617 (Accessed on 10/21/2022)
[18]
Github. 2022. GitHub REST API - GitHub Docs. https://docs.github.com/en/rest (Accessed on 10/21/2022)
[19]
Github. 2022. GitHub: Where the world builds software · GitHub. https://github.com/ (Accessed on 10/21/2022)
[20]
Github. 2023. Available:. https://docs.github.com/en/github/collaborating-with-issues-and-pull-requests/about-pull-requests
[21]
Cristina Aguilera González, Laia Albors Zumel, Jesús Antonanzas Acero, Valentina Lenarduzzi, Silverio Martíncz-Fernández, and Sonia Rabanaque Rodríguez. 2021. A preliminary investigation of developer profiles based on their activities and code quality: Who does what? In 2021 IEEE 21st International Conference on Software Quality, Reliability and Security (QRS). 938–945. https://doi.org/10.1109/QRS54544.2021.00103
[22]
Georgios Gousios, Martin Pinzger, and Arie van Deursen. 2014. An exploratory study of the pull-based software development model. In Proceedings of the 36th international conference on software engineering. 345–355. https://doi.org/10.1145/2568225.2568260
[23]
Georgios Gousios, Andy Zaidman, Margaret-Anne Storey, and Arie Van Deursen. 2015. Work practices and challenges in pull-based development: The integrator’s perspective. In 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering. 1, 358–368. https://doi.org/10.1109/ICSE.2015.55
[24]
Naina Handa, Anil Sharma, and Amardeep Gupta. 2022. Framework for prediction and classification of non functional requirements: a novel vision. Cluster Computing, 25, 2 (2022), 1155–1173. https://doi.org/10.1007/s10586-021-03484-0
[25]
C. Hoskinson. 2011. Available:. http://www.politico.com/news/stories/0611/58051.html
[26]
Huaxi Jiang, Jie Zhu, Li Yang, Geng Liang, and Chun Zuo. 2021. Deeprelease: Language-agnostic release notes generation from pull requests of open-source software. In 2021 28th Asia-Pacific Software Engineering Conference (APSEC). 101–110. https://doi.org/10.1109/APSEC53868.2021.00018
[27]
Rajni Jindal, Ruchika Malhotra, Abha Jain, and Ankita Bansal. 2021. Mining Non-Functional Requirements using Machine Learning Techniques. e-Informatica Software Engineering Journal, 15, 1 (2021), https://doi.org/10.37190/e-inf
[28]
Mitchell Joblin, Sven Apel, Claus Hunsen, and Wolfgang Mauerer. 2017. Classifying developers into core and peripheral: An empirical study on count and network metrics. In 2017 IEEE/ACM 39th International Conference on Software Engineering (ICSE). 164–174. https://doi.org/10.1109/ICSE.2017.23
[29]
Eirini Kalliamvakou, Georgios Gousios, Kelly Blincoe, Leif Singer, Daniel M German, and Daniela Damian. 2016. An in-depth study of the promises and perils of mining GitHub. Empirical Software Engineering, 21, 5 (2016), 2035–2071. https://doi.org/10.1007/s10664-015-9393-5
[30]
Kamaljit Kaur and Parminder Kaur. 2022. SABDM: A self-attention based bidirectional-RNN deep model for requirements classification. Journal of Software: Evolution and Process, e2430. https://doi.org/10.1002/smr.2430
[31]
Zijad Kurtanović and Walid Maalej. 2017. Automatically classifying functional and non-functional requirements using supervised machine learning. In 2017 IEEE 25th International Requirements Engineering Conference (RE). 490–495. https://doi.org/10.1109/RE.2017.82
[32]
Johan Linaker, Sardar Muhammad Sulaman, Martin Höst, and Rafael Maiani de Mello. 2015. Guidelines for conducting surveys in software engineering v. 1.1. https://doi.org/10.1007/978-3-030-32489-6_3
[33]
Zhongxin Liu, Xin Xia, Christoph Treude, David Lo, and Shanping Li. 2019. Automatic generation of pull request descriptions. In 2019 34th IEEE/ACM International Conference on Automated Software Engineering (ASE). 176–188. https://doi.org/10.1109/ASE.2019.00026
[34]
J MacQueen. 1967. Classification and analysis of multivariate observations. In 5th Berkeley Symp. Math. Statist. Probability. 281–297.
[35]
Esmael Mohammed and Esubalew Alemneh. 2021. Identification of Architecturally Significant Non-Functional Requirement. In 2021 International Conference on Information and Communication Technology for Development for Africa (ICT4DA). 24–29. https://doi.org/10.1109/ICT4DA53266.2021.9672235
[36]
John Noll and Wei-Ming Liu. 2010. Requirements elicitation in open source software development: a case study. In Proceedings of the 3rd International Workshop on Emerging Trends in Free/Libre/Open Source Software Research and Development. 35–40. https://doi.org/10.1145/1833272.1833279
[37]
Bashar Nuseibeh. 2001. Weaving together requirements and architectures. Computer, 34, 3 (2001), 115–119. https://doi.org/10.1109/2.910904
[38]
Gabriela Robiolo, Ezequiel Scott, Santiago Matalonga, and Michael Felderer. 2019. Technical debt and waste in non-functional requirements documentation: An exploratory study. In Product-Focused Software Process Improvement: 20th International Conference, PROFES 2019, Barcelona, Spain, November 27–29, 2019, Proceedings 20. 220–235. https://doi.org/10.1007/978-3-030-35333-9_16
[39]
Peter J Rousseeuw. 1987. Silhouettes: a graphical aid to the interpretation and validation of cluster analysis. Journal of computational and applied mathematics, 20 (1987), 53–65. https://doi.org/0.1016/0377-0427(87)90125-7
[40]
Mehrdad Saadatmand, Antonio Cicchetti, and Mikael Sjödin. 2012. Toward model-based trade-off analysis of non-functional requirements. In 2012 38th Euromicro Conference on Software Engineering and Advanced Applications. 142–149. https://doi.org/10.1109/SEAA.2012.23
[41]
Walt Scacchi. 2009. Understanding requirements for open source software. In Design Requirements Engineering: A Ten-Year Perspective: Design Requirements Workshop, Cleveland, OH, USA, June 3-6, 2007, Revised and Invited Papers. 467–494.
[42]
John Slankas and Laurie Williams. 2013. Automated extraction of non-functional requirements in available documentation. In 2013 1st International workshop on natural language analysis in software engineering (NaturaLiSE). 9–16.
[43]
Daricélio Moreira Soares, Manoel Limeira de Lima Júnior, Leonardo Murta, and Alexandre Plastino. 2015. Acceptance factors of pull requests in open-source projects. In 30th Annual ACM Symposium on Applied Computing. 1541–1546. https://doi.org/10.1145/2695664.2695856
[44]
L. Sousa, A. Oliveira, W. Oizumi, S. Barbosa, A. Garcia, J. Lee, M. Kalinowski, R. de Mello, B. Fonseca, R. Oliveira, C. Lucena, and R. Paes. 2018. Identifying Design Problems in the Source Code: A Grounded Theory. In 40th International Conference on Software Engineering (ICSE ’18). ACM, New York, NY, USA. 921–931. isbn:978-1-4503-5638-1 https://doi.org/10.1145/3180155.3180239
[45]
Leonardo Sousa, Roberto Oliveira, Alessandro Garcia, Jaejoon Lee, Tayana Conte, Willian Oizumi, Rafael de Mello, Adriana Lopes, Natasha Valentim, Edson Oliveira, and Carlos Lucena. 2017. How Do Software Developers Identify Design Problems?: A Qualitative Analysis. In Proceedings of 31st Brazilian Symposium on Software Engineering (SBES’17). 12 pages. https://doi.org/10.1145/3131151.3131168
[46]
Spring. 2022. Spring | Community. https://spring.io/community (Accessed on 10/21/2022)
[47]
Spring. 2022. Spring | Home. https://spring.io/ (Accessed on 10/21/2022)
[48]
Spring. 2022. Spring | Team. https://spring.io/team (Accessed on 10/21/2022)
[49]
László Tóth and László Vidács. 2019. Comparative Study of The Performance of Various Classifiers in Labeling Non-Functional Requirements. Information Technology and Control, 48, 3 (2019), 432–445. https://doi.org/10.5755/j01.itc.48.3.21973
[50]
Chong Wang, Fan Zhang, Peng Liang, Maya Daneva, and Marten Van Sinderen. 2018. Can app changelogs improve requirements classification from app reviews? an exploratory study. In Proceedings of the 12th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement. 1–4. https://doi.org/10.1145/3239235.3267428
[51]
A Yamashita and L Moonen. 2012. Do code smells reflect important maintainability aspects? In ICSM12. 306–315. https://doi.org/10.1109/ICSM.2012.6405287
[52]
A. Yamashita and L. Moonen. 2013. Do developers care about code smells? An exploratory survey. In 20th Working Conference on Reverse Engineering. 242–251. issn:1095-1350 https://doi.org/10.1109/WCRE.2013.6671299
[53]
Jie Zou, Ling Xu, Mengning Yang, Xiaohong Zhang, and Dan Yang. 2017. Towards comprehending the non-functional requirements through developers’ eyes: An exploration of stack overflow using topic analysis. Information and Software Technology, 84 (2017), 19–32. https://doi.org/10.1016/j.infsof.2016.12.003
Index Terms
- Understanding Developers’ Discussions and Perceptions on Non-functional Requirements: The Case of the Spring Ecosystem
Recommendations
Are "non-functional" requirements really non-functional?: an investigation of non-functional requirements in practice
ICSE '16: Proceedings of the 38th International Conference on Software EngineeringNon-functional requirements (NFRs) are commonly distinguished from functional requirements by differentiating how the system shall do something in contrast to what the system shall do. This distinction is not only prevalent in research, but also ...
An Ontological Approach to Specify Conflicts among Non-Functional Requirements
ICGDA '19: Proceedings of the 2019 2nd International Conference on Geoinformatics and Data AnalysisIt is a usual practice for a user to narrate the Non-Functional Requirements (NFRs) in natural language and the requirements engineers manually try to express the same, using semi-formal or formal language notations. However, inaccurate and the ...
Toward improved traceability of non-functional requirements
TEFSE '05: Proceedings of the 3rd international workshop on Traceability in emerging forms of software engineeringThis position paper examines current practices and challenges for tracing non-functional requirements (NFRs). Anecdotal evidence suggests that many organizations do not effectively trace NFRs and that functional changes are often implemented with very ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
July 2024
2770 pages
Copyright © 2024 Owner/Author.
This work is licensed under a Creative Commons Attribution International 4.0 License.
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 12 July 2024
Published in PACMSE Volume 1, Issue FSE
Badges
Distinguished Paper
Author Tags
Qualifiers
- Research-article
Funding Sources
- CNPq
- FAPERJ PDR-10
- CAPES
- CAPES\/Procad
- FAPERJ
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 319Total Downloads
- Downloads (Last 12 months)319
- Downloads (Last 6 weeks)56
Reflects downloads up to 29 Jan 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in