short-paper

Maintaining systematic literature reviews: benefits and drawbacks

Authors:

Vilmar Nepomuceno,

Sergio SoaresAuthors Info & Claims

ESEM '18: Proceedings of the 12th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement

Article No.: 47, Pages 1 - 4

https://doi.org/10.1145/3239235.3267432

Published: 11 October 2018 Publication History

Get Access

Abstract

Background: Maintenance and traceability (versioning) are constant concerns in Software Engineering (SE), however, few works related to these topics in Systematic Literature Reviews (SLR) were found. Goal: The goal of this research is to elucidate how SLRs can be maintained and what are the benefits and drawbacks in this process. Method: This work presents a survey where experienced researchers that conducted SLRs between 2011 and 2015 answered questions about maintenance and traceability and, using software maintenance concepts, it addresses the SLRs maintenance process. From the 79 e-mails sent we reach 28 answers. Results: 19 of surveyed researchers have shown interest in keeping their SLRs up-to-date, but they have expressed concerns about the effort to be made to accomplish it. It was also observed that 20 participants would be willing to share their SLRs in common repositories, such as GitHub. Conclusions: There is a need to perform maintenance on SLRs. Thus, we are proposing a SLR maintenance process, taking into account some benefits and drawbacks identified during our study and presented through the paper.

References

[1]

F. Q. B. da Silva, A. L. M. Santos, S. Soares, A. C. C. França, C. V. F. Monteiro, and F. F. Maciel. 2011. Six Years of Systematic Literature Reviews in Software Engineering: An Updated Tertiary Study. Inf. Softw. Technol. 53, 9 (Sept. 2011), 899--913.

Digital Library

Google Scholar

[2]

K. R. Felizardo, E. Mendes, M. Kalinowski, E. F. Souza, and N. L. Vijaykumar. 2016. Using Forward Snowballing to Update Systematic Reviews in Software Engineering (ESEM '16).

Digital Library

Google Scholar

[3]

J. Higgins and S. Green(editors). 2011. Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 {updated March 2011}. Available from www.cochrane-handbook.org.

Google Scholar

[4]

S. Jalali and C. Wohlin. 2012. Systematic literature studies: Database searches vs. backward snowballing (ESEM '12).

Digital Library

Google Scholar

[5]

B. Kitchenham, O. Pearl Brereton, and D. Budgen. 2008. Protocol for extending a tertiary study of systematic literature reviews in software engineering.

Google Scholar

[6]

B. Kitchenham, O. Pearl Brereton, D. Budgen, M. Turner, J. Bailey, and S. Linkman. 2009. Systematic literature reviews in software engineering - A systematic literature review. Information and Software Technology 51, 1 (2009), 7 -- 15.

Digital Library

Google Scholar

[7]

B. Kitchenham, O. Pearl Brereton, Z. Li, D. Budgen, and A. Burn. 2011. Repeatability of systematic literature reviews. In 15th Annual Conference on Evaluation Assessment in Software Engineering (EASE 2011). 46--55.

Crossref

Google Scholar

[8]

B. Kitchenham and S. Charters. 2007. Guidelines for performing Systematic Literature Reviews in Software Engineering.

Google Scholar

[9]

B. Kitchenham, R. Pretorius, D. Budgen, O. Pearl Brereton, M. Turner, M. Niazi, and S. Linkman. 2010. Systematic Literature Reviews in Software Engineering - A Tertiary Study. Inf. Softw. Technol. 52, 8 (2010), 792--805.

Digital Library

Google Scholar

[10]

S. MacDonell, M. Shepperd, B. Kitchenham, and E. Mendes. 2010. How Reliable Are Systematic Reviews in Empirical Software Engineering? IEEE Trans. Softw. Eng. 36, 5 (2010), 676--687.

Digital Library

Google Scholar

[11]

L. M. Garcés Rodriguez, K. R. Felizardo, L. B. R. Oliveira, and E. Y. Nakagawa. 2017. An Experience Report on Update of Systematic Literature Reviews. In The 29th International Conference on Software Engineering and Knowledge Engineering.

Google Scholar

[12]

Ian Sommerville. 2010. Software Engineering (9th ed.). Addison-Wesley Publishing Company, USA.

Digital Library

Google Scholar

[13]

C. Wohlin. 2016. Second-generation Systematic Literature Studies Using Snowballing (EASE '16).

Digital Library

Google Scholar

Cited By

View all

Suescún-Monsalve ESampaio-do-Prado-Leite JPardo-Calvache C(2022)Semi-Automatic Mapping Technique Using Snowballing to Support Massive Literature Searches in Software EngineeringRevista Facultad de Ingeniería10.19053/01211129.v31.n60.2022.1418931:60(e14189)Online publication date: 21-May-2022
https://doi.org/10.19053/01211129.v31.n60.2022.14189
Nepomuceno VSoares S(2020)Avoiding Plagiarism in Systematic Literature ReviewsProceedings of the 14th ACM / IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM)10.1145/3382494.3422170(1-6)Online publication date: 5-Oct-2020
https://dl.acm.org/doi/10.1145/3382494.3422170
Díaz OMedina HAnfurrutia FAli SGarousi V(2019)Coding-Data Portability in Systematic Literature ReviewsProceedings of the 23rd International Conference on Evaluation and Assessment in Software Engineering10.1145/3319008.3319025(178-187)Online publication date: 15-Apr-2019
https://dl.acm.org/doi/10.1145/3319008.3319025

Index Terms

Maintaining systematic literature reviews: benefits and drawbacks
1. General and reference
  1. Cross-computing tools and techniques
    1. Empirical studies

Recommendations

Towards Sustainability of Systematic Literature Reviews
ESEM '21: Proceedings of the 15th ACM / IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM)

Background: The software engineering community has increasingly conducted systematic literature reviews (SLR) as a means to summarize evidence from different studies and bring to light the state of the art of a given research topic. While SLR provide ...
Avoiding Plagiarism in Systematic Literature Reviews: An Update Concern
ESEM '20: Proceedings of the 14th ACM / IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM)

Background: The number of Systematic Literature Reviews published in Software Engineering has been increasing in recent years. Due to this fact, one point not addressed is the possibility of plagiarism when dealing with SLRs, especially when dealing ...
Guidelines for snowballing in systematic literature studies and a replication in software engineering
EASE '14: Proceedings of the 18th International Conference on Evaluation and Assessment in Software Engineering

Background: Systematic literature studies have become common in software engineering, and hence it is important to understand how to conduct them efficiently and reliably.

Objective: This paper presents guidelines for conducting literature reviews using ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

ESEM '18: Proceedings of the 12th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement

October 2018

487 pages

ISBN:9781450358231

DOI:10.1145/3239235

General Chair:
Markku Oivo
University of Oulu, Finland
,
Program Chairs:
Daniel Méndez
Technical University of Munich, Germany
,
Audris Mockus
University of Tennessee

© 2018 Association for Computing Machinery. ACM acknowledges that this contribution was authored or co-authored by an employee, contractor or affiliate of a national government. As such, the Government retains a nonexclusive, royalty-free right to publish or reproduce this article, or to allow others to do so, for Government purposes only.

In-Cooperation

IEEE CS

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 11 October 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper

Conference

ESEM '18

Sponsor:

SIGSOFT

ESEM '18: ACM / IEEE International Symposium on Empirical Software Engineering and Measurement

October 11 - 12, 2018

Oulu, Finland

Acceptance Rates

Overall Acceptance Rate 130 of 594 submissions, 22%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
301
Total Downloads

Downloads (Last 12 months)48
Downloads (Last 6 weeks)7

Reflects downloads up to 13 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Suescún-Monsalve ESampaio-do-Prado-Leite JPardo-Calvache C(2022)Semi-Automatic Mapping Technique Using Snowballing to Support Massive Literature Searches in Software EngineeringRevista Facultad de Ingeniería10.19053/01211129.v31.n60.2022.1418931:60(e14189)Online publication date: 21-May-2022
https://doi.org/10.19053/01211129.v31.n60.2022.14189
Nepomuceno VSoares S(2020)Avoiding Plagiarism in Systematic Literature ReviewsProceedings of the 14th ACM / IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM)10.1145/3382494.3422170(1-6)Online publication date: 5-Oct-2020
https://dl.acm.org/doi/10.1145/3382494.3422170
Díaz OMedina HAnfurrutia FAli SGarousi V(2019)Coding-Data Portability in Systematic Literature ReviewsProceedings of the 23rd International Conference on Evaluation and Assessment in Software Engineering10.1145/3319008.3319025(178-187)Online publication date: 15-Apr-2019
https://dl.acm.org/doi/10.1145/3319008.3319025

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

Towards Sustainability of Systematic Literature Reviews

Avoiding Plagiarism in Systematic Literature Reviews: An Update Concern

Guidelines for snowballing in systematic literature studies and a replication in software engineering