research-article

Analyzing Test Smells Refactoring from a Developers Perspective

Authors:

Humberto Damasceno,

Emanuel Coutinho,

Ivan MachadoAuthors Info & Claims

SBQS '22: Proceedings of the XXI Brazilian Symposium on Software Quality

Article No.: 4, Pages 1 - 10

https://doi.org/10.1145/3571473.3571487

Published: 27 January 2023 Publication History

Abstract

Test smells represent a set of poorly designed tests, which can harm a test code’s maintenance and quality criteria. Although fundamental steps to understand test smells have been investigated, there is still an evident lack of studies evaluating the impact of test smell refactoring from internal quality attributes, such as size, cohesion, coupling, and complexity. In addition, the literature still lacks research that addresses the difficulties developers encounter during test smell refactoring. The paper investigates the impact of test smell refactoring from developers’ perspectives and internal quality attributes. We investigated the perceptions and difficulties encountered by 20 developers while removing 5 types of test smells in 4 open-source projects over two months. Through this experiment, we analyzed: (i) the impact that test smell refactoring has on internal quality attributes; (ii) developers’ perception of test smells as actual problems within a software system; and (iii) main difficulties encountered by developers during test smell refactoring. Our findings can help developers design a prioritization scheme for test smell refactoring and make them aware of the real benefits of test smell refactoring.

References

[1]

Mamdouh Alenezi and Khaled Almustafa. 2015. Empirical analysis of the complexity evolution in open-source software systems. International Journal of Hybrid Information Technology 8, 2 (2015), 257–266.

[2]

Wajdi Aljedaani, Anthony Peruma, Ahmed Aljohani, Mazen Alotaibi, Mohamed Wiem Mkaouer, Ali Ouni, Christian D. Newman, Abdullatif Ghallab, and Stephanie Ludi. 2021. Test Smell Detection Tools: A Systematic Mapping Study. In Evaluation and Assessment in Software Engineering (Trondheim, Norway) (EASE 2021). Association for Computing Machinery, New York, NY, USA, 170–180. https://doi.org/10.1145/3463274.3463335

Digital Library

[3]

Gabriele Bavota, Abdallah Qusef, Rocco Oliveto, Andrea De Lucia, and David Binkley. 2012. An empirical analysis of the distribution of unit test smells and their impact on software maintenance. In 2012 28th IEEE International Conference on Software Maintenance (ICSM). 56–65. https://doi.org/10.1109/ICSM.2012.6405253

Digital Library

[4]

Gabriele Bavota, Abdallah Qusef, Rocco Oliveto, Andrea De Lucia, and Dave Binkley. 2015. Are test smells really harmful? an empirical study. Empirical Software Engineering 20, 4 (2015), 1052–1094. https://doi.org/10.1007/s10664-014-9313-0

Digital Library

[5]

Moritz Beller, Georgios Gousios, Annibale Panichella, Sebastian Proksch, Sven Amann, and Andy Zaidman. 2019. Developer Testing in the IDE: Patterns, Beliefs, and Behavior. IEEE Transactions on Software Engineering 45, 3 (2019), 261–284. https://doi.org/10.1109/TSE.2017.2776152

Digital Library

[6]

Moritz Beller, Georgios Gousios, Annibale Panichella, and Andy Zaidman. 2015. When, How, and Why Developers (Do Not) Test in Their IDEs. In Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering (Bergamo, Italy) (ESEC/FSE 2015). Association for Computing Machinery, New York, NY, USA, 179–190. https://doi.org/10.1145/2786805.2786843

Digital Library

[7]

S. Berner, R. Weber, and R.K. Keller. 2005. Observations and lessons learned from automated testing. In Proceedings. 27th International Conference on Software Engineering, 2005. ICSE 2005.571–579. https://doi.org/10.1109/ICSE.2005.1553603

[8]

Antonia Bertolino. 2007. Software Testing Research: Achievements, Challenges, Dreams. In Future of Software Engineering (FOSE ’07). 85–103. https://doi.org/10.1109/FOSE.2007.25

Digital Library

[9]

James M. Bieman and Byung-Kyoo Kang. 1995. Cohesion and Reuse in an Object-Oriented System. SIGSOFT Softw. Eng. Notes 20, SI (aug 1995), 259–262. https://doi.org/10.1145/223427.211856

Digital Library

[10]

Denivan Campos, Larissa Rocha, and Ivan Machado. 2021. Developers perception on the severity of test smells: an empirical study. arXiv preprint arXiv:2107.13902(2021). https://doi.org/10.48550/arXiv.2107.13902

[11]

George Candea, Stefan Bucur, and Cristian Zamfir. 2010. Automated Software Testing as a Service. In Proceedings of the 1st ACM Symposium on Cloud Computing (Indianapolis, Indiana, USA) (SoCC ’10). Association for Computing Machinery, New York, NY, USA, 155–160. https://doi.org/10.1145/1807128.1807153

Digital Library

[12]

Alexander Chávez, Isabella Ferreira, Eduardo Fernandes, Diego Cedrim, and Alessandro Garcia. 2017. How Does Refactoring Affect Internal Quality Attributes? A Multi-Project Study. In Proceedings of the 31st Brazilian Symposium on Software Engineering (Fortaleza, CE, Brazil) (SBES’17). Association for Computing Machinery, New York, NY, USA, 74–83. https://doi.org/10.1145/3131151.3131171

Digital Library

[13]

Shyam R. Chidamber and Chris F. Kemerer. 1991. Towards a Metrics Suite for Object Oriented Design. SIGPLAN Not. 26, 11 (nov 1991), 197–211. https://doi.org/10.1145/118014.117970

Digital Library

[14]

D.P. Darcy, C.F. Kemerer, S.A. Slaughter, and J.E. Tomayko. 2005. The structural complexity of software an experimental test. IEEE Transactions on Software Engineering 31, 11 (2005), 982–995. https://doi.org/10.1109/TSE.2005.130

Digital Library

[15]

Robert Dyer, Hridesh Rajan, and Yuanfang Cai. 2012. An Exploratory Study of the Design Impact of Language Features for Aspect-Oriented Interfaces. In Proceedings of the 11th Annual International Conference on Aspect-Oriented Software Development (Potsdam, Germany) (AOSD ’12). Association for Computing Machinery, New York, NY, USA, 143–154. https://doi.org/10.1145/2162049.2162067

Digital Library

[16]

Eduardo Fernandes, Alexander Chávez, Alessandro Garcia, Isabella Ferreira, Diego Cedrim, Leonardo Sousa, and Willian Oizumi. 2020. Refactoring effect on internal quality attributes: What haven’t they told you yet?Information and Software Technology 126 (2020), 106347. https://doi.org/10.1016/j.infsof.2020.106347

[17]

Martin Fowler. 1999. Refactoring: improving the design of existing code. ilustrated edition.

[18]

César França, Fabio Q. B. da Silva, and Helen Sharp. 2020. Motivation and Satisfaction of Software Engineers. IEEE Transactions on Software Engineering 46, 2 (2020), 118–140. https://doi.org/10.1109/TSE.2018.2842201

[19]

ISO. 2011. IEC 25010: 2011 systems and software engineering–systems and software quality requirements and evaluation (square)–system and software quality models. International Organization for Standardization 34 (2011), 2910.

[20]

Amandeep Kaur and Gaurav Dhiman. 2019. A review on search-based tools and techniques to identify bad code smells in object-oriented systems. In Harmony search and nature inspired optimization algorithms. Springer, 909–921. https://doi.org/10.1007/978-981-13-0761-4_86

[21]

Dong Jae Kim. 2020. An Empirical Study on the Evolution of Test Smell. In Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering: Companion Proceedings (Seoul, South Korea) (ICSE ’20). Association for Computing Machinery, New York, NY, USA, 149–151. https://doi.org/10.1145/3377812.3382176

Digital Library

[22]

Dong Jae Kim, Tse-Hsun (Peter) Chen, and Jinqiu Yang. 2021. The Secret Life of Test Smells - an Empirical Study on Test Smell Evolution and Maintenance. Empirical Softw. Engg. 26, 5 (sep 2021), 47 pages. https://doi.org/10.1007/s10664-021-09969-1

Digital Library

[23]

Mark Lorenz and Jeff Kidd. 1994. Object-oriented software metrics: a practical guide. Prentice-Hall, Inc.

Digital Library

[24]

Ruchika Malhotra and Anuradha Chug. 2016. An empirical study to assess the effects of refactoring on software maintainability. In 2016 International Conference on Advances in Computing, Communications and Informatics (ICACCI). 110–117. https://doi.org/10.1109/ICACCI.2016.7732033

[25]

Júlio Martins, Carla Bezerra, Anderson Uchôa, and Alessandro Garcia. 2021. How Do Code Smell Co-Occurrences Removal Impact Internal Quality Attributes? A Developers’ Perspective. In Brazilian Symposium on Software Engineering(Joinville, Brazil) (SBES ’21). Association for Computing Machinery, New York, NY, USA, 54–63. https://doi.org/10.1145/3474624.3474642

Digital Library

[26]

T.J. McCabe. 1976. A Complexity Measure. IEEE Transactions on Software Engineering SE-2, 4 (1976), 308–320. https://doi.org/10.1109/TSE.1976.233837

Digital Library

[27]

Sandro Morasca. 2009. A probability-based approach for measuring external attributes of software artifacts. In 2009 3rd International Symposium on Empirical Software Engineering and Measurement. 44–55. https://doi.org/10.1109/ESEM.2009.5316048

Digital Library

[28]

Glenford J Myers, Corey Sandler, and Tom Badgett. 2011. The art of software testing. John Wiley & Sons.

Digital Library

[29]

Alessandro Orso and Gregg Rothermel. 2014. Software Testing: A Research Travelogue (2000–2014). In Future of Software Engineering Proceedings (Hyderabad, India) (FOSE 2014). Association for Computing Machinery, New York, NY, USA, 117–132. https://doi.org/10.1145/2593882.2593885

Digital Library

[30]

Matheus Paixão, Anderson Uchôa, Ana Carla Bibiano, Daniel Oliveira, Alessandro Garcia, Jens Krinke, and Emilio Arvonio. 2020. Behind the Intents: An In-Depth Empirical Study on Software Refactoring in Modern Code Review. Association for Computing Machinery, New York, NY, USA, 125–136. https://doi.org/10.1145/3379597.3387475

Digital Library

[31]

Fabio Palomba, Gabriele Bavota, Massimiliano Di Penta, Rocco Oliveto, and Andrea De Lucia. 2014. Do They Really Smell Bad? A Study on Developers’ Perception of Bad Code Smells. In 2014 IEEE International Conference on Software Maintenance and Evolution. 101–110. https://doi.org/10.1109/ICSME.2014.32

Digital Library

[32]

Fabio Palomba, Andy Zaidman, and Andrea De Lucia. 2018. Automatic Test Smell Detection Using Information Retrieval Techniques. In 2018 IEEE International Conference on Software Maintenance and Evolution (ICSME). 311–322. https://doi.org/10.1109/ICSME.2018.00040

[33]

Fabiano Pecorelli, Fabio Palomba, and Andrea De Lucia. 2021. The relation of test-related factors to software quality: A case study on apache systems. Empirical Software Engineering 26, 2 (2021), 1–42. https://doi.org/10.1007/s10664-020-09891-y

Digital Library

[34]

Anthony Peruma, Khalid Saeed Almalki, Christian D Newman, Mohamed Wiem Mkaouer, Ali Ouni, and Fabio Palomba. 2019. On the distribution of test smells in open source android applications: An exploratory study. (2019). https://dl.acm.org/doi/10.5555/3370272.3370293

[35]

Elvys Soares, Márcio Ribeiro, Guilherme Amaral, Rohit Gheyi, Leo Fernandes, Alessandro Garcia, Baldoino Fonseca, and André Santos. 2020. Refactoring Test Smells: A Perspective from Open-Source Developers. In Proceedings of the 5th Brazilian Symposium on Systematic and Automated Software Testing (Natal, Brazil) (SAST 20). Association for Computing Machinery, New York, NY, USA, 50–59. https://doi.org/10.1145/3425174.3425212

Digital Library

[36]

Davide Spadini, Fabio Palomba, Andy Zaidman, Magiel Bruntink, and Alberto Bacchelli. 2018. On the Relation of Test Smells to Software Code Quality. In 2018 IEEE International Conference on Software Maintenance and Evolution (ICSME). 1–12. https://doi.org/10.1109/ICSME.2018.00010

[37]

Davide Spadini, Martin Schvarcbacher, Ana-Maria Oprescu, Magiel Bruntink, and Alberto Bacchelli. 2020. Investigating Severity Thresholds for Test Smells. In Proceedings of the 17th International Conference on Mining Software Repositories (Seoul, Republic of Korea) (MSR ’20). Association for Computing Machinery, New York, NY, USA, 311–321. https://doi.org/10.1145/3379597.3387453

Digital Library

[38]

Amjed Tahir, Steve Counsell, and Stephen G. MacDonell. 2016. An Empirical Study into the Relationship Between Class Features and Test Smells. In 2016 23rd Asia-Pacific Software Engineering Conference (APSEC). 137–144. https://doi.org/10.1109/APSEC.2016.029

[39]

Sandhya Tarwani and Anuradha Chug. 2016. Sequencing of refactoring techniques by Greedy algorithm for maximizing maintainability. In 2016 International Conference on Advances in Computing, Communications and Informatics (ICACCI). 1397–1403. https://doi.org/10.1109/ICACCI.2016.7732243

[40]

Anderson Uchôa, Caio Barbosa, Daniel Coutinho, Willian Oizumi, Wesley K. G. Assunção, Silvia Regina Vergilio, Juliana Alves Pereira, Anderson Oliveira, and Alessandro Garcia. 2021. Predicting Design Impactful Changes in Modern Code Review: A Large-Scale Empirical Study. In 2021 IEEE/ACM 18th International Conference on Mining Software Repositories (MSR). 471–482. https://doi.org/10.1109/MSR52588.2021.00059

[41]

Arie Van Deursen, Leon Moonen, Alex Van Den Bergh, and Gerard Kok. 2001. Refactoring test code. In Proceedings of the 2nd international conference on extreme programming and flexible processes in software engineering (XP2001). Citeseer, 92–95.

[42]

Bart Van Rompaey, Bart Du Bois, Serge Demeyer, and Matthias Rieger. 2007. On The Detection of Test Smells: A Metrics-Based Approach for General Fixture and Eager Test. IEEE Transactions on Software Engineering 33, 12 (2007), 800–817. https://doi.org/10.1109/TSE.2007.70745

Digital Library

[43]

Tássio Virgínio, Luana Martins, Larissa Rocha, Railana Santana, Adriana Cruz, Heitor Costa, and Ivan Machado. 2020. JNose: Java Test Smell Detector. In Proceedings of the 34th Brazilian Symposium on Software Engineering (Natal, Brazil) (SBES ’20). Association for Computing Machinery, New York, NY, USA, 564–569. https://doi.org/10.1145/3422392.3422499

Digital Library

[44]

Guowu Xie, Jianbo Chen, and Iulian Neamtiu. 2009. Towards a better understanding of software evolution: An empirical study on open source software. In 2009 IEEE International Conference on Software Maintenance. 51–60. https://doi.org/10.1109/ICSM.2009.5306356

[45]

Vahid Garousi Yusifoğlu, Yasaman Amannejad, and Aysu Betin Can. 2015. Software test-code engineering: A systematic mapping. Information and Software Technology 58 (2015), 123–147. https://doi.org/10.1016/j.infsof.2014.06.009

Cited By

Job RHora A(2025)How and why developers implement OS-specific testsEmpirical Software Engineering10.1007/s10664-024-10571-430:1Online publication date: 1-Feb-2025
https://dl.acm.org/doi/10.1007/s10664-024-10571-4
Martins LPontillo VCosta HFerrucci FPalomba FMachado I(2024)Test code refactoring unveiled: where and how does it affect test code quality and effectiveness?Empirical Software Engineering10.1007/s10664-024-10577-y30:1Online publication date: 16-Nov-2024
https://doi.org/10.1007/s10664-024-10577-y
Ngetich MOtieno CKimwele MGitahi S(2023)Advancements in Code Restructuring: Enhancing System Quality through Object-Oriented Coding Practices2023 IEEE 27th International Conference on Intelligent Engineering Systems (INES)10.1109/INES59282.2023.10297764(000125-000130)Online publication date: 26-Jul-2023
https://doi.org/10.1109/INES59282.2023.10297764
Show More Cited By

Index Terms

Analyzing Test Smells Refactoring from a Developers Perspective
1. General and reference
  1. Cross-computing tools and techniques
    1. Measurement
2. Software and its engineering
  1. Software creation and management
    1. Software verification and validation
      1. Empirical software validation

Recommendations

Prioritising Refactoring Using Code Bad Smells
ICSTW '11: Proceedings of the 2011 IEEE Fourth International Conference on Software Testing, Verification and Validation Workshops

We investigated the relationship between six of Fowler et al.'s Code Bad Smells (Duplicated Code, Data Clumps, Switch Statements, Speculative Generality, Message Chains, and Middle Man) and software faults. In this paper we discuss how our results can ...
A comprehensive catalog of refactoring strategies to handle test smells in Java-based systems
Abstract
Test smells, detrimental coding practices that hinder high-quality test code development, pose a significant challenge in software testing and maintenance. Software refactoring, traditionally a powerful approach for addressing code smells and ...
The smell of fear: on the relation between test smells and flaky tests
Abstract
Regression testing is the activity performed by developers to check whether new modifications have not introduced bugs. A crucial requirement to make regression testing effective is that test cases are deterministic. Unfortunately, this is not ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

SBQS '22: Proceedings of the XXI Brazilian Symposium on Software Quality

November 2022

352 pages

ISBN:9781450399999

DOI:10.1145/3571473

Editors:
Edna Dias Canedo
(UnB)
,
Davi Viana
(UFMA)
,
Vinicius Garcia
(UFPE)
,
Carla Bezerra
(UFC)
,
Ismayle de Sousa Santos
(UFC)
,
Bruno Gadelha
(UFAM)
,
Ivan Machado
(UFBA)
,
Sérgio Soares
(UFPE)
,
Uirá Kulesza
(UFRN)
,
Breno de França
(UNICAMP)
,
Tayana Conte
(UFAM)
,
José Carlos Maldonado
(USP-ICMC)
,
Sheila Reinehr
(PUCPR)
,
Andreia Malucelli
(PUCPR)
,
Adriano Bessa Albuquerque
(UNIFOR)
,
Gleison Santos
(UNIRIO)
,
Ivan Machado
(UFBA)
,
Monalessa Perini Barcellos
(UFES)
,
Rodrigo Santos
(UNIRIO)
,
Sheila Reinehr
(PUCPR)
,
Tayana Conte
(UFAM)
,
Crescencio Lima
(IFBA)
,
Danilo Monteiro
(ZUP Innovation)
,
Adriana Damian
(Instituto Eldorado)
,
Larissa Rocha
(UEFS)

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 January 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

SBQS '22

SBQS '22: XXI Brazilian Symposium on Software Quality

November 7 - 10, 2022

Curitiba, Brazil

Acceptance Rates

Overall Acceptance Rate 35 of 99 submissions, 35%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
86
Total Downloads

Downloads (Last 12 months)23
Downloads (Last 6 weeks)2

Reflects downloads up to 27 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Job RHora A(2025)How and why developers implement OS-specific testsEmpirical Software Engineering10.1007/s10664-024-10571-430:1Online publication date: 1-Feb-2025
https://dl.acm.org/doi/10.1007/s10664-024-10571-4
Martins LPontillo VCosta HFerrucci FPalomba FMachado I(2024)Test code refactoring unveiled: where and how does it affect test code quality and effectiveness?Empirical Software Engineering10.1007/s10664-024-10577-y30:1Online publication date: 16-Nov-2024
https://doi.org/10.1007/s10664-024-10577-y
Ngetich MOtieno CKimwele MGitahi S(2023)Advancements in Code Restructuring: Enhancing System Quality through Object-Oriented Coding Practices2023 IEEE 27th International Conference on Intelligent Engineering Systems (INES)10.1109/INES59282.2023.10297764(000125-000130)Online publication date: 26-Jul-2023
https://doi.org/10.1109/INES59282.2023.10297764
Simatupang JSoeparno HGaol FArifin Y(2023)A Survey of Refactoring Techniques to Maximize Code Coverage Metric2023 15th International Congress on Advanced Applied Informatics Winter (IIAI-AAI-Winter)10.1109/IIAI-AAI-Winter61682.2023.00015(32-36)Online publication date: 11-Dec-2023
https://doi.org/10.1109/IIAI-AAI-Winter61682.2023.00015

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents