Towards Predicting Fragility in End-to-End Web Tests
Pages 387 - 392
Abstract
Automated end-to-end web tests are typically implemented as scripts that leverage dedicated libraries to simulate user interactions with web pages in a remotely controlled web browser. These tests are crucial for ensuring the functionality and reliability of web applications, as well as confirming non-regression on new releases. Still, as web applications evolve, maintaining the end-to-end test code is one of the main challenges faced by practitioners. Indeed, even minor alterations in web pages can easily break existing test code, rendering it unable to correctly locate and interact with web page elements. This issue is commonly known as web test fragility.
Many works in the literature have investigated the root causes of web test fragility and proposed approaches to repair these tests automatically or to produce more robust tests. In this short paper, we tackle the fragility problem from a different perspective. Rather than trying to repair broken tests, the proposal we are currently working on aims at estimating the extent to which a web test is prone to breakage, by considering both the source code of the web test and the web pages of the application under test. We envision that our proposal could offer substantial benefits to web testing processes, allowing testers to improve their test suites before breakages occur, saving time and money in the process. Preliminary experiments we conducted on an open-source web application showed that the computed fragility scores are highly correlated with test breakages in subsequent web application versions.
References
[1]
Ermanno Battista, Sergio Di Martino, Sergio Di Meglio, Fabio Scippacercola, and Luigi Libero Lucio Starace. 2023. E2E-Loader: A Framework to Support Performance Testing of Web Applications. In 2023 IEEE Conference on Software Testing, Verification and Validation (ICST). IEEE, 351–361.
[2]
Matteo Biagiola, Andrea Stocco, Ali Mesbah, Filippo Ricca, and Paolo Tonella. 2019. Web test dependency detection. In Proceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (Tallinn, Estonia) (ESEC/FSE 2019). Association for Computing Machinery, New York, NY, USA, 154–164. https://doi.org/10.1145/3338906.3338948
[3]
Sacha Brisset, Romain Rouvoy, Lionel Seinturier, and Renaud Pawlak. 2022. Erratum: Leveraging Flexible Tree Matching to repair broken locators in web automation scripts. Information and Software Technology 144 (2022), 106754.
[4]
Shauvik Roy Choudhary, Dan Zhao, Husayn Versee, and Alessandro Orso. 2011. Water: Web application test repair. In Proceedings of the First International Workshop on End-to-End Test Script Engineering. 24–29.
[5]
D. Clark-Carter. 2010. Measures of Central Tendency. In International Encyclopedia of Education (Third Edition) (third edition ed.), Penelope Peterson, Eva Baker, and Barry McGaw (Eds.). Elsevier, Oxford, 264–266. https://doi.org/10.1016/B978-0-08-044894-7.01343-9
[6]
Anna Corazza, Sergio Di Martino, Adriano Peron, and Luigi Libero Lucio Starace. 2021. Web application testing: Using tree kernels to detect near-duplicate states in automated model inference. In Proceedings of the 15th ACM-IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM). 1–6.
[7]
Sergio Di Meglio and Libero Lucio Starace. 2023. Replication package for the paper titled “Towards predicting fragility in end-to-end web tests”. https://zenodo.org/records/10165245
[8]
Boni García, Micael Gallego, Francisco Gortázar, and Mario Munoz-Organero. 2020. A survey of the selenium ecosystem. Electronics 9, 7 (2020), 1067.
[9]
Mouna Hammoudi, Gregg Rothermel, and Andrea Stocco. 2016. Waterfall: An incremental approach for repairing record-replay tests of web applications. (2016), 751–762.
[10]
Mouna Hammoudi, Gregg Rothermel, and Paolo Tonella. 2016. Why do Record/Replay Tests of Web Applications Break?. In 2016 IEEE International Conference on Software Testing, Verification and Validation (ICST). 180–190. https://doi.org/10.1109/ICST.2016.16
[11]
Javaria Imtiaz, Muhammad Zohaib Iqbal, 2021. An automated model-based approach to repair test suites of evolving web applications. Journal of Systems and Software 171 (2021), 110841.
[12]
Javaria Imtiaz, Salman Sherin, Muhammad Uzair Khan, and Muhammad Zohaib Iqbal. 2019. A systematic literature review of test breakage prevention and repair techniques. Information and Software Technology 113 (2019), 1–19.
[13]
Pavneet Singh Kochhar, Ferdian Thung, and David Lo. 2015. Code coverage and test suite effectiveness: Empirical study with real bugs in large systems. In 2015 IEEE 22nd international conference on software analysis, evolution, and reengineering (SANER). IEEE, 560–564.
[14]
Maurizio Leotta, Boni García, Filippo Ricca, and Jim Whitehead. 2023. Challenges of end-to-end testing with selenium WebDriver and how to face them: A survey. In 2023 IEEE Conference on Software Testing, Verification and Validation (ICST). IEEE, 339–350.
[15]
Maurizio Leotta, Filippo Ricca, and Paolo Tonella. 2021. Sidereal: Statistical adaptive generation of robust locators for web testing. Software Testing, Verification and Reliability 31, 3 (2021), e1767.
[16]
Maurizio Leotta, Andrea Stocco, Filippo Ricca, and Paolo Tonella. 2014. Reducing web test cases aging by means of robust XPath locators. In 2014 IEEE International Symposium on Software Reliability Engineering Workshops. IEEE, 449–454.
[17]
Maurizio Leotta, Andrea Stocco, Filippo Ricca, and Paolo Tonella. 2015. Using multi-locators to increase the robustness of web test cases. In 2015 IEEE 8th International Conference on Software Testing, Verification and Validation (ICST). IEEE, 1–10.
[18]
Maurizio Leotta, Andrea Stocco, Filippo Ricca, and Paolo Tonella. 2016. ROBULA+: An algorithm for generating robust XPath locators for web testing. Journal of Software: Evolution and Process 28, 3 (2016), 177–204.
[19]
Michel Nass, Emil Alégroth, Robert Feldt, Maurizio Leotta, and Filippo Ricca. 2023. Similarity-Based Web Element Localization for Robust Test Automation. ACM Trans. Softw. Eng. Methodol. 32, 3, Article 75 (apr 2023), 30 pages. https://doi.org/10.1145/3571855
[20]
M Niranjanamurthy, R Arun Kumar, and Manoj RK Sahana Srinivas. 2014. Research Study on Web ApplicationTesting using Selenium Testing Framework. International Journal of Computer Science and Mobile Computing 3, 10 (2014), 121–126.
[21]
Filippo Ricca and Andrea Stocco. 2021. Web test automation: Insights from the grey literature. In SOFSEM 2021: Theory and Practice of Computer Science: 47th International Conference on Current Trends in Theory and Practice of Computer Science, SOFSEM 2021, Bolzano-Bozen, Italy, January 25–29, 2021, Proceedings 47. Springer, 472–485.
[22]
David J Sheskin. 2003. Handbook of parametric and nonparametric statistical procedures. Chapman and Hall/CRC.
[23]
Andrea Stocco, Maurizio Leotta, Filippo Ricca, and Paolo Tonella. 2017. APOGEN: automatic page object generator for web testing. Software Quality Journal 25, 3 (2017), 1007–1039.
[24]
Andrea Stocco, Rahulkrishna Yandrapally, and Ali Mesbah. 2018. Visual web test repair. In Proceedings of the 2018 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering. 503–514.
Index Terms
- Towards Predicting Fragility in End-to-End Web Tests
Recommendations
Web test dependency detection
ESEC/FSE 2019: Proceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software EngineeringE2E web test suites are prone to test dependencies due to the heterogeneous multi-tiered nature of modern web apps, which makes it difficult for developers to create isolated program states for each test case. In this paper, we present the first ...
Automated generation of visual web tests from DOM-based web tests
SAC '15: Proceedings of the 30th Annual ACM Symposium on Applied ComputingFunctional test automation is increasingly adopted by web applications developers. In particular, 2nd generation tools overcome the limitations of 1st generation tools, based on screen coordinates, by providing APIs for easy selection and interaction ...
Carving UI Tests to Generate API Tests and API Specification
ICSE '23: Proceedings of the 45th International Conference on Software EngineeringModern web applications make extensive use of API calls to update the UI state in response to user events or server-side changes. For such applications, API-level testing can play an important role, in-between unit-level testing and UI-level (or end-...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
June 2024
728 pages
ISBN:9798400717017
DOI:10.1145/3661167
Copyright © 2024 Owner/Author.
This work is licensed under a Creative Commons Attribution International 4.0 License.
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 18 June 2024
Check for updates
Author Tags
Qualifiers
- Short-paper
- Research
- Refereed limited
Conference
EASE 2024
EASE 2024: 28th International Conference on Evaluation and Assessment in Software Engineering
June 18 - 21, 2024
Salerno, Italy
Acceptance Rates
Overall Acceptance Rate 71 of 232 submissions, 31%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 114Total Downloads
- Downloads (Last 12 months)114
- Downloads (Last 6 weeks)27
Reflects downloads up to 12 Nov 2024
Other Metrics
Citations
View Options
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML FormatGet Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in