short-paper

Automated generation of realistic test inputs for web APIs

Author:

Juan C. AlonsoAuthors Info & Claims

ESEC/FSE 2021: Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering

Pages 1666 - 1668

https://doi.org/10.1145/3468264.3473491

Published: 18 August 2021 Publication History

Abstract

Testing web APIs automatically requires generating input data values such as addressess, coordinates or country codes. Generating meaningful values for these types of parameters randomly is rarely feasible, which means a major obstacle for current test case generation approaches. In this paper, we present ARTE, the first semantic-based approach for the Automated generation of Realistic TEst inputs for web APIs. Specifically, ARTE leverages the specification of the API under test to search for meaningful test inputs for the API parameters in knowledge bases like DBpedia. Our approach has been integrated into RESTest, a state-of-the-art tool for API testing, achieving an unprecedented level of automation which allows to generate up to 100% more valid API calls than existing fuzzing techniques, 30% on average. Evaluation results on a set of 26 real-world APIs show that ARTE can generate realistic inputs for 7 out of every 10 parameters, outperforming related approaches.

References

[1]

2020. DHL Location Finder API. https://developer.dhl.com/api-reference/location-finder Accessed December 2020.

[2]

2020. OpenAPI Specification. https://www.openapis.org accessed December 2020.

[3]

2020. RapidAPI API directory. https://rapidapi.com/marketplace Accessed November 2020.

[4]

2020. SPARQL 1.1 Overview. https://www.w3.org/TR/2013/REC-sparql11-overview-20130321/ Accessed January 2020.

[5]

2021. Apache Jena. https://jena.apache.org/index.html Accessed February 2021.

[6]

2021. Stanford CoreNLP. https://stanfordnlp.github.io/CoreNLP/ Accessed February 2021.

[7]

Vimala Balakrishnan and Ethel Lloyd-Yemoh. 2014. Stemming and lemmatization: a comparison of retrieval performances. Lecture Notes on Software Engineering, 262–267. https://doi.org/10.7763/LNSE.2014.V2.134

[8]

Alberto Bartoli, Andrea De Lorenzo, Eric Medvet, and Fabiano Tarlao. 2016. Can a machine replace humans in building regular expressions? A case study. IEEE Intelligent Systems, 31, 6 (2016), 15–21. https://doi.org/10.1109/MIS.2016.46

Digital Library

[9]

Alberto Bartoli, Andrea De Lorenzo, Eric Medvet, Fabiano Tarlao, and Marco Virgolin. 2015. Evolutionary Learning of Syntax Patterns for Genic Interaction Extraction. In Proceedings of the 2015 on Genetic and Evolutionary Computation Conference. 1183–1190. https://doi.org/10.1145/2739480.2754706

Digital Library

[10]

A. Bartoli, A. De Lorenzo, E. Medvet, and F. Tarlao. 2016. Inference of Regular Expressions for Text Extraction from Examples. IEEE Transactions on Knowledge and Data Engineering, 28, 5 (2016), May, 1217–1230. issn:1041-4347 https://doi.org/10.1109/TKDE.2016.2515587

Digital Library

[11]

Steven Bird, Ewan Klein, and Edward Loper. 2009. Natural language processing with Python: analyzing text with the natural language toolkit. " O’Reilly Media, Inc.".

Digital Library

[12]

C. Bizer, T. Heath, and Tim Berners-Lee. 2009. Linked Data - The Story So Far. Int. J. Semantic Web Inf. Syst., 5 (2009), 1–22. https://doi.org/10.4018/jswis.2009081901

[13]

Christian Bizer, Jens Lehmann, Georgi Kobilarov, Sören Auer, Christian Becker, Richard Cyganiak, and Sebastian Hellmann. 2009. DBpedia-A crystallization point for the Web of Data. Journal of web semantics, 7, 3 (2009), 154–165. https://doi.org/10.1016/j.websem.2009.07.002

Digital Library

[14]

Roy Thomas Fielding. 2000. Architectural Styles and the Design of Network-based Software Architectures. Ph.D. Dissertation. isbn:0-599-87118-0

Digital Library

[15]

Ramanathan Guha and Dan Brickley. 2004. RDF Vocabulary Description Language 1.0: RDF Schema. W3C. https://www.w3.org/TR/2004/REC-rdf-schema-20040210/.

[16]

Daniel Jacobson, Greg Brail, and Dan Woods. 2011. APIs: A Strategy Guide. O’Reilly Media, Inc. isbn:1449308929, 9781449308926

[17]

Christopher Manning and Hinrich Schutze. 1999. Foundations of statistical natural language processing. MIT press.

Digital Library

[18]

Leonardo Mariani, Mauro Pezzè, Oliviero Riganelli, and Mauro Santoro. 2014. Link: Exploiting the Web of Data to Generate Test Inputs. In Proceedings of the 2014 International Symposium on Software Testing and Analysis. 373–384. https://doi.org/10.1145/2610384.2610397

Digital Library

[19]

Alberto Martin-Lopez, Sergio Segura, and Antonio Ruiz-Cortés. 2020. RESTest: Black-Box Constraint-Based Testing of RESTful Web APIs. In International Conference on Service-Oriented Computing. 459–475. https://doi.org/10.1007/978-3-030-65310-1_33

Digital Library

[20]

Leonard Richardson, Mike Amundsen, and Sam Ruby. 2013. RESTful Web APIs. O’Reilly Media, Inc. isbn:1449358063, 9781449358068

[21]

Kristina Toutanova, Dan Klein, Christopher D Manning, and Yoram Singer. 2003. Feature-rich part-of-speech tagging with a cyclic dependency network. In Proceedings of the 2003 conference of the North American chapter of the association for computational linguistics on human language technology-volume 1. 173–180. https://doi.org/10.3115/1073445.1073478

Digital Library

[22]

Emanuele Viglianisi, Michael Dallago, and Mariano Ceccato. 2020. RestTestGen: Automated Black-Box Testing of RESTful APIs. In International Conference on Software Testing, Verification and Validation. https://doi.org/10.1109/ICST46399.2020.00024

[23]

S Vijayarani, Ms J Ilamathi, and Ms Nithya. 2015. Preprocessing techniques for text mining-an overview. International Journal of Computer Science & Communication Networks, 5, 1 (2015), 7–16.

[24]

Denny Vrandečić and Markus Krötzsch. 2014. Wikidata: A Free Collaborative Knowledge base. Commun. ACM, 57, 10 (2014), Sept., 78–85. issn:0001-0782 https://doi.org/10.1145/2629489

Digital Library

[25]

Tanapuch Wanwarang, Nataniel P. Borges, Leon Bettscheider, and Andreas Zeller. 2020. Testing Apps With Real-World Inputs. In Proceedings of the IEEE/ACM 1st International Conference on Automation of Software Test (AST ’20). Association for Computing Machinery, New York, NY, USA. 1–10. isbn:9781450379571 https://doi.org/10.1145/3387903.3389310

Digital Library

Cited By

Wu HXu LNiu XNie CDwyer MDamian DZeller A(2022)Combinatorial testing of RESTful APIsProceedings of the 44th International Conference on Software Engineering10.1145/3510003.3510151(426-437)Online publication date: 21-May-2022
https://dl.acm.org/doi/10.1145/3510003.3510151
Alsaedi AAlhuzali ABamasag O(2022)Effective and scalable black-box fuzzing approach for modern web applicationsJournal of King Saud University - Computer and Information Sciences10.1016/j.jksuci.2022.10.00634:10(10068-10078)Online publication date: Nov-2022
https://doi.org/10.1016/j.jksuci.2022.10.006

Index Terms

Automated generation of realistic test inputs for web APIs
1. Computer systems organization
  1. Dependable and fault-tolerant systems and networks
    1. Redundancy
  2. Embedded and cyber-physical systems
    1. Embedded systems
    2. Robotics
2. Networks
  1. Network properties
    1. Network reliability

Recommendations

Link: exploiting the web of data to generate test inputs
ISSTA 2014: Proceedings of the 2014 International Symposium on Software Testing and Analysis

Applications that process complex data, such as maps, personal data, book information, travel data, etc., are becoming extremely common. Testing such applications is hard, because they require realistic and coherent test inputs that are expensive to ...
AGORA: Automated Generation of Test Oracles for REST APIs
ISSTA 2023: Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis

Test case generation tools for REST APIs have grown in number and complexity in recent years. However, their advanced capabilities for automated input generation contrast with the simplicity of their test oracles, which limit the types of failures ...
Combinatorial testing of RESTful APIs
ICSE '22: Proceedings of the 44th International Conference on Software Engineering

This paper presents RestCT, a systematic and fully automatic approach that adopts Combinatorial Testing (CT) to test RESTful APIs. RestCT is systematic in that it covers and tests not only the interactions of a certain number of operations in RESTful ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

ESEC/FSE 2021: Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering

August 2021

1690 pages

ISBN:9781450385626

DOI:10.1145/3468264

General Chairs:
Diomidis Spinellis
Athens University of Economics and Business, Greece
,
Georgios Gousios
Facebook, Netherlands / Delft University of Technology, Netherlands
,
Program Chairs:
Marsha Chechik
University of Toronto, Canada
,
Massimiliano Di Penta
University of Sannio, Italy

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGSOFT: ACM Special Interest Group on Software Engineering

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 18 August 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper

Funding Sources

FEDER/Ministerio de Ciencia e Innovación ð Agencia Estatal de Investigación
European Commission (FEDER) and Junta de Andalucia

Conference

ESEC/FSE '21

Sponsor:

SIGSOFT

ESEC/FSE '21: 29th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering

August 23 - 28, 2021

Athens, Greece

Acceptance Rates

Overall Acceptance Rate 112 of 543 submissions, 21%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
228
Total Downloads

Downloads (Last 12 months)37
Downloads (Last 6 weeks)2

Reflects downloads up to 26 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Wu HXu LNiu XNie CDwyer MDamian DZeller A(2022)Combinatorial testing of RESTful APIsProceedings of the 44th International Conference on Software Engineering10.1145/3510003.3510151(426-437)Online publication date: 21-May-2022
https://dl.acm.org/doi/10.1145/3510003.3510151
Alsaedi AAlhuzali ABamasag O(2022)Effective and scalable black-box fuzzing approach for modern web applicationsJournal of King Saud University - Computer and Information Sciences10.1016/j.jksuci.2022.10.00634:10(10068-10078)Online publication date: Nov-2022
https://doi.org/10.1016/j.jksuci.2022.10.006

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents