research-article

Link: exploiting the web of data to generate test inputs

Authors:

Leonardo Mariani,

Oliviero Riganelli,

Mauro SantoroAuthors Info & Claims

ISSTA 2014: Proceedings of the 2014 International Symposium on Software Testing and Analysis

Pages 373 - 384

https://doi.org/10.1145/2610384.2610397

Published: 21 July 2014 Publication History

Abstract

Applications that process complex data, such as maps, personal data, book information, travel data, etc., are becoming extremely common. Testing such applications is hard, because they require realistic and coherent test inputs that are expensive to generate manually and difficult to synthesize automatically. So far the research on test case generation techniques has focused mostly on generating test sequences and synthetic test inputs, and has payed little attention to the generation of complex test inputs.

This paper presents Link, a technique to automatically generate test cases for applications that process complex data. The novel idea of Link is to exploit the Web of Data to generate test data that match the semantics of the related fields, and satisfy the semantic constraints that arise among interrelated fields. Link automatically analyzes the GUI of the application under test, generates a model of the required inputs, queries DBPedia to extract the data that can be used in the tests, and uses the extracted data to generate complex system test inputs.

The experimental results show that Link can generate realistic and coherent test inputs that can exercise behaviors difficult to exercise with currently available techniques.

References

[1]

aMetro. subway on android. http://www.ametro.org, 2013.

[2]

Data crow. the ultimate cataloguer. http://sourceforge.net/projects/datacrow/, 2013.

[3]

Ldap address book. http://sourceforge.net/projects/ldapaddrbook/, 2013.

[4]

Lyrics finder. https://github.com/Danguilherme/lyrics-finder, 2013.

[5]

MetaQuerier: Exploring and integrating the deep web. http://metaquerier.cs.uiuc.edu, 2013.

[6]

Myflights. https://github.com/markrebhan/MyFlights, 2013.

[7]

Osmand. http://osmand.net, 2013.

[8]

Regexlib. http://regexlib.com, 2013.

[9]

Robotium. https://code.google.com/p/robotium/, 2013.

[10]

S. Anand, E. K. Burke, T. Y. Chen, J. Clark, M. B. Cohen, W. Grieskamp, M. Harman, M. J. Harrold, and P. McMinn. An orchestrated survey of methodologies for automated software test case generation. Journal of Systems and Software, 86(8):1978 – 2001, 2013.

Digital Library

[11]

A. Arcuri, M. Z. Iqbal, and L. Briand. Formal analysis of the effectiveness and predictability of random testing. In proceedings of the International Symposium on Software Testing and Analysis, 2010.

Digital Library

[12]

G. Becce, L. Mariani, O. Riganelli, and M. Santoro. Extracting widget descriptions from GUIs. In proceedings of the International Conference on Fundamental Approaches to Software Engineering, 2012.

Digital Library

[13]

T. Berners-Lee, R. Fielding, and L. Masinter. Uniform resource identifier (uri): Generic syntax. Request for comments: 3986, The Internet Society, 2005.

[14]

C. Bizer, T. Heath, and T. Berners-Lee. Linked data - the story so far. International Journal on Semantic Web and Information Systems, 5(3):1–2, 2009.

[15]

C. Bizer, A. Jentzsch, and R. Cyganiak. State of the lod cloud. http://lod-cloud.net/state/, 2011.

[16]

C. Bizer, J. Lehmann, G. Kobilarov, S. Auer, C. Becker, R. Cyganiak, and S. Hellmann. Dbpedia - a crystallization point for the web of data. Journal of Web Semantics, 7(3):154–165, 2009.

Digital Library

[17]

M. Bozkurt and M. Harman. Automatically generating realistic test input from web services. In proceedings of the International Symposium on Service Oriented System Engineering, 2011.

Digital Library

[18]

D. Brickley and R. V. Guha. Rdf vocabulary description language 1.0: Rdf schema. W3c recommendation, W3C, 2004.

[19]

J. Burnim and K. Sen. Heuristics for scalable dynamic test generation. In proceedings of the International Conference on Automated Software Engineering, 2008.

Digital Library

[20]

C. Cadar, D. Dunbar, and D. Engler. KLEE: Unassisted and automatic generation of high-coverage tests for complex systems programs. In proceedings of the Symposium on Operating Systems Design and Implementation, 2008.

Digital Library

[21]

K. C.-C. Chang, B. He, and Z. Zhang. Toward large scale integration: Building a MetaQuerier over databases on the web. In proceedings of the Conference on Innovative Data Systems Research, 2005.

[22]

T. Y. Chen, F.-C. Kuo, R. G. Merkel, and T. H. Tse. Adaptive random testing: The art of test case diversity. J. Syst. Softw., 83(1):60–66, 2010.

Digital Library

[23]

I. Ciupa, A. Leitner, M. Oriol, and B. Meyer. ARTOO: adaptive random testing for object-oriented software. In proceedings of the International Conference on Software engineering, 2008.

Digital Library

[24]

P. Devaki, S. Thummalapenta, N. Singhania, and S. Sinha. Eﬃcient and flexible gui test execution via test merging. In proceedings of the International Symposium on Software Testing and Analysis, 2013.

Digital Library

[25]

C. Fellbaum. Wordnet and wordnets. In K. Brown, editor, Encyclopedia of Language and Linguistics, pages 665–670. Oxford, Elsevier, second edition edition, 2005.

[26]

G. Fraser and A. Arcuri. Evolutionary generation of whole test suites. In proceedings of the International Conference on Quality Software, 2011.

Digital Library

[27]

P. Godefroid, N. Klarlund, and K. Sen. DART: Directed automated random testing. In proceedings of the International Conference on Programming Language, Design and Implementation, 2005.

Digital Library

[28]

M. Grechanik, Q. Xie, and C. Fu. Maintaining and evolving gui-directed test scripts. In proceedings of the International Conference on Software Engineering, 2009.

Digital Library

[29]

F. Gross, G. Fraser, and A. Zeller. Search-based system testing: High coverage, no false alarms. In proceedings of the International Symposium on Software Testing and Analysis, 2012.

Digital Library

[30]

H. Hemmati, A. Arcuri, and L. Briand. Achieving scalable model-based testing through test case diversity. Transactions on Software Engineering and Methodologies, 22(1):6:1–6:42, 2013.

Digital Library

[31]

IBM. IBM rational functional tester. http://www- 01.ibm.com/software/awdtools/tester/functional/, 2013.

[32]

H. Jaygarl, S. Kim, T. Xie, and C. K. Chang. OCAT: object capture-based automated testing. In proceedings of the International Symposium on Software Testing and Analysis, 2010.

Digital Library

[33]

G. Klyne and J. J. Carroll. Resource description framework (rdf): Concepts and abstract syntax. W3c recommendation, W3C, 2004.

[34]

L. Mariani, M. Pezzè, O. Riganelli, and M. Santoro. AutoBlackTest: Automatic black-box testing of interactive applications. In proceedings of the International Conerence on Software Testing, Verification and Validation, 2012.

Digital Library

[35]

P. McMinn, M. Shahbaz, and M. Stevenson. Search-based test input generation for string data types using the results of web queries. In proceedings of the International Conference on Software Testing, Verification and Validation, 2012.

Digital Library

[36]

A. M. Memon and Q. Xie. Studying the fault-detection effectiveness of GUI test cases for rapidly evolving software. Transactions on Software Engineering, 31(10):884–896, 2005.

Digital Library

[37]

A. Mesbah, A. van Deursen, and D. Roest. Invariant-based automatic testing of modern web applications. Transactions on Software Engineering, 38(1):35–53, 2012.

Digital Library

[38]

D. C. Nguyen, A. Perini, and P. Tonella. Ontology-based test generation for multiagent systems. In proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems, 2008.

Digital Library

[39]

C. Pacheco and M. D. Ernst. Randoop: feedback-directed random testing for java. In proceedings of the International Conference Companion on Object-Oriented Programming, Systems, and Applications, 2007.

Digital Library

[40]

T. V. Paschos. A survey of approximately optimal solutions to some covering and packing problems. ACM Computing Surveys, 29(2):171–209, June 1997.

Digital Library

[41]

A. Passant. Measuring semantic distance on linking data and using it for resources recommendations. In proceedings of the AAAI Spring Symposium: Linked Data Meets Artificial Intelligence, 2010.

[42]

M. Shahbaz, P. McMinn, and M. Stevenson. Automated discovery of valid test strings from the web using dynamic regular expressions collation and natural language processing. In proceedings of the International Conference on Quality Software, 2012.

Digital Library

[43]

M. Stoer and F. Wagner. A simple min-cut algorithm. Journal of the ACM, 44(4):585–591, 1997.

Digital Library

[44]

N. Tillmann and J. D. Halleux. Pex: white box test generation for .NET. In proceedings of the International Conference on Tests and Proofs, 2008.

Digital Library

[45]

M. Utting and B. Legeard. Practical Model-Based Testing: A Tools Approach. Morgan-Kaufmann, 2006.

Digital Library

[46]

W3C SPARQL Working Group. Sparql 1.1 overview. W3c recommendation, W3C, 2013.

Cited By

Rasool ZBarnett SWillie DKurniawan SBalugo SThudumu SAbdelrazek MBosch JLewis GCleland-Huang JMuccini H(2024)LLMs for Test Input Generation for Semantic ApplicationsProceedings of the IEEE/ACM 3rd International Conference on AI Engineering - Software Engineering for AI10.1145/3644815.3644948(160-165)Online publication date: 14-Apr-2024
https://dl.acm.org/doi/10.1145/3644815.3644948
Clerissi DDenaro GMobilio MMariani L(2024)DBInputs: Exploiting Persistent Data to Improve Automated GUI TestingIEEE Transactions on Software Engineering10.1109/TSE.2024.343900250:9(2412-2436)Online publication date: Sep-2024
https://doi.org/10.1109/TSE.2024.3439002
Alonso JMartin-Lopez ASegura SGarcia JRuiz-Cortes A(2023)ARTE: Automated Generation of Realistic Test Inputs for Web APIsIEEE Transactions on Software Engineering10.1109/TSE.2022.315061849:1(348-363)Online publication date: 1-Jan-2023
https://doi.org/10.1109/TSE.2022.3150618
Show More Cited By

Index Terms

Link: exploiting the web of data to generate test inputs
1. Software and its engineering
  1. Software creation and management
    1. Software verification and validation
      1. Software defect analysis
        Software testing and debugging

Recommendations

Automated generation of realistic test inputs for web APIs
ESEC/FSE 2021: Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering

Testing web APIs automatically requires generating input data values such as addressess, coordinates or country codes. Generating meaningful values for these types of parameters randomly is rarely feasible, which means a major obstacle for current test ...
The Risks of Coverage-Directed Test Case Generation
A number of structural coverage criteria have been proposed to measure the adequacy of testing efforts. In the avionics and other critical systems domains, test suites satisfying structural coverage criteria are mandated by standards. With the advent of ...
System testing for object-oriented systems with test case prioritization

This paper presents an approach to generate test cases from UML 2.0 sequence diagrams and subsequently prioritize those test cases using model information encapsulated in the sequence diagrams. The test cases generated according to the proposed approach ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

ISSTA 2014: Proceedings of the 2014 International Symposium on Software Testing and Analysis

July 2014

460 pages

ISBN:9781450326452

DOI:10.1145/2610384

General Chair:
Corina S. Păsăreanu
NASA Ames, USA
,
Program Chair:
Darko Marinov
University of Illinois at Urbana-Champaign, USA

Copyright © 2014 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGSOFT: ACM Special Interest Group on Software Engineering

In-Cooperation

SIGPLAN: ACM Special Interest Group on Programming Languages

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 21 July 2014

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

ISSTA '14

Sponsor:

SIGSOFT

ISSTA '14: International Symposium on Software Testing and Analysis

July 21 - 25, 2014

CA, San Jose, USA

Acceptance Rates

Overall Acceptance Rate 58 of 213 submissions, 27%

Upcoming Conference

ISSTA '25

Sponsor:
sigsoft

34th ACM SIGSOFT International Symposium on Software Testing and Analysis

June 25 - 28, 2025

Trondheim , Norway

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

18
Total Citations
View Citations
216
Total Downloads

Downloads (Last 12 months)12
Downloads (Last 6 weeks)1

Reflects downloads up to 14 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Rasool ZBarnett SWillie DKurniawan SBalugo SThudumu SAbdelrazek MBosch JLewis GCleland-Huang JMuccini H(2024)LLMs for Test Input Generation for Semantic ApplicationsProceedings of the IEEE/ACM 3rd International Conference on AI Engineering - Software Engineering for AI10.1145/3644815.3644948(160-165)Online publication date: 14-Apr-2024
https://dl.acm.org/doi/10.1145/3644815.3644948
Clerissi DDenaro GMobilio MMariani L(2024)DBInputs: Exploiting Persistent Data to Improve Automated GUI TestingIEEE Transactions on Software Engineering10.1109/TSE.2024.343900250:9(2412-2436)Online publication date: Sep-2024
https://doi.org/10.1109/TSE.2024.3439002
Alonso JMartin-Lopez ASegura SGarcia JRuiz-Cortes A(2023)ARTE: Automated Generation of Realistic Test Inputs for Web APIsIEEE Transactions on Software Engineering10.1109/TSE.2022.315061849:1(348-363)Online publication date: 1-Jan-2023
https://doi.org/10.1109/TSE.2022.3150618
Alsaedi AAlhuzali ABamasag O(2022)Effective and scalable black-box fuzzing approach for modern web applicationsJournal of King Saud University - Computer and Information Sciences10.1016/j.jksuci.2022.10.00634:10(10068-10078)Online publication date: Nov-2022
https://doi.org/10.1016/j.jksuci.2022.10.006
Alonso JSpinellis DGousios GChechik MDi Penta M(2021)Automated generation of realistic test inputs for web APIsProceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering10.1145/3468264.3473491(1666-1668)Online publication date: 20-Aug-2021
https://dl.acm.org/doi/10.1145/3468264.3473491
Martin-Lopez AArcuri ASegura SRuiz-Cortes A(2021)Black-Box and White-Box Test Case Generation for RESTful APIs: Enemies or Allies?2021 IEEE 32nd International Symposium on Software Reliability Engineering (ISSRE)10.1109/ISSRE52982.2021.00034(231-241)Online publication date: Oct-2021
https://doi.org/10.1109/ISSRE52982.2021.00034
Wanwarang TBorges NBettscheider LZeller A(2020)Testing Apps With Real-World InputsProceedings of the IEEE/ACM 1st International Conference on Automation of Software Test10.1145/3387903.3389310(1-10)Online publication date: 7-Oct-2020
https://dl.acm.org/doi/10.1145/3387903.3389310
Clerissi DDenaro GMobilio MMariani LGrundy JLe Goues CLo D(2020)Plug the database & play with automatic testingProceedings of the 35th IEEE/ACM International Conference on Automated Software Engineering10.1145/3324884.3416561(66-77)Online publication date: 21-Dec-2020
https://dl.acm.org/doi/10.1145/3324884.3416561
Denaro GGuglielmo LMariani LRiganelli O(2019)GUI testing in productionCompanion Proceedings of the 3rd International Conference on the Art, Science, and Engineering of Programming10.1145/3328433.3328452(1-3)Online publication date: 1-Apr-2019
https://dl.acm.org/doi/10.1145/3328433.3328452
Mariani LPezzè MZuddas DChaudron MCrnkovic IChechik MHarman M(2018)AugustoProceedings of the 40th International Conference on Software Engineering10.1145/3180155.3180162(280-290)Online publication date: 27-May-2018
https://dl.acm.org/doi/10.1145/3180155.3180162
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents