research-article

Complete IOCO test cases: a case study

Authors:

Sofia Costa Paiva,

Adenilso Simao,

Mahsa Varshosaz,

Mohammad Reza MousaviAuthors Info & Claims

A-TEST 2016: Proceedings of the 7th International Workshop on Automating Test Case Design, Selection, and Evaluation

Pages 38 - 44

https://doi.org/10.1145/2994291.2994297

Published: 18 November 2016 Publication History

Abstract

Input/Output Transition Systems (IOTSs) have been widely used as test models in model-based testing. Traditionally, input output conformance testing (IOCO) has been used to generate random test cases from IOTSs. A recent test case generation method for IOTSs, called Complete IOCO, applies fault models to obtain complete test suites with guaranteed fault coverage for IOTSs. This paper measures the efficiency of Complete IOCO in comparison with the traditional IOCO test case generation implemented in the JTorX tool. To this end, we use a case study involving five specification models from the automotive and the railway domains. Faulty mutations of the specifications were produced in order to compare the efficiency of both test generation methods in killing them. The results indicate that Complete IOCO is more efficient in detecting deep faults in large state spaces while IOCO is more efficient in detecting shallow faults in small state spaces.

References

[1]

A. Belinfante. JTorX: A Tool for On-Line Model-Driven Test Derivation and Execution. In Proc. of TACAS’10, vol. 6015 of LNCS, pages 266–270. Springer, 2010.

Digital Library

[2]

C. Braunstein, A. E. Haxthausen, W.-l. Huang, F. Hübner, J. Peleska, U. Schulze, and L. Vu Hong. Complete model-based equivalence class testing for the etcs ceiling speed monitor. In Proc. of SEFM, vol. 8829 of LNCS, pages 380–395. Springer, 2014.

[3]

C. Braunstein, J. Peleska, U. Schulze, F. Hübner, W.-l. Huang, A. E. Haxthausen, and L. Vu Hong. A SysML test model and test suite for the ETCS ceiling speed monitor. Work Package 4 OETCS/WP4/CSM – 01/00, University of Bremen, 2014.

[4]

M. Broy, B. Jonsson, J.-P. Katoen, M. Leucker, and A. Pretschner. Model-Based Testing of Reactive Systems. vol. 3472 of LNCS. Springer, 2005.

[5]

T. S. Chow. Testing Software Design Modeled by Finite-State Machines. IEEE TSE, 4(3):178–187, 1978.

Digital Library

[6]

S. Fujiwara, G. von Bochmann, F. Khendek, M. Amalou, and A. Ghedamsi. Test Selection Based on Finite State Models. IEEE TSE, 17(6):591–603, 1991.

Digital Library

[7]

R. M. Hierons. Implementation Relations for Testing Through Asynchronous Channels. The Computer Journal, pages 107–122, 2012.

[8]

R. M. Hierons, K. Bogdanov, J. P. Bowen, R. Cleaveland, J. Derrick, J. Dick, M. Gheorghe, M. Harman, K. Kapoor, P. Krause, G. Lüttgen, A. J. H. Simons, S. Vilkomir, M. R. Woodward, and H. Zedan. Using Formal Specifications to Support Testing. ACM Computing Surveys, 41(2):9:1–9:76, 2009.

Digital Library

[9]

C. Jard and T. Jéron. TGV: Theory, Principles and Algorithms. STTT, 7(4):297–315, 2005.

Digital Library

[10]

D. Lee and M. Yannakakis. Principles and methods of testing finite state machines - a survey. Proceedings of the IEEE, 84(8):1090–1123, 1996.

[11]

S. Lity, R. Lachmann, M. Lochau, and I. Schaefer. Delta-oriented software product line test models - the body comfort system case study. Technical Report 2012-07, TU Braunschweig, 2013.

[12]

N. Noroozi, R. Khosravi, M. Mousavi, and T. Willemse. Synchronizing Asynchronous Conformance Testing. In Proc. of SEFM’11, volume 7041 of LNCS, pages 334–349. Springer, 2011.

Digital Library

[13]

S. C. Paiva and A. Simao. Generation of complete test suites from Mealy Input/Output Transition Systems. Formal Aspects of Computing, 28(1):65–78, 2016.

Digital Library

[14]

J. Peleska, A. Honisch, F. Lapschies, H. Loding, H. Schmid, P. Smuda, E. Vorobev, and C. Zahlten. A real-world benchmark model for testing concurrent real-time systems in the automotive domain. In Proc. of ICTSS’11, vol. 1, pages 146–161, Springer, 2011.

Digital Library

[15]

A. Simao and A. Petrenko. Checking Completeness of Tests for Finite State Machines. IEEE Transactions on Computers, 59(8):1023–1032, 2010.

Digital Library

[16]

A. Simao and A. Petrenko. Generating Asynchronous Test Cases from Test Purposes. Information and Software Technology, 53(11):1252–1262, 2011.

[17]

J. Tretmans. Model Based Testing with Labelled Transition Systems. In FORTEST ’08, vol. 4949 of LNCS, pp. 1–38, Springer, 2008.

Digital Library

[18]

M. van der Bijl and F. Peureux. I/O-automata Based Testing. In Model-Based Testing of Reactive Systems, vol. 3472 of LNCS, pp. 173–200. Springer 2005.

[19]

M. Weiglhofer and B. Aichernig. Unifying Input Output Conformance. vol. 5713 of LNCS, pp. 181–201. Springer, 2010.

Digital Library

[20]

M. Weiglhofer and F. Wotawa. Asynchronous Input-Output Conformance Testing. In Proc. of COMPSAC’09, vol. 1, pp. 154–159, 2009.

Digital Library

Cited By

Türker UHierons RMousavi MTyukin IGrundy J(2021)Efficient state synchronisation in model-based testing through reinforcement learningProceedings of the 36th IEEE/ACM International Conference on Automated Software Engineering10.1109/ASE51524.2021.9678566(368-380)Online publication date: 15-Nov-2021
https://dl.acm.org/doi/10.1109/ASE51524.2021.9678566
Paiva SSimão ACavalcante EDantas FBatista T(2020)An Experimental Study for Complete-IOCO TheoryProceedings of the 5th Brazilian Symposium on Systematic and Automated Software Testing10.1145/3425174.3425232(107-116)Online publication date: 20-Oct-2020
https://dl.acm.org/doi/10.1145/3425174.3425232

Index Terms

Complete IOCO test cases: a case study
1. Software and its engineering
  1. Software creation and management
    1. Software verification and validation
      1. Empirical software validation
      2. Software defect analysis
        Software testing and debugging

Recommendations

n-Complete test suites for IOCO

An n-complete test suite for automata guarantees to detect all faulty implementations with a bounded number of states. We propose a construction of such a test suite for ioco conformance on labeled transition systems, which we derive from construction ...
Generation of New March Tests with Low Test Power and High Fault Coverage by Test Sequence Reordering Using Genetic Algorithm
ARTCOM '09: Proceedings of the 2009 International Conference on Advances in Recent Technologies in Communication and Computing

March tests have been found to be very effective for fault detection and diagnosis in memories. Effectiveness of a March test is usually decided by the number of faults that can be detected using the test. The more the number of fault models covered, ...
Evaluating test suite characteristics, cost, and effectiveness of FSM-based testing methods

Context: Testing from finite state machines has been investigated due to its well-founded and sound theory as well as its practical application. There has been a recurrent interest in developing methods capable of generating test suites that detect all ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

A-TEST 2016: Proceedings of the 7th International Workshop on Automating Test Case Design, Selection, and Evaluation

November 2016

77 pages

ISBN:9781450344012

DOI:10.1145/2994291

General Chair:
Tanja Vos,
Program Chairs:
Sigrid Eldh,
Wishnu Prasetya

Copyright © 2016 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGSOFT: ACM Special Interest Group on Software Engineering

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 18 November 2016

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

FSE'16

Sponsor:

SIGSOFT

FSE'16: 24nd ACM SIGSOFT International Symposium on the Foundations of Software Engineering

November 18, 2016

WA, Seattle, USA

Upcoming Conference

ISSTA '25

Sponsor:
sigsoft

34th ACM SIGSOFT International Symposium on Software Testing and Analysis

June 25 - 28, 2025

Trondheim , Norway

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
131
Total Downloads

Downloads (Last 12 months)2
Downloads (Last 6 weeks)0

Reflects downloads up to 17 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Türker UHierons RMousavi MTyukin IGrundy J(2021)Efficient state synchronisation in model-based testing through reinforcement learningProceedings of the 36th IEEE/ACM International Conference on Automated Software Engineering10.1109/ASE51524.2021.9678566(368-380)Online publication date: 15-Nov-2021
https://dl.acm.org/doi/10.1109/ASE51524.2021.9678566
Paiva SSimão ACavalcante EDantas FBatista T(2020)An Experimental Study for Complete-IOCO TheoryProceedings of the 5th Brazilian Symposium on Systematic and Automated Software Testing10.1145/3425174.3425232(107-116)Online publication date: 20-Oct-2020
https://dl.acm.org/doi/10.1145/3425174.3425232

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents