Nothing Special   »   [go: up one dir, main page]

skip to main content
10.1145/2896971.2896979acmconferencesArticle/Chapter ViewAbstractPublication PagesisstaConference Proceedingsconference-collections
research-article

Agile metamorphic model-based testing

Published: 14 May 2016 Publication History

Abstract

Testing becomes difficult when we cannot easily determine whether the system delivers the correct result or not. To address this issue, we have developed a framework for automated testing of NASA's DAT system using metamorphic testing principles combined with model based testing. Based on the results from using the framework to test DAT we have determined that this is a cost beneficial solution allowing for comprehensive testing that detects defects without having to develop complex testing infrastructure to determine the oracle. In our approach, DAT queries are automatically generated from a set of metamorphic testing models where each model encodes one or more of the identified equivalences. In addition, we use a scheme for generating time stamps that we use to automatically create time pairs that add another equivalence dimension.

References

[1]
M. Lindvall, D. Muthig, A. Dagnino, C. Wallin, M. Stupperich, D. Kiefer, J. May, and T. Kahk " onen, "Agile software development " in large organizations," IEEE Computer, vol. 37, no. 12, pp. 26--34, 2004. {Online}. Available: http://doi.ieeecomputersociety.org/10.1109/MC.2004.231.
[2]
D. Cohen, M. Lindvall, and P. Costa, "An introduction to agile methods," Advances in Computers, vol. 62, pp. 1--66, 2004. {Online}. Available: http://dx.doi.org/10.1016/S0065-2458(03)62001-2.
[3]
M. Lindvall, D. Ganesan, R. Ardal, and R. E. Wiegand, "Metamorphic ' model-based testing applied on NASA DAT: An experience report," in Proceedings of the 37th International Conference on Software Engineering - Volume 2, ser. ICSE '15. Piscataway, NJ, USA: IEEE Press, 2015, pp. 129--138. {Online}. Available: http://dl.acm.org/citation.cfm?id=2819009.2819030.
[4]
Data Access Toolkit (DAT) User's Guide, 2014, version 0.8.
[5]
H. Liu, F. Kuo, D. Towey, and T. Y. Chen, "How effectively does metamorphic testing alleviate the oracle problem?" IEEE Trans. Software Eng., vol. 40, no. 1, pp. 4--22, 2014. {Online}. Available: http://doi.ieeecomputersociety.org/10.1109/TSE.2013.46.
[6]
C. Schulze, D. Ganesan, M. Lindvall, D. McComas, and A. Cudmore, "Model-based testing of NASA's OSAL API - an experience report." in ISSRE, 2013, pp. 300--309.
[7]
C. Schulze, D. Ganesan, M. Lindvall, R. Cleaveland, and D. Goldman, "Assessing model-based testing: An empirical study conducted in industry," in Companion Proceedings of the 36th International Conference on Software Engineering, ser. ICSE Companion 2014. New York, NY, USA: ACM, 2014, pp. 135--144. {Online}. Available: http://doi.acm.org/10.1145/2591062.2591180.
[8]
V. Gudmundsson, C. Schulze, D. Ganesan, M. Lindvall, and R. Wiegand, "An initial evaluation of model-based testing." in ISSRE (Supplemental Proceedings), 2013, pp. 13--14.
[9]
S. Segura, R. M. Hierons, D. Benavides, and A. Ruiz-Cortes, ' "Automated metamorphic testing on the analyses of feature models," Inf. Softw. Technol., vol. 53, no. 3, pp. 245--258, Mar. 2011. {Online}. Available: http://dx.doi.org/10.1016/j.infsof.2010.11.002.
[10]
R. Guderlei and J. Mayer, "Towards automatic testing of imaging software by means of random and metamorphic testing," International Journal of Software Engineering and Knowledge Engineering, vol. 17, no. 06, pp. 757--781, 2007.
[11]
A. Groce, G. Holzmann, and R. Joshi, "Randomized differential testing as a prelude to formal verification," in Software Engineering, 2007. ICSE 2007. 29th International Conference on. IEEE, 2007, pp. 621--631.
[12]
H. Robinson, "Model-based testing." {Online}. Available: http://www.harryrobinson.net/Ding, W. and Marchionini, G. 1997. A Study on Video Browsing Strategies. Technical Report. University of Maryland at College Park.

Cited By

View all
  • (2024)MRGS‐ART: Metamorphic Relation and Group Selection Based on Adaptive Random TestingSoftware Testing, Verification and Reliability10.1002/stvr.1908Online publication date: 27-Nov-2024
  • (2021)Metamorphosis Relationship Generation Based on Fixed Memory Step Gradient Descent Method with Noise2021 IEEE 12th International Conference on Software Engineering and Service Science (ICSESS)10.1109/ICSESS52187.2021.9522332(282-286)Online publication date: 20-Aug-2021
  • (2019)Increasing Validity of Simulation Models Through Metamorphic TestingIEEE Transactions on Reliability10.1109/TR.2018.285031568:1(91-108)Online publication date: Mar-2019
  • Show More Cited By

Recommendations

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences
MET '16: Proceedings of the 1st International Workshop on Metamorphic Testing
May 2016
59 pages
ISBN:9781450341639
DOI:10.1145/2896971
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 14 May 2016

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Agile development
  2. metamorphic testing
  3. model-based testing

Qualifiers

  • Research-article

Conference

ICSE '16
Sponsor:

Upcoming Conference

ISSTA '25

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)20
  • Downloads (Last 6 weeks)8
Reflects downloads up to 25 Nov 2024

Other Metrics

Citations

Cited By

View all
  • (2024)MRGS‐ART: Metamorphic Relation and Group Selection Based on Adaptive Random TestingSoftware Testing, Verification and Reliability10.1002/stvr.1908Online publication date: 27-Nov-2024
  • (2021)Metamorphosis Relationship Generation Based on Fixed Memory Step Gradient Descent Method with Noise2021 IEEE 12th International Conference on Software Engineering and Service Science (ICSESS)10.1109/ICSESS52187.2021.9522332(282-286)Online publication date: 20-Aug-2021
  • (2019)Increasing Validity of Simulation Models Through Metamorphic TestingIEEE Transactions on Reliability10.1109/TR.2018.285031568:1(91-108)Online publication date: Mar-2019
  • (2018)Metamorphic Testing of RESTful Web APIsIEEE Transactions on Software Engineering10.1109/TSE.2017.276446444:11(1083-1099)Online publication date: 1-Nov-2018
  • (2018)Tapir: Automation Support of Exploratory Testing Using Model Reconstruction of the System Under TestIEEE Transactions on Reliability10.1109/TR.2018.279995767:2(557-580)Online publication date: Jun-2018
  • (2017)A template-based approach to describing metamorphic relationsProceedings of the 2nd International Workshop on Metamorphic Testing10.5555/3103620.3103626(3-9)Online publication date: 20-May-2017
  • (2017)A Template-Based Approach to Describing Metamorphic Relations2017 IEEE/ACM 2nd International Workshop on Metamorphic Testing (MET)10.1109/MET.2017.3(3-9)Online publication date: May-2017

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media