research-article

A regression test selection technique for embedded software

Authors:

Swarnendu Biswas,

Manoranjan SatpathyAuthors Info & Claims

ACM Transactions on Embedded Computing Systems (TECS), Volume 13, Issue 3

Article No.: 47, Pages 1 - 39

https://doi.org/10.1145/2539036.2539043

Published: 24 December 2013 Publication History

Abstract

The current approaches for regression test selection of embedded programs are usually based on data- and control-dependency analyses, often augmented with human reasoning. Existing techniques do not take into account additional execution dependencies which may exist among code elements in such programs due to features such as tasks, task deadlines, task precedences, and intertask communications. In this context, we propose a model-based regression test selection technique for such programs. Our technique first constructs a graph model of the program; the proposed graph model has been designed to capture several characteristics of embedded programs, such as task precedence order, priority, intertask communication, timers, exceptions and interrupt handlers, which we consider important for regression-test selection. Our regression test selection technique selects test cases based on an analysis of the constructed graph model. We have implemented our technique to realize a prototype tool. The experimental results obtained using this tool show that, on average, our approach selects about 28.33% more regression test cases than those selected by a traditional approach. We observed that, on average, 36.36% of the fault-revealing test cases were overlooked by the existing regression test selection technique.

References

[1]

Aho, A., Sethi, R., and Ullman, J. 2008. Compilers: Principles, Techniques and Tools 2nd Ed. Dorling Kindersley (India) Pvt Ltd.

Digital Library

[2]

Allen, M. and Horwitz, S. 2003. Slicing java programs that throw and catch exceptions. In Proceedings of the ACM SIGPLAN Workshop on Partial Evaluation and Semantics-Based Program Manipulation (PEPM'03). ACM, New York, NY, 44--54.

Digital Library

[3]

Bates, S. and Horwitz, S. 1993. Incremental program testing using program dependence graphs. In Proceedings of the Conference Record of the 20th ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages. ACM, New York, NY, 384--396.

Digital Library

[4]

Binkley, D. 1997. Semantics guided regression test cost reduction. IEEE Trans. Softw. Eng. 23, 8, 498--516.

Digital Library

[5]

Biswas, S. 2011. Model-based regression test selection and optimization for embedded programs. M.S. thesis, Indian Institute of Technology, Kharagpur, India.

[6]

Biswas, S., Mall, R., Satpathy, M., and Sukumaran, S. 2009. A model-based regression test selection approach for embedded applications. ACM SIGSOFT Softw. Eng. Notes 34, 4, 1--9.

Digital Library

[7]

Biswas, S., Mall, R., Satpathy, M., and Sukumaran, S. 2011. Task dependency analysis for regression test selection of embedded programs. IEEE Embed. Syst. Lett. 3, 4, 117--120.

Digital Library

[8]

Cartaxo, E., Machado, P., and Neto, F. 2011. On the use of a similarity function for test case selection in the context of model-based testing. Softw. Test. Verification Reliab. 21, 2, 75--100.

Digital Library

[9]

Cleve, A., Henrard, J., and Hainaut, J. 2006. Data reverse engineering using system dependency graphs. In Proceedings of the 13th Working Conference on Reverse Engineering. IEEE Computer Society, Los Alamitos, CA, 157--166.

Digital Library

[10]

Do, H., Mirarab, S., Tahvildari, L., and Rothermel, G. 2010. The effects of time constraints on test case prioritization: A series of controlled experiments. IEEE Trans. Softw. Eng. 36, 5, 593--617.

Digital Library

[11]

Ferrante, J., Ottenstein, K., and Warren, J. 1987. The program dependence graph and its use in optimization. ACM Trans. Program. Lang. Syst. 9, 3, 319--349.

Digital Library

[12]

Guan, J., Offutt, J., and Ammann, P. 2006. An industrial case study of structural testing applied to safety-critical embedded software. In Proceedings of the ACM/IEEE International Symposium on Empirical Software Engineering. ACM, New York, NY, 272--277.

Digital Library

[13]

Harrold, M., Jones, J., Li, T., Liang, D., Orso, A., Pennings, M., Sinha, S., Spoon, S. A., and Gujarathi, A. 2001. Regression test selection for java software. In Proceedings of the 16th ACM SIGPLAN Conference on Object-Oriented Programming, Systems, Languages and Applications. ACM, New York, NY, 312--326.

Digital Library

[14]

Hatley, D. and Pirbhai, I. 1987. Strategies for Real-Time System Specification. Dorset House Publishing Company.

Digital Library

[15]

Horwitz, S., Reps, T., and Binkley, D. 1990. Interprocedural slicing using dependence graphs. ACM Trans. Program. Lang. Syst. 12, 1, 26--61.

Digital Library

[16]

Jiang, S., Zhou, S., Shi, Y., and Jiang, Y. 2006. Improving the preciseness of dependence analysis using exception analysis. In Proceedings of the 15th IEEE International Conference on Computing. IEEE Computer Society, Los Alamitos, CA, 277--282.

Digital Library

[17]

Kapfhammer, G. 2004. The Computer Science Handbook 2nd Ed. CRC Press, Boca Raton, FL, (Chapter on Software Testing.)

[18]

Knuth, D. 1974. Structured programming with go to statements. ACM Comput. Surv. 6, 4, 261--301.

Digital Library

[19]

Kondoh, H. and Futatsugi, K. 2006. To use or not to use the goto statement: Programming styles viewed from Hoare Logic. Sci. Comput. Program. 60, 1, 82--116.

Digital Library

[20]

Leung, H. and White, L. 1989. Insights into regression testing. In Proceedings of the Conference on Software Maintenance. 60--69.

[21]

Liang, D. and Harrold, M. 1998. Slicing objects using system dependence graphs. In Proceedings of the International Conference on Software Maintenance. IEEE Computer Society, Los Alamitos, CA, 358--367.

Digital Library

[22]

Mall, R. 2007. Real-Time Systems Theory and Practice 1st Ed. Pearson Education.

Digital Library

[23]

Marwedel, P. 2007. Embedded System Design. Springer.

Digital Library

[24]

Mathworks. 2011. MATLAB. http://www.mathworks.com.

[25]

McConell, S. 2004. Code Complete: A Practical Handbook of Software Construction 2nd Ed. Microsoft Press.

[26]

Netkow, M. and Brylow, D. 2010. Xest: An automated framework for regression testing of embedded software. In Proceedings of the Workshop on Embedded Systems Education (WESE'10). ACM, New York, NY, 7:1--7:8.

Digital Library

[27]

Orso, A., Shi, N., and Harrold, M. 2004. Scaling regression testing to large software systems. In Proceedings of the 12th ACM SIGSOFT 12th International Symposium on Foundations of Software Engineering. ACM, New York, NY, 241--251.

Digital Library

[28]

OSEK. 2001. OSEK/VDX time-triggered operating system specification 1.0. http://portal.osek-vdx.org.

[29]

Romanovsky, A., Xu, J., and Randell, B. 1998. Exception handling in object-oriented real-time distributed systems. In Proceedings of the 1st IEEE International Symposium on Object-Oriented Real-Time Distributed Computing (ISORC'98). IEEE Computer Society, Los Alamitos, CA 32--42.

Digital Library

[30]

Rothermel, G. and Harrold, M. 1996. Analyzing regression test selection techniques. IEEE Trans. Softw. Eng. 22, 8, 529--551.

Digital Library

[31]

Rothermel, G. and Harrold, M. 1997. A safe, efficient regression test selection technique. ACM Trans. Softw. Eng. Methodol. 6, 2, 173--210.

Digital Library

[32]

Sales, J. 2005. Symbian OS Internals: Real-Time Kernel Programming. John Wiley & Sons.

Digital Library

[33]

Salewski, F. and Taylor, A. 2007. Fault handling in FPGAs and microcontrollers in safety-critical embedded applications: A comparative survey. In Proceedings of the ACM/IEEE International Symposium on Empirical Software Engineering (ISESE'06). 124--131.

Digital Library

[34]

Sangiovanni-Vincentelli, A. and Natale, M. D. 2007. Embedded system design for automotive applications. IEEE Computer 40, 42--51.

Digital Library

[35]

Schotland, T. and Petersen, P. 2011. Exception Handling in C without C++. http://www.on-time.com/ddj0011.htm.

[36]

Seo, J., Ki, Y., Choi, B., and La, K. 2008. Which spot should I test for effective embedded software testing&quest; In Proceedings of the 2nd International Conference on Secure System Integration and Reliability Improvement (SSIRI'08). IEEE Computer Society, Los Alamitos, CA, 135--142.

Digital Library

[37]

Silberschatz, A., Galvin, P., and Gagne, G. 2010. Operating System Concepts 8th Ed. Wiley India Pvt Ltd.

Digital Library

[38]

Sinha, S., Harrold, M., and Rothermel, G. 1999. System-dependence-graph-based slicing of programs with arbitrary interprocedural control flow. In Proceedings of the 21st International Conference on Software Engineering. ACM, New York, NY, 432--441.

Digital Library

[39]

Sinha, S. and Harrold, M. J. 1998. Analysis of programs with exception-handling constructs. In Proceedings of the International Conference on Software Maintenance (ICSM'98). IEEE Computer Society, 348.

Digital Library

[40]

Sundmark, D., Pettersson, A., Eldh, S., Ekman, M., and Thane, H. 2007. Efficient system-level testing of embedded real-time software. In Proceedings of the Work in Progress Session of the 17th Eurmicro Conference on Real-Time System. 53--56.

[41]

Vahid, F. and Givargis, T. 2002. Embedded System Design: A Unified Hardware/Software Introduction 1st Ed. John Wiley & Sons.

Digital Library

[42]

Vokolos, F. and Frankl, P. 1997. Pythia: A regression test selection tool based on textual differencing. In Proceedings of the 3rd International Conference on Reliability, Quality & Safety of Software-Intensive Systems (ENCRESS'97). Chapman & Hall, Ltd., London, 3--21.

Digital Library

[43]

Ward, P. and Mellor, S. 1991. Structured Development for Real-Time Systems. Prentice Hall Professional Technical Reference.

Digital Library

[44]

Wind River Systems. 2010. Wind River VxWorks: Embedded RTOS with support for POSIX and SMP. http://www.windriver.com/products/vxworks/.

[45]

Zheng, J., Robinson, B., Williams, L., and Smiley, K. 2006. Applying regression test selection for COTS-based applications. In Proceedings of the 28th International Conference on Software Engineering (ICSE'06). ACM, New York, NY, 512--522.

Digital Library

Cited By

Babaei MDingel J(2023)Efficient regression testing of distributed real-time reactive systems in the context of model-driven developmentSoftware and Systems Modeling10.1007/s10270-023-01086-522:5(1565-1587)Online publication date: 6-Mar-2023
https://doi.org/10.1007/s10270-023-01086-5
Babaei MDingel J(2021)Efficient Replay-based Regression Testing for Distributed Reactive Systems in the Context of Model-driven Development2021 ACM/IEEE 24th International Conference on Model Driven Engineering Languages and Systems (MODELS)10.1109/MODELS50736.2021.00018(89-100)Online publication date: Oct-2021
https://doi.org/10.1109/MODELS50736.2021.00018
Babaei MDingel J(2021)MRegTest: A Replay-Based Regression Testing Tool for Distributed UML-RT Models2021 ACM/IEEE International Conference on Model Driven Engineering Languages and Systems Companion (MODELS-C)10.1109/MODELS-C53483.2021.00095(620-624)Online publication date: Oct-2021
https://doi.org/10.1109/MODELS-C53483.2021.00095
Show More Cited By

Index Terms

A regression test selection technique for embedded software
1. Software and its engineering
  1. Software creation and management
    1. Software verification and validation
      1. Software defect analysis
        Software testing and debugging

Recommendations

A model-based regression test selection approach for embedded applications

Regression test selection techniques for embedded programs have scarcely been reported in the literature. In this paper, we propose a model-based regression test selection technique for embedded programs. Our proposed model, in addition to capturing the ...
A safe, efficient regression test selection technique

Regression testing is an expensive but necessary maintenance activity performed on modified software to provide confidence that changes are correct and do not adversely affect other portions of the softwore. A regression test selection technique choses, ...
Empirical Studies of a Safe Regression Test Selection Technique

Regression testing is an expensive testing procedure utilized to validate modified software. Regression test selection techniques attempt to reduce the cost of regression testing by selecting a subset of a program's existing test suite. Safe regression ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Transactions on Embedded Computing Systems

ACM Transactions on Embedded Computing Systems Volume 13, Issue 3

December 2013

385 pages

ISSN:1539-9087

EISSN:1558-3465

DOI:10.1145/2539036

Issue’s Table of Contents

Copyright © 2013 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Journal Family

ACM Journals for the Design of Smart and Connected Systems

Publication History

Published: 24 December 2013

Accepted: 01 September 2012

Revised: 01 May 2012

Received: 01 August 2011

Published in TECS Volume 13, Issue 3

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
380
Total Downloads

Downloads (Last 12 months)9
Downloads (Last 6 weeks)0

Reflects downloads up to 29 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Babaei MDingel J(2023)Efficient regression testing of distributed real-time reactive systems in the context of model-driven developmentSoftware and Systems Modeling10.1007/s10270-023-01086-522:5(1565-1587)Online publication date: 6-Mar-2023
https://doi.org/10.1007/s10270-023-01086-5
Babaei MDingel J(2021)Efficient Replay-based Regression Testing for Distributed Reactive Systems in the Context of Model-driven Development2021 ACM/IEEE 24th International Conference on Model Driven Engineering Languages and Systems (MODELS)10.1109/MODELS50736.2021.00018(89-100)Online publication date: Oct-2021
https://doi.org/10.1109/MODELS50736.2021.00018
Babaei MDingel J(2021)MRegTest: A Replay-Based Regression Testing Tool for Distributed UML-RT Models2021 ACM/IEEE International Conference on Model Driven Engineering Languages and Systems Companion (MODELS-C)10.1109/MODELS-C53483.2021.00095(620-624)Online publication date: Oct-2021
https://doi.org/10.1109/MODELS-C53483.2021.00095
(2018)Hierarchical regression test case selection using slicingInternational Journal of Computational Science and Engineering10.1504/IJCSE.2017.08288214:2(179-197)Online publication date: 19-Dec-2018
https://dl.acm.org/doi/10.1504/IJCSE.2017.082882
Rosero RGómez ORodríguez G(2016)15 Years of Software Regression Testing Techniques — A SurveyInternational Journal of Software Engineering and Knowledge Engineering10.1142/S021819401630001326:05(675-689)Online publication date: Jun-2016
https://doi.org/10.1142/S0218194016300013

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents