Article

Demand-driven structural testing with dynamic instrumentation

Authors:

Jonathan Misurda,

James A. Clause,

Juliya L. Reed,

Bruce R. Childers,

Mary Lou SoffaAuthors Info & Claims

ICSE '05: Proceedings of the 27th international conference on Software engineering

Pages 156 - 165

https://doi.org/10.1145/1062455.1062496

Published: 15 May 2005 Publication History

Abstract

Producing reliable and robust software has become one of the most important software development concerns in recent years. Testing is a process by which software quality can be assured through the collection of information. While testing can improve software reliability, current tools typically are inflexible and have high over-heads, making it challenging to test large software projects. In this paper, we describe a new scalable and flexible framework for testing programs with a novel demand-driven approach based on execution paths to implement test coverage. This technique uses dynamic instrumentation on the binary code that can be inserted and removed on-the-fly to keep performance and memory overheads low. We describe and evaluate implementations of the framework for branch, node and defuse testing of Java programs. Experimental results for branch testing show that our approach has, on average, a 1.6 speed up over static instrumentation and also uses less memory.

References

[1]

T. Ball and J. R. Larus, "Efficient path profiling", Int'l. Symp. on Microarchitecture, 1996.T. Ball and J. R. Larus, "Efficient path profiling", Int'l. Symp. on Microarchitecture, 1996.

Digital Library

[2]

M. Burke, J-D. Choi, S. Sink, et al., "The Jalapeno dynamic optimizing compiler for Java", ACM Java Grande Conference, 1999.

Digital Library

[3]

B. Childers, M. L. Soffa, J. Beaver et al., "SoftTest: A framework for software testing of Java programs", Eclipse Technology eXchange Workshop, 2003.

Digital Library

[4]

Clover, http://www.cenqua.com/clover/.

[5]

Eclipse Integrated Development Environment, http://www.eclipse.org

[6]

P. G. Frankl and E. J. Weyuker, "An applicable family of data flow testing criteria", IEEE Trans. on Software Engineering, 14(10), October 1988.

Digital Library

[7]

P. G. Frankl, S. N. Weiss, and E. J. Weyuker, "ASSET: A system to select and evaluate tests", Proceedings of the IEEE Conference on Software Tools, 1985.

[8]

M. J. Harrold and M. L. Soffa, "Interprocedural data flow testing", Testing, Analysis and Verification Symp., 1989.

Digital Library

[9]

J. Hollingsworth, B. Miller, M. Goncalves, et al., "MDL: A language and compiler for dynamic program instrumentation", Conf. on Parallel Architecture and Compilation Techniques, 1997.

Digital Library

[10]

IBM, Rational PurifyPlus, http://www.ibm.com/rational.

[11]

JCover, http://www.codework.com/JCover/

[12]

P. Kessler, "Fast breakpoints: Design and implementation", ACM SIGPLAN Conf. on Programming Languages, Design and Implementation, 1990.

Digital Library

[13]

B. P. Miller, M. D. Callaghan, J. M. Cargille, J. K. Holling-sworth, et al., "The Paradyn parallel performance measurement tools", IEEE Computer, 11(28), 1995.

Digital Library

[14]

J. Misurda, J. Clause, J. Reed, B. R. Childers, and M. L. Soffa, "Jazz: A Tool for Demand-Driven Structural Testing", International Conference on Compiler Construction, 2005.

Digital Library

[15]

L. Osterweil et al., "Strategic directions in software quality", ACM Computing Surveys, Vol. 4, 1996.

Digital Library

[16]

C. Pavlopoulou and M. Young, "Residual test coverage monitoring", Int'l. Conf. on Software Engineering, 1999.

Digital Library

[17]

W. Perry, Effective Methods for Software Testing, John Wiley & Sons, Inc., New York, New York, 1995.

Digital Library

[18]

Pin, http://rogue.colorado.edu/Pin/

[19]

S. Rapps and E. Weyuker, "Selecting software test data using data flow information", IEEE Trans. on Software Engineering, 11(4):367--375, 1985.

Digital Library

[20]

Standard Performance Evaluation Corporation, http://www.spec.org/jvm98

[21]

M. Tikir and J. Hollingsworth, "Efficient instrumentation for code coverage testing", Int'l. Symp. on Software Testing and Analysis, 2002.

Digital Library

Cited By

Burchell HLarose OKaleba SMarr SBruno RMoss E(2023)Don’t Trust Your Profiler: An Empirical Study on the Precision and Accuracy of Java ProfilersProceedings of the 20th ACM SIGPLAN International Conference on Managed Programming Languages and Runtimes10.1145/3617651.3622985(100-113)Online publication date: 19-Oct-2023
https://dl.acm.org/doi/10.1145/3617651.3622985
Khorram FBousse EGarmendia AMottu JSunyé GWimmer MFischer BBurgueño LCazzola W(2022)From Coverage Computation to Fault Localization: A Generic Framework for Domain-Specific LanguagesProceedings of the 15th ACM SIGPLAN International Conference on Software Language Engineering10.1145/3567512.3567532(235-248)Online publication date: 29-Nov-2022
https://dl.acm.org/doi/10.1145/3567512.3567532
Chatterjee NChakrabarti ADas P(2018)Software Regression and Migration Assistance Using Dynamic InstrumentationAdvanced Computing and Systems for Security10.1007/978-981-13-3250-0_12(159-170)Online publication date: 8-Dec-2018
https://doi.org/10.1007/978-981-13-3250-0_12
Show More Cited By

Index Terms

Demand-driven structural testing with dynamic instrumentation
1. Software and its engineering
  1. Software creation and management
    1. Software verification and validation
      1. Software defect analysis
        Software testing and debugging
  2. Software notations and tools
    1. Compilers
      1. Runtime environments
    2. General programming languages
      1. Language features

Recommendations

Jazz2: a flexible and extensible framework for structural testing in a Java VM
PPPJ '11: Proceedings of the 9th International Conference on Principles and Practice of Programming in Java

This paper describes a framework for structural testing in a Java Virtual Machine (JVM), called Jazz2. The framework provides instrumentation, memory management, control flow analysis, and callback handling services for structural tests, such as branch ...
Efficient instrumentation for code coverage testing

Evaluation of Code Coverage is the problem of identifying the parts of a program that did not execute in one or more runs of a program. The traditional approach for code coverage tools is to use static code instrumentation. In this paper we present a ...
Efficient instrumentation for code coverage testing
ISSTA '02: Proceedings of the 2002 ACM SIGSOFT international symposium on Software testing and analysis

Evaluation of Code Coverage is the problem of identifying the parts of a program that did not execute in one or more runs of a program. The traditional approach for code coverage tools is to use static code instrumentation. In this paper we present a ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

ICSE '05: Proceedings of the 27th international conference on Software engineering

May 2005

754 pages

ISBN:1581139632

DOI:10.1145/1062455

General Chair:
Gruia-Catalin Roman
Washington University in St. Louis
,
Program Chairs:
William Griswold
University of California, San Diego
,
Bashar Nuseibeh
The Open University, UK

Copyright © 2005 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 15 May 2005

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

ICSE05

Sponsor:

ICSE05: 27th International Conference on Software Engineering

May 15 - 21, 2005

MO, St. Louis, USA

Acceptance Rates

Overall Acceptance Rate 276 of 1,856 submissions, 15%

Upcoming Conference

ICSE 2025

2025 IEEE/ACM 46th International Conference on Software Engineering

April 26 - May 3, 2025

Ottawa , ON , Canada

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

37
Total Citations
View Citations
807
Total Downloads

Downloads (Last 12 months)4
Downloads (Last 6 weeks)1

Reflects downloads up to 14 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Burchell HLarose OKaleba SMarr SBruno RMoss E(2023)Don’t Trust Your Profiler: An Empirical Study on the Precision and Accuracy of Java ProfilersProceedings of the 20th ACM SIGPLAN International Conference on Managed Programming Languages and Runtimes10.1145/3617651.3622985(100-113)Online publication date: 19-Oct-2023
https://dl.acm.org/doi/10.1145/3617651.3622985
Khorram FBousse EGarmendia AMottu JSunyé GWimmer MFischer BBurgueño LCazzola W(2022)From Coverage Computation to Fault Localization: A Generic Framework for Domain-Specific LanguagesProceedings of the 15th ACM SIGPLAN International Conference on Software Language Engineering10.1145/3567512.3567532(235-248)Online publication date: 29-Nov-2022
https://dl.acm.org/doi/10.1145/3567512.3567532
Chatterjee NChakrabarti ADas P(2018)Software Regression and Migration Assistance Using Dynamic InstrumentationAdvanced Computing and Systems for Security10.1007/978-981-13-3250-0_12(159-170)Online publication date: 8-Dec-2018
https://doi.org/10.1007/978-981-13-3250-0_12
Su TWu KMiao WPu GHe JChen YSu Z(2017)A Survey on Data-Flow TestingACM Computing Surveys10.1145/302026650:1(1-35)Online publication date: 10-Mar-2017
https://dl.acm.org/doi/10.1145/3020266
Ohmann PBrown DNeelakandan NLinderoth JLiblit BLo DApel SKhurshid S(2016)Optimizing customized program coverageProceedings of the 31st IEEE/ACM International Conference on Automated Software Engineering10.1145/2970276.2970351(27-38)Online publication date: 25-Aug-2016
https://dl.acm.org/doi/10.1145/2970276.2970351
Whalen MPerson SRungta NStaats MGrijincu DBertolino ACanfora GElbaum S(2015)A flexible and non-intrusive approach for computing complex structural coverage metricsProceedings of the 37th International Conference on Software Engineering - Volume 110.5555/2818754.2818817(506-516)Online publication date: 16-May-2015
https://dl.acm.org/doi/10.5555/2818754.2818817
Wedyan FGhosh SVijayasarathy L(2015)An approach and tool for measurement of state variable based data-flow test coverage for aspect-oriented programsInformation and Software Technology10.1016/j.infsof.2014.11.00859:C(233-254)Online publication date: 1-Mar-2015
https://dl.acm.org/doi/10.1016/j.infsof.2014.11.008
Wang HChan WTse T(2014)Improving the Effectiveness of Testing Pervasive Software via Context DiversityACM Transactions on Autonomous and Adaptive Systems10.1145/26200009:2(1-28)Online publication date: 17-Jul-2014
https://dl.acm.org/doi/10.1145/2620000
Patil ASidnal N(2014)CodeCoverACM SIGSOFT Software Engineering Notes10.1145/2557833.255785039:1(1-4)Online publication date: 11-Feb-2014
https://dl.acm.org/doi/10.1145/2557833.2557850
Araujo RChaim M(2014)Data-Flow Testing in the LargeProceedings of the 2014 IEEE International Conference on Software Testing, Verification, and Validation10.1109/ICST.2014.19(81-90)Online publication date: 31-Mar-2014
https://dl.acm.org/doi/10.1109/ICST.2014.19
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents