Abstract
To achieve software quality, testing is an essential component in all software development. It involves the execution of a deterministic software system with test data and a comparison of the results with the expected output, which must satisfy the users' requirements. This accounts for over 25% of the cost of a software development. Therefore, automation has considerable potential. The quality programming introduced by Cho can automatically generate data for testing, based on a so-called ‘SIAD tree’ which is used to represent the hierarchical and ‘network’ relation between input elements and also incorporates rules into the tree for using the inputs. However, it lacks a clear framework which would show how automated testing can be achieved. To address this problem, we present a Framework for Automating Statistics-based Testing (FAST), which is an extension of the testing concept in quality programming to achieve automated testing. In FAST, we propose a SOAD tree, which is similar to the structure of the SIAD tree, to describe the syntactic structure of the product unit and its defectiveness. Based on this tool, the inspection of test results can be automatically achieved by lexical and syntax analysis. The implementation of automated software testing or Command File Interpreter (CFI) software which incorporates the framework is also described.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
J.D. Musa, and W.W. Everett. Software-reliability engineering: technology for the 1990s, 1EEE Software, 7(6) (1990) 36-43.
M.S. Deutsch and R.R. Willis. Software Quality Engineering - A Total Technical and Management Approach (Prentice Hall International, London, 1988).
D. Ince. The automatic generation of test data, Computer Journal, 30(1987) 63-69.
G.J. Myers. The Art of Software Testing (John Wiley & Sons, New York, 1978).
S. Norman. Software Testing Tools (Ovum Ltd, London, 1993).
B. Marre, P. Thévenod-Fosse, H. Waeselynck, P.L. Gall, and Y. Grouzet. An experimental evaluation of formal testing and statistical testing, in Predictably Dependable Computing Systems (Springer, London, 1995), pp. 273-281.
P.A. Curritt, M Dyer and H.D. Mills. Certifying the reliability of software, IEEE Transactions on Software Engineering, SE-12(1986) 3-11.
J.A. Whittaker and M.G. Thomason. A Markov chain model for statistical software testing, IEEE Transactions on Software Engineering, SE-20(1994) 812-824.
M. Dyer. The Cleanroom Approach to Quality Software Development (John Wiley & Sons, New York, 1992).
P. Thévenod-Fosse, P. Waeselynck, H. and Y. Crouzet. Software statistical testing. In Predictably Dependable Computing Systems (Springer, London, 1995), pp. 253-272.
C.K. Cho. Quality Programming - Development and Testing Software with Statistical Quality Control (John Wiley & Sons, New York, 1988).
P. Thévenod-Fosse and H. Waeselynck. An investigation of statistical software testing, Journal of Software Testing, Verification and Reliability, 1(2) (1991), 5-25.
M.A. Ould and C. Unwin. Testing in Software Development (Cambridge University Press, Cambridge, 1986).
M. Roper. Software Testing (McGraw-Hill, London, 1994)
I. Sommerville. Software Engineering, 5th edn (Addison-Wesley, Wokingham, 1996).
H.V. Vliet. Software Engineering: Principles and Practice (John Wiley & Sons, New York, 1994.
W.S. Humphrey. Characterizing the software process: a maturity framework, IEEE Software, 5(2) (1988) 73-79.
M.E. Fagan. Design and code inspections to reduce errors in program development, IBM Systems Journal, 15(1976) 182-211.
R.A. DeMillo, R.J. Lipton and A.J. Perlis. Social processes and the proofs of theorems and programs, Communications of the ACM, 22(1979) 271-280.
M.E. Fagan. Advances in inspections, IEEE Transactions on Software Engineering, SE-12(1986) 744-751.
H.D. Mills, M. Dyer and R. Linger. Cleanroom software engineering, IEEE Software 4(1987) 19-25.
B. Beizer. Black-Box Testing (John Wiley & Sons, New York, 1995).
V.R. Basili and R.W. Selby. Comparing the effectiveness of software testing strategies, IEEE Transactions on Software Engineering, SE-13(1987) 1278-1296.
R.M. Poston. Automating Specification-Based Software Testing (IEEE Computer Society Press, Los Alamitos, CA, 1996).
E. Kit. Software Testing in the Real World: Improving the Process (Addison-Wesley, Wokingham, 1995).
D. Hamlet. Special section on software testing. Communication of the ACM, 31(6) (1988) 662-667.
J.D. Musa and A.F. Ackerman. Quantifying software validation: when to stop testing? IEEE Software, 6(3) (1989) 19-27.
D. Parnas, J.V. Schouwen and S.P. Kwan. Evaluation of safety-critical software. Communication of the CAM, 33(6) (1990) 636-648.
H.M. Hörcher, and J. Peleska. Using formal specifications to support software testing, Software Quality Journal, 4(1995) 309-327.
H. Zhu. A formal analysis of the subsume relation between software testing adequacy criteria, IEEE Transactions on Software Engineering, SE-22(1996) 248-255.
M. Deck. Cleanroom practice: a theme and variations, in Proceedings of the 9th International Software Quality Week (Software Research Institute, San Francisco, 1996).
I.C. Liu, R.D. Yang, H.D. Chu, F.H Liu, and C.S. Chang. Software statistical quality assurance, Technical Report, Institute for Information and Industry, Taiwan, 1992.
R.V. Megen, and D.B. Meyerhoff. Costs and benefits of early defect detection: experiences from developing client serve and host applications, Software Quality Journal, 4(1995) 247-256.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Chu, HD., Dobson, J.E. & Liu, IC. FAST: a framework for automating statistics-based testing. Software Quality Journal 6, 13–36 (1997). https://doi.org/10.1023/A:1018535229843
Issue Date:
DOI: https://doi.org/10.1023/A:1018535229843