On Functional Statistical Testing Designed from Software Behavior Models
Statistical testing involves exercising a piece of software by supplying it with input values that are randomly selected according to a defined probability distribution over its input domain. This paper focuses on functional statistical testing, that is, when an input distribution and a number of random inputs are determined according to criteria relating to software functionality. The criteria based on models of behavior deduced from specification, i.e., finite-state machines and decision tables, are defined. The modeling approach involves a hierarchical decomposition of software functionality. It is applied to a module from the nuclear field. Functional statistical test sets are designed and applied to two versions of the module: the real version, and that developed by a student. Twelve residual faults are revealed, eleven of which affect the student’s version. The other fault is quite subtle, since it resides in the driver that we have developed for the real version in our experimental test harness. Two other input distributions are experimented with: the uniform distribution over the input domain and a structural distribution determined so as to rapidly exercise all the instructions of the student’s version. The results show that the functional statistical test sets have the highest fault revealing power and are the most cost-effective.
KeywordsTest Size Random Input Input Distribution Hierarchical Decomposition Input Domain
Unable to display preview. Download preview PDF.
- B. Beizer. Software testing techniques. Van Nostrand Reinhold, New York, 1983. Second Edition, 1990.Google Scholar
- R. Hamlet. Theoretical comparison of testing methods. Proc. 3rd IEEE Symposium on Software Testing, Analysis and Verification, Key West, USA, December 1989, pp. 28-37.Google Scholar
- W. E. Howden. Functional program testing and analysis. Computer Science Series, McGraw-Hill Book Company, 1987.Google Scholar
- J-C. Laprie (Ed.). Dependability: Basic Concepts and Terminology. Vol. 5 in the Series on Dependable Computing and Fault-Tolerant Systems, Springer-Verlag, Austria, 1992.Google Scholar
- G. J. Myers. The art of software testing. Wiley, New York, 1979.Google Scholar
- P. Thévenod-Fosse. Software validation by means of statistical testing: retrospect and future direction. Proc. 1st IEEE Working Conference on Dependable Computing for Critical Applications, Santa Barbara, USA, August 1989, pp. 15-22. Published in Dependable Computing for Critical Applications, Vol. 4 in the Series on Dependable Computing and Fault-Tolerant Systems, Springer-Verlag, 1991, pp. 23-50.Google Scholar
- P. Thévenod-Fosse. On the efficiency of statistical testing with respect to software structural test criteria. Proc. IFIP Working Conference on Approving Software Products, Garmisch, Germany, Elsevier Science Publishers B.V., North-Holland, 1990, pp. 29–42.Google Scholar
- P. Thévenod-Fosse, H. Waeselynck, Y. Crouzet. An experimental study on software structural testing: deterministic versus random input generation. Proc. 21st IEEE Int. Symposium on Fault-Tolerant Computing (FTCS-21), Montréal, Canada, June 1991, pp. 410-417.Google Scholar
- P. Thévenod-Fosse, H. Waeselynck. An investigation of statistical software testing. Journal of Software Testing, Verification and Reliability, Vol. 1, No. 2, July–September 1991, pp. 5–25.Google Scholar
- P. Thévenod-Fosse, H. Waeselynck, Y. Crouzet. Software structural testing: an evaluation of the efficiency of deterministic and random test data. LAAS Report 91.389, December 1991.Google Scholar