Advertisement

On Functional Statistical Testing Designed from Software Behavior Models

  • Pascale Thévenod-Fosse
  • Hélène Waeselynck
Part of the Dependable Computing and Fault-Tolerant Systems book series (DEPENDABLECOMP, volume 8)

Abstract

Statistical testing involves exercising a piece of software by supplying it with input values that are randomly selected according to a defined probability distribution over its input domain. This paper focuses on functional statistical testing, that is, when an input distribution and a number of random inputs are determined according to criteria relating to software functionality. The criteria based on models of behavior deduced from specification, i.e., finite-state machines and decision tables, are defined. The modeling approach involves a hierarchical decomposition of software functionality. It is applied to a module from the nuclear field. Functional statistical test sets are designed and applied to two versions of the module: the real version, and that developed by a student. Twelve residual faults are revealed, eleven of which affect the student’s version. The other fault is quite subtle, since it resides in the driver that we have developed for the real version in our experimental test harness. Two other input distributions are experimented with: the uniform distribution over the input domain and a structural distribution determined so as to rapidly exercise all the instructions of the student’s version. The results show that the functional statistical test sets have the highest fault revealing power and are the most cost-effective.

Keywords

Test Size Random Input Input Distribution Hierarchical Decomposition Input Domain 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. [1]
    B. Beizer. Software testing techniques. Van Nostrand Reinhold, New York, 1983. Second Edition, 1990.Google Scholar
  2. [2]
    D. B. Brown et al. An automated oracle for software testing. IEEE Transactions on Reliability, Vol. 41, No. 2, June 1992, pp. 272–280.CrossRefGoogle Scholar
  3. [3]
    A. M. Davis. A comparison of techniques for the specification of external system behavior. Communications of the ACM, Vol. 31, No. 9, September 1988, pp. 1098–1115.CrossRefGoogle Scholar
  4. [4]
    R. A. DeMillo, R. J. Lipton, F. G. Sayward. Hints on test data selection: help for the practicing programmer. IEEE Computer Magazine, Vol. 11, No. 4, April 1978, pp. 34–41.CrossRefGoogle Scholar
  5. [5]
    J. W. Duran, S. C. Ntafos. An evaluation of random testing. IEEE Transactions on Software Engineering, Vol. SE-10, No. 4, July 1984, pp. 438–444.CrossRefGoogle Scholar
  6. [6]
    B. Goodenough, S. L. Gerhart. Toward a theory of test data selection. IEEE Transactions on Software Engineering, Vol. SE-1, No. 2, June 1975, pp. 156–173.MathSciNetCrossRefGoogle Scholar
  7. [7]
    R. Hamlet. Theoretical comparison of testing methods. Proc. 3rd IEEE Symposium on Software Testing, Analysis and Verification, Key West, USA, December 1989, pp. 28-37.Google Scholar
  8. [8]
    D. Harel et al. STATEMATE: a working environment for the development of complex reactive systems. IEEE Transactions on Software Engineering, Vol. SE-16, No. 4, April 1990, pp. 403–414.CrossRefGoogle Scholar
  9. [9]
    W. E. Howden. A functional approach to program testing and analysis. IEEE Transactions on Software Engineering. Vol. SE-12, No. 10, October 1986, pp. 997–1005.CrossRefGoogle Scholar
  10. [10]
    W. E. Howden. Functional program testing and analysis. Computer Science Series, McGraw-Hill Book Company, 1987.Google Scholar
  11. [11]
    J-C. Laprie (Ed.). Dependability: Basic Concepts and Terminology. Vol. 5 in the Series on Dependable Computing and Fault-Tolerant Systems, Springer-Verlag, Austria, 1992.Google Scholar
  12. [12]
    G. J. Myers. The art of software testing. Wiley, New York, 1979.Google Scholar
  13. [13]
    S. C. Ntafos. A comparison of some structural testing strategies. IEEE Transactions on Software Engineering, Vol. SE-14, No. 6, June 1988, pp. 868–874.CrossRefGoogle Scholar
  14. [14]
    T. J. Ostrand, M. J. Balcer. The category-partition method for specifying and generating functional tests. Communications of the ACM, Vol. 31, No. 6, June 1988, pp. 676–686.CrossRefGoogle Scholar
  15. [15]
    S. Rapps, E. J. Weyuker. Selecting software test data using data flow information. IEEE Transactions on Software Engineering, Vol. SE-11, No. 4, April 1985, pp. 367–375.CrossRefGoogle Scholar
  16. [16]
    P. Thévenod-Fosse. Software validation by means of statistical testing: retrospect and future direction. Proc. 1st IEEE Working Conference on Dependable Computing for Critical Applications, Santa Barbara, USA, August 1989, pp. 15-22. Published in Dependable Computing for Critical Applications, Vol. 4 in the Series on Dependable Computing and Fault-Tolerant Systems, Springer-Verlag, 1991, pp. 23-50.Google Scholar
  17. [17]
    P. Thévenod-Fosse. On the efficiency of statistical testing with respect to software structural test criteria. Proc. IFIP Working Conference on Approving Software Products, Garmisch, Germany, Elsevier Science Publishers B.V., North-Holland, 1990, pp. 29–42.Google Scholar
  18. [18]
    P. Thévenod-Fosse, H. Waeselynck, Y. Crouzet. An experimental study on software structural testing: deterministic versus random input generation. Proc. 21st IEEE Int. Symposium on Fault-Tolerant Computing (FTCS-21), Montréal, Canada, June 1991, pp. 410-417.Google Scholar
  19. [19]
    P. Thévenod-Fosse, H. Waeselynck. An investigation of statistical software testing. Journal of Software Testing, Verification and Reliability, Vol. 1, No. 2, July–September 1991, pp. 5–25.Google Scholar
  20. [20]
    P. Thévenod-Fosse, H. Waeselynck, Y. Crouzet. Software structural testing: an evaluation of the efficiency of deterministic and random test data. LAAS Report 91.389, December 1991.Google Scholar
  21. [21]
    H. Ural. Formal methods for test sequence generation. Computer Communications, Vol. 15, No. 5, June 1992, pp. 311–325.CrossRefGoogle Scholar
  22. [22]
    E. J. Weyuker, T. J. Ostrand. Theories of program testing and the application of revealing subdomains. IEEE Transactions on Software Engineering, Vol. SE-6, No. 3, May 1980, pp. 236–246.MathSciNetCrossRefGoogle Scholar
  23. [23]
    E. J. Weyuker. On testing non-testable programs. The Computer Journal, Vol. 25, No. 4, 1982, pp. 465–470.CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Wien 1993

Authors and Affiliations

  • Pascale Thévenod-Fosse
    • 1
  • Hélène Waeselynck
    • 1
  1. 1.Laboratoire d’Automatique et d’Analyse des Systèmes du C.N.R.S.Toulouse CedexFrance

Personalised recommendations