Applicability of Conventional Software Verification and Validation to Knowledge-Based Components

A Qualitative Assessment
  • Anca Vermesan
  • Frode Høgberg


Verification, validation and testing techniques developed for use with conventional development practices, are not always applicable when developing knowledge-based software. This paper presents an experimental framework to determine whether a technique is applicable or not, based on concepts from mutation testing. The framework itself comprises of a number of steps guiding the researcher/practitioner in the assessment process. Mutation testing is used to simulate faults in an example programme to determine the technique’s ability to detect them. The framework has been applied to two techniques: control-flow analysis and cause-effect graphing. The conclusion is that the framework gives a good basis for a qualitative assessment of the applicability and efficiency of applying specific traditional VV&T techniques to knowledge-based components.


Expert System Mutation Operator Testing Technique Mutation Testing Control Flow Graph 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Budd, T.A., R.A. DeMillo, R.J. Lipton, and F.G. Sayward. 1980. “Theoretical and empirical studies on using program mutation to test the functional correctness of programs” Conference Record of the Seventh ACM Symposium on Principles of Programming Languages (POPL `80). ACM Press, pp. 220–233.Google Scholar
  2. CLIPS. 1997. CLIPS Reference Manual Version 6.05 — Volume I: Basic Programming Guide. Google Scholar
  3. IEEE Standards Board. 1993. “IEEE Standard Classification for Software Anomalies”. In: IEEE Standards Collection — Software Engineering, ed.Google Scholar
  4. IEEE Standards Board. IEEE National Bureau of Standards. 1981. Validation, Verification and Testing of ComputerSoftware,Special Publication 500–75, National Bureau of Standards.Google Scholar
  5. O’Keefe, R.M. and D.E. O’Leary. 1993. “Expert system verification and validation: a survey and tutorial” Artificial Intelligence Review, vol. 7, pp. 3–42.CrossRefGoogle Scholar
  6. Preece, A.D., P.G. Chander, C. Grossner, and T. Radhakrishnan. 1993. “Modeling Rule Base Structure for Expert System Quality Assurance” International Joint Conference on Artificial Intelligence Workshop on Validation of Knowledge-Based Systems., pp. 37–49.Google Scholar
  7. Preece, A.D., S. Talbot, and L. Vignollet. 1997. “Evaluation of verification tools for knowledge-based systems” International Journal of Human-Computer Studies, vol. 47, pp. 629–258.CrossRefGoogle Scholar
  8. Preece, A.D., C. Grossner, P.G. Chander, and T. Radhakrishnan. 1998. “Structure-based validation of rule-based systems” Data and Knowledge Engineering, vol. 26, pp. 161–189.zbMATHCrossRefGoogle Scholar
  9. Vermesan, A.I. and T. Bench-Capon. 1995. “Techniques for the Verification and Validation of Knowledge-Based Systems: A Survey Based on the Symbol/Knowledge Level Distinction”, Software, Testing, Verification & Reliability, John Wiley & Sons, Ltd, pp. 233–271.Google Scholar
  10. Vermesan, A.I. 1998. “Foundation and Application of Expert System Verification and Validation”. In: The Handbook of Applied Expert Systems, ed. J. Liebowitz. Boca Raton: CRC Press LLC., pp. 5. 1–5. 32.Google Scholar
  11. [1]
    T. Bench-Capon, D. Castelli, F. Coenen, L. Devendeville-Brisoux, B. Eaglestone, N. Fiddian, A. Gray, A. Ligeza and A. Vermesan (1999). Validation, Verification and Integrity Issues in Expert and Database Systems. Expert Update, Vol 2, No 1, pp 31–35.Google Scholar
  12. [2]
    F. Bouali, S. Loiseau, M-C. Rousset (1997). Verification and Revision of Rule Bases In Hunt, J. and Miles, R. (Eds.), Research and Development in Expert Systems XIV, proceedings of ES’97, SGES publications, pp253–276.Google Scholar
  13. [3]
    F. Bouali, S. Loiseau, M-C. Rousset (1997). Revision of Rule Bases. Proceedings EUROVAV’97, Katholieke Universiteit Leuven, Belgium, pp 193–203.Google Scholar
  14. [4]
    Coenen, F.P. and Bench-Capon, T.J.M. (1993). Maintenance of Knowledge Based Systems: Theory, Tools and Techniques. Academic Press, London.Google Scholar
  15. [5]
    S.A. Cook (1971) Complexity of Theorem-Proving Procedures In Proc. of the 3rd Annual ACM Symposium on Theory of Computing, pp151–158.Google Scholar
  16. [6]
    S. Craw (1998). Krust Works: Developing a Generic Refinement Toolkit. Expert Update, Vol 1, No 2, pp 35–47.Google Scholar
  17. [7]
    D. Fensel (1995). Formal Specification Languages in Knowledge and software Engineering. Knowledge Engineering Review, Vol 9, No4.Google Scholar
  18. [8]
    M.R. Garey, D.S. Johnson (1979). Computers and Intractability — A Guide to the Theory of NP-Completeness. W.H. Freeman and Co., San Francisco.Google Scholar
  19. [9]
    N. Guarino and P. Giaretta, Ontologies and knowledge bases: towards a terminological clarification, In: N.J.I. Mars ed., Towards Very Large Knowledge Bases (IOS Press 1995 ).Google Scholar
  20. [10]
    F. van Harmelen and A. ten Teije (1997). Validation and Verification of Conceptual Models of Diagnosis. Proceedings EUROVAV’97, Katholieke Universiteit Leuven, Belgium.Google Scholar
  21. [11]
    G. van Heijst, The role of ontologies in knowledge engineering, (Doctoral Thesis, University of Amsterdam, Amsterdam, The Netherlands 1995 ).Google Scholar
  22. [12]
    Newell, The knowledge level, Artificial Intelligence 18 (1982) 87–127.CrossRefGoogle Scholar
  23. [13]
    J.F. Sowa (1984). Conceptual Structures: Information Processing in Mind and Machine. Addison-WesleyGoogle Scholar
  24. [14]
    A.I. Vermesan (1997). Quality Assessment of Knowledge-Based Software: Certification Considerations. In Proceedings of Third IEEE International Software Engineering Standards Symposium (ISESS ’97) Walnut Creek, CA.Google Scholar

Copyright information

© Springer Science+Business Media New York 1999

Authors and Affiliations

  • Anca Vermesan
    • 1
  • Frode Høgberg
    • 1
  1. 1.Det Norske Veritas ASNorway

Personalised recommendations