Abstract
Verification, validation and testing techniques developed for use with conventional development practices, are not always applicable when developing knowledge-based software. This paper presents an experimental framework to determine whether a technique is applicable or not, based on concepts from mutation testing. The framework itself comprises of a number of steps guiding the researcher/practitioner in the assessment process. Mutation testing is used to simulate faults in an example programme to determine the technique’s ability to detect them. The framework has been applied to two techniques: control-flow analysis and cause-effect graphing. The conclusion is that the framework gives a good basis for a qualitative assessment of the applicability and efficiency of applying specific traditional VV&T techniques to knowledge-based components.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Budd, T.A., R.A. DeMillo, R.J. Lipton, and F.G. Sayward. 1980. “Theoretical and empirical studies on using program mutation to test the functional correctness of programs” Conference Record of the Seventh ACM Symposium on Principles of Programming Languages (POPL `80). ACM Press, pp. 220–233.
CLIPS. 1997. CLIPS Reference Manual Version 6.05 — Volume I: Basic Programming Guide.
IEEE Standards Board. 1993. “IEEE Standard Classification for Software Anomalies”. In: IEEE Standards Collection — Software Engineering, ed.
IEEE Standards Board. IEEE National Bureau of Standards. 1981. Validation, Verification and Testing of ComputerSoftware,Special Publication 500–75, National Bureau of Standards.
O’Keefe, R.M. and D.E. O’Leary. 1993. “Expert system verification and validation: a survey and tutorial” Artificial Intelligence Review, vol. 7, pp. 3–42.
Preece, A.D., P.G. Chander, C. Grossner, and T. Radhakrishnan. 1993. “Modeling Rule Base Structure for Expert System Quality Assurance” International Joint Conference on Artificial Intelligence Workshop on Validation of Knowledge-Based Systems., pp. 37–49.
Preece, A.D., S. Talbot, and L. Vignollet. 1997. “Evaluation of verification tools for knowledge-based systems” International Journal of Human-Computer Studies, vol. 47, pp. 629–258.
Preece, A.D., C. Grossner, P.G. Chander, and T. Radhakrishnan. 1998. “Structure-based validation of rule-based systems” Data and Knowledge Engineering, vol. 26, pp. 161–189.
Vermesan, A.I. and T. Bench-Capon. 1995. “Techniques for the Verification and Validation of Knowledge-Based Systems: A Survey Based on the Symbol/Knowledge Level Distinction”, Software, Testing, Verification & Reliability, John Wiley & Sons, Ltd, pp. 233–271.
Vermesan, A.I. 1998. “Foundation and Application of Expert System Verification and Validation”. In: The Handbook of Applied Expert Systems, ed. J. Liebowitz. Boca Raton: CRC Press LLC., pp. 5. 1–5. 32.
T. Bench-Capon, D. Castelli, F. Coenen, L. Devendeville-Brisoux, B. Eaglestone, N. Fiddian, A. Gray, A. Ligeza and A. Vermesan (1999). Validation, Verification and Integrity Issues in Expert and Database Systems. Expert Update, Vol 2, No 1, pp 31–35.
F. Bouali, S. Loiseau, M-C. Rousset (1997). Verification and Revision of Rule Bases In Hunt, J. and Miles, R. (Eds.), Research and Development in Expert Systems XIV, proceedings of ES’97, SGES publications, pp253–276.
F. Bouali, S. Loiseau, M-C. Rousset (1997). Revision of Rule Bases. Proceedings EUROVAV’97, Katholieke Universiteit Leuven, Belgium, pp 193–203.
Coenen, F.P. and Bench-Capon, T.J.M. (1993). Maintenance of Knowledge Based Systems: Theory, Tools and Techniques. Academic Press, London.
S.A. Cook (1971) Complexity of Theorem-Proving Procedures In Proc. of the 3rd Annual ACM Symposium on Theory of Computing, pp151–158.
S. Craw (1998). Krust Works: Developing a Generic Refinement Toolkit. Expert Update, Vol 1, No 2, pp 35–47.
D. Fensel (1995). Formal Specification Languages in Knowledge and software Engineering. Knowledge Engineering Review, Vol 9, No4.
M.R. Garey, D.S. Johnson (1979). Computers and Intractability — A Guide to the Theory of NP-Completeness. W.H. Freeman and Co., San Francisco.
N. Guarino and P. Giaretta, Ontologies and knowledge bases: towards a terminological clarification, In: N.J.I. Mars ed., Towards Very Large Knowledge Bases (IOS Press 1995 ).
F. van Harmelen and A. ten Teije (1997). Validation and Verification of Conceptual Models of Diagnosis. Proceedings EUROVAV’97, Katholieke Universiteit Leuven, Belgium.
G. van Heijst, The role of ontologies in knowledge engineering, (Doctoral Thesis, University of Amsterdam, Amsterdam, The Netherlands 1995 ).
Newell, The knowledge level, Artificial Intelligence 18 (1982) 87–127.
J.F. Sowa (1984). Conceptual Structures: Information Processing in Mind and Machine. Addison-Wesley
A.I. Vermesan (1997). Quality Assessment of Knowledge-Based Software: Certification Considerations. In Proceedings of Third IEEE International Software Engineering Standards Symposium (ISESS ’97) Walnut Creek, CA.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 1999 Springer Science+Business Media New York
About this chapter
Cite this chapter
Vermesan, A., Høgberg, F. (1999). Applicability of Conventional Software Verification and Validation to Knowledge-Based Components. In: Vermesan, A., Coenen, F. (eds) Validation and Verification of Knowledge Based Systems. Springer, Boston, MA. https://doi.org/10.1007/978-1-4757-6916-6_23
Download citation
DOI: https://doi.org/10.1007/978-1-4757-6916-6_23
Publisher Name: Springer, Boston, MA
Print ISBN: 978-1-4419-5107-6
Online ISBN: 978-1-4757-6916-6
eBook Packages: Springer Book Archive