Abstract
Verification and validation (V&V) techniques have always been an essential part of the knowledge engineering process, because they offer the only way to judge the success (or otherwise) of a knowledge base development project. This remains true in the context of knowledge management: V&V techniques provide ways to measure the quality of knowledge in a knowledge base, and to indicate where work needs to be done to rectify anomalous knowledge. This paper provides a critical assessment of the state of the practice in knowledge base V&V, including a survey of available evidence as to the effectiveness of various V&V techniques in real-world knowledge base development projects. For the knowledge management practitioner, this paper offers guidance and recommendations for the use of V&V techniques; for researchers in knowledge management, the paper offers pointers to areas where further work needs to be done on developing more effective V&V techniques.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Liebowitz, J., and Wilcox, L., 1997, Knowledge Management and Its Integrative Elements. CRC Press, New York.
Rushby, J., 1990, Validation and testing of Knowledge-Based Systems: how bad can it get? In (Gupta, 1990).
Batarekh, A., Preece, A., Bennett. A., and Grogono, P., 1996, Specifying an expert system. Expert Systems with Applications, Vol. 2(4).
Boehm, B., 1984, Verifying and validating software requirements and design specifications. IEEE Software, Vol. 1(1).
Meseguer, P. and Preece, A., 1995, Verification and Validation of Knowledge-Based Systems with Formal Specifications. Knowledge Engineering Review, Vol. 10(4).
Preece, A., 1990, Towards a methodology for evaluating expert systems. Expert Systems, Vol. 7(4).
Preece, A., 1995, Towards a Quality Assessment Framework for Knowledge-Based Systems. Journal of Systems and Software, Vol. 29(3).
Wielinga, B.J., Schreiber, A.Th., and Breuker, J.A., 1992, KADS: a modelling approach to knowledge engineering. Knowledge Acquisition, Vol. 4(1).
Brazier, F., Keplics, B., Jennings, N. and Treur, J., 1997, DESIRE: Modelling multi-agent systems in a compositional formal framework. International Journal of Cooperative Information Systems, Vol. 6(1).
Kingston, J., and Macintosh, A., 1999, Knowledge Management through Multi-Perspective Modelling, In Research and Development in Intelligent Systems XVI, pages 221–239, Springer-Verlag, Berlin.
Gupta, U.G., 1990, Validating and Verifying Knowledge-based Systems. IEEE Press, Los Alamitos, CA.
Ayel, M. and Laurent, J-P., Eds., 1991, Verification, Validation and Test of Knowledge-based Systems. John Wiley & Sons, New York.
O’Leary, D.E., 1991, Design, development and validation of expert systems: a survey of developers. In (Ayel and Laurent, 1991), pages 3–20.
Preece, A., Shinghal, R. and Batarekh, A., 1992, Principles and practice in verifying rule-based systems. Knowledge Engineering Review, Vol. 7(2).
O’Leary, D.E., 1998, On a common language for a “best practices” knowledge base. In Using AI for Knowledge Management and Business Process Reengineering: Papers from the AAAI-98 Workshop, AAAI Press, 1998..
Kirani, S., Zualkernan, I.A., and Tsai, W.T., 1992, Comparative Evaluation of Expert System Testing Methods. Technical report TR 92–30, Computer Science Department, University of Minnesota, Minneapolis.
Rushby, J. and J Crow, J., 1990, Evaluation of an Expert System for Fault Detection, Isolation, and Recovery in the Manned Maneuvering Unit. NASA Contractor Report CR-187466, SRI International, Menlo Park CA.
Preece, A. and Shinghal, R., 1994, Foundation and application of knowledge base verification. International Journal of Intelligent Systems, Vol. 9(8).
Miller, L., Hayes, J., and Mirsky, S., 1993, Evaluation of Knowledge Base Certification Methods. SAIC Report for U.S. Nuclear Regulatory Commission and Electrical Power Research Institute NUREG/CR-6316 SAIC-95/1028 Vol. 4.
Preece, A., Talbot, S. and Vignollet, L., 1997, Evaluation of Verification Tools for Knowledge-Based Systems. International Journal of Human-Computer Studies, Vol. 47.
Ayel, M. and Vignollet, L., 1993, SYCOJET and SACCO, two tools for verifying expert systems. International Journal of Expert Systems: Research and Applications, Vol. 6(2).
Fisher, M. and Wooldridge, M., 1993, Specifying and verifying distributed artificial intelligent systems. In Progress in Artificial Intelligence-Sixth Portugese Conference on Artificial Intelligence (LNAI Volume 727), pages 13–28, Springer-Verlag, Berlin.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2001 Springer-Verlag London
About this chapter
Cite this chapter
Preece, A. (2001). Evaluating Verification and Validation Methods in Knowledge Engineering. In: Roy, R. (eds) Industrial Knowledge Management. Springer, London. https://doi.org/10.1007/978-1-4471-0351-6_6
Download citation
DOI: https://doi.org/10.1007/978-1-4471-0351-6_6
Publisher Name: Springer, London
Print ISBN: 978-1-4471-1075-0
Online ISBN: 978-1-4471-0351-6
eBook Packages: Springer Book Archive