Challenges of Evaluation in Medical Informatics

  • Charles P. Friedman
  • Jeremy C. Wyatt
Part of the Computers and Medicine book series (C+M)


This chapter develops in a general and intuitive way many issues that are explored in more detail in later chapters of this book. It gives a first definition of evaluation, describes why evaluation is needed, and notes some of the problems of evaluation in medical informatics that distinguish it from evaluation in other areas. In addition, it lists some of the many clinical information systems and resources, questions that can be asked about them, and the various perspectives of those concerned.


Information Resource Medical Informatics Clinical Data System Word Processor Holiday Destination 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Wyatt J, Spiegelhalter D: Evaluating medical expert systems: what to test, and how? Med Inf (Lond) 1990;15:205–217.CrossRefGoogle Scholar
  2. 2.
    Heathfield H, Wyatt J: The road to professionalism in medical informatics: a proposal for debate. Methods Inf Med 1995;34:426–433.PubMedGoogle Scholar
  3. 3.
    Brahams D, Wyatt J: Decision-aids and the law. Lancet 1989;2:632–634.PubMedCrossRefGoogle Scholar
  4. 4.
    Donabedian A: Evaluating the quality of medical care. Millbank Mem Q 1966;44:166–206.CrossRefGoogle Scholar
  5. 5.
    Young D: An aid to reducing unnecessary investigations. BMJ 1980;281:1610–1611.PubMedCrossRefGoogle Scholar
  6. 6.
    Brannigan V: Software quality regulation under the Safe Medical Devices Act, 1990: hospitals are now the canaries in the software mine. In: Clayton P (ed) Proceedings of the 15th Symposium on Computer Applications in Medical Care. New York: McGraw-Hill, 1991:238–242.Google Scholar
  7. 7.
    Wyatt J: Use and sources of medical knowledge. Lancet 1991;338:1368–1373.PubMedCrossRefGoogle Scholar
  8. 8.
    Pauker S, Gorry G, Kassirer J, Schwartz W: Towards the simulation of clinical cognition: taking a present illness by computer. Am J Med 1976;60:981–996.PubMedCrossRefGoogle Scholar
  9. 9.
    McDonald CJ, Hui SL, Smith DM, et al: Reminders to physicians from an introspective computer medical record: a two-year randomized trial. Ann Intern Med 1984;100:130–138.PubMedGoogle Scholar
  10. 10.
    Tierney WM, Miller ME, Overhage JM, McDonald CJ: Physician order writing on microcomputer workstations. JAMA 1993;269:379–383.PubMedCrossRefGoogle Scholar
  11. 11.
    Wyatt J: Lessons learned from the field trial of ACORN, an expert system to advise on chest pain. In: Barber B, Cao D, Qin D (eds) Proceedings of the Sixth World Conference on Medical Informatics, Singapore. Amsterdam: North Holland, 1989:111–115.Google Scholar
  12. 12.
    Heathfield HA, Wyatt J: Philosophies for the design and development of clinical decision-support systems. Methods Inf Med 1993;32:1–8.PubMedGoogle Scholar
  13. 13.
    Elstein A, Shulman L, Sprafka S: Medical Problem Solving: An Analysis of Clinical Reasoning. Cambridge, MA: Harvard University Press, 1978.Google Scholar
  14. 14.
    Evans D, Patel V (eds): Cognitive Science in Medicine. London: MIT Press, 1989.Google Scholar
  15. 15.
    Van der Lei J, Musen M, van der Does E, in’t Veld A, van Bemmel J: Comparison of computer-aided and human review of general practitioners’ management of hypertension. Lancet 1991;338:1504–1508.PubMedCrossRefGoogle Scholar
  16. 16.
    Musen M: The strained quality of medical data. Methods Inf Med 1989;28:123–125.PubMedGoogle Scholar
  17. 17.
    Wyatt JC: Clinical data systems. Part I. Data and medical records. Lancet 1994;344:1543–47.PubMedCrossRefGoogle Scholar
  18. 18.
    Leitch D: Who should have their cholesterol measured? What experts in the UK suggest. BMJ 1989;298:1615–1616.PubMedCrossRefGoogle Scholar
  19. 19.
    Gaschnig J, Klahr P, Pople H, Shortliffe E, Terry A: Evaluation of expert systems: issues and case studies. In: Hayes-Roth F, Waterman DA, Lenat D (eds) Building Expert Systems, Reading, MA: Addison-Wesley, 1983.Google Scholar
  20. 20.
    Wyatt J, Spiegelhalter D: Field trials of medical decision-aids: potential problems and solutions. In: Clayton P (ed) Proceedings of the 15th Symposium on Computer Applications in Medical Care, Washington. New York: McGraw-Hill, 1991:3–7.Google Scholar
  21. 21.
    Roethligsburger F, Dickson W: Management and the Worker. Cambridge, MA: Harvard University Press, 1939.Google Scholar
  22. 22.
    Stead W, Haynes RB, Fuller S, et al: Designing medical informatics research and library projects to increase what is learned. J Am Med Inf Assoc 1994;1:28–34.CrossRefGoogle Scholar
  23. 23.
    Johnston ME, Langton KB, Haynes RB, Matthieu D: A critical appraisal of research on the effects of computer-based decision support systems on clinician performance and patient outcomes. Ann Intern Med 1994;120:135–142.PubMedGoogle Scholar
  24. 24.
    Greenes RA, Shortliffe EH: Medical informatics: an emerging academic discipline and institutional priority. JAMA 1990;263:1114–1120.PubMedCrossRefGoogle Scholar
  25. 25.
    Friedman CP: Where’s the science in medical informatics? J Am Med Inf Assoc 1995;2:65–67.CrossRefGoogle Scholar
  26. 26.
    Clayton P: Assessing our accomplishments. Symp Comput Applications Med Care 1991;15:viii–x.Google Scholar
  27. 27.
    Anderson JG, Aydin CE, Jay SE (eds): Evaluating Health Care Information Systems. Thousand Oaks, CA: Sage, 1994.Google Scholar
  28. 28.
    Cohen P: Empirical Methods for Artificial Intelligence. Cambridge, MA: MIT Press, 1995.Google Scholar
  29. 29.
    Jain R: The Art of Computer Systems Performance Analysis. New York: Wiley, 1991.Google Scholar
  30. 30.
    Wyatt JC: Medical informatics: artifacts or science? Meth Inf Med 1996;35:3.Google Scholar

Copyright information

© Springer Science+Business Media New York 1997

Authors and Affiliations

  • Charles P. Friedman
    • 1
    • 2
  • Jeremy C. Wyatt
    • 3
  1. 1.University of North CarolinaPittsburghUSA
  2. 2.Center for Biomedical InformaticsUniversity of PittsburghPittsburghUSA
  3. 3.Imperial Cancer Research FundLondonUK

Personalised recommendations