Testing System Accuracy
Evaluation is a crucial component in the development of any clinical diagnostic decision support system (CDDSS). Much of it takes place informally as part of the development process and is used by the CDDSS developers for system improvement. Once a system is sufficiently mature, more formal evaluation studies should be done, initially of system accuracy and later, of system impact. A wide range of study design choices can be appropriate for assessing accuracy, but once the CDDSS appears to be ready for use in practice, there is a need for more rigorous evaluation. Most published evaluation studies have focused on the issue of system accuracy, with few studies evaluating the impact of using a CDDSS on clinical care. This chapter will address issues involved in assessing the accuracy of CDDSS. Key results from research or evaluation studies of system accuracy will be summarized and discussed. The reader who is interested in the details of individual studies should read the references at the end of this chapter.
KeywordsDecision Support System Primary User Clinical Decision Support System System Accuracy Proc Amia
Unable to display preview. Download preview PDF.
- 13.Georgakis DC, Trace DA, Naeymi-Rad F et al. A statistical evaluation of the diagnostic performance of MEDAS—the medical emergency decision assistance system. Proc Annu Symp Comput Appl Med Care 1990:815–819.Google Scholar
- 15.Hammersley JR, Cooney K. Evaluating the utility of available differential diagnosis systems. Proc Annu Symp Comput Appl Med Care 1988:229–231.Google Scholar
- 29.Miller RA, Masarie FE Jr. The quick medical reference (QMR) relationships function: description and evaluation of a simple, efficient “multiple diagnoses” algorithm. Medinfo 1992:512–518.Google Scholar
- 33.Bankowitz, RA. The effectiveness of QMR in medical decision support. Executive summary and final report. Springfield, VA: U.S. Department of Commerce, National Technical Information Service, 1994.Google Scholar
- 34.Murphy GC, Friedman CP, Elstein AS. The influence of a decision support system on the differential diagnosis of medical practitioners at three levels of training. Proc AMIA Fall Symp Comput 1996: 219–223.Google Scholar
- 36.Innis MD. Medisets. Computer-assisted diagnosis using a modified set theory. Proc Second Natl Health Conf, Health Informatics Conf, ’94, Gold Coast Australia, 1994, 286–291.Google Scholar
- 51.Weed LL. Problem-knowledge couplers: Philosophy, use and interpretation. PKC Corporation 1982; pgs. 2–22.Google Scholar
- 52.Weed LL. Reengineering medicine: questions and answers. Federation Bull 1995; 82:24–36.Google Scholar
- 55.Wyatt J, Spiegelhalter D. Field trials of medical decision-aids: potential problems and solutions. Proc Annu Symp Comput Appl Med Care 1991:3–7.Google Scholar
- 59.Berner ES, Maisiak RS. Physician use of interactive functions in diagnostic decision support systems. Proc AMIA Fall Symp 1997; 842.Google Scholar
- 61.Welford CR. A comprehensive computerized patient record with automated linkage to QMR. Proc Annu Symp Comput Appl Med Care 1994: 814–818.Google Scholar