Clinical Trials of Information Interventions

  • E. Andrew Balas
  • Suzanne Austin Boren
Part of the Health Informatics book series (HI)


Chapter 3 discussed the results of studies that have been conducted to determine the accuracy of clinical diagnostic decision support systems (CDDSS). When a CDDSS passes the test of accuracy and is ready for clinical implementation, the need for replicable and gener-alizable measurement of practical impact emerges. It is increasingly acknowledged that measurement of system performance and impact represents the research component of informatics projects and such evaluations should guide the development of decision support technologies.1 This chapter discusses the methodology for systematic evaluation of information interventions. It provides a framework for designing appropriate tests of the clinical impact of CDDSS.


Decision Support System Health Service Research Health Care Organization Clinical Decision Support System Information Intervention 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Friedman CP. Where’s the science in medical informatics? JAMIA 1995; 2:65–67.PubMedCrossRefGoogle Scholar
  2. 2.
    Peck CC, Sheiner LB, Martin MM et al. Computer-assisted digoxin therapy. N Engl J Med 1973; 289:441–446.PubMedCrossRefGoogle Scholar
  3. 3.
    McAlister NH, Cowey HD, Tong C et al. Randomized controlled trial of computer-assisted management of hypertension in primary care. Br Med J 1986; 293:670–674.CrossRefGoogle Scholar
  4. 4.
    Tierney WM, Miller ME, McDonald CJ. The effect on test ordering of informing physicians of the charges for outpatient diagnostic tests. N Engl J Med 1990; 322:1499–1504.PubMedCrossRefGoogle Scholar
  5. 5.
    Wones RG. Failure of low-cost audits with feedback to reduce laboratory test utilization. Med Care 1987; 25:78–82.PubMedCrossRefGoogle Scholar
  6. 6.
    Wyatt J, Spiegelhalter D. Field trials of medical decision-aids: potential problems and solutions. Proc Annu Symp Comput Appl Med Care 1991:3–7.Google Scholar
  7. 7.
    Haynes RB, Walker CJ. Computer-aided quality assurance: a critical appraisal. Arch Intern Med 1987; 147:1297–1301.PubMedCrossRefGoogle Scholar
  8. 8.
    Piantadosi S, Byar DP. A proposal for registering clinical trials. Control Clin Trials 1988; 9:82–84.PubMedCrossRefGoogle Scholar
  9. 9.
    Gorry GA. Computer-assisted clinical decision making. Methods Inf Med 1973; 12:45.PubMedGoogle Scholar
  10. 10.
    de Dombal FT, Leaper DJ, Staniland JR et al. Computer aided diagnosis of acute abdominal pain. Br Med J 1972; 2:9–13.PubMedCrossRefGoogle Scholar
  11. 11.
    McAdam WA, Brock BM, Armitage T et al. Twelve years experience of computer-aided diagnosis in a district general hospital. Ann R Coll Surg Engl 1990; 72:140–146.PubMedGoogle Scholar
  12. 12.
    Sutton GC. How accurate is computer-aided diagnosis? Lancet 1989; 2:905–908.PubMedCrossRefGoogle Scholar
  13. 13.
    Paterson-Brown S, Vipond MN, Simms K et al. Clinical decisionmaking and laparoscopy versus computer prediction in the management of the acute abdomen. Br J Surg 1989; 76:1011–1013.PubMedCrossRefGoogle Scholar
  14. 14.
    Gabert HA, Stenchever MA. Continuous electronic monitoring of fetal heart rate during labor. Am J Obstet Gynecol 1973; 115:919–923.PubMedGoogle Scholar
  15. 15.
    Paul RH, Hon EH. Clinical fetal monitoring. V. Effect on perinatal outcome. Am J Obstet Gynecol 1974; 118:529–533.PubMedGoogle Scholar
  16. 16.
    McDonald D, Grant A, Sheridan-Pereira M et al. The Dublin randomized controlled trial of intrapartum fetal heart rate monitoring. Am J Obstet Gynecol 1985; 152:524–539.Google Scholar
  17. 17.
    Shy KK, Luthy DA, Bennett FC et al. Effects of electronic fetal heart-rate monitoring, as compared with periodic auscultation, on the neurologic development of premature infants. N Engl J Med 1990; 322:588–593.PubMedCrossRefGoogle Scholar
  18. 18.
    Freeman R. Intrapartum fetal monitoring—a disappointing story. N Engl J Med 1990; 322:624–626.PubMedCrossRefGoogle Scholar
  19. 19.
    Tyson JE, Furzan JA, Resich JS et al. An evaluation of the quality of therapeutic studies in perinatal medicine. J Pediatr 1983; 102:10–13.PubMedCrossRefGoogle Scholar
  20. 20.
    Simborg DW, Whiting-O’Keefe QE. Evaluation methodology for ambulatory care information systems. Med Care 1982; 20:255–265.PubMedCrossRefGoogle Scholar
  21. 21.
    Smith L. The coming health care shakeout. Fortune 1993; May 17:70–75.Google Scholar
  22. 22.
    Cochrane AL. Effectiveness and Efficiency: Random Reflections on Health Services. London, England: Nuffield Provincial Hospitals Trust, 1972.Google Scholar
  23. 23.
    Banta HD, Thacker SB. The case for reassessment of health care technology: once is not enough. JAMA 1990; 264:235–240.PubMedCrossRefGoogle Scholar
  24. 24.
    Berwick DM. Continuous improvement as an ideal in health care. N Engl J Med 1989; 320:53–56.PubMedCrossRefGoogle Scholar
  25. 25.
    Vinicor F, Cohen SJ, Mazzuca SA et al. DIABEDS: a randomized trial of the effects of physician and/or patient education on diabetes patient outcomes. J Chronic Dis 1987; 40:345–356.PubMedCrossRefGoogle Scholar
  26. 26.
    McPhee SJ, Bird JA, Jenkins CN et al. Promoting cancer screening: a randomized, controlled trial of three interventions. Arch Intern Med 1989; 149:1866–1872.PubMedCrossRefGoogle Scholar
  27. 27.
    Tape TG, Campbell JR. Computerized medical records and preventive health care: success depends on many factors. Am J Med. 1993; 94:619–625.PubMedCrossRefGoogle Scholar
  28. 28.
    Melin AL, Bygren LO. Efficacy of the rehabilitation of elderly primary health care patients after short-stay hospital treatment. Med Care 1992; 30:1004–1015.PubMedCrossRefGoogle Scholar
  29. 29.
    Starmer CF, Lee KL, Harell FE et al. On the complexity of investigating chronic illness. Biometrics 1980; 36:333–335.PubMedCrossRefGoogle Scholar
  30. 30.
    Byar DP. Why data bases should not replace randomized clinical trials. Biometrics 1980; 36:337–342.PubMedCrossRefGoogle Scholar
  31. 31.
    Ingelfinger JA, Mosteller F, Thibodeau LA et al. Biostatistics in Clinical Medicine. New York: Macmillan, 1987.Google Scholar
  32. 32.
    Balas EA, Stockham MG, Mitchell JA et al. The Columbia registry of information and utilization management trials. JAMIA 1995; 2:307–315.PubMedCrossRefGoogle Scholar
  33. 33.
    Johnston ME, Langton KB, Haynes RB et al. Effects of computer-based clinical decision support systems on clinician performance and patient outcomes critical appraisal of research. Ann Intern Med 1994; 120:135–142.PubMedCrossRefGoogle Scholar
  34. 34.
    Williams LS. Microchips versus stethoscopes: Calgary hospital, MDs face off over controversial computer system. Can Med Assoc J 1992; 147:1534–1547.Google Scholar
  35. 35.
    Massaro TA. Introducing physician order entry at a major academic medical center: I. Impact on organizational culture and behavior. Acad Med 1993; 68:20–25.PubMedCrossRefGoogle Scholar
  36. 36.
    Teach RL, Shortliffe EH. An analysis of physician attitudes regarding computer-based clinical consultation systems. Comput Biomed Res 1981; 14:542–558.PubMedCrossRefGoogle Scholar
  37. 37.
    Gardner RM, Lundsgaarde HP. Evaluation of user acceptance of a clinical expert system. JAMIA 1994; 1:428–438.PubMedCrossRefGoogle Scholar
  38. 38.
    Hershey CO, Porter DK, Breslau D et al. Influence of simple computerized feedback on prescription charges in an ambulatory clinic. Med Care 1986; 24:472–481.PubMedCrossRefGoogle Scholar
  39. 39.
    Pozen MW, D’Agostino RB, Selker HP et al. A predictive instrument to improve coronary-care-unit admission practices in acute ischemic heart disease. N Engl J Med 1984; 310:1273–1278.PubMedCrossRefGoogle Scholar
  40. 40.
    GCP: Nordic and European Guidelines. Austin: Pharmaco Dynamics Research, 1990.Google Scholar
  41. 41.
    Spilker B. Guide to Clinical Trials. New York: Raven, 1991.Google Scholar
  42. 42.
    Balas EA, Austin SM, Ewigman BG et al. Methods of randomized controlled clinical trials in health services research. Med Care 1995; 33:687–699.PubMedCrossRefGoogle Scholar
  43. 43.
    Cochrane AL. 1931–1971, a critical review with particular reference to the medical profession. In G Teeling-Smith and N Wells, Medicine for the Year 2000. London: Office of Health Economics, 1979.Google Scholar
  44. 44.
    Chalmers I, Hetherington J, Newdick M et al. The Oxford Database of Perinatal Trials: developing a register of published reports of controlled trials. Control Clin Trials 1986; 7:306–324.PubMedCrossRefGoogle Scholar
  45. 45.
    Dickersin K. Why register clinical trials?—Revisited. Control Clin Trials 1992; 13:170–177.PubMedCrossRefGoogle Scholar
  46. 46.
    Davis DA, Thomson MA, Oxman AD et al. Evidence for the effectiveness of CME: a review of 50 randomized control trials. JAMA 1992; 268:1111–1117.PubMedCrossRefGoogle Scholar
  47. 47.
    Antman EM, Lau J, Kupelnick B et al. A comparison of results of meta-analysis of randomized control trials and recommendations of clinical experts. Treatments for myocardial infarction. JAMA 1992; 268:240–248.PubMedCrossRefGoogle Scholar
  48. 48.
    Light RJ, Smith PV. Accumulating evidence: procedures for resolving contradictions among different research studies. Harvard Educational Review 1971; 41:420–471.Google Scholar
  49. 49.
    Balas EA, Austin SM, Mitchell JA et al. The clinical value of computerized information services: a review of 98 randomized clinical trials. Arch Fam Med 1996; 5:271–278.PubMedCrossRefGoogle Scholar
  50. 50.
    Begg EJ, Atkinson HC, Jeffery GM et al. Individualized aminoglycoside dosage based on pharmacokinetic analysis is superior to dosage based on physician intuition at achieving target plasma drug concentrations. Br J Clin Pharmac 1989; 28:137–141.CrossRefGoogle Scholar
  51. 51.
    Chiarelli F, Tumini S, Morgese G et al. Controlled study in diabetic children comparing insulin-dosage adjustment by manual and computer algorithms. Diab Care 1990; 13:1080–1084.CrossRefGoogle Scholar
  52. 52.
    Privitera MD, Homan RW, Ludden TM et al. Clinical utility of a Bayesian dosing program for phenytoin. Ther Drug Monit 1989; 11:285–294.PubMedCrossRefGoogle Scholar
  53. 53.
    Reid JA, Kenny GNC. Evaluation of closed-loop control of arterial pressure after cardiopulmonary bypass. Br J Anaesth 1987; 59:247–255.PubMedCrossRefGoogle Scholar
  54. 54.
    Rodman JH, Jelliffe RW, Kolb E et al. Clinical studies with computer-assisted initial lidocaine therapy. Arch Intern Med 1984; 144:703–709.PubMedCrossRefGoogle Scholar
  55. 55.
    Ziegler DK, Hurwitz A, Hassanein RS et al. Migraine prophylaxis: a comparison of Propranolol and Amitriptyline. Arch Neurol 1987; 44:486–489.PubMedCrossRefGoogle Scholar
  56. 56.
    Chambers CV, Balaban DJ, Carlson BL et al. Microcomputer-generated reminders: improving the compliance of primary care physicians with mammography screening guidelines. J Fam Pract 1989; 3:273–280.Google Scholar
  57. 57.
    Fordham D, McPhee SJ, Bird JA et al. The cancer prevention reminder system. MD Comput 1990; 7:289–295.PubMedGoogle Scholar
  58. 58.
    Chambers CV, Balaban DJ, Carlson BL. The effect of microcomputer-generated reminders on influenza vaccination rates in a university-based family practice center. J Am Board Fam Pract 1991; 4:19–26.PubMedGoogle Scholar
  59. 59.
    McDonald CJ, Hui SL, Smith DM et al. Reminders to physicians from an introspective computer medical record: a two-year randomized trial. Ann Int Med 1984; 100:130–138.PubMedCrossRefGoogle Scholar
  60. 60.
    Ornstein SM, Garr DR, Jenkins RG et al. Computer-generated physician and patient reminders: tools to improve population adherence to selected preventive services. J Fam Pract 1991; 32:82–90.PubMedGoogle Scholar
  61. 61.
    Soljak MA, Handford S. Early results from the Northland immunization register. N Z Med J 1987; 100:244–246.PubMedGoogle Scholar
  62. 62.
    Austin SM, Balas EA, Mitchell JA et al. Effect of physician reminders on preventive care: Meta-analysis of randomized clinical trials. Proc Annu Symp Comput Appl Med Care 1994:121–124.Google Scholar

Copyright information

© Springer Science+Business Media New York 1999

Authors and Affiliations

  • E. Andrew Balas
  • Suzanne Austin Boren

There are no affiliations available

Personalised recommendations