International Journal of Clinical Pharmacy

, Volume 35, Issue 2, pp 195–201 | Cite as

Risk analysis and user satisfaction after implementation of computerized physician order entry in Dutch hospitals

  • Willem van der VeenEmail author
  • Han J. J. de Gier
  • Tjerk van der Schaaf
  • Katja Taxis
  • Patricia M. L. A. van den Bemt
Research Article


Background Computerized physician order entry (CPOE) in hospitals is widely considered to be important for patient safety, but implementation is lagging behind and user satisfaction is often low. Risk analysis methods may improve the implementation process and thus user satisfaction. Objective The aim of our study was to determine the association of performing risk analysis with user satisfaction after implementation of CPOE. Setting All hospitals in the Netherlands. Method A cross-sectional study using a questionnaire was performed. All Dutch hospital pharmacies were asked about the extent of implementation of CPOE in the hospitals they served, the performance of (retrospective or prospective) risk analysis and the satisfaction with CPOE of doctors, nurses and pharmacists. Only hospitals that had implemented inpatient CPOE on at least 70 % of the wards were included in the primary analysis. Main outcome measure The primary outcome measure was the proportion of hospital pharmacists with a satisfaction level of 4 or 5 (i.e. ‘satisfied’). The secondary outcome measure was the proportion of medical doctors and nurses with a satisfaction level of 4 or 5 (i.e. satisfied). The main determinant was the performance of a formal method of prospective or retrospective risk analysis. Results The questionnaire was sent to all 79 Dutch hospital pharmacies. Questionnaires were returned by 70 hospital pharmacies, serving 72 separate hospitals. In 40 hospitals the CPOE was implemented on at least 70 % of the wards. The association of risk analysis with the proportion of satisfied users was determined within this group of 40 hospitals. For hospital pharmacists we found that the performance of risk analysis showed a statistically non-significant trend towards an association with satisfaction [OR 3.3 (95 % CI 0.8–14.1)]. For medical doctors the performance of risk analysis was associated with satisfaction [OR 10.0 (95 % CI 1.8–56.0)]. Also a statistically non-significant trend towards an association with satisfaction was found for nurses [OR 4.5 (95 % CI 0.8–24.7)]. Conclusion Although not statistically significant, the user satisfaction with CPOE seems to be associated with the performance of risk analysis during the implementation of CPOE. This suggests that the CPOE implementation process can be optimized by performing risk analysis before and/or after implementation.


CPOE FMEA Implementation RCA Risk analysis User satisfaction The Netherlands 



We would like to express our gratitude to the Dutch Hospital Pharmacists who cooperated in this study. Also, we would like to thank Lisette Hoekstra and Fleur Kos (students) for performing the pilot of this study.


No funding was received for this study.

Conflicts of interest

All authors declared that they have no competing interests: no support from any organization for the submitted work; no financial relationships with any organization that might have an interest in the submitted work in the previous five years; no other relationships or activities that could appear to have influenced the submitted work.

Supplementary material

11096_2012_9727_MOESM1_ESM.doc (45 kb)
Supplementary material 1 (DOC 45 kb)


  1. 1.
    Niazkhani Z, Pirnejad H, Berg M, Aarts J. The impact of computerized provider order entry systems on inpatient clinical workflow: a literature review. J Am Med Inform Assoc. 2009;16:539–49.PubMedCrossRefGoogle Scholar
  2. 2.
    Yu FB, Menachemi N, Berner ES, Allison JJ, Weissman NW, Houston TK. Full implementation of computerized physician order entry and medication-related quality outcomes: a study of 3364 hospitals. Am J of Med Qual. 2009;24:278–86.CrossRefGoogle Scholar
  3. 3.
    Wolfstadt JI, Gurwitz JH, Field TS, Lee M, Kalkar S, Wu W, et al. The effect of computerized physician order entry with clinical decision support on the rates of adverse drug events: a systematic review. J Gen Int Med. 2008;23:451–8.CrossRefGoogle Scholar
  4. 4.
    Kaushal R, Jha AK, Franz C, Glaser J, Shetty KD, Jaggi T, et al. Return on investment for a computerized physician order entry system. J Am Med Inform Assoc. 2006;13:261–6.PubMedCrossRefGoogle Scholar
  5. 5.
    Garnica MP. CPOE: an essential tool for evidence-based practice. JNP. 2011;36:12–5.Google Scholar
  6. 6.
    Hagland M. CPOE and patient safety. Healthcare inform. 2011;28:76–8.Google Scholar
  7. 7.
    Hollister D Jr, Messenger A. Implementation of computerized physician order entry at a community hospital. Conn Med. 2011;75:227–33.PubMedGoogle Scholar
  8. 8.
    Reckmann MH, Westbrook JI, Koh Y, Lo C, Day RO. Does computerized provider order entry reduce prescribing errors for hospital inpatients? A systematic review. J Am Med Inform Assoc. 2009;16:613–23.PubMedCrossRefGoogle Scholar
  9. 9.
    Koppel R, Metlay JP, Cohen A, Abaluck B, Localio AR, Kimmel SE, et al. Role of computerized physician order entry systems in facilitating medication errors. J Am Med Inform Assoc. 2005;293:1197–203.CrossRefGoogle Scholar
  10. 10.
    Han YY, Carcillo JA, Venkataraman ST, Clark RS, Watson RS, Nguyen TC, et al. Unexpected increased mortality after implementation of a commercially sold computerized physician order entry system. Pediatrics. 2005;116:1506–12.PubMedCrossRefGoogle Scholar
  11. 11.
    van Doormaal JE, van den Bemt PM, Mol PG, Zaal RJ, Egberts AC, Haaijer-Ruskamp FM, et al. Medication errors: the impact of prescribing and transcribing errors on preventable harm in hospitalised patients. J Qual Saf Health Care. 2009;18:22–7.CrossRefGoogle Scholar
  12. 12.
    van Doormaal JE, Mol PG, Zaal RJ, van den Bemt PM, Kosterink JG, Vermeulen KM, et al. Computerized physician order entry (CPOE) system: expectations and experiences of users. J Eval Clin Pract. 2010;16:738–43.PubMedCrossRefGoogle Scholar
  13. 13.
    Chen AR, Lehmann CU. Computerized provider order entry in pediatric oncology: design, implementation, and outcomes. J Oncol Pract. 2011;7:218–22.PubMedCrossRefGoogle Scholar
  14. 14.
    IOM. Health IT and patient safety: building safer systems for better care. Washington DC: The National Academies Press; 2012.Google Scholar
  15. 15.
    Sittig DF, Singh H. Defining health information technology-related errors: new developments since to err is human. Arch Int Med. 2011;171:1281–4.CrossRefGoogle Scholar
  16. 16.
    Niazkhani Z, Pirnejad H, van der Sijs H, Aarts J. Evaluating the medication process in the context of CPOE use: the significance of working around the system. J Am Med Inform Assoc. 2011;80:490–506.Google Scholar
  17. 17.
    Coiera E, Aarts J, Kulikowski C. The dangerous decade. J Am Med Inform Assoc. 2012;19:2–5.PubMedCrossRefGoogle Scholar
  18. 18.
    Niazkhani Z, Pirnejad H, van der Sijs H, de Bont A, Aarts J. Computerized provider order entry system—does it support the inter-professional medication process? Lessons from a Dutch academic hospital. Methods Inform Med. 2010;49:20–7.Google Scholar
  19. 19.
    Bates DW. Computerized physician order entry and medication errors: finding a balance. J Biomed Inform. 2005;38:259–61.PubMedCrossRefGoogle Scholar
  20. 20.
    Strom BL, Schinnar R. Center for E, Research on Therapeutics UoP. Evaluating health information technology’s clinical effects. LDI issue brief. 2011;16:1–4.PubMedGoogle Scholar
  21. 21.
    Rahimi B, Timpka T, Vimarlund V, Uppugunduri S, Svensson M. Organization-wide adoption of computerized provider order entry systems: a study based on diffusion of innovations theory. BMC Med Inform Decis Mak. 2009;9:52.PubMedCrossRefGoogle Scholar
  22. 22.
    Maslove DM, Rizk N, Lowe HJ. Computerized physician order entry in the critical care environment: a review of current literature. J Intensive Care Med. 2011;26:165–71.PubMedCrossRefGoogle Scholar
  23. 23.
    Classen DC, Avery AJ, Bates DW. Evaluation and certification of computerized provider order entry systems. J Am Med Inform Assoc. 2007;14:48–55.PubMedCrossRefGoogle Scholar
  24. 24.
    Ozdas A, Miller RA. In: Geissbuhler A, Haux R, Kulikowski C, editors. IMIA Yearbook of Medical Informatics 2007. Methods Inf Med 2007;46(1):128–37.Google Scholar
  25. 25.
    Bartos CE, Butler BS, Crowley RS. Ranked Levels of Influence model: selecting influence techniques to minimize IT resistance. J Biomed Inform. 2011;44:497–504.PubMedCrossRefGoogle Scholar
  26. 26.
    Harrison MI, Koppel R, Bar-Lev S. Unintended consequences of information technologies in health care—an interactive sociotechnical analysis. J Am Med Inform Assoc. 2007;14:542–9.PubMedCrossRefGoogle Scholar
  27. 27.
    Bonnabry P, Despont-Gros C, Grauser D, Casez P, Despond M, Pugin D, et al. A risk analysis method to evaluate the impact of a computerized provider order entry system on patient safety. J Am Med Inform Assoc. 2008;15:453–60.PubMedCrossRefGoogle Scholar
  28. 28.
    Saizy-Callaert S, Causse R, Thebault A, Chouaïd C. Analysis of mode of failure, their effects and criticality: improving of the hospital drug prescribing process. Therapie. 2001;56:525–31.PubMedGoogle Scholar
  29. 29.
    Williams E, Talley R. The use of failure mode effect and criticality analysis in a medication error subcommittee. Hospital pharmacy 1994; 29:331–2, 34–6, 39.Google Scholar
  30. 30.
    Habraken MM, Van der Schaaf TW, Leistikow IP, Reijnders-Thijssen PM. Prospective risk analysis of health care processes: a systematic evaluation of the use of HFMEA in Dutch health care. Ergonomics. 2009;52:809–19.PubMedCrossRefGoogle Scholar
  31. 31.
    Stalhandske E, DeRosier J, Patail B, Gosbee J. How to make the most of failure mode and effect analysis. Biomed Instrum Technol. 2003;37:96–102.PubMedCrossRefGoogle Scholar
  32. 32.
    DeRosier J, Stalhandske E, Bagian JP, Nudell T. Using health care failure mode and effect analysis: the VA National Center for Patient Safety’s prospective risk analysis system. Jt Comm J Qual Improv. 2002;28(248–67):09.Google Scholar
  33. 33.
    Nicolini D, Waring J, Mengis J. Policy and practice in the use of root cause analysis to investigate clinical adverse events: mind the gap. Soc Sci Med. 2011;73:217–25.PubMedCrossRefGoogle Scholar
  34. 34.
    Nicolini D, Waring J, Mengis J. The challenges of undertaking root cause analysis in health care: a qualitative study. J Health Serv Res Policy. 2011;16(Suppl 1):34–41.PubMedCrossRefGoogle Scholar
  35. 35.
    Franklin BD, Shebl NA, Barber N. Failure mode and effects analysis: too little for too much? BMJ Qual Saf (Published Online First: March 23, 2012) doi:  10.1136/bmjqs-2011-000723.
  36. 36.
    Giesen D, Meertens V, Vis R, Beukenhorst D. Vragenlijstontwikkeling (questionnaire development). The Hague, The Netherlands: Dutch Statistics 2010.Google Scholar
  37. 37.
    Wierenga PC, Lie-A-Huen L, de Rooij SE, Klazinga NS, Guchelaar HJ, Smorenburg SM. Application of the Bow-Tie model in medication safety risk analysis: consecutive experience in two hospitals in the Netherlands. Drug Saf. 2009;32:663–73.PubMedCrossRefGoogle Scholar
  38. 38.
    Hudson PT, Guchelaar HJ. Risk assessment in clinical pharmacy. Pharm World Sci. 2003;25:98–103.PubMedCrossRefGoogle Scholar
  39. 39.
    Kessels-Habraken M, De Jonge J, Van der Schaaf T, Rutte C. Prospective risk analysis prior to retrospective incident reporting and analysis as a means to enhance incident reporting behaviour: a quasi-experimental field study. Soc Sci Med. 2010;70:1309–16.PubMedCrossRefGoogle Scholar
  40. 40.
    Percarpio KB, Watts BV, Weeks WB. The effectiveness of root cause analysis: what does the literature tell us? Jt Comm J Qual Pat Saf. 2008;34:391–8.Google Scholar
  41. 41.
    Amo MF. Root cause analysis. A tool for understanding why accidents occur. Balance. 1998;2:12–5.PubMedGoogle Scholar
  42. 42.
    Snijders C, van der Schaaf TW, Klip H, van Lingen RA, Fetter WP, Molendijk A. Feasibility and reliability of PRISMA-medical for specialty-based incident analysis. Qual Saf Health Care. 2009;18:486–91.PubMedCrossRefGoogle Scholar
  43. 43.
    Likert R. Public opinion polls. Sci Am. 1948;179:7–11.PubMedGoogle Scholar
  44. 44.
    Apkon M, Leonard J, Probst L, Lee M, Kalkar S, Wu W, et al. Design of a safer approach to intravenous drug infusions: failure mode effects analysis. Qual Saf Health Care. 2004;13:265–71.PubMedCrossRefGoogle Scholar
  45. 45.
    Aarts J, Koppel R. Implementation of computerized physician order entry in seven countries. Health Aff. 2009;28:404–14.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2012

Authors and Affiliations

  • Willem van der Veen
    • 1
    • 2
    Email author
  • Han J. J. de Gier
    • 3
  • Tjerk van der Schaaf
    • 4
  • Katja Taxis
    • 3
  • Patricia M. L. A. van den Bemt
    • 5
  1. 1.Department of Hospital PharmacyHospital Röpcke-ZweersHardenbergThe Netherlands
  2. 2.Research Institute SHARE, Graduate School of Medical SciencesUniversity of GroningenGroningenThe Netherlands
  3. 3.Division of Pharmacotherapy and Pharmaceutical Care, Department of PharmacyUniversity of GroningenGroningenThe Netherlands
  4. 4.PRISMA Safety Management SystemsEindhovenThe Netherlands
  5. 5.Department of Hospital PharmacyErasmus MCRotterdamThe Netherlands

Personalised recommendations