International Journal of Clinical Pharmacy

, Volume 36, Issue 4, pp 742–749 | Cite as

A national drug related problems database: evaluation of use in practice, reliability and reproducibility

  • Lene Juel KjeldsenEmail author
  • Trine Birkholm
  • Hanne Fischer
  • Trine Graabæk
  • Merete Kjær Hansen
  • Karina Porsborg Kibsdal
  • Lene Vestergaard Ravn-Nielsen
  • Tania Holtum Truelshøj
Research Article


Background A drug related problems database (DRP-database) was developed on request by clinical pharmacists. The information from the DRP-database has only been used locally e.g. to identify focus areas and to communicate identified DRPs to the hospital wards. Hence the quality of the data at the national level is unknown, which may compromise national analyses for benchmarking and identification of national focus areas. Objective The aim of the study was to evaluate the use in practice, reliability and reproducibility of the DRPs documented in the Danish drug related problems database. Setting Danish hospital pharmacies. Methods Practice use of the DRP-database was explored by an electronic questionnaire distributed to hospital pharmacies, and consisted of questions regarding current and previous use of the DRP-database. The reliability was evaluated by comparing the categorization of 24 cases by clinical pharmacists with categorization performed by the project group. Reproducibility was explored by re-categorization of a sample of existing records in the DRP-database by two project group members individually. Main outcome measures Observed proportion of agreement and Fleiss’ kappa as measures of inter-rater reliability and reproducibility. Results The practice use study of 12 hospital pharmacy locations revealed that when implementing the DRP-database, the majority of identified DRPs are documented in the DRP-database, however, some variations throughout the country exist. The interrater reliability study of 34 clinical pharmacists showed high inter-rater reliability with the project group (Fleiss’ kappa = 0.79 with 95 % CI (0.70; 0.88)), and the reproducibility study also documented high inter-rater reliability of a sample of 379 records from the DRP-database re-categorized by two project group members (Fleiss’ kappa = 0.81 with 95 % CI (0.78; 0.85)). Conclusion The study showed high reliability and reproducibility of the DRP-database, however, some local variation in the use of the DRP-database throughout the country existed affecting the overall quality. These findings indicate that data in the DRP-database may be pooled, and national analyses may be conducted to explore development areas for common interest.


Clinical pharmacy Database Denmark DRP Hospital Medication review 



We would like to thank the clinical pharmacy staff of the different hospitals for their for their contributions.



Conflicts of interest

The authors declare that they have no conflict of interest.

Supplementary material

11096_2014_9957_MOESM1_ESM.doc (68 kb)
Caption of the data object (DOC 68 kb)


  1. 1.
    van den Bemt PM, Egberts TC, van den de Jong Berg LT, Brouwers JR. Drug-related problems in hospitalised patients. Drug Saf. 2000;22(4):321–33.PubMedCrossRefGoogle Scholar
  2. 2.
    Krähenbühl-Melcher A, Schlienger R, Lampert M, Haschke M, Drewe J, Krähenbühl S. Drug-related problems in hospitals: a review of the recent literature. Drug Saf. 2007;30(5):379–407.PubMedCrossRefGoogle Scholar
  3. 3.
    Graabaek T, Kjeldsen LJ. Medication reviews by clinical pharmacists at hospitals lead to improved patient outcomes: a systematic review. Basic Clin Pharmacol Toxicol. 2013;112(6):359–73.PubMedCrossRefGoogle Scholar
  4. 4.
    Van Mil F, Westerlund T, Hersberger KE, Schaefer MA. Drug-related problem classification systems. Ann Pharmacother. 2004;38:859–67.PubMedCrossRefGoogle Scholar
  5. 5.
    Hepler CD, Strand LM. Opportunities and responsibilities in pharmaceutical care. Am J Hosp Pharm. 1990;47:533–43.PubMedGoogle Scholar
  6. 6.
    Hanlon JT, Schmader KE, Samsa GP, Weinberger M, Uttech KM, Lewis IK, et al. A method for assessing drug therapy appropriateness. J Clin Epidemiol. 1992;45:1045–51.PubMedCrossRefGoogle Scholar
  7. 7.
    ASHP guidelines on a standardized method for pharmaceutical care. In: Deffenbaugh J, editor. Best practices for health-system pharmacy. Bethesda, MD: American Society of Health-System Pharmacists, 1996. pp. 109–11.Google Scholar
  8. 8.
    Pharmaceutical Care Network Europe. DRP-classification V5.01 [internet] [updated 2006 Jun 29; cited 2014 Feb 14]. Available from:
  9. 9.
    WHO Collaborating Centre for Drug Statistics Methodology. The Anatomical Therapeutic Chemical (ATC) classification system [internet] [updated 2011 Mar 25; cited 2014 Feb 14]. Available from:
  10. 10.
    Fleiss J. Measuring nominal scale agreement among many raters. Psychol Bull. 1971;76(5):378–82.CrossRefGoogle Scholar
  11. 11.
    Gamer M, Lemon J and Singh IRR. Various Coefficients of Interrater Reliability and Agreement. R package version 0.84 [internet] [updated 2012 Jul 16; cited 2014 Feb 14]. Available from::
  12. 12.
    Canty A, Ripley B. Boot: bootstrap R (S-Plus) functions. R package version 1.3-9, 2013.Google Scholar
  13. 13.
    Efron B. Better bootstrap confidence intervals. J Am Stat Assoc. 1987;82(397):171–85.CrossRefGoogle Scholar
  14. 14.
    Hall P. Theoretical comparison of bootstrap confidence intervals. Ann Stat. 1988;16(3):927–53.CrossRefGoogle Scholar
  15. 15.
    Sim J, Wright CC. The kappa statistic in reliability studies: use, interpretation, and sample size requirements. Phys Ther. 2005;85:257–68.PubMedGoogle Scholar
  16. 16.
    Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33:159–74.PubMedCrossRefGoogle Scholar
  17. 17.
    R Core Team. R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria [internet]; 2013 [updated 2013 Sep 25; cited 2014 Feb 14]. Available from:
  18. 18.
    Nielsen TR, Andersen SE, Rasmussen M, Honoré PH. Clinical pharmacist service in the acute ward. Int J Clin Pharm. 2013;35(6):1137–51.PubMedCrossRefGoogle Scholar
  19. 19.
    Hohmann C, Eickhoff C, Klotz JM, Schulz M, Radziwill R. Development of a classification system for drug-related problems in the hospital setting (APS-Doc) and assessment of the inter-rater reliability. J Clin Pharm Ther. 2012;37(3):276–81.PubMedCrossRefGoogle Scholar
  20. 20.
    Allenet B, Bedouch P, Rose FX, Escofier L, Roubille R, Charpiat B, Juste M, Conort O. Validation of an instrument for the documentation of clinical pharmacists’ interventions. Pharm World Sci. 2006;28(4):181–8.PubMedCrossRefGoogle Scholar
  21. 21.
    Somers A, Mallet L, van der Cammen T, Robays H, Petrovic M. Applicability of an adapted medication appropriateness index for detection of drug-related problems in geriatric inpatients. Am J Geriatr Pharmacother. 2012;10(2):101–9.PubMedCrossRefGoogle Scholar
  22. 22.
    Hoth AB, Carter BL, Ness J, Bhattacharyya A, Shorr RI, Rosenthal GE, Kaboli PJ. Development and reliability testing of the clinical pharmacist recommendation taxonomy. Pharmacotherapy. 2007;27(5):639–46.PubMedCrossRefGoogle Scholar
  23. 23.
    Sorensen L, Grobler MP, Roberts MS. Development of a quality use of medicines coding system to rate clinical pharmacists’ medication review recommendations. Pharm World Sci. 2003;25(5):212–7.PubMedCrossRefGoogle Scholar
  24. 24.
    Somers A, Robays H, De Paepe P, Van Maele G, Perehudoff K, Petrovic M. Evaluation of clinical pharmacist recommendations in the geriatric ward of a Belgian university hospital. Clin Interv Aging. 2013;8:703–9.PubMedCentralPubMedCrossRefGoogle Scholar

Copyright information

© Koninklijke Nederlandse Maatschappij ter bevordering der Pharmacie 2014

Authors and Affiliations

  • Lene Juel Kjeldsen
    • 1
    Email author
  • Trine Birkholm
    • 2
  • Hanne Fischer
    • 1
  • Trine Graabæk
    • 3
  • Merete Kjær Hansen
    • 4
  • Karina Porsborg Kibsdal
    • 5
  • Lene Vestergaard Ravn-Nielsen
    • 6
  • Tania Holtum Truelshøj
    • 7
  1. 1.The Danish Research Unit for Hospital PharmacyCopenhagenDenmark
  2. 2.Region Zealand Hospital PharmacyNæstvedDenmark
  3. 3.Centre for Clinical ResearchHospital South West JutlandEsbjergDenmark
  4. 4.DTU Compute, Section for Statistics and Data AnalysisTechnical University of DenmarkCopenhagenDenmark
  5. 5.The Hospital Pharmacy North Denmark RegionAalborgDenmark
  6. 6.Clinical Pharmacy UnitOdense University HospitalOdenseDenmark
  7. 7.The Hospital Pharmacy Central Denmark RegionAarhusDenmark

Personalised recommendations