Introduction

Available literature indicates that 25–50% of hospitalized patients receive antibiotics, of which 20–50% are either unnecessary or inappropriate [1,2,3,4,5]. Antimicrobial stewardship programs (ASPs) are coordinated programs designed to improve the appropriateness of antibiotic use [6,7,8]. One of the cornerstones of an ASP is to monitor the total amount of local antibiotic use and use this information to guide and evaluate targeted ASP interventions [7, 9].

Several units of measurement are available to standardize total antibiotic use. Recommended metrics are defined daily dose (DDD) and days of therapy (DOT) [10,11,12,13]. DDD is defined by the World Health Organization (WHO) as the assumed average maintenance dose per day for a drug used for its main indication in adults [14, 15]. One DOT represents the administration of a single agent on a given day regardless of the number of doses administered or dosage strength [10].

Advantages and disadvantages of both metrics have been recognized. DDDs allow for standardized comparison of aggregate antibiotic use between hospitals and are usually extracted from hospital billing or hospital dispensing records, which makes the metric applicable even in countries with limited access to computerized pharmacy or prescription data. However, there are substantial limitations to this metric. For example, DDDs are influenced by dose adjustment and will therefore underestimate antibiotic use in patients in whom dose adjustment is required, for instance children or patients with renal impairment. Also, DDD is a unit of measurement and does not necessarily reflect the recommended or prescribed daily dose [7, 10]. Using DOT is recommended by the IDSA guidelines, as it is usually based on patient-level prescription data and therefore not influenced by dose adjustment. However, prescription data are difficult to assess without computerized physician order entry (CPOE) of individual patients [7, 10].

In Europe and in the USA, surveillance reports on antibiotic use and resistance rates are issued annually. The European Surveillance of Antimicrobial Consumption Network (ESAC-NET), which is managed and coordinated by the European Centre for Disease Prevention and Control (ECDC), and the Dutch Working Party on Antibiotic Policy (SWAB) provide reference data on hospital antibiotic consumption using DDD [16, 17]. In contrast, the US Center for Disease Control and Prevention (CDC) uses DOT [18]. These reports are used for surveillance purposes, but they have not shown to be useful in ASPs, as they do not provide a detailed assessment of quantitative antibiotic use, e.g., per diagnosis or medical specialty, which is needed to guide and evaluate targeted ASP interventions.

A recent literature review stated that electronic assessment of antibiotic use data is potentially useful for the purpose of antimicrobial stewardship; however, the best approach to retrieve reliable quantitative data is not yet clear and might also be determined by local hospital settings, procedures, and budget [19]. The objectives of the present study were (1) to explore for the hospital setting the possibilities of quantitative data retrieval on the level of medical specialty and (2) to describe factors affecting the usability and interpretation of these quantitative metrics.

Materials and methods

Study setting and population

This study was part of a cluster-randomized, multicenter trial (http://www.trialregister.nl; NTR 5933) that compared three different methods intended to measure the appropriateness of antibiotic use in hospitalized adult patients. One of these methods assessed total antibiotic use in adults. For one university and 13 non-university hospitals allocated to this study arm, data on systemic antibiotic use, number of admissions, and number of patient days were retrospectively collected for a 1-year period, from December 1st 2014 to December 1st 2015. Seven hospitals reported data for three surgical specialties: general surgery (including abdominal and vascular surgery), urology, and orthopedic surgery (surgical cluster); the other seven hospitals reported antibiotic use data for three non-surgical specialties: internal medicine (including nephrology and geriatric patients), pulmonology, and gastro-enterology (non-surgical cluster).

Retrieval of quantitative antibiotic use on a level of medical specialty

Hospital pharmacies were requested to report monthly use of antibiotics belonging to ATC-group J01 (antibiotics for systemic use, http://whocc.no) using two data sources: (1) dispensing data and (2) prescription data. If a hospital pharmacy was able to extract data on the level of medical specialty (i.e., the medical specialism that holds responsibility for the patient of interest), they were requested to do so. If not, they were requested to extract data for the medical ward (i.e., the functional sub-unit within the hospital) where the majority of patients from the participating medical specialty were admitted to.

Dispensing data included the generic name of each antibiotic drug, ATC code, dosage form, the total amount of antibiotics dispensed, unit of measurement, and admission specialty or ward. DDDs were calculated from this data source using the ATC/DDD classification developed by WHO Collaborating Centre for Drug Statistics Methodology (http://whocc.no; 2016.). Prescription data consisted of information on antibiotics prescribed to individual patients (patient level) and included the generic name of each antibiotic drug, ATC code, dosage form, unit of measurement, start date and time, stop date and time, and admission specialty or ward. The multicenter trial focused on antibiotics used as therapy for a suspected bacterial infection. Therefore, antibiotics used as surgical prophylaxis had to be excluded. Antibiotic courses with a duration of less than 24 h were seen as surgical prophylaxis if they concerned the antibiotics cefazolin, cefuroxime, clindamycin, and vancomycin. DOTs were calculated from this data source using Visual FoxPro 6.

DDDs and DOTs, as defined above, were standardized per 100 patient days and per 100 admissions, respectively, to control for differences in hospital census and length of hospital stay [10, 11, 14]. Patient days were defined as the aggregate sum of days that patients were admitted to the hospital.

Assessment of the relationship between DDDs and DOTs

Over a 1-year study period, a monthly mean of DDD per 100 admissions, DDD per 100 patient days, DOT per 100 admissions, and DOT per 100 patient days was calculated for each hospital by aggregating monthly data on antibiotic use, patient days, and admissions from each hospital in our sample [20, 21]. The relationship between DDDs and DOTs was first examined graphically for each hospital. DOT estimates were log transformed to approach a more normal distribution. The relationship between mean DDDs and DOTs was assessed for hospitals with corresponding extraction methods for DDD/DOT and denominators using Spearman’s rank correlation test. P values < .05 were considered statistically significant. Statistical analyses were performed using IBM SPSS statistics 23.

Identification of factors affecting the usability and interpretation of quantitative metrics

Based on expert opinion, we explored whether the following factors affected the retrieved DDDs and DOTs: organizational factors, data sources, data registration, and data extraction.

Results

Hospital characteristics

Of 14 participating acute care hospitals, one was a university hospital, the others were non-university hospitals. The mean number of hospital beds was 556 (range 255–854). Twelve out of 14 hospitals (86%) had mixed wards, indicating that patients from different specialties (surgical and/or non-surgical) were admitted to one ward. All hospitals used an electronic patient system and CPOE (Table 1).

Table 1 Characteristics and data extraction details of the participating hospitals
Full size table

Retrieval of quantitative antibiotic use on the level of medical specialty

Twelve out of 14 hospitals (86%) were able to extract dispensing data (DDDs) on a ward or specialty level. Two hospitals could only extract dispensing data for the entire hospital, because these hospitals had central distribution units in which antibiotics were aggregately stored. Three of 12 hospitals were able to extract the dispensing data on the level of medical specialty and nine hospitals could only extract on a ward level. Thirteen out of 14 hospitals were able to extract patient-level prescription data (DOTs), of which 11 hospitals could extract the prescription data on the level of specialty. One hospital was not able to extract patient-level prescription data due to migration of the electronic patient system at the time of the study. Data on “patient days” and “admissions” could be retrieved by all hospitals. Six of 14 hospitals (43%) were able to retrieve these denominators per specialty (Table 1); the rest could only retrieve the data hospital-wide.

Assessment of the relationship between DDDs and DOTs

Mean monthly antibiotic use expressed in DDD was 440/100 admissions for the non-surgical cluster and 290/100 admissions for the surgical cluster, respectively 49/100 patient days for the non-surgical cluster and 61/100 patient days for the surgical cluster. When expressed in DOT, mean monthly antibiotic use was 515/100 admissions for the non-surgical cluster, 300/100 admissions for the surgical cluster, 52/100 patient days for the non-surgical cluster, and 67/100 patient days for the surgical cluster. Appendices 1–4 illustrate monthly antibiotic use per cluster for DDD and DOT.

Mean monthly antibiotic use varied substantially between hospitals (Fig. 1). Hospitals 7, 10, and 14 showed a disproportionally high mean monthly antibiotic use and a wide range in monthly antibiotic use for DOT. Hospitals 4 and 10 showed a large variation in monthly antibiotic use for DDD (Fig. 1 and Appendices 1–4). The overall correlation between DDDs and DOTs in our study was low, as well as the correlation between DDD and DOT per hospital (Figs. 1 and 2). Hospital 12 was the only hospital that showed a significant correlation between both metrics (Table 2).

Fig. 1
figure 1

Monthly antibiotic use in 14 hospitals, expressed in DOT versus DDD, standardized per 100 admissions

Full size image
Fig. 2
figure 2

Monthly antibiotic use in 14 hospitals, expressed in DOT versus DDD, standardized per 100 patient days

Full size image
Table 2 Detailed information on data extraction and correlation of antibiotic use per hospital
Full size table

Identification of factors affecting the usability and interpretation of quantitative metrics

The following factors were identified as possible causes affecting the retrieved DDDs and DOTs. A summary of our findings, together with our supplementary recommendations, is provided in Table 3.

Table 3 Reasons for low correlation between DDD and DOT in the assessment of quantitative antibiotic use on a level of specialty and recommendations for hospitals regarding data sources, data extraction, and units of measurement
Full size table

Organizational factors

Twelve out of 14 hospitals (86%) had mixed wards and the composition of contributing specialties within a mixed ward differed per hospital. In five of these hospitals, patients from surgical and non-surgical specialties were mixed at the same ward. As antibiotics were usually dispensed aggregately to a ward and not registered per specialty, DDDs were not able to give an adequate estimate of antibiotic use at specialty level in the hospitals with mixed wards. Hospital 10 had a reorganization of wards during the study period, causing a wide variation in monthly antibiotic use, both for DDD and DOT (Fig. 1 and Table 2).

Data sources

Aggregated dispensing data, used to calculate DDD, do not tell which of the dispensed antibiotics are actually administered to a patient. Patient-level prescription data, used to calculate DOT, however, are able to give a more adequate estimation of the actually administered antibiotics. Furthermore, monthly dispensing data may fluctuate substantially as monthly hospital stock orders are based on their previous antibiotic usage rates and predictions on patient admission rates (e.g., taking into account seasonal influences). This could result in a large variation in mean monthly antibiotic use within hospitals. Aggregated dispensing data are therefore more accurate when calculated over longer intervals.

Data registration and data extraction

An inconsistency in registration was found in hospitals 7 and 14: prescription data were extracted based on the prescribers’ specialty instead of the patients’ specialty of admission. As residents often rotate between specialties, while they are registered in the electronic prescribing system under the responsibility of their main specialty (internal medicine or surgery), this resulted in disproportionately high prescription rate for the main specialty (Fig. 1 and Table 2).

Eight hospitals did not have corresponding extraction methods for DDD/DOT and denominators (i.e., dispensing or prescription data were extracted per specialty and the denominators were extracted per ward), see Table 2. As these hospitals also had mixed wards with differences in composition of contributing specialties, we were not able to compare DDDs and DOTs from these hospitals. From the remaining hospitals, two were unable to extract either prescription or dispensing data. The remaining hospitals 3, 5, 7, and 12 used comparable extraction methods for both DDD/DOT and denominator. The low correlation for hospital 7 was explained by the problems in data registration described above. Other possible explanations for low correlation between DDD and DOT for hospitals 3 and 5 are listed in Table 3. A significant correlation between DDD and DOT was only found in hospital 12, as data registration and extraction methods were similar for both metrics and denominators, and data were extracted per medical specialty, neglecting the influence of mixed wards.

Discussion

In this observational multicenter study, we observed a large variation in antibiotic use between and within hospitals and a low correlation between DDD and DOT as metrics of total antibiotic use in hospitalized adult patients. We explored several factors potentially affecting the retrieved data and found that part of the variation in quantitative antibiotic use is likely caused by differences in organizational factors, data sources, data registration, and data extraction. Also, we showed that for measuring quantitative antibiotic use for ASP purposes at the level of medical specialty, it is currently preferable to use patient-level prescription data.

Previous literature showed that differences in antibiotic use between hospitals can be partially explained by patient mix or hospital characteristics. For example, vancomycin use is significantly higher in university hospitals as compared to large teaching or general hospitals, due to differences in patient mix [22]. However, in our study, the variation in mean monthly antibiotic use between and within the hospitals was more extensive than the differences in antibiotic use presented in the annual antibiotic consumption reports [22], and therefore unlikely to result exclusively from differences in patient mix and hospital type.

We found a low correlation between DDD and DOT for three out of four hospitals that used corresponding extraction procedures for numerators and denominators, in contrary to a large US study by Polk et al. who found an overall linear association between DDD and DOT. In the USA, billing records are used as a data source to measure antibiotic use, and both metrics were calculated from the same data source [10]. In the Netherlands, costs for in-patient antibiotics are integrated in the overall hospital budget; thus, billing data cannot be used as a data source. Data sources that can be used to measure antibiotic use in the Netherlands are dispensing data, to calculate DDD, and prescription data, to calculate DOT. The low correlation we found between DDD and DOT might therefore be partly explained by the use of different data sources for each metric. In addition, the low correlation could be explained by errors in data registration and extraction procedures, e.g., hospitals extracted prescription data based on the prescribers’ specialty instead of the patients’ specialty of admission (see Table 3).

The Dutch healthcare system at present stimulates reorganization of hospital wards into “mixed wards” (i.e., physical locations with a mix of medical specialties). Basically, all hospitals in our study consisted of mixed wards, and the distribution of medical specialties within mixed wards differed per hospital. The majority of Dutch hospital pharmacies in our study were able to extract prescription data on the level of medical specialty (79%). The main reasons for three hospitals not to be able to extract prescription data were technical difficulties and lack of knowledge of the extraction procedure by the IT specialist at the time of the study. Dispensing data, however, could be extracted on specialty level by only 21% of the pharmacies, as antibiotics are usually dispensed aggregated per ward or unit and are not registered per medical specialty. In addition, antibiotics dispensed to a ward do not provide accurate information whether these antibiotics are actually administered to a patient. Compared to dispensing data, patient-level prescription data are able to give a more adequate estimation of the actually administered antibiotics. In addition, patient-level data can be linked to other data registered in the patient system (such as indication and culture results) making it possible to relate quantitative antibiotic use data to resistance data. As a result, for ASP purposes, prescription data currently provide a more valid metric to compare levels of antibiotic use between medical specialties.

Finally, a study on retrieval of antibiotic use data from computerized pharmacy data on the intensive care unit found that computerized patient-level measures can be derived easily, but the magnitude of discrepancies between computerized antibiotic use data and manual chart review varied, with electronic medication administration records (eMAR) providing maximal accuracy [23]. In the future, the use of administration registration would be preferable over prescription data; however, calculation of antibiotic use from administration records is not yet possible in most clinical settings.

Our study has several strengths. This is the first study to focus on the process of data registration and extraction, in order to understand variation between hospitals in quantitative antibiotic use and to detect inconsistencies between DDD and DOT as measures of antibiotic use. Also, our study focusses on medical clusters of specialties in the assessment of quantitative antibiotic use, which is highly recommended for the evaluation of ASPs, whereas most studies evaluated antibiotic use on a hospital level or only compared wards with a relatively high antibiotic use, e.g., intensive care units [10,11,12, 24, 25].

Our study was limited by the relatively small number of hospitals participating in the study. Also, we only included Dutch hospitals. However, as differences between Dutch hospitals likely reflect the (even larger) variability in healthcare organizations throughout Europe, using a variety of electronic patient systems, each with different registration modes and extraction possibilities, the findings of our study are of relevance for other countries as well, including cross-country comparisons.

A recent literature review described the difficulties in secondary use of data from hospital electronic prescribing and pharmacy systems to support ASP, including data access, data accuracy, and completeness, and discussed the complexity of data extraction from multiple electronic systems or hospital sites [19]. Our study showed that differences between hospitals in organizational factors, data sources, data registration, and data extraction contribute to the variation between hospitals in quantitative use and a low correlation between DDDs and DOTs. A clear understanding of these factors, together with a uniform and transparent approach in defining organizational units within hospitals, and uniform data sources, registration, and extraction procedures are necessary for reliable measurement and valid comparison of antibiotic use.