FormalPara Key Summary Points

This study was unique in that the safety data from both prospective and retrospective methods were collected for comparison in a single post-marketing surveillance (PMS) study

Propensity score matching was used to reduce selection bias and to adjust confounding variables between the prospective and retrospective methods. Overall, the prospective method detected safety outcomes well. Adverse events and serious adverse events were more frequently reported in the prospective data collection method

The incidence of adverse drug reactions (ADRs) was similar between the two methods, especially in patients treated with tigecycline for > 14 days

A retrospective PMS study may be used as an alternative for detecting ADRs, especially for the antimicrobial agent administered for several weeks, when various limitations prevent prospective data collection

Digital Features

This article is published with digital features, including a summary slide, to facilitate understanding of the article. To view digital features for this article go to https://doi.org/10.6084/m9.figshare.13525796.

Introduction

The re-examination of drugs, including new ones for post-marketing safety management, has been required in many countries, including South Korea. The re-examination systems aim to determine the occurrence of adverse events (AEs), including AEs that did not appear in pre-marketing clinical trials for the period set by the health authority (e.g., 4 or 6 years), and to monitor both common and rare AEs and adverse drug reactions (ADRs) [1,2,3,4].

Tigecycline is a first-in-class glycylcycline antibacterial drug approved for the treatment of complicated skin structure infections (cSSTI), complicated intraabdominal infections (cIAI) and community-acquired bacterial pneumonia (CAP) in patients 18 years of age and older. Tigecycline was approved in July 2007 in South Korea. As per the South Korea’s Ministry of Food and Drug Safety requirements, safety and efficacy information for new drugs should be prospectively collected in at least 3000 patients who receive the drug in routine clinical practice within the 6 years after its approval (until July 2013 for tigecycline) [2]. South Korea has a national health insurance system with universal coverage. In most cases, tigecycline is administered to severely ill patients as it is reimbursed only as a second-line therapy. Thus, enrollment of patients for the prospective post-marketing surveillance (PMS) was difficult, leading to an insufficient number of patients during the original prospective re-examination period. Accordingly, the Ministry of Food and Drug Safety permitted an additional 1 year in order to enroll a sufficient number of patients through the retrospective method.

Tigecycline PMS was the first PMS study conducted in South Korea through both prospective and retrospective methods, and this PMS study has the most tigecycline real-world data in Asia. No other PMS studies compare the results of prospective and retrospective safety data collections for an antimicrobial agent. Thus, the purpose of this study is to compare the safety data of tigecycline gathered via prospective and retrospective methods. The collected safety data included AEs, serious AEs (SAEs), ADRs and serious ADRs (SADRs).

Methods

Data Source

Tigecycline PMS was an open-label, noncomparative, observational, noninterventional and multicenter study, examining the safety of tigecycline therapy for patients receiving it. Safety data were collected in two periods using prospective and retrospective methods. In the prospective method, 27 hospitals were involved, but the target sample size of 3000 patients was not achieved. Additional safety data were then collected in the following period using the retrospective method wherein 17 hospitals were involved. Patients who received at least one dose of tigecycline for any of the approved indications on the product label were eligible for the study.

Period I (July 2007 to July 2013) involved prospective collection of data. Patients receiving tigecycline according to the product label, who agreed to sign a data privacy statement, were enrolled in this study. Observations were conducted from the initial administration of tigecycline until the end of treatment or test of cure, which was performed within 14 days after the last tigecycline administration. Comorbidities and the use of concomitant medications were recorded in case report forms. All abnormal findings including AEs during tigecycline therapy were assessed and documented by physicians during the observation period. For the retrospective data collection in period II (April 2014 to April 2015), relevant data were collected by reviewing medical records of patients who had received at least one dose of tigecycline. Similar to the prospective phase, data were collected from the start of tigecycline therapy until the end of treatment or test of cure. The eligibility criteria were the same as those of the prospective method. Detailed information about all concomitant medications and comorbidities was also recorded.

Data from 3172 patients, including those prospectively collected from 738 and those retrospectively collected from 2434, were included in this study. Seven patients with lacking data on the demographic and baseline characteristics were excluded. Finally, 3165 patients were included in the safety evaluation population.

This was a surveillance study mandated by the Ministry of Food and Drug Safety to collect information from patients receiving the drug. The Institutional Review Board of each participating center approved this study according to institutional rules (Supplementary Table S4). In the prospective study, informed consent was obtained from each participant.

Variables

Safety End Point

Safety end points included the incidence of AEs, ADRs, SAEs and SADRs that occurred after tigecycline administration during the observation period. AE was defined as an adverse outcome that occurred while a patient was receiving tigecycline, but the event was not necessarily caused by tigecycline. Meanwhile, ADR was defined as a harmful or unpleasant reaction that had a causal relationship with tigecycline. SAE was defined as any AE that resulted in death, was life-threatening (immediate risk of death), required hospitalization (or prolonged hospitalization), resulted in persistent or significant disability/incapacity (substantial disruption of the ability to conduct normal life functions), resulted in congenital anomaly/birth defect or was an important medical event that had jeopardized the patient or required intervention to prevent one of the other AEs listed in the definition above [5]. Lastly, SADR was defined as any ADR with the same definition as above.

Demographic and Baseline Characteristics

A physician collected the demographic characteristics and other medical conditions at baseline and further recorded the information in each patient’s case report form. Demographic characteristics included sex and age. Medical conditions at baseline included infection site, severity of infection, underlying disease/condition, kidney disorders, liver disorders and concomitant medications. The total administration period of tigecycline was also recorded.

Statistical Analysis

Adjustments of Covariate

If demographic and/or baseline characteristics varied depending on the data collection method, the safety end points may be affected by these variables. To correct bias associated with the data collection method, the effects of demographic characteristics and medical conditions at baseline were adjusted using propensity score matching before conducting the safety analysis. Propensity score matching was performed in four steps. First, we selected covariates for the propensity score calculation in patients with underlying diseases/conditions and concurrent medication. Covariates were as follows: sex, age, infection sites, severity of infection, kidney disorders and liver disorders. Second, a logistic regression model was utilized with the selected covariate variables in calculating the propensity score for each patient. Third, the nearest neighbor method was used to match the propensity scores; the ratio of prospective and retrospective methods was 1:2. Finally, significant differences in covariate variables between data collection methods were determined using chi-square and Fisher’s exact tests to examine covariate balance before and after matching. As part of the exploratory analysis, additional analyses were conducted using the data from 14 hospitals that were involved in both prospective and retrospective data collections.

Safety Analyses

The incidences of AEs, ADRs, SAEs and SADRs for the two methods of data collection were compared using chi-square and Fisher’s exact tests at a statistically significant level of 5%. All statistical analyses were performed using the SAS package version 9.4 (SAS Institute Inc., Cary, NC) or R version 3.3.0 programs.

Results

No significant differences in sex, age and the total duration of tigecycline therapy were noted between the prospective and retrospective methods (Table 1). However, patients in the retrospective method group were significantly more likely to have cIAI, severe infections, underlying diseases/conditions, kidney disorders, liver disorders and concomitant medications compared to those in the prospective method group. Therefore, propensity score matching was performed to adjust for differences in demographic characteristics and medical conditions at baseline. As a result, 482 patients in the prospective method group and 964 in the retrospective method group were selected. The differences between the two groups were corrected after propensity score matching, as shown in Table 1.

Table 1 Assessment of balance of the distributions of demographic and baseline characteristics before and after propensity score matching
Full size table

Safety outcomes for tigecycline are provided in Table 2. AEs, ADRs and SADRs were determined to be similar in the two groups. SAEs occurred more frequently in the retrospective group than in the prospective group, reflecting the severe underlying condition of the former group (P = 0.02). However, in the propensity-adjusted analysis, AEs and SAEs were more frequently reported in the prospective group than in the retrospective group (P < 0.001 and P = 0.004, respectively). The incidences of ADRs were not significantly different between the two groups, and the types and proportions of ADRs were similar (Supplementary Table S1). When the propensity-matched cohort was divided into two subgroups according to the duration of tigecycline therapy (≤ 14 days vs. > 14 days), the demographic and baseline characteristics were generally similar between the prospective and retrospective groups regardless of the duration of therapy. (Supplementary Table S2). In patients treated for ≤ 14 days, AEs, ADRs and SAEs were significantly more frequently reported in the prospective group than in the retrospective group (P < 0.001, P = 0.01 and P = 0.002, respectively) (Supplementary Table S3). However, in patients treated for > 14 days, the incidences of ADRs and SAEs were similar between the two groups (P = 0.85 and P = 0.66, respectively).

Table 2 Comparison of safety outcomes between the prospective and retrospective methods
Full size table

To decrease the inter-investigator differences, we compared the data collected from 14 hospitals involved in both the prospective and retrospective data collections. We analyzed an additional 2397 patients, including 141 from the prospective method and 2256 from the retrospective method. In total, 120 patients in the prospective method and 240 patients in the retrospective method were selected by propensity score matching. The demographic and baseline characteristics were found to be comparable between the two groups after propensity score matching (Table 3). The safety outcomes for tigecycline in these patients are shown in Table 4. As per our findings, AEs were more frequently reported in the prospective group than in the retrospective group (P < 0.001). However, the incidences of ADRs and SAEs were not significantly different between the two groups.

Table 3 Demographic and baseline characteristics of patients from 14 hospitals that participated in both prospective and retrospective studies
Full size table
Table 4 Comparison of safety outcomes between the prospective and retrospective methods in propensity-matched patients from 14 hospitals that participated in both prospective and retrospective studies
Full size table

Discussion

Safety outcomes for tigecycline from prospectively collected data were compared to retrospectively collected PMS data in patients with cSSTI, cIAI and CAP during routine clinical practice in South Korea. We were able to reduce selection bias and adjusted confounding variables using propensity score matching as patients in the retrospective group had more underlying diseases/conditions and were more severely ill than those in the prospective group. AEs and SAEs were more frequently detected using the prospective method unlike when the retrospective method was used, but the incidences of ADRs and SADRs were not statistically different between the two data collection methods. These results indicate that the retrospective method detects AEs less than the prospective method in real clinical practice; however, ADRs were detected at similar rates using both methods. This reflects real clinical practice in which ADRs are rarely omitted from the medical record.

In a PMS study of tigecycline, treatment-emergent AEs were determined to occur in 60.4% of patients regardless of study drug causality. The most common treatment-emergent AEs in this pivotal study were nausea, vomiting and diarrhea [6, 7]. These common AEs were similarly detected in our PMS study, although the incidence of AEs was slightly lower compared with previous PMS studies. In a retrospective study of elderly patients with multidrug-resistant bacterial infections, 42.6% of these patients experienced AEs associated with the use of tigecycline; these AEs include gastrointestinal symptoms, such as nausea and vomiting [8]. In another retrospective study of tigecycline in the treatment of ventilator-associated pneumonia, 26% of the patients experienced side effects, including liver damage and gastrointestinal symptoms [9]. In our PMS study, 32.6% of patients from the retrospective data collection group experienced AEs, such as nausea and vomiting. Thus, our data are consistent with other studies confirming proper data collection.

In our study, 14 hospitals participated in both prospective and retrospective studies. In the subgroup analysis of data from these hospitals, data were analyzed through propensity score matching to decrease inter-investigator variability. More AEs were detected through the prospective method (P < 0.001). However, no significant differences were noted in the incidences of ADRs and SAEs between the prospective and retrospective methods. These results indicate that severe safety outcome and drug-related events are appropriately detected by both methods.

Moreover, the prospective method has been determined to be more valuable in terms of collecting safety information since data collection can be controlled and monitored and variables can be measured consistently [10]. Meanwhile, retrospective data collection is relatively inexpensive, quick and easy to perform, and more patients can be recruited [10,11,12]. Our findings suggest that a retrospective PMS study may be used as an alternative for detecting ADRs and SAEs, especially for the drug administered for several weeks, when various limitations prevent the prospective collection of data or when safety data from a large number of patients are needed for rare diseases.

Our study has several limitations. First, the number of enrolled patients was quite different between the prospective and retrospective methods. Second, a limited number of hospitals participated in both data collection methods. Although propensity score matching and site matching were used to overcome these limitations, further studies should be conducted to generalize our findings. Despite these limitations, this analysis is deemed meaningful. The safety information was collected in a single study through both prospective and retrospective methods. Furthermore, to the best of our knowledge, this is the first analysis to compare prospective and retrospective methods in terms of collecting PMS safety data for an antimicrobial agent.

Conclusion

A prospective data collection method can detect safety data more precisely than a retrospective method for PMS studies. However, retrospective data collection may be a good alternative option for collecting ADR data when a prospective PMS study is not deemed feasible.