Introduction

Pharmacovigilance centers have a major role in the post-marketing monitoring of drug safety for which the spontaneous reporting system is the best recognized. [1, 2] The main function of this system is the early detection of new, rare, or serious adverse drug reactions (ADRs) or those with a long time of onset, which may not have been detected in pre-marketing trials or post-marketing surveillance studies. [3] The major limitation of this system is the high level of underreporting of ADRs, with an estimated median under-reporting rate across 37 studies of 94% (interquartile range 82–98%). [4, 5] This underreporting may hinder optimal ADR monitoring, may mask true side effect profiles, and extend exposure to possibly harmful drugs. [4, 6].

Despite the diverse and numerous interventions to decrease the level of underreporting by physicians and pharmacists, they have not been as successful as anticipated. [7] Patient reporting, especially in some European countries, has increased the number of ADR-reports (doubled the number in the Netherlands), has provided an additional source of information and has strengthened medication safety signals. [8, 9] The downside is that patient reports contain more subjective elements of information and less clinically related information. [10, 11] The involvement of specialist nurses may increase the number of high quality reports, because (specialist) nurses administer most drugs, they have prescribing privileges (specialist nurses), they are often aware of the occurrence of ADRs in their patients and they are an objective source of information. [5, 12,13,14].

Many studies have already demonstrated the importance of specialist nurses to pharmacovigilance. [13,14,15] This is especially true for medical specialties that use high risk medications and which have low reporting rates, such as hematology and oncology. [16] Our previous study involving specialist oncology nurses (SONs) also showed that, after an educational intervention, these specialist nurses were adequately prepared, had sufficient knowledge and adequate abilities, and had a positive attitude to monitoring and reporting ADRs. [17].

Although specialist nurses seem prepared for their role in pharmacovigilance, the rate of underreporting by nurses in general is especially high. For instance, in the Netherlands the reporting rate per 1000 nurse-years is 0.80. [18, 19] Multiple studies have investigated the reasons and determinants of underreporting [4, 5], which can be broadly classified in three groups: (i) ‘this is not part of my profession’ i.e. attitude about professional activities (financial incentives, legal aspects, and ambition to publish); (ii) ‘this is not part of my knowledge domain’ i.e. ADR-related knowledge and attitudes (complacency, insecurity, diffidence, indifference, and ignorance); and (iii) ‘this is not part of my interest’ i.e. excuses made by professionals (lethargy). Despite this knowledge, educational interventions for nurses in general have not been specifically designed to address these determinants.

A weak point of previous educational interventions is the use of outdated educational techniques and surrogate outcome measures (attitude and knowledge) to predict the effectiveness of an intervention. Our recent literature review showed an urgent need for more innovative learning initiatives in pharmacovigilance training [20] and a recent study stressed the use of clinical and real-life continuing learning initiatives. [21] It also advised studying the long-term effects and clinical value of educational initiatives. [20].

We chose to study SONs because they support current physicians and pharmacists and high risk medications are used in oncology and ADR reporting rates are low. [22, 23] Therefore, the aim of this study was to investigate the long-term clinical and educational effects of a real-life pharmacovigilance educational intervention for healthcare professionals, by analyzing the quantity and quality of their ADR reports and assessing their attitudes, skills, and knowledge of pharmacovigilance.

Methods

This longitudinal prospective cohort study to evaluate the long-term effects of real-life pharmacovigilance education was carried out in the three colleges that offer a course on prescribing for SONs in the Netherlands.

Setting

Three colleges (“The Amstel Academy”, “Radboudumc Health Academy”, and “Wenckebach Instituut UMCG”) offer registered SONs a nationally established course to enable them to qualify to prescribe a limited set of frequently prescribed drugs (anti-diarrhea drugs, anti-emetics, non-opioid analgesics, and benzodiazepines). SONs follow the course in addition to their work in different (mostly non-academic) hospitals in the Netherlands. The course consists of 4 days (6 h/day) of lectures and small group exercises spread over half a year, and is completed by a prescribing assessment. The national course overview is displayed in Fig. 1a.

Fig. 1
figure 1

Overview of the course curriculum and follow-up period

Full size image

The Amstel Academy, the college with the largest group size is the only college that also provided the pharmacovigilance intervention. The intervention consisted of a lecture on basic pharmacovigilance, a real-life reporting assignment of (unknown, exceptional, or unexpected) ADRs during work, and group discussion of the reported ADRs. The course was led by a pharmacotherapy teacher (MR & TS) and assessor from the Pharmacovigilance Center Lareb (R.v.E.). After the prescribing assignment, participants evaluated their experience by writing a short portfolio essay on their ADR report.

Population and inclusion

All SONs enrolled in the courses in the three colleges in the Netherlands from November 2015 – September 2017 were invited to participate in this study voluntarily. Participation included giving written or digital approval for an assessor from the Pharmacovigilance Centre Lareb (RvE) to search their database for submitted ADR reports (using email-address, name and institution). Additionally, SONs were asked to fill in an e-questionnaire at three different times after the course, to analyze changes in attitudes and knowledge after the prescribing qualification course. Inclusion was done by personal invitation by the local course coordinator, via e-mail.

Instruments

Three aspects of ADR reporting were evaluated: the number of reports submitted before and after the intervention; the quality and relevance of these reports; and the competence of SONs with regard to pharmacovigilance and ADR reporting (Fig. 1b).

The number of ADR reports submitted was measured by extracting all reports submitted by the participants from the Lareb database up to 2 years after course completion. This was done by drafting a query based on their email address and hospital name, from 2015 to March 2019. All retrieved reports were checked manually.

The ClinDoc algorithm [24] was used to determine the quality of the clinical documentation of each ADR report. Two researchers (MR and RVE) individually assessed the first five reports, to check whether they scored reports in a similar way (all reports had the same score). As this was the case, the other reports were divided among the two researchers. Two reports were difficult to score and were discussed before agreement was reached on the score. Aspects that indicate relevance of the ADR report were scored automatically (seriousness) or manually, using the report form or pharmacovigilance center assessment information (labeling information, off-label use, additional monitoring and withdrawal of chemotherapeutic treatment).

The competence of the SONs was evaluated with three e-questionnaires (Supplementary Table 1), using Castor EDC. The e-questionnaires consisted of three parts – baseline characteristics, intention/attitudes, and knowledge/skills of the SONs (16questions). The first e-questionnaire was sent within 2 weeks after the prescribing assessment and the second and third at 1-year intervals. If participants did not respond, two reminders were sent at a 2-weekly intervals.

Table 1 Baseline characteristics of specialist oncology nurses
Full size table

Data analysis

All data were imported in SPSS Statistics 22 (IBM Corp.; Armonk, New York). Descriptive statistics were used to report frequencies and means/median/standard deviations (SD) and interquartile range (IQR) of survey results. Differences between baseline characteristics were compared using Chi-squared and Mann–Whitney U test (alpha 0.5 and p < 0.05). Attitude and knowledge scores between intervention and control groups of SONs were computed with.

Mann–Whitney U test, with Bonferroni’s correction (p < 0.0025). Reported ADRs were categorized in two groups: 1. Reports submitted during the intervention and 2. Reports submitted after the educational intervention. The prescribing assessment was used as cut-off date. Reporting rates per 1000 nurse years in the study (intervention group) and the national average were compared by dividing the ADR-reporting rates (according to the Pharmacovigilance Center Lareb) [18, 19] by the number of BIG-registered nurses (including specialist nurses) [18]. Open questions were analyzed using thematic analysis [25].

Ethical aspects

This study did not fall under the scope of the Dutch Medical Research Involving Human Subjects Act. (reference number 2017.034). Participation was voluntary and based on informed consent. Participants did not receive credit or other incentives to participate. The ethics review board of the Netherlands Association for Medical Education (NVMO) reviewed the protocol and approved this study (ID: 692 & 826).

Results

All SONs enrolled in the three prescribing qualification courses in the Netherlands (n = 113) were invited to participate in this study, 88 (78%) of whom signed the informed consent form (65 in the intervention group and 23 in the control group). All included SONs filled in the first questionnaire (T0), 73 (83%) filled in the second questionnaire (T1), and 62 (70%) filled in all three questionnaires (T2).

At baseline, no significant differences were found in the characteristics of the SONs in the intervention and control groups, except for a sex difference (more female SONs in the control group). Overall, 76 SONs were women, the median age was 45–55 years, and SONs had a median of 14 years of clinical experience as a nurse. Most SONs worked on an oncology and hematology ward (68%) in a non-academic hospital (90%). While 49 SONs (56%) reported having had a lecture on ADRs before the prescribing qualification course, only 7 SONs (8%) indicated that their previous training covered when or how to report an ADR. Before starting the course, 6 SONs (5%) had reported one or more ADRs to the Netherlands Pharmacovigilance Centre Lareb (Table 1).

Clinical outcomes

ADR reporting (number)

During the study period, 82 ADR reports (concerning 220 different ADRs) were reported to Lareb. All intervention-group SONs submitted at least one ADR report during and after the course, whereas none of the control-group SONs did. Five participants submitted more than one ADR report during the educational intervention. Seven SONs (11%) submitted one or more ADR reports after the intervention, accounting for 11 additional reports (Table 2a).

Table 2 ADR reports characteristics
Full size table

ADR reporting (quality)

Since only the intervention-group reported ADRs, the reporting quality of the intervention and control groups could not be compared. ClinDoc scores higher than 75% indicate a well-documented ADR report; 69 of the 82 (84%) ADR reports had a score higher than 75% (Table 2b / Supplementary Fig. 1). The scores for the reports submitted during and after the intervention were not significantly different (p = 0.227).

ADR characteristics

Most of the ADR reports concerned monoclonal antibodies (17%), pyrimidine analogues (17%) and platinum compounds (14%) and associated skin reactions (19%), gastrointestinal disorders (15%), and nervous system disorders (14%) (Table 2c). Thirty-five (43%) reports met criteria for seriousness (CIOMS), with hospitalization being mentioned in 24 reports (Table 2c). The main serious reactions reported were peripheral edema, dizziness, diarrhea, and dehydration. More serious conditions were acute myocardial infarction (n = 2), pneumonitis (n = 2), toxic skin eruption (possible Stevens-Johnson syndrome) (n = 1), anaphylactic reaction (n = 2). Four fatal reactions occurred and involved ascites, cardiogenic shock, hypovolemia due to diarrhea and gastrointestinal hemorrhage, and pulmonary embolism. Although most (70%) of the reported reactions were mentioned in 4.8 section (Side effects) of the Summary of Product Characteristics (SmPC) 30% of the reported reactions were unexpected.

Educational outcomes

Intentions and attitudes

The intervention-group SONs had significantly higher scores for intention to report unknown and serious ADRs than the control group SONs up to 2 years after the prescribing qualification (Table 3). All SONs thought that reporting ADRs would “contribute to medication safety”, “improve patient safety”, and would “educate others about drug risks”; however, the intervention-group had a more positive attitude to these outcomes than the control group (p < 0.05). The intervention-group SONs were significantly more likely to consider that reporting “disrupts the normal workflow” and “is time consuming” at all-time points (p < 0.05). All SONs agreed that physicians and pharmacists are important healthcare professionals with regard to reporting ADRs, but only the intervention-group SONs considered themselves to be an important healthcare professional for reporting ADRs (p < 0.05).

Table 3 Intention, attitudes, and opinions regarding the reporting of adverse drug reactions
Full size table

Knowledge (quantitative)

The intervention-group SONs had significantly (p < 0.05) better scores on most (8/11) dichotomous knowledge questions directly after course completion compared with the control group. Three questions regarding “anonymous reporting”, “adverse experiences with cosmetics”, and “natural and homeopathic products” were answered correctly by only 50% of the SONs in both groups. After 2 years, the intervention group still significantly outperformed the control group on most questions (6/11) (Supplementary Table 2).

Skills (qualitative)

Analysis of the open-ended questions showed that the intervention-group SONs were more aware of “what to do when they suspect an ADR”. They would significantly more often perform “causality assessments”, would try to reduce symptoms by “stopping the suspected drug”, “search for alternative drug”, “lower dose”, or even perform a “re- or dechallenge”. They also spontaneously mentioned “reporting the ADR” more often than did the control-group SONs (Supplementary Fig. 2).

When asked what essential information is needed for a qualitatively good report, the intervention-group SONs reported more options. The control-group SONs primarily mentioned “description of the adverse drug reaction”, “suspected drug” and “co-medication”, whereas the intervention-group SONs also mentioned “start-stop dates”, “latency time”, “re- and dechallenge information” and would supply additional information and laboratory data (Supplementary Fig. 3).

Discussion

SONs who received the context-based pharmacovigilance educational intervention submitted significantly more, qualitatively good ADR reports than did SONs who did not receive the intervention in the first 2 years after they completed a prescribing qualification course. The latter group did not submit any ADR reports. The SONs who received the intervention considered themselves more ready for their role in pharmacovigilance, had sustainable and more positive attitudes, and had a better knowledge of pharmacovigilance and ADR reporting up to 2 years after course completion. Additionally, their ADR reports provided valuable, relevant, and well-documented information in a specialized field with a low reporting rate.

SONs who received pharmacovigilance training had a reporting rate of 84.6 per 1000 nurse-years in the first two years after the intervention. This made the SONS 105 times more likely to report an ADR after the prescribing course than the national reporting average for nurses in the Netherlands [18, 19]. This major increase should be interpreted with caution since reporting rates among nurses in the Netherlands are lower than in other European countries and the specialist oncology nurses in our study would be expected to be more aware of ADRs and report more frequently, because they have prescribing authority.

Our results were similar to those of other educational interventions in nurses. Bäckström et al. [26] reported a large (tenfold) increase in ADR reporting rates among nurses who had followed repeated educational lectures. Other interventions for non-nurse healthcare professionals (physicians/pharmacists) had a similar effect on ADR reporting rates (increase of 2- to sixfold). [27,28,29,30] These educational interventions are typically short (1 h) lectures or workshops, with follow-up reminders, educational materials or emails, which makes them more difficult to implement.

Despite our clinically relevant increase in the number of ADR reports, we did not observe a further growth during the follow-up period, 11% of SONs submitted an ADR report in the first year after the intervention but only 6% in the second year. Previous intervention studies also reported that under-reporting increased with increasing time after an intervention [7, 25,26,27,28]

Hazell et al. [4] and Lopez-Gonzalez et al. [5] identified three distinct groups of determinants of underreporting (Supplementary Table 3). Our real-life pharmacovigilance intervention improved “attitudes relating to professional activities” and “increased ADR-related knowledge and attitudes” (the first two determinants). Our intervention did not address the third determinant: “excuses made by professionals”. These excuses include: “I don’t have time”, “I have different care priorities”, and “reporting is bureaucratized”, and are factors not easily addressed in educational interventions. Therefore, we strongly recommend the development of guidelines and national/international regulations for reporting ADRs. If medical directors or policy makers put more emphasis on the importance of reporting ADRs, healthcare professionals will eventually change their care priorities and time will be made available for reporting ADRs during work. Many initiatives, such as making reporting forms more accessible [34], reporting with apps or discharge letters and Adverse Drug Event Manager (ADEMs) [35] teams, have made ADR reporting more effective; however, these initiatives have not succeeded in changing healthcare professionals’ views on pharmacovigilance and ADR reporting.

The strength of this study lies in its design, a prospective cohort study with a 2-year follow-up for clinical and educational outcomes. Moreover, the participants formed a homogenous group as they all had a similar educational background and had followed a nationally established prescribing course. In total, 78% of the entire population of SONs in the Netherlands with a prescribing qualification were included. Lastly, the dropout rate was low. No SON withdrew their consent to search for reported ADRs in the National Pharmacovigilance Database and e-questionnaire response rates were 100% (directly after prescribing assessment), and 84% and 73% at 1 and 2 years of follow-up, respectively.

The main limitation of this study is the relatively small control group of 23 SONs. Since the included SONs were probably the ones most interested in pharmacovigilance, the results of this group could therefore be an overestimation of the practice, knowledge, and attitude to ADR reporting of SONS in general. A second limitation could be caused by the educational intervention itself. The intervention consisted of a real ADR report, analysis, class discussion and a portfolio assignment which took much longer than a normal ADR report. The extensiveness of the intervention could have influenced the reporting rates by conditioning SONs to expect ADR reporting to be time consuming. This is borne out by the finding that the intervention-group SONs had significantly higher scores on questions about the time required (“time consuming to report” and “disrupts the normal workflow”) (Table 3). A second study where SONs perform a short (and quick) ADR reporting assignment could distinguish between the two reasons.

Taking these limitations into account, this study is the first to show a significant and clinically relevant increase in the quantity and quality of ADRs reported after a single educational intervention. Although we studied SONs, we hypothesize that training other healthcare professionals will improve the level of ADR reporting. Therefore, we would recommend that other healthcare curricula (medicine, pharmacy, or nursing) incorporate this simple real-life, problem- and context-based pharmacovigilance educational intervention as it fits perfectly into the proposed curriculum. [21] Further research is needed to analyze whether a shorter version of the reporting assignment will further encourage ADR reporting. Although our findings are encouraging, reporting rates were limited by “Excuses made by professionals”, an aspect not adequately addressed in our educational intervention. This emphasizes the importance of increasing the time available in medical curricula to teach medication safety and pharmacovigilance, so as to increase medication awareness. We recommend guidelines and national/international regulations in clinical practice to maximize reporting rates.