Smart pumps improve medication safety but increase alert burden in neonatal care
- 192 Downloads
Smart pumps have been widely adopted but there is limited evidence to understand and support their use in pediatric populations. Our objective was to assess whether smart pumps are effective at reducing medication errors in the neonatal population and determine whether they are a source of alert burden and alert fatigue in an intensive care environment.
Using smart pump records, over 370,000 infusion starts for continuously infused medications used in neonates and infants hospitalized in a level IV NICU from 2014 to 2016 were evaluated. Attempts to exceed preset soft and hard maximum limits, percent variance from those limits, and pump alert frequency, patterns and salience were evaluated.
Smart pumps prevented 160 attempts to exceed the hard maximum limit for doses that were as high as 7–29 times the maximum dose and resulted in the reprogramming or cancellation of 2093 infusions after soft maximum alerts. While the overall alert burden from smart pumps for continuous infusions was not high, alerts clustered around specific patients and medications, and a small portion (17%) of infusions generated the majority of alerts. Soft maximum alerts were often overridden (79%), consistent with low alert salience.
Smart pumps have the ability to improve neonatal medication safety when compliance with dose error reducing software is high. Numerous attempts to administer high doses were intercepted by dosing alerts. Clustered alerts may generate a high alert burden and limit safety benefit by desensitizing providers to alerts. Future efforts should address ways to improve alert salience.
KeywordsPatient safety Medication administration errors Smart infusion pumps Infusion pump alerts Alert fatigue
Dose error reduction software
Intensive care unit
Neonatal intensive care unit
Medication errors are common in pediatrics, and errors with the potential to cause harm are significantly increased in neonates cared for in intensive care environments . Younger and critically ill patients are also more likely to suffer harm from a medication error [1, 2]. Neonates are a particularly vulnerable population, due to many factors, including their physiologic immaturity and rapidly changing weights that affect weight-based medication dosing [1, 3]. For this reason and others, hospitals have quickly and widely adopted technologies that have the potential to reduce medication errors in the neonatal intensive care unit (NICU).
Smart pumps are a promising technology to prevent medication administration errors. Smart pumps are infusion pumps manufactured with software that checks the nurse-programmed medication administration against pre-established institutional limits in customized medication libraries before beginning infusion. Because administration is the last step in the medication use pathway after ordering and dispensing, there are few opportunities to intercept errors in administration. As such, nurse administration errors are more likely to reach the patient than physician ordering or pharmacy dispensing errors . Smart pumps provide alerts around many types of potentially unsafe infusion conditions, providing both ‘soft alerts’ and ‘hard stops’ when programmed dosing limits are violated. The goal of soft alerts is to raise awareness by notifying the user that the programmed medication infusion parameters are outside of the usual dosing range. These alerts can be overridden, however, and the infusion continued. Hard stops notify the user that the medication infusion parameters are outside of safe limits and force cancellation of the infusion. The American Society of Health-System Pharmacists national survey of 2014 suggested that 80.5% of surveyed hospitals in the US use smart pumps, up from 32.2% in 2005, demonstrating the rise in use of smart pumps [5, 6]. Despite rising rates of adoption, little data exists on the impact of smart pump alerts on medication safety and alert fatigue in the pediatric and neonatal populations.
Some studies have raised the concern that smart pumps may actually introduce error themselves and require a high rate of compliance to observe safety benefits [7, 8, 9]. As with all technology, the implementation of smart pumps may come with unintended consequences, and organizational structure, the technology itself or patient needs may drive the development of workarounds [10, 11]. Issues that have been raised in the literature include workarounds like the use of basic infusions that do not employ the drug library dosing limits, improper patient or medication identification during pump programming, and high override rates for soft alerts, all of which negate the safety features of smart pumps [8, 12]. The use of an additional alerting system in the ICU environment also raises the concern of alert fatigue and the salience of smart pump alerts, due to suboptimal levels of actionable alerts . As a result of alert fatigue from cognitive overload or densitization with time, responsiveness to smart pump alerts may be influenced by other factors, such as work shift, month, repetitiveness of alerts or work complexity [14, 15]. The goal of this study was to assess whether smart pumps prevent medication errors for continuous intravenous medications used in the neonatal population and to assess whether smart pumps are a source of alert burden in the neonatal intensive care unit (NICU). The immediate goal of this study was to characterize the data that smart pumps generate, describe how it changes over time, and understand whether contextual factors, such as work shift, influence the data. We chose to focus our work on neonates as one of the most vulnerable populations but anticipate broader pediatric applicability.
Study site and patient population
The study took place in a level four NICU that provides the highest level of neonatal intensive care to complex and critically ill newborns requiring extracorporeal mechanical oxygenation (ECMO), surgical and subspecialty care. At the time of the study, the NICU had 59 registered beds and an average of 725 yearly admissions. Patients had an average gestational age of 35 weeks and an average length of stay of 23 days.
NICU safety technologies in place at the time of the study include the use of computerized provider order entry with electronic medication administration records and clinical decision support, a barcode medication administration system, smart infusion pump technology with a customized neonatal library of 158 medications, daily rounding and prescription review by dedicated NICU pharmacists and clinical guidelines for high-risk medications.
Alaris™ (Becton, Dickinson, and Company, Franklin Lakes, NJ) intravenous infusion smart pumps with dose error reduction software (DERS) were introduced in January of 2014. Data from pumps was wirelessly downloaded daily and distributed for analysis monthly. Pump reports included the pump identification number (Point-of-Care Unit and module), infusion number, patient medical record number, drug name, drug concentration and rate, alerts, alarms and actions, and date and time stamps for all actions. Each action or change the infusion pump undergoes is captured in a smart pump record (SPR). The NICU medication library contained in the smart pumps was customized in 2013 through a collaborative effort with neonatologists, pharmacists, and neonatal nurse practitioners. Smart pump libraries and medication issues were reviewed monthly by a multidisciplinary medication use team to assess medication additions and adequacy of dosing limits. Changes to the library have been tracked since 2014 with updates occurring on average 4–5 times/year. Sixty percent of medications and fluids in the library have hard maximum limits, and this has been consistent over time.
Study design/methodological approach
We evaluated retrospective smart pump data for all neonatal patients from January 2014–December 2016, which included 727, 745 and 655 patients by year. We focused our evaluation on continuous medications since these medications typically represent high-risk medications and have been the emphasis of our previous area of study. The data presented are limited to smart pump data on continuous medications and their associated medication alerts. We sought to understand the baseline data generated by smart pumps in the NICU and to understand how dynamic the data were over the study period through use of descriptive statistics and trend analysis. Patterns in the data were examined to determine the dynamic nature of the system and the reliability of the system, the quality of the data, alert burden, response to alerts, and contextual effects, such as shift or day of the week, on the data.
Infusion Frequency: number of unique infusions and number of infusions started
Alert Frequency: number of alerts and percentage of alerts per infusions started
Compliance with Drug Library: percent compliance of clinician programming the pump with drug library utilization
User Response to Pump Alerts: actions by percentage and number
Alert Burden by Time: alert number and percentage by shift, day, week/weekend, and month.
Alert Burden by Medication: alert number and percentage by medication
Salience of Alerts: We calculated alert salience as one measure of the effectiveness of alerts as it measures the extent to which a provider takes corrective action in response to the presented alert. Alert salience has been used to demonstrate provider response to dosing alerts issued during medication ordering and is suggested to be a proxy for alert fatigue [16, 17]. We calculated alert salience as the number of cancelled or reprogrammed infusions/total number of alerts. A low alert salience indicates that a high number of alerts are being generated but do not result in significant action.
Alert Burden by Library Threshold Violation: number of ‘soft maximum’ and ‘hard maximum’ violations by year and user response by type of violation, as well as number of ‘soft minimum’ and ‘hard minimum’ violations.
Percent variance from soft and hard maximum thresholds: percent variance over soft and hard maximum thresholds
Unique and Total Infusion Starts
Each new bag or syringe for a medication is given a unique infusion ID. Given that a unique infusion may be started and stopped multiple times, as a result of alarms and alerts, the total number of infusion starts will be greater than the number of unique infusions. We considered each infusion start an opportunity to evaluate the pump.
Provider cancels the pump programming after an alert and starts over. Cancellation is forced by the pump when a hard limit is exceeded.
Provider resets the pump programming for medication dose or rate after an alert.
Provider receives an alert and chooses to proceed without changes in pump programming.
Pump programming mode that requires all information to be manually entered and which bypasses all safety features of the pump except volume to be infused.
Dose Error Reduction Software
Software present in the pump that allows the creation of a population-specific drug library with established minimum and maximum dose and/or rate limits by medication.
System stability and drug library compliance
Infusion and Alert Data
All Years Average
# of Infusion Starts
# of unique infusion IDs
# of Infusion Starts Using DERS
# of Infusion Starts Using Basic Mode
% Infusion Starts with Alerts
Response to Alerts
% of Alerts by Shift
% of Infusion Starts with Alerts by shift
Alerts, response to alerts and overall alert burden
Alerts occurred in an average of 4.3% of all infusion starts (range 2.8–5.4% across the three years) with 15,720 alerts for 264,470 unique infusions. There was a small but statistically significant change in the percentage of infusion starts with alerts that occurred each year (p < 0.0001; Table 1). An average of 2.5% of unique infusions had alerts (range 2.0–2.9% across three years, p < 0.0001).
The majority of alerts were overridden with an average of 73.6% of alerts overridden (range 71.1–76.4%), as shown in Table 2. An average of 23.3% (21.5–25.7%) of alerts were cancelled and 3.1% (2.2–4.8%) reprogrammed. The responses to alerts showed a small statistical change over time by year, with more alerts being overridden and fewer being cancelled or reprogrammed in 2016 when compared to 2014 (p < 0.0001).
To understand the effect of context on alerts, we evaluated the effect of work shift, day and month. There was a statistically significant (p < .0001) difference regarding the overall number and percent of alerts by shift across the three years, with a higher proportion of alerts occurring on evening shift. However, the percent of infusion starts with alerts did not differ by shift. (P = 0.17; Table 2) The percentages of infusions with alerts also did not vary significantly by day of the week, or weekday versus weekend (p = 0.38; not shown).
Alert burden by medication
We evaluated medications that generated the highest number of alerts per infusion starts as a reflection of their dosing rules. Fentanyl, insulin, vasopressin, morphine, TPN and midazolam generated the highest number of alerts per infusion starts (Fig. 2a) and demonstrated a large number of infusions with both one alert and multiple alerts (Fig. 2b). There was some variation in the percent of medication infusions that generated alerts by year, but the top 6 medications remained essentially the same during the time period studied (Fig. 2a). Override rates remained high (63.1–84.8%) for medications with the highest alert rates and similar to all medications with an average override rate at 74.7% (data not shown).
Drug library threshold violations
Drug Library Threshold Violations by Type, Year, and User Response
Percent variance from soft and hard maximum limits
For neonates, who are at particularly high risk for overdose infusion errors, smart pumps have the potential to be particularly valuable. However, this value is only realized when the pumps and libraries are tailored to use with neonates and the equipment and software are used as intended. Because medication administration errors are more likely to reach the patient, smart pumps have the potential to provide additional protection by double-checking nurse programming. The goal of our study was to assess whether smart pumps are able to prevent medication overdoses in continuous medications administered to the neonatal population. Overall, we found that nurses took advantage of the safety features of smart pumps and programmed the majority of infusions using the drug library with dose error reduction software. This high rate of compliance has been shown to be critical to the effective use of smart pumps [8, 18, 19]. Although our rate of compliance is higher than previously reported rates [12, 13], basic infusions, which lack the safety features of the DERS, were still used for 11–16% of infusions. Targeted discussions with frontline smart pump users revealed that our library had limited options for intravenous fluid selection that did not match the large variety of fluids being used. Work is underway to expand fluid options in our neonatal library, and our experience supports the need for regular reassessment to understand the reasons for basic infusion use in all institutions.
Along with the use of basic infusions, other workarounds that generated alerts were evident in our data. When evaluating hard stops, we found that nurses programmed the pump rate at 2–4 times the dose to facilitate rapid pump priming in 15–30 min instead of one hour. Discussion with the nurses revealed the need for efficiency in the face of urgent patient needs. Smart pumps are not perfect, and workarounds occur when there is a mismatch between the capabilities of a technology and demands of care [20, 21]. Our findings are consistent with the causes of workarounds identified by others, which include patient issues, the need for efficiency, technology-related issues, and organizational issues where policies or libraries do not align with clinical practice [20, 21, 22].
Previous studies have used the number of infusions that are reprogrammed or cancelled to assess prevented errors. We found that there were many attempts to exceed the soft maximum limits, less frequent attempts to exceed the hard maximum limits, and that many infusions were cancelled or reprogrammed. Looking specifically at attempts to exceed hard maximums, which we used as a proxy for potentially prevented errors, we found that smart pumps identified and prevented significant overdoses of several high-risk medications, including narcotics, sedatives and insulin, demonstrating their utility in improving patient safety in the NICU. While some hard stops may be contributed to attempts at pump priming, we found examples of incorrect programmed doses, decimal point errors, confusion between two simultaneous medications, and attempts to bolus using the continuous infusion setting. Although these events were infrequent compared to the number of overall infusion starts, the possibility of delivering highly dangerous doses was prevented through the use of the dose error reduction software. This rate of potential error prevention by smart pumps is in line with that reported in other ICU environments . For soft alerts, given that the typical response is to override the alert, cancelling or reprogramming of the infusion is more likely to represent a true prevented error. We found 2093 examples of cancelled or reprogrammed infusions after soft alerts.
The second goal of our study was to assess whether smart pumps are a source of alert burden and potential alert fatigue in the NICU. When looking across the unit as a whole, our data suggest that smart pump alerts for continuous infusions do not contribute significantly to overall alert burden with only 3–5% of infusions alerting per infusion start, which is significantly less than the 12–16 alarms/hour nurses experience from physiologic monitors in the NICU [24, 25]. While there was a statistically significant change in the percentage of alerts per year, it is unlikely that this represents a clinically significant change given the relatively low frequency of infusion-related alerts. When we evaluate alerts from the perspective of the patient, however, we found that alerts cluster around specific patients and specific infusions, with 17% of infusions generating significant alert numbers. In these unique and interesting situations, we found that a single medication in a single patient could repetitively alert, creating a very high alert burden and potentially making it difficult for a caregiver to identify and act upon appropriate alerts representing true errors. The evidence on alert fatigue shows that fatigue is connected to the proportion of repeated alerts , potentially creating unique situations where individual patients are placed at increased risk for experiencing missed alerts or delayed responses. In our data, repetitive alerts occurred during the initial start of medications and clustered around patients that required doses of medication that were significantly higher than normal, most commonly at the end of life, when sedatives and analgesics had been significantly escalated. There are patients that will be exceptions to the standards of the library based on their individual physiology or during end-of-life care. Current smart pumps do not have a way to account for these individual situations. While smart pumps may be working appropriately to raise awareness of a dose being given outside of the usual range, repeated alerts are a new source of alert fatigue at the patient level; this may decrease nurse sensitivity to the alert, placing the patient at especially high risk for a nurse to miss a true error. Future work should focus on ways to mitigate this issue, identifying ways to individualize smart pump performance.
Given the opportunity, the vast majority of nurses override alerts, calling into question the ability to appropriately recognize alerts that require attention. This parallels the observed phenomenon seen with medication alerts produced during the prescribing phase of the medication lifecycle [26, 27]. Of concern, many high-risk medications that have been associated with harmful errors are generating a high number of smart pump alerts. The medications responsible for a high alert burden have high individual alert rates per infusion starts, suggesting that their dosing rules may not be optimized. Additionally, soft minimum alerts, which have high override rates  and low potential to reduce harm, contributed significantly to the alert burden in our study. Ideally, the best alerts identify true errors and require action.
In looking at the amount that programmed doses exceed limits, we found that several medications that generated high alert rates, such as morphine, alerted for doses that were just 20% above the soft dosing limit. This may indicate that adjustment of the dosing limits in the drug library could significantly decrease alert burden while continuing to promote patient safety by alerting for more significant overdoses. Several authors have demonstrated the utility of small dosing limit adjustments to decrease the number of nonactionable alerts, but more evidence is needed to understand the effect of changing dosing limits on patient safety [13, 28, 29].
Others have shown that alert number differs by time of day, day of the week, and month . We found that overall alert number is highest on the evening shift but we contribute this to the fact that the majority of new fluids, TPN and lipids are hung during this shift. The overall percentage of alerts per infusion starts was not different based on shift or day of the week. Our data also suggest that nurses may experience some desensitization to infusion pump alerts over time. In desensitization, repeated exposure to alerts leads to declining responsiveness over time [14, 30, 31, 32]. We found a small but statistically significant increase in the number of alerts that were overridden and a decrease in the number of alerts that were reprogrammed or cancelled over time, suggesting that nurses may experience some densitization with continued infusion pump experience. No significant changes to the pump library limits or to hospital policy occurred during that time to otherwise account for the change. Given the small rate of change, further evaluation over time is required to determine if this is a true phenomenon.
Our study has a number of strengths. We took advantage of wirelessly downloaded smart pump information to build a large neonatal database evaluating over 370,000 infusions over multiple years. The granularity of this large dataset allowed us to identify unique trends in the data to understand some of the benefits and inherent risks of smart pumps. The current literature on medication administration errors in neonates is based primarily on small observational studies. This study is the first neonatal study to use non-observational methodology and a large dataset to assess the effect of smart pumps on neonatal safety.
There are limits to this approach as well. First, by utilizing a data approach, we lack the ability to assess human factors and contextual factors, such as patient and unit acuity, or workload complexity, which may influence the caregiver response to alerts and can affect how data is interpreted. While we met with frontline smart pump users to understand data trends and workarounds, we did not perform intensive observations or structured interviews. Second, there is not a one-to-one correlation between orders and infusions, which makes it impossible to ascertain the number of expected infusions to confirm complete data capture. Third, our study was limited to a single intensive care unit in a single site, which may limit generalizability to other environments and populations. Finally, we chose to focus on continuous medications, and the issues around dosing limits, alerts and alert fatigue may differ for intermittent medications compared to continuous medications.
Given the large number of infusions utilized in neonates, and essentially in all hospitalized patients, it is clear that measures that improve medication and fluid safety can have a significant impact. What appears to be a small number of prevented errors becomes large when analyzed at the level of the institution or hospital system. To further optimize the effectiveness of smart pumps, future work should evaluate the effect of smart pump alert burden on an individual caregiver level as individual alert fatigue or desensitization may contribute to medication errors.
Smart pumps, when used appropriately, have the ability to improve neonatal medication safety in a small but important way. However, this benefit must be balanced with the effects of alert burden and alert fatigue, as individual medications or individual patients with a high alert burden may limit safety benefit through provider desensitization to alerts. Future work should address ways to improve alert salience, especially when alert rates are high, and identify ways to target smart pump performance when individualization is required, particularly to decrease repeated nonactionable alerts. Regular assessment of smart pump data to assess violations of limits, current library limits, alert frequency, provider actions, and use of basic infusions will help improve the usefulness of smart pump technology and the ability to prevent medication administration errors in our most vulnerable patients.
KM and EK conceived the study; KT acquired and analyzed the data; KM, KT, KW, JMD and EK analyzed and interpreted the data; JMD performed the statistical analysis; KM and EK drafted the manuscript and all authors revised and approved the submitted version.
This work was supported by the National Institutes of Health grant number 1R01LM012230, which had no role in the study design, collection, analysis and interpretation of data, or in the writing of the manuscript.
Ethics approval and consent to participate
This work was evaluated by the Cincinnati Children’s Hospital Medical Center Institutional Review Board and was determined to not be human subject research. The IRB approved access to the data and confirmed that no formal ethical approval or consent was needed.
Consent for publication
The authors declare that they have no competing interests.
- 4.Leape, L.L., et al., Systems analysis of adverse drug events. ADE Prevention Study Group. JAMA, 1995. 274(1): p. 35–43.Google Scholar
- 11.Coiera E, Ash J, Berg M. The unintended consequences of health information technology revisited. Yearb Med Inform. 2016;1:163–9.Google Scholar
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.