Implementation study of a 5-component pediatric early warning system (PEWS) in an emergency department in British Columbia, Canada, to inform provincial scale up
The rapid identification of deterioration in the pediatric population is complex, particularly in the emergency department (ED). A comprehensive multi-faceted Pediatric Early Warning System (PEWS) might maximize early recognition of clinical deterioration and provide a structured process for the reassessment and escalation of care. The objective of the study was to evaluate the implementation fidelity, effectiveness, and utility of a 5-component PEWS implemented in the ED of an urban public general hospital in British Columbia, Canada, and to guide provincial scale up.
We used a before-and-after design to evaluate the implementation fidelity, effectiveness, and utility of a 5-component PEWS (pediatric assessment flowsheet, PEWS score, situational awareness, escalation aid, and communication framework). Sources of data included patient medical records, surveys of direct care staff, and key-informant interviews. Data were analyzed using mixed-methods approaches.
The majority of medical records had documented PEWS scores at triage (80%) and first bedside assessment (81%), indicating that the intervention was implemented with high fidelity. The intervention was effective in increasing vital signs documentation, both at first beside assessment (84% increase) and throughout the ED stay (> 100% increase), in improving staff’s self-perceived knowledge and confidence in providing pediatric care, and self-reported communication between staff. Satisfaction levels were high with the PEWS scoring system, flowsheet, escalation aid, and to a lesser extent with the situational awareness tool and communication framework. Reasons for dissatisfaction included increased paperwork and incidence of false-positives. Overall, the majority of providers indicated that implementation of PEWS and completing a PEWS score at triage alongside the Canadian Triage and Acuity Scale (CTAS) added value to pediatric care in the ED. Results also suggest that the intervention is aligned with current practice in the ED.
Our study shows that high-fidelity implementation of PEWS in the ED is feasible. We also show that a multi-component PEWS can be effective in improving pediatric care and be well-accepted by staff. Results and lessons learned from this pilot study are being used to scale up implementation of PEWS in ED settings across the province of British Columbia.
KeywordsPediatric early warning system Pediatric early warning score PEWS Emergency department Pediatrics Implementation study Mixed methods
Canadian Triage and Acuity Scale
Pediatric Early Warning System
Situation, Background, Assessment, Recommendation
Globally, failure to identify and intervene with pediatric patients experiencing clinical deterioration is a source of unintended harm including prolonged hospital stays or readmission, disability and death [1, 2, 3]. Care inequities can arise due to diversity of provider experience and knowledge, and lack of standardized approaches in the management of critically ill children .
To maximize early recognition of clinical deterioration in children, Pediatric Early Warning Systems (PEWS) have been implemented internationally with a growing body of evidence showing benefit of their use [4, 5, 6]. PEWS should support healthcare providers in identifying abnormal physiology, tracking trends across time and supporting structured processes for reassessment and escalation of care.
In emergency department (ED) settings, the ability to identify deterioration or risk “in the moment” – without tracking over time – is critical ; however, rapid identification of deterioration in the pediatric population has several complexities. These difficulties include: varying normal vital signs parameters by age [8, 9], changes in physiologic parameters related to factors beyond the illness or injury (medication, pain, fear and anxiety) , and compensation seen in pediatric patients when critically ill, underrepresenting the degree of deterioration [7, 8]. Using PEWS in the ED could offer benefit in the focus and guidance provided on age-appropriate pediatric norms, measuring physiologic parameters and acting on evidence of abnormality [7, 11, 12].
While implementation of PEWS in EDs is becoming increasingly common globally, with mounting evidence for this application [8, 10, 11], there is still much to learn. Most studies of PEWS in ED to date focus on the predictive value of a score in determining level of care at disposition, with the overarching conclusion being that a PEWS score is reasonable at predicting admission to the intensive care unit (ICU), but has limited value in predicting admission to a non-ICU bed [10, 11, 13]. No study has yet examined PEWS as a complex healthcare intervention . There has been minimal research focusing on the general utility of PEWS for the ED , yet utility is particularly important for ED as it is a dynamic environment with undifferentiated patients often seen for a relatively limited time [4, 10]. The operational characteristics of busy EDs include intermittent high patient loads, diagnostic uncertainty, multiple handovers, intense time pressures and diverse skills owing to high staffing levels and rapid turnover [7, 13]. Researchers, such as Roland et al. , built a convincing case for PEWS in ED as a means of minimizing cognitive factors and biases that impact decision making.
In British Columbia, one of the ten provinces of Canada, PEWS had not been used in ED settings prior to this study. In 2014, a provincial working group consisting of members of regional Health Authorities, academia, and healthcare providers came to a consensus that PEWS for inpatient settings should be pursued, not only to improve health outcomes but also to standardize assessment and care of children, facilitate cross-site communication processes and enhance patient transfer across the varied hospitals in the province. However, testing the effectiveness and utility of PEWS in ED settings was considered to be necessary before a provincial roll out, given a scarcity of scientific evidence.
The objective of the study was to evaluate the implementation fidelity, effectiveness, and utility of a 5-component PEWS implemented in the ED of an urban public general hospital in British Columbia (BC), Canada. This article will outline the planning, implementation, and results of this pilot study and discuss how the results are being used for provincial scale up.
The study setting was the ED of Richmond Hospital – an urban public general hospital located in British Columbia, Canada, with an average of 6800 pediatric visits per year; approximately 12% of total ED visits. No PEWS was in place before this study and documentation occurred on a non-pediatric specific emergency nursing assessment record. Richmond Hospital uses the Canadian Triage and Acuity Scale (CTAS)  to triage patients by severity and type of presenting symptoms and signs. The scale consists of five levels from 1 (resuscitation) to 5 (non-urgent).
Description and source of the five components that constitute the BC Pediatric Early Warning System (BC PEWS)
Pediatric assessment flowsheet
The double-sided flowsheet, designed for inpatients, comprehensively outlines documentation for 24 h of nursing assessment, including PEWS scoring parameters, full head-to-toe assessment and documentation of routine nursing care.
Adapted from BC Children’s Hospital
The flow sheets are available in six age grouping (0–3 months; 4–11 months; 1–3 years; 4–6 years; 7–11 years and 12+ years) to account for naturally-occurring variations vital signs norms . Staff were also provided with vital signs reference cards that could be worn on a lanyard.
The Brighton PEWS score embedded in the flowsheet is the most widely used and validated PEWS score available for inpatient care. It is a 13-point score (with 0 normal and 13 high risk) based on behavioural, cardiovascular and respiratory parameters.
Brighton PEWS score 
As the Brighton scoring tool is not age specific, vital signs references for PEWS scoring were based on the Canadian Triage Acuity Scale (CTAS) vital signs norms [14, 17]. These norms were chosen to promote internal consistency within and between sites based on a nationally accepted standard. The PEWS scoring section of the flowsheet has the norms embedded for easy plotting and is colour coded to provide a clear visual when vital signs are outside of the normal range.
The intent of situational awareness is to promote awareness, prediction, and mitigation of potential risk. Implemented tools in the ED setting included posters for visual cueing, discussion in staff reporting and regular documentation of four factors embedded in the flowsheet (caregiver concern, unusual therapy, watcher patient, and communication breakdown).
Adapted from the Cincinnati Situational Awareness Model 
The escalation aid outlines actions to support clinical decision making following assessment. Recommended mitigation actions (e.g. notification, reassessment, consultation) correspond to PEWS scores and situational awareness factors. A quick-view of the escalation aid was also embedded in the flowsheet.
Adapted from Cook Children’s Medical Center
The Situation, Background, Assessment, Recommendation (SBAR) toolkit was used to improve communication between team members regarding patient status.
SBAR toolkit 
Intervention implementation and monitoring
A team consisting of staff from nursing, medicine, and administration assumed active site leadership roles in implementation and support to health practitioners. Educational resources that had been developed for the provincial inpatient implementation were used for training the nurses in this ED pilot, including instructions for using the flowsheet, e-learning training modules, on-site training workshops, and case studies. Training reinforced the concept that PEWS was a tool to aid clinical judgment, not replace it.
We monitored implementation fidelity through quality audits of medical records at five points during the one-year implementation. These audits were based on selected quality indicators: utilization rates of flowsheets, completion and accuracy of PEWS scoring, documentation of situational awareness factors, and escalation of care. Results were used to provide further education where needed and to promote optimal uptake.
Dimensions, indicators, and sources of data selected for evaluation
Implementation fidelity: whether the program was implemented as designed and staff were satisfied with it.
PEWS score documentation
Post-implementation medical record eview
Accuracy of PEWS score calculation
Post-implementation medical record review
Satisfaction with PEWS in ED and its implementation
Online provider survey
Barriers and facilitators in implementation
Online provider survey and key-informant interviews
Effectiveness: whether the intervention changed practice.
Increased vital signs documentation
Pre and post implementation medical record review
Increased knowledge and confidence in pediatric care as a result of intervention
Online provider survey
Perceived changes in practice as a result of intervention
Online provider surveys and key-informant interviews
Improved communication between staff
Online provider survey
Utility: whether the staff found the intervention valuable and it aligned with current practice.
Alignment of PEWS score with CTAS
Post-implementation medical record review
Perceived usefulness and value of PEWS in ED and its components in provision of care
Online provider surveys and key-informant interviews
We used three types of data collection techniques: medical record review, online survey of providers and semi-structured interviews with key informants to allow us to explore this system from multiple angles and offset the limitations or biases of any one method . The medical record reviews allowed us to explore fidelity and feasibility of implementation, and the documented changes in practice. The qualitative methods allowed us to round out this information by answering questions related to what, why, how the system was working from the perspective of the ED providers .
Medical record review
We carried out retrospective medical record reviews covering the 12-months pre-implementation as baseline, and the 12 months post-implementation for comparison. This timeframe accounted for the full cycle of seasonal variation in ED presentations.
The inclusion criteria for medical record selection were patients who (i) were seen in ED in the 12 months pre and post implementation, and (ii) were under 17 years of age (the provincial age for defining a person receiving hospital care as “pediatric”), and (iii) had an ED length of stay greater than 2 h (to allow for trending), and (iv) had a disposition of either transfer to higher level of care or admission to hospital, and (v) had a CTAS assignment of 1–4 (as proxy measures of acuity and severity). We stratified our sample to include patients across all CTAS levels 1–4 and age groups (0–3 months, 4–11 months, 1–3 years, 4–6 years, 7–11 years, 12–17 years). The final sample included all patients transferred to a higher level of care, all CTAS 1 and 4 patients who were admitted to the hospital (as these were few in number) and 50% of the CTAS 2 and 3 patients who were admitted to the hospital, randomly selected within the stratified age groups.
Data were extracted from the medical records by two experienced pediatric acute care nurses and inputted in an electronic data capture tool hosted by the BC Children’s Hospital Research Institute. REDCap© (Research Electronic Data Capture) is a secure, web-based application designed to support data capture for research studies . Identifiers were removed and a unique study code was generated for each patient. The reviewers maintained field notes, and in the case of uncertainty, decisions were made within the team.
Online survey of providers
One year post implementation, all ED nursing staff and physicians were invited to participate in an online survey to explore and describe key issues related to implementing PEWS in ED. Survey questions focused on exploring perceptions related to changes in practice, knowledge and confidence in providing pediatric care, usefulness of PEWS and its components, satisfaction with the tools and training, as well as barriers and facilitators to implementation. The survey took approximately 20 min to complete, included multiple choice questions, checkboxes, Likert scales, and open-ended questions, and was administered anonymously through Fluid Surveys, an online data collection platform. The survey was pilot tested by three experienced registered nurses prior to data collection and adjustments were made as needed.
Semi-structured interviews with key informants
The hospital administrator, registered nurse educator and physician who championed and supported the intervention were interviewed face-to-face in English for approximately 40 min after 12 months of implementation. Interview questions focused on understanding the utility of PEWS in ED and identifying the barriers and facilitators to implementation; this information was important for understanding how this complex system might function in a busy ED setting, to inform changes required and implementation processes. To confirm accuracy and credibility, the interviewer (YT) took notes capturing the responses and then summarized these for participants at the end of the interview.
Quantitative data analysis
We used descriptive statistics to calculate the overall characteristics of patient medical records and Fisher’s exact test to determine if they were significantly different in any aspect. We used two-samples tests for equality of proportions to determine whether vital signs documentations significantly increased post-implementation compared to baseline. We used Spearman’s correlation to evaluate the relationship between PEWS scores and CTAS scores. Statistical analyses were carried out using Microsoft Excel and RStudio .
Response options of the online survey with 5-level Likert scales were collapsed into a measure consisting of 3 levels. Frequencies of responses were calculated for the different categories. In reporting survey responses, the denominators for the calculated percentages represent number of persons who responded to that particular question, not the whole survey.
Qualitative data analysis
Qualitative data from open-ended survey questions and interviews with key informants were analyzed using thematic content analysis to identify the key elements of the responses . Responses were reviewed, coded and categorized into commonly occurring themes by two separate members of the research team to enhance trustworthiness of the data. These researchers (TM and Research Associate JM-N) had experience and training with qualitative analysis in health research. The recurrent themes as identified by the researchers underwent member checking to ensure they accurately captured the perceptions of practitioners. This analysis method was chosen to address the primary aim of the qualitative methods: to explore and report on participant’s key perceptions as users of the new system. Saturation was obtained in the themes noted across the survey and interview responses.
Characteristics of patients included in medical record review pre and post implementation (n = 96 for each sample)
Female / male
52 (54.2) / 44 (45.8)
39 (40.6) / 57 (59.4)
Most responsible diagnosis a
Discharge disposition from ED
Transferred to higher level of care
As for provider surveys, the response rate was 38% (n = 37) for registered nurses and 72% (n = 13) for physicians. Half of the practitioners had less than 5 years of clinical work experience. 39% (n = 5) of physicians and 21% (n = 8) of nurses reported having more than 15 years of work experience. Most estimated that 10–25% of their practice was pediatric care.
Three key-informant interviews were conducted.
Medical record reviews show that the intervention was implemented with high fidelity. At triage, 80.2% (n = 77) of the medical records had a PEWS score completed and of those 87% (n = 67) had been accurately calculated. At first bedside assessment, 81.2% (n = 78) of the medical records had a PEWS score present and of those 88.5% (n = 69) were accurate. Medical record review found the most common reason for inaccuracy in PEWS score to be adding the sub-scores instead of taking the highest score for the section, a matter that key informants suggested was improved through practice and more education. At both triage and first bedside assessment, highly urgent or non-urgent patients (CTAS scores 1 and 4) were more likely to be missing a PEWS score than others (p-value = 0.0375 for triage and 0.0354 for bedside assessment, Fisher’s exact test). In 52.1% (n = 25) of patients with an initial PEWS score of 0–3, the time in between assessments was 2 h or less, as per the implementation plan. In 61.4% (n = 59) of the records, a PEWS score was completed at all vital signs assessments throughout the ED visit. In an additional 29.2% (n = 28), PEWS score was completed for over half of the vital signs assessments.
Overall, majority of survey respondents were satisfied or very satisfied with PEWS scoring system (71.8% of nurses, n = 23; 81.8% of physicians, n = 9), PEWS flowsheet (56.2% of nurses, n = 18; 81.8% of physicians, n = 9), escalation guide (68.8% of nurses, n = 22; 81.8% of physicians, n = 9), and reference cards (75% of nurses, n = 24; 70% of physicians, n = 7). Satisfaction was relatively lower for situational awareness tools (41.9% of nurses, n = 13; 36.4% of physicians, n = 4) and the communication framework (54.5% of nurses, n = 17; 45.5% of physicians, n = 5).
Completeness of documentation based on medical record review pre and post implementation
PEWS score component
(n = 96)
(n = 96)
Documentation of parameters at first assessment in the ED
Capillary refill time
Consistent documentation of parameters throughout ED stay a
Capillary refill time
Using PEWS in ED further promoted documentation of two situational awareness factors throughout the patient stay: recording of “caregiver concern” increased from 10.4 to 67.7%, and “watcher patient” from 2.1 to 40.6%. “Watcher patient” was noted by the chart reviewer to be particularly important for documenting risks that PEWS score is not designed to capture, including: pain, surgical risk, abnormal lab values, neurologic status, and mental health risk.
We also observed a 51% increase in documentation of notification of most responsible physician and documentation of response by physician.
Intervention effectiveness: perception of change in knowledge and confidence of practitioners
Perceived change in knowledge and confidence
Not at all /to a slight extent (%)
To a moderate extent (%)
To a great / very great extent (%)
Change in knowledge
Identification of abnormal clinical signs
Identification of situational awareness factors that increase risk
Mitigation of deterioration
Escalation of care
Change in confidence
Identification of abnormal clinical signs
Identification of situational awareness factors that increase risk
Mitigation of deterioration
Escalation of care
Interviewed key-informants highlighted a few important issues about the effects of implementing PEWS at triage: 1) the tool was easily accepted and used, i.e. its adoption took little promotion or management from leadership, 2) time to complete PEWS scoring decreased with practice, and 3) the slight increase in triage time was outweighed by thoroughness of assessment and greater awareness of patient status; this was particularly important when the department was busy and wait times were long. Surveyed physicians noted an overall increase in staff awareness regarding pediatric patients, which promoted earlier notification. “I think the general increase in awareness of frontline staff has improved, and that either escalates the CTAS score, or prompts the nurses and others to bring the patient to our attention sooner” (Physician).
Implementation effectiveness: perception of improvement in verbal communication attributed to PEWS in ED
Perceived improved verbal communication between practitioners
Frequency of verbal communication
Timing of verbal communication
Clarity of verbal communication
Outcomes of verbal communication
Overall, the majority of survey respondents (78.9%, n = 30) agreed that implementation of PEWS in the ED was valuable for pediatric patient care. One nurse highlighted: “It increased our response to children with abnormal vital signs that are compensating or appear relatively well.” The leadership also noted: “PEWS helped validate their clinical decision making and offered a standardized approach to care.”
Intervention utility: perception of usefulness of different BC PEWS components
Not at all / to a slight extent (%)
To a moderate extent (%)
To a great / very great extent (%)
Pediatric assessment flowsheet
Situational awareness a
Overall value to pediatric care in ED
Some participants perceived little value in using PEWS (5.3%, n = 2). Their reasoning related to the need to further tailor the flowsheets to the ED environment: “PEWS form is good, but frustrating to have to use old form too. It would be great if there was an ED specific PEWS form” (Nurse).
To study alignment of the intervention with current practice, we explored how the PEWS score relates to CTAS score, the well-established triage scoring system. Seventy-seven charts included both a CTAS score and PEWS score at triage. Collapsing PEWS scores 5–13 into a single score (since they would trigger the same escalation guide), we found PEWS scores (0–5) and CTAS scores [1, 2, 3, 4] to be inversely correlated (Spearman’s rho = − 0.574, p-value < 0.001).
The majority of surveyed nurses (89.6%, n = 25) felt that it was “valuable” or “possibly valuable” to complete a PEWS score at triage alongside CTAS (physicians are not involved in triage, therefore were not asked these questions). Practitioners noted that the PEWS score provided a baseline for trending across the visit and gave more objective, thorough results by promoting assessment of all parameters consistently with all patients, which helped to correctly assign a CTAS score. “It (PEWS score) helps me a lot with assessment and priority settings in triage especially for the pediatric population who came in really sick and unable to obtain complete information/data from parents, guardians or significant other” (Nurse).
Themes of perceived positive and negative impacts of PEWS in ED on pediatric practice
Perceived positive impacts
• Prompts earlier recognition of risk, change, decline, and abnormality
• Increases provider’s general awareness of risk, concern, and abnormality
• Guides triage decisions
“With the PEWS scores that I gathered after assessment, it helped me alert the primary nurse and emergency doctors to see patient as soon as possible and be able to render care in a timely manner.”- Nurse
• Provides a standardized assessment framework
• Improves ease and comprehensiveness of assessment (full vital signs)
• Increases staff comfort with vital signs norms
“Caring for a pediatric patient can be very stressful for some nurses, the flowsheet helps guide the inexperienced nurse through a very thorough assessment.”- Nurse
• Provides a baseline for monitoring from triage onwards
• Increases frequency of vital signs assessment and closer observation
• Improves ability to trend across patient stay which helps with care and disposition decisions
“Children get vitals more often and all of their vitals more often! It is easy to see what vitals are normal and what are not with graphs and colours. It is easy to see changes and trends in vitals. It likely prompts more repeat vitals and assessments and documentation thereof. When done at triage prompts full vitals early.”- Physician
• Provides a standardized approach to communication (speaking the same language)
• Promotes earlier notification of physicians
• Enhances delivery of thorough information to physicians
• Improves confidence of nurses with notification (validation by score)
“Effective as a means of communicating with doctors, helpful in confirming if patient is improving as appearance can be deceiving, PEWS assists with confirmation.”- Nurse
• Provides a standardized approach to escalation
• Supports earlier response as notification occurs faster
“I think it is very important data when it comes to assessment, priority setting and rendering time sensitive actions to every peds patient.”- Nurse
• Promotes better overall care for pediatric patients
“PEWS has really played a huge role in the way in which we care for and treat our pediatric patients. This has been a real positive change.”-Nurse
Perceived negative impacts
• Scores may not accurately capture clinical status in some instances (e.g. false positive scores due to upset, post medication)
“Need to alert staff that PEWS can be elevated due to patient stress in ED environment/circumstances, crying, tantrum, etc. Thus, require a strategy for these circumstances i.e. using clinical judgment, repeating scoring when patient calms.” - Nurse Educator
• Standardized scoring and escalation can take away from clinical judgement
“In my experience as charge nurse, we use clinical judgment, vital signs anyways, and the PEWS is just a numerical value of our own clinical assessment.” - Nurse
• Can increase time for assessment (particularly at triage)
“Added paperwork and time spent especially at triage” - Nurse
Lack of tailoring to ED setting
• Increases paperwork because poorly integrated with current ED paperwork (double charting)
• Form is missing important information for ED (e.g. narrative space, medication record)
“Increased charting as not everything can be charted on one paper” - Nurse
• Lacks relevance or seems excessive for patients with single system or minor injuries
“It seems too time consuming and unnecessary to do for all children presenting in the ED. There are a lot of very simple presentations that do not seem to warrant the full set of vitals and score. For example, children coming in with very small lacerations or simple wounds, or even simple injuries.” - Nurse
This study presents the implementation of PEWS in the ED of a general urban hospital in British Columbia, Canada. Employing a framework for evaluation of complex healthcare interventions and using medical record reviews, surveys, and key-informant interviews, we evaluated the intervention’s fidelity, effectiveness, and utility . Our study supports that a multi-faceted PEWS can be valuable in supporting ED practitioners with providing timely and effective pediatric care. However, we concluded that it is important that the tools and training strategy are tailored to the ED context to enhance implementation fidelity and satisfaction.
Using PEWS and CTAS at triage
In this pilot intervention, PEWS scoring was completed at triage alongside CTAS scoring . These two scores have been designed for different purposes and their scoring is done differently: PEWS identifies early signs of deterioration and is scored through assessment of vital signs , while CTAS assigns acuity and is scored via assessment and clinical judgement . We were able to show that the scores correlate and that practitioners found them synergistic; we therefore conclude that there is benefit in using both systems with pediatric patients. The PEWS score calculation reinforces a more comprehensive, objective assessment informed by vital signs norms based on age. In our study, implementation of PEWS significantly increased documentation of physiologic parameters; a benefit reported by others . This systematically addresses issues of variance across providers and as highlighted by Roland et al. , may reduce cognitive bias, errors or missed assessment components that could otherwise occur because of the pressures of the triage environment .
Study participants noted a few limitations when using PEWS at triage. An increase in time at triage was highlighted, an issue which improved with practice and was largely acceptable to practitioners who indicated that the value of scoring outweighed time increase. There were also instances of false positive scores owing to factors such as irritability, which altered vital signs and skewed PEWS scores. In our study, we aimed to counteract this issue through reinforcing the importance of clinical judgement and repetition of assessment when score accuracy was in question. Limited specificity has also been reported elsewhere [9, 10, 12]. As parameters included in a PEWS score assessment are important for determining risk, this substantive increase in thorough assessment and documentation at triage is a very positive change. The limitations of PEWS were acceptable to practitioners who overall found value in using PEWS at triage alongside the CTAS triage tool.
Using the 5-component BC PEWS at bedside evaluation
It has been demonstrated that nurses and physicians may fail to recognize deteriorating children due to lack of consistency or accuracy in recording physiological observations [23, 24]. Consistent and comprehensive documentation has been shown to be particularly important in the ED environment where health care providers need to document improvement and appropriateness for discharge . Moreover, with pediatric patients, this documentation is even more critical because providers often rely on general appearance and vital signs as opposed to patients’ descriptions of improvement. This study reviewed documentation of PEWS components across the patient stay. The introduction of PEWS was associated with a substantive increase in documentation across all physiologic parameters. We conclude that the introduction of PEWS in ED can have a positive influence on rates and thoroughness of documented assessment parameters. This not only provides a more consistent “in the moment” picture of physiologic abnormality, but also allows for documentation of trending from the point of presentation.
The “active ingredients” of BC PEWS
BC PEWS was designed as a multifaceted safety system as advocated by Lambert et al. . In trying to understand its utility as a system, we asked practitioners to assess the usefulness of each of the components. While all of the components of BC PEWS were ranked useful by the majority of respondents, the PEWS escalation aid and score were perceived as most useful. Practitioners reported that the norm-based scoring enhanced or validated identification of sick children . The escalation aid justified escalating to senior review and encouraged repeated assessment or more thorough monitoring throughout the patient stay.
The systematic documentation and reporting of situational awareness factors were ranked as the least useful part of the system. Although there was a substantive rise in the documentation of “caregiver concern” and “watcher patient”, the medical records reviewer indicated instances where “watcher patient” was missed as an opportunity to elevate a risk profile. For instance, as has been shown in the literature, PEWS scores are more accurate in capturing significant medical illness (particularly respiratory related) than surgical risk . Medical record reviews noted instances such as surgical risk, mental health risk, abnormal labs or neurovitals, where “watcher patient” would have been an appropriate and systematic means of elevating risk profile and guiding appropriate action. Thus, despite lower usefulness rankings, we believe that situational awareness factors highlight risk and create awareness beyond scoring, which has value as part of a safety system in the ED. Moving forward, we believe more tailoring to ED context and further education of staff may improve uptake. Context and staff perceptions play crucial roles in PEWS implementation success as we and others have observed [25, 26]. In addition, as current research focuses on the inpatient setting, further investigation of the impact of situational awareness in the ED on care and patient outcomes will be important.
Communication within the team
Enhanced communication is a demonstrated factor in improving quality healthcare . Communication breakdown at handovers are a known point of risk that require consideration when planning an effective system . Because BC PEWS was implemented across the province in inpatient settings, our team chose to implement the same tools to promote a common language, understanding and process across sites and departments, and within the ED. Our findings support that this implementation strategy was effective in enhancing communication as our survey found positive influence on verbal communication and our medical records review found significant gains in documentation. Further, staff reported increased comfort with escalation and communication of risk for senior review, which is an important characteristic of an optimal PEWS tool. As noted above, the staff attributed this gain to having objective tools to validate the need and process for escalation . While we will be taking feedback from this research to revise an assessment flowsheet for ED, we found value in a consistent scoring tool to promote internal and external communication and will retain the Brighton  tool in scale up.
PEWS, knowledge and confidence through standardization
As described by Oldroyd and Day, there is a lack of exposure to seriously ill children among ED nurses in general hospitals . Low practice volumes present challenges with maintenance of competency, thus guidance provided by PEWS can assist with identification of serious illness . Our study showed that nurses reported improvements in knowledge and confidence post-PEWS implementation and were completing more comprehensive assessment of vital signs that improved their ability to trend the patient stay. Physician respondents indicated similar benefits in knowledge and confidence and site leaders expressed an overall perception that the healthcare team showed greater awareness of risk or abnormality in the pediatric population. Importantly, improvement in staff knowledge and confidence for providing provision of pediatric care has been noted to be one of the key outcomes of PEWS implementation, which unlike patient outcomes is robustly measurable .
Given the decrease in variability of assessment practices using BC PEWS, we conclude that the implementation of PEWS influenced the equalization of patient care between providers. Additionally, having vital signs reference ranges visually evident in PEWS decreased knowledge deficit and cognitive load that may be present for staff who are less familiar or experienced with the pediatric population.
A minority of survey respondents perceived that systems such as PEWS take away from autonomy of experienced nurses. While overall findings of the study support positive associations with the use of such standardized systems, this finding suggests a need to acknowledge and reinforce the role of such systems in supporting clinical judgement rather than replacing or usurping it [26, 28].
The overall value of PEWS for ED
Bonafide et al. demonstrated that beyond the marginal performance of PEWS when applied to data sets, clinicians who recently experienced PEWS score failures (false positives) still considered it valuable . In a survey of 254 general ED practitioners in the UK, Griffiths and Kidney found that the majority support the use of early warning systems in the ED, despite the evidence that such scores are low in sensitivity . As with other studies, we found limitations with PEWS in the ED; however these limitations did not appear to alter the overall value assigned to PEWS by staff.
Roland et al. outlined that early warning scores in ED should support earlier diagnosis, more accurate estimation of illness severity and improved communication . Our participants reported that BC PEWS fulfilled all of these functions.
An effective clinical tool is one that practitioners use successfully, and one they want to use [25, 26]. Notably, the hospital team continued to use BC PEWS after the conclusion of the study and have provided ongoing support to the re-design of the system and scale up for the province.
Informing scale up
This study demonstrated that BC PEWS is useful and beneficial to pediatric patient care when used with clinical judgment and alongside the currently accepted tools (CTAS) in the ED.
Based on these results and the growing body of evidence in the literature, provincial stakeholders made the unanimous decision to move to provincial implementation of PEWS in all ED settings in British Columbia. This pilot study’s results were used to redesign the 5-component BC PEWS specifically to address the needs of EDs (BC PEWS ED). This involved the creation of a provincial emergency nursing assessment record (short and long form versions) with a compatible PEWS scoring sheet for recording vital signs, and a provincial escalation guide for the ED, which corresponds to PEWS scores. Training materials were developed for ED staff including online training modules, in-person workshops, and a series of quick educational sessions that can be offered as refreshers. Health Authorities from across the province reviewed and accepted the new system to ensure buy-in and regional support. Beginning in February 2018, the province launched a phased implementation of BC PEWS ED, which included the full range of EDs serving children from the generalist rural/remote facilities to the provincial subspecialty pediatric facility. The implementation is being monitored through quarterly audits and planning is underway for researching the scale up of implementation.
The study has a number of limitations. First, there were no accurate measures of deterioration available when identifying the dataset for review so proxy measures of deterioration or severity (i.e. transfer to higher level of care, admission, triage score) were used to select medical records for review. However, we believe that the selected proxy measures captured the population of interest. Second, response rate for the health providers survey was limited to 55%, even after sending out reminders in an attempt to increase the rate. However, our response rate is high compared to a study looking at response rates to online surveys among Canadian physician specialists . Nevertheless, a non-response bias cannot be ruled out. The study did not account for an acclimatization phase in the implementation and did not look at impact on patient outcomes, matters that will be addressed during the research of the provincial scale up.
In this study, standardization of pediatric ED care through PEWS demonstrated several benefits: PEWS significantly improved comprehensiveness of assessment and documentation at triage and throughout the ED stay. Accordingly, there was a perception that identification and awareness of risk rose and equalized between practitioners. BC PEWS upgraded providers perceived knowledge and confidence related to pediatric care, and increased timely and effective communication between practitioners. Scoring and the escalation aid made it objectively clear when senior review was required, helping to remove ambiguity and variance. Consequently, we believe that BC PEWS contributed to addressing the issues of cognitive load seen in typical ED settings, particularly amongst practitioners for whom pediatrics is a small part of a heavy caseload. Finally, all components of the system were perceived and reported as useful, and their use continued and evolved beyond the end of the study period, most importantly in planning for provincial scale up.
The authors would like to acknowledge Suud Nahdi for his support in the development of the research plan; Melissa Coop and Meaghan Unger for their support of the medical record review; Jennifer McConnell-Nzunga for her help with qualitative data analysis, Michelle Fryer, Nicole Cave and Shannon Moffat for their work in the development and support for implementation of PEWS throughout British Columbia; Terry Chau for his help in development of the manuscript’s visuals; Boris Kuzeljevic for his support with statistical consultation; BC Children’s Hospital Learning & Development for their support in the design of BC PEWS, the BC Children’s Hospital Foundation who provides support for Child Health BC and the entire Richmond Hospital Emergency Department staff for their willingness to support innovation and research in order to improve emergency care for children in the province.
TM co-designed the study, prepared ethics application, coordinated roll out of PEWS within the Vancouver Coastal Health Authority, coordinated data collection and cleaning, analysed and interpreted data from all sources and was a major contributor in writing the manuscript. ES co-designed the study, reviewed results, provided input on analysis and interpretations and reviewed and revised the manuscript. KH contributed to data analysis and in writing the manuscript. SW performed analysis on medical record review data and was a major contributor in writing the manuscript. YT coordinated the provincial roll out of the PEWS system in British Columbia, reviewed results, provided input on analysis and interpretations and reviewed and revised the manuscript. CM led the roll out of the intervention for nursing at Richmond Hospital, monitored and addressed implementation fidelity at the site and reviewed and revised the manuscript. RC led the roll out of the intervention for physicians at Richmond Hospital, reviewed and revised the manuscript. DW was a principal investigator, co-designed the study, reviewed results, provided input on analysis and interpretations and reviewed and revised the manuscript. MO was a principal investigator, co-designed the study, reviewed results, provided input on analysis and interpretations and reviewed and revised the manuscript. All authors read and approved the final manuscript.
The authors received no specific funding for this work.
Ethics approval and consent to participate
Ethics approval was obtained through the Children’s and Women’s Research Ethics Board; reference # H15–02236-A002. Informed consent was obtained for online surveys and key informant interviews, including the right to confidentiality for all participants and withdrawal from the study for the interviewees (since online surveys were anonymous, withdrawal was not possible).
Consent for publication
The authors declare that they have no competing interests.
- 1.Adshead N, Thomson R. Use of a paediatric early warning system in emergency departments. Emerg Nurse. 2009;17(1):22–5 [cited 2019 May 17] Available from: http://rcnpublishing.com/doi/abs/10.7748/en2009.04.17.1.22.c6984.CrossRefGoogle Scholar
- 2.Brady PW, Muething S, Kotagal U, Ashby M, Gallagher R, Hall D, et al. Improving situation awareness to reduce unrecognized clinical deterioration and serious safety events. Pediatrics. 2013;131(1):e298–308 [cited 2019 May 17] Available from: http://pediatrics.aappublications.org/cgi/doi/10.1542/peds.2012-1364.CrossRefGoogle Scholar
- 3.Institute of Medicine (US) Committee on Quality of Health Care in America. To Err Is Human. Linda T. Kohn, Janet M. Corrigan and MSD, editor. Washington, D.C.: National Academies Press; 2000 [cited 2019 May 17]. Available from: http://www.nap.edu/catalog/9728
- 4.Lambert V, Matthews A, MacDonell R, Fitzsimons J. Paediatric early warning systems for detecting and responding to clinical deterioration in children: a systematic review. BMJ Open. 2017;7(3):e014497 [cited 2019 May 17] Available from: http://bmjopen.bmj.com/lookup/doi/10.1136/bmjopen-2016-014497.CrossRefGoogle Scholar
- 5.Corfield AR, Silcock D, Clerihew L, Kelly P, Stewart E, Staines H, et al. Paediatric early warning scores are predictors of adverse outcome in the pre-hospital setting: A national cohort study. Resuscitation. 2018;133:153–9 [cited 2019 May 17] Available from: https://linkinghub.elsevier.com/retrieve/pii/S030095721830995X.CrossRefGoogle Scholar
- 6.Agulnik A, Mora Robles LN, Forbes PW, Soberanis Vasquez DJ, Mack R, Antillon-Klussmann F, et al. Improved outcomes after successful implementation of a pediatric early warning system (PEWS) in a resource-limited pediatric oncology hospital. Cancer. 2017;123(15):2965–74 [cited 2019 May 17] Available from: http://www.ncbi.nlm.nih.gov/pubmed/28440868.CrossRefGoogle Scholar
- 9.O’Leary F, Hayen A, Lockie F, Peat J. Defining normal ranges and centiles for heart and respiratory rates in infants and children: a cross-sectional study of patients attending an Australian tertiary hospital paediatric emergency department. Arch Dis Child. 2015;100(8):733–7 [cited 2019 May 17] Available from: http://adc.bmj.com/lookup/doi/10.1136/archdischild-2014-307401.CrossRefGoogle Scholar
- 10.Gold DL, Mihalov LK, Cohen DM. Evaluating the Pediatric Early Warning Score (PEWS) system for admitted patients in the pediatric emergency department. Walthall J, editor. Acad Emerg Med. 2014;21(11):1249–56 [cited 2019 May 17] Available from: https://onlinelibrary.wiley.com/doi/abs/10.1111/acem.12514.CrossRefGoogle Scholar
- 11.Lillitos PJ, Hadley G, Maconochie I. Can paediatric early warning scores (PEWS) be used to guide the need for hospital admission and predict significant illness in children presenting to the emergency department? An assessment of PEWS diagnostic accuracy using sensitivity and specificity. Emerg Med J. 2016;33(5):329–37 [Cited 2019 May 17] Available from: http://www.ncbi.nlm.nih.gov/pubmed/26531861.CrossRefGoogle Scholar
- 14.Bullard MJ, Musgrave E, Warren D, Unger B, Skeldon T, Grierson R, et al. Revisions to the Canadian Emergency Department Triage and Acuity Scale (CTAS) Guidelines 2016. CJEM. 2017;19(S2):S18–27 [cited 2019 May 17] Available from: https://www.cambridge.org/core/product/identifier/S1481803517003657/type/journal_article.CrossRefGoogle Scholar
- 15.Monaghan A. Detecting and managing deterioration in children. Paediatr Care. 2005;17(1):32–35. [cited 2019 May 17] Available from: http://rcnpublishing.com/doi/abs/10.7748/paed2005.02.17.1.32.c964 CrossRefGoogle Scholar
- 16.Leonard M, Graham S, Bonacum D. The human factor: the critical importance of effective teamwork and communication in providing safe care. 2004;13. [cited 2019 May 17] Available from: http://www.ncbi.nlm.nih.gov/pubmed/15465961.
- 17.Fleming S, Thompson M, Stevens R, Heneghan C, Plüddemann A, Maconochie I, et al. Normal ranges of heart rate and respiratory rate in children from birth to 18 years of age: a systematic review of observational studies. Lancet. 2011;377(9770):1011–8. [cited 2019 May 17] Available from: http://www.ncbi.nlm.nih.gov/pubmed/21411136.CrossRefGoogle Scholar
- 19.Green J, Thorogood N. Qualitative methods for health research. 4th ed. Sage Publishing; 2018. 420 p.Google Scholar
- 20.Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)—A metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009;42(2):377–81 [cited 2019 May 17]Available from: https://www.sciencedirect.com/science/article/pii/S1532046408001226.CrossRefGoogle Scholar
- 21.RStudio Team. RStudio: integrated development for R [Internet]. Boston MA; 2016. Available from: www.rstudio.com
- 22.Gravel J, Opatrny L, Gouin S. High rate of missing vital signs data at triage in a paediatric emergency department. Paediatr Child Health. 2006;11(4):211–5 [cited 2019 May 17] Available from: https://academic.oup.com/pch/article-lookup/doi/10.1093/pch/11.4.211.CrossRefGoogle Scholar
- 24.Porter SC, Manzi SF, Volpe D, Stack AM. Getting the data right: information accuracy in pediatric emergency medicine. Qual Saf Health Care. 2006;15(4):296–301 [cited 2019 May 17] Available from: http://qualitysafety.bmj.com/lookup/doi/10.1136/qshc.2005.017442.CrossRefGoogle Scholar
- 26.Almblad A-C, Siltberg P, Engvall G, Målqvist M. Implementation of Pediatric Early Warning Score; Adherence to Guidelines and Influence of Context. J Pediatr Nurs. 2018;38:33–39. [cited 2019 May 17] Available from: https://linkinghub.elsevier.com/retrieve/pii/S0882596317300337 CrossRefGoogle Scholar
- 27.McBride DL. Do Pediatric Early Warning Systems for Hospitalized Children Reduce Mortality? J Pediatr Nurs. 2018;43:132–133. [cited 2019 May 17] Available from: https://linkinghub.elsevier.com/retrieve/pii/S0882596318301842 CrossRefGoogle Scholar
- 28.Jensen CS, Olesen HV, Aagaard H, MLO S, Kirkegaard H. Comparison of Two Pediatric Early Warning Systems: A Randomized Trial. J Pediatr Nurs. 2019;44:e58–65 [cited 2019 May 17] Available from: https://linkinghub.elsevier.com/retrieve/pii/S0882596318303610.CrossRefGoogle Scholar
- 29.Bonafide CP, Roberts KE, Weirich CM, Paciotti B, Tibbetts KM, Keren R, et al. Beyond statistical prediction: qualitative evaluation of the mechanisms by which pediatric early warning scores impact patient safety. J Hosp Med. 2013;8(5):248–53 [cited 2019 May 17] Available from: http://www.journalofhospitalmedicine.com/jhospmed/article/127141/early-warning-score-qualitative-study.CrossRefGoogle Scholar
- 30.Griffiths JR, Kidney EM. Current use of early warning scores in UK emergency departments. Emerg Med J. 2012;29(1):65–6 [cited 2019 May 17] Available from: http://emj.bmj.com/lookup/doi/10.1136/emermed-2011-200508.CrossRefGoogle Scholar
- 31.Cunningham CT, Quan H, Hemmelgarn B, Noseworthy T, Beck CA, Dixon E, et al. Exploring physician specialist response rates to web-based surveys. BMC Med Res Methodol. 2015;15(1):32 [cited 2019 May 17] Available from: https://bmcmedresmethodol.biomedcentral.com/articles/10.1186/s12874-015-0016-z.CrossRefGoogle Scholar
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.