It has been estimated that 5 billion people lack access to safe surgery; the vast majority (94%) of whom live in low- and middle-income countries (LMIC).1 The need for safer surgery in LMIC has been recognized as part of the Sustainable Development Goals2 and the Lancet commission on Global Surgery.2

In 2018, Biccard et al. identified that patients in Africa were twice as likely to die in the perioperative period compared with the world average.3 This is despite being younger and with fewer pre-existing comorbidities, which are characteristics usually associated with lower risk patient groups and fewer postoperative complications. Direct anesthesia-related mortality is also disproportionately high in LMIC.4 Zambia is a middle-income country in sub-Saharan Africa, and was included in the Biccard study. Perioperative morbidity and mortality in the emergency surgical setting is significant in Zambia. A 2012 study of 18,010 surgical patients reported that 114 (0.6%) patients had died by postoperative day 65; 30% of deaths were avoidable, 32% were “probably” avoidable, and failure to recognize the deteriorating patient was often a key factor in avoidable death.5 In a resource-limited environment, the early identification of critically ill patients is likely to be key to allowing appropriate resource allocation in a timely fashion.

The use of early warning scores (EWS) to identify patients at risk for sudden deterioration on admission to hospital is common in high-income countries (HIC). Patient’s physiologic parameters are tracked and given a numerical value, which triggers a medical response. Early detection and management of deteriorating patients is known to reduce morbidity and mortality.6 In the UK, the National Early Warning Score (NEWS) is used in all National Health Service hospitals.6 There is currently no internationally agreed EWS, and the role of EWS in LMICs has not yet been defined. Variants of the EWS, such as a Modified Early Warning Score (MEWS), have been suggested as potentially useful in low-resource environments.7 Nevertheless, patients in Zambia have different characteristics than those in the UK and other HICs, as was noted by Biccard.2 Patients are often younger and present later to healthcare services. Furthermore, trauma provides a larger proportion of surgical caseload. In addition, patients have a different co-morbidity profile with human immunodeficiency virus (HIV) and tuberculosis (TB) being more prevalent. The surgical burden of disease is different, as is the access to surgical care.8,9,10,11 Thus, scoring systems used in HICs cannot be considered automatically valid with regards to patients in LMICs. They require specific validation in this environment before implementation is considered.

Previous work done in Zambia by our group has identified that if the NEWS score was used, 45% of acute medical and surgical admissions would have a NEWS score of > 7, commonly used in the UK as the cut-off for a critical care referral.12 An accompanying editorial highlighted that NEWS does not consider diagnoses of HIV or TB.13 The authors suggested that in this population, in addition to physiologic abnormality, HIV and TB are likely to be independent predictors of mortality.13

The primary objective of this study was to identify if the NEWS,6 MEWS,7 a modified Hypotension, Oxygen Saturation, Temperature, Electrocardiogram, Loss of Independence (mHOTEL) score,14 or the Tachypnoea, Oxygen saturation, Temperature, Alertness, Loss of Independence (TOTAL) score14 could predict 30-day mortality in a Zambian emergency surgical population. Secondary objectives were to identify whether any one of these clinical prediction models had better discriminant predictive ability than others, and to investigate if the presence of human HIV or TB predicted an increased mortality in this group.

Methods

This study was a prospective observational study, which included adult emergency surgical patients admitted to the University Teaching Hospital in Lusaka, Zambia. The Transparent Reporting of a multivariate prediction model for Individual Prognosis or Diagnosis (TRIPOD) reporting guidelines were adhered to in this study.15 The biomedical research ethics committee at The University of Zambia approved the study on 30 January, 2017 (reference number 003-01-17). Written consent was taken from all participants. All patient data were anonymized prior to data analysis and were securely stored. All surgical specialties were included except obstetrics. Data were collected between March 2017 and July 2017.

The study inclusion criteria included patients age < 18 yr admitted for any emergency surgery (except obstetrics) who were in hospital for > 24 hr. We excluded pregnant patients because the normal physiologic parameters are different in the obstetric population. Separate EWS are available for this patient population.16 We also excluded those who were unable to obtain consent or were admitted to hospital for < 24 hr.

Local medical students were trained as research assistants and completed study recruitment and data collection. Data were collected in person on days 1, 2, and 3 of hospital admission. On day 30, the patient or their family was contacted by telephone. Figure 1 is a flowchart of the study design, detailing which data were collected at each point.

Fig. 1
figure 1

Study flow diagram. n refers to the number of patients for which data were available on that day. The data collected at each time point are shown on the right side of the figure

Full size image

A Lifebox™ pulse oximeter (Acare Technology Co. Ltd, New Taipei City, Taiwan) and a Pro Logic PL100™ Upper Arm Digital Blood Pressure Monitor non-invasive blood pressure machine (Omron, Osaka, Japan) was used as standard to take measurements from all patients. Axillary temperature was measured using unbranded digital thermometers routinely available on the surgical wards.

The observations taken were used to calculate four EWS: NEWS,6 MEWS,7 a modified HOTEL (mHOTEL),14 and a TOTAL14 score (Table 1). In all of these early warning scoring systems, the higher the numerical value assigned, the more abnormal the physiologic parameter. A higher overall score denotes increasingly deranged physiology and indicates that urgent medical attention or even critical care referral is required.

Table 1 Components of the early warning scoring systems studied
Full size table

The NEWS was calculated using systolic blood pressure, heart rate, oxygen saturation (and the presence of supplementary oxygen), respiratory rate, temperature, and level of consciousness. Consciousness was defined by the Alert, responds to Voice, responds to Pain or Unresponsive (AVPU)17 score. Of note, because of the limited availability of oxygen, if a patient’s oxygen saturation was < 94%, we elected to score those patients as receiving oxygen whether or not it was actually available on the ward. The MEWS was calculated using systolic blood pressure, heart rate, respiratory rate, temperature, and AVPU score. The mHOTEL score was calculated using systolic blood pressure, oxygen saturation, temperature, and loss of independence. Loss of independence was defined as being unable to mobilize unaided as previously described.10 The ECG was not included in the modified HOTEL score as it was not available. The TOTAL score was calculated using respiratory rate, oxygen saturation, temperature, AVPU score, and loss of independence (as defined above).

After data collection and planned analysis were completed, we elected to also consider a modification of the NEWS score, based on equipment that would usually be available on the wards at University Teaching Hospital (UTH) outside of the context of this study. This modified NEWS score (mNEWS) was calculated using respiratory rate, heart rate, temperature, consciousness level, and whether the patient was receiving oxygen. It was scored as per NEWS but excluded blood pressure and pulse oximetry. Because of the post hoc nature of this element of the study, we consider this aspect to be an exploratory analysis.

Statistical analysis

To calculate sample size, we aimed to have sufficient power for both the primary and secondary analysis. As the comparison between tools requires a larger sample size, we based the sample size calculation on the ability to be able to show, when comparing two different early warning scales, the difference between an AUROC of 0.8 (considered to be a “good” predictive test) and an AUROC of 0.7 (a “fair” predictive test). Assuming a mortality rate of 5% (from pilot data), a moderate correlation between the two EWS (to be conservative, estimated r = 0.6), a power of 80%, and an alpha value of 0.05, we calculated that we would need to recruit 1,260 patients (i.e., 1,200 survivors and 60 deaths).

Receiver operating curves (ROC) were created for each EWS and area under the curve (AUROC) calculated. As a secondary analysis, the AUROC curves were compared between early warning scoring systems using multiple pair-wise comparisons; P values were not adjusted for multiple comparisons. A multiple logistic regression analysis identified whether HIV, TB, or patient characteristics predict mortality. Sensitivity and specificity were reported for each cut-off of the EWS with the highest AUROC. A P < 0.05 was considered to be statistically significant. Statistical analysis was performed using SPSS (version 24, IBM, Armonk, NY, USA) for descriptive statistics and logistic regression and MedCalc (version 18.5, Ostend, Belgium) for comparing the area under ROC curves and for calculating sample size.

Results

Logistical issues relating to funding and the availability of research assistants resulted in only 293 patients being recruited, and analysis proceeded at that point. Complete data were available for 254 patients (Fig. 1). For 41 patients, data were collected on days 1–3, but we could not contact then on day 30 so they were not included in the 30-day mortality analysis. Patients involved in trauma accounted for the majority of admissions (110/293, 37%). Patient characteristics are described in Table 2. Nineteen of the 254 (7.5%) patients with a complete data set died within 30 days of admission. Fifty of the 293 recruited patients (19.7%) were HIV positive. Of those, 49 (98%) were on anti-retroviral agent therapy (ART) and of the 18 patients with CD4 counts known, ten had a CD4 count less than 500 cells·mm−3. Twenty-eight (9.6%) of the total study population were not tested for HIV at any point. Five patients were known to have TB on admission (5/293, 1.7%), and all five were receiving chemotherapy. Two inpatients were tested for TB and one was found to have active TB and was started on therapy. By day three of admission, 155 of the 254 patients with a complete data set (61%) had undergone a surgical procedure. Some may have had surgery after this time, but data were not collected on patients who underwent a surgical procedure by day 30.

Table 2 Patient characteristics
Full size table

Primary analysis

Table 3 shows the AUROC for the NEWS and MEWS scores collected on days 1 and 2 of admission, and for the HOTEL and TOTAL scores on day 1 (there were no data for HOTEL and TOTAL scores on day 2 because mobility status was only collected on day 1). Figure 2 shows the ROC curves for all four scores.

Table 3 Receiver operating curve characteristics
Full size table
Fig. 2
figure 2

Receiver operating curves. Receiver operating curves for five different early warning scoring systems: NEWS, MEWS, mHOTEL, TOTAL, and modified NEWS. NEWS = National Early Warning Score; MEWS = Modified Early Warning Score; mHOTEL = modified hypotension, oxygen saturation, low temperature, and loss of independence score; TOTAL = tachypnea, oxygen saturation, temperature, and loss of independence score; mNEWS = modified news score that includes only respiratory rate, heart rate, temperature, consciousness level, and whether the patient is receiving oxygen

Full size image

Figure 3 shows the point estimates and 95% confidence intervals (95% CI) for 30-day mortality for each NEWS score. Table 4 shows the sensitivity and specificity of the NEWS score on day 1 of admission for different cut-points. No patients were referred or admitted to critical care during their admission.

Fig. 3
figure 3

Red dots are point estimates of percentage 30-day mortality plotted against each NEWS score, the error bars are 95% confidence intervals. NEWS = National Early Warning Score

Full size image
Table 4 NEWS cut-points and their predictive value
Full size table

Post hoc exploratory analyses

Table 3 includes the AUROC for the exploratory post hoc modified NEWS score.

In addition, because our sample size was lower than intended, the following are considered exploratory analyses. We found no significant difference between the AUROC for the NEWS score and for any of the other EWS: NEWS day 1 vs NEWS day 2 (P = 0.77); NEWS day 1 vs MEWS day 1 (P = 0.31); NEWS day 1 vs MEWS day 2 (P = 0.49); NEWS vs HOTEL (P = 0.44); NEWS vs TOTAL (P = 0.69). We elected to compare the score with the highest AUC (NEWS on day 1) with the other scores.

Logistic regression did not find that HIV status, sex, or weekend admission were associated with mortality at 30 days (Table 5). Age was independently predictive of mortality at 30 days, with almost 1.5 times the risk of mortality for each additional decade of life (95% CI, 1.1 to 1.8).

Table 5 Results of logistic regression analysis
Full size table

Discussion

Our study found that the NEWS, MEWS, mHOTEL, TOTAL, and mNEWS scores were all predictive of 30-day mortality in a Zambian emergency surgical population. We found no difference in the AUROC for the different EWS. Nevertheless, our study was underpowered to identify this difference and so our data do not support any one EWS over another. Almost 20% of patients were HIV positive, but HIV status did not predict mortality. Increasing age was strongly associated with 30-day mortality.

The 7.5% 30-day mortality in this study was much higher than the 0.6% seven-day mortality found in a previous study in the same institution.15 This is unlikely to only be due to the longer follow-up, as most perioperative deaths occur in the first few days after surgery. We note that there was a difference in the populations included, as the current study recruited patients that ultimately did not have surgery, and may have been too sick to have had surgery, as well as excluding obstetric patients. The current study was also prospective, whereas the previous study was retrospective, limited by a large amount of missing data, and likely underestimated the surgical mortality. Finally, our sample only included emergency and urgent admissions to hospital, whereas the previous study also included elective patients.

If all patients in our cohort with a NEWS score equal or greater than 5 were referred to a critical care team, then there would be a good balance of sensitivity (74%) and specificity (72%). Nevertheless, this would require around 20% of surgical admissions being referred to critical care. If a cut-point of ≥ 7 was used, as is the trigger for critical care review in the UK, then NEWS would still identify almost two thirds of patients at risk for mortality. Nevertheless, this cut-point would still represent around 10% of the surgical population, a number that far exceeds the current critical care capacity at the University Teaching Hospital, Lusaka. In the same institution, Dart et al identified that when including medical inpatients, 45% of admissions to UTH had a NEWS score of more than 7.12 This would suggest that a larger implementation of an EWS system would have an even larger impact on critical care services, which would need to be considered. We suggest that if an EWS such as NEWS was to be introduced in Zambia, it would be necessary in the short term to pick a cut-off for escalation of care that had a high specificity at the expense of sensitivity, to be workable within current resources. These data can also advocate for increased critical care resources, as it is clearly an ethical imperative that patients with a high risk of mortality should be able to access timely medical care, and that a cut-off score for an EWS should have an acceptable sensitivity, which would be a priority for Zambian patients and their families.

No patients in our study were referred or admitted to a critical care unit. A detailed description of why this may have happened is beyond the scope of this study, but recognized barriers to critical care access include a lack of recognition of clinical deterioration, no clear referral trigger or pathway, and no clear means of communication (such as pagers or hospital phones) between the ward and intensive care unit. This is in addition to a lack of critical care beds.

If an EWS scoring system could be successfully implemented in Zambia it may allow scarce resources to be directed to the patients most in need. This study aimed to provide evidence that EWS systems can predict mortality in this Zambian population. As the implementation of an EWS system would require significant investment in equipment, training, communication systems, and critical care services, it is imperative that there is evidence of validity. It is not clear if the implementation of an EWS, and any subsequent enhancement of critical care services, would reduce in mortality in this group. There are currently no studies in LMICs that have shown improvements in mortality with the implementation of an EWS. Nevertheless, in the UK, the implementation of EWS has reduced hospital mortality.6

Previous work in 2012 highlighted that a large proportion of Zambian perioperative deaths were avoidable.5 Obtaining an EWS on admission could alert clinical staff to those patients most at risk of death and allow them to triage as appropriate. It could also be used to develop training courses in the immediate management of the acutely unwell patient. Another approach would be to develop a critical care outreach service, targeted at those with high EWS. This would allow the most unwell patients to receive immediate critical care input.

An additional barrier to implementation is that there is not widespread availability of equipment such as pulse oximetry and non-invasive blood pressure monitors. A score that does not require either monitor, such as the mMEWS, may be the most appropriate score in this setting. This would avoid the need for new equipment to be purchased. Nevertheless, the analysis of this score was only exploratory, so further work is required before it can be used.

There is currently conflicting evidence with regards to which EWS is most appropriate in LMICs. A MEWS score of > 5 predicted seven-day mortality in a Ugandan acute medical and surgical population.7 In Malawi, the HOTEL score was superior to MEWS in predicting three-day mortality in acute medical patients. The group in Malawi created a new score from their analysis (TOTAL score) that consists of respiratory rate, oxygen saturation, temperature, consciousness level, and loss of independence. They found this TOTAL score to be superior to both HOTEL and MEWS in this setting.14 Two studies have been conducted in Tanzania, on differing populations. One looked at patients admitted from a medical clinic and concluded that normal physiologic parameters predicted low risk of death, that there was a positive correlation between EWS and risk of mortality, and that the ability to walk unaided was associated with low risk of death.18 The second study looked at patients admitted to the intensive care unit and showed that a NEWS score of > 7 was associated with mortality. Nevertheless, they also identified that the presence of a single deranged physiologic parameter was also associated with mortality and concluded that the use of a single parameter could be as useful as an EWS in identifying unwell patients.19 Our study is the first study in sub-Saharan Africa that has examined the use of EWS in the acute surgical population. Our study has shown that all these scores can predict 30-day mortality in this population, and supports further validation of EWS implementation in the Zambian setting. Larger multi-centre international trials are required to assess if this is also the case in other LMICs; our data from a Zambian population are not generalizable to the other countries that are categorized by the World Bank as LMICs.

Our study had limitations related to recruitment and 30-day follow up. Our target recruitment was 1,260 participants, but we were unable to recruit this many because of practical limitations and financial constraints. Patients were lost to 30-day follow up because of communication issues, including patients or families not having mobile phones, incorrect phone numbers given, or patients not answering their phones. It is not clear if those that did not answer had died within the 30 days, so they were not included in the analysis.

In conclusion, our data support the implementation of EWS systems in a Zambian emergency surgical population. A larger study is needed to identify which scoring system performs best in Zambia. Previous research has shown that the majority of perioperative deaths are avoidable in this context. The implementation of an EWS may identify patients most at risk and allow scarce resources to be given to the highest-risk patients. Nevertheless, successful implementation is likely to require significant investment in human resources, equipment, and training.