Introduction

With therapy regimens consisting regularly of two or more immunosuppressive agents, kidney transplant recipients (KTR) are prone to severe infectious complications [1]. Septicemia and pneumonia are among the ten most frequent causes for admission to the emergency department in KTR [2], while pneumonia is the most common life-threatening infection in KTR [3, 4]. In fact, due to improved immunosuppressive regimens and increased life expectancy, the number of KTR presenting with community-acquired pneumonia (CAP) is increasing constantly [5].

Characterizing this population of patients and identifying KTR at risk for severe CAP or death through risk scores is crucial to improve initial management and patient outcomes. However, management recommendations for immunocompromised patients with CAP mostly rely on expert consensus statements since these patients have so far usually been excluded from national guidelines [6,7,8]. Even though new guidelines for solid organ recipients with CAP were published in 2019, risk stratification in these patients remains difficult due to lack of data from clinical trials [9].

Pneumonia-specific risk scores such as CRB-65 [10], CURB-65 [11], DS-CRB-65 [12], PSI (pneumonia severity index) [13], and IDSA/ATS (Infectious Diseases Society of America)/ATS (American Thoracic Society) minor criteria [14] are well established in immunocompetent patients presenting with CAP. Moreover, qSOFA (quick sequential organ failure assessment) [15] and SOFA (sequential organ failure assessment) [16], initially developed to predict sepsis outcome, have been used more frequently to predict CAP severity [17,18,19]. NEWS-2 (national early warning score 2) is the currently recommended score for determining the degree of illness of a patient by the National Health Service (NHS) [20].

In different cohorts of immunocompromised patients with CAP, the prognostic value of CRB-65 and qSOFA was found to be limited. Carrabba et al. showed poor prognostic value of C(U)RB-65 and PSI in patients with immunosuppression (AUC for mortality between 0.55 and 0.64) [21]. Frantz et al. found comparable results in a cohort of 198 immunocompromised patients (AUC for severe ***CAP-CRB-65: 0.63 and qSOFA: 0.69) [19]. While the first cohort did not include any solid organ transplant recipients, the latter included only 18 KTR. In summary, there are no enough data to recommend for or against using risk scores and to choose among them in KTR presenting with CAP.

Therefore, we characterized a cohort of 390 KTR with CAP at our tertiary care center and compared the validity of eight different risk scores for prediction of in-hospital mortality and severe CAP analyzing 310 first CAP episodes in 310 KTR.

Methods

Study population

The study was approved by the ethics committee of the Charité–Universitätsmedizin Berlin (EA1/330/21). We screened our proprietary electronic health record and transplant database TBase [22] for patients with pneumonia, who were treated at Charité–Universitätsmedizin Berlin between 01.01.2006 and 31.03.2022, were at least 18 years old and had a functioning kidney transplant at the time of diagnosis as detailed in Item S1. Next, we reviewed all medical records of the respective 1103 medical cases with suspected CAP to include only patients meeting the CAP definition and none of the exclusion criteria as shown in Table 1, as well as to extract demographic and clinical data detailed in Item S2. The main analysis was performed for the first pneumonia for each patient in our records to ensure statistical independence. Subsequently, we analyzed the recurrent cases in patients with more than one CAP episode.

Table 1 Inclusion and exclusion criteria
Full size table

Outcomes

The primary endpoint was severe pneumonia, a composite endpoint consisting of in-hospital mortality, respiratory failure requiring invasive mechanical ventilation (IMV), acute kidney injury (AKI) requiring kidney replacement therapy (KRT), and need for vasopressor therapy.

The secondary outcomes were in-hospital mortality, 28-day mortality, ICU admission, need for vasopressor therapy, IMV, high-flow nasal cannula (HFNC) or non-invasive mechanical ventilation (NIV), acute kidney injury (AKI) stage according to KDIGO or need for KRT, and persistent impairment of estimated glomerular filtration rate (eGFR) at discharge in comparison to baseline eGFR. To ensure the validity of the data on 28-day mortality, we verified that at least one follow-up visit was performed at our transplant center or by the home nephrologist more than 28 days after the initial hospital admission for patients, who did not experience in-hospital death and whose hospitalization was shorter than 28 days.

Microbiology

Only pathogens identified within the first 7 days after admission were considered to be related to the acute CAP episode and were included in the analysis. Causative pathogens were identified as detailed in Item S3.

Risk scores and ROC analysis

Missing values for predictor variables were either calculated based on other variables available or by performing multiple imputation (MI) as detailed in Item S4. To assess the effect of MI on the results of the subsequent analyses, complete case analysis was performed as a sensitivity analysis. The following eight risk scores were calculated for each patient’s first pneumonia, separately for each of the five imputed datasets.

CRB-65 [10], CURB-65 [11], DS-CRB-65 [12], qSOFA [13], and NEWS-2 [20] were calculated as described elsewhere from the first clinical data available from each patient after admission. It was ensured that no data were included after the endpoint was reached.

SOFA [16], PSI [13], and IDSA [14] for CAP were slightly modified to account for unavailable information as detailed in Item S5. In case of complete case analysis, the scores were calculated only for patients, for whom all necessary information was available.

Statistical analysis

Statistical analysis was performed using R studio 2021.09.2 with R version 4.1.2. as detailed in Item S6. Descriptive analysis was performed using base R and R package psych [23]. Plotting was performed using R package ggplot2 [24]. Multiple imputation was performed using R package mice [25]. ROC analysis was performed using R package pROC [26] and pooling of performance metrics was performed using the function pool_auc from R package psfmi [27].

Results

In total, 1103 KTR cases were screened. After applying all inclusion and exclusion criteria described in Table 1, 390 cases of CAP were retrieved from our database. To ensure statistical independence for each case, we included the first CAP for patients with more than one CAP episode into our main analysis, resulting in 310 cases. In 50 patients, more than one CAP episode occurred, resulting in 80 recurrent CAP episodes. The patient flow is shown in Fig. 1, and comorbidities, transplant-related characteristics, imaging as well as laboratory parameters are summarized in Table 2. The number and distribution of missing values for each risk score are shown in Figures S1–S8.

Fig.1
figure 1

Patient flow diagram. After applying all inclusion and exclusion criteria, 390 cases of confirmed CAP in 310 adult KTR were retrieved from our database. We included the first CAP episode for each patient into the final analysis to ensure statistical independence and applied multiple imputation for missing variables necessary to calculate risk scores. For complete case analysis, risk scores were calculated only when all variables were available, resulting in different sample size depending on the risk score, KTR—kidney transplant recipients, CAP—community-acquired pneumonia

Full size image
Table 2 Patient charcteristics
Full size table

Outcomes

The combined primary endpoint was reached in 48 patients (15.5%); 16 out of 310 patients (5.2%) died in hospital (Fig. 2 and Table 3).

Fig. 2
figure 2

Frequencies of organ failure treatments and in-hospital death qualifying for the primary endpoint. IMV invasive mechanical ventilation, KRT kidney replacement therapy

Full size image
Table 3 Outcomes of community-acquired pneumonia in kidney transplant recipients
Full size table

Among the 310 patients, 43 patients had COVID-19. Patients with COVID-19 developed severe pneumonia more frequently (32.6%; 14/43) than non-COVID-19 patients (12.7%; 34/267; p = 0.011) and had higher rates of secondary endpoints. In-hospital mortality was 18.6% (8/43) vs. 3.0% (8/267; p = 0.014), IMV was performed in 23.3% (10/43) vs. 5.6% (15/267; p = 0.011), vasopressor treatment in 25.6% (11/43) vs. 8.6% (23/267; p = 0.018), and KRT in 23.3% (10/46) vs. 10.5% (28/267, p = 0.066) (Table 3) in COVID-19 vs. non-COVID-19 patients, respectively.

With respect to renal outcomes, 27.1% (84/310) of all patients had no AKI, 49.7% (154/310) stage 1 AKI, 5.5% (17/310) stage 2 AKI, 17.7% (55/310) stage 3 AKI, and 12.3% (38/310) required KRT during the admission. We further analyzed which proportion of patients had persistent impairment of eGFR at discharge. While eGFR was completely restored in the majority of patients (77.1%—239/310), 11.0% (34/310) had eGFR loss of 5–10 ml/min/1.73 m2, 8.7% (27/310) eGFR loss of 10–20 ml/min/1.73 m2, 3.2% (10/310) eGFR loss of > 20 ml/min/1.73m2, and 1.9% (6/323) lost their graft function in-hospital, meaning permanent return to dialysis.

Pathogens

In 64.5% (200/310) of cases, no causative pathogen was identified. In the remaining 110 cases, SARS-CoV-2 (39.0%), Pneumocystis jirovecii (PjP; 14.5%), Streptococcus pneumoniae (7.3%), CMV (5.5%), and influenza A (5.5%) were the most frequent ones (Fig. 3 and Table 4). In the first year after transplantation, PjP and CMV were more common than in the following years (Figures S9 and S10), with discontinuation of prophylactic treatment often preceding infection (Table S1). Recurrent pneumonias show comparable results for causative pathogens with the exception that pneumocystis is not found as causative pathogen in recurrent pneumonia (Figure S11). A more detailed analysis of CMV and PjP pneumonia is shown in Table S1 and Item S7.

Fig. 3
figure 3

Distribution of causative pathogens for the first episode of community-acquired pneumonia in kidney transplant recipients. SARS-CoV-2 severe acute respiratory syndrome coronavirus 2, CMV cytomegalovirus, MRSA methicillin-resistant staphylococcus aureus, MSSA methicillin-susceptible staphylococcus aureus, HSV herpes simplex virus, RSV respiratory syncytial virus

Full size image
Table 4 Causative pathogens isolated in kidney transplant recipients presenting with community-acquired pneumonia
Full size table

Risk scores

Next, we validated eight different risk scores for pneumonia, sepsis, or general risk assessment, namely CRB-65, CURB-65, DS-CRB-65, qSOFA, SOFA, PSI, IDSA/ATS minor criteria, and NEWS-2. For every score, prediction of in-hospital mortality as well as severe pneumonia were assessed by ROC analysis. Instead of using the previously described cutoffs, we assessed threshold-dependent metrics such as sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) based on the Youden index.

For prediction of death, all risk scores achieved a NPV of at least 0.97 with the cutoffs shown in Table 5. SOFA and NEWS-2 showed the highest AUC, with 0.794 (0.679–0.875) and 0.741 (0.574–0.858), respectively (Table 5 and Fig. 4).

Table 5 Predictive performance of eight different risk scores for prediction of in-hospital mortality in kidney transplant recipients hospitalized for community-acquired pneumonia
Full size table
Fig. 4
figure 4

ROC analysis of eight different risk scores for prediction of in-hospital mortality in 310 kidney transplant recipients with community-acquired pneumonia. Variables for the risk scores were assessed up to 48 h after hospital admission, but always before an endpoint occurred. Missing values were imputed performing multiple imputation and the ROC curves for one out of five multiply imputed datasets is shown here

Full size image

For prediction of severe pneumonia, SOFA and NEWS-2 achieved the highest AUC, with 0.823 (0.747–0.880) and 0.784 (0.691–0.855), respectively (Table 6 and Fig. 5). Notably, SOFA showed a significantly higher AUC than all other risk scores. The only exception was NEWS-2, which showed significantly higher AUC than all other risk scores except for SOFA and IDSA/ATS minor criteria.

Table 6 Predictive performance of eight different risk scores for prediction of severe pneumonia in kidney transplant recipients hospitalized for community-acquired pneumonia
Full size table
Fig. 5
figure 5

ROC analysis of eight different risk scores for predicting the occurrence of the primary endpoint. The composite endpoint of invasive mechanical ventilation, vasopressor treatment, dialysis, or in-hospital mortality in 310 kidney transplant recipients with community-acquired pneumonia was assessed up to 48 h after hospital admission, but always before an endpoint occurred. Missing values were imputed performing multiple imputation and the ROC analysis for one out of five multiply imputed datasets is shown here

Full size image

The observed rate of severe pneumonia and in-hospital death for each value of each score are shown in Table S2. To summarize those results, all risk scores were rescaled to [0,1] and plotted against the observed rate of severe pneumonia for each observed score value in Fig. 6. qSOFA, SOFA, and NEWS-2 all run closely to the diagonal, suggesting good calibration. This means that higher score values indicate higher event rate, while the lowest score values indicate an event rate close to 0% and the highest score values indicate event rates close to 100%. The other risk scores slightly overestimate the risk of severe pneumonia for higher score values, but in general show an increase in event rate with higher score values as well.

Fig. 6
figure 6

Percentage of patients with primary endpoint (severe pneumonia) in dependence on risk scores. Scores were rescaled to the unit interval for this purpose. We pooled scores in order to deal with sparsely filled score classes as follows: for DS-CRB-65 ≤ 1, for SOFA ≤ 1 and ≥ 10, for ATS-Minor ≥ 5, for NEWS-2 10–13, and 14–15, and for PSI < 60 and ≥ 180. Missing values were imputed performing multiple imputation and the data from one out of five multiply imputed datasets are shown here

Full size image

We separately analyzed the correlation of each variable included into the risk scores with the primary outcome. In the univariable analysis, we found 15 out of 27 variables to be significantly correlated with the primary outcome: altered mental status, heart rate, respiratory rate, SpO2, need for oxygen supplementation, Horovitz index, sodium, blood glucose, BUN, creatinine, bilirubin, congestive heart failure, pleural effusion, and multilobular infiltrates (Table S3).

Sensitivity analyses

To verify that computational decisions do not affect the results and conclusions, the following sensitivity analyses were performed.

For all scores, comparable or slightly better results were achieved when no multiple imputation, but complete case analysis was performed as shown in Tables S4/S5.

Since patients with COVID-19 were included when meeting the CAP definition above, we analyzed the predictive performance of all eight risk scores separately for COVID vs. Non-COVID patients. We found that SOFA and NEWS-2 still performed best in detecting the primary endpoint with AUC of 0.804 (0.648–0.901) and 0.787 (0.614–0.895) for COVID-19 pneumonia, and 0.843 (0.749–0.906) and 0.790 (0.684–0.867) for non-COVID-19 pneumonia, respectively (Table S6).

Discussion

In the present study, we validated a comprehensive set of eight different risk scores established to describe disease severity or predict outcomes in pneumonia and sepsis in a large cohort of KTR hospitalized for CAP. We compared the discriminative power with respect to in-hospital mortality and severe pneumonia.

We included risk scores developed to predict CAP outcomes such as CRB-65, CURB-65, DS-CRB-65, PSI, and IDSA/ATS minor criteria, and added SOFA, qSOFA and NEWS-2. Although SOFA as well as its simplified version qSOFA were initially defined to describe the sequence of complications in distinct organs in sepsis, they have been widely used to predict mortality and severe disease course in CAP [17, 18, 28]. Similarly, early warning scores such as NEWS have been investigated as risk stratification tools for CAP, with NEWS-2 being extensively studied within patients with COVID-pneumonia in the last years [29,30,31,32].

ROC analysis revealed that SOFA and NEWS-2 discriminate KTR at risk for severe pneumonia significantly better than the other risk scores investigated. This finding can be explained by three clinically important observations:

First, in contrast to immunocompetent patients [17, 28, 33], age did not significantly correlate with severe pneumonia in our analysis. Accordingly, risk scores containing an age criterion, such as CRB-65, predicted the occurrence of severe pneumonia with less accuracy than risk scores containing similar amounts of variables without an age criterion, such as qSOFA. In comparison to immunocompetent patients with CAP in Germany, the median age of our study population was considerably lower (59 vs. 76 years) [33], which may further explain why age-dependent scores like CRB-65 perform worse in our cohort. This is in line with a secondary analysis from a recent international multi-center study, where immunocompromised patients were significantly younger than immunocompetent patients with CAP [34]. As described before, immunosuppression might facilitate the development of severe pneumonia courses with poor prognosis at a younger age [21, 35]. Therefore, risk scores not relying on age might be preferable in immunocompromised patients with pneumonia [19], which might be applicable for KTR hospitalized for CAP as well.

Second, scores including granular information on pulmonary changes (IDSA/ATS-Minor criteria, PSI, NEWS-2, SOFA) as well as extrapulmonary organ failure (SOFA) performed better, except for PSI. The latter may be explained by extensive imputation, which was necessary for PSI. It is not surprising that SOFA performed best in identifying severe pneumonia, since increases in creatinine and decreases in mean arterial pressure and Horovitz index regularly precede each single outcome composing the primary endpoint.

Third, NEWS-2 showed comparable discrimination as SOFA by only including clinical variables. A possible explanation is that NEWS-2 assesses clinical variables such as respiratory rate, temperature, and blood pressure with the greatest granularity. This might be relevant, since clinical characteristics of immunocompromised patients may differ from immunocompetent patients with CAP [35]. For example, temperature changes indicating systemic infection may be less pronounced in KTR with CAP due to immunosuppression. As opposed to PSI or IDSA/ATS-Minor criteria detecting only large temperature deviations, NEWS-2 might be better suited in this cohort.

To date, only a few studies have investigated risk scores in immunocompromised patients with CAP. In line with our results, other authors found a moderate prognostic value of CRB-65 and qSOFA in immunocompromised patients with pneumonia. Frantz et al. found similar AUC of 0.630 for CRB-65 and 0.688 for qSOFA in the prediction of severe CAP in a cohort of 198 immunocompromised patients, including 18 KTR [19]. Carrabba et al. found a lower AUC of 0.57 for CRB-65 than in our study, but comparable AUC of 0.68 for PSI and 0.62 for CURB-65 when predicting mortality in immunocompromised patients with pneumonia [21].

Reduced performance of qSOFA compared to SOFA has also been shown in the large German cohort PROGRESS predicting severe CAP in immunocompetent patients with CAP [17]. In line with our results, improvement of CRB-65 by adding oxygenation and comorbidities (DS-CRB-65) was observed in the CAPNETZ cohort of immunocompetent German CAP patients [37].

While AUC is a suitable summary statistic with respect to discrimination, practical benefit of risk scores depends on threshold- and incidence-dependent metrics such as PPV and NPV. Standard pneumonia scores like CRB-65 have initially been developed to predict mortality. The threshold of CRB-65 ≥ 1 to indicate hospital admission is chosen to achieve high NPV, so that patients with the lowest score have a very low risk of mortality and can in most cases receive outpatient treatment.

In our analysis, all investigated scores had NPV for in-hospital mortality above 97%. This is due to a low mortality rate of 5.2% and does not suffice to identify patients with low mortality risk for most scores. Only zero points in CURB-65, SOFA, or IDSA/ATS-Minor criteria indicated 0% risk of mortality in our cohort. Correspondingly, only zero points in CURB-65 or IDSA/ATS-Minor criteria indicated 0% risk for development of severe pneumonia. One could argue that for patients with zero points in CURB-65 or IDSA/ATS-Minor criteria, outpatient treatment could be considered due to a very low risk of severe disease. For the other scores, even patients with low score values develop severe pneumonia in a relevant proportion and should be hospitalized and closely monitored.

Furthermore, all the scores were calibrated with respect to the primary endpoint. This means that for the lowest score values, an event rate close to 0% was observed, and for higher score values, a proportionally higher event rate was observed. The latter was most pronounced for SOFA, NEWS-2, and qSOFA, for which the highest score values indicated a 100% risk of severe pneumonia. The other scores overestimate the risk of severe pneumonia to a different extent.

Previous research on risk scores for CAP in immunocompromised patients was mostly performed in heterogeneous groups in terms of underlying disease, treatment regimen, and severity of immunosuppression. Moreover, controversy remains regarding the conditions to be included in the definition of immunocompromised patients with CAP [5]. Hence, studying CAP in a distinct cohort of KTR can lead to more reliable results than studying patients receiving different types of immunosuppressive therapy.

To our knowledge, this is the first study to comprehensively assess the prognostic value of risk scores in a large cohort of KTR hospitalized for CAP.

Limitations

Generally, due to the retrospective and single-center design, the results of this study remain explorative and hypothesizing and need replication within prospective multi-center cohorts.

While many risk scores for CAP have been developed to predict mortality, we chose severe CAP as our primary endpoint for two reasons: (1) due to the small number of deaths, the estimation of discrimination would be less reliable for death as primary endpoint, (2) every item of our composite endpoint is of high importance for KTR with CAP, and intensified management strategies have been shown to improve clinical outcomes in immunocompetent patients [38], as well as in immunocompromised patients with CAP, who are critically ill [39, 40].

Consequently, the exact values for AUC and threshold-dependent metrics, such as sensitivity and specificity, are less reliable for in-hospital death than for the primary endpoint.

Owing to missing values for all risk scores, we used multiple imputations to perform a direct comparison of all risk scores. While sensitivity analysis showed that complete case analysis yielded comparable results in general, this was not true for PSI, where imputation was necessary for most patients. This is due to missing pH values in a large proportion of patients. Hence, PSI results must be interpreted carefully.

To reduce the amount of missing data, we used the earliest value available for each variable from the first 48 h after hospital admission for prediction. We ensured that no data after the occurrence of any endpoint of interest were included into the prediction to balance data recovery and the validity of our results.

In contrast to a recently published study [41], identification of causal pathogens was not the primary objective. The low overall detection rate of 36% might be partially explained by the retrospective study design and lack of a systematic approach to microbiological sampling.

In line with recent study results in KTR with CAP [41], we found a surprisingly low mortality rate, which might be further explained by the exclusion of patients with documented treatment restrictions in our study.

As described above, patients with COVID-pneumonia had a more severe disease course than those with non-COVID-pneumonia with respect to all relevant outcome measures, which has been described in several cohorts of KTR. Since SARS-CoV-2 will probably continue to be an important cause of CAP, especially in immunosuppressed patients, we included COVID-pneumonia in our analysis. Subgroup analysis showed differences in the AUC for most scores when used for COVID vs. non-COVID CAP. Nevertheless, NEWS-2 and SOFA-Score showed superior discrimination in predicting severe CAP for both COVID and non-COVID CAP.

Conclusion

SOFA and NEWS-2, assessed in the first 48 h after hospital admission due to CAP, are best suited to identify KTR at risk for the development of severe CAP. CRB-65 should not be used to guide outpatient treatment in KTR, since there is a 7% risk for the development of severe pneumonia even in patients with a score of zero.