Meta-analysis of Calibration, Discrimination, and Stratum-Specific Likelihood Ratios for the CRB-65 Score

  • Mark H. EbellEmail author
  • Mary E. Walsh
  • Tom Fahey
  • Maggie Kearney
  • Christian Marchello
Review Paper



The CRB-65 score is recommended as a decision support tool to help identify patients with community-acquired pneumonia (CAP) who can safely be treated as outpatients.


To perform an updated meta-analysis of the accuracy, discrimination, and calibration of the CRB-65 score using a novel approach to calculation of stratum-specific likelihood ratios.


Meta-analysis of accuracy, discrimination, and calibration.


We searched PubMed, Google, previous systematic reviews, and reference lists of included studies. Data was abstracted and quality assessed in parallel by two investigators. The quality assessment used an adaptation of the TRIPOD and PROBAST criteria. Measures of discrimination, calibration, and stratum-specific likelihood ratios are reported.

Key Results

Twenty-nine studies met our inclusion criteria and provided usable data. Most studies were set in Europe, none in North America, and 12 were judged to be at low risk of bias. The pooled estimate of area under the receiver operating characteristic curve was 0.74 (95% CI 0.71–0.77) for all studies. Calibration was good although there was significant heterogeneity; the pooled estimate of the ratio of observed to expected mortality for all studies was 1.04 (95% CI 0.91–1.19). The corresponding values for studies at low risk of bias where patients could be treated as outpatients or inpatients were 0.76 (0.70–0.81) and 0.88 (0.69–1.13). Summary estimates of stratum-specific likelihood ratios for all studies were 0.19 for the low-risk group, 1.1 for the moderate-risk group, and 4.5 for the high-risk group, and 0.13, 1.3, and 5.6 for studies at low risk of bias where patients could be treated as outpatients or inpatients.


The CRB-65 is useful for identifying low-risk patients for outpatient therapy. Given a 4% overall mortality risk, patients classified as low risk by the CRB-65 had an outpatient mortality risk of no more than 0.5%.


community-acquired pneumonia risk prediction models clinical decision rules CRB-65 adults meta-analysis 


Funding Source

Dr. Ebell’s collaboration with Drs. Fahey and Walsh was supported by a 2018/2019 Fulbright Teaching/Research award.

Role of Investigators

Mark Ebell and Tom Fahey were responsible for overall design and conduct of the study. Mark Ebell participated in data abstraction, analysis, and primary authorship of the manuscript. Maggie Kearney, Mary Walsh, and Christian Marchello participated in data abstraction. Mary Walsh performed the random effects meta-analysis of observed/expected ratio and AUC, while Mark Ebell performed meta-analysis of stratum-specific likelihood ratios. Tom Fahey also participated in writing the manuscript and in the analysis, and all authors reviewed and approved the final manuscript.

Compliance with Ethical Standards

Conflict of Interest

The authors declare that they do not have a conflict of interest.

Supplementary material

11606_2019_4869_MOESM1_ESM.docx (67 kb)
ESM 1 (DOCX 66.5 kb)


  1. 1.
    Mandell LA, et al. Infectious Diseases Society of America/American Thoracic Society consensus guidelines on the management of community-acquired pneumonia in adults. Clin Infect Dis. 2007;44(Suppl 2):S27–72.CrossRefGoogle Scholar
  2. 2.
    Fine MJ, et al. A prediction rule to identify low-risk patients with community-acquired pneumonia. N Engl J Med. 1997;336(4):243–50.CrossRefGoogle Scholar
  3. 3.
    Lim WS, et al. Defining community acquired pneumonia severity on presentation to hospital: an international derivation and validation study. Thorax. 2003;58(5):377–82.CrossRefPubMedCentralGoogle Scholar
  4. 4.
    McNally M, et al. Validity of British Thoracic Society guidance (the CRB-65 rule) for predicting the severity of pneumonia in general practice: systematic review and meta-analysis. Br J Gen Pract. 2010;60(579):e423–33.CrossRefPubMedCentralGoogle Scholar
  5. 5.
    Lim WS, et al. BTS guidelines for the management of community acquired pneumonia in adults: update 2009. Thorax. 2009;64(Suppl 3): iii1–55.CrossRefGoogle Scholar
  6. 6.
    Debray TPA, et al. A guide to systematic review and meta-analysis of prediction model performance. BMJ. 2017;356.Google Scholar
  7. 7.
    Buising KL, et al. Identifying severe community-acquired pneumonia in the emergency department: a simple clinical prediction tool. Emerg Med Australas. 2007;19(5):418–26.Google Scholar
  8. 8.
    Ochoa-Gondar O, et al. Validation of the CORB75 (confusion, oxygen saturation, respiratory rate, blood pressure, and age >/= 75 years) as a simpler pneumonia severity rule. Infection. 2014;42(2):371–8.CrossRefGoogle Scholar
  9. 9.
    Kontopantelis E, Reeves D. metaan: Random-effects meta-analysis. Stata J, 2010. 10(3): p. 395–407.CrossRefGoogle Scholar
  10. 10.
    Snell KI, et al. Meta-analysis of prediction model performance across multiple studies: Which scale helps ensure between-study normality for the C-statistic and calibration measures? Stat Methods Med Res. 2017;962280217705678.Google Scholar
  11. 11.
    Higgins JP, Thompson SG. Quantifying heterogeneity in a meta-analysis. Stat Med. 2002;21(11):1539–58.CrossRefGoogle Scholar
  12. 12.
    Barlow G, Nathwani D, Davey P. The CURB65 pneumonia severity score outperforms generic sepsis and early warning scores in predicting mortality in community-acquired pneumonia. Thorax. 2007;62(3):253–9.CrossRefGoogle Scholar
  13. 13.
    Bauer TT, et al. CRB-65 predicts death from community-acquired pneumonia. J Intern Med. 2006;260(1):93–101.CrossRefGoogle Scholar
  14. 14.
    Bont J, et al. Predicting death in elderly patients with community-acquired pneumonia: a prospective validation study reevaluating the CRB-65 severity assessment tool. Arch Intern Med. 2008;168(13):1465–8.CrossRefGoogle Scholar
  15. 15.
    Capelastegui A, et al. Validation of a predictive rule for the management of community-acquired pneumonia. Eur Respir J. 2006;27(1):151–7.CrossRefGoogle Scholar
  16. 16.
    Chalmers JD, Singanayagam A, Hill AT. Systolic blood pressure is superior to other haemodynamic predictors of outcome in community acquired pneumonia. Thorax. 2008;63(8):698–702.CrossRefGoogle Scholar
  17. 17.
    Ewig S, et al. New perspectives on community-acquired pneumonia in 388 406 patients. Results from a nationwide mandatory performance measurement programme in healthcare quality. Thorax. 2009;64(12):1062–9.CrossRefPubMedCentralGoogle Scholar
  18. 18.
    Kruger S, et al. Procalcitonin predicts patients at low risk of death from community-acquired pneumonia across all CRB-65 classes. Eur Respir J. 2008;31(2):349–55.CrossRefGoogle Scholar
  19. 19.
    Man SY, et al. Prospective comparison of three predictive rules for assessing severity of community-acquired pneumonia in Hong Kong. Thorax. 2007;62(4):348–53.CrossRefGoogle Scholar
  20. 20.
    Menendez R, et al. Biomarkers improve mortality prediction by prognostic scales in community-acquired pneumonia. Thorax. 2009;64(7):587–91.CrossRefGoogle Scholar
  21. 21.
    British Thoracic S, et al. Severity assessment criteria recommended by the British Thoracic Society (BTS) for community-acquired pneumonia (CAP) and older patients. Should SOAR (systolic blood pressure, oxygenation, age and respiratory rate) criteria be used in older people? A compilation study of two prospective cohorts. Age Ageing. 2006;35(3):286–91.CrossRefGoogle Scholar
  22. 22.
    Schaaf B, et al. Sepsis severity predicts outcome in community-acquired pneumococcal pneumonia. Eur Respir J. 2007;30(3):517–24.CrossRefGoogle Scholar
  23. 23.
    Schuetz P, et al. Predicting mortality with pneumonia severity scores: importance of model recalibration to local settings. Epidemiol Infect. 2008;136(12):1628–37.CrossRefPubMedCentralGoogle Scholar
  24. 24.
    Zuberi FF, Khan JA. Prospective comparison of prediction rules of mortality risk for CAP in a developing country. Int J Tuberc Lung Dis. 2008;12(4):447–52.Google Scholar
  25. 25.
    Kolditz M, et al. Assessment of oxygenation and comorbidities improves outcome prediction in patients with community-acquired pneumonia with a low CRB-65 score. J Intern Med. 2015;278(2):193–202.CrossRefGoogle Scholar
  26. 26.
    Dwyer R, et al. Improvement of CRB-65 as a prognostic tool in adult patients with community-acquired pneumonia. BMJ Open Respir Res. 2014;1(1):e000038.CrossRefPubMedCentralGoogle Scholar
  27. 27.
    Partouche H, et al. Suspected community-acquired pneumonia in an ambulatory setting (CAPA): a French prospective observational cohort study in general practice. NPJ Prim Care Respir Med. 2015;25:15010.CrossRefPubMedCentralGoogle Scholar
  28. 28.
    Pflug MA, et al. Short-term mortality of adult inpatients with community-acquired pneumonia: external validation of a modified CURB-65 score. Postgrad Med J. 2015;91(1072):77–82.CrossRefGoogle Scholar
  29. 29.
    Kabundji DM, et al. Determining need for hospitalisation: Evaluation of the utility of the CRB-65 score in patients with community-acquired pneumonia presenting to an emergency department. S Afr Med J. 2014;104(11):769–772.CrossRefGoogle Scholar
  30. 30.
    Ewig S, et al. Prediction of in-hospital death from community-acquired pneumonia by varying CRB-age groups. Eur Respir J. 2013;41(4):917–22.CrossRefGoogle Scholar
  31. 31.
    Ochoa Gondar O, et al. Ability of the modified CRB75 severity scale in assessing elderly patients with community acquired pneumonia. Aten Primaria. 2013;45(4):208–15.CrossRefGoogle Scholar
  32. 32.
    Kolditz M, et al. Serum cortisol predicts death and critical disease independently of CRB-65 score in community-acquired pneumonia: a prospective observational cohort study. BMC Infect Dis. 2012;12:90.CrossRefPubMedCentralGoogle Scholar
  33. 33.
    Ochoa-Gondar O, et al. Comparison of three predictive rules for assessing severity in elderly patients with CAP. Int J Clin Pract. 2011;65(11):1165–72.CrossRefGoogle Scholar
  34. 34.
    Lepper PM, et al. Serum glucose levels for predicting death in patients admitted to hospital for community acquired pneumonia: prospective cohort study. BMJ. 2012;344:e3397.CrossRefPubMedCentralGoogle Scholar
  35. 35.
    Arnold FW, et al. Predictive accuracy of the pneumonia severity index vs CRB-65 for time to clinical stability: results from the Community-Acquired Pneumonia Organization (CAPO) International Cohort Study. Respir Med. 2010;104(11):1736–43.CrossRefGoogle Scholar
  36. 36.
    Ronan D, et al. Predicting mortality in patients with community-acquired pneumonia and low CURB-65 scores. Eur J Clin Microbiol Infect Dis. 2010;29(9):1117–24.CrossRefGoogle Scholar
  37. 37.
    Vila Corcoles A, Ochoa Gondar O, Blanco TR. Usefulness of the CRB-65 scale for prognosis assessment of patients 65 years or older with community-acquired pneumonia. Med Clin (Barc). 2010;135(3):97–102.CrossRefGoogle Scholar
  38. 38.
    Tessmer A, et al. Impact of intravenous {beta}-lactam/macrolide versus {beta}-lactam monotherapy on mortality in hospitalized patients with community-acquired pneumonia. J Antimicrob Chemother. 2009;63(5):1025–33.CrossRefGoogle Scholar
  39. 39.
    Francis NA, et al. Severity assessment for lower respiratory tract infections: potential use and validity of the CRB-65 in primary care. Prim Care Respir J. 2012;21(1):65–70.CrossRefGoogle Scholar
  40. 40.
    Nullmann H, et al. External validation of the CURSI criteria (confusion, urea, respiratory rate and shock index) in adults hospitalised for community-acquired pneumonia. BMC Infect Dis. 2014;14:39.CrossRefPubMedCentralGoogle Scholar
  41. 41.
    von Hippel PT. The heterogeneity statistic I(2) can be biased in small meta-analyses. BMC Med Res Methodol. 2015;15:35.CrossRefGoogle Scholar

Copyright information

© Society of General Internal Medicine 2019

Authors and Affiliations

  • Mark H. Ebell
    • 1
    Email author
  • Mary E. Walsh
    • 2
  • Tom Fahey
    • 2
  • Maggie Kearney
    • 1
  • Christian Marchello
    • 1
  1. 1.Department of Epidemiology and Biostatistics, College of Public Health University of GeorgiaAthensUSA
  2. 2.HRB Centre for Primary Care Research, Department of General PracticeRoyal College of Surgeons in IrelandDublinRepublic of Ireland

Personalised recommendations