Skip to main content
SpringerLink
Log in
Book cover

Conference on Artificial Intelligence in Medicine in Europe

AIME 2011: Artificial Intelligence in Medicine pp 230–239Cite as

Repeated Prognosis in the Intensive Care: How Well Do Physicians and Temporal Models Perform?

  • Conference paper

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 6747))

Abstract

Recently, we devised a method to develop prognostic models incorporating patterns of sequential organ failure to predict the eventual hospital mortality at each day of intensive care stay. In this study, we aimed to understand, using a real world setting, how these models perform compared to physicians, who are exposed to additional information than the models. We found a slightly better discriminative ability for physicians (AUC range over days: 0.73-0.83 vs. 0.70-0.80) and a slightly better accuracy for the models (Brier score range: 0.14-0.19 vs. 0.16-0.19). However when we combined both sources of predictions we arrived at a significantly superior discrimination as well as accuracy (AUC range: 0.81-0.88; Brier score range: 0.11-0.15). Our results show that the models and the physicians draw on complementary information that can be best harnessed by combining both prediction sources. Extensive external validation and impact studies are imperative to further investigate the ability of the combined model.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Lucas, P.J., Abu-Hanna, A.: Prognostic methods in medicine. Artif. Intell. Med. 15, 105–119 (1999)

    Article  Google Scholar 

  2. Abu-Hanna, A., Lucas, P.J.: Prognostic models in medicine. AI and statistical approaches. Methods Inf. Med. 40, 1–5 (2001)

    Google Scholar 

  3. Fried, T.R., Bradley, E.H., Towle, V.R., Allore, H.: Understanding the treatment preferences of seriously ill patients. N. Engl. J. Med. 346, 1061–1066 (2002)

    Article  Google Scholar 

  4. Murphy, D.J., Burrows, D., Santilli, S., Kemp, A.W., Tenner, S., Kreling, B., et al.: The influence of the probability of survival on patients’ preferences regarding cardiopulmonary resuscitation. N. Engl. J. Med. 330, 545–549 (1994)

    Article  Google Scholar 

  5. Rocker, G., Cook, D., Sjokvist, P., Weaver, B., Finfer, S., McDonald, E., et al.: Clinician predictions of intensive care unit mortality. Crit. Care Med. 32, 1149–1154 (2004)

    Article  Google Scholar 

  6. Whitecotton, S.M., Sanders, D.E., Norris, K.B.: Improving Predictive Accuracy with a Combination of Human Intuition and Mechanical Decision Aids. Organ Behav. Hum. Decis. Process 76, 325–348 (1998)

    Article  Google Scholar 

  7. Knaus, W.A., Draper, E.A., Wagner, D.P., Zimmerman, J.E.: APACHE II: a severity of disease classification system. Crit. Care Med. 13, 818–829 (1985)

    Article  Google Scholar 

  8. Le Gall, J.R., Lemeshow, S., Saulnier, F.: A new Simplified Acute Physiology Score (SAPS II) based on a European/North American multicenter study. JAMA 270, 2957–2963 (1993)

    Article  Google Scholar 

  9. Lemeshow, S., Teres, D., Klar, J., Avrunin, J.S., Gehlbach, S.H., Rapoport, J.: Mortality Probability Models (MPM II) based on an international cohort of intensive care unit patients. JAMA 270, 2478–2486 (1993)

    Article  Google Scholar 

  10. Christakis, N.A., Lamont, E.B.: Extent and determinants of error in doctors’ prognoses in terminally ill patients: prospective cohort study. BMJ 320, 469–472 (2000)

    Article  Google Scholar 

  11. McClish, D.K., Powell, S.H.: How well can physicians estimate mortality in a medical intensive care unit? Med. Decis. Making 9, 125–132 (1989)

    Article  Google Scholar 

  12. Poses, R.M., Bekes, C., Copare, F.J., Scott, W.E.: The answer to ”What are my chances, doctor?” depends on whom is asked: prognostic disagreement and inaccuracy for critically ill patients. Crit. Care Med. 17, 827–833 (1989)

    Article  Google Scholar 

  13. Christakis, N.A., Iwashyna, T.J.: Attitude and self-reported practice regarding prognostication in a national sample of internists. Arch. Intern. Med. 158, 2389–2395 (1998)

    Article  Google Scholar 

  14. Scholz, N., Basler, K., Saur, P., Burchardi, H., Felder, S.: Outcome prediction in critical care: physicians’ prognoses vs. scoring systems. Eur. J. Anaesthesiol. 21, 606–611 (2004)

    Google Scholar 

  15. Garrouste-Orgeas, M., Montuclard, L., Timsit, J.F., Misset, B., Christias, M., Carlet, J.: Triaging patients to the ICU: a pilot study of factors influencing admission decisions and patient outcomes. Intensive Care Med. 29, 774–781 (2003)

    Article  Google Scholar 

  16. Knaus, W.A., Harrell Jr., F.E., Lynn, J., Goldman, L., Phillips, R.S., Connors Jr., A.F., et al.: he SUPPORT prognostic model. Objective estimates of survival for seriously ill hospitalized adults. Study to understand prognoses and preferences for outcomes and risks of treatments. Ann. Intern. Med. 122, 191–203 (1995)

    Article  Google Scholar 

  17. Christensen, C., Cottrell, J.J., Murakami, J., Mackesy, M.E., Fetzer, A.S., Elstein, A.S.: Forecasting survival in the medical intensive care unit: a comparison of clinical prognoses with formal estimates. Methods Inf. Med. 32, 302–308 (1993)

    Google Scholar 

  18. Marks, R.J., Simons, R.S., Blizzard, R.A., Browne, D.R.: Predicting outcome in intensive therapy units–a comparison of Apache II with subjective assessments. Intensive Care Med. 17, 159–163 (1991)

    Article  Google Scholar 

  19. Poses, R.M., Bekes, C., Copare, F.J., Scott, W.E.: What difference do two days make? The inertia of physicians’ sequential prognostic judgments for critically ill patients. Med. Decis. Making 10, 6–14 (1990)

    Article  Google Scholar 

  20. Poses, R.M., Bekes, C., Winkler, R.L., Scott, W.E., Copare, F.J.: Are two (inexperienced) heads better than one (experienced) head? Averaging house officers’ prognostic judgments for critically ill patients. Arch. Intern. Med. 150, 1874–1878 (1990)

    Article  Google Scholar 

  21. Poses, R.M., McClish, D.K., Bekes, C., Scott, W.E., Morley, J.N.: Ego bias, reverse ego bias, and physicians’ prognostic. Crit. Care Med. 19, 1533–1539 (1991)

    Article  Google Scholar 

  22. McClish, D.K., Powell, S.H.: How well can physicians estimate mortality in a medical intensive care unit? Med. Decis. Making 9, 125–132 (1989)

    Article  Google Scholar 

  23. Brannen, A.L., Godfrey, L.J., Goetter, W.E.: Prediction of outcome from critical illness. A comparison of clinical judgment with a prediction rule. Arch. Intern. Med. 149, 1083–1086 (1989)

    Article  Google Scholar 

  24. Kruse, J.A., Thill-Baharozian, M.C., Carlson, R.W.: Comparison of clinical assessment with APACHE II for predicting mortality risk in patients admitted to a medical intensive care unit. JAMA 260, 1739–1742 (1988)

    Article  Google Scholar 

  25. Sinuff, T., Adhikari, N.K., Cook, D.J., Schunemann, H.J., Griffith, L.E., Rocker, G., et al.: Mortality predictions in the intensive care unit: comparing physicians with scoring systems. Crit. Care Med. 34, 878–885 (2006)

    Article  Google Scholar 

  26. Chang, R.W., Lee, B., Jacobs, S., Lee, B.: Accuracy of decisions to withdraw therapy in critically ill patients: clinical judgment versus a computer model. Crit. Care Med. 17, 1091–1097 (1989)

    Article  Google Scholar 

  27. Meyer, A.A., Messick, W.J., Young, P., Baker, C.C., Fakhry, S., Muakkassa, F., et al.: Prospective comparison of clinical judgment and APACHE II score in predicting the outcome in critically ill surgical patients. J. Trauma 32, 747–753 (1992)

    Article  Google Scholar 

  28. Toma, T., Bosman, R.J., Siebes, A., Peek, N., Abu-Hanna, A.: Learning predictive models that use pattern discovery–a bootstrap evaluative approach applied in organ functioning sequences. J. Biomed. Inform. 43, 578–586 (2010)

    Article  Google Scholar 

  29. Toma, T., Abu-Hanna, A., Bosman, R.J.: Discovery and inclusion of SOFA score episodes in mortality prediction. J. Biomed. Inform. 40, 649–660 (2007)

    Article  Google Scholar 

  30. Toma, T., Abu-Hanna, A., Bosman, R.J.: Discovery and integration of univariate patterns from daily individual organ-failure scores for intensive care mortality prediction. Artif. Intell. Med. 43, 47–60 (2008)

    Article  Google Scholar 

  31. de Jonge, E., Bosman, R.J., van der Voort, P.H., Korsten, H.H., Scheffer, G.J., de Keizer, N.F.: [Intensive care medicine in the Netherlands, 1997-2001. I. Patient population and treatment outcome]. Ned Tijdschr Geneeskd 147, 1013–1017 (2003)

    Google Scholar 

  32. Fretschner, R., Bleicher, W., Heininger, A., Unertl, K.: Patient data management systems in critical care. J. Am. Soc. Nephrol. 12 suppl 17, S83–S86 (2001)

    Google Scholar 

  33. Minne, L., Abu-Hanna, A., de Jonge, E.: Evaluation of SOFA-based models for predicting mortality in the ICU: A systematic review. Crit. Care 12, R161 (2008)

    Article  Google Scholar 

  34. Burnham, K.P., Anderson, D.R.: Model selection and multimodel inference: a practical-theoretic approach. Springer, Heidelberg (2002)

    MATH  Google Scholar 

  35. Frick, S., Uehlinger, D.E., Zuercher Zenklusen, R.M.: Medical futility: predicting outcome of intensive care unit patients by nurses and doctors–a prospective comparative study. Crit. Care Med. 31, 456–461 (2003)

    Article  Google Scholar 

  36. Moons, K.G., Royston, P., Vergouwe, Y., Grobbee, D.E., Altman, D.G.: Prognosis and prognostic research: what, why, and how? BMJ 338, b375 (2009)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Medical Informatics, Academic Medical Center, P.O. Box 22660, 1100 DD, Amsterdam, The Netherlands

    Lilian Minne & Ameen Abu-Hanna

  2. Department of Intensive Care, Leiden University Medical Center, P.O. Box 9600, 2300 RC, Leiden, The Netherlands

    Evert de Jonge

Authors
  1. Lilian Minne
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Evert de Jonge
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ameen Abu-Hanna
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Faculty of Social Sciences, Department of Information Systems, University of Haifa,, Rabin Bldg., 31905, Haifa, Israel

    Mor Peleg

  2. Department of Knowledge Technologies, Jožef Stefan Institute, Jamova 39, 1000, Ljubljana, Slovenia

    Nada Lavrač

  3. Department of Computer Science, University of Verona, Ca’ Vignal 2, strada le Grazie 15, 37134, Verona, Italy

    Carlo Combi

Rights and permissions

Reprints and permissions

Copyright information

© 2011 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Minne, L., de Jonge, E., Abu-Hanna, A. (2011). Repeated Prognosis in the Intensive Care: How Well Do Physicians and Temporal Models Perform?. In: Peleg, M., Lavrač, N., Combi, C. (eds) Artificial Intelligence in Medicine. AIME 2011. Lecture Notes in Computer Science(), vol 6747. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-22218-4_29

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-22218-4_29

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-22217-7

  • Online ISBN: 978-3-642-22218-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation