Abstract
Risk stratification of patients with gastrointestinal bleeding (GIB) is recommended, but current risk assessment tools have variable performance. Machine learning (ML) has promise to improve risk assessment. We performed a systematic review to evaluate studies utilizing ML techniques for GIB. Bibliographic databases and conference abstracts were searched for studies with a population of overt GIB that used an ML algorithm with outcomes of mortality, rebleeding, hemostatic intervention, and/or hospital stay. Two independent reviewers screened titles and abstracts, reviewed full-text studies, and extracted data from included studies. Risk of bias was assessed with an adapted Quality in Prognosis Studies tool. Area under receiver operating characteristic curves (AUCs) were the primary assessment of performance with AUC ≥ 0.80 predefined as an acceptable threshold of good performance. Fourteen studies with 30 assessments of ML models met inclusion criteria. No study had low risk of bias. Median AUC reported in validation datasets for predefined outcomes of mortality, intervention, or rebleeding was 0.84 (range 0.40–0.98). AUCs were higher with artificial neural networks (median 0.93, range 0.78–0.98) than other ML models (0.81, range 0.40–0.92). ML performed better than clinical risk scores (Glasgow-Blatchford, Rockall, Child–Pugh, MELD) for mortality in upper GIB. Limitations include heterogeneity of ML models, inconsistent comparisons of ML models with clinical risk scores, and high risk of bias. ML generally provided good–excellent prognostic performance in patients with GIB, and artificial neural networks tended to outperform other ML models. ML was better than clinical risk scores for mortality in upper GIB.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Peery AF, Crockett SD, Barritt AS, et al. Burden of gastrointestinal, liver, and pancreatic diseases in the United States. Gastroenterology. 2015;149:e1733.
Barkun AN, Bardou M, Kuipers EJ, et al. International consensus upper gastrointestinal bleeding conference G: International consensus recommendations on the management of patients with nonvariceal upper gastrointestinal bleeding. Ann Intern Med. 2010;152:101–113.
Laine L, Jensen DM. Management of patients with ulcer bleeding. Am J Gastroenterol. 2012;107:345–360.
Gralnek IM, Dumonceau JM, Kuipers EJ, et al. Diagnosis and management of nonvariceal upper gastrointestinal hemorrhage: European society of gastrointestinal endoscopy (esge) guideline. Endoscopy. 2015;47:a1–46.
Strate LL, Gralnek IM. Acg clinical guideline: Management of patients with acute lower gastrointestinal bleeding. Am J Gastroenterol. 2016;111:755.
Rockall TA, Logan RF, Devlin HB, Northfield TC. Risk assessment after acute upper gastrointestinal haemorrhage. Gut. 1996;38:316–321.
Blatchford O, Murray WR, Blatchford M. A risk score to predict need for treatment for upper-gastrointestinal haemorrhage. Lancet. 2000;356:1318–1321.
Saltzman JR, Tabak YP, Hyett BH, Sun X, Travis AC, Johannes RS. A simple risk score accurately predicts in-hospital mortality, length of stay, and cost in acute upper gi bleeding. Gastrointest Endosc. 2011;74:1215–1224.
Strate LL, Saltzman JR, Ookubo R, Mutinga ML, Syngal S. Validation of a clinical prediction rule for severe acute lower intestinal bleeding. Am J Gastroenterol. 2005;100:1821–1827.
Velayos FS, Williamson A, Sousa KH, et al. Early predictors of severe lower gastrointestinal bleeding and adverse outcomes: A prospective study. Clin Gastroenterol Hepatol. 2004;2:485–490.
Newman J, Fitzgerald JE, Gupta S, von Roon AC, Sigurdsson HH, Allen-Mersh TG. Outcome predictors in acute surgical admissions for lower gastrointestinal bleeding. Colorectal Dis. 2012;14:1020–1026.
Aoki T, Nagata N, Shimbo T, et al. Development and validation of a risk scoring system for severe acute lower gastrointestinal bleeding. Clin Gastroenterol Hepatol. 2016;14:e1562.
Stanley AJ, Ashley D, Dalton HR, et al. Outpatient management of patients with low-risk upper-gastrointestinal haemorrhage: Multicentre validation and prospective evaluation. Lancet. 2009;373:42–47.
Abu-Mostafa YS, Magdon-Ismail M, Lin H. Learning from data: A short course. United States of America; 2012.
Mitchell TM. Machine Learning. New York: McGraw-Hill; 1997.
Wilson FP, Shashaty M, Testani J, et al. Automated, electronic alerts for acute kidney injury: A single-blind, parallel-group, randomised controlled trial. Lancet. 2015;385:1966–1974.
Andrew W, Albert TY, April SL, Ralph G, Vanja CD, Dexter H. Development and validation of an electronic health record—based machine learning model to estimate delirium risk in newly hospitalized patients without known cognitive impairment. JAMA Netw Open. 2018;1(4):e181018.
Shimabukuro DW, Barton CW, Feldman MD, Mataraso SJ, Das R. Effect of a machine learning-based severe sepsis prediction algorithm on patient survival and hospital length of stay: A randomised clinical trial. BMJ Open Respir Res. 2017;4:e000234.
Moher D, Liberati A, Tetzlaff J, Altman DG, Group P. Preferred reporting items for systematic reviews and meta-analyses: The prisma statement. PLoS Med. 2009;6:e1000097.
Hayden JA, van der Windt DA, Cartwright JL, Cote P, Bombardier C. Assessing bias in studies of prognostic factors. Ann Intern Med. 2013;158:280–286.
Moons KG, de Groot JA, Bouwmeester W, et al. Critical appraisal and data extraction for systematic reviews of prediction modelling studies: The charms checklist. PLoS Med. 2014;11:e1001744.
Zarshenas S, Tam L, Colantonio A, Alavinia SM, Cullen N. Predictors of discharge destination from acute care in patients with traumatic brain injury. BMJ Open. 2017;7:e016694.
Mandrekar JN. Receiver operating characteristic curve in diagnostic test assessment. J Thorac Oncol. 2010;5:1315–1316.
Ali A, Swingland J, Choi CH, et al. OC-143 artificial neural network for the risk stratification of acute upper gastrointestinal bleeding: Multicentre comparative analysis vs the Glasgow Blatchford and Rockall scores. Gut. 2012;61:A62–A62.
Augustin S, Muntaner L, Altamirano JT, et al. Predicting early mortality after acute variceal hemorrhage based on classification and regression tree analysis. Clin Gastroenterol Hepatol. 2009;7:1347–1354.
Ayaru L, Ypsilantis PP, Nanapragasam A, et al. Prediction of outcome in acute lower gastrointestinal bleeding using gradient boosting. PLoS One. 2015;10:e0132485.
Choi CH, Swingland J, Ali A, Bose S, Ayaru L. Assessing risk of adverse outcome in acute lower gastrointestinal bleeding: Artificial neural network vs sign guidelines and bleed score. Gut. 2012;61:A156–A157.
Chu A, Ahn H, Halwan B, et al. A decision support system to facilitate management of patients with acute gastrointestinal bleeding. Artif Intell Med. 2008;42:247–259.
Das A, Ben-Menachem T, Cooper GS, et al. Prediction of outcome in acute lower-gastrointestinal haemorrhage based on an artificial neural network: Internal and external validation of a predictive model. Lancet (London, England). 2003;362:1261–1266.
Das A, Ben-Menachem T, Farooq FT, et al. Artificial neural network as a predictive instrument in patients with acute nonvariceal upper gastrointestinal hemorrhage. Gastroenterology. 2008;134:65–74.
Grossi E, Marmo R, Intraligi M, Buscema M. Artificial neural networks for early prediction of mortality in patients with non variceal upper gi bleeding (ugib). Biomed Inf Insights. 2008;1:7–19.
Loftus TJ, Brakenridge SC, Croft CA, et al. Neural network prediction of severe lower intestinal bleeding and the need for surgical intervention. J Surg Res. 2017;212:42–47.
Lyles T, Elliott A, Rockey DC. A risk scoring system to predict in-hospital mortality in patients with cirrhosis presenting with upper gastrointestinal bleeding. J Clin Gastroenterol. 2014;48:712–720.
Rotondano G, Cipolletta L, Grossi E, et al. Italian registry on upper gastrointestinal B: Artificial neural networks accurately predict mortality in patients with nonvariceal upper gi bleeding. Gastrointest Endosc. 2011;73:226.e211.
Thon K, Stoltzing H, Ohmann C, Lorenz W, Roher HD. Decision-making and clinical problem solving in upper gastrointestinal bleeding. Theor Surg. 1988;2:185–198.
Lee HH, Park JM, Han S, et al. A simplified prognostic model to predict mortality in patients with acute variceal bleeding. Dig Liver Dis. 2018;50:247–253.
D’Amico G, De Franchis R. Upper digestive bleeding in cirrhosis. Post-therapeutic outcome and prognostic indicators. Hepatology. 2003;38:599–612.
Stanley AJ, Laine L, Dalton HR, et al. International gastrointestinal bleeding C: Comparison of risk scoring systems for patients presenting with upper gastrointestinal bleeding: International multicentre prospective study. BMJ. 2017;356:i6432.
Laursen SB, Hansen JM. Schaffalitzky de Muckadell OB: The glasgow blatchford score is the most accurate assessment of patients with upper gastrointestinal hemorrhage. Clin Gastroenterol Hepatol. 2012;10:e1131.
Rajkomar A, Oren E, Chen K, et al. Scalable and accurate deep learning with electronic health records. npj Dig Med. 2018;1:18.
Esteva A, Kuprel B, Novoa RA, et al. Dermatologist-level classification of skin cancer with deep neural networks. Nature. 2017;542:115.
Goldstein BA, Navar AM, Pencina MJ, Ioannidis JP. Opportunities and challenges in developing risk prediction models with electronic health records data: A systematic review. J Am Med Inform Assoc. 2017;24:198–208.
Waljee AK, Joyce JC, Wang S, et al. Algorithms outperform metabolite tests in predicting response of patients with inflammatory bowel disease to thiopurines. Clin Gastroenterol Hepatol. 2010;8:143–150.
Shah ND, Steyerberg EW, Kent DM. Big data and predictive analytics: Recalibrating expectations. JAMA. 2018;320:27–28.
Funding
Grant support for this study was obtained from the National Institutes of Health T32 DK007017 (D.S.)
Author information
Authors and Affiliations
Contributions
Dennis Shung M.D. took part in study concept and design, data acquisition, data analysis and interpretation, and drafting of manuscript. Michael Simonov M.D. performed data acquisition, data analysis and interpretation, and critical revision of manuscript. Mark Gentry, M.A., M.L.S., conducted study design, data acquisition, and critical revision of manuscript. Benjamin Au, M.S., carried out data analysis and interpretation, technical support with machine learning expertise, and critical revision of manuscript. Loren Laine M.D. is involved in study concept and design, data analysis and interpretation, and drafting of manuscript.
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Shung, D., Simonov, M., Gentry, M. et al. Machine Learning to Predict Outcomes in Patients with Acute Gastrointestinal Bleeding: A Systematic Review. Dig Dis Sci 64, 2078–2087 (2019). https://doi.org/10.1007/s10620-019-05645-z
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10620-019-05645-z