Advertisement

Journal of Gastrointestinal Surgery

, Volume 23, Issue 11, pp 2225–2231 | Cite as

Evaluating the ACS NSQIP Risk Calculator in Primary Pancreatic Neuroendocrine Tumor: Results from the US Neuroendocrine Tumor Study Group

  • Apeksha Dave
  • Eliza W. Beal
  • Alexandra G. Lopez-Aguiar
  • George Poultsides
  • Eleftherios Makris
  • Flavio G. Rocha
  • Zaheer Kanji
  • Sean Ronnekleiv-Kelly
  • Victoria R. Rendell
  • Ryan C. Fields
  • Bradley A. Krasnick
  • Kamran Idrees
  • Paula Marincola Smith
  • Hari Nathan
  • Megan Beems
  • Shishir K. Maithel
  • Timothy M. Pawlik
  • Carl R. Schmidt
  • Mary E. DillhoffEmail author
Original Article

Abstract

Background

In a changing health care environment where patient outcomes will be more closely scrutinized, the ability to predict surgical complications is becoming increasingly important. The American College of Surgeons National Surgical Quality Improvement Program (ACS NSQIP) online risk calculator is a popular tool to predict surgical risk. This paper aims to assess the applicability of the ACS NSQIP calculator to patients undergoing surgery for pancreatic neuroendocrine tumors (PNETs).

Methods

Using the US Neuroendocrine Tumor Study Group (USNET-SG), 890 patients who underwent pancreatic procedures between 1/1/2000–12/31/2016 were evaluated. Predicted and actual outcomes were compared using C-statistics and Brier scores.

Results

The most commonly performed procedure was distal pancreatectomy, followed by standard and pylorus-preserving pancreaticoduodenectomy. For the entire group of patients studied, C-statistics were highest for discharge destination (0.79) and cardiac complications (0.71), and less than 0.7 for all other complications. The Brier scores for surgical site infection (0.1441) and discharge to nursing/rehabilitation facility (0.0279) were below the Brier score cut-off, while the rest were equal to or above and therefore not useful for interpretation.

Conclusion

This work indicates that the ACS NSQIP risk calculator is a valuable tool that should be used with caution and in coordination with clinical assessment for PNET clinical decision-making.

Keywords

ACS NSQIP risk calculator Pancreatic neuroendocrine tumor PNET Pan-NET 

Notes

Authorship

AD, EWB, AGL, GP, EM, FGR, ZK, SRK, VRR, RCF, BAK, KI, PMS, HN, MB, SKM, TMP, CRS, and MD contributed to the conception and design of this work and acquisition of data. EWB, AD, CRS, and MD performed data analysis and interpretation of the data and prepared the manuscript. AD, EWB, AGL, GP, EM, FGR, ZK, SRK, VRR, RCF, BAK, KI, PMS, HN, MB, SKM, TMP, CRS, and MD revised the manuscript critically for important intellectual content, approved the final version to be published, and agree to be accountable for all aspects of the work.

References

  1. 1.
    Ries LAG, Young JL, Keel GE, Eisner MP, Lin YD, Horner M-JD et al. Cancer Survival Among Adults: U.S. SEER Program, 1988-2001. Patient and Tumor Characteristics. 2007.Google Scholar
  2. 2.
    Metz DC, Jensen RT. Gastrointestinal neuroendocrine tumors: pancreatic endocrine tumors. Gastroenterology. 2008;135(5):1469–92. doi: https://doi.org/10.1053/j.gastro.2008.05.047.CrossRefPubMedPubMedCentralGoogle Scholar
  3. 3.
    Fischer L, Kleeff J, Esposito I, Hinz U, Zimmermann A, Friess H et al. Clinical outcome and long-term survival in 118 consecutive patients with neuroendocrine tumours of the pancreas. Br J Surg. 2008;95(5):627–35. doi: https://doi.org/10.1002/bjs.6051.CrossRefPubMedGoogle Scholar
  4. 4.
    McPhee JT et al. Perioperative mortality for pancreatectomy: A national perspective. Ann. Surg. 2007; 246:246–253.CrossRefGoogle Scholar
  5. 5.
    Goldman L, Caldera DL, Nussbaum SR, Southwick FS, Krogstad D, Murray B et al. Multifactorial index of cardiac risk in noncardiac surgical procedures. N Engl J Med. 1977;297(16):845–50. doi: https://doi.org/10.1056/NEJM197710202971601.CrossRefPubMedGoogle Scholar
  6. 6.
    Lee TH, Marcantonio ER, Mangione CM, Thomas EJ, Polanczyk CA, Cook EF et al. Derivation and prospective validation of a simple index for prediction of cardiac risk of major noncardiac surgery. Circulation. 1999;100(10):1043–9.CrossRefGoogle Scholar
  7. 7.
    Hill JS, Zhou Z, Simons JP, Ng SC, McDade TP, Whalen GF et al. A simple risk score to predict in-hospital mortality after pancreatic resection for cancer. Ann Surg Oncol. 2010;17(7):1802–7. doi: https://doi.org/10.1245/s10434-010-0947-x.CrossRefPubMedGoogle Scholar
  8. 8.
    Are C, Afuh C, Ravipati L, Sasson A, Ullrich F, Smith L. Preoperative nomogram to predict risk of perioperative mortality following pancreatic resections for malignancy. J Gastrointest Surg. 2009;13(12):2152–62. doi: https://doi.org/10.1007/s11605-009-1051-z.CrossRefPubMedGoogle Scholar
  9. 9.
    Parikh P, Shiloach M, Cohen ME, Bilimoria KY, Ko CY, Hall BL et al. Pancreatectomy risk calculator: an ACS-NSQIP resource. HPB (Oxford). 2010;12(7):488–97. doi: https://doi.org/10.1111/j.1477-2574.2010.00216.x.CrossRefPubMedPubMedCentralGoogle Scholar
  10. 10.
    Bilimoria KY, Liu Y, Paruch JL, Zhou L, Kmiecik TE, Ko CY et al. Development and evaluation of the universal ACS NSQIP surgical risk calculator: a decision aid and informed consent tool for patients and surgeons. J Am Coll Surg. 2013;217(5):833–42.e1–3. doi: https://doi.org/10.1016/j.jamcollsurg.2013.07.385.CrossRefPubMedGoogle Scholar
  11. 11.
    Kohut A, Orfanelli T, Poggio JL, Gibbon D, Buckley De Meritens A, Richard S. Morbidity and Mortality Risk Assessment in Gynecologic Oncology Surgery Using the American College of Surgeons National Surgical Quality Improvement Program Database. Int J Gynecol Cancer. 2018;28(4):840–7. doi: https://doi.org/10.1097/IGC.0000000000001234.CrossRefPubMedGoogle Scholar
  12. 12.
    Vosler PS, Orsini M, Enepekides DJ, Higgins KM. Predicting complications of major head and neck oncological surgery: an evaluation of the ACS NSQIP surgical risk calculator. J Otolaryngol Head Neck Surg. 2018;47(1):21. doi: https://doi.org/10.1186/s40463-018-0269-8.CrossRefPubMedPubMedCentralGoogle Scholar
  13. 13.
    Beal EW, Saunders ND, Kearney JF, Lyon E, Wei L, Squires MH et al. Accuracy of the ACS NSQIP Online Risk Calculator Depends on How You Look at It: Results from the United States Gastric Cancer Collaborative. Am Surg. 2018;84(3):358–64.PubMedGoogle Scholar
  14. 14.
    Margolick J, Wiseman SM. Risk of major complications following thyroidectomy and parathyroidectomy: Utility of the NSQIP surgical risk calculator. Am J Surg. 2018;215(5):936–41. doi: https://doi.org/10.1016/j.amjsurg.2018.01.006.CrossRefPubMedGoogle Scholar
  15. 15.
    Golan S, Adamsky MA, Johnson SC, Barashi NS, Smith ZL, Rodriguez MV et al. National Surgical Quality Improvement Program surgical risk calculator poorly predicts complications in patients undergoing radical cystectomy with urinary diversion. Urol Oncol. 2018;36(2):77.e1-.e7. doi: https://doi.org/10.1016/j.urolonc.2017.09.015.CrossRefGoogle Scholar
  16. 16.
    Jiang HY, Kohtakangas EL, Asai K & Shum JB. Predictive Power of the NSQIP Risk Calculator for Early Post-Operative Outcomes After Whipple: Experience from a Regional Center in Northern Ontario. J Gastrointest. Cancer. 2018;49:288–294.Google Scholar
  17. 17.
    Beal EW, Lyon E, Kearney J, Wei L, Ethun CG, Black SM et al. Evaluating the American College of Surgeons National Surgical Quality Improvement project risk calculator: results from the U.S. Extrahepatic Biliary Malignancy Consortium. HPB (Oxford). 2017;19(12):1104–11. doi: https://doi.org/10.1016/j.hpb.2017.08.009.CrossRefGoogle Scholar
  18. 18.
    Xia J, Broadhurst DI, Wilson M, Wishart DS. Translational biomarker discovery in clinical metabolomics: an introductory tutorial. Metabolomics. 2013:9, 280–299.CrossRefGoogle Scholar
  19. 19.
    Cohen ME, Ko CY, Bilimoria KY, Zhou L, Huffman K, Wang X et al. Optimizing ACS NSQIP modeling for evaluation of surgical quality and risk: patient risk adjustment, procedure mix adjustment, shrinkage adjustment, and surgical focus. J Am Coll Surg. 2013;217(2):336–46.e1. doi: https://doi.org/10.1016/j.jamcollsurg.2013.02.027.CrossRefGoogle Scholar
  20. 20.
    Merkow RP, Hall BL, Cohen ME, Dimick JB, Wang E, Chow WB et al. Relevance of the c-statistic when evaluating risk-adjustment models in surgery. J Am Coll Surg. 2012;214(5):822–30. doi: https://doi.org/10.1016/j.jamcollsurg.2011.12.041.CrossRefGoogle Scholar
  21. 21.
    Steyerberg EW, Vickers AJ, Cook NR, Gerds T, Gonen M, Obuchowski N et al. Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology. 2010;21(1):128–38. doi: https://doi.org/10.1097/EDE.0b013e3181c30fb2.CrossRefGoogle Scholar
  22. 22.
    Roberts KJ, Hodson J, Mehrzad H, Marudanayagam R, Sutcliffe RP, Muiesan P et al. A preoperative predictive score of pancreatic fistula following pancreatoduodenectomy. HPB (Oxford). 2014;16(7):620–8. doi: https://doi.org/10.1111/hpb.12186.CrossRefGoogle Scholar
  23. 23.
    Yamamoto Y, Sakamoto Y, Nara S, Esaki M, Shimada K, Kosuge T. A preoperative predictive scoring system for postoperative pancreatic fistula after pancreaticoduodenectomy. World J Surg. 2011;35(12):2747–55. doi: https://doi.org/10.1007/s00268-011-1253-x.CrossRefPubMedGoogle Scholar
  24. 24.
    Vin Y, Sima CS, Getrajdman GI, Brown KT, Covey A, Brennan MF et al. Management and outcomes of postpancreatectomy fistula, leak, and abscess: results of 908 patients resected at a single institution between 2000 and 2005. J Am Coll Surg. 2008;207(4):490–8. doi: https://doi.org/10.1016/j.jamcollsurg.2008.05.003.CrossRefPubMedGoogle Scholar
  25. 25.
    Kneuertz PJ, Pitt HA, Bilimoria KY, Smiley JP, Cohen ME, Ko CY et al. Risk of morbidity and mortality following hepato-pancreato-biliary surgery. J Gastrointest Surg. 2012;16(9):1727–35. doi: https://doi.org/10.1007/s11605-012-1938-y.CrossRefPubMedGoogle Scholar

Copyright information

© The Society for Surgery of the Alimentary Tract 2019

Authors and Affiliations

  • Apeksha Dave
    • 1
  • Eliza W. Beal
    • 1
  • Alexandra G. Lopez-Aguiar
    • 2
  • George Poultsides
    • 3
  • Eleftherios Makris
    • 3
  • Flavio G. Rocha
    • 4
  • Zaheer Kanji
    • 4
  • Sean Ronnekleiv-Kelly
    • 5
  • Victoria R. Rendell
    • 5
  • Ryan C. Fields
    • 6
  • Bradley A. Krasnick
    • 6
  • Kamran Idrees
    • 7
  • Paula Marincola Smith
    • 7
  • Hari Nathan
    • 8
  • Megan Beems
    • 8
  • Shishir K. Maithel
    • 2
  • Timothy M. Pawlik
    • 1
  • Carl R. Schmidt
    • 1
  • Mary E. Dillhoff
    • 1
    • 9
    Email author
  1. 1.Division of Surgical OncologyThe Ohio State University Wexner Medical Center and James Comprehensive Cancer CenterColumbusUSA
  2. 2.Division of Surgical Oncology, Department of Surgery, Winship Cancer InstituteEmory UniversityAtlantaUSA
  3. 3.Department of SurgeryStanford UniversityPalo AltoUSA
  4. 4.Department of SurgeryVirginia Mason Medical CenterSeattleUSA
  5. 5.Department of SurgeryUniversity of Wisconsin School of Medicine and Public HealthMadisonUSA
  6. 6.Department of SurgeryWashington University School of MedicineSt. LouisUSA
  7. 7.Division of Surgical Oncology, Department of SurgeryVanderbilt UniversityNashvilleUSA
  8. 8.Division of Hepatopancreatobiliary and Advanced Gastrointestinal Surgery, Department of SurgeryUniversity of MichiganAnn ArborUSA
  9. 9.Department of Surgery, Division of Surgical OncologyThe Ohio State University Wexner Medical CenterColumbusUSA

Personalised recommendations