Advertisement

Quality of MBSAQIP data: bad luck, or lack of QA plan?

  • K. K. NoyesEmail author
  • A. A. Myneni
  • S. D. Schwaitzberg
  • A. B. Hoffman
2019 SAGES Oral
  • 10 Downloads

Abstract

Background

National clinical registries are commonly used in clinical research, quality improvement, and health policy. However, little is known about methodological challenges associated with these registry analyses that could limit their impact and compromise patient safety. This study examined the quality of Metabolic and Bariatric Surgery Accreditation and Quality Improvement Program (MSBASQIP) data to assess its usability potential and improve data collection methodologies.

Methods

We developed a single flat file (n = 168,093) using five subsets (Main, BMI, Readmission, Reoperation, and Intervention) of the 2015 MBSAQIP Participant User Data File (PUF). Logic and validity tests included (1) individual profiles of patient’s body mass index (BMI) changes over time, (2) individual patient care pathways, and (3) correlation analysis between variable pairs associated with the same clinical encounters.

Results

8888 (5.3%) patients did not have postoperative weight/BMI data; 20% of patients had different units for preoperative and postoperative weights. Postoperative weight measurements ranged between − 71 and 132% of preoperative weight. There were 325 (3.7%) hospital readmissions reported on the day of or day after MBS. The self-reporting of “emergency” vs. “planned” interventions did not correlate with the type of procedure and its indication. Up to 20% of data could potentially be unused for analysis due to data quality issues.

Conclusions

Our analysis revealed various data quality issues in the 2015 MBSAQIP PUF related to completeness, accuracy, and consistency. Since information on where the surgery was performed is lacking, it is not possible to conclude whether these issues represent data errors, patient outliers, or inappropriate care. Including automated data checks and biomedical informatics oversight, standardized coding for complications, additional de-identified facility and provider information, and training/mentorship opportunities in data informatics for all researchers who get access to the data have been shown to be effective in improving data quality and minimizing patient safety concerns.

Keywords

MBSAQIP PUF Data quality Bariatric surgery Surgical outcomes 

Notes

Acknowledgements

The authors would like to thank Monami Majumder and Evan Kessler for their assistance with the preparation of this manuscript.

Funding

This manuscript was in part funded by Surgical Outcomes and Research (UB SOAR) program at the Department of Surgery, Jacobs School of Medicine and Biomedical Sciences, University at Buffalo, Buffalo, New York.

Compliance with ethical standards

Disclosures

The earlier version of the study was presented at the 2019 SAGES Annual Meeting. Dr. Hoffman is a paid consultant for Ethicon US, LLC (not related to the study). Dr. Steven Schwaitzberg is a paid consultant for Nu View Surgical, Acuity Bio, Activ Surgical, Human Extensions, Levitra Magnetics and Arch Therapeutics (not related to the study). K. K. Myneni, A. A. Noyes, S. D. Schwaitzberg, and A. B. Hoffman declare that they have no conflict of interest or financial ties to disclose.

References

  1. 1.
    Shiloach M, Frencher SK Jr, Steeger JE, Rowell KS, Bartzokis K, Tomeh MG, Richards KE, Ko CY, Hall BL (2010) Toward robust information: data quality and inter-rater reliability in the American College of Surgeons National Surgical Quality Improvement Program. J Am Coll Surg 210:6–16CrossRefGoogle Scholar
  2. 2.
    Telem DA, Dimick JB (2018) Practical guide to surgical data sets: Metabolic and Bariatric Surgery Accreditation and Quality Program (MBSAQIP). JAMA Surg 153:766–767CrossRefGoogle Scholar
  3. 3.
    Brethauer SA, Kim J, el Chaar M, Papasavas P, Eisenberg D, Rogers A, Ballem N, Kligman M, Kothari S, Committee ACI (2015) Standardized outcomes reporting in metabolic and bariatric surgery. Surg Obes Relat Dis 11:489–506CrossRefGoogle Scholar
  4. 4.
    Ehrenstein V, Nielsen H, Pedersen AB, Johnsen SP, Pedersen L (2017) Clinical epidemiology in the era of big data: new opportunities, familiar challenges. Clin Epidemiol 9:245–250CrossRefGoogle Scholar
  5. 5.
    Kruse CS, Goswamy R, Raval Y, Marawi S (2016) Challenges and opportunities of big data in health care: a systematic review. JMIR Med Inform 4:e38CrossRefGoogle Scholar
  6. 6.
    Ardila-Gatas J, Sharma G, Lloyd SJ, Khorgami Z, Tu C, Schauer PR, Brethauer SA, Aminian A (2019) A nationwide safety analysis of discharge on the first postoperative day after bariatric surgery in selected patients. Obes Surg 29:15–22CrossRefGoogle Scholar
  7. 7.
    Berger ER, Clements RH, Morton JM, Huffman KM, Wolfe BM, Nguyen NT, Ko CY, Hutter MM (2016) The impact of different surgical techniques on outcomes in laparoscopic sleeve gastrectomies: the first report from the metabolic and bariatric surgery accreditation and quality improvement program (MBSAQIP). Ann Surg 264:464–473CrossRefGoogle Scholar
  8. 8.
    Chaar ME, Lundberg P, Stoltzfus J (2018) Thirty-day outcomes of sleeve gastrectomy versus Roux-en-Y gastric bypass: first report based on Metabolic and Bariatric Surgery Accreditation and Quality Improvement Program database. Surg Obes Relat Dis 14:545–551CrossRefGoogle Scholar
  9. 9.
    Clapp B, Hahn J, Dodoo C, Guerra A, de la Rosa E, Tyroch A (2018) Evaluation of the rate of marginal ulcer formation after bariatric surgery using the MBSAQIP database. Surg Endosc.  https://doi.org/10.1007/s00464-018-6468-6 Google Scholar
  10. 10.
    Daigle CR, Brethauer SA, Tu C, Petrick AT, Morton JM, Schauer PR, Aminian A (2018) Which postoperative complications matter most after bariatric surgery? Prioritizing quality improvement efforts to improve national outcomes. Surg Obes Relat Dis 14:652–657CrossRefGoogle Scholar
  11. 11.
    Dang JT, Switzer N, Delisle M, Laffin M, Gill R, Birch DW, Karmali S (2019) Predicting venous thromboembolism following laparoscopic bariatric surgery: development of the BariClot tool using the MBSAQIP database. Surg Endosc 33:821–831CrossRefGoogle Scholar
  12. 12.
    Dang JT, Switzer NJ, Sun WYL, Raghavji F, Birch DW, Karmali S (2018) Evaluating the safety of intragastric balloon: an analysis of the Metabolic and Bariatric Surgery Accreditation and Quality Improvement Program. Surg Obes Relat Dis 14:1340–1347CrossRefGoogle Scholar
  13. 13.
    Demeusy A, Sill A, Averbach A (2018) Current role of staple line reinforcement in 30-day outcomes of primary laparoscopic sleeve gastrectomy: an analysis of MBSAQIP data, 2015-2016 PUF. Surg Obes Relat Dis 14:1454–1461CrossRefGoogle Scholar
  14. 14.
    El Chaar M, Stoltzfus J (2018) Assessment of sleeve gastrectomy surgical technique: first look at 30-day outcomes based on the MBSAQIP database. J Am Coll Surg 227:564–572CrossRefGoogle Scholar
  15. 15.
    El Chaar M, Stoltzfus J, Melitics M, Claros L, Zeido A (2018) 30-Day outcomes of revisional bariatric stapling procedures: first report based on MBSAQIP data registry. Obes Surg 28:2233–2240CrossRefGoogle Scholar
  16. 16.
    Fazl Alizadeh R, Li S, Inaba CS, Dinicu AI, Hinojosa MW, Smith BR, Stamos MJ, Nguyen NT (2019) Robotic versus laparoscopic sleeve gastrectomy: a MBSAQIP analysis. Surg Endosc 33:917–922CrossRefGoogle Scholar
  17. 17.
    Feng X, Andalib A, Brethauer SA, Schauer PR, Aminian A (2018) How safe is bariatric surgery in patients with class I obesity (body mass index 30–35 kg/m(2))? Surg Obes Relat Dis 15:253–260CrossRefGoogle Scholar
  18. 18.
    Gondal AB, Hsu CH, Khoubyari R, Ghaderi I (2019) Development of a bariatric surgery specific risk assessment tool for perioperative myocardial infarction. Surg Obes Relat Dis 15:462–468CrossRefGoogle Scholar
  19. 19.
    Haskins IN, Ju T, Whitlock AE, Rivas L, Amdur RL, Lin PP, Vaziri K (2018) Older age confers a higher risk of 30-day morbidity and mortality following laparoscopic bariatric surgery: an analysis of the metabolic and bariatric surgery quality improvement program. Obes Surg 28:2745–2752CrossRefGoogle Scholar
  20. 20.
    Inaba CS, Koh CY, Sujatha-Bhaskar S, Zhang L, Nguyen NT (2018) Same-day discharge after laparoscopic Roux-en-Y Gastric Bypass: an analysis of the metabolic and bariatric surgery accreditation and quality improvement program database. J Am Coll Surg 226:868–873CrossRefGoogle Scholar
  21. 21.
    Janik MR, Mustafa RR, Rogula TG, Alhaj Saleh A, Abbas M, Khaitan L (2018) Safety of laparoscopic sleeve gastrectomy and Roux-en-Y gastric bypass in elderly patients—analysis of the MBSAQIP. Surg Obes Relat Dis 14:1276–1282CrossRefGoogle Scholar
  22. 22.
    Janik MR, Mustafa RR, Rogula TG, Alhaj Saleh A, Abbas M, Khaitan L (2018) Application of HARM score to measure surgical quality and outcomes in bariatric patients. Obes Surg 28:2815–2819CrossRefGoogle Scholar
  23. 23.
    Janik MR, Rogula TG, Mustafa RR, Alhaj Saleh A, Khaitan L (2019) Safety of revision sleeve gastrectomy compared to Roux-Y Gastric bypass after failed gastric banding: analysis of the MBSAQIP. Ann Surg 269:299–303CrossRefGoogle Scholar
  24. 24.
    Jurgensen JA, Reidt W, Kellogg T, Mundi M, Shah M, Collazo Clavell ML (2019) Impact of patient attrition from bariatric surgery practice on clinical outcomes. Obes Surg 29:579–584CrossRefGoogle Scholar
  25. 25.
    Kanters AE, Shubeck SP, Varban OA, Dimick JB, Telem DA (2018) Incidence and efficacy of stent placement in leak management after bariatric surgery: an MBSAQIP analysis. Ann Surg.  https://doi.org/10.1097/SLA.0000000000003023 Google Scholar
  26. 26.
    Kumar SB, Hamilton BC, Wood SG, Rogers SJ, Carter JT, Lin MY (2018) Is laparoscopic sleeve gastrectomy safer than laparoscopic gastric bypass? a comparison of 30-day complications using the MBSAQIP data registry. Surg Obes Relat Dis 14:264–269CrossRefGoogle Scholar
  27. 27.
    Lundberg PW, Stoltzfus J, El Chaar M (2019) 30-day outcomes of robot-assisted versus conventional laparoscopic sleeve gastrectomy: first analysis based on MBSAQIP. Surg Obes Relat Dis 15:1–7CrossRefGoogle Scholar
  28. 28.
    Lundberg PW, Wolfe S, Seaone J, Stoltzfus J, Claros L, El Chaar M (2018) Robotic gastric bypass is getting better: first results from the Metabolic and Bariatric Surgery Accreditation and Quality Improvement Program. Surg Obes Relat Dis 14:1240–1245CrossRefGoogle Scholar
  29. 29.
    Rahman U, Docimo S, Pryor AD, Bates A, Obeid NR, Spaniolas K (2018) Routine contrast imaging after bariatric surgery and the effect on hospital length of stay. Surg Obes Relat Dis 14:517–520CrossRefGoogle Scholar
  30. 30.
    Sebastian R, Howell MH, Chang KH, Adrales G, Magnuson T, Schweitzer M, Nguyen H (2018) Robot-assisted versus laparoscopic Roux-en-Y gastric bypass and sleeve gastrectomy: a propensity score-matched comparative analysis using the 2015–2016 MBSAQIP database. Surg Endosc 33:1600–1612CrossRefGoogle Scholar
  31. 31.
    Sharma G, Strong AT, Tu C, Brethauer SA, Schauer PR, Aminian A (2018) Robotic platform for gastric bypass is associated with more resource utilization: an analysis of MBSAQIP dataset. Surg Obes Relat Dis 14:304–310CrossRefGoogle Scholar
  32. 32.
    Zafar SN, Miller K, Felton J, Wise ES, Kligman M (2019) Postoperative bleeding after laparoscopic Roux en Y gastric bypass: predictors and consequences. Surg Endosc 33:272–280CrossRefGoogle Scholar
  33. 33.
    Zafar SN, Shockcor N, Prasad N, Wise ES, Jackson HT, Kligman M (2019) Laparoscopic sleeve gastrectomy has a lower risk of postoperative bleeding than laparoscopic Roux-en-Y gastric bypass. Surg Laparosc Endosc Percutan Tech 29:53–57CrossRefGoogle Scholar
  34. 34.
    Metabolic and Bariatric Surgery Accreditation and Quality Improvement Program (Published January 2017) User Guide for the 2015 Participant Use Data File (PUF)Google Scholar
  35. 35.
    Department of Health and Human Services Office for Human Research Protections (2018) 45 CFR 46-Protection of Human Subjects, 2018 Requirements: 104 Exempt research. Department of Health and Human Services Office for Human Research Protections, RockvilleGoogle Scholar
  36. 36.
    Metabolic and Bariatric Surgery Accreditation and Quality Improvement Program (October 2016) Standards Manual version 2.0. Resources for Optimal Care of the Metabolic and Bariatric Surgery Patient. 2016Google Scholar
  37. 37.
    American Society for Metabolic and Bariatric Surgery (2018) Who is a Candidate for Bariatric Surgery?. American Society for Metabolic and Bariatric Surgery, GainsvilleGoogle Scholar
  38. 38.
    Fryar CD, Kruszon-Moran D, Gu Q, Ogden CL (2018) Mean body weight, height, waist circumference, and body mass index among adults: United States, 1999–2000 through 2015–2016. Natl Health Stat Report 1:16Google Scholar
  39. 39.
    Aminian A, Chang J, Brethauer SA, Kim JJ, American Society for M, Bariatric Surgery Clinical Issues C (2018) ASMBS updated position statement on bariatric surgery in class I obesity (BMI 30-35 kg/m(2)). Surg Obes Relat Dis 14:1071–1087CrossRefGoogle Scholar
  40. 40.
    AHIMA e-HIM Work Group on EHR Data Content (2006) Data standard time: data content standardization and the HIM role. J AHIMA 77:26–32Google Scholar
  41. 41.
    Amster A, Jentzsch J, Pasupuleti H, Subramanian KG (2015) Completeness, accuracy, and computability of National Quality Forum-specified eMeasures. J Am Med Inform Assoc 22:409–416Google Scholar
  42. 42.
    Hall BL, Hamilton BH, Richards K, Bilimoria KY, Cohen ME, Ko CY (2009) Does surgical quality improve in the American College of Surgeons National Surgical Quality Improvement Program: an evaluation of all participating hospitals. Ann Surg 250:363–376Google Scholar
  43. 43.
    Huser V, Kahn MG, Brown JS, Gouripeddi R (2018) Methods for examining data quality in healthcare integrated data repositories. Pac Symp Biocomput 23:628–633Google Scholar
  44. 44.
    Ko C (2012) The critical importance of good data to improving quality. Patient Safety & Quality HealthcareGoogle Scholar
  45. 45.
    Kupzyk KA, Cohen MZ (2015) Data validation and other strategies for data entry. West J Nurs Res 37:546–556CrossRefGoogle Scholar
  46. 46.
    Marc DT, Beattie J, Herasevich V, Gatewood L, Zhang R (2016) Assessing metadata quality of a federally sponsored health data repository. AMIA Annu Symp Proc 2016:864–873Google Scholar
  47. 47.
    Nouraei SA, Hudovsky A, Virk JS, Saleh HA (2017) Impact of developing a multidisciplinary coded dataset standard on administrative data accuracy for septoplasty, septorhinoplasty and nasal trauma surgery. J Laryngol Otol 131:341–346CrossRefGoogle Scholar
  48. 48.
    Raval MV, Pawlik TM (2018) Practical guide to surgical data sets: national surgical quality improvement program (NSQIP) and Pediatric NSQIP. JAMA Surg 153:764–765CrossRefGoogle Scholar
  49. 49.
    Royal College of Surgeons of England (2018) Improving surgical data. Royal College of Surgeons of England, LondonGoogle Scholar
  50. 50.
    American College of Surgeons National Surgical Quality Improvement Program (2017) User guide for the 2016 ACS NSQIP procedure targeted participant use data file (PUF). Chicago, ILGoogle Scholar
  51. 51.
    Research Data Assistance Center (ResDAC) (2019) Learn about CMS data. University of Minnesota School of Public Health, MinneapolisGoogle Scholar
  52. 52.
    Ferguson TB Jr, Hammill BG, Peterson ED, DeLong ER, Grover FL, Committee STSND (2002) A decade of change-risk profiles and outcomes for isolated coronary artery bypass grafting procedures, 1990–1999: a report from the STS National Database Committee and the Duke Clinical Research Institute. Society of Thoracic Surgeons. Ann Thorac Surg 73:480–489 (discussion 489–490) CrossRefGoogle Scholar
  53. 53.
    Welke KF, Ferguson TB Jr, Coombs LP, Dokholyan RS, Murray CJ, Schrader MA, Peterson ED (2004) Validity of the society of thoracic surgeons national adult cardiac surgery database. Ann Thorac Surg 77:1137–1139CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  • K. K. Noyes
    • 1
    • 2
    Email author
  • A. A. Myneni
    • 2
  • S. D. Schwaitzberg
    • 2
  • A. B. Hoffman
    • 2
  1. 1.Division of Health Services Policy and Practice, Department of Epidemiology and Environmental Health, School of Public Health and Health ProfessionsUniversity at BuffaloBuffaloUSA
  2. 2.Department of Surgery, Jacobs School of Medicine and Biomedical SciencesUniversity at BuffaloBuffaloUSA

Personalised recommendations