Quality of MBSAQIP data: bad luck, or lack of QA plan?
- 10 Downloads
National clinical registries are commonly used in clinical research, quality improvement, and health policy. However, little is known about methodological challenges associated with these registry analyses that could limit their impact and compromise patient safety. This study examined the quality of Metabolic and Bariatric Surgery Accreditation and Quality Improvement Program (MSBASQIP) data to assess its usability potential and improve data collection methodologies.
We developed a single flat file (n = 168,093) using five subsets (Main, BMI, Readmission, Reoperation, and Intervention) of the 2015 MBSAQIP Participant User Data File (PUF). Logic and validity tests included (1) individual profiles of patient’s body mass index (BMI) changes over time, (2) individual patient care pathways, and (3) correlation analysis between variable pairs associated with the same clinical encounters.
8888 (5.3%) patients did not have postoperative weight/BMI data; 20% of patients had different units for preoperative and postoperative weights. Postoperative weight measurements ranged between − 71 and 132% of preoperative weight. There were 325 (3.7%) hospital readmissions reported on the day of or day after MBS. The self-reporting of “emergency” vs. “planned” interventions did not correlate with the type of procedure and its indication. Up to 20% of data could potentially be unused for analysis due to data quality issues.
Our analysis revealed various data quality issues in the 2015 MBSAQIP PUF related to completeness, accuracy, and consistency. Since information on where the surgery was performed is lacking, it is not possible to conclude whether these issues represent data errors, patient outliers, or inappropriate care. Including automated data checks and biomedical informatics oversight, standardized coding for complications, additional de-identified facility and provider information, and training/mentorship opportunities in data informatics for all researchers who get access to the data have been shown to be effective in improving data quality and minimizing patient safety concerns.
KeywordsMBSAQIP PUF Data quality Bariatric surgery Surgical outcomes
The authors would like to thank Monami Majumder and Evan Kessler for their assistance with the preparation of this manuscript.
This manuscript was in part funded by Surgical Outcomes and Research (UB SOAR) program at the Department of Surgery, Jacobs School of Medicine and Biomedical Sciences, University at Buffalo, Buffalo, New York.
Compliance with ethical standards
The earlier version of the study was presented at the 2019 SAGES Annual Meeting. Dr. Hoffman is a paid consultant for Ethicon US, LLC (not related to the study). Dr. Steven Schwaitzberg is a paid consultant for Nu View Surgical, Acuity Bio, Activ Surgical, Human Extensions, Levitra Magnetics and Arch Therapeutics (not related to the study). K. K. Myneni, A. A. Noyes, S. D. Schwaitzberg, and A. B. Hoffman declare that they have no conflict of interest or financial ties to disclose.
- 1.Shiloach M, Frencher SK Jr, Steeger JE, Rowell KS, Bartzokis K, Tomeh MG, Richards KE, Ko CY, Hall BL (2010) Toward robust information: data quality and inter-rater reliability in the American College of Surgeons National Surgical Quality Improvement Program. J Am Coll Surg 210:6–16CrossRefGoogle Scholar
- 7.Berger ER, Clements RH, Morton JM, Huffman KM, Wolfe BM, Nguyen NT, Ko CY, Hutter MM (2016) The impact of different surgical techniques on outcomes in laparoscopic sleeve gastrectomies: the first report from the metabolic and bariatric surgery accreditation and quality improvement program (MBSAQIP). Ann Surg 264:464–473CrossRefGoogle Scholar
- 19.Haskins IN, Ju T, Whitlock AE, Rivas L, Amdur RL, Lin PP, Vaziri K (2018) Older age confers a higher risk of 30-day morbidity and mortality following laparoscopic bariatric surgery: an analysis of the metabolic and bariatric surgery quality improvement program. Obes Surg 28:2745–2752CrossRefGoogle Scholar
- 30.Sebastian R, Howell MH, Chang KH, Adrales G, Magnuson T, Schweitzer M, Nguyen H (2018) Robot-assisted versus laparoscopic Roux-en-Y gastric bypass and sleeve gastrectomy: a propensity score-matched comparative analysis using the 2015–2016 MBSAQIP database. Surg Endosc 33:1600–1612CrossRefGoogle Scholar
- 34.Metabolic and Bariatric Surgery Accreditation and Quality Improvement Program (Published January 2017) User Guide for the 2015 Participant Use Data File (PUF)Google Scholar
- 35.Department of Health and Human Services Office for Human Research Protections (2018) 45 CFR 46-Protection of Human Subjects, 2018 Requirements: 104 Exempt research. Department of Health and Human Services Office for Human Research Protections, RockvilleGoogle Scholar
- 36.Metabolic and Bariatric Surgery Accreditation and Quality Improvement Program (October 2016) Standards Manual version 2.0. Resources for Optimal Care of the Metabolic and Bariatric Surgery Patient. 2016Google Scholar
- 37.American Society for Metabolic and Bariatric Surgery (2018) Who is a Candidate for Bariatric Surgery?. American Society for Metabolic and Bariatric Surgery, GainsvilleGoogle Scholar
- 38.Fryar CD, Kruszon-Moran D, Gu Q, Ogden CL (2018) Mean body weight, height, waist circumference, and body mass index among adults: United States, 1999–2000 through 2015–2016. Natl Health Stat Report 1:16Google Scholar
- 40.AHIMA e-HIM Work Group on EHR Data Content (2006) Data standard time: data content standardization and the HIM role. J AHIMA 77:26–32Google Scholar
- 41.Amster A, Jentzsch J, Pasupuleti H, Subramanian KG (2015) Completeness, accuracy, and computability of National Quality Forum-specified eMeasures. J Am Med Inform Assoc 22:409–416Google Scholar
- 42.Hall BL, Hamilton BH, Richards K, Bilimoria KY, Cohen ME, Ko CY (2009) Does surgical quality improve in the American College of Surgeons National Surgical Quality Improvement Program: an evaluation of all participating hospitals. Ann Surg 250:363–376Google Scholar
- 43.Huser V, Kahn MG, Brown JS, Gouripeddi R (2018) Methods for examining data quality in healthcare integrated data repositories. Pac Symp Biocomput 23:628–633Google Scholar
- 44.Ko C (2012) The critical importance of good data to improving quality. Patient Safety & Quality HealthcareGoogle Scholar
- 46.Marc DT, Beattie J, Herasevich V, Gatewood L, Zhang R (2016) Assessing metadata quality of a federally sponsored health data repository. AMIA Annu Symp Proc 2016:864–873Google Scholar
- 49.Royal College of Surgeons of England (2018) Improving surgical data. Royal College of Surgeons of England, LondonGoogle Scholar
- 50.American College of Surgeons National Surgical Quality Improvement Program (2017) User guide for the 2016 ACS NSQIP procedure targeted participant use data file (PUF). Chicago, ILGoogle Scholar
- 51.Research Data Assistance Center (ResDAC) (2019) Learn about CMS data. University of Minnesota School of Public Health, MinneapolisGoogle Scholar
- 52.Ferguson TB Jr, Hammill BG, Peterson ED, DeLong ER, Grover FL, Committee STSND (2002) A decade of change-risk profiles and outcomes for isolated coronary artery bypass grafting procedures, 1990–1999: a report from the STS National Database Committee and the Duke Clinical Research Institute. Society of Thoracic Surgeons. Ann Thorac Surg 73:480–489 (discussion 489–490) CrossRefGoogle Scholar