Annals of Surgical Oncology

, Volume 26, Issue 3, pp 739–745 | Cite as

Lessons Learned Regarding Missing Clinical Stage in the National Cancer Database

  • Tanya L. HoskinEmail author
  • Judy C. Boughey
  • Courtney N. Day
  • Elizabeth B. Habermann
Health Services Research and Global Oncology



The National Cancer Database (NCDB) is a valuable resource for studying national cancer treatment patterns. However, data abstraction rules from 2004 to 2007 resulted in missing clinical stage for a high percentage of cases. We investigated how this missingness can bias results in breast cancer studies including patients treated with neoadjuvant chemotherapy (NAC).


The impact of missing clinical stage on the estimated percentage of breast cancers treated with NAC versus adjuvant chemotherapy (AC) was examined from 2004 to 2013. Trends in NAC use were presented, excluding those cases with missing clinical stage, and compared with trends after multiple imputation, performed using the chained equations approach with predictive mean matching.


Clinical stage was missing for 56% of cases in 2004–2007, versus 12% in 2008–2013, and was missing more than twice as often for AC patients versus NAC patients (31% vs. 12% overall), with the largest difference occurring in 2004–2007 (60% vs. 27% missing). Because stage was more frequently missing in AC patients, excluding those missing clinical stage introduced bias when considering NAC versus AC trends. With multiple imputation, significant increases in NAC use were identified between 2004 and 2013 for each stage: use for stage I was 2% in 2004 and 5% in 2013, use for stage II was 11% in 2004 and 24% in 2013, use for stage III was 34% in 2004 and 46% in 2013, in contrast to an analysis excluding those missing stage, which suggested little or no increase within any stage.


NCDB data abstraction rules from 2004 to 2007 resulted in missing clinical stage for > 50% of breast cancers, which may introduce substantial bias. Multiple imputation or exclusion of the years 2004–2007 should be considered to mitigate the problem of missing clinical stage in NCDB.



The NCDB is a joint project of the CoC of the American College of Surgeons and the American Cancer Society. The CoC’s NCDB, and the hospitals participating in the CoC NCDB, are the source of the de-identified data used herein; they have not verified and are not responsible for the statistical validity of the data analysis or the conclusions derived by the authors.


The Mayo Clinic Robert D. and Patricia E. Kern Center for the Science of Health Care Delivery provides salary support for Dr. Habermann, Ms. Hoskin, and Ms. Day. No external funding was used.


  1. 1.
    Raval MV, Bilimoria KY, Stewart AK, Bentrem DJ, Ko CY. Using the NCDB for cancer care improvement: an introduction to available quality assessment tools. J Surg Oncol. 2009;99(8):488–90.CrossRefGoogle Scholar
  2. 2.
    Bilimoria KY, Stewart AK, Winchester DP, Ko CY. The National Cancer Data Base: a powerful initiative to improve cancer care in the United States. Ann Surg Oncol. 2008;15(3):683–90.CrossRefGoogle Scholar
  3. 3.
    Boffa DJ, Rosen JE, Mallin K, et al. Using the National Cancer Database for outcomes research: a review. JAMA Oncol. 2017;3(12):1722–8.CrossRefGoogle Scholar
  4. 4.
    National Cancer Database. Participant Use Data File (PUF) data dictionary. Accessed 13 Aug 2018.
  5. 5.
    National Cancer Database. Participant Use Data File (PUF) supplemental documents. Accessed 13 Aug 2018.
  6. 6.
    Merkow RP, Rademaker AW, Bilimoria KY. Practical guide to surgical data sets: National Cancer Database (NCDB). JAMA Surg. 2018;153(9):850–1.CrossRefGoogle Scholar
  7. 7.
    Haider AH, Bilimoria KY, Kibbe MR. A checklist to elevate the science of surgical database research. JAMA Surg. 2018;153(6):505–7.CrossRefGoogle Scholar
  8. 8.
    Kaji AH, Rademaker AW, Hyslop T. Tips for analyzing large data sets from the JAMA surgery statistical editors. JAMA Surg. 2018;153(6):508–9.CrossRefGoogle Scholar
  9. 9.
    Little RJ, Rubin DB. Statistical analysis with missing data. Wiley, London; 2014.Google Scholar
  10. 10.
    Sterne JA, White IR, Carlin JB, et al. Multiple imputation for missing data in epidemiological and clinical research: potential and pitfalls. BMJ. 2009;338:b2393.CrossRefGoogle Scholar
  11. 11.
    Horton NJ, Kleinman KP. Much ado about nothing: A comparison of missing data methods and software to fit incomplete data regression models. Am Stat. 2007;61(1):79-90.CrossRefGoogle Scholar
  12. 12.
    Moons KG, Donders RA, Stijnen T, Harrell FE. Using the outcome for imputation of missing predictor values was preferred. J Clin Epidemiol. 2006;59(10):1092–101.CrossRefGoogle Scholar
  13. 13.
    Eisemann N, Waldmann A, Katalinic A. Imputation of missing values of tumour stage in population-based cancer registration. BMC Med Res Methodol. 2011;11(1):129.CrossRefGoogle Scholar
  14. 14.
    White IR, Royston P, Wood AM. Multiple imputation using chained equations: issues and guidance for practice. Stat Med. 2011;30(4):377–99.CrossRefGoogle Scholar
  15. 15.
    Van Buuren S, Groothuis-Oudshoorn K. mice: multivariate imputation by chained equations in R. J Stat Softw. 2011;45(3):1–67.CrossRefGoogle Scholar
  16. 16.
    Knol MJ, Janssen KJ, Donders ART, et al. Unpredictable bias when using the missing indicator method or complete case analysis for missing confounder values: an empirical example. J Clin Epidemiol. 2010;63(7):728–36.CrossRefGoogle Scholar
  17. 17.
    Mougalian SS, Soulos PR, Killelea BK, et al. Use of neoadjuvant chemotherapy for patients with stage I to III breast cancer in the United States. Cancer. 2015;121(15):2544–52.CrossRefGoogle Scholar
  18. 18.
    Mackinnon A. The use and reporting of multiple imputation in medical research: a review. J Intern Med. 2010;268(6):586–93.CrossRefGoogle Scholar

Copyright information

© Society of Surgical Oncology 2019

Authors and Affiliations

  • Tanya L. Hoskin
    • 1
    Email author
  • Judy C. Boughey
    • 2
  • Courtney N. Day
    • 1
  • Elizabeth B. Habermann
    • 1
    • 2
    • 3
  1. 1.Department of Health Sciences ResearchMayo ClinicRochesterUSA
  2. 2.Department of SurgeryMayo ClinicRochesterUSA
  3. 3.Robert D. and Patricia E. Kern Center for the Science of Health Care DeliveryMayo ClinicRochesterUSA

Personalised recommendations