Limited evidence of non-response bias despite modest response rate in a nationwide survey of long-term cancer survivors—results from the NOR-CAYACS study
Declining response rates threaten the generalizability of health surveys. We investigate (1) the effect of item order on response rate; (2) characteristics of early , late and non-responders; and (3) potential non-response bias in a population-based health survey of childhood, adolescent and young adult cancer survivors (CAYACS).
We mailed a questionnaire survey to 5361 eligible CAYACS identified by the Cancer Registry of Norway (CRN), representing a range of cancer diagnoses. The 302-item questionnaire included a range of survivorship-related questions and validated patient-reported outcome measures. To investigate item-order effects on response rates, we constructed two versions of the questionnaire presenting cancer-related or socio-demographic items first. The CRN provided demographic and clinical information for the total population. Risk of non-response bias was estimated by (1) comparing outcomes between early and late responders (answered after a reminder), and (2) by applying inverse probability of participation weights to construct a total population (with 100% response) and then compare 21 a priori selected outcomes between early responders, all responders (early + late) and the total population (all eligible).
Survey item order did not affect response rates (cancer first 49.8% vs socio-demographic first 50.2%). Shorter time since diagnosis, male gender and a malignant melanoma diagnosis remained significant predictors of non-response in a multivariable multinomial regression model. There were no significant differences on 16/21 survey outcomes between early and late responders, and 18/21 survey outcomes between early responders, all responders and the total population.
Despite a modest response rate, we found little evidence for a response bias in our study.
Implications for Cancer Survivors
Surveys of survivor-reported outcomes with low response rates may still be valuable and generalizable to the total survivor population.
KeywordsNon-response bias Childhood cancer survivors Health survey Response rate
The project was funded by The Norwegian Cancer Society (45980) and The Norway Research Council (218312). CSR has received funding from the European Union Seventh Framework Programme (FP7-PEOPLE-2013-COFUND) under grant agreement no. 609020 - Scientia Fellows. SDF received funding from The Radiumhospital Fund (335007). ER and HCL were partially funded by the Regional health authorities of South-Eastern Norway (2015084).
Compliance with ethical standards
The study was granted concession by The Norwegian Data Protection Authority (15/00395-2/CGN) and approved by the Regional Committee for Medical Research Ethics (2015/232 REK sør-øst B), and the Data Protection Officer at Oslo University and the Norwegian Cancer Registry. Informed consent was collected for both participation in the survey and data linkage to information in the CRN
Conflict of interests
The authors declare that they have no conflicts of interest.
- 1.Kilsdonk E, Wendel E, van Dulmen-den Broeder E, van Leeuwen FE, van den Berg MH, Jaspers MW. Participation rates of childhood cancer survivors to self-administered questionnaires: a systematic review. Eur J Cancer Care. 2017; e12462. https://doi.org/10.1111/ecc.12462.
- 5.Brinkman TM, Recklitis CJ, Michel G, Grootenhuis MA, Klosky JL. Psychological symptoms, social outcomes, socioeconomic attainment, and health behaviors among survivors of childhood cancer: current state of the literature. J Clin Oncol. 2018;36(21):2190–7. https://doi.org/10.1200/jco.2017.76.5552.CrossRefGoogle Scholar
- 8.Robison LL, Mertens AC, Boice JD, Breslow NE, Donaldson SS, Green DM, et al. Study design and cohort characteristics of the Childhood Cancer Survivor Study: a multi-institutional collaborative project. Ped Blood Cancer. 2002;38(4):229–39.Google Scholar
- 9.Shaw AK, Morrison HI, Speechley KN, Maunsell E, Barrera M, Schanzer D, et al. The late effects study: design and subject representativeness of a Canadian, multi-centre study of late effects of childhood cancer. Chronic Dis Inj Can. 2004;25(3/4):119.Google Scholar
- 10.Overbeek A, van den Berg MH, Kremer LC, Van den Heuvel-Eibrink MM, Tissing WJ, Loonen JJ, et al. A nationwide study on reproductive function, ovarian reserve, and risk of premature menopause in female survivors of childhood cancer: design and methodological challenges. BMC Cancer. 2012;12(1):363. https://doi.org/10.1186/1471-2407-12-363. CrossRefGoogle Scholar
- 11.Hawkins MM, Lancashire ER, Winter DL, Frobisher C, Reulen RC, Taylor AJ, et al. The British Childhood Cancer Survivor Study: objectives, methods, population structure, response rates and initial descriptive information. Ped Blood Cancer. 2008;50(5):1018–25. https://doi.org/10.1002/pbc.21335.CrossRefGoogle Scholar
- 13.De Heer W, De Leeuw E. Trends in household survey nonresponse: a longitudinal and international comparison. Survey nonresp. 2002:41.Google Scholar
- 16.Cheung KL, ten Klooster PM, Smit C, de Vries H, Pieterse ME. The impact of non-response bias due to sampling in public health studies: a comparison of voluntary versus mandatory recruitment in a Dutch national survey on adolescent health. BMC Public Health. 2017;17(1):276. https://doi.org/10.1186/s12889-017-4189-8.CrossRefGoogle Scholar
- 17.Rueegg CS, Gianinazzi ME, Michel G, Zwahlen M, von der Weid NX, Kuehni CE. No evidence of response bias in a population-based childhood cancer survivor questionnaire survey - results from the Swiss Childhood Cancer Survivor Study. PLoS One. 2017;12(5):e0176442. https://doi.org/10.1371/journal.pone.0176442.CrossRefGoogle Scholar
- 18.Ojha RP, Oancea SC, Ness KK, Lanctot JQ, Srivastava DK, Robison LL, et al. Assessment of potential bias from non-participation in a dynamic clinical cohort of long-term childhood cancer survivors: results from the St. Jude lifetime cohort study. Ped Blood Cancer. 2013;60(5):856–64. https://doi.org/10.1002/pbc.24348.CrossRefGoogle Scholar
- 21.Bliddal M, Liew Z, Pottegård A, Kirkegaard H, Olsen J, Nohr EA. Examining non-participation to the maternal follow-up within the Danish National Birth Cohort. Am J Epidemiol. 2018 Jul 1;187(7):1511-1519. https://doi.org/10.1093/aje/kwy002.
- 25.Dillman DA, Smyth JD, Christian LM. Internet, phone, mail, and mixed-mode surveys: the tailored design method. 4th ed. Hoboken: John Wiley & Sons, Inc.; 2014.Google Scholar
- 26.Hernán MA, Hernández-Díaz S, Robins JM. A structural approach to selection bias. Epidemiol. 2004;15(5):615–25. https://doi.org/10.1097/01.ede.0000135174.63482.43.CrossRefGoogle Scholar
- 27.Ness KK, Li C, Mitby PA, Radloff GA, Mertens AC, Davies SM, et al. Characteristics of responders to a request for a buccal cell specimen among survivors of childhood cancer and their siblings. Ped Blood Cancer. 2010;55(1):165–70.Google Scholar