During the past few decades, response rates have declined across social science research. Although some researchers are willing to accept this state of affairs, in this comment, we argue that low response rates weaken the value of social science research, thereby jeopardizing science research and associated knowledge. We describe the case of a Hong Kong-based multi-site study of addiction treatment seekers to examine approaches to stimulating clinical research and response rates. For this case of addiction treatment seekers, the overall response rate was acceptable (69.7%), despite variation among the three study sites. We discuss factors that likely contributed to this response rate variation to illustrate practical steps that investigators can undertake to improve response rates instead of capitulating to the status quo.
This is a preview of subscription content, log in to check access.
Buy single article
Instant access to the full article PDF.
Tax calculation will be finalised during checkout.
Subscribe to journal
Immediate online access to all issues from 2019. Subscription will auto renew annually.
Tax calculation will be finalised during checkout.
Arfken, C. L., & Balon, R. (2011). Declining participation in research studies. Psychotherapy and Psychosomatics, 80(6), 325–328. https://doi.org/10.1159/000324795.
Babbie, E. R. (2007). The practice of social research (11th ed.). Belmont: Thomson Wadsworth.
Basson, D. (2008). Completion rate. In P. Lavrakas (Ed.), Encyclopedia of Survey Research Methods (p. 113). Thousand Oaks: Sage Publications, Inc..
de Leeuw, E., & de Heer, W. (2002). Trends in household survey nonresponse: a longitudinal and international comparison. In R. M. Groves, D. A. Dillman, J. L. Eltinge, & R. J. A. Little (Eds.), Survey Nonresponse (pp. 41–54). New York: Wiley.
Ekholm, O., Gundgaard, J., Rasmussen, N. K. R., & Hansen, E. H. (2010). The effect of health, socio-economic position, and mode of data collection on non-response in health interview surveys. Scandinavian Journal of Public Health, 38(7), 699–706. https://doi.org/10.1177/1403494810382474.
Frankel, J. R. (1982). On the definition of response rates: a special report of the CASRO task force on completion rates. Retrieved from Port Jefferson, NY.
Grant, B. F., Dawson, D. A., Stinson, F. S., Chou, S. P., Dufour, M. C., & Pickering, R. P. (2004). The 12-month prevalence and trends in DSM-IV alcohol abuse and dependence: United States, 1991-1992 and 2001-2002. Drug and Alcohol Dependence, 74(3), 223–234.
Groves, R. M. (2006). Nonresponse rates and nonresponse bias in household surveys. Public Opinion Quarterly, 70(5), 646–675.
Hall, M. N., Amodeo, M., Shaffer, H. J., & Vander Bilt, J. (2000). Social workers employed in substance abuse treatment programs: a training needs assessment. Social Work, 45(2), 141–154.
Hulley, S. B., Gove, S., Browner, W. S., & Cummings, S. R. (1988). Choosing the study subjects: specification and sampling. In S. B. Hulley & S. R. Cummings (Eds.), Designing clinical research: an epidemiologic approach (p. 247). Baltimore: Williams & Wilkins.
Johnson, T., & Owens, L. (2003). Survey response rate reporting in the professional literature. American Association for Public Opinion Research, 127–133.
Kazdin, A. E. (2003). Research design in clinical psychology (4th ed.). Boston: Allyn and Bacon.
Kempf, A. M., & Remington, P. L. (2007). New challenges for telephone survey research in the twenty-first century. Annual Review of Public Health, 28(1), 113–126.
Kennedy, J. F. (1962). Yale University Commencement Address. American Rhetoric. Retrieved from American Rhetoric website: https://www.americanrhetoric.com/speeches/jfkyalecommencement.htm. Accessed 24 Oct 2019.
Kessler, R. C., Berglund, P., Chiu, W. T., Demler, O., Heeringa, S., Hiripi, E., et al. (2004). The US National Comorbidity Survey Replication (NCS-R): design and field procedures. International Journal of Methods in Psychiatric Research, 13(2), 69–92.
Kessler, R. C., Hwang, I., Labrie, R. A., Petukhova, M., Sampson, N. A., Winters, K. C., & Shaffer, H. J. (2008). DSM-IV pathological gambling in the National Comorbidity Survey Replication. Psychological Medicine, 38, 1351–1360. https://doi.org/10.1017/S0033291708002900.
Klitzman, R. (2013). How IRBs view and make decisions about coercion and undue influence. Journal of Medical Ethics, 39(4), 224–229. https://doi.org/10.1136/medethics-2011-100439.
Lim, L. E. C., Poo, K. P., Lein, T., & Chew, S. K. (1995). Why patients fail to attend psychiatric outpatient follow-up: a pilot study. Singapore Medical Journal, 36, 403–405.
Pinsonneault, A., & Kraemer, K. (1993). Survey research methodology in management information systems: an assessment. Journal of Management Information Systems, 10(2), 75–105. https://doi.org/10.1080/07421222.1993.11518001.
Shaffer, H. J., Tom, M., Wong, M. F. Y., Chan, E. M. L., Cheng, G. L. F., Lo, C. K. M., . . . Wong, R. H. Y. (2017). Treating addiction as a syndrome: bridging research and clinical practice. Trends in Medicine, (April). Retrieved from Trends in Medicine website: https://trends.hms.harvard.edu. Accessed 24 Oct 2019.
Shaffer, H. J., Tom, M. A., Wiley, R. C., Wong, M. F. Y., Chan, E. M. L., Cheng, G. L. F., et al. (2018). Using the syndrome model of addiction: a preliminary consideration of psychological states and traits. International Journal of Mental Health and Addiction, 16(6), 11373–11393. https://doi.org/10.1007/s11469-018-9952-2.
Simons, D. J. (2018). Introducing advances in methods and practices in psychological science. Advances in Methods and Practices in Psychological Science, 1(1), 3–6. https://doi.org/10.1177/2515245918757424.
Singleton, R., & Straits, B. C. (2005). Approaches to social research (4th ed.). New York: Oxford University Press.
Sivo, S., Saunders, C., Chang, Q., & Jiang, J. (2006). How low should you go? Low response rates and the validity of inference in IS questionnaire research. Journal of the Association for Information Systems, 7(6), 351–356,359-360,362-372,374-414. https://doi.org/10.17705/1jais.00093.
Studer, J., Baggio, S., Mohler-Kuo, M., Dermota, P., Gaume, J., Bertholet, N., et al. (2013). Examining non-response bias in substance use research—are late respondents proxies for non-respondents? Drug and Alcohol Dependence, 132, 316–323. https://doi.org/10.1016/j.drugalcdep.2013.02.029.
Tercyak, K., Johnson, S. B., Kirkpatrick, K. A., & Silverstein, J. H. (1998). Offering a randomized trial of intensive therapy for IDDM to adolescents. Diabetes Care, 21(2), 213.
The Insight Association. (2011). Code of standards & ethics for market, opinion, and social research. Port Jefferson: The Insight Association.
Vachon, D. O., Susman, M., Wynne, M. E., Birringer, J., Olshefsky, L., & Cox, K. (1995). Reasons therapists give for refusing to participate in psychotherapy process research. Journal of Counseling Psychology, 42(3), 380–382.
Vander Bilt, J., Hall, M. N., Shaffer, H. J., Storti, S., & Church, O. (1997). An assessment of substance abuse treatment training needs among nurses (part I): evaluating skill, knowledge, and training characteristics. Journal of Substance Misuse for Nursing, Health and Social Care, 2(3), 150–157.
Volberg, R. A., & Bernhard, B. (2006). The 2006 study of gambling and problem gambling in New Mexico: report to the Responsible Gaming Association of New Mexico. Retrieved from University of Calgary Libraries and Cultural Resources website: https://prism.ucalgary.ca/handle/1880/44211. Accessed 24 Oct 2019.
Wiseman, F., & Billington, M. (1984). Journal of Marketing Research, 21(3), 336–338.
Funding and Conflict of Interest
During approximately the past 5 years, the Division on Addiction received funding from a variety of sources. We do not believe any of the following funding sources constitute a conflict of interest for this project; however, we want to fully disclose our funding sources. The Division on Addiction received funding from the Foundation for Advancing Alcohol Responsibility (FAAR), The Healing Lodge of the Seven Nations via the National Institutes of Health (NIH) and Indian Health Services (IHS), the Integrated Centre on Addiction Prevention and Treatment of the Tung Wah Group of Hospitals—which receives funding from The Hong Kong Jockey Club Charities Trust, DraftKings, the Massachusetts Gaming Commission, GVC Holdings PLC, and the Massachusetts Department of Public Health. In addition, during approximately the past 5 years, Shaffer or the Division on Addiction received funding from National Center for Responsible Gambling, National Institutes of Health, the Alcohol Beverage Management Research Fund, the Danish Council for Independent Research, Heineken USA, Inc., bwin.party, St. Francis House, the State of Florida (i.e., as a subcontract to Spectrum Gaming Group), the Massachusetts Residential Substance Abuse Treatment for State Prisoners Grant Program (i.e., as a subcontracted evaluator for Worcester House of Corrections), and the Massachusetts Juvenile Accountability Block Grant Program—as a subcontracted evaluator for Cambridge Police Department). Howard Shaffer also has received speaker honoraria, or royalties, and compensation for consultation from the American Psychological Association, Las Vegas Sands Corp., Davies Ward Phillips and Vineberg, LLP, Freshfields Bruckhaus Deringer, LLP, and from the Dunes of Easthampton, a residential addiction treatment program, for serving as a consultant. Regarding his work with the international group on responsible gambling, he received reimbursement from Laval University for travel expenses, but no honorarium.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Public Policy Statement
Although response rates in social science research are of vital importance, these rates have been decreasing. Some researchers consider this state of affairs acceptable, but we argue that it is imperative and feasible to incorporate various techniques that can increase response rates into study designs. This paper provides an example from a clinical research program with addiction treatment seekers in Hong Kong.
About this article
Cite this article
Ching, F.Y., Wiley, R.C., Tom, M.A. et al. A Brief Comment About Sample Size and Response Rates for Clinical Research: A Multi-site Case Example. Int J Ment Health Addiction 18, 1335–1345 (2020). https://doi.org/10.1007/s11469-019-00146-6
- Response rate
- Research methods
- Sample size