A Brief Comment About Sample Size and Response Rates for Clinical Research: A Multi-site Case Example


During the past few decades, response rates have declined across social science research. Although some researchers are willing to accept this state of affairs, in this comment, we argue that low response rates weaken the value of social science research, thereby jeopardizing science research and associated knowledge. We describe the case of a Hong Kong-based multi-site study of addiction treatment seekers to examine approaches to stimulating clinical research and response rates. For this case of addiction treatment seekers, the overall response rate was acceptable (69.7%), despite variation among the three study sites. We discuss factors that likely contributed to this response rate variation to illustrate practical steps that investigators can undertake to improve response rates instead of capitulating to the status quo.

This is a preview of subscription content, log in to check access.

Fig. 1


  1. 1.

    Wiseman and Billington (1984) respond to a classic contribution by Frankel (1982). However, Frankel’s article is difficult to access. Nevertheless, we include it here so that interested readers are aware of this classic contribution.


  1. Arfken, C. L., & Balon, R. (2011). Declining participation in research studies. Psychotherapy and Psychosomatics, 80(6), 325–328. https://doi.org/10.1159/000324795.

    Article  PubMed  Google Scholar 

  2. Babbie, E. R. (2007). The practice of social research (11th ed.). Belmont: Thomson Wadsworth.

    Google Scholar 

  3. Basson, D. (2008). Completion rate. In P. Lavrakas (Ed.), Encyclopedia of Survey Research Methods (p. 113). Thousand Oaks: Sage Publications, Inc..

    Google Scholar 

  4. de Leeuw, E., & de Heer, W. (2002). Trends in household survey nonresponse: a longitudinal and international comparison. In R. M. Groves, D. A. Dillman, J. L. Eltinge, & R. J. A. Little (Eds.), Survey Nonresponse (pp. 41–54). New York: Wiley.

    Google Scholar 

  5. Ekholm, O., Gundgaard, J., Rasmussen, N. K. R., & Hansen, E. H. (2010). The effect of health, socio-economic position, and mode of data collection on non-response in health interview surveys. Scandinavian Journal of Public Health, 38(7), 699–706. https://doi.org/10.1177/1403494810382474.

    Article  PubMed  Google Scholar 

  6. Frankel, J. R. (1982). On the definition of response rates: a special report of the CASRO task force on completion rates. Retrieved from Port Jefferson, NY.

  7. Grant, B. F., Dawson, D. A., Stinson, F. S., Chou, S. P., Dufour, M. C., & Pickering, R. P. (2004). The 12-month prevalence and trends in DSM-IV alcohol abuse and dependence: United States, 1991-1992 and 2001-2002. Drug and Alcohol Dependence, 74(3), 223–234.

    Article  Google Scholar 

  8. Groves, R. M. (2006). Nonresponse rates and nonresponse bias in household surveys. Public Opinion Quarterly, 70(5), 646–675.

    Article  Google Scholar 

  9. Hall, M. N., Amodeo, M., Shaffer, H. J., & Vander Bilt, J. (2000). Social workers employed in substance abuse treatment programs: a training needs assessment. Social Work, 45(2), 141–154.

    CAS  Article  Google Scholar 

  10. Hulley, S. B., Gove, S., Browner, W. S., & Cummings, S. R. (1988). Choosing the study subjects: specification and sampling. In S. B. Hulley & S. R. Cummings (Eds.), Designing clinical research: an epidemiologic approach (p. 247). Baltimore: Williams & Wilkins.

    Google Scholar 

  11. Johnson, T., & Owens, L. (2003). Survey response rate reporting in the professional literature. American Association for Public Opinion Research, 127–133.

  12. Kazdin, A. E. (2003). Research design in clinical psychology (4th ed.). Boston: Allyn and Bacon.

    Google Scholar 

  13. Kempf, A. M., & Remington, P. L. (2007). New challenges for telephone survey research in the twenty-first century. Annual Review of Public Health, 28(1), 113–126.

    Article  Google Scholar 

  14. Kennedy, J. F. (1962). Yale University Commencement Address. American Rhetoric. Retrieved from American Rhetoric website: https://www.americanrhetoric.com/speeches/jfkyalecommencement.htm. Accessed 24 Oct 2019.

  15. Kessler, R. C., Berglund, P., Chiu, W. T., Demler, O., Heeringa, S., Hiripi, E., et al. (2004). The US National Comorbidity Survey Replication (NCS-R): design and field procedures. International Journal of Methods in Psychiatric Research, 13(2), 69–92.

    Article  Google Scholar 

  16. Kessler, R. C., Hwang, I., Labrie, R. A., Petukhova, M., Sampson, N. A., Winters, K. C., & Shaffer, H. J. (2008). DSM-IV pathological gambling in the National Comorbidity Survey Replication. Psychological Medicine, 38, 1351–1360. https://doi.org/10.1017/S0033291708002900.

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  17. Klitzman, R. (2013). How IRBs view and make decisions about coercion and undue influence. Journal of Medical Ethics, 39(4), 224–229. https://doi.org/10.1136/medethics-2011-100439.

    Article  PubMed  Google Scholar 

  18. Lim, L. E. C., Poo, K. P., Lein, T., & Chew, S. K. (1995). Why patients fail to attend psychiatric outpatient follow-up: a pilot study. Singapore Medical Journal, 36, 403–405.

    CAS  PubMed  Google Scholar 

  19. Pinsonneault, A., & Kraemer, K. (1993). Survey research methodology in management information systems: an assessment. Journal of Management Information Systems, 10(2), 75–105. https://doi.org/10.1080/07421222.1993.11518001.

    Article  Google Scholar 

  20. Shaffer, H. J., Tom, M., Wong, M. F. Y., Chan, E. M. L., Cheng, G. L. F., Lo, C. K. M., . . . Wong, R. H. Y. (2017). Treating addiction as a syndrome: bridging research and clinical practice. Trends in Medicine, (April). Retrieved from Trends in Medicine website: https://trends.hms.harvard.edu. Accessed 24 Oct 2019.

  21. Shaffer, H. J., Tom, M. A., Wiley, R. C., Wong, M. F. Y., Chan, E. M. L., Cheng, G. L. F., et al. (2018). Using the syndrome model of addiction: a preliminary consideration of psychological states and traits. International Journal of Mental Health and Addiction, 16(6), 11373–11393. https://doi.org/10.1007/s11469-018-9952-2.

    Article  Google Scholar 

  22. Simons, D. J. (2018). Introducing advances in methods and practices in psychological science. Advances in Methods and Practices in Psychological Science, 1(1), 3–6. https://doi.org/10.1177/2515245918757424.

    Article  Google Scholar 

  23. Singleton, R., & Straits, B. C. (2005). Approaches to social research (4th ed.). New York: Oxford University Press.

    Google Scholar 

  24. Sivo, S., Saunders, C., Chang, Q., & Jiang, J. (2006). How low should you go? Low response rates and the validity of inference in IS questionnaire research. Journal of the Association for Information Systems, 7(6), 351–356,359-360,362-372,374-414. https://doi.org/10.17705/1jais.00093.

    Article  Google Scholar 

  25. Studer, J., Baggio, S., Mohler-Kuo, M., Dermota, P., Gaume, J., Bertholet, N., et al. (2013). Examining non-response bias in substance use research—are late respondents proxies for non-respondents? Drug and Alcohol Dependence, 132, 316–323. https://doi.org/10.1016/j.drugalcdep.2013.02.029.

    Article  PubMed  Google Scholar 

  26. Tercyak, K., Johnson, S. B., Kirkpatrick, K. A., & Silverstein, J. H. (1998). Offering a randomized trial of intensive therapy for IDDM to adolescents. Diabetes Care, 21(2), 213.

    Article  Google Scholar 

  27. The Insight Association. (2011). Code of standards & ethics for market, opinion, and social research. Port Jefferson: The Insight Association.

    Google Scholar 

  28. Vachon, D. O., Susman, M., Wynne, M. E., Birringer, J., Olshefsky, L., & Cox, K. (1995). Reasons therapists give for refusing to participate in psychotherapy process research. Journal of Counseling Psychology, 42(3), 380–382.

    Article  Google Scholar 

  29. Vander Bilt, J., Hall, M. N., Shaffer, H. J., Storti, S., & Church, O. (1997). An assessment of substance abuse treatment training needs among nurses (part I): evaluating skill, knowledge, and training characteristics. Journal of Substance Misuse for Nursing, Health and Social Care, 2(3), 150–157.

    Article  Google Scholar 

  30. Volberg, R. A., & Bernhard, B. (2006). The 2006 study of gambling and problem gambling in New Mexico: report to the Responsible Gaming Association of New Mexico. Retrieved from University of Calgary Libraries and Cultural Resources website: https://prism.ucalgary.ca/handle/1880/44211. Accessed 24 Oct 2019.

  31. Wiseman, F., & Billington, M. (1984). Journal of Marketing Research, 21(3), 336–338.

    Article  Google Scholar 

Download references

Author information



Corresponding author

Correspondence to Matthew A. Tom.

Ethics declarations

Funding and Conflict of Interest

During approximately the past 5 years, the Division on Addiction received funding from a variety of sources. We do not believe any of the following funding sources constitute a conflict of interest for this project; however, we want to fully disclose our funding sources. The Division on Addiction received funding from the Foundation for Advancing Alcohol Responsibility (FAAR), The Healing Lodge of the Seven Nations via the National Institutes of Health (NIH) and Indian Health Services (IHS), the Integrated Centre on Addiction Prevention and Treatment of the Tung Wah Group of Hospitals—which receives funding from The Hong Kong Jockey Club Charities Trust, DraftKings, the Massachusetts Gaming Commission, GVC Holdings PLC, and the Massachusetts Department of Public Health. In addition, during approximately the past 5 years, Shaffer or the Division on Addiction received funding from National Center for Responsible Gambling, National Institutes of Health, the Alcohol Beverage Management Research Fund, the Danish Council for Independent Research, Heineken USA, Inc., bwin.party, St. Francis House, the State of Florida (i.e., as a subcontract to Spectrum Gaming Group), the Massachusetts Residential Substance Abuse Treatment for State Prisoners Grant Program (i.e., as a subcontracted evaluator for Worcester House of Corrections), and the Massachusetts Juvenile Accountability Block Grant Program—as a subcontracted evaluator for Cambridge Police Department). Howard Shaffer also has received speaker honoraria, or royalties, and compensation for consultation from the American Psychological Association, Las Vegas Sands Corp., Davies Ward Phillips and Vineberg, LLP, Freshfields Bruckhaus Deringer, LLP, and from the Dunes of Easthampton, a residential addiction treatment program, for serving as a consultant. Regarding his work with the international group on responsible gambling, he received reimbursement from Laval University for travel expenses, but no honorarium.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Public Policy Statement

Although response rates in social science research are of vital importance, these rates have been decreasing. Some researchers consider this state of affairs acceptable, but we argue that it is imperative and feasible to incorporate various techniques that can increase response rates into study designs. This paper provides an example from a clinical research program with addiction treatment seekers in Hong Kong.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Ching, F.Y., Wiley, R.C., Tom, M.A. et al. A Brief Comment About Sample Size and Response Rates for Clinical Research: A Multi-site Case Example. Int J Ment Health Addiction 18, 1335–1345 (2020). https://doi.org/10.1007/s11469-019-00146-6

Download citation


  • Response rate
  • Research methods
  • Sample size