A Brief Comment About Sample Size and Response Rates for Clinical Research: A Multi-site Case Example

  • Fung Yee Ching
  • Rhiannon Chou Wiley
  • Matthew A. TomEmail author
  • Ryan Hing Yan Wong
  • Howard J. Shaffer
  • Elda Mei Lo Chan
Original Article


During the past few decades, response rates have declined across social science research. Although some researchers are willing to accept this state of affairs, in this comment, we argue that low response rates weaken the value of social science research, thereby jeopardizing science research and associated knowledge. We describe the case of a Hong Kong-based multi-site study of addiction treatment seekers to examine approaches to stimulating clinical research and response rates. For this case of addiction treatment seekers, the overall response rate was acceptable (69.7%), despite variation among the three study sites. We discuss factors that likely contributed to this response rate variation to illustrate practical steps that investigators can undertake to improve response rates instead of capitulating to the status quo.


Response rate Research methods Sample size 


Compliance with Ethical Standards

Funding and Conflict of Interest

During approximately the past 5 years, the Division on Addiction received funding from a variety of sources. We do not believe any of the following funding sources constitute a conflict of interest for this project; however, we want to fully disclose our funding sources. The Division on Addiction received funding from the Foundation for Advancing Alcohol Responsibility (FAAR), The Healing Lodge of the Seven Nations via the National Institutes of Health (NIH) and Indian Health Services (IHS), the Integrated Centre on Addiction Prevention and Treatment of the Tung Wah Group of Hospitals—which receives funding from The Hong Kong Jockey Club Charities Trust, DraftKings, the Massachusetts Gaming Commission, GVC Holdings PLC, and the Massachusetts Department of Public Health. In addition, during approximately the past 5 years, Shaffer or the Division on Addiction received funding from National Center for Responsible Gambling, National Institutes of Health, the Alcohol Beverage Management Research Fund, the Danish Council for Independent Research, Heineken USA, Inc.,, St. Francis House, the State of Florida (i.e., as a subcontract to Spectrum Gaming Group), the Massachusetts Residential Substance Abuse Treatment for State Prisoners Grant Program (i.e., as a subcontracted evaluator for Worcester House of Corrections), and the Massachusetts Juvenile Accountability Block Grant Program—as a subcontracted evaluator for Cambridge Police Department). Howard Shaffer also has received speaker honoraria, or royalties, and compensation for consultation from the American Psychological Association, Las Vegas Sands Corp., Davies Ward Phillips and Vineberg, LLP, Freshfields Bruckhaus Deringer, LLP, and from the Dunes of Easthampton, a residential addiction treatment program, for serving as a consultant. Regarding his work with the international group on responsible gambling, he received reimbursement from Laval University for travel expenses, but no honorarium.


  1. Arfken, C. L., & Balon, R. (2011). Declining participation in research studies. Psychotherapy and Psychosomatics, 80(6), 325–328. Scholar
  2. Babbie, E. R. (2007). The practice of social research (11th ed.). Belmont: Thomson Wadsworth.Google Scholar
  3. Basson, D. (2008). Completion rate. In P. Lavrakas (Ed.), Encyclopedia of Survey Research Methods (p. 113). Thousand Oaks: Sage Publications, Inc..Google Scholar
  4. de Leeuw, E., & de Heer, W. (2002). Trends in household survey nonresponse: a longitudinal and international comparison. In R. M. Groves, D. A. Dillman, J. L. Eltinge, & R. J. A. Little (Eds.), Survey Nonresponse (pp. 41–54). New York: Wiley.Google Scholar
  5. Ekholm, O., Gundgaard, J., Rasmussen, N. K. R., & Hansen, E. H. (2010). The effect of health, socio-economic position, and mode of data collection on non-response in health interview surveys. Scandinavian Journal of Public Health, 38(7), 699–706. Scholar
  6. Frankel, J. R. (1982). On the definition of response rates: a special report of the CASRO task force on completion rates. Retrieved from Port Jefferson, NY.Google Scholar
  7. Grant, B. F., Dawson, D. A., Stinson, F. S., Chou, S. P., Dufour, M. C., & Pickering, R. P. (2004). The 12-month prevalence and trends in DSM-IV alcohol abuse and dependence: United States, 1991-1992 and 2001-2002. Drug and Alcohol Dependence, 74(3), 223–234.CrossRefGoogle Scholar
  8. Groves, R. M. (2006). Nonresponse rates and nonresponse bias in household surveys. Public Opinion Quarterly, 70(5), 646–675.CrossRefGoogle Scholar
  9. Hall, M. N., Amodeo, M., Shaffer, H. J., & Vander Bilt, J. (2000). Social workers employed in substance abuse treatment programs: a training needs assessment. Social Work, 45(2), 141–154.CrossRefGoogle Scholar
  10. Hulley, S. B., Gove, S., Browner, W. S., & Cummings, S. R. (1988). Choosing the study subjects: specification and sampling. In S. B. Hulley & S. R. Cummings (Eds.), Designing clinical research: an epidemiologic approach (p. 247). Baltimore: Williams & Wilkins.Google Scholar
  11. Johnson, T., & Owens, L. (2003). Survey response rate reporting in the professional literature. American Association for Public Opinion Research, 127–133.Google Scholar
  12. Kazdin, A. E. (2003). Research design in clinical psychology (4th ed.). Boston: Allyn and Bacon.Google Scholar
  13. Kempf, A. M., & Remington, P. L. (2007). New challenges for telephone survey research in the twenty-first century. Annual Review of Public Health, 28(1), 113–126.CrossRefGoogle Scholar
  14. Kennedy, J. F. (1962). Yale University Commencement Address. American Rhetoric. Retrieved from American Rhetoric website: Accessed 24 Oct 2019.
  15. Kessler, R. C., Berglund, P., Chiu, W. T., Demler, O., Heeringa, S., Hiripi, E., et al. (2004). The US National Comorbidity Survey Replication (NCS-R): design and field procedures. International Journal of Methods in Psychiatric Research, 13(2), 69–92.CrossRefGoogle Scholar
  16. Kessler, R. C., Hwang, I., Labrie, R. A., Petukhova, M., Sampson, N. A., Winters, K. C., & Shaffer, H. J. (2008). DSM-IV pathological gambling in the National Comorbidity Survey Replication. Psychological Medicine, 38, 1351–1360. Scholar
  17. Klitzman, R. (2013). How IRBs view and make decisions about coercion and undue influence. Journal of Medical Ethics, 39(4), 224–229. Scholar
  18. Lim, L. E. C., Poo, K. P., Lein, T., & Chew, S. K. (1995). Why patients fail to attend psychiatric outpatient follow-up: a pilot study. Singapore Medical Journal, 36, 403–405.PubMedGoogle Scholar
  19. Pinsonneault, A., & Kraemer, K. (1993). Survey research methodology in management information systems: an assessment. Journal of Management Information Systems, 10(2), 75–105. Scholar
  20. Shaffer, H. J., Tom, M., Wong, M. F. Y., Chan, E. M. L., Cheng, G. L. F., Lo, C. K. M., . . . Wong, R. H. Y. (2017). Treating addiction as a syndrome: bridging research and clinical practice. Trends in Medicine, (April). Retrieved from Trends in Medicine website: Accessed 24 Oct 2019.
  21. Shaffer, H. J., Tom, M. A., Wiley, R. C., Wong, M. F. Y., Chan, E. M. L., Cheng, G. L. F., et al. (2018). Using the syndrome model of addiction: a preliminary consideration of psychological states and traits. International Journal of Mental Health and Addiction, 16(6), 11373–11393. Scholar
  22. Simons, D. J. (2018). Introducing advances in methods and practices in psychological science. Advances in Methods and Practices in Psychological Science, 1(1), 3–6. Scholar
  23. Singleton, R., & Straits, B. C. (2005). Approaches to social research (4th ed.). New York: Oxford University Press.Google Scholar
  24. Sivo, S., Saunders, C., Chang, Q., & Jiang, J. (2006). How low should you go? Low response rates and the validity of inference in IS questionnaire research. Journal of the Association for Information Systems, 7(6), 351–356,359-360,362-372,374-414. Scholar
  25. Studer, J., Baggio, S., Mohler-Kuo, M., Dermota, P., Gaume, J., Bertholet, N., et al. (2013). Examining non-response bias in substance use research—are late respondents proxies for non-respondents? Drug and Alcohol Dependence, 132, 316–323. Scholar
  26. Tercyak, K., Johnson, S. B., Kirkpatrick, K. A., & Silverstein, J. H. (1998). Offering a randomized trial of intensive therapy for IDDM to adolescents. Diabetes Care, 21(2), 213.CrossRefGoogle Scholar
  27. The Insight Association. (2011). Code of standards & ethics for market, opinion, and social research. Port Jefferson: The Insight Association.Google Scholar
  28. Vachon, D. O., Susman, M., Wynne, M. E., Birringer, J., Olshefsky, L., & Cox, K. (1995). Reasons therapists give for refusing to participate in psychotherapy process research. Journal of Counseling Psychology, 42(3), 380–382.CrossRefGoogle Scholar
  29. Vander Bilt, J., Hall, M. N., Shaffer, H. J., Storti, S., & Church, O. (1997). An assessment of substance abuse treatment training needs among nurses (part I): evaluating skill, knowledge, and training characteristics. Journal of Substance Misuse for Nursing, Health and Social Care, 2(3), 150–157.CrossRefGoogle Scholar
  30. Volberg, R. A., & Bernhard, B. (2006). The 2006 study of gambling and problem gambling in New Mexico: report to the Responsible Gaming Association of New Mexico. Retrieved from University of Calgary Libraries and Cultural Resources website: Accessed 24 Oct 2019.
  31. Wiseman, F., & Billington, M. (1984). Journal of Marketing Research, 21(3), 336–338.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Tung Wah Group of Hospitals Integrated Centre on Addiction Prevention and TreatmentHong Kong SARChina
  2. 2.Division on AddictionThe Cambridge Health AllianceMaldenUSA
  3. 3.Harvard Medical SchoolBostonUSA
  4. 4.St. John’s Cathedral Counselling ServiceHong Kong SARChina

Personalised recommendations