Skip to main content

Advertisement

SpringerLink
Log in

A Brief Comment About Sample Size and Response Rates for Clinical Research: A Multi-site Case Example

  • Original Article
  • Published:
International Journal of Mental Health and Addiction Aims and scope Submit manuscript

Abstract

During the past few decades, response rates have declined across social science research. Although some researchers are willing to accept this state of affairs, in this comment, we argue that low response rates weaken the value of social science research, thereby jeopardizing science research and associated knowledge. We describe the case of a Hong Kong-based multi-site study of addiction treatment seekers to examine approaches to stimulating clinical research and response rates. For this case of addiction treatment seekers, the overall response rate was acceptable (69.7%), despite variation among the three study sites. We discuss factors that likely contributed to this response rate variation to illustrate practical steps that investigators can undertake to improve response rates instead of capitulating to the status quo.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

Notes

  1. Wiseman and Billington (1984) respond to a classic contribution by Frankel (1982). However, Frankel’s article is difficult to access. Nevertheless, we include it here so that interested readers are aware of this classic contribution.

References

  • Arfken, C. L., & Balon, R. (2011). Declining participation in research studies. Psychotherapy and Psychosomatics, 80(6), 325–328. https://doi.org/10.1159/000324795.

    Article  PubMed  Google Scholar 

  • Babbie, E. R. (2007). The practice of social research (11th ed.). Belmont: Thomson Wadsworth.

    Google Scholar 

  • Basson, D. (2008). Completion rate. In P. Lavrakas (Ed.), Encyclopedia of Survey Research Methods (p. 113). Thousand Oaks: Sage Publications, Inc..

    Google Scholar 

  • de Leeuw, E., & de Heer, W. (2002). Trends in household survey nonresponse: a longitudinal and international comparison. In R. M. Groves, D. A. Dillman, J. L. Eltinge, & R. J. A. Little (Eds.), Survey Nonresponse (pp. 41–54). New York: Wiley.

    Google Scholar 

  • Ekholm, O., Gundgaard, J., Rasmussen, N. K. R., & Hansen, E. H. (2010). The effect of health, socio-economic position, and mode of data collection on non-response in health interview surveys. Scandinavian Journal of Public Health, 38(7), 699–706. https://doi.org/10.1177/1403494810382474.

    Article  PubMed  Google Scholar 

  • Frankel, J. R. (1982). On the definition of response rates: a special report of the CASRO task force on completion rates. Retrieved from Port Jefferson, NY.

  • Grant, B. F., Dawson, D. A., Stinson, F. S., Chou, S. P., Dufour, M. C., & Pickering, R. P. (2004). The 12-month prevalence and trends in DSM-IV alcohol abuse and dependence: United States, 1991-1992 and 2001-2002. Drug and Alcohol Dependence, 74(3), 223–234.

    Article  Google Scholar 

  • Groves, R. M. (2006). Nonresponse rates and nonresponse bias in household surveys. Public Opinion Quarterly, 70(5), 646–675.

    Article  Google Scholar 

  • Hall, M. N., Amodeo, M., Shaffer, H. J., & Vander Bilt, J. (2000). Social workers employed in substance abuse treatment programs: a training needs assessment. Social Work, 45(2), 141–154.

    Article  CAS  Google Scholar 

  • Hulley, S. B., Gove, S., Browner, W. S., & Cummings, S. R. (1988). Choosing the study subjects: specification and sampling. In S. B. Hulley & S. R. Cummings (Eds.), Designing clinical research: an epidemiologic approach (p. 247). Baltimore: Williams & Wilkins.

    Google Scholar 

  • Johnson, T., & Owens, L. (2003). Survey response rate reporting in the professional literature. American Association for Public Opinion Research, 127–133.

  • Kazdin, A. E. (2003). Research design in clinical psychology (4th ed.). Boston: Allyn and Bacon.

    Google Scholar 

  • Kempf, A. M., & Remington, P. L. (2007). New challenges for telephone survey research in the twenty-first century. Annual Review of Public Health, 28(1), 113–126.

    Article  Google Scholar 

  • Kennedy, J. F. (1962). Yale University Commencement Address. American Rhetoric. Retrieved from American Rhetoric website: https://www.americanrhetoric.com/speeches/jfkyalecommencement.htm. Accessed 24 Oct 2019.

  • Kessler, R. C., Berglund, P., Chiu, W. T., Demler, O., Heeringa, S., Hiripi, E., et al. (2004). The US National Comorbidity Survey Replication (NCS-R): design and field procedures. International Journal of Methods in Psychiatric Research, 13(2), 69–92.

    Article  Google Scholar 

  • Kessler, R. C., Hwang, I., Labrie, R. A., Petukhova, M., Sampson, N. A., Winters, K. C., & Shaffer, H. J. (2008). DSM-IV pathological gambling in the National Comorbidity Survey Replication. Psychological Medicine, 38, 1351–1360. https://doi.org/10.1017/S0033291708002900.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Klitzman, R. (2013). How IRBs view and make decisions about coercion and undue influence. Journal of Medical Ethics, 39(4), 224–229. https://doi.org/10.1136/medethics-2011-100439.

    Article  PubMed  Google Scholar 

  • Lim, L. E. C., Poo, K. P., Lein, T., & Chew, S. K. (1995). Why patients fail to attend psychiatric outpatient follow-up: a pilot study. Singapore Medical Journal, 36, 403–405.

    CAS  PubMed  Google Scholar 

  • Pinsonneault, A., & Kraemer, K. (1993). Survey research methodology in management information systems: an assessment. Journal of Management Information Systems, 10(2), 75–105. https://doi.org/10.1080/07421222.1993.11518001.

    Article  Google Scholar 

  • Shaffer, H. J., Tom, M., Wong, M. F. Y., Chan, E. M. L., Cheng, G. L. F., Lo, C. K. M., . . . Wong, R. H. Y. (2017). Treating addiction as a syndrome: bridging research and clinical practice. Trends in Medicine, (April). Retrieved from Trends in Medicine website: https://trends.hms.harvard.edu. Accessed 24 Oct 2019.

  • Shaffer, H. J., Tom, M. A., Wiley, R. C., Wong, M. F. Y., Chan, E. M. L., Cheng, G. L. F., et al. (2018). Using the syndrome model of addiction: a preliminary consideration of psychological states and traits. International Journal of Mental Health and Addiction, 16(6), 11373–11393. https://doi.org/10.1007/s11469-018-9952-2.

    Article  Google Scholar 

  • Simons, D. J. (2018). Introducing advances in methods and practices in psychological science. Advances in Methods and Practices in Psychological Science, 1(1), 3–6. https://doi.org/10.1177/2515245918757424.

    Article  Google Scholar 

  • Singleton, R., & Straits, B. C. (2005). Approaches to social research (4th ed.). New York: Oxford University Press.

    Google Scholar 

  • Sivo, S., Saunders, C., Chang, Q., & Jiang, J. (2006). How low should you go? Low response rates and the validity of inference in IS questionnaire research. Journal of the Association for Information Systems, 7(6), 351–356,359-360,362-372,374-414. https://doi.org/10.17705/1jais.00093.

    Article  Google Scholar 

  • Studer, J., Baggio, S., Mohler-Kuo, M., Dermota, P., Gaume, J., Bertholet, N., et al. (2013). Examining non-response bias in substance use research—are late respondents proxies for non-respondents? Drug and Alcohol Dependence, 132, 316–323. https://doi.org/10.1016/j.drugalcdep.2013.02.029.

    Article  PubMed  Google Scholar 

  • Tercyak, K., Johnson, S. B., Kirkpatrick, K. A., & Silverstein, J. H. (1998). Offering a randomized trial of intensive therapy for IDDM to adolescents. Diabetes Care, 21(2), 213.

    Article  Google Scholar 

  • The Insight Association. (2011). Code of standards & ethics for market, opinion, and social research. Port Jefferson: The Insight Association.

    Google Scholar 

  • Vachon, D. O., Susman, M., Wynne, M. E., Birringer, J., Olshefsky, L., & Cox, K. (1995). Reasons therapists give for refusing to participate in psychotherapy process research. Journal of Counseling Psychology, 42(3), 380–382.

    Article  Google Scholar 

  • Vander Bilt, J., Hall, M. N., Shaffer, H. J., Storti, S., & Church, O. (1997). An assessment of substance abuse treatment training needs among nurses (part I): evaluating skill, knowledge, and training characteristics. Journal of Substance Misuse for Nursing, Health and Social Care, 2(3), 150–157.

    Article  Google Scholar 

  • Volberg, R. A., & Bernhard, B. (2006). The 2006 study of gambling and problem gambling in New Mexico: report to the Responsible Gaming Association of New Mexico. Retrieved from University of Calgary Libraries and Cultural Resources website: https://prism.ucalgary.ca/handle/1880/44211. Accessed 24 Oct 2019.

  • Wiseman, F., & Billington, M. (1984). Journal of Marketing Research, 21(3), 336–338.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Tung Wah Group of Hospitals Integrated Centre on Addiction Prevention and Treatment, Hong Kong SAR, China

    Fung Yee Ching, Ryan Hing Yan Wong & Elda Mei Lo Chan

  2. Division on Addiction, The Cambridge Health Alliance, 350 Main Street, Suite 630, Malden, MA, 02148, USA

    Rhiannon Chou Wiley, Matthew A. Tom & Howard J. Shaffer

  3. Harvard Medical School, Boston, MA, USA

    Matthew A. Tom & Howard J. Shaffer

  4. St. John’s Cathedral Counselling Service, Hong Kong SAR, China

    Elda Mei Lo Chan

Authors
  1. Fung Yee Ching
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rhiannon Chou Wiley
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Matthew A. Tom
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ryan Hing Yan Wong
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Howard J. Shaffer
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Elda Mei Lo Chan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Matthew A. Tom.

Ethics declarations

Funding and Conflict of Interest

During approximately the past 5 years, the Division on Addiction received funding from a variety of sources. We do not believe any of the following funding sources constitute a conflict of interest for this project; however, we want to fully disclose our funding sources. The Division on Addiction received funding from the Foundation for Advancing Alcohol Responsibility (FAAR), The Healing Lodge of the Seven Nations via the National Institutes of Health (NIH) and Indian Health Services (IHS), the Integrated Centre on Addiction Prevention and Treatment of the Tung Wah Group of Hospitals—which receives funding from The Hong Kong Jockey Club Charities Trust, DraftKings, the Massachusetts Gaming Commission, GVC Holdings PLC, and the Massachusetts Department of Public Health. In addition, during approximately the past 5 years, Shaffer or the Division on Addiction received funding from National Center for Responsible Gambling, National Institutes of Health, the Alcohol Beverage Management Research Fund, the Danish Council for Independent Research, Heineken USA, Inc., bwin.party, St. Francis House, the State of Florida (i.e., as a subcontract to Spectrum Gaming Group), the Massachusetts Residential Substance Abuse Treatment for State Prisoners Grant Program (i.e., as a subcontracted evaluator for Worcester House of Corrections), and the Massachusetts Juvenile Accountability Block Grant Program—as a subcontracted evaluator for Cambridge Police Department). Howard Shaffer also has received speaker honoraria, or royalties, and compensation for consultation from the American Psychological Association, Las Vegas Sands Corp., Davies Ward Phillips and Vineberg, LLP, Freshfields Bruckhaus Deringer, LLP, and from the Dunes of Easthampton, a residential addiction treatment program, for serving as a consultant. Regarding his work with the international group on responsible gambling, he received reimbursement from Laval University for travel expenses, but no honorarium.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Public Policy Statement

Although response rates in social science research are of vital importance, these rates have been decreasing. Some researchers consider this state of affairs acceptable, but we argue that it is imperative and feasible to incorporate various techniques that can increase response rates into study designs. This paper provides an example from a clinical research program with addiction treatment seekers in Hong Kong.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ching, F.Y., Wiley, R.C., Tom, M.A. et al. A Brief Comment About Sample Size and Response Rates for Clinical Research: A Multi-site Case Example. Int J Ment Health Addiction 18, 1335–1345 (2020). https://doi.org/10.1007/s11469-019-00146-6

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11469-019-00146-6

Keywords

Search

Navigation