Skip to main content
SpringerLink
Log in

Derivation and Evaluation of a Risk-Scoring Tool to Predict Participant Attrition in a Lifestyle Intervention Project

  • Published:
Prevention Science Aims and scope Submit manuscript

Abstract

Participant attrition in clinical trials and community-based interventions is a serious, common, and costly problem. In order to develop a simple predictive scoring system that can quantify the risk of participant attrition in a lifestyle intervention project, we analyzed data from the Special Diabetes Program for Indians Diabetes Prevention Program (SDPI-DP), an evidence-based lifestyle intervention to prevent diabetes in 36 American Indian and Alaska Native communities. SDPI-DP participants were randomly divided into a derivation cohort (n = 1600) and a validation cohort (n = 801). Logistic regressions were used to develop a scoring system from the derivation cohort. The discriminatory power and calibration properties of the system were assessed using the validation cohort. Seven independent factors predicted program attrition: gender, age, household income, comorbidity, chronic pain, site’s user population size, and average age of site staff. Six factors predicted long-term attrition: gender, age, marital status, chronic pain, site’s user population size, and average age of site staff. Each model exhibited moderate to fair discriminatory power (C statistic in the validation set: 0.70 for program attrition, and 0.66 for long-term attrition) and excellent calibration. The resulting scoring system offers a low-technology approach to identify participants at elevated risk for attrition in future similar behavioral modification intervention projects, which may inform appropriate allocation of retention resources. This approach also serves as a model for other efforts to prevent participant attrition.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Barnes, P. M., Adams, P. F., & Powell-Griner, E. (2005). Health characteristics of the American Indian and Alaska Native adult population: United States, 1999–2003 advance data, from vital and health statistics (Vol. 356). Hyattsville: US Department of Health and Human Services, National Center for Health Statistics.

    Google Scholar 

  • Blanton, S., Morris, D. M., Prettyman, M. G., McCulloch, K., Redmond, S., Light, K. E., et al. (2006). Lessons learned in participant recruitment and retention: The EXCITE trial. Physical Therapy, 86, 1520–1533. doi:10.2522/ptj.20060091.

    Article  PubMed  Google Scholar 

  • Bradley, E. H., Yakusheva, O., Horwitz, L. I., Sipsma, H., & Fletcher, J. (2013). Identifying patients at increased risk for unplanned readmission. Medical Care, 51, 761–766. doi:10.1097/MLR.0b013e3182a0f492.

    Article  PubMed  PubMed Central  Google Scholar 

  • Brown, D. M., Thorne, J. E., Foster, G. L., Duncan, J. L., Brune, L. M., Munana, A., et al. (2006). Factors affecting attrition in a longitudinal study of patients with AIDS. AIDS Care, 18, 821–829. doi:10.1080/09540120500466747.

    Article  CAS  PubMed  Google Scholar 

  • Carlsson, A. M. (1983). Assessment of chronic pain. I. Aspects of the reliability and validity of the visual analogue scale. Pain, 16, 87–101.

    Article  CAS  PubMed  Google Scholar 

  • CDC. (2011). 2011 National Diabetes Fact Sheet. Retrieved 08 April 2011, from http://www.cdc.gov/diabetes/pibs/estimates11.htm.

  • Clark, M. M., Niaura, R., King, T. K., & Pera, V. (1996). Depression, smoking, activity level, and health status: Pretreatment predictors of attrition in obesity treatment. Addictive Behaviors, 21, 509–513.

    Article  CAS  PubMed  Google Scholar 

  • D’Agostino, R. B., Sr., Vasan, R. S., Pencina, M. J., Wolf, P. A., Cobain, M., Massaro, J. M., et al. (2008). General cardiovascular risk profile for use in primary care: The Framingham Heart Study. Circulation, 117, 743–753. doi:10.1161/CIRCULATIONAHA.107.699579.

    Article  PubMed  Google Scholar 

  • Dalle Grave, R., Calugi, S., Molinari, E., Petroni, M. L., Bondi, M., Compare, A., et al. (2005). Weight loss expectations in obese patients and treatment attrition: An observational multicenter study. Obesity Research, 13, 1961–1969. doi:10.1038/oby.2005.241.

    Article  PubMed  Google Scholar 

  • Donze, J., Aujesky, D., Williams, D., & Schnipper, J. L. (2013). Potentially avoidable 30-day hospital readmissions in medical patients: Derivation and validation of a prediction model. JAMA Internal Medicine, 173, 632–638. doi:10.1001/jamainternmed.2013.3023.

    Article  PubMed  Google Scholar 

  • Exalto, L. G., Biessels, G. J., Karter, A. J., Huang, E. S., Katon, W. J., Minkoff, J. R., et al. (2013). Risk score for prediction of 10 year dementia risk in individuals with type 2 diabetes: A cohort study. Lancet Diabetes Endocrinology, 1, 183–190. doi:10.1016/S2213-8587(13)70048-2.

    Article  PubMed  PubMed Central  Google Scholar 

  • Fabricatore, A. N., Wadden, T. A., Moore, R. H., Butryn, M. L., Heymsfield, S. B., & Nguyen, A. M. (2009). Predictors of attrition and weight loss success: Results from a randomized controlled trial. Behavioral Research Therapy, 47, 685–691. doi:10.1016/j.brat.2009.05.004.

    Article  Google Scholar 

  • Fitzpatrick, S. L., Jeffery, R., Johnson, K. C., Roche, C. C., Van Dorsten, B., Gee, M., et al. (2014). Baseline predictors of missed visits in the Look AHEAD study. Obesity, 22, 131–140. doi:10.1002/oby.20613.

    Article  PubMed  PubMed Central  Google Scholar 

  • Garfield, S. A., Malozowski, S., Chin, M. H., Narayan, K. M., Glasgow, R. E., Green, L. W., et al. (2003). Considerations for diabetes translational research in real-world settings. Diabetes Care, 26, 2670–2674.

    Article  PubMed  Google Scholar 

  • Hanley, J. A., Negassa, A., Edwardes, M. D., & Forrester, J. E. (2003). Statistical analysis of correlated data using generalized estimating equations: An orientation. American Journal of Epidemiology, 157, 364–375.

    Article  PubMed  Google Scholar 

  • Honas, J. J., Early, J. L., Frederickson, D. D., & O’Brien, M. S. (2003). Predictors of attrition in a large clinic-based weight-loss program. Obesity Research, 11, 888–894. doi:10.1038/oby.2003.122.

    Article  PubMed  Google Scholar 

  • Hosmer, D. W., & Lemeshow, S. (2000). Applied logistic regression (2nd ed.). New York: Wiley.

    Book  Google Scholar 

  • Hosmer, D. W., Hosmer, T., Le Cessie, S., & Lemeshow, S. (1997). A comparison of goodness-of-fit tests for the logistic regression model. Statistics in Medicine, 16, 965–980.

    Article  CAS  PubMed  Google Scholar 

  • Jiang, L., Manson, S. M., Beals, J., Henderson, W. G., Huang, H., Acton, K. J., et al. (2013). Translating the Diabetes Prevention Program into American Indian and Alaska Native communities: Results from the Special Diabetes Program for Indians Diabetes Prevention demonstration project. Diabetes Care, 36, 2027–2034. doi:10.2337/dc12-1250.

    Article  PubMed  PubMed Central  Google Scholar 

  • Jiang, L., Manson, S. M., Dill, E. J., Beals, J., Johnson, A., Huang, H., et al. (2015). Participant and site characteristics related to participant retention in a diabetes prevention translational project. Prevention Science, 16, 41–52. doi:10.1007/s11121-013-0451-1.

    Article  PubMed  PubMed Central  Google Scholar 

  • Johnson, S. B., Lynch, K. F., Lee, H. S., Smith, L., Baxter, J., Lernmark, B., et al. (2014). At high risk for early withdrawal: Using a cumulative risk model to increase retention in the first year of the TEDDY study. Journal of Clinical Epidemiology, 67, 609–611. doi:10.1016/j.jclinepi.2014.01.004.

    Article  PubMed  PubMed Central  Google Scholar 

  • Kahn, H. S., Cheng, Y. J., Thompson, T. J., Imperatore, G., & Gregg, E. W. (2009). Two risk-scoring systems for predicting incident diabetes mellitus in U.S. adults age 45 to 64 years. Annals of Internal Medicine, 150, 741–751.

    Article  PubMed  Google Scholar 

  • Kansagara, D., Englander, H., Salanitro, A., Kagen, D., Theobald, C., Freeman, M., et al. (2011). Risk prediction models for hospital readmission: A systematic review. Journal of the American Medical Association, 306, 1688–1698. doi:10.1001/jama.2011.1515.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Kealey, K. A., Ludman, E. J., Mann, S. L., Marek, P. M., Phares, M. M., Riggs, K. R., et al. (2007). Overcoming barriers to recruitment and retention in adolescent smoking cessation. Nicotine & Tobacco Research, 9, 257–270. doi:10.1080/14622200601080315.

    Article  Google Scholar 

  • Kong, W., Langlois, M. F., Kamga-Ngande, C., Gagnon, C., Brown, C., & Baillargeon, J. P. (2010). Predictors of success to weight-loss intervention program in individuals at high risk for type 2 diabetes. Diabetes Research Clinical Practice, 90, 147–153. doi:10.1016/j.diabres.2010.06.031.

    Article  PubMed  Google Scholar 

  • Lee, E. W., Wei, L. J., & Amato, D. (1992). Cox-type regression analysis for large numbers of small groups of correlated failure time observations survival analysis: State of the Art (pp. 237–347). Netherlands: Kluwer.

    Google Scholar 

  • Lee, E. T., Howard, B. V., Wang, W., Welty, T. K., Galloway, J. M., Best, L. G., et al. (2006). Prediction of coronary heart disease in a population with high prevalence of diabetes and albuminuria: The Strong Heart Study. Circulation, 113, 2897–2905. doi:10.1161/CIRCULATIONAHA.105.593178.

    Article  PubMed  Google Scholar 

  • Lindstrom, J., & Tuomilehto, J. (2003). The diabetes risk score: A practical tool to predict type 2 diabetes risk. Diabetes Care, 26, 725–731.

    Article  PubMed  Google Scholar 

  • Manson, S. M., Jiang, L., Zhang, L., Beals, J., Acton, K. J., & Roubideaux, Y. (2011). Special diabetes program for Indians: Retention in cardiovascular risk reduction. Gerontologist, 51, S21–S32. doi:10.1093/geront/gnq083.

    Article  PubMed  PubMed Central  Google Scholar 

  • McGeechan, K., Macaskill, P., Irwig, L., Liew, G., & Wong, T. Y. (2008). Assessing new biomarkers and predictive models for use in clinical practice: A clinician’s guide. Archives of Internal Medicine, 168, 2304–2310. doi:10.1001/archinte.168.21.2304.

    Article  PubMed  Google Scholar 

  • McGuigan, W. M., Katzev, A. R., & Pratt, C. C. (2003). Multi-level determinants of retention in a home-visiting child abuse prevention program. Child Abuse & Neglect, 27, 363–380.

    Article  Google Scholar 

  • Noble, D., Mathur, R., Dent, T., Meads, C., & Greenhalgh, T. (2011). Risk models and scores for type 2 diabetes: Systematic review. British Medical Journal, 343, d7163. doi:10.1136/bmj.d7163.

    Article  PubMed  PubMed Central  Google Scholar 

  • O’Brien, R. A., Moritz, P., Luckey, D. W., McClatchey, M. W., Ingoldsby, E. M., & Olds, D. L. (2012). Mixed methods analysis of participant attrition in the nurse-family partnership. Prevention Science, 13, 219–228. doi:10.1007/s11121-012-0287-0.

    Article  PubMed  PubMed Central  Google Scholar 

  • Probstfield, J. L., & Frye, R. L. (2011). Strategies for recruitment and retention of participants in clinical trials. Journal of the American Medical Association, 306, 1798–1799. doi:10.1001/jama.2011.1544.

    CAS  PubMed  Google Scholar 

  • Rothberg, A. E., McEwen, L. N., Kraftson, A. T., Ajluni, N., Fowler, C. E., Miller, N. M., et al. (2015). Factors associated with participant retention in a clinical, intensive, behavioral weight management program. BMC Obesity, 2, 11. doi:10.1186/s40608-015-0041-9.

    Article  PubMed  PubMed Central  Google Scholar 

  • Sangha, O., Stucki, G., Liang, M. H., Fossel, A. H., & Katz, J. N. (2003). The self-administered comorbidity questionnaire: A new method to assess comorbidity for clinical and health services research. Arthritis & Rheumatology, 49, 156–163. doi:10.1002/art.10993.

    Article  Google Scholar 

  • Snow, W. M., Connett, J. E., Sharma, S., & Murray, R. P. (2007). Predictors of attendance and dropout at the Lung Health Study 11-year follow-up. Contemporary Clinical Trials, 28, 25–32. doi:10.1016/j.cct.2006.08.010.

    Article  PubMed  PubMed Central  Google Scholar 

  • Spring, B., Sohn, M. W., Locatelli, S. M., Hadi, S., Kahwati, L., & Weaver, F. M. (2014). Individual, facility, and program factors affecting retention in a national weight management program. BMC Public Health, 14, 363. doi:10.1186/1471-2458-14-363.

    Article  PubMed  PubMed Central  Google Scholar 

  • The DPP Research Group. (2002). The diabetes prevention program (DPP): Description of lifestyle intervention. Diabetes Care, 25, 2165–2171.

    Article  Google Scholar 

  • Warren-Findlow, J., Prohaska, T. R., & Freedman, D. (2003). Challenges and opportunities in recruiting and retaining underrepresented populations into health promotion research. The Gerontologist, 43, 37–46.

    Article  PubMed  PubMed Central  Google Scholar 

  • Williams, P. L., Van Dyke, R., Eagle, M., Smith, D., Vincent, C., Ciupak, G., et al. (2008). Association of site-specific and participant-specific factors with retention of children in a long-term pediatric HIV cohort study. American Journal of Epidemiology, 167, 1375–1386. doi:10.1093/aje/kwn072.

    Article  PubMed  PubMed Central  Google Scholar 

  • Wilson, P. W., D’Agostino, R. B., Levy, D., Belanger, A. M., Silbershatz, H., & Kannel, W. B. (1998). Prediction of coronary heart disease using risk factor categories. Circulation, 97, 1837–1847.

    Article  CAS  PubMed  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Epidemiology, School of Medicine, University of California Irvine, 205B Irvine Hall, Irvine, CA, 92697-7550, USA

    Luohua Jiang

  2. Department of Epidemiology and Biostatistics, School of Rural Public Health, Texas A&M Health Science Center, College Station, TX, USA

    Luohua Jiang & Jing Yang

  3. Centers for American Indian and Alaska Native Health, Colorado School of Public Health, University of Colorado Anschutz Medical Campus, Aurora, CO, USA

    Haixiao Huang, Ann Johnson, Janette Beals & Spero M. Manson

  4. Department of Psychology, University of Colorado Denver, Denver, CO, USA

    Edward J. Dill

  5. Elson S. Floyd College of Medicine, Washington State University, Spokane, WA, USA

    Yvette Roubideaux

Authors
  1. Luohua Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jing Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Haixiao Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ann Johnson
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Edward J. Dill
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Janette Beals
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Spero M. Manson
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Yvette Roubideaux
    View author publications

    You can also search for this author in PubMed Google Scholar

Consortia

the Special Diabetes Program for Indians Diabetes Prevention Demonstration Project

Corresponding author

Correspondence to Luohua Jiang.

Ethics declarations

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards. The SDPI-DP protocol was approved by the institutional review board (IRB) of the University of Colorado Denver and the IHS IRB. When required, grantees obtained approval from other entities charged with overseeing research in their programs (e.g., tribal review boards).

Conflict of Interest

The authors declare that they have no competing interests.

Informed Consent

All participants provided written informed consent and Health Insurance Portability and Accountability Act authorization.

Funding

Funding for SDPI-DP project was provided by the Indian Health Service (HHSI242200400049C, S. Manson). Manuscript preparation was supported in part by American Diabetes Association (ADA #7-12-CT-36, L. Jiang) and the National Institute of Diabetes and Digestive and Kidney Diseases (1P30DK092923, S.M. Manson).

Additional information

Grant programs participating in the Special Diabetes Program for Indians Diabetes Prevention Demonstration Project: Confederated Tribes of the Chehalis Reservation, Cherokee Nation, Cheyenne River Sioux Tribe, the Chickasaw Nation, Coeur d’Alene Tribe, Colorado River Indian Tribes, Colville Confederated Tribes, Cow Creek Band of Umpqua Tribe, Fond du Lac Reservation, Gila River Health Care, Haskell Health Center, Ho-Chunk Nation, Indian Health Board of Minneapolis, Urban Native Diabetes Prevention Consortium, Kenaitze Indian Tribe IRA, Lawton IHS Service Unit, Menominee Indian Tribe of Wisconsin, Mississippi Band of Choctaw Indians, Norton Sound Health Corporation, Pine Ridge IHS Service Unit, Pueblo of San Felipe, Quinault Indian Nation, Rapid City IHS Diabetes Program, Red Lake Comprehensive Health Services, Rocky Boy Health Board, Seneca Nation of Indians, Sonoma County Indian Health Project, South East Alaska Regional Health Consortium, Southcentral Foundation, Trenton Indian Service Area, Tuba City Regional Health Care Corporation, United American Indian Involvement, Inc., United Indian Health Services, Inc., Warm Springs Health & Wellness Center, Winnebago Tribe of Nebraska, Zuni Pueblo.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jiang, L., Yang, J., Huang, H. et al. Derivation and Evaluation of a Risk-Scoring Tool to Predict Participant Attrition in a Lifestyle Intervention Project. Prev Sci 17, 461–471 (2016). https://doi.org/10.1007/s11121-015-0628-x

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11121-015-0628-x

Keywords

Search

Navigation