Skip to main content
SpringerLink
Log in

Planning for Long-Term Follow-Up: Strategies Learned from Longitudinal Studies

  • Published:
Prevention Science Aims and scope Submit manuscript

Abstract

Preventive interventions are often designed and tested with the immediate program period in mind, and little thought that the intervention sample might be followed up for years or even decades beyond the initial trial. However, depending on the type of intervention and the nature of the outcomes, long-term follow-up may well be appropriate. The advantages of long-term follow-up of preventive interventions are discussed and include the capacity to examine program effects across multiple later life outcomes, the ability to examine the etiological processes involved in the development of the outcomes of interest, and the ability to provide more concrete estimates of the relative benefits and costs of an intervention. In addition, researchers have identified potential methodological risks of long-term follow-up such as inflation of type 1 error through post hoc selection of outcomes, selection bias, and problems stemming from attrition over time. The present paper presents a set of seven recommendations for the design or evaluation of studies for potential long-term follow-up organized under four areas: Intervention Logic Model, Developmental Theory and Measurement Issues; Design for Retention; Dealing with Missing Data; and Unique Considerations for Intervention Studies. These recommendations include conceptual considerations in the design of a study, pragmatic concerns in the design and implementation of the data collection for long-term follow-up, as well as criteria to be considered for the evaluation of an existing intervention for potential for long-term follow-up. Concrete examples from existing intervention studies that have been followed up over the long term are provided.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  • Aos, S., Lieb, R., Mayfield, J., Miller, M., & Pennucci, A. (2004). Benefits and costs of prevention and early intervention programs for youth. Olympia: Washington State Institute for Public Policy.

    Google Scholar 

  • Bailey, J. A., Hill, K. G., Hawkins, J. D., Catalano, R. F., & Abbott, R. D. (2008). Men’s and women’s patterns of substance use around pregnancy. Birth: Issues in Perinatal Care, 35, 50–59.

    Article  Google Scholar 

  • Beldavs, Z., Forgatch, M. S., Patterson, G. R., & DeGarmo, G. (2006). Reducing the detrimental effects of divorce: enhancing the parental competence of single mothers. In N. Heinrichs, K. Haalweg, & M. Döpfner (Eds.), Strengthening families: evidence-based approaches to support child mental health (pp. 143–185). Munster: Psychotherapie.

    Google Scholar 

  • Beskow, L. M., Dame, L., & Costello, E. J. (2008). Certificates of confidentiality and compelled disclosure of data. Science, 322, 1054–1055.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Brown, E. C., Hawkins, J. D., Arthur, M. W., Briney, J. S., & Abbott, R. D. (2007). Effects of Communities That Care on prevention services systems: findings from the community youth development study at 1.5 years. Prevention Science, 8, 180–191.

    Article  PubMed  Google Scholar 

  • Catalano, R. F., & Hawkins, J. D. (1996). The social development model: a theory of antisocial behavior. In J. D. Hawkins (Ed.), Delinquency and crime: current theories (pp. 149–197). New York: Cambridge University Press.

    Google Scholar 

  • Chen, H.-T. (1990). Theory-driven evaluations. Thousand Oaks: Sage.

    Google Scholar 

  • Cohen, P. (1991). A source of bias in longitudinal investigations of change. In L. M. Collins & J. L. Horn (Eds.), Best methods for the analysis of change: recent advances, unanswered questions, future directions (pp. 18–30). Washington: American Psychological Association.

    Chapter  Google Scholar 

  • Collins, L. M. (1991). Measurement in longitudinal research. In L. M. Collins & J. L. Horn (Eds.), Best methods for the analysis of change: recent advances, unanswered questions, future directions (pp. 137–148). Washington: American Psychological Association.

    Chapter  Google Scholar 

  • Cotter, R. B., Burke, J. D., Loeber, R., & Navratil, J. L. (2002). Innovative retention methods in longitudinal research: a case study of the developmental trends study. Journal of Child and Family Studies, 11, 485–498.

    Article  Google Scholar 

  • Cotter, R. B., Burke, J. D., Stouthamer-Loeber, M., & Loeber, R. (2005). Contacting participants for follow-up: How much effort is required to retain participants in longitudinal studies? Evaluation and Program Planning, 28, 15–21.

    Article  Google Scholar 

  • Elder, G. H., Jr. (1998). The life course as developmental theory. Child Development, 69, 1–12.

    Article  PubMed  Google Scholar 

  • Epstein, M., Hill, K. G., Bailey, J. A., & Hawkins, J. D. (2013). The effect of general and drug-specific family environments on comorbid and drug-specific problem behavior: a longitudinal examination. Developmental Psychology, 49, 1151–1164.

    Article  PubMed  Google Scholar 

  • Farrington, D. P. (2006). Key longitudinal-experimental studies in criminology. Journal of Experimental Criminology, 2, 121–141.

    Article  Google Scholar 

  • Farrington, D. P., Gallagher, B., Morley, L., St. Ledger, R. J., & West, D. J. (1990). Minimizing attrition in longitudinal research: methods of tracing and securing cooperation in a 24-year follow-up study. In D. Magnusson & L. Bergman (Eds.), Data quality in longitudinal research (pp. 122–147). Cambridge: Cambridge University Press.

    Google Scholar 

  • Fernandez, C. V., Skedgel, C., & Weijer, C. (2004). Considerations and costs of disclosing study findings to research participants. Canadian Medical Association Journal, 170, 1417–1419.

    Article  PubMed  PubMed Central  Google Scholar 

  • Flay, B.R., Biglan, A., Boruch, R.F., Castro, F.G., Gottfredson, D., Kellam, S, Ji, P. (2005). Standards of evidence: criteria for efficacy, effectiveness and dissemination. Prevention Science, 6(3), 151–175.

  • Forgatch, M. S., & DeGarmo, D. S. (2007). Accelerating recovery from poverty: prevention effects for recently separated mothers. Journal of Early and Intensive Behavioral Intervention, 4, 681–702.

    Article  Google Scholar 

  • Foster, E. M., & McCombs-Thornton, K. (2012). The economics of evidence-based practice in disorders of childhood and adolescence. In P. Sturmey & M. Hersen (Eds.), Handbook of evidence-based practice in clinical psychology, vol 1: child and adolescent disorders (pp. 103–127). Hoboken: Wiley.

    Google Scholar 

  • Gest, S. D., & Davidson, A. J. (2011). A developmental perspective on risk, resilience, and prevention. In M. K. Underwood & L. H. Rosen (Eds.), Social development: relationships in infancy, childhood, and adolescence (pp. 427–454). New York: Guilford.

    Google Scholar 

  • Gottfredson, D.C., Cook, T.D., Gardner, F.E.M., Gorman-Smith, D., Howe, G.W., Sandler, I. N., Zafft, K.M. (2015). Standards of evidence for efficacy, effectiveness, and scale up research in prevention science: next generation. Prevention Science, 16, 893–926. doi:10.1007/s11121-015-0555-x.

  • Graham, J. W. (2009). Missing data analysis: making it work in the real world. Annual Review of Psychology, 60, 549–576.

    Article  PubMed  Google Scholar 

  • Grant, S., Mayo-Wilson, E., Hopewell, S., Macdonald, G., Moher, D., & Montgomery, P. (2013). Developing a reporting guideline for social and psychological intervention trials. Journal of Experimental Criminology, 9, 355–367.

    Article  Google Scholar 

  • Haggerty, K. P., Fleming, C. B., Catalano, R. F., Petrie, R. S., Rubin, R. J., & Grassley, M. H. (2008). Ten years later: locating and interviewing children of drug abusers. Evaluation and Program Planning, 31, 1–9.

    Article  PubMed  Google Scholar 

  • Hansen, W.B., & Collins, L.M. (1994). Seven ways to increase power without increasing. In National Institute on Drug Abuse Research monograph series vol. 142 (pp. 184–195). Washington, DC: US Department of Health and Human Services.

  • Hansen, W. B., Tobler, N. S., & Graham, J. W. (1990). Attrition in substance abuse prevention research: a meta-analysis of 85 longitudinally followed cohorts. Evaluation Review, 14, 677–685.

    Article  Google Scholar 

  • Hawkins, J. D., Catalano, R. F., Morrison, D. M., O’Donnell, J., Abbott, R. D., & Day, L. E. (1992). The Seattle Social Development Project: effects of the first four years on protective factors and problem behaviors. In J. McCord & R. E. Tremblay (Eds.), Preventing antisocial behavior: interventions from birth through adolescence (pp. 139–161). New York: Guilford.

    Google Scholar 

  • Hawkins, J. D., Catalano, R. F., Kosterman, R., Abbott, R., & Hill, K. G. (1999). Preventing adolescent health-risk behaviors by strengthening protection during childhood. Archives of Pediatrics and Adolescent Medicine, 153, 226–234.

    Article  CAS  PubMed  Google Scholar 

  • Hawkins, J. D., Guo, J., Hill, K. G., Battin-Pearson, S., & Abbott, R. D. (2001). Long-term effects of the Seattle Social Development intervention on school bonding trajectories. Applied Developmental Science: Special Issue: Prevention as Altering the Course of Development, 5, 225–236.

    Google Scholar 

  • Hawkins, J. D., Kosterman, R., Catalano, R. F., Hill, K. G., & Abbott, R. D. (2005). Promoting positive adult functioning through social development intervention in childhood: long-term effects from the Seattle Social Development Project. Archives of Pediatrics and Adolescent Medicine, 159, 25–31.

    Article  PubMed  Google Scholar 

  • Hawkins, J. D., Brown, E. C., Oesterle, S., Arthur, M. W., Abbott, R. D., & Catalano, R. F. (2008a). Early effects of Communities That Care on targeted risks and initiation of delinquent behavior and substance use. Journal of Adolescent Health, 43, 15–22.

    Article  PubMed  Google Scholar 

  • Hawkins, J. D., Catalano, R. F., Arthur, M. W., Egan, E., Brown, E. C., Abbott, R. D., & Murray, D. M. (2008b). Testing Communities that Care: the rationale, design and behavioral baseline equivalence of the community youth development study. Prevention Science, 9, 178–190.

    Article  PubMed  PubMed Central  Google Scholar 

  • Hawkins, J. D., Kosterman, R., Catalano, R. F., Hill, K. G., & Abbott, R. D. (2008c). Effects of social development intervention in childhood 15 years later. Archives of Pediatrics & Adolescent Medicine, 162, 1133–1141.

    Article  Google Scholar 

  • Hill, K. G., Hawkins, J. D., Catalano, R. F., Abbott, R. D., & Guo, J. (2005). Family influences on the risk of daily smoking initiation. Journal of Adolescent Health, 37, 202–210.

    Article  PubMed  Google Scholar 

  • Hill, K.G., Hawkins, J.D., Bailey, J.A., Catalano, R.F., Abbott, R.D., Shapiro, V.B. (2010). Person-environment interaction in the prediction of alcohol abuse and alcohol dependence in adulthood. Drug and Alcohol Dependence, [Epub ahead of print.] (March 16).

  • Hill, K. G., Bailey, J. A., Hawkins, J. D., Catalano, R. F., Kosterman, R., Oesterle, S., & Abbott, R. D. (2013). The onset of STI diagnosis through age 30: results from the Seattle Social Development Project intervention. Prevention Science. doi:10.1007/s11121-013-0382-x.

    PubMed  PubMed Central  Google Scholar 

  • Holder, H. (2009). Prevention programs in the 21st century: what we do not discuss in public. Addiction, 105, 578–581.

    Article  PubMed  Google Scholar 

  • Howard, W. J., Rhemtulla, M., & Little, T. D. (2015). Using Principal Components as Auxiliary Variables in Missing Data Estimation. Multivariate Behavioral Research, 50, 285–299. doi:10.1080/00273171.2014.999267.

  • Kuklinski, M. R., Briney, J. S., Hawkins, J. D., & Catalano, R. F. (2012). Cost-benefit analysis of Communities That Care outcomes at eighth grade. Prevention Science, 13, 150–161.

    Article  PubMed  PubMed Central  Google Scholar 

  • Latimer, N. R. (2013). Survival analysis for economic evaluations alongside clinical trials—extrapolation with patient-level data: inconsistencies, limitations, and a practical guide. Medical Decision Making, 33, 743–754.

    Article  PubMed  Google Scholar 

  • Lonczak, H. S., Abbott, R. D., Hawkins, J. D., Kosterman, R., & Catalano, R. F. (2002). Effects of the Seattle Social Development Project on sexual behavior, pregnancy, birth, and sexually transmitted disease outcomes by age 21 years. Archives of Pediatrics and Adolescent Medicine, 156, 438–447.

    Article  PubMed  Google Scholar 

  • MacKinnon, D. P. (2008). Introduction to statistical mediation analysis. New York: Taylor & Francis/Lawrence Erlbaum Associates.

    Google Scholar 

  • Magnusson, D., & Bergman, L. R. (Eds.). (1990). Data quality in longitudinal research. New York: Cambridge University Press.

    Google Scholar 

  • Masten, A. S., & Cicchetti, D. (2010). Developmental cascades. Development and Psychopathology, 22, 491–495.

    Article  PubMed  Google Scholar 

  • Moher, D., Hopewell, S., Schulz, K. F., Montori, V., Gøtzsche, P. C., Devereaux, P. J., Altman, D. G. (2010). CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trials. Journal of Clinical Epidemiology, 63(8), e1–e37. doi:10.1016/j.jclinepi.2010.03.004

  • Nicolaas, G. (2011). Survey paradata: a review. Retrieved February 2015 from Economic and Social Research Council National Center for Research Methods http://eprints.ncrm.ac.uk/1719/

  • Oesterle, S., Hawkins, J. D., Hill, K. G., & Bailey, J. A. (2010). Men’s and women’s pathways to adulthood and their adolescent precursors. Journal of Marriage and Family, 72, 1436–1453.

    Article  PubMed  PubMed Central  Google Scholar 

  • Olweus, D., & Limber, S. P. (2010). Bullying in school: evaluation and dissemination of the Olweus Bullying Prevention Program. American Journal of Orthopsychiatry, 80, 124–134.

    Article  PubMed  Google Scholar 

  • Patterson, G. R., & Fleischman, M. J. (1979). Maintenance of treatment effects: some considerations concerning family systems and follow-up data. Behavior Therapy, 10, 168–185.

    Article  Google Scholar 

  • Patterson, G. R., Forgatch, M. S., & DeGarmo, D. S. (2010). Cascading effects following intervention. Development and Psychopathology, 22, 949–970.

    Article  PubMed  PubMed Central  Google Scholar 

  • Plotnick, R. D. (1994). Applying benefit-cost analysis to substance use prevention programs. International Journal of the Addictions, 29, 339–359.

    Article  CAS  PubMed  Google Scholar 

  • Sanders, M.R., & Kirby, J.N. (2014). Surviving or thriving: quality assurance mechanisms to promote innovation in the development of evidence-based parenting interventions. Prevention Science, 1–11. doi:10.1007/s11121-014-0475-1

  • Sandler, I. N., Schoenfelder, E. N., Wolchik, S. A., & MacKinnon, D. P. (2011). Long-term impact of prevention programs to promote effective parenting: lasting effects but uncertain processes. Annual Review of Psychology, 62, 299–329.

    Article  PubMed  PubMed Central  Google Scholar 

  • Slade, E. P., & Becker, K. D. (2014). Understanding proximal–distal economic projections of the benefits of childhood preventive interventions. Prevention Science, 15, 807–817.

    Article  PubMed  PubMed Central  Google Scholar 

  • Stouthamer-Loeber, M., & van Kammen, W. B. (1995). Data collection and management: a practical guide. Thousand Oaks: Sage.

    Book  Google Scholar 

  • van Kammen, W. B., & Stouthamer-Loeber, M. (1998). Practical aspects of interview data collection and data management. In L. Bickman, D. J. Rog, L. Bickman, & D. J. Rog (Eds.), Handbook of applied social research methods (pp. 375–397). Thousand Oaks: Sage.

    Google Scholar 

  • Wolf, L. E., Dame, L. E., Patel, M. J., Williams, B. A., Austin, J. A., & Beskow, L. M. (2012). Certificates of confidentiality: legal counsels’ experiences with and perspectives on legal demands for research data. Journal of Empirical Research on Human Research Ethics, 7, 1–9.

    PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgments

This project was supported by the National Institute on Drug Abuse (NIDA; R01DA009679, R01DA024411-05-07) and grant 21548 from the Robert Wood Johnson Foundation. The content is solely the responsibility of the authors. The authors gratefully acknowledge our study participants for their continued contribution to the longitudinal studies. They also acknowledge the SDRG Survey Research Division for their hard work in maintaining high panel retention. Earlier versions of this manuscript were presented at an expert panel meeting on “Impact of Early Interventions on Trajectories of Violence” in October of 2010 organized by CDC as well as at the Society for Prevention Research meeting in 2011. The paper has been expanded to consider long-term follow-up of preventive interventions in general.

Author information

Authors and Affiliations

  1. Social Development Research Group, School of Social Work, University of Washington, 9725 3rd Ave. N.E., Suite 401, Seattle, WA, 98115, USA

    Karl G. Hill, Danielle Woodward, Tiffany Woelfel, J. David Hawkins & Sara Green

Authors
  1. Karl G. Hill
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Danielle Woodward
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tiffany Woelfel
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. J. David Hawkins
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Sara Green
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Karl G. Hill.

Ethics declarations

Conflict of Interest

The authors declare that they have no competing interests.

Ethical Approval

Formal ethics review is not applicable for this theoretical/methodological paper; however, the work is consistent with COPE guidelines.

Informed Consent

Informed consent is not applicable for this work.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hill, K.G., Woodward, D., Woelfel, T. et al. Planning for Long-Term Follow-Up: Strategies Learned from Longitudinal Studies. Prev Sci 17, 806–818 (2016). https://doi.org/10.1007/s11121-015-0610-7

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11121-015-0610-7

Keywords

Search

Navigation