Skip to main content

Intervening with Practitioners to Improve the Quality of Prevention: One-Year Findings from a Randomized Trial of Assets-Getting To Outcomes

The Journal of Primary Prevention volume 34, pages 173–191 (2013)Cite this article

Abstract

There continues to be a gap in prevention outcomes achieved in research trials versus those achieved in “real-world” practice. This article reports interim findings from a randomized controlled trial evaluating Assets-Getting To Outcomes (AGTO), a two-year intervention designed to build prevention practitioners’ capacity to implement positive youth development–oriented practices in 12 community coalitions in Maine. A survey of coalition members was used to assess change on individual practitioners’ prevention capacity between baseline and one year later. Structured interviews with 32 program directors (16 in the intervention group and 16 in the control group) were used to assess changes in programs’ prevention practices during the same time period. Change in prevention capacity over time did not differ significantly between the intervention and control groups. However, in secondary analyses of only those assigned to the AGTO intervention, users showed greater improvement in their self-efficacy to conduct Assets–based programming and increases in the frequency with which they engaged in AGTO behaviors, whereas among non-users, self-efficacy to conduct Assets–based programming declined. Interview ratings showed improvement in several key areas of performance among intervention programs. Improvement was associated with the number of technical assistance hours received. These results suggest that, after one year, AGTO is beginning to improve the capacity of community practitioners who make use of it.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price includes VAT (USA)
Tax calculation will be finalised during checkout.

Fig. 1
Fig. 2
Fig. 3

Notes

  1. “Getting To Outcomes” and “GTO” are trademarks registered by the University of South Carolina and the RAND Corporation.

  2. “Developmental Assets” is a trademark registered by the Search Institute.

  3. Of the 60 programs initially randomized, nine folded between the time the study was planned and begun, leaving only 51 programs.

References

  • Acosta, J., & Chinman, M. (2011). Building capacity for outcomes measurement in human service organizations: The “Getting To Outcomes” (GTO) method. In J. L. Magnabosco & R. W. Manderscheid (Eds.), Outcomes measurement in the human services: Cross-cutting issues and methods (2nd ed.). Washington, DC: NASW Press.

    Google Scholar 

  • Ajzen, I., & Fishbein, M. (1977). Attitude-behavior relations: A theoretical analysis and review of empirical research. Psychological Bulletin, 84, 888–918.

    Article  Google Scholar 

  • Backer, T. E. (2001). Finding the balance: Program fidelity and adaptation in substance abuse prevention: A state-of-the-art review. Rockville, MD: Substance Abuse and Mental Health Administration.

    Google Scholar 

  • Bandura, A. (2004). Health promotion by social cognitive means. Health Education & Behavior, 31, 143–164.

    Article  Google Scholar 

  • Battelle. (2008). Annual findings report: Drug-free communities support program national evaluation. Washington, DC: Executive Office of the President Office of Administration Office of National Drug Control Policy.

  • Benson, P. L. (2002). Toward asset-building communities: How does change occur? In R. M. Lerner & P. L. Benson (Eds.), Developmental assets and asset-building communities: Implications for research policy and practice. Norwell, MA: Kluwer Academic.

    Google Scholar 

  • Benson, P. L., & Scales, P. C. (2011). Developmental assets. In R. J. R. Levesque (Ed.), Encyclopedia of adolescence. New York, NY: Springer.

    Google Scholar 

  • Benson, P. L., Scales, P. C., Leffert, N., & Roehlkepartain, E. R. (1999). A fragile foundation: The state of developmental assets among American youth. Minneapolis, MN: Search Institute.

    Google Scholar 

  • Benson, P. L., Scales, P. C., & Mannes, M. (2003). Developmental strengths and their sources: Implications for the study and practice of community building. In R. M. Lerner, F. Jacobs, & D. Wertlieb (Eds.), Handbook of applied developmental science. Applying developmental science for youth and families (Vol. 1, pp. 369–406). Thousand Oaks, CA: Sage.

    Google Scholar 

  • Benson, P. L., Scales, P. C., Hamilton, S. F., & Sesma, A. (2006). Positive youth development: Theory, research, and applications. In W. Damon, & R. M. Lerner (Eds.), Handbook of child psychology (6th ed., pp. 894–941). New York, NY: Wiley.

  • Botvin, G. J., Baker, E., Dusenbury, L., Botvin, E. M., & Diaz, T. (1995). Long-term follow-up results of a randomized drug abuse prevention trial in a white middle-class population. Journal of the American Medical Association, 273, 1106–1112.

    PubMed  Article  CAS  Google Scholar 

  • Bryt, T., Bishop, J., & Carlin, J. B. (1993). Bias, prevalence and kappa. Journal of Clinical Epidemiology, 46, 423–429.

    Article  Google Scholar 

  • Butterfoss, F. D., Goodman, R. M., & Wandersman, A. (1993). Community coalitions for prevention and health promotion. Health Education Research, 8, 315–330.

    PubMed  Article  CAS  Google Scholar 

  • Cantor, D., Crosse, S., Hagen, C. A., Mason, M. J., Siler, A. J., & von Glatz, A. (2001). A closer look at drug and violence prevention efforts in American schools: Report on the study on school violence and prevention. Washington, DC: U.S. Department of Education, Planning and Evaluation Service.

    Google Scholar 

  • Chinman, M., Imm, P., & Wandersman, A. (2004). Getting To Outcomes 2004: Promoting accountability through methods and tools for planning, implementation, and evaluation (No. TR-TR101). Retrieved June 1, 2011, from http://www.rand.org/publications/TR/TR101/.

  • Chinman, M., Hannah, G., Wandersman, A., Ebener, P., Hunter, S., Imm, P., et al. (2005). Developing a community science research agenda for building community capacity for effective preventive interventions. American Journal of Community Psychology, 3, 143–157.

    Article  Google Scholar 

  • Chinman, M., Hunter, S., Ebener, P., Paddock, S., Stillman, L., Imm, P., et al. (2008). The Getting To Outcomes demonstration and evaluation: An illustration of the Prevention Support System. American Journal of Community Psychology, 41, 206–224.

    PubMed  Article  Google Scholar 

  • Chinman, M., Tremain, B., Imm, P., & Wandersman, A. (2009). Strengthening prevention performance using technology: A formative evaluation of interactive Getting To Outcomes. The American Journal of Orthopsychiatry, 79, 469–481.

    PubMed  Article  Google Scholar 

  • Chinman, M., Acosta, J., Ebener, P., Burkhart, Q., Clifford, M., Corsello, M., et al. (2012). Establishing and evaluating the key functions of an interactive systems framework using an assets-getting to outcomes intervention. American Journal of Community Psychology, 50, 295–310.

    PubMed  Article  Google Scholar 

  • Cividin, T. M., & Ottoson, J. M. (1997). Linking reasons for continuing professional education participation with post-program application. Journal of Continuing Education in the Health Professions, 17, 46–55.

    Article  Google Scholar 

  • Collins, C., Harshbarger, C., Sawyer, R., & Hamdallah, M. (2006). The Diffusion of Effective Behavioral Interventions project: Development, implementation, and lessons learned. AIDS Education and Prevention, 18(4 Suppl. A), 5–20.

    Google Scholar 

  • Crosse, S., Burr, M., Cantor, D., Hagen, C. A., & Hantman, I. (2001). Wide scope, questionable quality: Drug and violence prevention efforts in American schools. Report on the study on school violence and prevention. Washington, DC: U.S. Department of Education, Planning and Evaluation Service.

  • Crosse, S., Williams, B., Hagen, C. A., Harmon, M., Ristow, L., DiGaetano, R., et al. (2011). Prevalence and implementation fidelity of research-based prevention programs in public schools: Final report. Washington, DC: U.S. Department of Education, Planning and Evaluation Service.

    Google Scholar 

  • Donner, A., & Klar, N. (2004). Pitfalls of and controversies in cluster randomization trials. American Journal of Public Health, 94, 416–422.

    PubMed  Article  Google Scholar 

  • Ellickson, P. L., & Bell, R. M. (1990). Drug prevention in junior high: A multi-site longitudinal test. Science, 247, 1299–1305.

    PubMed  Article  CAS  Google Scholar 

  • Ennett, S. T., Ringwalt, C. L., Thorne, J., Rohrbach, L. A., Vincus, A., Simons-Rudolph, A., et al. (2003). A comparison of current practice in school-based substance abuse prevention programs with meta-analysis findings. Prevention Science, 4, 1–14.

    PubMed  Article  Google Scholar 

  • Feinberg, M. E., Ridenour, T. A., & Greenberg, M. T. (2008). The longitudinal effect of technical assistance dosage on the functioning of Communities That Care prevention boards in Pennsylvania. The Journal of Primary Prevention, 29(2), 145–165.

    PubMed  Article  Google Scholar 

  • Fetterman, D. M., & Wandersman, A. (2005). Empowerment evaluation principles in practice. New York: The Guilford Press.

    Google Scholar 

  • Fishbein, M., & Ajzen, I. (1974). Attitudes toward objects as predictive of single and multiple behavioral criteria. Psychological Review, 81, 59–74.

    Article  Google Scholar 

  • Fishbein, M., & Ajzen, I. (1975). Belief, attitude, intention, and behavior: An introduction to theory and research. Reading, MA: Addison-Wesley.

    Google Scholar 

  • Fisher, D., Imm, P. S., Chinman, M., & Wandersman, A. (2006). Getting To Outcomes with developmental assets: Ten steps to measuring success in youth programs and communities. Minneapolis, MN: Search Institute.

    Google Scholar 

  • Fixsen, D. L., Naoom, S. F., Blase, K. A., Friedman, R. M., & Wallace, F. (2005). Implementation research: A synthesis of the literature. Tampa, FL: University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network (FMHI Publication #231).

  • Flaspohler, P., Duffy, J., Wandersman, A., Stillman, L., & Maras, M. A. (2008). Unpacking prevention capacity: An intersection of research-to-practice models and community-centered models. American Journal of Community Psychology, 41, 182–196.

    PubMed  Article  Google Scholar 

  • Green, L. (2001). From research to “Best Practices” in other settings and populations. American Journal of Health Behavior, 25, 165–178.

    PubMed  Article  CAS  Google Scholar 

  • Hall, G. E., & Hord, S. M. (2001). Implementing change. Boston, MA: Allyn and Bacon.

    Google Scholar 

  • Hall, G. E., & Loucks, S. F. (1977). A developmental model for determining whether the treatment is actually implemented. American Educational Research Journal, 14, 263–276.

    Article  Google Scholar 

  • Hall, G. E., Loucks, S. F., Rutherford, W. L., & Newlove, B. W. (1975). Levels of use of the innovation: A framework for analyzing innovation adoption. Journal of Teacher Education, 26, 52–56.

    Article  Google Scholar 

  • Hallfors, D., & Godette, D. (2002). Will the ‘principles of effectiveness’ improve prevention practice? Early findings from a diffusion study. Health Education Research, 17, 461–470.

    PubMed  Article  CAS  Google Scholar 

  • Hawkins, J. D., Oesterle, S., Brown, E. C., Arthur, M. W., Abbott, R. D., Fagan, A. A., et al. (2009). Results of a type 2 translational research trial to prevent adolescent drug use and delinquency: A test of Communities that Care. Archives of Pediatric and Adolescent Medicine, 163, 789–798.

    Article  Google Scholar 

  • Hunter, S., Paddock, S., Ebener, P., Burkhart, Q., & Chinman, M. (2009). Promoting evidence based practices: The adoption of a prevention support system in community settings. The Journal of Community Psychology, 37, 579–593.

    Article  Google Scholar 

  • Jemmott, J. B., Jemmott, L. S., Fong, G. T., & McCaffree, K. (1999). Reducing HIV risk-associated sexual behavior among African American adolescents: Testing the generality of intervention effects. American Journal of Community Psychology, 27, 161–187.

    PubMed  Article  Google Scholar 

  • Jemmott, J. B., Jemmott, L. S., Spears, H., Hewitt, N., & Cruz-Collins, M. (1992a). Self-efficacy, hedonistic expectancies, and condom-use intentions among inner-city black adolescent women: A social cognitive approach to AIDS risk behavior. Journal of Adolescent Health, 13, 512–519.

    PubMed  Article  Google Scholar 

  • Jemmott, J. B., Jemmott, L. S., Spears, H., Hewitt, N., & Cruz-Collins, M. (1992b). Increasing condom-use intentions among sexually active inner-city black adolescent women: Effects of an AIDS prevention program. Nursing Research, 41, 273–279.

    PubMed  CAS  Google Scholar 

  • Keener, D. C. (2007). Towards a science of capacity building: An examination of technical assistance following a training of prevention professionals. (Doctoral dissertation). Retrieved from ProQuest Dissertations and Theses. (Accession Order No. AAT 3272459).

  • Kitson, A., Harvey, G., & McCormack, B. (1998). Enabling the implementation of evidence based practice: A conceptual framework. Quality in Health Care, 7, 149–158.

    PubMed  Article  CAS  Google Scholar 

  • Leffert, N., Benson, P. L., Scales, P. C., Sharma, A. R., Drake, D. R., & Blyth, D. A. (1998). Developmental assets: Measurement and prediction of risk behaviors among adolescents. Applied Developmental Science, 2, 209–230.

    Article  Google Scholar 

  • Lehman, W. E. K., Greener, J. M., & Simpson, D. D. (2002). Assessing organizational readiness for change. Journal of Substance Abuse Treatment, 22, 197–209.

    PubMed  Article  Google Scholar 

  • Livet, M., & Wandersman, A. (2005). Organizational functioning: Facilitating effective interventions and increasing the odds of programming success. In D. M. Fetterman & A. Wandersman (Eds.), Empowerment evaluation in practice (pp. 123–154). New York, NY: Guilford.

    Google Scholar 

  • Maine Office of Substance Abuse. (2006). Maine youth drug and alcohol survey local summary reports. Retrieved March 25, 2009 from http://www.maine.gov/maineosa/survey/home.php.

  • Mihalic, S. F., & Irwin, K. (2003). Blueprints for violence prevention: From research to real-world settings—Factors influencing the successful replication of model programs. Youth Violence and Juvenile Justice, 1, 1–23.

    Article  Google Scholar 

  • Miller, T., & Hendrie, D. (2009). Substance abuse prevention dollars and cents: A cost-benefit analysis (DHHS Pub. No. (SMA) 07-4298). Rockville, MD: Center for Substance Abuse Prevention, Substance Abuse and Mental Health Services Administration.

  • Mitchell, R. E., Stone-Wiggins, B., Stevenson, J. F., & Florin, P. (2004). Cultivating capacity: Outcomes of a statewide support system for prevention coalitions. Journal of Prevention & Intervention in the Community, 27, 67–87.

    Article  Google Scholar 

  • Murray, D. M. (1998). Design and analysis of group-randomized trials. Oxford, UK: Oxford University Press.

    Google Scholar 

  • O’Donnell, L. O., Scattergood, P., Adler, M., San Doval, A., Barker, M., et al. (2000). The role of technical assistance in the replication of effective HIV interventions. AIDS Education and Prevention, 12(Supl A), 99–111.

    Google Scholar 

  • Ottoson, J. M. (1997). After the applause: Exploring multiple influences on application following an adult education program. Adult Education Quarterly, 47, 92–107.

    Article  Google Scholar 

  • Philliber, S., & Nolte, K. (2008). Implementation science: Promoting science-based approaches to prevent teen pregnancy. Prevention Science, 9, 166–177.

    PubMed  Article  Google Scholar 

  • Reuter, P., & Timpane, J. M. (2001). Assessing options for the safe and drug free schools and communities act (No. MR-1328). Santa Monica, CA: RAND Corporation.

  • Ringwalt, C. L., Ennett, S., Vincus, A., Thorne, J., Rohrbach, L. A., & Simons-Rudolph, A. (2002). The prevalence of effective substance use prevention curricula in US middle schools. Prevention Science, 3, 257–265.

    PubMed  Article  Google Scholar 

  • Robertson, E. B., David, S. L., & Rao, S. A. (2003). Preventing drug use among children and adolescents: A research-based guide for parents, educators, and community leaders (2nd ed.) (NIH Publication No. 04-4212[A]). Bethesda, MD: National Institute on Drug Abuse.

  • Rycroft-Malone, J., Harvey, G., Seers, K., Kitson, A., McCormack, B., & Titchen, A. (2002). An exploration of the factors that influence the implementation of evidence into practice. Journal of Clinical Nursing, 13, 913–924.

    Article  Google Scholar 

  • Scales, P. C. (1999). Reducing risks and building developmental assets: Essential actions for promoting adolescent health. Journal of School Health, 69, 113–119.

    PubMed  Article  CAS  Google Scholar 

  • Scales, P. C., Benson, P. L., Leffert, N., & Blyth, D. A. (2000). Contribution of Developmental Assets to the prediction of thriving among adolescents. Applied Developmental Science, 4, 27–46.

    Article  Google Scholar 

  • Silvia, E. S., & Thorne, J. (1997). School-based drug prevention programs: A longitudinal study in selected school districts. Research Triangle Park, NC: Research Triangle Institute.

    Google Scholar 

  • Spoth, R., Redmond, C., Shin, C., Greenberg, M., Clair, S., & Feinberg, M. (2007). Substance-use outcomes at 18 months past baseline: The PROSPER Community-University partnership trial. American Journal of Preventive Medicine, 32, 395–402.

    PubMed  Article  Google Scholar 

  • Stetler, C. B., Legro, M. W., Rycroft-Malone, J., Bowman, C., Curran, G., Guihan, M., et al. (2006). Role of “external facilitation” in implementation of research findings: A qualitative evaluation of facilitation experiences in the Veterans Health Administration. Implementation Science, 1, 1–15.

    Article  Google Scholar 

  • Stevenson, J. F., Florin, P., Mills, D. S., & Andrade, M. (2002). Building evaluation capacity in human service organizations: A case study. Evaluation and Program Planning, 25, 233–243.

    Article  Google Scholar 

  • U.S. Census Bureau. (2000). State and county Quickfacts: Maine. Retrieved March 25, 2009, from http://quickfacts.census.gov.

  • Wandersman, A. (2009). Four keys to success (theory, implementation, evaluation, and resource/system support): High hopes and challenges in participation. American Journal of Community Psychology, 43, 3–21.

    PubMed  Article  Google Scholar 

  • Wandersman, A., & Florin, P. (2003). Community interventions and effective prevention: Bringing researchers/evaluators, funders and practitioners together for accountability. American Psychologist, 58, 441–448.

    PubMed  Article  Google Scholar 

  • Wandersman, A., Chien, V. H., & Katz, J. (2012). Toward an evidence-based system for innovation support for implementing innovations with quality: Tools, training, technical assistance, and quality assurance/quality improvement. American Journal of Community Psychology, 50, 445–449.

    PubMed  Article  Google Scholar 

  • Weiss, E., Miller-Anderson, R., & Lasker, R. (2002). Making the most of collaboration: Exploring the relationship between partnership synergy and partnership functioning. Health Education and Behavior, 29, 683–698.

    PubMed  Article  Google Scholar 

  • Wenger, E., McDermott, R., & Snyder, W. M. (2002). Cultivating communities of practice. Boston, MA: Harvard Business School Press.

    Google Scholar 

  • Yin, R. K., Kaftarian, S. J., Yu, P., & Jansen, M. A. (1997). Outcomes from CSAP’s Community Partnership Program: Findings from the National Cross Site Evaluation. Evaluation and Program Planning, 20, 345–355.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. RAND Corporation, 4570 Fifth Ave, Pittsburgh, PA, 15213, USA

    Matthew Chinman, Joie Acosta, Patricia Ebener, Q. Burkhart, Susan M. Paddock, Sarah Hunter & Andrea Phillips

  2. University of South Carolina, Barnwell College, Columbia, SC, 29208, USA

    Patrick S. Malone

  3. Vision Training Associates, PO Box 338, Gorham, ME, 04038-0338, USA

    Michael Clifford, Tim Duffey & Margaret Jones

  4. University of Southern Maine, 45 Commerce Drive, Suite 11, Portland, ME, 04330, USA

    Michel Lahti

  5. Spurwink, 899 Riverside St., Portland, ME, 04103, USA

    Maryann Corsello & Susan Savell

  6. Search Institute, The Banks Building, 615 First Avenue NE, Suite 125, Minneapolis, MN, 55413, USA

    Peter C. Scales & Nancy Tellett-Royce

Authors
  1. Matthew Chinman
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Joie Acosta
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Patricia Ebener
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Q. Burkhart
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Patrick S. Malone
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Susan M. Paddock
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Michael Clifford
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Maryann Corsello
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Tim Duffey
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Sarah Hunter
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Margaret Jones
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Michel Lahti
    View author publications

    You can also search for this author in PubMed Google Scholar

  13. Andrea Phillips
    View author publications

    You can also search for this author in PubMed Google Scholar

  14. Susan Savell
    View author publications

    You can also search for this author in PubMed Google Scholar

  15. Peter C. Scales
    View author publications

    You can also search for this author in PubMed Google Scholar

  16. Nancy Tellett-Royce
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Matthew Chinman.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Chinman, M., Acosta, J., Ebener, P. et al. Intervening with Practitioners to Improve the Quality of Prevention: One-Year Findings from a Randomized Trial of Assets-Getting To Outcomes. J Primary Prevent 34, 173–191 (2013). https://doi.org/10.1007/s10935-013-0302-7

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10935-013-0302-7

Keywords

  • Technical assistance
  • Implementation support
  • Capacity building