Journal of Youth and Adolescence

, Volume 44, Issue 12, pp 2337–2358 | Cite as

A Longitudinal Evaluation of the Positive Action Program in a Low-Income, Racially Diverse, Rural County: Effects on Self-Esteem, School Hassles, Aggression, and Internalizing Symptoms

  • Shenyang Guo
  • Qi Wu
  • Paul R. Smokowski
  • Martica Bacallao
  • Caroline B. R. Evans
  • Katie L. Cotter
Empirical Research


Positive Action is a school-based program that aims to decrease problem behaviors (e.g., violence, substance use) and increase positive behaviors (e.g., school engagement, academic achievement). Although a number of studies have shown that Positive Action successfully achieves these goals, few studies have evaluated the program’s effectiveness in rural schools. Given that rural youth are at an increased risk for risky behaviors (e.g., violence, substance use), this is a critical gap in the existing Positive Action research base. The current study assesses the impact of Positive Action on change rates of self-esteem, school hassles, aggression, and internalizing symptoms in a group (N = 1246, 52 % female) of ethnically/racially diverse (27 % White, 23 % African American, 12 % mixed race/other, 8 % Latino, 30 % as American Indian) middle school youth (age range 9–20) located in two violent, low-income rural counties in North Carolina. One county engaged in Positive Action over the 3-year study window while the other county did not. Following multiple imputation and propensity score analysis, 4 two-level hierarchical linear models were run using each of the outcome measures as dependent variables. The results indicate that the program generates statistically significant beneficial effects for youth from the intervention county on self-esteem scores and school hassles scores. Although the program generates beneficial effects for intervention youth on the change in aggression scores, the finding is not statistically significant. The finding on the change in internalizing scores shows a non-significant detrimental effect: the youth from the comparison county have lower internalizing scores than those from the intervention county. Implications are discussed.


School-based interventions Positive Action Self-esteem Aggression Rural Propensity score analysis 



Funding for this research was provided through a cooperative agreement with the U.S. Centers for Disease Control and Prevention’s National Center for Injury Prevention and Control (5 U01 CE001948-03).

Authors’ Contributions

S.G. conceived of the study by selecting the independent variables and statistical methodologies to use and oversaw the statistical analysis. Q.W. conducted the statistical analysis and wrote the methods and analysis sections with support and guidance from S.G. P.R.S. obtained the funding to make the current study possible, implemented the data collection, and assisted in drafting and editing the manuscript. M.B. oversaw the implementation of the program and worked in the field ensuring the fidelity of Positive Action; she also wrote the implementation section. C.B.R.E. and K.L.C. wrote the remaining sections of the manuscript. All authors read and approved the final manuscript.

Conflict of interest

The authors report no conflicts of interest.


  1. Achenbach, T. M., & Rescorla, L. A. (2001). Manual for ASEBA school-age forms and profiles. Burlington, VT: University of Vermont, Research Center for Children, Youth & Families.Google Scholar
  2. Allison, P. D. (2002). Missing data. Thousand Oaks, CA: Sage.Google Scholar
  3. Atav, S., & Spencer, G. A. (2002). Health risk behaviors among adolescents attending rural, suburban, and urban schools: A comparative study. Family and Community Health, 17(12), 53–64. doi: 10.1097/00003727-200207000-00007.CrossRefGoogle Scholar
  4. Barnett, T., Clements-Nolle, K., Lensch, T., Frankenberger, D., Larson, S., & Yang, W. (2015). Non-medical prescription drug use in a representative sample of high school students: A need for prevention in rural settings. American Public Health Association. Retrevied from
  5. Bavarian, N., Lewis, K. M., DuBois, D. L., Acock, A., Vuchinich, S., Silverthorn, N., & Flay, B. R. (2013). Using social-emotional and character development to improve academic outcomes: A matched-pair cluster-randomized controlled trial in low-income, urban schools. Journal of School Health, 83, 771–779. doi: 10.1111/josh.12093.CrossRefPubMedGoogle Scholar
  6. Beets, M. W., Ray, B. R., Vuchinich, S., Snyder, F. J., Acock, A., Li, K. K., & Durlak, J. (2009). Use of social and character development program to prevent substance use, violent behaviors, and sexual activity among elementary-school students in Hawaii. American Journal of Public Health, 99, 1438–1445. doi: 10.2105/AJPH.2008.142919.PubMedCentralCrossRefPubMedGoogle Scholar
  7. Blueprints for Healthy Youth Development. (2012). Program criteria. Retrieved from
  8. Bowen, G. L., & Richman, J. M. (2008). The School Success Profile. Chapel Hill, NC: University of North Carolina.Google Scholar
  9. Bowen, G. L., Rose, R. A., & Bowen, N. K. (2005). The reliability and validity of the school success profile. Philadelphia, PA: Xlibris.Google Scholar
  10. Bureau of Labor Statistics. (2012). Local area unemployment statistics map. Retrieved from
  11. Centers for Disease Control and Prevention. (2013). 1995–2013 Middle school youth risk behavior survey data.
  12. Cohen, J., McCabe, L., Michelli, N. M., & Pickeral, T. (2009). School climate: Research, policy, practice, and teacher education. Teachers College Record, 111, 180–213. Retrieved from
  13. Deal, T. E., & Peterson, K. D. (2009). Shaping School Cutlture: Pitfalls, Paradoxes, and Promises. San Francisco, CA: Jossey-Bass.Google Scholar
  14. Derdikman-Eiron, R., Indredavik, M. S., Bratberg, G. H., Taraldsen, G., Bakken, I. J., & Colton, M. (2011). Gender differences in subjective well-being, self-esteem, and psychosocial functioning in adolescents with symptoms of anxiety and depression: Findings from the Nord-Trøndelag health study. Scandinavian Journal of Psychology, 52, 261–267. doi: 10.1111/j.1467-9450.2010.00859.x.CrossRefPubMedGoogle Scholar
  15. Donnellan, M. B., Trzesniewski, K. H., Robins, R. W., Moffitt, T. E., & Caspi, A. (2005). Low self-esteem is related to aggression, antisocial behavior, and delinquency. Psychological Science, 16, 328–335. doi: 10.1111/j.0956-7976.2005.01535.x.CrossRefPubMedGoogle Scholar
  16. DuBois, D. L., Flay, B. R., & Fagen, M. C. (2009). Self-esteem enhancement theory: An emerging framework for promoting health across the life-span. In R. J. DiClemente, M. C. Kegler, & R. A. Crosby (Eds.), Emerging theories in health promotion practice and research (2nd ed., pp. 97–130). San Francisco, CA: Jossey-Bass.Google Scholar
  17. Dulmus, C. N., Theriot, M. T., Sowers, K. M., & Blackburn, J. A. (2004). Student reports of peer bullying victimization in a rural school. Stress, Trauma, and Crisis, 7, 1–16. doi: 10.1080/15434610490281093.CrossRefGoogle Scholar
  18. Erikson, E. H. (1950). Childhood and society. New York, NY: Norton.Google Scholar
  19. Finley, C. & Stewart, A. (2013). Working with rural teens: Adolescent reproductive health in rural America. The National Campaign to Prevent Teen and Unplanned Pregnancy. Third Annual Teen Pregnancy Prevention Grantee Conference. Retrieved from
  20. Flay, B. R., & Allred, C. G. (2003). Long-term effects of the positive-action program. American Journal of Health Behavior, 27, S6–S21. doi: 10.5993/AJHB.27.1.s1.2.CrossRefPubMedGoogle Scholar
  21. Flay, B. R., Allred, C. G., & Ordway, N. (2001). Effects of the positive action program on achievement and discipline: Two match-control comparisons. Prevention Science, 2(2), 71–89. doi: 10.1023/A:1011591613728.CrossRefPubMedGoogle Scholar
  22. Geertz, C. (1973). The Interpretation of Culture. New York, NY: Basic Books.Google Scholar
  23. Gil, A. G., Vega, W. A., & Dimas, J. M. (1994). Acculturative stress and personal adjustment among Hispanic adolescent joys. Journal of Community Psychology, 22, 43–54. doi: 10.1002/1520-6629(199401)22:1<43::AID-JCOP2290220106>3.0.CO;2-T.CrossRefGoogle Scholar
  24. Greene, W. H. (2003). Econometric analysis (5th ed.). Upper Saddle River, NJ: Prentice Hall.Google Scholar
  25. Guo, S., & Fraser, W. M. (2015). Propensity score analysis: Statistical methods and applications (2nd ed.). Thousand Oaks, CA: Sage.Google Scholar
  26. Horvitz, D., & Thompson, D. (1952). A generalization of sampling without replacement from a finite population. Journal of the American Statistical Association, 47, 663–685. doi: 10.1080/01621459.1952.10483446.CrossRefGoogle Scholar
  27. Huitsing, G., Veenstra, R., Sainio, M., & Salmivalli, C. (2012). “It must be me” or “It could be them?” The impact of the social network position of bullies and victims on victims’ adjustment. Social Networks, 34, 279–386. doi: 10.1016/j.socnet.2010.07.002.Google Scholar
  28. Imbens, G. W. (2004). Nonparametric estimation of average treatment effects under exogeneity: A review. Review of Economics and Statistics, 86, 4–29. doi: 10.1162/003465304323023651.CrossRefGoogle Scholar
  29. Imbens, G. W., & Wooldridge, J. M. (2009). Recent developments in the econometrics of program evaluation. Journal of Economic Literature, 47, 5–86. doi: 10.1257/jel.47.1.5.CrossRefGoogle Scholar
  30. Lee, C. H., & Song, J. (2012). Functions of parental involvement and effects of school climate on bullying behaviors among South Korean middle school students. Journal of Interpersonal Violence, 27, 2437–2464. doi: 10.1177/0886260511433508.CrossRefPubMedGoogle Scholar
  31. Lewis, K. M., DuBois, D. L., Bavarian, N., Acock, A., Silverthorn, N., Day, S., & Flay, B. R. (2013a). Effects of Positive Action on the emotional health of urban youth: A cluster-randomized trial. Journal of Adolescent Health, 53, 706–711. doi: 10.1016/j.jadohealth.2013.06.012.CrossRefPubMedGoogle Scholar
  32. Lewis, K. M., Schure, M. B., Bavarian, N., DuBois, D. L., Day, J., Ji, P., & Flay, B. R. (2013b). Problem behavior and urban, low-income youth: A randomized controlled trial of Positive Action in Chicago. American Journal of Preventative Medicine, 44, 622–630. doi: 10.1016/j.amepre.2013.01.030.CrossRefGoogle Scholar
  33. Li, K. K., Washburn, I., DuBois, D. L., Vuchinich, S., Ji, P., Brechling, V., & Flay, B. R. (2011). Effects of the Positive Action programme on problem behaviors in elementary school students: A match-pair randomised control trial in Chicago. Psychology and Health, 26(2), 187–204. doi: 10.1080/08870446.2011.531574.CrossRefPubMedGoogle Scholar
  34. Nansel, T. R., Overpeck, M., Pilla, R. S., Ruan, W. J., Simons-Morton, B., & Scheidt, P. (2001). Bullying behaviors among US youth: Prevalence and association with psychosocial adjustment. Journal of the American Medical Association, 285, 2094–2110. doi: 10.1001/jama.285.16.2094.PubMedCentralCrossRefPubMedGoogle Scholar
  35. National Institute of Justice. (2015), n.d.). Program profile: Positive Action. Retrieved from
  36. National Registry of Evidence-Based Programs and Practices. (2014). Minimum requirements. Retrieved from
  37. NC State Center for Health Statistics. (2015). North Carolina statewide and county trends in key health indicators. Retrieved from
  38. Positive Action (2015) n.d.) Positive Action overview. Retrieved from
  39. Price, M., Chin, M. A., Higa-McMillan, C., Kim, S., & Frueh, B. C. (2013). Prevalence and internalizing problems of ethnoracially diverse victims of traditional and cyberbullying. School Mental Health, 5, 183–191. doi: 10.1007/s12310-013-9104-6.CrossRefGoogle Scholar
  40. Prinz, R. J., Foster, S., Kent, R. N., & O’Leary, K. D. (1979). Multivariate assessment of conflict in distressed and nondistressed mother-adolescent dyads. Journal of Applied Behavior Analysis, 12, 691–700. doi: 10.1901/jaba.1979.12-691.PubMedCentralCrossRefPubMedGoogle Scholar
  41. Provasnik, S., Ramani, A. K., Coleman, M. M., Gilbertson, L., Herring, W., & Xie, Q. (2007). Status of education in rural America (NCES 2007-040). Washington, DC: National Center for Education Statistics.Google Scholar
  42. Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models: Applications and data analysis methods (2nd ed.). Newbury Park, CA: Sage.Google Scholar
  43. Robers, S., Kemp, J., & Truman, J. (2013). Indicators of school crime and safety: 2012 (NCES 2013-036/NCJ241446). Washington, DC: National Center for Education Statistics. Retrieved from
  44. Rosenbaum, P. R., & Rubin, D. B. (1983). The central role of the propensity score in observational studies for causal effects. Biometrika, 70, 41–55. doi: 10.1093/biomet/70.1.41.CrossRefGoogle Scholar
  45. Rosenbaum, P. R., & Rubin, D. B. (1985). Constructing a control group using multivariate matched sampling methods that incorporate the propensity score. American Statistician, 39, 33–38. doi: 10.2307/2683903.Google Scholar
  46. Rosenberg, M. (1965). Society and the adolescent self-image. Princeton, NJ: Princeton University Press.Google Scholar
  47. Rubin, D. B. (1987). Multiple imputation for nonresponse in surveys. New York, NY: J. Wiley & Sons.CrossRefGoogle Scholar
  48. Smokowski, P. R., Guo, S., Cotter, K. L., Evans, C. B. R., & Rose, R. A. (2015). in press). Multi-level risk factors and developmental assets associated with aggressive behavior in disadvantaged adolescents: Modeling longitudinal trajectories from the rural adaptation project. Aggressive Behavior. Google Scholar
  49. Smokowski, P. R., Guo, S., Rose, R., Evans, C. B., Cotter, K. L., & Bacallao, M. (2014). Multilevel risk factors and developmental assets for internalizing symptoms and self-esteem in disadvantaged adolescents: Modeling longitudinal trajectories from the Rural Adaptation Project. Development and Psychopathology, 26, 1495–1513. doi: 10.1017/S0954579414001163.CrossRefPubMedGoogle Scholar
  50. Snyder, F. J., Acock, A. C., Vuchinich, S., Beets, M. W., Washburn, I., & Flay, B. R. (2013). Preventing negative behaviors among elementary-school students through enhancing students’ social-emotional and character development. American Journal of Health Promotion, 28(1), 50–58. doi: 10.4278/ajhp.120419-QUAN-207.2.PubMedCentralCrossRefPubMedGoogle Scholar
  51. Snyder, F., Flay, B., Vuchinich, S., Acock, A., Washburn, I., Beets, M., & Li, K. K. (2010). Impact of a social-emotional and character development program on school-level indicators of academic achievement, absenteeism, and disciplinary outcome: A matched-pair, cluster-randomized, controlled trial. Journal of Research on Educational Effectiveness, 3, 26–55. doi: 10.1080/19345740903353436.PubMedCentralCrossRefPubMedGoogle Scholar
  52. Substance Abuse and Mental Health Services Administration. (2014). Results from the 2013 national survey on drug use and health: summary of national findings, NSDUH Series H-48, HHS Publication No. (SMA) 14-4863. Rockville, MD: Substance Abuse and Mental Health Services Administration, 2014.Google Scholar
  53. U.S Census Bureau. (2015). State and county quick facts. Retrieved from
  54. U.S. Department of Education. (2007). What works clearing house. Retrieved from
  55. U.S. Department of Education. (2009). Save, disciplined, and drug-free schools. Retrieved from
  56. U.S. Department of Justice, Office for Victims of Crime. (2001). Rural victim assistance: A victim/witness guide for rural prosecutors (NCJ No. 211106). Retrieved from
  57. Washburn, I. J., Acock, A., Vuchinich, S., Snyder, F., Li, K. K., Ji, P., & Flay, B. R. (2011). Effects of a social-emotional and character development program on the trajectory of behaviors associated with social-emotional and character development: Findings from three randomized trials. Prevention Science, 12, 314–323. doi: 10.1007/s11121-011-0230-9.CrossRefPubMedGoogle Scholar
  58. Witherspoon, D., & Ennett, S. (2011). Stability and change in rural youths’ educational outcomes through the middle and high school years. Journal of Youth and Adolescence, 40, 1077–1090. doi: 10.1007/s10964-010-9614-6.PubMedCentralCrossRefPubMedGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2015

Authors and Affiliations

  • Shenyang Guo
    • 1
  • Qi Wu
    • 2
  • Paul R. Smokowski
    • 3
  • Martica Bacallao
    • 3
  • Caroline B. R. Evans
    • 2
  • Katie L. Cotter
    • 4
  1. 1.Brown School of Social WorkWashington UniversitySt. LouisUSA
  2. 2.School of Social WorkUniversity of North CarolinaChapel HillUSA
  3. 3.School of Social WelfareUniversity of KansasLawrenceUSA
  4. 4.School of Social WorkArizona State UniversityPhoenixUSA

Personalised recommendations