Advertisement

Initial Development and Validation of the Student Wellbeing Teacher-Report Scale

  • Anthony J. Roberson
  • Tyler L. Renshaw
Original Paper

Abstract

Given that youth mental health is associated with success in school and life more broadly, it is important that school-based psychological service providers embrace best-practice prevention and intervention strategies that target mental health when working with student populations. One line of study in this area has begun exploring the incorporation of a dual-factor model of mental health within universal screening systems in schools. The dual-factor model is differentiated from the traditional unidimensional mental health model, which focuses on the presence or absence of psychopathology, by conceptualizing mental health alternatively as consisting of both psychopathology and wellbeing dimensions. The present study involved the preliminary development and validation of the Student Wellbeing Teacher-Report Scale (SWTRS)—a brief behavior rating scale intended as a screening tool for measuring the wellbeing dimension of youths’ mental health at school. Specifically, the study involved drafting pilot items for the SWTRS and explored their latent factor structure, concurrent validity with school-related outcomes (i.e., attendance, academic achievement, and time on-task), as well as concurrent and incremental validity in comparison with two psychopathology screeners. Results suggested that the SWTRS items may better represent three context-specific indicators of youth wellbeing behavior—(a) academic, (b) social, and (c) emotional wellbeing—rather than the hypothesized “feeling good” and “functioning well” dimensions. The SWTRS scores also demonstrated incremental validity and were uniformly stronger concurrent predictors of all school-related outcomes compared to the psychopathology scales. Implications for theory and future research are discussed.

Keywords

Wellbeing Student mental health Universal screening Dual-factor mental health Behavior rating scales 

Notes

Compliance with Ethical Standards

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical Approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Informed Consent

Informed consent was obtained from all individual participants included in the study.

References

  1. Akaike, H. (1987). Factor analysis and AIC. Psychometrika, 52, 317–332.  https://doi.org/10.1007/BF02294359.CrossRefGoogle Scholar
  2. Appleton, J. J., Christenson, S. L., & Furlong, M. J. (2008). Student engagement with school: Critical conceptual and methodological issues of the construct. Psychology in the Schools, 45, 369–386.  https://doi.org/10.1002/pits.CrossRefGoogle Scholar
  3. Armistead, R. J., & Smallwood, D. L. (2014). The National Association of School Psychologists model for comprehensive and integrated school psychological services. In P. Harrison & A. Thomas (Eds.), Best practices in school psychology: Data-based and collaborative decision making (pp. 9–23). Bethesda, MD: The National Association of School Psychologists.Google Scholar
  4. Bradshaw, C. P., Buckley, J. A., & Ialongo, N. S. (2008). School-based service utilization among urban children with early onset educational and mental health problems: The squeaky wheel phenomenon. School Psychology Quarterly, 23, 169–186.  https://doi.org/10.1037/1045-3830.23.2.169.CrossRefGoogle Scholar
  5. Burns, B. J., Costello, E. J., Angold, A., Tweed, D., Stangl, D., Farmer, E. M., et al. (1995). Children’s mental health service use across service sectors. Health Affairs, 14, 147–159.  https://doi.org/10.1377/hlthaff.14.3.147.CrossRefGoogle Scholar
  6. Caldarella, P., & Merrell, K. W. (1997). Common dimensions of social skills of children and adolescents: A taxonomy of positive behaviors. School Psychology Review, 26, 264–278.Google Scholar
  7. Clark, L. A., & Watson, D. (1995). Constructing validity: Basic issues in objective scale development. Psychological Assessment, 7, 309–319.  https://doi.org/10.1037/1040-3590.7.3.309.CrossRefGoogle Scholar
  8. Cook, C. R. (2013). Manual: Student internalizing behavior screener and student externalizing behavior screener. Unpublished manual.Google Scholar
  9. Cook, C. R., Rasetshwane, K. B., Truelson, E., Grant, S., Dart, E. H., Collins, T. A., et al. (2011). Development and validation of the Student Internalizing Behavior Screener: Examination of reliability, validity, and classification accuracy. Assessment for Effective Intervention, 36, 71–79.  https://doi.org/10.1177/1534508410390486.CrossRefGoogle Scholar
  10. Costello, A. B., & Osborne, J. W. (2005). Best practices in exploratory factor analysis: Four recommendations for getting the most from your analysis. Practical Assessment, Research & Evaluation, 10, 1–9.Google Scholar
  11. Cowen, E. L., & Kilmer, R. P. (2002). “Positive Psychology”: Some plusses and some open issues. Journal of Community Psychology, 30, 449–460.  https://doi.org/10.1002/jcop.10014.CrossRefGoogle Scholar
  12. D’Agostino, R. B., Belanger, A., & D’Agostino, R. B., Jr. (1990). A suggestion for using powerful and informative tests of normality. The American Statistician, 44, 316–321.Google Scholar
  13. DeVellis, R. F. (2012). Scale development: Theory and applications (3rd ed., Vol. 26). Thousand Oaks, CA: Sage.Google Scholar
  14. DiPerna, J. C. (2006). Academic enablers and student achievement: Implications for assessment and intervention services in the schools. Psychology in the Schools, 43, 7–17.  https://doi.org/10.1002/pits.20125.CrossRefGoogle Scholar
  15. Dowdy, E., Furlong, M., Raines, T. C., Bovery, B., Kauffman, B., Kamphaus, R. W., et al. (2014). Enhancing school-based mental health services with a preventive and promotive approach to universal screening for complete mental health. Journal of Educational and Psychological Consultation, 25, 179–197.  https://doi.org/10.1080/10474412.2014.929951.Google Scholar
  16. Drummond, T. (1993). The Student Risk Screening Scale (SRSS). Grants Pass, OR: Josephine County Mental Health Program.Google Scholar
  17. Dunn, T. J., Baguley, T., & Brunsden, V. (2014). From alpha to omega: A practical solution to the pervasive problem of internal consistency estimation. British Journal of Psychology, 105, 399–412.  https://doi.org/10.1111/bjop.12046.CrossRefGoogle Scholar
  18. Earhart, J., Jr., Jimerson, S. R., Eklund, K., Hart, S. R., Jones, C. N., Dowdy, E., et al. (2009). Examining relationships between measures of positive behaviors and negative functioning for elementary school children. The California School Psychologist, 14, 97–104.  https://doi.org/10.1007/BF03340955.CrossRefGoogle Scholar
  19. Eklund, K., & Dowdy, E. (2013). Screening for behavioral and emotional risk versus traditional school identification methods. School Mental Health, 6, 40–49.  https://doi.org/10.1007/s12310-013-9109-1.CrossRefGoogle Scholar
  20. Fredricks, J., Blumenfeld, P. C., & Paris, A. H. (2004). School engagement: Potential of the concept, state of the evidence. Review of Educational Research, 74, 59–109.  https://doi.org/10.3102/00346543074001059.CrossRefGoogle Scholar
  21. Fredricks, J., McColskey, W., Meli, J., Mordica, J., Montrosse, B., & Mooney, K. (2011). Measuring student engagement in upper elementary through high school: A description of 21 instruments. (Issues & Answers Report, REL 2011–No. 098). Washington, DC: U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, Regional Educational Laboratory Southeast. Retrieved from http://ies.ed.gov/ncee/edlabs.
  22. Furlong, M. J., Gilman, R., & Huebner, E. S. (Eds.). (2014). Handbook of positive psychology in the schools (2nd ed.). New York, NY: Routledge.Google Scholar
  23. Furlong, M. J., You, S., Renshaw, T. L., O’Malley, M. D., & Rebelez, J. (2013). Preliminary development of the Positive Experiences at School Scale for elementary school children. Child Indicators Research, 6, 753–775.  https://doi.org/10.1007/s12187-013-9193-7.CrossRefGoogle Scholar
  24. Garrido, L. E., Abad, F. J., & Ponsoda, V. (2013). A new look at Horn’s parallel analysis with ordinal variables. Psychological Methods, 18, 454–474.  https://doi.org/10.1037/a0030005.CrossRefGoogle Scholar
  25. Glover, T. A., & Albers, C. A. (2007). Considerations for evaluating universal screening assessments. Journal of School Psychology, 45, 117–135.  https://doi.org/10.1016/j.jsp.2006.05.005.CrossRefGoogle Scholar
  26. Goodman, R. (1997). The Strengths and Difficulties Questionnaire: A research note. Journal of Child Psychology and Psychiatry and Allied Disciplines, 38, 581–586.CrossRefGoogle Scholar
  27. Greco, L. A., Lambert, W., & Baer, R. A. (2008). Psychological inflexibility in childhood and adolescence: Development and evaluation of the Avoidance and Fusion Questionnaire for Youth. Psychological Assessment, 20, 93–102.  https://doi.org/10.1037/1040-3590.20.2.93.CrossRefGoogle Scholar
  28. Greenspoon, P. J., & Saklofske, D. H. (2001). Toward an integration of subjective wellbeing and psychopathology. Social Indicators Research, 54, 81–108.  https://doi.org/10.1023/A:1007219227883.CrossRefGoogle Scholar
  29. Hayes, S. C., Nelson, R. O., & Jarrett, R. B. (1987). The treatment utility of assessment: A functional approach to evaluating assessment quality. American Psychologist, 42, 963–974.CrossRefGoogle Scholar
  30. Holgado-Tello, F. P., Chacon-Moscoso, S., Barbero-Garcia, I., & Vila-Abad, E. (2010). Polychoric versus Pearson correlations in exploratory and confirmatory factor analysis of ordinal variables. Quality & Quantity, 44, 153–166.  https://doi.org/10.1007/s11135-008-9190-y.CrossRefGoogle Scholar
  31. Individuals with Disabilities Education Improvement Act, 20 U.S.C. § 1400 et seq. (2004).Google Scholar
  32. Iverson, R. D., & Maguire, C. (2000). The relationship between job and life satisfaction: Evidence from a remote mining community. Human Relations, 53, 807–839.  https://doi.org/10.1177/0018726700536003.CrossRefGoogle Scholar
  33. Jimerson, S. R., Campos, E., & Greif, J. L. (2003). Toward an understanding of definitions and measures of school engagement and related terms. California School Psychologist, 8, 7–27.  https://doi.org/10.1007/BF03340893.CrossRefGoogle Scholar
  34. Kamphaus, R. W., DiStefano, C., Dowdy, E., Eklund, K., & Dunn, A. R. (2010). Determining the presence of a problem: Comparing two approaches for detecting youth behavioral risk. School Psychology Review, 39, 395–407.Google Scholar
  35. Kelley, K. (2016). MBESS: The MBESS R package (R package version 4.1.0). Retrieved December 27, 2018 from https://CRAN.R-project.org/package=MBESS.
  36. Kelley, K., & Pornprasertmanit, S. (2016). Confidence intervals for population reliability coefficients: Evaluation of methods, recommendations, and software for composite measures. Psychological Methods, 21, 69–92.  https://doi.org/10.1037/a0040086.CrossRefGoogle Scholar
  37. Keyes, C. L. M. (2006). Mental health in adolescence: Is America’s youth flourishing? American Journal of Orthopsychiatry, 76, 395–402.  https://doi.org/10.1037/0002-9432.76.3.395.CrossRefGoogle Scholar
  38. Keyes, C. L. M. (2007). Promoting and protecting mental health as flourishing: A complementary strategy for improving national mental health. American Psychologist, 62, 95–108.  https://doi.org/10.1080/17439760902844228.CrossRefGoogle Scholar
  39. Keyes, C. L. M., & Annas, J. (2009). Feeling good and functioning well: Distinctive concepts in ancient philosophy and contemporary science. The Journal of Positive Psychology, 4, 197–201.  https://doi.org/10.1080/17439760902844228.CrossRefGoogle Scholar
  40. Kilgus, S. P., Chafouleas, S. M., & Riley-Tillman, T. C. (2013). Development and initial validation of the Social and Academic Behavior Risk Screener for elementary grades. School Psychology Quarterly, 28, 210–226.  https://doi.org/10.1037/spq0000024.CrossRefGoogle Scholar
  41. Kilgus, S. P., Eklund, K., von der Embse, N. P., Taylor, C. N., & Sims, W. A. (2016). Psychometric defensibility of the Social, Academic, and Emotional Behavior Risk Screener (SAEBRS) Teacher Rating Scale and multiple gating procedure within elementary and middle school samples. Journal of School Psychology, 58, 21–39.  https://doi.org/10.1016/j.jsp.2016.07.001.CrossRefGoogle Scholar
  42. Kline, R. B. (2016). Principles and practice of structural equation modeling (4th ed.). New York, NY: Guilford.Google Scholar
  43. Kolenikov, S., & Bollen, K. A. (2012). Testing negative error variances: Is a Heywood case a symptom of misspecification? Sociological Methods & Research, 41(1), 124–167.CrossRefGoogle Scholar
  44. Lane, K. L., Little, M. A., Casey, A. M., Lambert, W., Wehby, J., Weisenbach, J. L., et al. (2009). A comparison of systematic screening tools for emotional and behavioral disorders. Journal of Emotional and Behavioral Disorders, 17, 93–105.  https://doi.org/10.1177/1063426608326203.CrossRefGoogle Scholar
  45. Lane, K. L., Oakes, W., & Menzies, H. (2010). Systematic screenings to prevent the development of learning and behavior problems: Considerations for practitioners, researchers, and policy makers. Journal of Disability Policy Studies, 21, 160–172.  https://doi.org/10.1177/1044207310379123.CrossRefGoogle Scholar
  46. Lyons, M. D., Huebner, E. S., & Hills, K. J. (2012). The dual-factor model of mental health: A short-term longitudinal study of school-related outcomes. Social Indicators Research, 114, 549–565.  https://doi.org/10.1007/s11205-012-0161-2.CrossRefGoogle Scholar
  47. Marmorstein, N. R., Iacono, W. G., & Malone, S. M. (2010). Longitudinal associations between depression and substance dependence from adolescence through early adulthood. Drug and Alcohol Dependence, 107, 154–160.  https://doi.org/10.1016/j.drugalcdep.2009.10.002.CrossRefGoogle Scholar
  48. Mathyssek, C. M., Olino, T. M., Velhurst, F. C., & van Oort, F. V. A. (2012). Childhood internalizing and externalizing problems predict the onset of clinical panic attacks over adolescence: The TRAILS study. PLoS ONE, 7, e51564.  https://doi.org/10.1371/journal.pone.0051564.CrossRefGoogle Scholar
  49. Menard, S. (1995). Applied logistic regression analysis. Sage University Paper Series on Quantitative Applications in the Social Sciences. Thousand Oaks, CA: Sage.Google Scholar
  50. Merikangas, K. J., He, J. P., Burstein, M., Swanson, S. A., Avenevoli, S., Cui, L., et al. (2010). Lifetime prevalence of mental disorder in U.S. adolescents: Results from the national comorbidity study—Adolescent supplement (NCS-A). Journal of the American Academy of Child and Adolescent Psychiatry, 49, 980–989.  https://doi.org/10.1016/j.jaac.2010.05.017.CrossRefGoogle Scholar
  51. Merrell, K. W. (2008). Behavioral, social, and emotional assessment of children and adolescents (3rd ed.). New York, NY: Taylor & Francis.Google Scholar
  52. Miller, F. G., Cohen, D., Chafouleas, S. A., Riley-Tillman, T. C., Welsh, M. E., & Fabiano, G. A. (2015). A comparison of measures to screen for social, emotional, and behavioral risk. School Psychology Quarterly, 30, 184–196.  https://doi.org/10.1037/spq0000085.CrossRefGoogle Scholar
  53. Myers, R. (1990). Classical and modern regression with applications (2nd ed.). Boston, MA: Duxbury.Google Scholar
  54. Naglieri, J. A., LeBuffe, P., & Shapiro, V. B. (2011). Universal screening for social–emotional competencies: A study of the reliability and validity of the DESSA-mini. Psychology in the Schools, 48, 660–671.  https://doi.org/10.1002/pits.20586.CrossRefGoogle Scholar
  55. Naglieri, J. A., LeBuffe, P. A., & Shapiro, V. (2010). Devereux Student Strengths Assessment-mini. Lewisville, NC: Kaplan Press.Google Scholar
  56. National Association of School Psychologists. (2010). Model for comprehensive and integrated school psychological services. Retrieved December 27, 2018 from http://www.nasponline.org/standards/2010standards/2_PracticeModel.pdf.
  57. Podsakoff, P. M., MacKenzie, S. B., Lee, J.-Y., & Podsakoff, N. P. (2003). Common method biases in behavioral research: A critical review of the literature and recommended remedies. Journal of Applied Psychology, 88, 879–903.  https://doi.org/10.1037/0021-9010.88.5.879.CrossRefGoogle Scholar
  58. Presaghi, F., & Desimoni, M. (2016). random.polychor.pa: A parallel analysis with polychoric correlation matrices. R package version 1.1.4-2.Google Scholar
  59. Quay, H. C. (1986). Classification. In H. C. Quay & J. S. Werry (Eds.), Psychopathological disorders of childhood (3rd ed., pp. 1–34). New York, NY: Wiley.Google Scholar
  60. R Core Team. (2016). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. Retrieved December 27, 2018 from https://www.R-project.org/.
  61. Rapport, M. D., Denney, C. B., Chung, K.-M., & Hustace, K. (2001). Internalizing behavior problems and scholastic achievement in children: Cognitive and behavioral pathways as mediators of outcome. Journal of Clinical Child & Adolescent Psychology, 30, 536–551.  https://doi.org/10.1207/S15374424JCCP3004_10.CrossRefGoogle Scholar
  62. Renshaw, T. L. (2015). A replication of the technical adequacy of the Student Subjective Wellbeing Questionnaire. Journal of Psychoeducational Assessment.  https://doi.org/10.1177/0734282915580885.Google Scholar
  63. Renshaw, T. L. (2017). Technical adequacy of the Positive Experiences at School Scale with adolescents. Journal of Psychoeducational Assessment, 35, 323–335.  https://doi.org/10.1177/0734282915627920.CrossRefGoogle Scholar
  64. Renshaw, T. L., & Bolognino, S. J. (2017). Psychometrics of the Psychological Wellbeing and Distress Screener: A brief measure of youths’ bidimensional mental health. Assessment for Effective Intervention, 42, 160–167.  https://doi.org/10.1177/1534508416678970.CrossRefGoogle Scholar
  65. Renshaw, T. L., & Cook, C. R. (2016). Initial development and validation of the Youth Internalizing Problems Screener. Journal of Psychoeducational Assessment.  https://doi.org/10.1177/0734282916679757.Google Scholar
  66. Renshaw, T. L., Eklund, K., Bolognino, S. J., & Adodo, I. (2016). Bidimensional emotional health in college students: A comparison of categorical and continuous analytic approaches. Journal of Psychopathology & Behavioral Assessment, 38, 681–694.  https://doi.org/10.1007/s10862-016-9558-6.CrossRefGoogle Scholar
  67. Renshaw, T. L., Furlong, M. J., Dowdy, E., Rebelez, J., Smith, D. C., O’Malley, M. D., et al. (2014a). Covitality: A synergistic conception of youths’ mental health. In M. J. Furlong, R. Gilman, & E. S. Huebner (Eds.), Handbook of positive psychology in the schools (2nd ed., pp. 12–32). New York, NY: Routledge.Google Scholar
  68. Renshaw, T. L., Long, A. C. J., & Cook, C. R. (2014b). Assessing adolescents’ positive psychological functioning at school: Development and validation of the Student Subjective Wellbeing Questionnaire. School Psychology Quarterly, 30, 534–552.  https://doi.org/10.1037/spq0000088.CrossRefGoogle Scholar
  69. Revelle, W. (2017). psych: Procedures for personality and psychological research. Northwestern University, Evanston, Illinois, USA. Retrieved December 27, 2018 from https://CRAN.R-project.org/package=psych. Version = 1.7.5.
  70. Rudolph, K. D., & Klein, D. N. (2009). Exploring depressive personality traits in youth: Origins, correlates, and developmental consequences. Development and Psychopathology, 21, 1155–1180.  https://doi.org/10.1017/S0954579409990095.CrossRefGoogle Scholar
  71. Schwartz, G. (1978). Estimating the dimension of a model. Annals of Statistics, 6, 461–464.CrossRefGoogle Scholar
  72. Seligman, M. E. P. (2002). Positive psychology, positive prevention, and positive therapy. In C. R. Snyder & S. J. Lopez (Eds.), Handbook of positive psychology (pp. 3–12). New York, NY: Oxford University Press.Google Scholar
  73. Seligman, M. E. P., & Csikszentmihalyi, M. (2000). Positive psychology: An introduction. American Psychologist, 55, 5–14.  https://doi.org/10.1037//0003-066X.55.1.5.CrossRefGoogle Scholar
  74. Strein, W., Hoagwood, K., & Cohn, A. (2003). School psychology: A public health perspective I. Prevention, populations, and systems change. Journal of School Psychology, 41, 23–38.  https://doi.org/10.1016/S0022-4405(02)00142-5.CrossRefGoogle Scholar
  75. Suldo, S., Thalji, A., & Ferron, J. (2011). Longitudinal academic outcomes predicted by early adolescents’ subjective wellbeing, psychopathology, and mental health status yielded from a dual factor model. The Journal of Positive Psychology, 6, 17–30.  https://doi.org/10.1080/17439760.2010.536774.CrossRefGoogle Scholar
  76. Suldo, S. M., & Shaffer, E. J. (2008). Looking beyond psychopathology: The dual-factor model of mental health in youth. School Psychology Review, 37, 52–68.Google Scholar
  77. Volpe, R. J., DuPaul, G. J., DiPerna, J. C., Jitendra, A. K., Lutz, J. G., Tresco, K., et al. (2006). Attention deficit hyperactivity disorder and scholastic achievement: A model of mediation via academic enablers. School Psychology Review, 35, 47–61.Google Scholar
  78. von der Embse, N. P., Pendergast, L. L., Kilgus, S. P., & Eklund, K. R. (2015). Evaluating the applied use of a mental health screener: Structural validity of the Social, Academic, and Emotional Behavior Risk Screener. Psychological Assessment, 28, 1265–1275.  https://doi.org/10.1037/pas0000253.CrossRefGoogle Scholar
  79. Walker, H. M., & Severson, H. H. (1990). Systematic Screening for Behavior Disorders (SSBD). Longmont, CO: Sopris West.CrossRefGoogle Scholar
  80. Warnes, E. D., Sheridan, S. M., Geske, J., & Warnes, W. A. (2005). A contextual approach to the assessment of social skills: Identifying meaningful behaviors for social competence. Psychology in the Schools, 42, 173–187.  https://doi.org/10.1002/pits.20052.CrossRefGoogle Scholar
  81. Wickham, H. (2014). Tidy data. The Journal of Statistical Software, 59, 1–24. Retrieved December 27, 2018 from http://www.jstatsoft.org/v59/i10/.
  82. World Health Organization. (2004). Promoting mental health: Concepts, emerging evidence, practice (summary report). Geneva, Switzerland: Author. Retrieved December 27, 2018 from http://www.who.int/mental_health/evidence/en/promoting_mhh.pdf.
  83. Ysseldyke, J., & Reschly, D. J. (2014). The evolution of school psychology: Origins, contemporary status, and future directions. In P. Harrison & A. Thomas (Eds.), Best practices in school psychology: Data-based and collaborative decision making (pp. 71–84). Bethesda, MD: The National Association of School Psychologists.Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Department of PsychologyLouisiana State UniversityBaton RougeUSA
  2. 2.Department of PsychologyUtah State UniversityLoganUSA

Personalised recommendations