Advertisement

Mindfulness

pp 1–13 | Cite as

Validation of a Scale for Assessing Social Validity in Mindfulness-Based Educational Programs

  • Luis López-González
  • David Herrero-Fernández
  • Alberto AmutioEmail author
  • Txemi Santamaría
  • William Van Gordon
ORIGINAL PAPER
  • 20 Downloads

Abstract

Objectives

Social validity (SV) is a concept used in intervention research and is concerned with the overall acceptability, relevance, and utility of an intervention to all intervention stakeholders. SV not only takes into account efficacy in respect of the pre-defined study outcomes, but also participants’ perceptions of the intervention as well as the wider social context in which it will be applied. There are a growing number of mindfulness-based educational programs (MBEPs) being empirically evaluated and implemented in educational settings. However, due to a lack of scientifically validated instruments that can assess SV in MBEPs, a systematic evaluation of SV in such programs has not been undertaken to date. The aim of this study was to investigate the psychometric properties of the Social Validity Scale of Mindfulness-Based Programs for Adolescents (Escala de Validez Social de Programas de Mindfulness para Adolescentes—EVSPM-A), composed of 20 items.

Methods

The sample comprised 512 compulsory secondary education and high school students (mean age = 14.5; SD = 1.57) from three Spanish educational centers that had completed an MBEP known as the TREVA Program.

Results

The final version of the scale showed good psychometric properties and factor analyses yielded five factors: global impact-satisfaction, acceptance and viability, individual perceived effectiveness, perceived classroom climate; training feasibility, and applicability of techniques.

Conclusions

The EVSPM-A appears to be a suitable means of assessing SV in MBEPs delivered to adolescents. Using the EVSPM-A to evaluate SV can help improve the implementation and long-term efficacy of MBEPs.

Keywords

Assessment Social validity Mindfulness-based interventions Educational programs 

Notes

Acknowledgments

We wish to express our deepest appreciation to the individuals who voluntarily participated in the project and university employees for their support with the study.

Author’s Contributions

LLG and AA conceived and designed the study and wrote and edited all the different versions of the ms and the final paper. DHF conducted data analyses. TS participated in recruiting the sample, and in the administration of the different instruments. WVG contributed to the writing and final editing of the manuscript.

Compliance with Ethical Standards

Ethical Approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. The study was approved by the Bioethics Committee of the University of Barcelona (Spain).

Conflict of Interest

The authors declare that they have no conflict of interest.

Informed Consent

Informed consent was obtained from all individual participants included in the study.

References

  1. Ager, K., Albrecht, N. J., & Cohen, M. (2015). Mindfulness in schools research project: Exploring Student’s perspectives of mindfulness—what are students’ perspectives of learning mindfulness practices at school. Psychology, 6(7), 896–914.  https://doi.org/10.4236/psych.2015.67088.CrossRefGoogle Scholar
  2. Baltzell, A. (2016). Mindfulness and performance. London: Cambridge University Press.Google Scholar
  3. Barret, P. M., Short, A. L., & Wescombe, K. (2001). Examining the social validity of the FRIENDS treatment program for anxious children. Behaviour Change, 18(2), 63–67.CrossRefGoogle Scholar
  4. Beauchemin, J., Hutchins, T. L., & Patterson, F. (2008). Mindfulness meditation may lessen anxiety, promote social skills, and improve academic performance among adolescents with learning disabilities. Complementary Health Practice Review, 13(1), 34–45.  https://doi.org/10.1177/1533210107311624.CrossRefGoogle Scholar
  5. Bergomi, C., Tschacher, W., & Kupper, Z. (2013). The assessment of mindfulness with self-report measures: Existing scales and open issues. Mindfulness, 4(3), 191–202.  https://doi.org/10.1007/s12671-012-0110-9.CrossRefGoogle Scholar
  6. Bluth, K., & Blanton, P. W. (2015). The influence of self-compassion on emotional well-being among early and older adolescent males and females. The Journal of Positive Psychology, 10(3), 219–230.  https://doi.org/10.1080/17439760.2014.936967.CrossRefGoogle Scholar
  7. Bollen, K. A., & Curran, P. J. (2005). Latent curve models: a structural equation perspective. Hoboken, NJ: Wiley.CrossRefGoogle Scholar
  8. Carmines, E. G., & McIver, J. P. (1981). Analyzing models with unobserved variables: analysis of covariance structures. In G. W. Bornstedt & E. F. Borgatta (Eds.), Social Measurement: Current issues (pp. 112–133). Beverly Hills: Sage.Google Scholar
  9. Carroll, R., & St. Peter, C. C. (2014). Methods for assessing social validity of behavioral intervention plans for children with attention deficit hyperactivity disorder. Acta de Investigación Psicológica, 4(3), 1642–1656.  https://doi.org/10.1016/S2007-4719(14)70971-8.CrossRefGoogle Scholar
  10. Carter, S. L. (2009). The social validity manual: a guide to subjective evaluation of behavior interventions. London: Academic Press.Google Scholar
  11. Crane, R. S., & Hecht, F. M. (2018). Intervention integrity in mindfulness-based research. Mindfulness, 9, 1370–1380.  https://doi.org/10.1007/s12671-018-0886-3.CrossRefGoogle Scholar
  12. Crane, R. S., Eames, C., Kuyken, W., Hastings, R. P., Williams, J. M. G., Bartley, T., et al. (2013). Development and validation of the mindfulness-based interventions–teaching assessment criteria (MBI: TAC). Assessment, 20(6), 681–688.  https://doi.org/10.1177/1073191113490790.CrossRefGoogle Scholar
  13. Da Fonceca, V. (1998). Manual de Observación Psicomotriz. Zaragoza: Inde.Google Scholar
  14. Dariotis, J., Miralbal-Beltran, R., Cluxtin-Keller, F., Gould, L. F., Greenberg, M. T., & Mendelson, M. (2016). A qualitative evaluation of student learning and skills use in a school-based mindfulness and yoga program. Mindfulness, 7(1), 76-89.  https://doi.org/10.1007/s12671-015-0463-y.
  15. Davidson, R. J., Dunne, J., Eccles, J. S., Engle, A., Greenberg, M., Jennings, P., & Vago, D. (2012). Contemplative practices and mental training: prospects for American education. Child Development Perspectives, 6(2), 146–153.  https://doi.org/10.1111/j.1750-8606.2012.00240.x.CrossRefGoogle Scholar
  16. Eklund, K., O’malley, M., & Meyer, L. (2017). Gauging mindfulness in children and youth: school-based applications. Psychology in the Schools, 54(1), 101–114.  https://doi.org/10.1002/pits.21983.CrossRefGoogle Scholar
  17. Emerson, L. M., Leyland, A., Hudson, K., Rowse, G., Hanley, P., & Hugh-Jones, S. (2017). Teaching mindfulness to teachers: a systematic review and narrative synthesis. Mindfulness, 8(5), 1136-1149.  https://doi.org/10.1007/s12671-017-0691-4.
  18. Felver, J. C., Celis-de Hoyos, C. E., Tezanos, K., & Singh, N. N. (2016). A systematic review of mindfulness-based interventions for youth in school settings. Mindfulness, 7(1), 34–45.  https://doi.org/10.1007/s12671-015-0389-4.CrossRefGoogle Scholar
  19. Flook, L., Smalley, S. L., Kitil, M. J., Galla, B. M., Kaiser-Greenland, S., & Locke, J. (2010). Effects of mindful awareness practices on executive functions in elementary school children. Journal of Applied School Psychology, 26, 70–95.  https://doi.org/10.1080/15377900903379125.CrossRefGoogle Scholar
  20. Forman, K., & Soloff, G. (2014). Preparing for educational leadership (2nd ed.). New York: Pearson Education.Google Scholar
  21. Fornell, C., & Larcker, D. F. (1981). Evaluating structural equation models with unobservable variables and measurement error. Journal of Marketing Research, 27, 39–50.CrossRefGoogle Scholar
  22. Greenberg, M. T., & Harris, A. R. (2012). Nurturing mindfulness in children and youth: Current state of research. Child Development Perspectives, 6, 161–166.  https://doi.org/10.1111/j.1750-8606.2011.00215.x.CrossRefGoogle Scholar
  23. Hanley, G. P. (2012). Functional assessment of problem behavior: dispelling myths, overcoming implementation obstacles, and developing new lore. Behavior Analysis in Practice, 5(1), 54–72.  https://doi.org/10.1007/BF03391818.CrossRefGoogle Scholar
  24. Himelstein, S. (2016). A mindfulness-based approach to working with high-risk adolescents. London: Routledge.Google Scholar
  25. Hokke, H., Lievense, P., & Sterkenburg, P. S. (2014). The effects of mindfulness for people with visual and moderate intellectual disability and their caregivers. Journal of Applied Research in Intellectual Disabilities, 27(4), 311.Google Scholar
  26. Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1–55.  https://doi.org/10.1080/10705519909540118.CrossRefGoogle Scholar
  27. Husley, J., & Meyer, G. (2003). The incremental validity of psychological testing and assessment: conceptual, methodological, and statistical issues. Psychological Assessment, 15, 466–455.  https://doi.org/10.1037/1040-3590.15.4.446.Google Scholar
  28. Jennings, P. A., Brown, J. L., Frank, J. L., Doyle, S., Oh, Y., Davis, R., Rasheed, D., De Weese, A., DeMauro, A. A., Cham, H., & Greenberg, M. T. (2017). Impacts of the CARE for Teachers Program on teachers’ social and emotional competence and classroom interactions. Journal of Educational Psychology. Advance online publication.  https://doi.org/10.1037/edu0000187.
  29. Jöreskog, K., & Sörbom, D. (1997). Lisrel 8: user’s reference guide. Lincolnwood: Scientific Software International.Google Scholar
  30. Kallapiran, K., Koo, S., Kirubakaran, R., & Hancock, K. (2015). Review: effectiveness of mindfulness in improving mental health symptoms of children and adolescents: a meta-analysis. Child and Adolescent Mental Health, 20(4), 182–194.  https://doi.org/10.1111/camh.12113.CrossRefGoogle Scholar
  31. Kern, L., & Manz, P. (2004). A look at current validity issues of school-wide behavior support. Behavioral Disorders, 30, 47–59.CrossRefGoogle Scholar
  32. Klingbeil, D. A., Renshaw, T. L., Willenbrink, J. B., Copek, R. A., Chan, K. T., Haddock, A., et al. (2017a). Mindfulness-based interventions with youth: a comprehensive meta-analysis of group-design studies. Journal of School Psychology, 63, 77–103.  https://doi.org/10.1016/j.jsp.2017.03.006.CrossRefGoogle Scholar
  33. Klingbeil, D. A., Fischer, A. J., Renshaw, T. L., Bloomfield, B. S., Polakoff, B., Willenbrink, J. B., et al. (2017b). Effects of mindfulness-based interventions on disruptive behavior: a meta-analysis of single-case research. Psychology in the Schools, 54(1), 70–87.  https://doi.org/10.1002/pits.21982.CrossRefGoogle Scholar
  34. Kuby, A. K., McLean, N., & Allen, K. (2015). Validation of the child and adolescent mindfulness measure (CAMM) with non-clinical adolescents. Mindfulness, 6(6), 1448–1455.  https://doi.org/10.1007/s12671-015-0418-3.CrossRefGoogle Scholar
  35. Kuyken, W., Weare, K., Ukoumunne, O.C Vicary, R., Motton, N., Burnett, R., Cullen, C., Hennelly, S., & Huppert, F. (2013). Effectiveness of the mindfulness in schools programme: non-randomised controlled feasibility study. The British Journal of Psychiatry, 203(2), 126–131; doi:  https://doi.org/10.1192/bjp.bp.113.126649.
  36. Leonard, N. R., Jha, A. P., Casarjian, B., Goolsarran, B., Garcia, C., Cleland, C. M., Gwadz, M. V., & Massey, Z. (2013). Mindfulness training improves attentional task performance in incarcerated youth: a group randomized controlled intervention trial. Frontiers in Psychology, 4(792), 1–10.  https://doi.org/10.3389/fpsyg.2013.00792.Google Scholar
  37. Liehr, P., & Diaz, N. (2010). A pilot study examining the effect of mindfulness on depression and anxiety for minority children. Archives of Psychiatric Nursing, 24(1), 69–71.  https://doi.org/10.1016/j.apnu.2009.10.001.CrossRefGoogle Scholar
  38. López-González, L. (2010). Disseny i desenvolupament d’un programa de relaxació vivencial aplicada a l’aula. Doctoral thesis, University of Barcelona - (UB). Barcelona, Spain. Available at: http://tdx.cat/bitstream/handle/10803/2364/LLG_TESI.pdf?sequence=1. Accessed 19 March 2019.
  39. López-González, L., & Bisquerra, R. (2013). Validación y análisis de una escala breve para evaluar el clima de clase en Educación Secundaria. ISEP SCIENCE, 5, 62–77.Google Scholar
  40. López-González, L., Álvarez-González, M., & Bisquerra, R. (2016a). Mindfulness and action research in secondary schools. Gestation of the TREVA program. Revista Interuniversitaria de Formación del Profesorado, 87(30,3), 75–91.Google Scholar
  41. López-González, L., Amutio, A., Herrero-Fernández, D., & Bisquerra, R. (2016b). Validation of a relaxation-mindfulness skills and states scale for adolescents. Revista Interuniversitaria de Formación del Profesorado, 87(30,3), 93–105.Google Scholar
  42. López-González, L., Amutio, A., Oriol, X., & Bisquerra, R. (2016c). Habits related to relaxation and mindfulness of high school students: influence on classroom climate and academic performance. Journal of Psychodidactics, 21(1), 121.138.  https://doi.org/10.1387/RevPsicodidact.13866.Google Scholar
  43. López-González, L., Amutio, A., & Herrero-Fernández, D. (2018a). The relaxation-mindfulness competence of secondary and high school students and its influence on classroom climate and academic performance. European Journal of Education and Psychology, 11(1), 5–17.CrossRefGoogle Scholar
  44. López-González, L., Amutio, A., Oriol, X., Gázquez, J. J., Pérez-Fuentes, M. C., & Molero, M. M. (2018b). Development and validation of the relaxation-mindfulness scale for adolescents (EREMIND-A). Psicothema, 30(2), 224–231.Google Scholar
  45. Lorenzo-Seva, U., & Ferrando, P. J. (2013). Factor 9.2: a comprehensive program for fitting exploratory and Semiconfirmatory factor analysis and IRT models. Applied Psychological Measurement, 37(6), 497–498.CrossRefGoogle Scholar
  46. Luiselli, J. K., Worthen, D., Carbonell, L., & Queen, A. H. (2017). Social validity assessment of mindfulness education and practices among high school students. Journal of Applied School Psychology, 33(2), 124–135.  https://doi.org/10.1080/15377903.2016.1264531.CrossRefGoogle Scholar
  47. Marchant, M., Heath, M., & Miramontes, N. Y. (2013). Merging empiricism and humanism role of social validity in the school-wide positive behavior support model. Journal of Positive Behavior Interventions, 15(4), 221–230.  https://doi.org/10.1177/1098300712459356.CrossRefGoogle Scholar
  48. Mendelson, T., Greenberg, M. T., Dariotis, J. K., Gould, L. F., Rhoades, B. L., & Leaf, P. J. (2010). Feasibility and preliminary outcomes of a school-based mindfulness intervention for urban youth. Journal of Abnormal Child Psychology, 38, 985–994.  https://doi.org/10.1007/s10802-010-9418-x.CrossRefGoogle Scholar
  49. Metz, S. M., Frank, J. L., Reibel, D., Cantrell, T., Sanders, R., & Broderick, P. C. (2013). The effectiveness of the learning to BREATHE program on adolescent emotion regulation. Research in Human Development, 10(3), 252–272.  https://doi.org/10.1080/15427609.2013.818488.CrossRefGoogle Scholar
  50. Mîndrilă, D. (2010). Maximum likelihood (ML) and diagonally weighted least squares (DWLS) estimation procedures: a comparison of estimation bias with ordinal and multivariate non-normal data. International Journal of Digital Society, 1(1), 60–66.CrossRefGoogle Scholar
  51. Pallozzi, R., Wertheim, E., Paxton, S., & Ong, B. (2016). Trait mindfulness measures for use with adolescents: a systematic review. Mindfulness.  https://doi.org/10.1007/s12671-016-0567-z.
  52. Parker, A. E., & Kupersmidt, J. B. (2016). Two universal mindfulness education programs for elementary and middle-school students: master mind and moment. In K. Schonert-Reichl & R. Roeser (Eds.), Handbook of mindfulness in education. Mindfulness in Behavioral Health. New York, NY: Springer.  https://doi.org/10.1007/978-1-4939-3506-2_21.Google Scholar
  53. Rennekamp, R. A., & Arnold, M. E. (2009). What progress, program evaluation? Reflections on a quarter-century of extension evaluation practice. Journal of Extension [On-line], 47(3), Article 3COM1 Available at: https://www.joe.org/joe/2009june/pdf/JOE_v47_3comm1.pdf. Accessed 19 March 2019.
  54. Sainio, M., Herkama, S., Turunen, T., Rönkkö, M., Kontio, M., Poskiparta, E., & Salmivalli, C. (2018). Sustainable antibullying program implementation: school profiles and predictors. Scandinavian Journal of Psychology.  https://doi.org/10.1111/sjop.12487.
  55. Saltzman, A., & Goldin, P. (2008). Mindfulness-based stress reduction for school-age children. In L. A. Greco & S. C. Hayes (Eds.), Acceptance and mindfulness treatments for children and adolescents: a practitioner's guide (pp. 139–161). Oakland, CA, US: New Harbinger Publications.Google Scholar
  56. Sanger, K. L., & Dorjee, D. (2015). Mindfulness training for adolescents: a neurodevelopmental perspective on investigating modifications in attention and emotion regulation using event-related brain potentials. Cognitive, Affective, & Behavioral Neuroscience, 15(3), 696–711.  https://doi.org/10.3758/s13415-015-0354-7.CrossRefGoogle Scholar
  57. Sapthiang, S., Van Gordon, W., & Shonin, E. (2019). Mindfulness in schools: a health promotion approach to improving adolescent mental health. International Journal of Mental Health and Addiction, 17, 112-119.  https://doi.org/10.1007/s11469-018-0001-y.
  58. Schonert-Reichl, K. A., & Roeser, R. W. (2016). Handbook of mindfulness in education. New York: Springer.Google Scholar
  59. Semple, R. J., Droutman, V., & Reid, B. A. (2017). Mindfulness goes to school: things learned (so far) from research and real-world experiences. Psychology in the Schools, 54(1), 29–52.  https://doi.org/10.1002/pits.21981.CrossRefGoogle Scholar
  60. Shababi, S. (2014). Use of a mindfulness practice to decrease problem behavior and increase engaged time of three students in an elementary school setting. Vancouver: University of British Columbia.Google Scholar
  61. Sheinman, N., Levy, S., & Lahav, A. (2011). Mindfulness for a whole school–mindfulness and mind-body skills with children. Unpublished manuscript.Google Scholar
  62. Shonani, A., & Slone, M. (2013). Middle school transition from the strengths perspective: young adolescents’ character strengths, subjective well-being, and school adjustment. Journal of Happiness Studies, 14, 1163–1181.  https://doi.org/10.1007/s10902-012-9374-y.CrossRefGoogle Scholar
  63. Shonin, E., Van Gordon, W., & Griffiths, M. D. (2012). The health benefits of mindfulness-based interventions for children and adolescents. Education and Health, 30, 94–97.Google Scholar
  64. Steiger, J. H. (2007). Understanding the limitations of global fit assessment in structural equation modeling. Personality and Individual Differences, 42(5), 893–898.  https://doi.org/10.1016/j.paid.2006.09.017.CrossRefGoogle Scholar
  65. Stone, J. R. A. (2014). Mindfulness in schools: taking present practice into account. DECP Debate Edition. Available at: https://northampton.academia.edu/JacquelineStone/Papers. Accessed 19 March 2019.
  66. Tan, L. B. G. (2016). A critical review of adolescent mindfulness-based programmes. Clinical Child Psychology and Psychiatry, 21(2), 175–177.  https://doi.org/10.1177/1359104515577486.CrossRefGoogle Scholar
  67. Van Gordon, W., Shonin, E., & Griffiths, M. D. (2015). Towards a second-generation of mindfulness-based interventions. Australia and New Zealand Journal of Psychiatry, 49, 591–591.CrossRefGoogle Scholar
  68. Wiesner, M., & Schanding, G. T. (2013). Exploratory structural equation modeling, bifactor models, and standard confirmatory factor analysis models: application to the BASC-2 behavioral and emotional screening system teacher form. Journal of School Psychology, 51, 751–763.  https://doi.org/10.1016/j.jsp.2013.09.001.CrossRefGoogle Scholar
  69. Wisner, B. (2017). Mindfulness and meditation for adolescents. London: Springer.Google Scholar
  70. Wolf, M. M. (1978). Social validity: the case for subjective measurement or how applied behavior analysis is finding its heart. Journal of Applied Behavior Analysis, 11, 203–214.  https://doi.org/10.1901/2Fjaba.1978.11-203.CrossRefGoogle Scholar
  71. Worthen, D., & Luiselli, J. K. (2017). Social validity assessment and intervention evaluation of mindfulness education and practices with high school students. Mindfulness, 8(4), 903–910.  https://doi.org/10.1007/s12671-016-0664-z.CrossRefGoogle Scholar
  72. Zenner, C., Herrnleben-Kurz, S., & Walach, H. (2014). Mindfulness-based interventions in schools—a systematic review and meta-analysis. Frontiers in Psychology, 5, 603.  https://doi.org/10.3389/fpsyg.2014.00603.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Institut de Desenvolupament Professional, University of BarcelonaBarcelonaSpain
  2. 2.Universidad Europea del AtlánticoSantanderSpain
  3. 3.Facultad de Relaciones Laborales y Trabajo SocialUniversity of the Basque Country (UPV/EHU)LeioaSpain
  4. 4.Faculty of Psychology and EducationUniversity of DeustoBilbaoSpain
  5. 5.Human Sciences Research CentreUniversity of DerbyDerbyUK

Personalised recommendations