Abstract
The purpose of this investigation was to systematically review and provide a critical narrative commentary on validation investigations from studies published in the Journal of Sport and Exercise Psychology (JSEP) to determine how modern validity theory has been incorporated or omitted in validation studies. Out of the 405 articles published between 2002 and 2012, 50 met inclusion criteria. Coded items pertaining to validity theory were based on the Standards validation framework (American Educational Research Association [AERA], American Psychological Association [APA], National Council on Measurement in Education [NCME], Standards for educational and psychological testing. American Educational Research Association, Washington, DC, 1999). Results revealed that in general, researchers publishing validation investigations in JSEP are not basing their work on established validation frameworks, or using validity theory developed by psychometricians (e.g., 70 % of studies provided no validation citation). With respect to the types of validity evidence presented, researchers ascertained score validity from evidence based on content, internal structure, and relations to other variables. Validity evidence based on response processes (2 %) and consequences (0 %) were largely omitted from investigations. Critical analysis while coding yielded three common misconceptions about validity and validation. Recommendations to improve the state of validation work in sport and exercise psychology are forwarded.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
The coding scheme is available from the first author upon request.
- 2.
Some authors refer to all validity evidence as construct validity (i.e., a unified perspective). Construct validity was only coded as ‘construct validity’ under relations to other variables if the authors described it as a type of evidence bearing on the relationship with other variables.
- 3.
We recognize that different researchers have different conceptualizations of discriminant and convergent evidence (e.g., Brown 2006; Kline 2010). If researchers are going to use alternative ways of examining convergent and discriminant evidence, they should ensure they cite relevant sources to support their analysis.
References
American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (1999). Standards for educational and psychological testing. Washington, DC: American Educational Research Association.
Bartholomew, K. J., Ntoumanis, N., & Thøgersen-Ntoumani, C. (2010). The controlling interpersonal style in a coaching context: Development and initial validation of a psychometric scale. Journal of Sport & Exercise Psychology, 32, 193–216.
Boardley, I. D., & Kavussanu, M. (2007). Development and validation of the moral disengagement in sport scale. Journal of Sport & Exercise Psychology, 29, 608–628.
Borsboom, D., Mellenbergh, G. J., & van Heerden, J. (2004). The concept of validity. Psychological Review, 111, 1061–1071.
Borsboom, D., Cramer, A. O. J., Kievit, R. A., Scholten, A. Z., & Franic, S. (2009). The end of construct validity. In R. W. Lissitz (Ed.), The concept of validity: Revisions, new directions and applications (pp. 135–170). Charlotte: Information Age Publishing.
Brown, A. (2006). Confirmatory factor analysis for applied research. New York: Guilford Press.
Campbell, D. R., & Fiske, D. W. (1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56, 81–105.
Cizek, G. J., Rosenberg, S. L., & Koons, H. H. (2008). Sources of validity evidence for educational and psychological tests. Educational and Psychological Measurement, 68, 397–410.
Dunn, J. G. H., Bouffard, M., & Rogers, W. T. (1999). Assessing item content-relevance in sport psychology scale-construction research: Issue and recommendations. Measurement in Physical Education and Exercise Science, 3, 15–36.
Freeman, P., Coffee, P., & Rees, T. (2011). The PASS-Q: The Perceived Available Support in Sport Questionnaire. Journal of Sport & Exercise Psychology, 33, 54–74.
Goodwin, L. D. (2002). Changing conceptions of measurement validity: An update on the new Standards. Journal of Nursing Education, 41, 100–106.
Gucciardi, D. (2011). The relationship between developmental experiences and mental toughness in adolescent cricketers. Journal of Sport & Exercise Psychology, 33, 370–393.
Hagger, M. S., & Chatzisarantis, N. L. D. (2009). Assumptions in research in sport and exercise psychology. Psychology of Sport & Exercise, 10, 511–519.
Hubley, A. M., & Zumbo, B. D. (1996). A dialectic on validity: Where we have been and where we are going. The Journal of General Psychology, 123, 207–215.
Jones, M., Lane, A. M., Bray, S. R., Uphill, M., & Catlin, J. (2005). Development and validation of the Sport Emotion Questionnaire. Journal of Sport & Exercise Psychology, 27, 407–431.
Kane, M. T. (1992). An argument-based approach to validity. Psychological Bulletin, 112, 527–535.
Kane, M. T. (2001). Current concerns in validity theory. Journal of Educational Measurement, 38, 319–342.
Kane, M. T. (2013). Validating the interpretations and uses of test scores. Journal of Educational Measurement, 50, 1–73.
Kline, R. B. (2010). Principles and practice of structural equation modeling (3rd ed.). New York: Guilford Press.
Lonsdale, C., Hodge, K., & Rose, E. A. (2008). The Behavioral Regulation in Sport Questionnaire (BRSQ): Instrument development and initial validity evidence. Journal of Sport & Exercise Psychology, 30, 323–355.
Markland, D., & Tobin, V. (2004). A modification to the Behavioural Regulation in Exercise Questionnaire to include an assessment of amotivation. Journal of Sport & Exercise Psychology, 26, 191–196.
Messick, S. (1995). Validity of psychological assessment: Validations of inferences from persons’ responses and performances as scientific inquiry into score meaning. American Psychologist, 50, 741–774.
Morton, K. L., Barling, J., Rhodes, R. E., Mâsse, L. C., Zumbo, B. D., & Beauchamp, M. R. (2011). The application of transformational leadership theory to parenting: Questionnaire development implications for adolescent self-regulatory efficacy and life satisfaction. Journal of Sport & Exercise Psychology, 33, 688–709.
Myers, N. D., Feltz, D. L., Guillén, F., & Dithurbide, L. (2012). Development of, and initial validity evidence for, the Referee Self-Efficacy Scale: A multistudy report. Journal of Sport & Exercise Psychology, 43, 737–765.
Schmidt, F. L., & Hunter, J. E. (1977). Development and general solution to the problem of validity generalization. Journal of Applied Psychology, 62, 529–540.
Schutz, R. W., & Gessaroli, M. E. (1993). Use, misuse, and disuse of psychometric in sport psychology research. In R. N. Singer, M. Murphy, & L. K. Tennant (Eds.), Handbook of research on sport psychology (pp. 901–917). New York: Macmillan.
Sebire, S. J., Standage, M., & Vansteenkiste, M. (2008). Development and validation of the goal content for exercise questionnaire. Journal of Sport & Exercise Psychology, 30, 353–377.
Shepard, L. A. (1993). Evaluating test validity. Review of Research in Education, 19, 405–450.
Sireci, S. G. (2009). Packing and unpacking sources of validity evidence: History repeats itself again. In R. W. Lissitz (Ed.), The concept of validity: Revisions, new directions and applications (pp. 19–37). Charlotte: Information Age Publishing.
Thompson, B. (1992). Two and one-half decades of leadership and measurement and devaluation. Journal of Counseling and Development, 70, 434–438.
Tukey, J. W. (1986). Sunset salvo. The American Statistician, 40, 72–76.
Williams, S. E., & Cumming, J. (2011). Measuring athlete imagery ability: The sport imagery ability questionnaire. Journal of Sport & Exercise Psychology, 33, 416–440.
Williams, S. E., Cumming, J., Ntoumanis, N., Nordin-Bates, S. M., Ramsey, R., & Hall, C. (2012). Further validation and development of the movement imagery questionnaire. Journal of Sport & Exercise Psychology, 34, 621–646.
Zhu, W. (1998). Comments on “development of a cadence curl-up test for college students” (Sparling, Millard-Stafford, & Snow, 1997): Concerns about validity and practicality. Research Quarterly for Exercise and Sport, 69, 308–310.
Zhu, W. (2012). Measurement practice in sport and exercise psychology: A historical comparative, and psychometric view. In G. Tenenbaum, R. C. Eklund, & A. Kamata (Eds.), Measurement in sport and exercise psychology (pp. 293–302). Champaign: Human Kinetics.
Zumbo, B. D. (2007). Validity: Foundational issues and statistical methodology. In C. R. Rao & S. Sinharay (Eds.), Handbook of statistics (Pyschometrics, Vol. 26, pp. 45–79). Amsterdam: Elsevier.
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this chapter
Cite this chapter
Gunnell, K.E., Schellenberg, B.J.I., Wilson, P.M., Crocker, P.R.E., Mack, D.E., Zumbo, B.D. (2014). A Review of Validity Evidence Presented in the Journal of Sport and Exercise Psychology (2002–2012): Misconceptions and Recommendations for Validation Research. In: Zumbo, B., Chan, E. (eds) Validity and Validation in Social, Behavioral, and Health Sciences. Social Indicators Research Series, vol 54. Springer, Cham. https://doi.org/10.1007/978-3-319-07794-9_8
Download citation
DOI: https://doi.org/10.1007/978-3-319-07794-9_8
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-07793-2
Online ISBN: 978-3-319-07794-9
eBook Packages: Humanities, Social Sciences and LawSocial Sciences (R0)