Abstract
The purpose of this paper was to provide a systematic review to examine how researchers who have used the Behavioural Regulation in Exercise Questionnaire (BREQ; Mullan et al. Person Individ Differ 23:745–752, 1997) have applied validity theory to their investigations. Of the published studies (n = 138) included in the sample using the BREQ, only 29 coded studies explicitly presented evidence of score validity. Less than a quarter of the coded studies (20.69 %) cited a contemporary validity framework. Evidence of internal structure was the most common source of validity evidence presented, followed by relations to other variables (e.g., discriminant and convergent evidence) and content evidence. Validity evidence pertaining to response processes or consequences was not directly analyzed in the studies coded. Findings indicated that researchers need to incorporate modern validity theory and validation procedures into their investigations and analyses.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
In addition to the Standards, there are other validity frameworks from which researchers could base their validation efforts. For example, Messick (1995) advocated a progressive matrix of validity based on construct validity. Kane (2001) calls for a program of evaluation that is based on an argument of score validity. Zumbo (2007) has developed a validity framework that focuses on sample homogeneity and context of validity. Finally, Borsboom et al. (2004) proposed a conceptual framework for validity that opposes the unified view. Researchers are encouraged to read the original documents for a more in depth understanding of each framework.
- 2.
The Standards were never intended to be a prescription, or cookbook type document in which researchers can follow (Goodwin 2002). The process of validation is ongoing and as such, is based on the accumulation of knowledge.
- 3.
- 4.
Using a translation tool represents a limitation of the current study. For studies that required translation, it is possible that the quality of the translation impacted coding and interpretations. Consequently, caution is warranted when interpreting our results, especially for investigations that were published in a language other than English.
- 5.
We acknowledge that under the unified conception of validity, all research informs the evidence base informing score validity. Consequently a limitation of this study is that we narrowed our search to only articles in which the authors explicitly discuss evidence of score validity. Therefore, the validity evidence presented herein may not represent all validity evidence available for scores of the BREQ; however, we believe the sampling strategy will allow for stronger conclusions about the application of validity theory to the BREQ.
- 6.
Coding form can be obtained from the primary author upon request.
- 7.
A detailed table containing authors’ claims and how they were discrepant from The Standards can be obtained from the first author upon request.
- 8.
References marked with an asterisk indicate studies included in the validity evidence review. For a full list please contact the first author.
References
References marked with an asterisk indicate studies included in the validity evidence review. For a full list please contact the first author.
American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (1999). Standards for educational and psychological testing. Washington, DC: American Educational Research Association.
Bagozzi, R. P. (1981). An examination of the validity of two models of attitude. Multivariate Behavioral Research, 16, 323–359. doi:10.1207/s15327906mbr1603_4.
Borsboom, D., Mellenbergh, G. J., & van Heerden, J. (2004). The concept of validity. Psychological Review, 111, 1061–1071. doi:10.1037/0033-295X.111.4.1061.
Campbell, D. R., & Fiske, D. W. (1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56, 81–105. doi:10.1037/h0046016.
Cizek, G. J., Rosenberg, S. L., & Koons, H. H. (2008). Sources of validity evidence for educational and psychological tests. Educational and Psychological Measurement, 68, 397–410. doi:10.1177/0013164407310130.
Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16, 297–334.
*Daley, A. J., & Duda, J. L. (2006). Self-determination, stage of readiness to change for exercise, and frequency of physical activity in young people. European Journal of Sport Science, 6, 231–243. doi:10.1080/17461390601012637.
Deci, E. L., & Ryan, R. M. (2002). Self-determination research: Reflections and future directions. In E. L. Deci & R. M. Ryan (Eds.), Handbook of self-determination research (pp. 431–441). Rochester: University of Rochester Press.
Fornell, C., & Larcker, D. F. (1981). Evaluating structural equation models with unobservable variables and measurement error. Journal of Marketing Research, 18, 39–50.
Goodwin, L. D. (2002). Changing conceptions of measurement validity: An update on the new Standards. Journal of Nursing Education, 41, 100–106.
Gunnell, K. E., Schellenberg, B. J. I., Wilson, P. M., Crocker, P. R. E., Mack, D. E., & Zumbo, B. D. (2014). A review of validity evidence presented in the Journal of Sport and Exercise Psychology (2002–2012): Misconceptions and recommendations for validation research. In B. D. Zumbo & K. H. Chan (Eds.), Validity and validation in social, behavioral, and health sciences. Dordrecht: Springer.
Hagger, M. S., & Chatzisarantis, N. L. D. (2009). Assumptions in research in sport and exercise psychology. Psychology of Sport and Exercise, 10, 511–519. doi:10.1016/j.psychsport.2009.01.004.
*Hagger, M., Chatzisarantis, N. L. D., Hein, V., Soos, I., Karasai, I., Lintunen, T., & Leemans, S. (2009). Teacher, peer and parent autonomy support in physical education and leisure-time physical activity: A trans-contextual model of motivation in four nations. Psychology & Health, 24, 689–711. doi:10.1080/08870440801956192.
*Hein, V., & Hagger, M. S. (2007). Global self-esteem, goal achievement orientations, and self-determined behavioural regulations in a physical education setting. Journal of Sports Sciences, 25, 149–159. doi:10.1080/02640410600598315.
Jöreskog, K. G. (1970). Estimation and testing of simplex models. The British Journal of Mathematical and Statistical Psychology, 23, 121–145.
Kane, M. T. (2001). Current concerns in validity theory. Journal of Educational Measurement, 38, 319–342. doi:10.1111/j.1745-3984.2001.tb01130.x.
Kline, R. B. (2005). Principles and practice of structural equation modeling (2nd ed.). New York: The Guilford Press.
Marsh, H. W. (1998). Foreward. In J. L. Duda (Ed.), Advances in sport and exercise psychology measurement (pp. xv–xix). Morgantown: Fitness Information Technology.
Markland, D. (2010). Exercise motivation measurement index. Retrieved from http://www.bangor.ac.uk/~pes004/exercise_motivation/scales.htm
*Markland, D., & Tobin, V. (2004). A modification to the behavioural regulation in exercise questionnaire to include an assessment of amotivation. Journal of Sport & Exercise Psychology, 26, 191–196.
*McLachlan, S., Spray, C., & Hagger, M. S. (2011). The development of a scale measuring integrated regulation in exercise. British Journal of Health Psychology, 16, 722–743. doi:10.1348/2044-8287.002009.
Messick, S. (1995). Validity of psychological assessment: Validations of inferences from persons’ responses and performances as scientific inquiry into score meaning. American Psychologist, 50, 741–774.
Messick, S. (1998). Consequences of testing interpretation and use: The fusion of validity and values in psychological assessment (Research Report). Princeton: Educational Testing Service.
*Mullan, E., Markland, D., & Ingledew, D. K. (1997). A graded conceptualization of self-determination in the regulation of exercise behaviour: Development of a measure using confirmatory factor analytic procedures. Personality and Individual Differences, 23, 745–752. doi:10.1016/s0191-8869(97)00107-4.
Ryan, R. M., & Connell, J. P. (1989). Perceived locus of causality and internalisation: Examining reasons for acting in two domains. Journal of Personality and Social Psychology, 57, 749–761. doi:10.1037/0022-3514.57.5.749.
Sireci, S. G. (2009). Packing and unpacking sources of validity evidence: History repeats itself again. In R. W. Lissitz (Ed.), The concept of validity: Revisions, new directions and applications (pp. 19–37). Charlotte: Information Age Publishing.
Vallerand, R. J., Pelletier, L. G., Blais, M. R., Brière, N. M., Senécal, C., & Vallières, E. F. (1992). The academic motivation scale: A measure of internal, external and amotivation in education. Educational and Psychological Measurement, 52, 1003–1017. doi:10.1177/0013164492052004025.
Wilson, P. M. (2012). Exercise motivation. In G. Tenenbaum, R. C. Eklund, & A. Kamata (Eds.), Measurement in sport and exercise psychology (pp. 293–302). Champaign: Human Kinetics.
*Wilson, P. M., Rodgers, W. M., Loitz, C. C., & Scime, G. (2006). It’s who I am … Really! The importance of integrated regulation in exercise contexts. Journal of Applied Biobehavioral Research, 11, 79–104. doi:10.1111/j.1751-9861.2006.tb00021.x.
Wilson, P. M., Mack, D. E., & Sylvester, B. D. (2011). When a little myth goes a long way: The use (or misuse) of cut-points, interpretations and discourse with coefficient-alpha in exercise psychology. In A. M. Columbus (Ed.), Advances in psychology research (Vol. 77). Hauppauge: Nova.
Zhu, W. (1998). Comments on “development of a cadence curl-up test for college students” (Sparling, Millard-Stafford, & Snow, 1997): Concerns about validity and practicality. Research Quarterly for Exercise and Sport, 69, 308–310.
Zhu, W. (2012). Measurement practice in sport and exercise psychology: A historical comparative, and psychometric view. In G. Tenenbaum, R. C. Eklund, & A. Kamata (Eds.), Measurement in sport and exercise psychology (pp. 293–302). Champaign: Human Kinetics.
Zumbo, B. D. (2007). Validity: Foundational issues and statistical methodology. In C. R. Rao & S. Sinharay (Eds.), Handbook of statistics (Pyschometrics, Vol. 26, pp. 45–79). Amsterdam: Elsevier Science.
Zumbo, B. D. (2009). Validity as contextualized and pragmatic explanation and its implications for validation practice. In R. W. Lissitz (Ed.), The concept of validity: Revisions, new directions, and applications. Charlotte: Information Age Publishing.
Acknowledgement
The first and sixth authors were supported by a scholarship (doctoral award) from the Social Sciences and Humanities Research Council of Canada (SSHRC) during the preparation of this manuscript. The second and fifth authors were jointly supported by grant funding from the SSHRC during manuscript preparation and are affiliated with the Center for Bone and Muscle Health (Brock University). The fourth author was supported by a grant from SSHRC during manuscript preparation.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this chapter
Cite this chapter
Gunnell, K.E., Wilson, P.M., Zumbo, B.D., Crocker, P.R.E., Mack, D.E., Schellenberg, B.J.I. (2014). Validity Theory and Validity Evidence for Scores Derived from the Behavioural Regulation in Exercise Questionnaire. In: Zumbo, B., Chan, E. (eds) Validity and Validation in Social, Behavioral, and Health Sciences. Social Indicators Research Series, vol 54. Springer, Cham. https://doi.org/10.1007/978-3-319-07794-9_10
Download citation
DOI: https://doi.org/10.1007/978-3-319-07794-9_10
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-07793-2
Online ISBN: 978-3-319-07794-9
eBook Packages: Humanities, Social Sciences and LawSocial Sciences (R0)