Comparing Likert Scale Functionality Across Culturally and Linguistically Diverse Groups in Science Education Research: an Illustration Using Qatari Students’ Responses to an Attitude Toward Science Survey
Surveying is a common methodology in science education research, including cross-national and cross-cultural comparisons. The literature surrounding students’ attitudes toward science, in particular, illustrates the prevalence of efforts to translate instruments with the eventual goal of comparing groups. This paper utilizes survey data from a nationally representative cross-sectional study of Qatari students in grades 3 through 12 to frame a discussion around the adequacy and extent to which common adaptations allow comparisons to be made among linguistically or culturally different respondents. The analytic sample contained 2615 students who responded to a previously validated 32-item instrument, 1704 of whom completed the survey in Modern Standard Arabic and 911 in English. The purpose of using these data is to scrutinize variation in the performance of the instrument between groups of respondents as determined by language of survey completion and cultural heritage. Multi-group confirmatory factor analysis was employed to investigate issues of validity associated with the performance of the survey with each group and to evaluate the appropriateness of using this instrument to make simultaneous comparisons across the distinct groups. Findings underscore the limitations of group comparability that may persist even when issues of translation and adaptation were heavily attended to during instrument development.
KeywordsAttitudes toward science Cross-sectional Multi-group CFA Translation Validity
- Adolphe, F. (2002). A cross-national study of classroom environment and attitudes among junior secondary science students in Australia and in Indonesia (Doctoral dissertation). Curtin University, Australia.Google Scholar
- Ali, M. S., Mohsin, M. N., & Iqbal, M. Z. (2013). The discriminant validity and reliability for Urdu version of Test of Science-Related Attitudes (TOSRA). International Journal of Humanities and Social Science, 3(2), 29–39.Google Scholar
- Bentler, P. M. (2005). EQS 6.1: Structural equations program manual. Encino, CA: Multivariate Software, Inc..Google Scholar
- Brewer, D. J., Augustine, C. H., Zellman, G. L., Ryan, G. W., Goldman, C. A., Stasz, C., & Constant, L. (2007). Education for a new era: Design and implementation of K–12 education reform in Qatar. Retrieved from http://www.rand.org/pubs/monographs/2007/RAND MG548.pdf.
- Boone, W. J., Townsend, J. S., & Staver, J. (2011). Using Rasch theory to guide the practice of survey development and survey data analysis in science education and to inform science reform efforts: An exemplar utilizing STEBI self-efficacy data. Science Education, 95, 258–280. https://doi.org/10.1002/sce.20413.
- Brislin, R. W., Lonner, W. J., & Thorndike, R. M. (1973). Cross-cultural research methods. New York, NY: Wiley.Google Scholar
- Byrne, B. M. (2006). Structural equation modeling with EQS: Basic concepts, applications, and programming (2nd ed.). Mahwah, NJ: Erlbaum.Google Scholar
- Byrne, B. M. (2008). Testing for multigroup equivalence of a measuring instrument: A walk through the process. Psicothema, 20(4), 872–882.Google Scholar
- Campbell, A. A., & Katona, G. (1953). The sample survey: A technique for social science research. In L. Festinger & D. Katz (Eds.), Research methods in the behavioral sciences (pp. 14–55). New York, NY: Dryden.Google Scholar
- Central Intelligence Agency (2013). Qatar. In The world factbook. Retrieved from https://www.cia.gov/library/publications/the-world-factbook/geos/qa.html.
- Curebal F (2004). Gifted students’ attitudes towards science and classroom environment based on gender and grade level (Unpublished Master's Thesis). Graduate School of Natural and Applied Sciences at Middle East Technical University, Ankara, Turkey.Google Scholar
- Fraser, B. (1981). Test of science related attitudes. Melbourne, Australia: Australian Council for Educational Research.Google Scholar
- Gall, M. D., Borg, W. R., & Gall, J. P. (1996). Educational research: An introduction. White Plains, NY: Longman.Google Scholar
- General Secretariat for Development Planning (2010). Qatar national vision 2030. Doha, Qatar: Authors.Google Scholar
- Harkness, J. A., & Schoua-Glusberg, A. (1998). Questionnaires in translation. ZUMA-Nachrichten Spezial, 3(1), 87–127.Google Scholar
- Harkness, J. A., Van de Vijver, F. J., & Mohler, P. P. (2003). Cross-cultural survey methods. Hoboken, NJ: Wiley-Interscience.Google Scholar
- Hayduk, L. A. (1987). Structural equation modeling with LISREL: Essentials and advances. Baltimore, MD: The Johns Hopkins University Press.Google Scholar
- Jenkins, E. W., & Pell, G. (2006). The Relevance of Science Education Project (ROSE) in England: A summary of findings. Leeds, England: University of Leeds Centre for Studies in Science and Mathematics Education.Google Scholar
- Jowell, R., Roberts, C., Fitzgerald, R., & Eva, G. (Eds.). (2007). Measuring attitudes cross-nationally: Lessons from the European Social Survey. London, England: Sage.Google Scholar
- Kellett, M., & Ding, S. (2004). Middle childhood. In S. Fraser, V. Lewis, S. Ding, M. Kellett, & C. Robinson (Eds.), Doing research with children and young people (pp. 161–174). London, England: Sage.Google Scholar
- Kline, P. (1979). Psychometrics and psychology. London, England: Academic Press.Google Scholar
- Lowe, J. P. (2004). The effect of a cooperative group work and assessment on the attitudes of students towards science in New Zealand (Unpublished doctoral dissertation). Curtin University of Technology, Curtin, Australia.Google Scholar
- Mourtaga, K. (2004). Investigating writing problems among Palestinian students: Studying English as a foreign language. Bloomington, Indiana: Author House.Google Scholar
- Navarro, M., Förster, C., González, C., & González-Pose, P. (2016). Attitudes toward science: Measurement and psychometric properties of the Test of Science-Related Attitudes for its use in Spanish-speaking classrooms. International Journal of Science Education, 38(9), 1459–1482.CrossRefGoogle Scholar
- Nunnally, J., & Bernstein, L. (1994). Psychometric theory. New York, NY: McGraw-Hill Higher, INC..Google Scholar
- Qatar Foundation (2009). Science and research. Retrieved December 6, 2009 from http://www.qf.org.qa/output/Page18.asp.
- Rashed, R. (2003). Report on ROSE project in Egypt. Retrieved from http://roseproject.no/network/countries/egypt/report-egy.pdf.
- Santiboon, T. (2013). School environments inventory in primary education in Thailand. Merit Research Journal of Education and Review, 1(10), 250–258.Google Scholar
- Schreiner, C., & Sjøberg, S. (2004). Sowing the seeds of ROSE: Background, rationale, questionnaire development and data collection for ROSE (the Relevance of Science Education)—A comparative study of students’ views of science and science education. Oslo, Norway: University of Oslo, Department of Teacher Education and School Development.Google Scholar
- Sechrest, L., Fay, T. L., & Zaidi, S. H. (1972). Problems of translation in cross-cultural research. Journal of Cosscultural Psychology, 3(1), 41–56.Google Scholar
- Squires, A., Aiken, L. H., van den Heede, K., Sermeus, W., Bruyneel, L., Lindqvist, R., . . . Matthews, A. (2013). A systematic survey instrument translation process for multi-country, comparative health workforce studies. International Journal of Nursing Studies, 50(2), 264–273.Google Scholar
- Streiner D.L., & Norman G.R. (1989). Health measurement scales: A practical guide to their development and use. New York, NY: Oxford University.Google Scholar
- Tavakol, M., & Dennick, R. (2011). Making sense of Cronbach's alpha. International Journal of Medical Education, 2, 53–55.Google Scholar
- Telli, S. (2006). Students’ perceptions of their science teachers’ interpersonal behaviour in two countries: Turkey and the Netherlands (Unpublished doctoral dissertation). The Graduate School of Natural and Applied Sciences, Middle East Technical University, Ankara, Turkey.Google Scholar
- The World Bank. (2008). The road not traveled: Education reform in the Middle East and North Africa. Washington, DC: Author.Google Scholar
- Turkmen, L., & Bonnstetter, R. (1999). A study of Turkish preservice science teachers’ attitudes toward science and science teaching. Retrieved from ERIC database. (ED444828).Google Scholar
- United Nations Development Programme. (2003). The Arab human development report: Building a knowledge society. New York, NY: UNDP regional Program and Arab Fund for Economic and social Development.Google Scholar
- Van de Vijver, F., & Leung, K. (1997). Methods and data analysis of comparative research. In J. W. Berry, Y. H. Poortinga, & J. Pandey (Eds.), Handbook of cross-cultural psychology: Theory and method (Vol. 1, 2nd ed., pp. 257–300). Needham Heights, MA: Allyn & Bacon.Google Scholar
- Vaske, J. J. (2008). Survey research and analysis: Applications in parks, recreation and human dimensions. State College, PA: Venture Publishing.Google Scholar
- Webb, A. (2014). A cross-cultural analysis of the Test of Science Related Attitudes (Master’s thesis). The Pennsylvania State University, Pennsylvania, EE.UU. Retrieved from https://etda.libraries.psu.edu/paper/22723/24112.
- Widaman, K. F., & Reise, S. P. (1997). Exploring the measurement invariance of psychological instruments: Applications in the substance abuse domain. In K. J. Bryant & M. Windle (Eds.), The science of prevention: Methodological advance from alcohol and substance abuse research (pp. 281–324). Washington, DC: American Psychological Association.CrossRefGoogle Scholar
- Zellman, G. L., Ryan, G. W., Karam, R., Constant, L., Salem, H., Gonzalez, G., . . . Al-Obaidli, K. (2007). Implementation of the K-12 education reform in Qatar’s schools. Santa Monica, CA: RAND Corporation.Google Scholar
- Zellman, G. L., Ryan, G. W., Karam, R., Constant, L., Salem, H., Gonzalez, G., . . . Al-Obaidli, K. (2009). Implementation of the K-12 education reform in Qatar’s schools. Santa Monica, CA: RAND Corporation. Retrieved from http://www.rand.org/pubs/monographs/2009/RAND MG880.pdf.