The purpose of this book is to provide a synthesis of thought and practice in research in literacy and science education intended to lead to evidence-based results and generalizations that will serve as a foundation for public policy and informed curriculum, teaching, and assessment practices in education. The Gold Standard of educational research funding in the United States can be viewed as a response to the general dissatisfaction with the utility of educational research; this federal mandate fosters a shift of educational research toward positivist empirical research approaches, such as random controlled trials (RCTs). There is an expectation of greater generalization and policy relevance as Gold Standard research is conducted and reported. It should be noted this dissatisfaction is not confined to the United States. An international response to this dissatisfaction with educational research—systematic reviews of educational research such as the Campbell Collaboration (Campbell Collaboration, n.d.), the What Works Clearinghouse of the US Office of Education (US Institute of Education Sciences, n.d.), and the Evidence for Policy and Practice Information and Co-ordinating Centre (EPPI-Centre) in the United Kingdom (EPPI-Centre, n.d.)— has also identified the general dearth of rigorous empirical research that can support meaningful generalization of research findings.
The deliberations about qualitative—quantitative approaches to educational research over the last 25 years have established parallels for quantitative data considerations (reliability, validity, significance, objectivity) that consider dependability, credibility, believability, and confirmability of information (Howe & Eisenhart, 1990; Husen, 1988; Lather, 1992; Phillips, 2005; Pring, 2000). The enactment of these considerations has produced a diverse collection of qualitative, quantitative, and mixed-methods research in literacy and science education. Inspection of these research studies during the 2nd Island Conference revealed concerns about the clarity of constructs involved in the research, measurements of these constructs, interpretative frameworks and scoring rubrics, data scales (nominal, ordinal, internal, ratio), and statistical analyses of these data. Procedural rigor, clarity, mining and secondary analysis of existing databases and information resources, generalization of research results, research ethics, and mixed-methods and other innovative approaches were identified as issues to enhance research quality (see Yore & Boscolo, Chap. 2).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Anderson, J. O., Lin, H.-L., Treagust, D. F., Ross, S. P., & Yore, L. D. (2007). Using large-scale assessment datasets for research in science and mathematics education: Programme for International Student Assessment (PISA). International Journal of Science & Mathematics Education, 5(4), 591–614.
Anderson, J. O., Monseur, C., & Cartwright, F. (2006, April). Procedures and issues associated with the data and analysis of PISA 2003 thematic research. Paper presented at the annual meeting of the American Educational Research Association, San Francisco, CA.
Beaton, A. E., Postlewaite, T. N., Ross, K. N., Spearritt, D., & Worl, R. M. (1999). The benefits and limitations of international educational achievements studies. Retrieved April 22, 2008, from http://unesdoc.unesco.org/images/0011/001176/117629e.pdf
Brooks-Gunn, J., Phelps, E., & Elder, G. H. (1991). Studying lives through time: Secondary data analyses in developmental psychology. Developmental Psychology, 27(6), 899–910.
Byrne, B. M. (2001). Structural equation modeling with AMOS: Basic concepts, applications and programming. Mahwah, NJ: Lawrence Erlbaum.
Campbell Collaboration (n.d.). Homepage. Retrieved October 3, 2007, from http://www.camp-bellcollaboration.org
Dohn, N. B. (2007). Knowledge and skills for PISA — Assessing the assessment. Journal of Philosophy of Education, 41(1), 1–16.
Evidence for Policy and Practice Information and Co-ordinating Centre. (n.d.). Homepage. Retrieved June 20, 2008, from http://eppi.ioe.ac.uk/cms/
George, R., & Kaplan, D. (1998). A structural model of parent and teacher influences on science attitudes of eighth graders: Evidence from NELS: 88. Science Education, 82(1), 93–109.
Goh, M. (2006). A multilevel analysis of mathematics literacy: The effects of intrinsic motivation, teacher support, and student-teacher relations. Unpublished master's thesis, University of Victoria, Victoria, British Columbia, Canada.
Goldstein, H. (2004). International comparisons of student attainment: Some issues arising from the PISA study. Assessment in Education: Principles, Policy & Practice, 11(3), 319–330.
Gu, Z. (2006). Students' beliefs about themselves, learning environment at school, and achievement. Unpublished master's thesis, University of Victoria, Victoria, British Columbia, Canada.
Hakim, C. (1982). Secondary analysis in social research. Boston: Allen & Unwin.
Hofferth, S. L. (2005). Secondary data analysis in family research. Journal of Marriage & Family, 67(4), 891–907.
Howe, K. R., & Eisenhart, M. (1990). Standards for qualitative (and quantitative) research: A prolegomenon. Educational Researcher, 19(4), 2–9.
Hsu, J. C. (2007). Comparing the relationships between mathematics achievement and student characteristics in Canada and Hong Kong through HLM. Unpublished master's thesis, University of Victoria, Victoria, British Columbia, Canada.
Husen, T. (1988). Research paradigms in education. In J. P. Keeves (Ed.), Educational research, methodology and measurement: An international handbook (pp. 17–20). Toronto, Ontario, Canada: Pergamon Press.
Hyman, H. (1972). Secondary analysis of sample surveys: Principles, procedures, and potentialities. Toronto, Ontario, Canada: Wiley & Sons.
Kennedy, M. M. (1999). Infusing educational decision making with research. In G. J. Cizek (Ed.), Handbook of educational policy (pp. 54–80). San Diego, CA: Academic Press.
Kiecolt, K. J., & Nathan, L. E. (1985). Secondary analysis of survey data. Newbury Park, CA: Sage.
Kline, R. B. (1998). Principles and practice of structural equation modeling. New York: Guilford.
Lather, P. (1992). Critical frames in educational research: Feminist and poststructural perspectives. Theory into Practice, 31(2), 87–99.
Lindblom, C. E. (1968). The policy-making process. Englewood Cliffs, NJ: Prentice Hall.
Lindblom, C. E. (1992). Inquiry and change: The troubled attempt to understand and shape society. New Haven, CT & New York: Yale University Press & Russell Sage Foundation.
Luke, D. A. (2004). Multilevel modeling. Thousand Oaks, CA: Sage.
Marsh, H. W., & Yeung, A. S. (1998). Longitudinal structural equation models of academic self-concept and achievement: Gender differences in the development of math and English constructs. American Educational Research Journal, 35(4), 705–738.
McQueen, J., & Mendelovits, J. (2003). PISA reading: Cultural equivalence in a cross-cultural study. Language Testing, 20(2), 208–224.
Norris, S. P., & Phillips, L. M. (2003). How literacy in its fundamental sense is central to scientific literacy. Science Education, 87(2), 224–240.
Organisation for Economic Co-operation and Development. (2001). Knowledge and skills for life: First report from the OECD programme for international student assessment. Paris: Author.
Organisation for Economic Co-operation and Development. (2002). PISA 2000 technical report. Paris: Author.
Organisation for Economic Co-operation and Development. (2003a). The PISA 2003 assessment framework — mathematics, reading, science and problem solving: Knowledge and skills. Paris: Author.
Organisation for Economic Co-operation and Development. (2003b). PISA 2003 technical report. Paris: Author.
Organisation for Economic Co-operation and Development. (2006). Assessing scientific, reading and mathematical literacy: A framework for PISA 2006. Paris: Author.
Organisation for Economic Co-operation and Development. (2007). What PISA assesses. Paris: Author.
Organisation for Economic Co-operation and Development. (n.d.). PISA Homepage. Retrieved June 30, 2008, from http://www.pisa.oecd.org/pages/0,2987,en_32252351_32235731_1_1_1_ 1_1,00.html
Phillips, D. C. (2005). The contested nature of empirical educational research (and why philosophy of education offers little help). Journal of Philosophy of Education, 39(4), 577–597.
Prais, S. J. (2003). Cautions on OECD's recent educational survey (PISA). Oxford Review of Education, 29(2), 139–163.
Pring, R. (2000). The ‘false dualism’ of educational research. Journal of Philosophy of Education, 34(2), 247–260.
Ram, A. (2007). A multilevel analysis of mathematics literacy in Canada and Japan: The effects of sex differences, teacher support, and the school learning environment. Unpublished master's thesis, University of Victoria, Victoria, British Columbia, Canada.
Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models: Applications and data analysis (2nd edn.). Thousand Oaks, CA: Sage.
Reynolds, A. J., & Walberg, H. J. (1991). A structural model of science achievement. Journal of Educational Psychology, 83(1), 97–107.
Rogers, W. T., Anderson, J. O., Klinger, D. A., & Dawber, T. (2006). Pitfalls and potential pitfalls of secondary data analysis of the Council of Ministers of Education, Canada, national assessment. Canadian Journal of Education, 29(3), 757–770.
Ross, S. P. (2008). Motivation correlates of academic achievement: Exploring how motivation influences academic achievement in the PISA 2003 dataset. Unpublished doctoral dissertation, University of Victoria, Victoria, British Columbia, Canada.
Statistics Canada. (n.d.). National longitudinal survey of children and youth for ages 16–17. Retrieved April 22, 2008, from http://www.statcan.ca/english/kits/microdata/microdata.htm
TIMSS & PIRLS International Study Center. (n.d.). Homepage. Retrieved July 11, 2008, from http://timss.bc.edu/
Turner, R., & Adams, R. J. (2007). The Programme for International Student Assessment: An overview. Journal of Applied Measurement, 8(3), 237–248.
United States Institute of Education Sciences. (n.d.). What Works Clearinghouse: Homepage. Retrieved June 20, 2008, from http://ies.ed.gov/ncee/wwc/
Yore, L. D., Craig, M. T., & Maguire, T. O. (1998). Index of science reading awareness: An interactive-constructive model, test verification, and grades 4–8 results. Journal of Research in Science Teaching, 35(1), 27–51.
Yore, L. D., Pimm, D., & Tuan, H.-L. (2007). The literacy component of mathematical and scientific literacy. International Journal of Science & Mathematics Education, 5(4), 559–589.
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer Science + Business Media B.V
About this chapter
Cite this chapter
Anderson, J.O., Milford, T., Ross, S.P. (2009). Multilevel Modeling with HLM: Taking a Second Look at PISA. In: Shelley, M.C., Yore, L.D., Hand, B. (eds) Quality Research in Literacy and Science Education. Springer, Dordrecht. https://doi.org/10.1007/978-1-4020-8427-0_13
Download citation
DOI: https://doi.org/10.1007/978-1-4020-8427-0_13
Publisher Name: Springer, Dordrecht
Print ISBN: 978-1-4020-8426-3
Online ISBN: 978-1-4020-8427-0
eBook Packages: Humanities, Social Sciences and LawEducation (R0)