Abstract
This study investigated the differences between two competing regression methods(single-level Means-on-Means regression and Multilevel Modelling) used to produce value-added performance-indicator information for the monitoring of school effectiveness. Data from 24 government secondary schools with a total of 2862 students in 132 Year 8 classes in Western Australia were used. The dependent variable was a Rasch-created linear measure of Year 8 Numeracy. The five independent variables were: (1) a Rasch-created, linear measure of Year 7 Numeracy; (2) gender; (3) ethnic group (Aboriginal and Torres Strait Islander, or non-Aboriginal and Torres Strait Islander status); (4) language background (English or other than English); and the school-level variable (5) school socioeconomic status. The findings of this study suggest that residual-based performance indicators calculated using the Multilevel model are more accurate and fair than those produced using the single-level Means-on-Means regression model, and would enable both schools and teachers to report on accountability and investigate a greater range of school effectiveness issues with more confidence.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Aitkin, M., Longford, N. (1986). Statistical modelling issues in school effectiveness studies. Journal of the Royal Statistical Society. Series A (General), 149(1), 1–43.
Andrich, D., Sheridan, B., & Luo, G. (2005). RUMM: A windows-based item analysis program employing Rasch unidimensional models. Perth, WA: RUMM Laboratory.
Browne, W. J., Subramanian, S. V., Jones, K., & Goldstein, H. (2005). Variance partitioning in multilevel logistic models that exhibit overdispersion. Journal of the Royal Statistical Society: Series A (Statistics in Society), 168(3), 599–613.
Cook, J. (2005). Reporting to systems and schools. Paper presented at Curriculum Corporation Conference, Brisbane, Australia. Retrieved June 20, 2007, from http://cmslive.curriculum.edu.au/verve/_resources/Cook_edited.pdf
Curriculum Corporation. (2007). National literacy and numeracy testing 2008. Curriculum Corporation. Retrieved November 6, 2007, from http://www.curriculum.edu.au/ccsite/nap,19875.html
Department of Employment, Education, Training and Youth Affairs (DEETYA). (1998). Literacy for all: The challenge for Australian schools. Commonwealth literacy programs for Australian schools. Australian Schooling Monograph series, 1/1998. Canberra: AGPS.
Department of Education and Training (DET). (2002). School accountability framework. Perth: Government of Western Australia.
Department of Education and Training (DET). (2007a). National and international perspectives and approaches to school accountability: Executive summary. Perth: Government of Western Australia. Retrieved March 31, 2007, from http://www.det.wa.edu.au/education/accountability/Docs/National%20and%20International%20Perspectives%20and%20Approaches%20-%20Exec%20Summary.pdf
Department of Education and Training (DET). (2007b). Western Australian literacy and numeracy assessment (website). Perth: Government of Western Australia. Retrieved August 3, 2007, from http://www.det.wa.gov.au/education/walna/index.html
Department of Education and Training (DET). (2007c). DataClub 2007. Perth: Government of Western Australia. Retrieved June 20, 2008, from http://www.det.wa.edu.au/education/accountability/Docs/DataClub.ppt
Department of Education and Training (DET). (2007d). Guidelines for analysis of WALNA:Years 3, 5 and 7. Perth: Government of Western Australia. Retrieved June 20, 2008, from http://www.det.wa.edu.au/education/mse/pdfs/walna.pdf
Fitz-Gibbon, C. T. (1996). Monitoring eduction: Indicators quality and effectiveness. London: Continuum.
Goldstein, H. (1997). Methods in school effectiveness research. School Effectiveness and School Improvement, 8(4), 369–395.
Goldstein, H. (2001). Using pupil performance data for judging schools and teachers: Scope and limitations. British Education Research Journal, 27(4), 433–442.
Hattie, J. A. C. (2003). Teachers make a difference: What is the research evidence? Keynote paper presented at the ACER Annual Conference, Melbourne, Australia. Retrieved April 24, 2008, from http://www.acer.edu.au/documents/RC2003_Hattie_TeachersMakeADifference.pdf
Hill, P. W., & Rowe, K. J. (1996). Multilevel modelling in school effectiveness research. School Effectiveness and School Improvement, 7(1), 1–34.
Hill, P. W., & Rowe, K. J. (1998). Modelling student progress in studies of educational effectiveness. School Effectiveness and School Improvement, 9(3), 310–333.
Louden, W., Rohl, M., Barratt-Pugh, C., Brown, C., Cairney, T., Elderfield, J., et al. (2005). In teachers’ hands: Effective literacy teaching practices in the early years of schooling. Final Report for Department of Education Science and Training. Perth: Edith Cowan University.
Louden, W., Rohl, M., & Hopkins, S. (2008). Teaching for growth: Effective teaching of literacy and numeracy. Final Report for the Department of Education and Training, Western Australia. Perth: The University of Western Australia.
Louden, W., & Wildy, H. (2001). Developing schools’ capacity to make performance judgements. Final Report. Report prepared for Australian Commonwealth Department of Education, Employment, Training and Youth Affairs (DETYA). Perth, Western Australia: Edith Cowan University.
Ministerial Council on Education, Employment, Training and Youth Affairs (MCEETYA). (1995). National report on schooling in Australia 1995. Carlton: Curriculum Corporation.
Rasch, G. (1960/1980). Probabilistic models for some intelligence and achievement tests. Copenhagen: Danish Institute for Educational Research. (Expanded edition, 1980. Chicago: University of Chicago Press)
Raudenbush, S. W., & Willms, J. D. (1995). The estimation of school effects. Journal of Educational and Behavioural Statistics, 20(4), 307–335.
Robinson, W. S. (1950). Ecological correlations and the behaviour of individuals. American Sociological Review, 15, 351–357.
Rowe, K. (2004, April 19–22). Analysing and reporting performance indicator data: ‘Caress’ the data and user beware! Background paper to invited address presented at the 2004 Public Sector Performance conference, under the auspices of the International Institute for Research, Sydney. Retrieved February 2, 2008, from http://www.acer.edu.au/documents/Rowe-IIR_Conf_2004_Paper.pdf
Rowe, K. (2005). Practical multilevel analysis with MlwiN & LISREL: An integrated course (4th Rev. ed.). Camberwell: Australian Council for Educational Research.
Strand, S. (1998). A ‘value added’ analysis of the 1996 primary school performance tables. Educational Research, 40(2), 123–137.
Thorndike, E. L. (1939). On the fallacy of imputing correlations found for groups to the individuals and in smaller groups composing them. American Journal of Psychology, 52, 122–124.
Waugh, R. F. (2006). Rasch measurement. In N. J. Salkind (Ed.), Encyclopedia of measurement and statistics (Vol. 3., pp. 820–825). Thousand Oaks, CA: Sage Publications.
Woodhouse, G., & Goldstein, H. (1988). Educational performance indicators and LEA tables. Oxford Review of Education, 14(3), 301–320.
Wright, B. D. (1999). Fundamental measurement. In S. E. Embretson & S. C. Hersberger (Ed.), The new rules of measurement (pp. 65–104). Mahwah, NJ: Lawrence Erlbaum Associates, Publishers.
Zhou, X.-H., Perkins, A. J., & Hui, S. L. (1999). Comparisons of software packages for generalized linear multilevel models. The American Statistician, 53(3), 282–290.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Sense Publishers
About this chapter
Cite this chapter
Elderfield, J.M., Waugh, R.F. (2011). High Stakes State-Wide Testing of the Learning Environment Outcomes. In: Cavanagh, R.F., Waugh, R.F. (eds) Applications of Rasch Measurement in Learning Environments Research. Advances in Learning Environments Research, vol 2. SensePublishers, Rotterdam. https://doi.org/10.1007/978-94-6091-493-5_2
Download citation
DOI: https://doi.org/10.1007/978-94-6091-493-5_2
Publisher Name: SensePublishers, Rotterdam
Online ISBN: 978-94-6091-493-5
eBook Packages: Humanities, Social Sciences and LawEducation (R0)
