Advertisement

International Large-Scale Assessments and Education System Reform

On the Power of Numbers
Living reference work entry
  • 90 Downloads
Part of the Springer International Handbooks of Education book series (SIHE)

Abstract

International large-scale assessments, such as PISA and TIMSS, have had a profound effect on educational policy and school accountability. In an open letter to the OECD in 2014, a large number of academics expressed their concerns over the continuous cycle of global testing and how it negatively affects student’s well-being and impoverishes classrooms, as it inevitably involves more and longer batteries of multiple-choice testing, more scripted “vendor”-made lessons, and less autonomy for teachers. Others have pointed to the ways in which international large-scale assessments have opened up education systems and built country’s assessment capacities. This chapter reviews the ways in which international standardized assessments permeate national policy and school-level decision-making, drawing on the sociology of numbers to understand the appeal of standardization and benchmarking. The OECD initiative “PISA for Schools” is described and how it potentially extends the influence of international assessments on the classroom. An alternative project within the OECD (the “Strategic Education Governance” project) is presented and how the approach of case studies and learning seminars allows countries to learn more holistically about how to hold schools accountable and improve education. Openness, situational awareness, and a consideration for the complexity of successful reform are presented as necessary “habits of mind” to successfully improve learning outcomes. The final conclusion reflects on the limitations and potential usage of international large-scale assessments for effective education reform.

Keywords

Accountability Commensuration Standardization Educational governance 

References

  1. Addey, C., & Sellar, S. (2019). Rationales for (non) participation in international large-scale learning assessments. UNESCO Working paper ED-2019/WP/2.Google Scholar
  2. Auld, E., & Morris, P. (2014). Comparative education, the ‘New Paradigm’ and policy borrowing: constructing knowledge for educational reform. Comparative Education, 50(2), 129–155.  https://doi.org/10.1080/03050068.2013.826497CrossRefGoogle Scholar
  3. Breakspear, S. (2014, November). How does PISA shape education policy making? Why how we measure learning determines what counts in education. In Centre for Strategic Education Seminar Series Paper (Vol. 40).Google Scholar
  4. Burns, T., & Köster, F. (2016). Governing education in a complex world. Educational Research and Innovation. OECD Publishing.Google Scholar
  5. Burns, B., Köster, F., & Fuster, M. (2016). Educational research and innovation education governance in action lessons from case studies: Lessons from case studies. OECD Publishing.CrossRefGoogle Scholar
  6. Carvalho, L. M., & Costa, E. (2015). Seeing education with one’s own eyes and through PISA lenses: Considerations of the reception of PISA in European countries. Discourse: Studies in the Cultural Politics of Education, 36(5), 638–646.  https://doi.org/10.1080/01596306.2013.871449CrossRefGoogle Scholar
  7. Chabbot, C., & Elliott, E. (Eds.). (2003). Understanding others, educating ourselves: Getting more from international comparative studies in education. National Academies Press.Google Scholar
  8. Desrosières, A. (1998). The politics of large numbers: A history of statistical reasoning. Harvard University Press.Google Scholar
  9. Diaz-Bone, R., & Didier, E. (2016). The sociology of quantification – Perspectives on an emerging field in the social sciences. Historical Social Research, 41(2), 7–26.  https://doi.org/10.12759/hsr.41.2016.2.7-26CrossRefGoogle Scholar
  10. Espeland, W., & Sauder, M. (2007). Rankings and reactivity: How public measures recreate social worlds. American Journal of Sociology, 113(1), 1–40.  https://doi.org/10.1086/517897CrossRefGoogle Scholar
  11. Espeland, W., & Stevens, M. L. (1998). Commensuration as a social process. Annual Review of Sociology, 24, 313–343.CrossRefGoogle Scholar
  12. Espeland, W. N., & Stevens, M. L. (2008). A sociology of quantification. European Journal of Sociology/Archives Européennes de Sociologie, 49(3), 401–436.  https://doi.org/10.1017/S0003975609000150CrossRefGoogle Scholar
  13. Ferrer, G., & Fiszbein, A. (2015). What has happened with learning assessment systems in Latin America? In Lessons from the last decade of experience. The World Bank.Google Scholar
  14. Frey, B. S., Homberg, F., & Osterloh, M. (2013). Organizational control systems and pay-for-performance in the public service. Organization Studies, 34(7), 949–972.  https://doi.org/10.1177/0170840613483655CrossRefGoogle Scholar
  15. Gregory, A. J. (2007). Target setting, lean systems and viable systems: a systems perspective on control and performance measurement. Journal of the Operational Research Society, 58(11), 1503–1517.  https://doi.org/10.1057/palgrave.jors.2602319CrossRefGoogle Scholar
  16. Grek, S. (2009). Governing by numbers: the PISA ‘effect’ in Europe. Journal of Education Policy, 24(1), 23–37.  https://doi.org/10.1080/02680930802412669CrossRefGoogle Scholar
  17. Gruber, K. H. (2006, May). The German ‘PISA-Shock’: some aspects of the extraordinary impact of the OECD’s PISA study on the German education system. In Cross-national attraction in education accounts from England and Germany. Symposium Books.Google Scholar
  18. Hegarty, S., & Finlay, S. (2020). Publications and dissemination. In Reliability and validity of international large-scale assessment (pp. 221–230). Springer publishers.CrossRefGoogle Scholar
  19. Hopfenbeck, T., Lenkeit, J., El Masri, Y., Cantrell, K., Ryan, J., & Baird, J. (2017). Lessons learned from PISA: A systematic review of Peer Reviewed Articles on the Programme for International Student Assessment. Scandinavian Journal of Educational Research.  https://doi.org/10.1080/00313831.2016.1258726
  20. Klieme, E. (2020). Policies and practices of assessment: A showcase for the use (and misuse) of international large scale assessments in educational effectiveness research. In International perspectives in educational effectiveness research (pp. 147–181). Springer.Google Scholar
  21. Lewis, S. (2016). Governing schooling through ‘what works’: the OECD’s PISA for Schools. Journal of Education Policy.  https://doi.org/10.1080/02680939.2016.1252855
  22. Lewis, S. (2017). Communities of practice and PISA for Schools: Comparative learning or a mode of educational governance? Education Policy Analysis Archives, 25(92).  https://doi.org/10.14507/epaa.25.2901
  23. Lockheed, M., Prokic-Bruer, T., & Shadrova, A. (2015). The experience of middle-income countries participating in PISA 2000–2015 (PISA series). The World Bank & OECD Publishing.  https://doi.org/10.1787/9789264246195-enCrossRefGoogle Scholar
  24. Meyer, H. D., & Zahedi, K. (2014). An Open Letter: To Andreas Schleicher, OECD, Paris. Mitteilungen der Gesellschaft für Didaktik der Mathematik, 40(97), 31–36.Google Scholar
  25. Morgan, C., & Volante, L. (2016). A review of the Organisation for Economic Cooperation and Development’s international education surveys: Governance, human capital discourses, and policy debates. Policy Futures in Education, 14(6), 775–792.  https://doi.org/10.1177/1478210316652024CrossRefGoogle Scholar
  26. OECD. (2012). How your school compares internationally: OECD test for schools (Based on PISA) Pilot Trial [US Version]. OECD. http://www.oecd.org/pisa/aboutpisa/pisa-based-test-for-schools-assessment.htm
  27. Pizmony-Levy, O., Harvey, J., Schmidt, W. H., Noonan, R., Engel, L., Feuer, M. J., & Chatterji, M. (2014). On the merits of, and myths about, international assessments. Quality Assurance in Education.  https://doi.org/10.1108/QAE-07-2014-0035
  28. Rogers, K. H., Luton, R., Biggs, H., Biggs, R., Blignaut, S., Choles, A. G., Palmer, C. G., & Tangwe, P. (2013). Fostering complexity thinking in action research for change in social–ecological systems. Ecology and Society, 18(2), 31.  https://doi.org/10.5751/ES-05330-180231CrossRefGoogle Scholar
  29. Rutkowski, D. J. (2007). Converging us softly: how intergovernmental organizations promote neoliberal educational policy. Critical Studies in Education, 48(2), 229–247.  https://doi.org/10.1080/17508480701494259CrossRefGoogle Scholar
  30. Rutkowski, L., & Rutkowski, D. (2016). A call for a more measured approach to reporting and interpreting PISA results. Educational Researcher, 45(4), 252–257.  https://doi.org/10.3102/0013189X16649961CrossRefGoogle Scholar
  31. Rutkowski, D., Thompson, G., & Rutkowski, L. (2020). Understanding the policy influence of international large-scale assessments in education. In Reliability and validity of international large-scale assessment, 261.  https://doi.org/10.1007/978-3-030-53081-5_15
  32. Sauder, M., & Lancaster, R. (2006). Do rankings matter? The effects of US News & World Report rankings on the admissions process of law schools. Law & Society Review, 40(1), 105–134.  https://doi.org/10.1111/j.1540-5893.2006.00261.xCrossRefGoogle Scholar
  33. Sellar, S., & Lingard, B. (2018). International large-scale assessments, affective worlds and policy impacts in education. International Journal of Qualitative Studies in Education, 31(5), 367–381Google Scholar
  34. Shewbridge, C., & Köster, F. (2019). Strategic education governance; policy toolkit, design. OECD.Google Scholar
  35. Sjøberg, S. (2015). PISA and global educational governance-a critique of the project, its uses and implications. Eurasia Journal of Mathematics, Science & Technology Education, 11(1).  https://doi.org/10.12973/eurasia.2015.1310a
  36. Van Petegem, P., & Vanhoof, J. (2004). Feedback over schoolprestatie-indicatoren als strategisch instrument voor schoolontwikkeling? Lessen uit twee Vlaamse cases [Feedback of performance indicators as a strategic instrument for school improvement? Lessons from two Flemish cases]. Pedagogische Studiën, 81(5), 338–353.Google Scholar
  37. Verger, A., & Parcerisa, L. (2017). Accountability and education in the post-2015 scenario: International trends, enactment dynamics and socio-educational effects. Paper commissioned for the 2017/8 Global Education Monitoring Report, Accountability in education: Meeting our commitments. Paris: UNESCO. ED/GEMR/MRT/2017/P1/1/REV.Google Scholar
  38. Verger, A., Fontdevila, C., & Parcerisa, L. (2019). Reforming governance through policy instruments: how and to what extent standards, tests and accountability in education spread worldwide. Discourse: Studies in the Cultural Politics of Education.  https://doi.org/10.1080/01596306.2019.1569882
  39. Wagemaker, H. (2020). Introduction to reliability and validity of international large-scale assessment. In Reliability and validity of international large-scale assessment (pp. 1–5). Springer publishers.CrossRefGoogle Scholar
  40. Waldow, F. (2009). What PISA did and did not do: Germany after the ‘PISA-shock’. European Educational Research Journal, 8(3), 476–483.  https://doi.org/10.2304/eerj.2009.8.3.476CrossRefGoogle Scholar
  41. Wiseman, A. (2013). Policy responses to PISA in comparative perspective. In H. D. Meyer & A. Benavot (Eds.), PISA, power, and policy. The emergence of global educational governance. Symposium Books.Google Scholar

Authors and Affiliations

  1. 1.Vrije Universiteit Amsterdam FGB – Room MF D455 Van der Boechorststraat 7 1081 BTAmsterdamThe Netherlands
  2. 2.University College London, Institute of EducationLondonUK

Section editors and affiliations

  • Sigrid Blömeke
    • 1
  1. 1.University of OsloOsloNorway

Personalised recommendations