Abstract
International large-scale comparisons struggle with the difficulty of maintaining standardisation in order to facilitate comparison, and at the same time being relevant and meaningful locally across diverse contexts. This study inquires into the practices involved in accomplishing these apparently contradictory tasks through an empirical case study of the OECD’s adaptation of the Programme for International Student Assessment (PISA) to make it relevant to low-income nations in the form of PISA for Development (PISA-D). Situated knowledges, political decisions, socio-material practices, subjective interpretations, and cultural imaginaries are folded into managing the tension between standardization and contextualisation. Local contexts are profoundly ‘re-contextualized’ to comply with global practice, and conversely, global practices must adjust and compromise to manage standardization from a distance as well as in the here-and-now.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
- 2.
- 3.
- 4.
Star and Lampland (2009).
- 5.
Breakspeare (2012).
- 6.
- 7.
Gorur and Wu (2014).
- 8.
Lockheed and Wagemaker (2013).
- 9.
- 10.
- 11.
Cf. Callon et al. (2009).
- 12.
Cf. Raasch and Sørensen (2014).
- 13.
- 14.
- 15.
- 16.
Gorur (2016a).
- 17.
- 18.
- 19.
Cf. Garfinkel (1967).
- 20.
- 21.
- 22.
OECD (2013).
- 23.
Lezaun and Woolgar (2013).
- 24.
- 25.
Maddox (2017).
- 26.
Latour (1999).
- 27.
- 28.
Law (2009).
- 29.
Bibliography
Bottani, Norberto. 1994. “The OECD International Education Indicators.” Assessment in Education: Principles, Policy and Practice 1 (3): 333–350.
Breakspeare, Simon. 2012. “The Policy Impact of PISA: An Exploration of the Normative Effects of International Benchmarking in School System Performance.” OECD Education Working Paper 71. OECD, Paris.
Callon, Michel, Pierre Lascoumes, and Yannick Barthe. 2009. Acting in an Uncertain World: An Essay on Technical Democracy. Translated by Graham Burchell. Cambridge, MA: MIT Press.
Desrosières, Alain. 1998. The Politics of Large Numbers: A History of Statistical Reasoning. Cambridge, MA and London: Harvard University Press.
Garfinkel, Harold. 1967. Studies in Ethnomethodology. Englewood Cliffs, NJ: Prentice-Hall.
Goffman, Erving. 1957. The Presentation of Self in Everyday Life. London: Penguin.
Gorur, Radhika. 2011. “ANT on the PISA Trail: Following the Statistical Pursuit of Certainty.” Educational Philosophy and Theory 43 (S1): 76–93.
Gorur, Radhika. 2015. “Producing Calculable Worlds: Education at a Glance.” Discourse-Studies in the Cultural Politics of Education 36 (4): 578–595. https://doi.org/10.1080/01596306.2015.974942.
Gorur, Radhika. 2016a. “Seeing Like PISA: A Cautionary Tale About the Performativity of International Assessments.” European Educational Research Journal 15 (5): 598–616.
Gorur, Radhika. 2016b. “The Performative Politics of NAPLAN and My School.” In National Testing in Schools: An Australian Assessment, edited by Greg Thompson, Sam Sellar, and Richard Lingard, 30–43. London: Routledge.
Gorur, Radhika, and Margaret Wu. 2014. “Leaning too Far? PISA, Policy and Australia’s ‘Top Five’ Ambitions.” Discourse: Studies in Cultural Politics of Education 36 (5): 647–664.
Haraway, Donna. 1988. “Situated Knowledge: The Science Question and the Privilege of Partial Perspective.” Feminist Studies 114 (3): 575–599.
Holzkamp, Klaus. 2016. “Conduct of Everyday Life as a Basic Concept of Critical Psychology.” In Psychology and the Conduct of Everyday Life, edited by Ernst Schraube and Charlotte Højholt, 65–99. Hove, East Sussex and New York, NY: Routledge.
Hopkins, David, Dianne Pennock, Jo Ritzen, Elpida Ahtaridou, and Karin Zimmer. 2008. “External Evaluation of the Policy Impact of PISA.” Paris: OECD, November 13, 2018. http://www.oecd.org/officialdocuments/publicdisplaydocumentpdf/?cote=EDU/PISA/GB(2008)35/REV1&docLanguage=En.
Jasanoff, Sheila. 2004. “The Idiom of Coproduction.” In States of Knowledge, edited by Sheila Jasanoff, 1–12. London and New York: Routledge.
Kristensen, Kasper, and Ernst Schraube. 2014. “Conduct of Everyday Life.” In Encyclopedia of Critical Psychology, edited by Thomas Teo, 291–293. New York: Springer.
Latour, Bruno. 1988. The Pasteurization of France. Cambridge, MA: Harvard University Press.
Latour, Bruno. 1999. Pandora’s Hope—Essays on the Reality of Science Studies. Cambridge, MA and London, UK: Harvard University Press.
Lave, Jean, and Etienne Wenger. 1992. Situated Learning: Legitimate Peripheral Participation. Cambridge, UK and New York: Cambridge University Press.
Law, John. 2004. After Methods: Mess in Social Science Research. New York: Routledge.
Law, John. 2009. “Actor Network Theory and Material Semiotics.” In The New Blackwell Companion to Social Theory, edited by Bryan. S. Turner, 141–158, Oxford: Wiley-Blackwell.
Lezaun, Javier, and Steven Woolgar. 2013. “The Wrong Bin Bag: A Turn to Ontology in Science and Technology Studies?” Social Studies of Science 43 (3): 321–340.
Lockheed, Marlaine E., and Hans Wagemaker. 2013. “International Large-Scale Assessments: Thermometers, Whips or Useful Policy Tools?” Research in Comparative and International Education 8 (3): 296–306.
Maddox, Brian. 2014. “Globalising Assessment: An Ethnography of Literacy Assessment, Camels and Fast Food in the Mongolian Gobi.” Comparative Education 50 (4): 474–489.
Maddox, Brian. 2015a. “The Neglected Situation: Assessment Performance and Interaction in Context.” Assessment in Education: Principles, Policy and Practice 22 (4): 427–443.
Maddox, Brian. 2015b. “Inside the Assessment Machine: The Life and Times of a Test Item.” In Literacy as Numbers, edited by Mary Hamilton, Brian Maddox, and Camilla Addey, 129–146. Cambridge: Cambridge University Press.
Maddox, Brian. 2017. “Talk and Gesture as Process Data.” Measurement: Interdisciplinary Research and Perspectives 15 (3–4): 113–127.
Maddox, Brian, and Bruno D. Zumbo. 2017. “Testing Situations Observed: Validation as Jazz.” In Understanding and Investigating Response Processes in Validation Research, edited by Anita Hubley and Bruno D. Zumbo, 179–192. Cham, Switzerland: Springer.
Maddox, Brian, Bruno D. Zumbo, Brenda Tay-Lim, and Demin (Iris) Qu. 2015. “An Anthropologist Among the Psychometricians: Assessment Events, Ethnography and DIF in the Mongolian Gobi.” International Journal of Testing 14 (2): 291–309.
Mol, Annemarie. 2002. The Body Multiple: Ontology in Medical Practice. Durham: Duke University Press.
OECD. 2013. PISA 2012 Assessment and Analytical Framework: Mathematics, Reading, Science, Problem Solving and Financial Literacy. Paris: OECD. https://www.oecd.org/pisa/pisaproducts/PISA%202012%20framework%20e-book_final.pdf.
Porter, Theodore M. 1995. Trust in Numbers: The Pursuit of Objectivity in Science and Public Life. Princeton, NJ: Princeton University Press.
Raasch, Josefine, and Estrid Sørensen. 2014. “Helen Verran: Pionierin der postkolonialen Science & Technology Studies.” In Schlüsselwerke der Science & Technology Studies, edited by Diana Lengersdorf and Matthias Wieser, 257–267. Berlin: Springer.
Rottenburg, Richard, Sally E. Merry, Sung-Joon Park, and Joanna Mugler. 2015. The World of Indicators: The Making of Governmental Knowledge Through Quantification. Cambridge: Cambridge University Press.
Schraube, Ernst, and Charlotte Højholt. 2016. “Introduction: Towards the Psychology of Everyday Living.” In Psychology and the Conduct of Everyday Life, edited by Ernst Schraube and Charlotte Højholt, 1–15. Hove, East Sussex and New York, NY: Routledge.
Scott, James C. 1998. Seeing Like a State: How Certain Schemes to Improve the Human Condition Have Failed. New Haven: Yale University Press.
Sørensen, Estrid. 2008. “Multi-sited Comparison of ‘Doing Regulation’.” Comparative Sociology 7 (3): 311–337.
Sørensen, Estrid. 2009. The Materiality of Learning: Technology and Knowledge in Educational Practice. New York: Cambridge University Press.
Sørensen, Estrid. 2010. “Producing Multi-sited Comparability.” In Thick Comparisons: Reviving an Ethnographic Ambition, edited by Jörg Niewöhner and Thomas Scheffer, 44–77. Leiden: Brill.
Sørensen, Estrid, Alison Marlin, and Jörg Niewöhner. 2017. “From Scholastic to Emic Comparison: Generating Comparability and Handling Difference in Ethnographic Research.” In The SAGE Handbook of Qualitative Data Collection, edited by Uwe Flick. Los Angeles, London, and New Delhi: Sage.
Star, Susan Leigh, and Martha Lampland. 2009. “Reckoning with Standards.” In Standards and Their Stories, edited by Martha Lampland and Susan Leigh Star, 3–31. Ithaca and London: Cornell University Press.
Verran, Helen. 2001. Science and African Logic. Chicago: University of Chicago Press.
Verran, Helen. 2010. “Numbers as an Inventive Frontier in Knowing and Working Australia’s Water Resources.” Anthropological Theory 10 (1–2): 171–178.
Acknowledgements
We would like to thank the OECD and contractor officers as well as the Examination Council Zambia Staff who invited us to look at their work in preparing PISA-D and spent time offering their insights into the progress of the program. Their contributions have been essential for the development of this paper. Gorur’s contributions to this paper were also supported by an Australian Research Council Grant (Project ID: DE170100460).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 The Author(s)
About this chapter
Cite this chapter
Gorur, R., Sørensen, E., Maddox, B. (2019). Standardizing the Context and Contextualizing the Standard: Translating PISA into PISA-D. In: Prutsch, M. (eds) Science, Numbers and Politics. Palgrave Macmillan, Cham. https://doi.org/10.1007/978-3-030-11208-0_14
Download citation
DOI: https://doi.org/10.1007/978-3-030-11208-0_14
Published:
Publisher Name: Palgrave Macmillan, Cham
Print ISBN: 978-3-030-11207-3
Online ISBN: 978-3-030-11208-0
eBook Packages: Political Science and International StudiesPolitical Science and International Studies (R0)