A Comparativistic Narrative of Expertise: International Large-Scale Assessments as the Encyclopaedia of Educational Knowledge
- 1.5k Downloads
In this chapter, Lyotard’s notion serves as an intellectual framing of how a specific reasoning (cf. Hacking, Studies in History and Philosophy of Science 23(1):1–20, 1992a) develops within the educational sciences that eventually leads to the construction of international large-scale assessments (ILSA) and how this reasoning gains legitimacy within both science and policy embedded in a larger societal frame of meritocracy. To this end, this chapter historicizes some of the historical trajectories facilitating the construction of the first truly comparative assessment based on a positivistic inspired aggregation of numbered data. This first IEA (the International Association for the Evaluation of Educational Achievement) study was eventually followed by a multitude of different ILSA. In this, special attention is given to five important trajectories necessary for ILSA to occupy their present role in today’s meritocratic system: (1) how the scientific revolution changed and framed epistemological beliefs, (2) how the role of experts and expertism (Popkewitz, Paradigm and ideology in educational research, London and New York, 1984) changed, (3) how the introduction of statistics facilitated new ways of demonstrating the world and ‘reality’, (4) how the long-forgotten work by the French empiricist Marc-Antoine Jullien was used for a longer and legitimate history of a special branch of comparative education and (5) how the governing of matter and minds has changed over time. All these trajectories are important for an understanding of how ILSA became intelligible in a meritocratic context.
In his seminal book The Postmodern Condition: A Report on Knowledge (1984), Lyotard writes that ‘Data banks are the Encyclopaedia of tomorrow. They transcend the capacity of each of their users. They are “nature” for postmodern man’ (Lyotard 1984: 51). Lyotard’s prophetic notion relates to a discussion about how knowledge has changed when state and society switch from modernity to postmodernism. Here he observes an epistemic displacement (cf. Latour 1988) of knowledge, which, instead of being ‘hidden’ and managed by experts, is governed by what he calls ‘perfect information’ (Lyotard 1984: 52), where data is considered as the prime knowledge. In principle, when data becomes the dominant form of knowledge, it also becomes accessible to any expert to the extent there are no longer any ‘scientific secrets’ (Lyotard 1984: 52). The role of experts also changes. Instead of being ‘hidden’ to the common man, visualized data is open to everyone and it means that anyone can become an expert.
In this chapter, Lyotard’s notion serves as an intellectual framing of how a specific reasoning (cf. Hacking 1992a) develops within the educational sciences that eventually leads to the construction of international large-scale assessments (ILSA) and how this reasoning gains legitimacy within both science and policy embedded in a larger societal frame of meritocracy. To this end, this chapter historicizes some of the historical trajectories facilitating the construction of the first truly comparative assessment based on a positivistic inspired aggregation of numbered data. This first IEA (the International Association for the Evaluation of Educational Achievement) study was eventually followed by a multitude of different ILSA. In this, special attention is given to five important trajectories necessary for ILSA to occupy their present role in today’s meritocratic system: (1) how the scientific revolution changed and framed epistemological beliefs, (2) how the role of experts and expertism (Popkewitz 1984) changed, (3) how the introduction of statistics facilitated new ways of demonstrating the world and ‘reality’, (4) how the long-forgotten work by the French empiricist Marc-Antoine Jullien was used for a longer and legitimate history of a special branch of comparative education and (5) how the governing of matter and minds has changed over time. All these trajectories are important for an understanding of how ILSA became intelligible in a meritocratic context.
17.2 Reasoning Embedded in the Frame of Meritocracy
We begin by elaborating on and explaining how meritocracy can be understood. In a critique of how liberal society constructs inequality through ideas of merit, British sociologist Michael Young’s book, The Rise of the Meritocracy, published in 1958, is a well-known example. In fact, the concept had already been used some years before this in an article by Alan Fox (1956) that gave merit a function by discussing institutions and ideologies as reproducing and legitimizing social stratification (Littler 2013). According to Fox, meritocracy is a societal concept in which the talented, energetic and ambitious are favoured as a result of both their talents and the interconnectedness between education, merits and social benefits. It became apparent that in the emerging modern society, differences could no longer be legitimized by referring to birth, rank or economic prerequisites. It was also evident that in the nineteenth century, suspicion of privilege and meritocracy was introduced as a safely elitist form of democracy (Porter 1995) and meant that relationships between the individual and the society had to be rewritten.
In modernity, reasoning about meritocratic selection is normally justified by referring to equal opportunities. This is often interpreted as individuals with the same talents and a desire to make use of them should have the same opportunities in life. The only hierarchy that can be accepted is based on meritocratic ideas aggregated from evaluations of individual performance. Consequently, inequality is only based on who has access to education and social positions based on merit. As such, meritocracy is not blind to inequality, but defines inequality and equality differently. As equality depends on merit, merit can also lead to inequality. In other words, meritocracy is both an ideology and a state-sanctioned technology that promotes the elimination of a traditional heritage-based inequality, but at the same time legitimizes inequalities based on individual performance. In fact, Lemann (1999) criticizes this by stating that American meritocracy is a lie, in that socioeconomic background and ethnicity are still the most dominant predictors of the future of individuals. As can be seen in discussions about meritocracy, Lemann’s observation is universally applicable. Meritocracy is a problematic and complex ideology, though. Michael Young and many others have pointed to some of the problems associated with meritocracy, for example, considering social and cultural heritage in terms of access to merits (Bourdieu 1971). These descriptions are often tied to the many unspoken assumptions and styles of reasoning (Hacking 1992b) on which meritocracy is based, such as the conceptualization of talent/intelligence, the ability to discern what essential knowledge, skills and abilities are and whether they are measurable—preferably by means of standardized and comparable tests.1 Others have criticized meritocratic technology for its inability to maintain the meritocratic ideal, the establishment of new hierarchies and that certain groups are systematically disadvantaged and discriminated (Bell 1972). Thus, here we note a combination of meritocratic reasoning related to categorizations or taxonomies of individuals or groups in criticisms of educational systems at work.
Consequently, rather like a gatekeeper, meritocratic technology affects and regulates the entrance to, passages in and outputs of education and the labour market (cf. Forsberg 2006). Within the education system, this is highlighted by administrative and pedagogical systems designed to assess, evaluate, document and compare students’ achievements. In other words, meritocracy as a technology is a combination of equality and competitive ideals. In this perspective, meritocracy as a just injustice or a just inequality can also be highlighted (cf. Forsberg and Pettersson 2015). Meritocracy is as such the frame within which ILSA are made intelligible and uphold a position. In the following, five different trajectories and displacements are historicized in order to explain how the technology of ILSA has developed historically into a technology in line with the reasoning of meritocracy.
To describe these five displacements and make comparative assessments intelligible, Gaston Bachelard’s notions of epistemological obstacles (obstacle épistémologique) and epistemological breaks (rupture épistémologique) (Bachelard 1938/2002) are used as intellectually organizing principles. The benefits of thinking about the developments in a Bachelardian way is that scientific and societal developments are never regarded as linear. Instead, Bachelard thinks about them as a constant process of obstacles and raptures continually involved in a process of legitimacy. But before historicizing these displacements, we need to explain what ILSA are and how they came about.
17.3 A Brief History of ILSA
Horkheimer and Adorno (1948) argue that civil society tends to make the incommensurable comparable by reducing it to abstract quantities. This strategy has been most visible in the field of international comparative research (e.g. Durkheim 1894/1938; cf. Steiner-Khamsi and Waldow 2012) in the humanities and social science traditions (Cowen and Kazamias 2009), including their long and intense controversies (see Rust et al. 2009). The practice and importance of comparison have been described in different ways. For instance, Nóvoa and Yariv-Mashal (2003) regard comparisons more important than cases and variables in comparative studies. In turn, Schriewer and Martinez (2004) analyse periods in the development of the comparative field emphasizing a reflexive turn. A major issue today in the field of comparative education research is the emphasis on comparisons based on concepts and the problem of comparisons between countries (e.g. Schriewer 2009). One observation is that very few comparative studies within education integrate case and variable studies on a conceptual basis (e.g. Stiegler and Hiebert 1999) and that data from empirical studies in the field are under-analysed (e.g. Lindblad et al. 2015). However, nowadays, international comparative analyses are vital for the development of scientific discourse in education and for counteracting theoretical chauvinism (e.g. Archer 2013).
Comparative educational research has developed rapidly since the late 1950s in terms of research programmes, a number of studies in these programmes and the number of publications (e.g. Forsberg and Pettersson 2015). Comparative educational research has also been examined in a number of research reviews and handbooks (e.g. Rutkowski et al. 2014) and also as an ingredient in other research traditions, such as school effectiveness and improvement research (e.g. Reynolds 2007) or comparative education (e.g. Beech 2009). Conclusively, it can be said that comparative educational research as a scientific branch is based on at least two important premises—comparisons and data. In the late nineteenth century, the production of numbered data was used for new visions of the social and economic world. The new construction of epistemic references for defining ‘reality’ with the aid of data is linked to the creation and management of the development of the self-defined ‘democratic’ state. Numerical data also provide more than an ‘objective way’ of seeing reality, in that it ‘institutes’ reality by creating a ‘common cognitive space’ that can be both observed and described through data (Lussi Borer and Lawn 2013).
After the Second World War, data was considered as the most objective way of understanding ‘reality’ (Lussi Borer and Lawn 2013). The reorganization required a standardized system of accounting. One offshoot was the creation of ILSA of student learning outcomes. This development was guided by the vision that if custom and law define what is educationally permissible within a nation, educational systems beyond national boundaries suggest the educationally possible (Foshay et al. 1962). This vison was used to introduce the first international comparative pilot study in mathematics, which not only described the origins of an emergent field, but also foreshadowed the subsequent growth of comparative assessment studies (Owens 2013). Consequently, ILSA can be seen as creating a practice that shows what is educationally possible.
17.4 IEA: The Mother of All ILSA
The International Association for the Evaluation of Educational Achievement (IEA) was the first organization to be formally established for this kind of activity in the 1950s. The founders viewed the world as a natural educational laboratory, where different school systems experimented with obtaining optimal results in the education of youth. They assumed that if research could obtain evidence from different national education systems, the variability would be sufficient to reveal important relationships that would otherwise escape detection in a single education system (Pettersson 2014b). The purpose was said to determine intellectual functioning using multiple-choice items, test the feasibility of large-scale assessments and be exploratory (Foshay et al. 1962). The first IEA study differed from other contemporary comparative studies in that it sought to introduce an empirical approach into the methodology of comparative education, a field that is said to have initially relied on cultural analysis (Foshay et al. 1962). IEA embarked on the task with great enthusiasm and ran a pilot study (beginning in June 1959 and ending in June 1961) in which the researchers concluded that cross-national comparisons of educational performance could be made with comparable results (Foshay et al. 1962). Such findings were startling at the time, but even more important was the clear sense that a group of researchers from different cultures and educational systems could agree on a common approach to testing and evaluation (Purves 1987). The original aim of studying intellectual functioning was changed to include a more sharply defined curriculum base in the test items. David Walker (1962) contributed to this with the phrase ‘opportunity to learn’, which became one of the important items of study in the following IEA projects, even though Walker’s analysis in the pilot study found that individual ability accounted for more of the explained variance in the successful completion of an item than the teacher’s emphasis in class (Walker 1962).
In 1961, researchers from 12 countries met within the organizational frames of IEA to discuss the pilot study assessments in mathematics, reading comprehension, geography, science and non-verbal ability. The study was considered a success and plans for another study in mathematics took shape. It was agreed at the outset that the project should be a cooperative enterprise. The major purpose of the inquiry was to measure achievements in mathematics and to relate this achievement to relevant factors in the home, school and society. In determining these factors, the investigation had to rely on the findings of previous research. The project, called the First International Mathematic Study (FIMS), was said to be an attempt to assess the efficiency or productivity of different educational systems and practices (Bloom 1969). The final results of FIMS were presented in a publication by Husén (1967). In addition to the main study, various reports were published (e.g. Keeves 1968; Pidgeon 1967; Kuusinen 1967; Hultin 1968). In the study, it became evident that there was a difference between how a subject was actually taught in the classroom and how it was described in the curriculum, and that this was a good predictor of the differences in student performance. FIMS also showed that there was a lack of equity between different groups of students in how they performed. After this study, IEA conducted a variety of studies on different subjects, time spans and periods (for a list of the different assessments conducted by IEA, see Lindblad et al. 2015).
17.5 OECD: The Queen of ILSA
The IEA studies led to many assessments being undertaken in various countries. The Programme for International Student Assessment (PISA) study, a project of the Organization for Economic Cooperation and Development (OECD), was similar to the IEA studies in many respects. Although OECD has primarily been concerned with economic policy, education has become increasingly important due to the fact that over the last 40 years’ education has been reframed to include economic competitiveness in an economic discourse related to human capital and ‘knowledge economy’ (Pettersson 2008). Through statistics, reports and studies, OECD has activated a ‘common sense’ in political decision-making by saying that scientific ‘proofs’ are indisputable (Martens 2007).
Martens (2007) argues that OECD’s greatest impact can be seen in its agenda with indicators and its role in constructing a global policy field of governance by comparison (cf. Grek 2009). Nóvoa and Lord (2002) state that comparisons like this may not be regarded as a method, but can in fact be seen as policy. The policy is driven by an expert discourse that, by means of comparative strategies, tends to impose natural or common-sense answers in national settings (cf. Pettersson 2008). While OECD serves national policymakers well with a comparable discourse in terms of statistics, it also provides them with a global policy lexicon concerning what education is and ought to be (cf. Pettersson 2014a). One way of creating this global policy lexicon is to look at what is measured in PISA. PISA provides comparisons of the competencies of 15-year-olds that are relevant to everyday adult life, rather than simply evaluating curricula-based knowledge (OECD 2001). It is also said that assessments that test curriculum only offer a measure of internal efficiency and cannot reveal how schools prepare students for adult life (OECD 2001). As such, PISA can be seen as a platform for policy construction, mediation and diffusion at the national, international and even global levels (Rizvi and Lingard 2006).
PISA assessments have been conducted several times. In every assessment, students’ knowledge in reading, mathematics and scientific literacy is tested, together with interests and backgrounds. Innovative domains are also assessed, such as collaborative problem-solving, and there are also plans to incorporate what is called global competency from the 2018 assessment onwards. The emphasis on ‘real-life’ circumstances and the capacity to enter the labour market with the relevant skills has been said to shift the focus of PISA away from less explicit educational aims that are complicated to measure (Grek 2009). PISA also easily connects to the idea of the self-governance of active subjects, which expands governance into a system of individual self-regulation (Ball 1998). Even though PISA both is constructed and operates under a clear policy framework that is designed to improve future results, it is not just a testing regime. PISA should also be seen in the light of its ability to improve and attract economic and human capital investments. For policymakers, PISA is therefore a two-sided coin in that it tests outcomes and attracts economic investment. In view of this, PISA can be said to have two functions—economic and educational—in the international policy discourse (Pettersson 2008). As these two aspects are interwoven and strengthen each other, they can hardly be analysed separately.
Through the 1980s and 1990s, ILSA proliferated in both type and design. IEA expanded its surveys from mathematics and science to include reading, pre-primary education, classroom environment, second language acquisition, technology and civics. However, while PISA imitated IEA’s 1959–1961 study with the intention of evaluating student performance close to the end of schooling, the objective of the assessment actually evolved from curriculum-based learning to a new concept of literacy. Literacy was employed by PISA to signify a mastery of broad concepts applicable to life beyond the classroom (OECD 2001).
How can the development of ILSA for investigating educational knowledge be understood? This has been a long and ongoing process as society has changed into a meritocratic system. Some displacements in the reasoning of education have been necessary to change assessments into something more than students’ scores in tests. Below, some of the displacements in the reasoning are elaborated on historically and discussed as raptures that made it possible to stage ILSA as intelligible for education and educational development on a large scale.
17.6 The Scientific Revolution
Has there been a scientific revolution? Revolutions are often understood as rapid occurrences in a short space of time. This is not the case when we talk about a scientific revolution—especially when discussing the evolution of science and how science became a dominant field of knowledge production. Normally when locating the development of modern science in time, we note that it happened somewhere around the sixteenth century and up to the early nineteenth century, which makes it an extremely slow revolution. Therefore, when the French historian Alexandre Koyré (1968) started to discuss the changes as a revolution in the 1930s and 1940s, he did it by saying that it was the most profound revolution achieved by the human mind since Greek antiquity. According to Koyré, the revolution was so profound that for centuries human culture was unable to grasp its bearing or meaning. The importance of scientific development can also be found in the work of the English historian Herbert Butterfield (1965) in the late 1940s, who states that the scientific revolution has outshone everything since the rise of Christianity and reduced both the Renaissance and the Reformation to mere episodes in history. He concludes that the scientific revolution is in fact the origin of modernity and modern mentality. The question of whether it was a revolution or not and whether it changed people’s mentality and society is a matter for debate amongst historians. Instead of acknowledging the development as one long single event, historians instead talk about a diverse array of cultural practices aimed at understanding, explaining and controlling the natural world, each of which had different characteristics and experienced different trajectories until they eventually started to be perceived as one single unit (for a discussion on these matters, see, e.g. Shapin 1996).
But why is the scientific revolution so important for understanding the construction of ILSA? The simple answer to this is because ILSA is science—although this is does not explain why ILSA are constructed as they are. For this, we have to begin by asking ourselves what science is and what the technologies developed within science are that make ILSA intelligible. This involves going back to the time when science was considered heretical: when religion, and to a lesser extent the monarch, was seen as the cultural and moral authority to decide what was true or false. Gradually, science occupied this position and instead of the church and the monarch defining ‘reality’, this was given to science. This started out with scientists like Galileo, Descartes, Huygens and Darwin, who began to question the Aristotelian cosmos sanctioned by the church. They did this by using a specific technology, which can be called a ‘mathematization’ of the study of motions (Shapin 1996). This mathematization of natural science became the method with highest legitimacy for investigating the natural order, which led to a dichotomy between what was considered subjective and objective. Here, objectivism evolved to the highest order in deciding what was true or false. Before this, the decision-making system had been perceived as far too subjective to have authority or legitimacy. Now, the mathematization method conducted by scientists was perceived as more objective and the ultimate authority for deciding falsehood or truth (Shapin 2010).
An early shift in this new way of using mathematization to say something about ‘reality’ is Marquis de Vauban’s suggestion to the French king, Louis XIV, of an annual census in 1686 for counting his subjects:
Would it not be a great satisfaction to the king to know at a designated moment every year the number of his subjects, in total & by region, with all the resources, wealth and poverty of each place; [the number] of his nobility and ecclesiastics of all kinds, of men of the robe, of Catholics and of those of the other religion, all separated according to the place of their residence? […] [Would it not be] a useful and necessary pleasure for him to be able, in his own office, to review in an hour’s time the present and past condition of a great realm of which he is the head, and be able himself to know with certitude in what consists his grandeur, his wealth, and his strengths? (Cited from Scott 1998: 11)
What is shown here is a shift in the reasoning about governing to one in which data collected from the emerging scientific field is used to provide the monarch with important information. Before the scientific development of using numbers to describe the natural state, knowledge about the state had to be collected subjectively by the king or his administrators. In this new way of reasoning, numbers in terms of resources could be collected and calculated for different purposes. Scott (1998) states that certain forms of knowledge and control require a narrowing of vision and that censuses provide this. One advantage of a narrow view is that it brings certain aspects into sharp focus and helps to make sense of an otherwise complex and unwieldy reality. A further advantage is that it highlights some aspects of reality and exaggerates their legitimacy, which in turn makes them even more susceptible to careful measurement and calculation. Consequently, what we measure is coupled with how we interpret ‘true’ reality. As such, what we choose to measure tends to constitute reality. However, quantifications and measurements of reality cannot just be seen as a social construction of reality, but can also be regarded as a specific technology of governing from a distance (cf. Porter 1995). A technology of numbers and quantifications minimized the need for intimate knowledge and personal trust, which had previously been regarded as necessary (Porter 1995). One reason for the development of governing at a distance is that quantifications are well suited for a communication that goes beyond the boundaries of the local community in that numbers are multilingual and can easily be adapted to different contexts.
The introduction of mathematization into the field of governing meant that the scientific revolution could also be discussed as a revolution in governing. The technologies that were developed and used in science also now interacted with society. Mathematization changed society in profound ways, for example, by making the connection between science and society institutionally stronger and changing people’s attitudes. Using science and numbers to govern became common sense. As such, ‘facts’ and ‘truths’ had to be based on science and science was usually described as resting on a strong objective foundation of quantifications and measurements. Overall, the experts who equated knowledge with authority and legitimacy changed with the scientific revolution—from the church and the monarch to scientists using a specific technology based on numbers.
17.7 Experts and Expertism
In order to fully understand the arguments and the ‘facts’ about ILSA, we need to acknowledge the changing roles of experts and how a specific discourse of expertism has developed in modernity. The term expert originates from the Latin expertus, the past participle of experiri—to try. If experts are defined in accordance with this archaic meaning, they will be seen as those who try to ‘convince’ others about alternative attitudes, ‘realities’ or common sense. As such, expertism can be understood as a social practice between different rival actors. Experts can thus be interpreted as agents of change (Popkewitz 1984) who identify the correct methods or procedures for getting other individuals to accept the call for change.
The role of experts is tightly interwoven with the rise of the scientific revolution as discussed previously. Before science developed into a hegemonic reasoning of how to gain and perceive knowledge, this position was occupied (at least in the West) by a sacred theory of knowledge in which God and his interpreters—the Church—had the exclusive right to define the only true knowledge. When nature no longer counts as God’s Truth, but instead has to be mathematized, the act of knowing is no longer seen as the imitation of otherworldly divinity. As such, the early development of natural science, with its mathematization of nature, led to a desacralized knowledge. The role of God and the Church as experts collapsed (Shapin and Lawrence 1998) and instead scientists became the experts. With this profound change in who had the legitimacy to claim ‘facts’ and ‘truths’, the way opened for the construction of scientific methods and theories for change.
This changing perception of who the expert is can be regarded as one of the most important transformations into modernity. Before the scientific revolution, epistemological obstacles and raptures were managed by the Church in a discourse of divinity. After it, these obstacles and raptures were dealt with inside the field of science. Mathematization thus became the new divinity, this time within science, for making sense of nature. Consequently, mathematization had to be explained and developed for better accuracy and legitimacy. One of the responses to this was the science of statistics.
17.8 Understanding the World by Numbers
In order to understand the relevance of statistics for making ILSA intelligible, we need to be aware of a long-lasting philosophical controversy about the use of statistics. This controversy can be placed into two different categories depending on whether they concern measurement or the object itself. If the reality of the thing being measured is considered to be independent of the measuring process, the discussion hinges on the reliability of the measurement made. However, if the object to be measured is seen as a convention, discussions about the existence and definition of the object become necessary (Desrosières 1998). The tension between these two different points of view—one describing the objects to measure as real and the other describing objects as conventions—are important to bear in mind when talking about the development of ILSA. But it is not enough to say that ILSA considers the knowledge that is measured as real, even though this may be the case. Instead, we have to consider the interpretations of this knowledge as conventions (this is visible in OECD work on indicators) about what good education is or ought to be. As such, ILSA acknowledges an intercommunion between the objects measured as real and as conventions (indicators of something greater than just ‘knowledge’ or students’ performances). ILSA also makes statements about education at large and even future societal and economic developments.
The notions about statistics and ILSA are based on two different observations. The first is Durkheim’s (1894/1938) description of the central role of social science (in his case sociology) and the need to consider social facts as things. This can be read in two different ways: that ‘social facts are things’ or that ‘social facts must be treated as if they were things’ (cf. Desrosières 1998). Treating social facts as if they were things requires a specific scientific language—the language of statistics. The other observation is that statistical tools facilitate the discovery or creation of entities that support descriptions of the world and how we act in it. These objects are simultaneously treated both as real things and as constructed, but when they continue to be assembled and circulated they are cut off from their origins and treated as though they really are things (Desrosières 1998).
Against this background, the development of modern statistics from science and administrative practice makes sense when we consider science and administration as two different trajectories that eventually merged. In the administrative context, objects were mostly treated as things that could be measured for governing and administrative reasons. In the science context, the debate centred on how to treat objects methodologically and construct measurable objects. A specific discussion emerged about the differences between prescriptive and descriptive views of statistics. This discussion is especially relevant in the development of probability calculus, with its categorization of subjective and objective probability. Hacking (1975) characterizes these differences as either epistemic or frequentist probability. In the epistemic perspective, probability is characterized as a degree of belief. In a situation in which the future seems uncertain and our knowledge incomplete, probabilities provide us with rules of behaviour when information is scarce. On the other hand, the frequentist view emphasizes diversity and risk as part of nature and not as part of incomplete knowledge. According to this view, diversity and risk are external to mankind and, as such, part of the essence of things. Consequently, it falls to science to describe the frequencies observed. In both cases, statistics becomes a way of dealing with uncertainty. Here, the relationship between statistics and administrative practices becomes evident. The history of statistics is closely connected to the construction of the state (cf. Scott 1998) in which general forms are established—categories of equivalence—that change the singularities of individual situations into whole classes in a process of encoding. Defining classes of equivalence and encoding became central to the performance of statistical work (Desrosières 1998). The most important aspect of this process is that disparate things can be held together to generate a different order (Thévenot 1986).
In order to understand ILSA, we have to acknowledge the ambiguity of statistics as a history of probabilistic thinking and as an administrative technology for governing. In this, statistics strives towards knowledge, action and descriptive and prescriptive acts. Desrosières (1998) describes the two trajectories of statistics by saying that science and practice are linked, where the task is to objectify and making things that hold together, ‘either because they are predictable or because, if unpredictable, their unpredictability can be mastered to some extent, thanks to the calculation of probability’ (ibid. p. 9). Managing uncertainty is the most important displacement that statistics can provide. When statistics are thought of in this way, ILSA can be interpreted as a way of acquiring knowledge and as leading to action, where the end results can be perceived as both descriptive and prescriptive. In this way, ILSA can be interpreted as a scientific and administrative activity. However, for this to happen, ILSA had to claim legitimacy from a historical tradition within science, namely comparative education.
17.9 The Claim and Construction of History
As indicated earlier, IEA was one of the first organizations to focus on large-scale assessments of students’ achievements. The organization was created to conduct comparative educational studies in the late 1950s and staged its first assessment in the early 1960s (Pettersson 2014b). The first IEA study differed from other comparative education studies of the period in that it tried to introduce an empirical, number-based approach into a field dominated by cultural analysis (Foshay et al. 1962). Before the first IEA study was undertaken, education comparisons had been based on humanistic ideals. With the formation of IEA by scientists interested in psychometrics and educational outputs, the social sciences and behavioural science became the ideal on which comparative achievement tests rested (cf. Kazamias and Massialas 1982).
When comparative education is described as a scientific field, it is not clear what its methodological and theoretical roots are. When IEA introduced its first survey, it was made clear that it was a comparative study that challenged some of the epistemological beliefs in the field of comparative education. Instead of claiming heritage to the most common theoretical starting points within humanities, such as Constantin Ushinsky or Wilhelm Dilthey and their emphasis on cultural understanding, or Verstehen for performing comparative education (Epstein 2008a), it placed itself within the social sciences and used statistics to investigate the field. This challenged the fundamental beliefs within comparative education. To gain legitimacy, IEA claims to be the inheritor of a long-lasting tradition in comparative education stemming from the French empiricist Marc-Antoine Jullien, who in fact developed his ideas before Ushinsky and Dilthey (Epstein 2017). Jullien became a legitimacy claim for IEA in saying that its assessments were in fact part of comparative education and also the oldest tradition in this field.
But how was this made possible? In 1935, a newly discovered book written by Jullien in 1817 was donated to the International Bureau of Education in Geneva. The book was read by Pedro Roselló, who worked at the Bureau and who in 1943 published a text that presented Jullien as the father of comparative education (Roselló 1943). What Jullien tried to do in the early nineteenth century was to introduce positivism as the basis for all comparative studies. In this, numbers became objective facts that had to be gathered to verify educational claims. The first IEA study made it possible to connect to a long history of comparative studies based on numbers and to demonstrate the historical legitimacy that was so important in the 1950s and 1960s, especially in an American research context in which positivism gained important ground (cf. Anderson 1961). However, the presentation of Jullien as the founding father of comparative education was contested in the field of comparative education (e.g. García Garrido 1996; Noah and Eckstein 1969) and still today can lead to heated discussions about comparative education’s roots and historical trajectories (e.g. Epstein 2008b). Nevertheless, Jullien’s thinking was used to legitimize a study such as that undertaken by IEA as a science-driven endeavour and served to give it both a history and historical legitimacy. Another way of looking at this problem is to suggest that IEA created something new in the history of comparative education. In short, it focused on an educational output that could be represented in numbers and created hierarchies of students, educational systems and nations based on these numbers, thereby creating a specific positivistic reasoning on education.
17.10 Governing Matter and Minds
Finally, we need to consider one last displacement to make ILSA fully intelligible, namely the changes in how matter and minds are thought to be governed. Foucault (2009) distinguishes between three different modalities in the history of power relations. First, we have the legal system, which defines itself through a normative code of what is considered legal and illegal. Second, the legal system establishes a system of disciplinary devices and techniques for the ordering, correction and modulation of subjects. Third, an apparatus of security is created. All these modalities coexist. Foucault identifies the origin of governmental technique in the Christian pastorate of ‘governing the souls’. This church hegemony started to be questioned with the scientific revolution and new forms of governing matter and minds sought. What came instead was the narrative of objectivity, where science, and especially the mathematization of observations, was considered as the highest order of things.
The language of science became the new Lingua Tertii Imperii (Klemperer 2011), in which ‘state simplifications’ (Scott 1998) determined how to govern with the opportunities provided by science, and especially the technology of mathematization. According to Scott, state simplifications have at least five characteristics. First, state simplifications are observations of aspects of social life that are of official interest. Second, they are mostly delivered in a written format and are often numerical and considered as documentary facts. Third, they are typically in the form of static facts, and fourth often aggregated facts in an impersonal assemblage of individual characteristics. Fifth, and finally, for most purposes, state officials need to group citizens in ways that permit them to make collective assessments. These aggregated facts, which can be presented as averages or distributions, must therefore be standardized facts. Even though the actual circumstances might be unique for individuals, it is the similarities and differences on a standardized scale or continuum that are of interest.
At least three steps are required for the use of these state simplifications. The first is the creation of common units of measurement or coding. The second step is that each item or instance in a category is counted and classified according to the new unit of assessment. The third step is the use of these classifications in various combinations to locate new illuminations of knowledge (Scott 1998). What is recognized in this discussion is the fact that science and state worked together to order society and people’s minds by using classifications that made a mathematization of social matters possible. When ILSA appeared, it was considered as an important joint venture for societal, economic and scientific development. This gave ILSA the legitimacy to categorize individuals for governmental and scientific reasons.
17.11 ILSA as Encyclopaedic Knowledge
As stated earlier, Horkheimer and Adorno (1948) argue that civil society tends to make incommensurables comparable by reducing them to abstract quantities. Lyotard (1984) also maintains that these abstract quantities are—like data—transformed into an encyclopaedia of knowledge. This strategy is, to a large extent, based on a belief in numbers as more objective (Porter 1995). Porter illuminates that strict quantification through measurement, counting and calculation is one of the most credible strategies for perceiving objectivity—a strategy that has enjoyed widespread and growing authority for at least two centuries in, for example, science and the organization of the state. In education, this strategy was discussed in relation to reasoning linking political theories of government with notions of democracy and merit that began to appear in the nineteenth century about numbers providing narratives on equality and social progress. The emergence of merit tied to individual capabilities and qualities replaced manners and gentlemanly conduct as a way of thinking about truth and competency (Sapin 1994). However, reasoning about merit is not unique to modern society, but is embedded in different systems of reason that are not based on notions of individuality, agency and the temporality of progress. Historically, societies made trade-offs between merit, seniority, heritage and divinity’s given orders when organizing the social order (Neves 2000). What the displacements presented earlier highlight is a notion of modernity that gives individuals their own history and the capacity to develop on merit. For instance, French philosophers talk about the need for an equal system of measurement for an equal society (see Popkewitz 2008; see Kett 2013). In this argument, we can begin to see the development of the idea that numbers are independent of human activity, but yet need to be applied in social arenas as procedures for correcting social wrongs and facilitating human equality in the organization of society.
Today, the use of numbers and statistical comparisons are taken for granted as ways of understanding how society grows and how schools respond to the social and political commitments associated with equality as expressed through ideologies of merit. Data from grades, examinations, student performances in national tests and regional and international knowledge assessments are aggregated and are now widely used to determine national results and make comparisons between them, thereby establishing a faith in numbers (Porter 1995) that affects both the reasoning and discussions about education. A way of presenting these results is through school or country rankings, which in turn often lead to discussions about reforms for achieving better performances.
How can the success of ILSA be explained today? To begin with, different aspects have contributed to making ILSA more relevant, most of which are embedded in an ideology of meritocracy. These include the changes that took place due to the scientific revolution, the changing role of experts, the development of statistics as a way to giving knowledge legitimacy, stating legitimacy for ILSA by claiming an old history and a changing discourse on how to govern. All these aspects are important to understand why ILSA was perceived as more or less ‘common sense’ when introduced in the early 1960s. What is constructed in these historical trajectories is a specific narrative that describes how ILSA serves meritocracy as the dominant organizing principle. This narrative is dependent on the legitimacy of comparisons and a mathematization of humans and human actions for making sense of the world.
It is clear that as a representative of a specific and legitimate technology, ILSA is closely connected to state administration and science and is based on the presumption that society, its citizens and knowledge can be quantified by a system of meritocracy. Donna Haraway (1997) claims that science not only has social causes, but also causes society. As such, we could say that ILSA as a phenomenon partly constructs how we perceive knowledge in today’s meritocratic society. We can also learn from Lyotard (1984), who maintains that when society changes, scientific knowledge also changes and can be understood as a kind of discourse. This discourse brings a certain logic, or reasoning, that determines what is accepted as ‘knowledge statements’, who is mandated to state them and why. We are in no position to claim that the knowledge measured by ILSA is the ‘right knowledge, although we can raise questions about comparisons of ILSA knowledge and the mathematization and aggregation of data in the meritocratic system. The frequency and spread of ILSA results in contemporary policy indicates that ILSA is not only a part of how ‘knowledge’ is made and perceived, but also helps to construct how we perceive and understand “society”’.
This chapter describes how a public sphere is constructed when questions about social aspects and knowledge are debated publicly. This is made possible when the role of experts changes from being performed in closed societies by ‘knighted’ representatives who debate and construct knowledge into what is considered as more ‘objective’ formats and aggregate data as statistics. Statistics has made it possible for anyone to draw conclusions and make statements on the basis of the presented data. In fact, in modern times open and accessible data is appreciated as being necessary for democracy and enlightened debates (cf. Desrosières 1998) and as the basis of meritocracy. What is evident in this development is that the construction of a statistical system cannot be separated from the construction of equivalence that guarantees consistency and permanence in the political and scientific sphere, or from a social reasoning about the objects under discussion. As such, statistics create a common reference, or a common language, about objects that are highlighted as important phenomena. Statistics thus create societal and cognitive frames for what can be understood as ‘social’ or as ‘knowledge’. Consequently, knowledge becomes something that can be measured and spoken about. The technology of ILSA is, to a large extent, based on this statistical language, where ‘knowledge’ is perceived as what is measured and calculated in the assessments. Consequently, ILSA illuminates some ‘knowledge’ and downplays others.
These two observations lead to our final conclusion. Encyclopaedia is etymologically derived from a Greek word meaning complete instruction or complete knowledge. This text both argues and criticizes that today ILSA is largely interpreted as the complete instruction or knowledge about education and plays a part in constructing education and ‘the world’. The knowledge derived from ILSA is not the only possible knowledge about education, although it can certainly appear to be if we only acknowledge international and national policies on education. By historicizing some of the trajectories in educational history, this chapter has shown how this was made possible and intelligible by specific displacements and raptures in some of the epistemological beliefs that are necessary for organizing society as a meritocratic state. As a consequence, ILSA has, to a large extent, been interpreted as the encyclopaedia of education due to the specific narrative that has been developed in the sphere of governing education, its comparative nature and a specific expertise in using the technology of mathematization embedded in meritocracy. This is true if we acknowledge the media and political coverage given to ILSA when it is presented in the international discourse on education.
- Archer, M. (2013). Solidarity and Governance. In Governance in a Changing World: Meeting the Challenges of Liberty, Legitimacy, Solidarity, and Subsidiarity. Pontifical Academy of Social Science, Extra Series 14. Vatican City: Vatican State. http://www.pass.va/content/dam/scienzesociali/pdf/es14/es14-archer.pdf.Google Scholar
- Bachelard, G. (1938/2002). The formation of the scientific mind. Clinamen: Bolton.Google Scholar
- Bell, D. (1972). On meritocracy and equality. The Public Interest, 29(48), 30–68.Google Scholar
- Bloom, B. (1969). Cross-National Study of educational attainment: Stage I of the IEA investigation in six subject areas. Final Report. Volume I. Chicago: University of Chicago.Google Scholar
- Bourdieu, P. (1971). Systems of education and systems of thought. In M. F. D. Young (Ed.), Knowledge and control: New directions in the sociology of education. London: Collier-Macmillan.Google Scholar
- Butterfield, H. (1965). The origins of modern science, 1300–1800. New York: Free Press.Google Scholar
- Desrosières, A. (1998). The politics of large numbers: A history of statistical reasoning. Cambridge & London: Harvard University Press.Google Scholar
- Durkheim, E. (1894/1938). The rules of sociological method. New York: Collier-Macmillan.Google Scholar
- Epstein, E. H. (2008a). Crucial benchmarks in the professionalization of comparative education. In C. Wolhuter, N. Popov, M. Manzon, & B. Leutwyler (Eds.), Comparative Education at Universities World Wide (pp. 9–24). Sofia: Bureau for Educational Services.Google Scholar
- Forsberg, E. (2006). International tests, national assessment cultures and reform history. Research project. Stockholm: The Swedish Research Council.Google Scholar
- Forsberg, E., & Pettersson, D. (2015). Meritokratin och jämförande kunskapsmätningar. In G.-B. Wärvik, C. Runesdotter, E. Forsberg, B. Hasselgren, & F. Sahlström (Eds.), Skola Lärare Samhälle: en vänbok till Sverker Lindblad. Göteborg: Göteborgs Universitet.Google Scholar
- Foshay, A. W., Thorndike, R. L., Hotyat, F., Pidgeon, D. A., & Walker, D. A. (1962). Educational achievements of 13 year olds in twelve countries. UNESCO Institute of Education: Hamburg.Google Scholar
- Foucault, M. (2009). Security, territory, population. Lectures at the Collège de France 1977–1978. New York: Picador.Google Scholar
- Fox, A. (1956). Class and equality. Socialist Commentary, 11–13.Google Scholar
- García Garrido, J. L. (1996). Fundamentos de Educación Comparada. Madrid: Editorial Dykinson S. L..Google Scholar
- Hacking, I. (1975). The emergence of probability. Cambridge: Cambridge University Press.Google Scholar
- Hacking, I. (1992b). Statistical language, statistical truth, and statistical reason: The self authentification of a style of scientific reasoning. The Social Dimensions of Science, 3, 130–157.Google Scholar
- Haraway, D. (1997). Modest_Witness@Second_Millennium. FemaleMan@_Meets_OncoMouse: Feminism and technoscience. New York: Routledge.Google Scholar
- Horkheimer, M., & Adorno, T. W. (1948). Dialektik der Aufklärung: Philosophische Fragmente. S. Fischer Verlag: Frankfurt.Google Scholar
- Hultin, M. (1968). Skolans produktivitet. Stockholm: Almqvist & Wiksell.Google Scholar
- Husén, T. (Ed.). (1967). International study of educational achievement in mathematics: A comparison of twelve countries. Volume I and II. New York: Wiley.Google Scholar
- Kazamias, A., & Massialas, B. G. (1982). Comparative education. In H. E. Mitzel (Ed.), Encyclopaedia of educational research (pp. 309–317). New York: Free Press.Google Scholar
- Keeves, J. P. (1968). Variation in mathematics education in Australia. ACER: Sydney.Google Scholar
- Kett, J. F. (2013). The history of a founding ideal from the American revolution to the twenty-first century (American institutions and society). Ithaca: Cornell University Press.Google Scholar
- Klemperer, V. (2011). The language of the third Reich. London & New York: Continuum.Google Scholar
- Koyré, A. (1968). From the closed world to the infinite universe. Baltimore: Johns Hopkins University Press.Google Scholar
- Kuusinen, J. (1967). School systems and level of achievement: Some notions on the basis of the IEA study in mathematics. Helsinki: University of Finland.Google Scholar
- Latour, B. (1988). The pasteurization of France. Cambridge & London: Harvard University Press.Google Scholar
- Lemann, N. (1999). The big test: The secret history of the American meritocracy. New York: Farrar, Straus & Giroux.Google Scholar
- Levinson, D., Cookson, P. W., & Sadovnik, A. R. (2002). Education and sociology: An encyclopaedia. London: Taylor & Francis.Google Scholar
- Lindblad, S., Pettersson, D., & Popkewitz, T. S. (2015). International comparisons of school results: A systematic review of research on large scale assessments in education. Stockholm: Vetenskapsrådets Rapporter.Google Scholar
- Lyotard, J.-F. (1984). The postmodern condition: A report on knowledge. Manchester: Manchester University Press.Google Scholar
- Martens, K. (2007). How to become an influential actor—The “comparative turn” in OECD education policy. In K. Martens, A. Rusconi, & K. Lutz (Eds.), Transformations of the state and global governance. London: Routledge.Google Scholar
- Noah, H., & Eckstein, M. (1969). Toward a science of comparative education. London: Macmillan.Google Scholar
- Nóvoa, A., & Lord, M. (Eds.). (2002). Fabricating Europe: The formation of an education space. Dordrecht/Boston/London: Kluwer Academic Publishers.Google Scholar
- Owens, T. L. (2013). Thinking beyond league tables: A review of key PISA research questions. In H.-D. Meyer & A. Benavot (Eds.), PISA, Power and Policy: the emergence of global educational governance (Oxford Studies in Comparative Education) (Vol. 23, p. 1). Oxford: Symposium Books.Google Scholar
- Pettersson, D. (2008). Internationell kunskapsbedömning som inslag i nationell styrning av skolan. Acta Universitatis Upsaliensis. Uppsala Studies in Education No. 120. Uppsala: Uppsala Universitet.Google Scholar
- Pettersson, D. (2014a). Three narratives: National Interpretations of PISA. Knowledge Cultures, 2(4), 172–191.Google Scholar
- Pettersson, D. (2014b). The development of the IEA: The rise of large-scale testing. In A. Nordin & D. Sundberg (Eds.), Transnational policy-flows in European education: Conceptualizing and governing knowledge. East Greenwich: Oxford Studies in Comparative Education. Symposium Books.Google Scholar
- Pidgeon, D. A. (1967). Achievement in mathematics: A National Study in secondary schools. Upton Park: NFER.Google Scholar
- Popkewitz, T. S. (1984). Paradigm and ideology in educational research. London & New York: Routledge.Google Scholar
- Popkewitz, T. S. (2008). Cosmopolitanism and the age of school reform: Science, education, and making society by making the child. New York: Routledge.Google Scholar
- Porter, T. M. (1995). Trust in numbers: The pursuit of objectivity in science and public life. Princeton, NJ: Princeton University Press.Google Scholar
- Reynolds, D. (2007). School effectiveness and school improvement (SESI): Links with the international standards/accountability agenda. In T. Townsend (Ed.), International handbook of school effectiveness and improvement. Basel: Springer.Google Scholar
- Rizvi, F., & Lingard, B. (2006). Globalisation and the changing nature of the OECD’s educational work. In H. Lauder, P. Brown, J. Dillabough, & A. H. Halsey (Eds.), Education, globalization and social change. Oxford: Oxford University Press.Google Scholar
- Roselló, P. (1943). Marc-Antoine Jullien de Paris: Père de l’Education Comparée et Précurseur du Bureau International de l’Education. Geneva: International Bureau of Education.Google Scholar
- Rust, V. D., Johnstone, B., & Allaf, C. (2009). Reflections on the development of comparative education. In R. Cowens & A. Kazamias (Eds.), International handbook of comparative education. Basel: Springer.Google Scholar
- Rutkowski, L., van Davier, M., & Rutkowski, D. (2014). Handbook of international large-scale assessment: Background, technical issues, and methods of data analysis. Boca Raton: CRC Press.Google Scholar
- Schriewer, J. (2009). Discourse formation in comparative education. Frankfurt am Main: Peter Lang.Google Scholar
- Schriewer, J., & Martinez, C. (2004). Construction of internationality in education. In G. Steiner-Khamsi (Ed.), The global politics of educational borrowing and lending. New York & London: Teachers College Press.Google Scholar
- Scott, J. C. (1998). Seeing like a state: How certain schemes to improve the human condition have failed. New Haven & London: Yale University Press.Google Scholar
- Shapin, S. (2010). Never pure: Historical studies of science as it was produced by people with bodies, situated in time, space, culture, and society, and struggling for credibility and authority. Baltimore: Johns Hopkins University Press.Google Scholar
- Shapin, S., & Lawrence, C. (1998). Introduction. In C. Lawrence & S. Shapin (Eds.), Science incarnate: Historical embodiments of natural knowledge. Chicago & London: The University of Chicago Press.Google Scholar
- Stiegler, J., & Hiebert, J. (1999). The teaching gap: Best ideas from the world’s teachers for improving education in the classroom. New York: The Free Press.Google Scholar
- Thévenot, L. (1986). Les investissements de forme. In Les Conventions économiques, Cahiers du Centre d’études de l’emploi (Vol. 29, pp. 21–71). Paris: PUF.Google Scholar
- Walker, D. A. (1962). An analysis of the reactions of Scottish teachers and pupils to items in the geography, mathematics and science. In A. W. Foshay, R. L. Thorndike, F. Hotyat, D. A. Pidgeon, & D. A. Walker (Eds.), Educational achievements of 13 year olds in twelve countries. Hamburg: UNESCO Institute of Education.Google Scholar
- Young, M. (1958). The rise of meritocracy. New Brunswick & London: Transaction Publishers.Google Scholar
Open Access This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.
The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.