Supporting Mathematics Teaching for Mastery in England
- 22k Downloads
Supporting Mathematics Teaching for Mastery in England examines the mathematics mastery reform introduced in England in 2014. The reform sought to develop a new pedagogical approach to teaching mathematics in England through an innovative professional development program and initiative to design and introduce new textbooks. The pedagogical approach, professional development, and textbooks were strongly influenced by approaches studied in high-performing South-East Asian countries. The chapter begins by identifying the catalysts for the reform’s inception, before describing the pedagogical approach and implementation of the reform. Finally, the chapter provides an outline of what has been achieved to date, and the preliminary conclusions of interested stakeholders.
In this chapter, we will analyze the mathematics mastery reform in England since 2014. This reform aims to introduce a new pedagogy inspired by high-performing South-East Asian countries to improve mathematics teaching and student outcomes and address England’s stagnant performance in mathematics in international league tables. The reform seeks to implement a mastery approach to teaching mathematics, developed through innovative professional development programs. The initial focus of the reform from 2014 was to develop mastery teaching in primary schools for pupils aged 4–11 years, and the first stage of expansion into secondary began in 2018–2019. It is an opt-in reform, funded by the Department for Education, which is currently being scaled up. Our study focuses only on the mastery reform at the primary level. This reform will be situated within the context of wider educational reform and mathematics teaching since 2010. We will then examine the pedagogical approach and professional development programs that have been implemented to support educators in teaching for mastery. The chapter will conclude by evaluating the results that have been achieved so far.
We draw on information from the National Centre for Excellence in the Teaching of Mathematics (NCETM), the main body responsible for implementation, and from official statements and reports by the Department for Education (DfE). To compare the intended goals of the reform with twenty-first century skills, we utilize frameworks presented by Reimers and Chung (2016) in Teaching and Learning for the Twenty-First Century as well as the 2012 report by Pellegrino and Hilton on the development of transferable knowledge and skills in the twenty-first century, Education for Life and Work: Developing Transferable Knowledge and Skills in the 21st Century. Weiss’ (2001) theory of change framework is used to analyze the implementation of the reform and to examine whether outcomes have been carried out in alignment with the intended goals. Additionally, evaluations conducted by Boylan Maxwell, Wolstenholme and Jay (2018, 2019) as well as reports from randomized controlled trials (Jerrim & Vignoles, 2015) are utilized to evaluate the results of the reform. In order to gain a better understanding of the reform, we conducted interviews with headteachers; deputy headteachers; mathematics Mastery Specialists; teachers; a professor of mathematics focused on curriculum, pedagogy and assessment; and the Director for Primary at the NCETM. This evaluation is limited by the relatively short time-period since the implementation of the reform. Due to this constraint, changes in national and international standardized test results cannot be utilized to evaluate the impact of the reform on mathematics achievement.
The introduction of the reform originated within the coalition government’s agenda for education reform from 2010 to 2015 to raise standards in education through a school-led system. Preceding efforts to improve mathematics teaching shaped the context for the reform.
Mathematics teaching in England changed dramatically with the National Numeracy Strategy, implemented in 1998 as part of the National Strategies reform. This was the first “systematic attempt at a national level to drive improvements in standards through a focused programme of managing changes in the way that core subjects are taught in classrooms” (DfE, 2011, p. 2). It was a government-mandated professional development program, providing schools with teacher training and detailed schemes-of-work. The implementation of the National Numeracy Strategy, which promoted prescriptive whole-class interactive teaching and was based on comparative studies of international practice, was a decisive move away from textbooks. An impact evaluation published in 2011 found that a key success of the reform was focusing attention on how to improve teaching (DfE, 2011). While national and international data suggested a positive impact on attainment in the early years, by 2011, the improvement in standards and progress had slowed down (DfE, 2011, p. 4). In 2011, the coalition government recognized the successes of the reform but announced that it was abandoning the National Strategies in favor of moving toward more collaborative practice between teachers and schools (DfE, 2011).
Another preceding reform aimed at improving primary mathematics teaching was the Mathematics Specialist Teacher programme (MaST), implemented in 2007 in response to Peter Williams’ report for the Department for Education identifying professional development as a key priority. The aim of MaST was “to provide each participating teacher with a Masters-level programme of training and professional support in order that they can carry out their mathematics specialist role working with teachers in their school” (DfE, 2013, p. 13). An evaluation of the programme in 2013 suggested not only initial success but also challenges to whole-school improvement due to mathematics specialists facing a “lack of senior leader support to work with colleagues across the school” (DfE, 2013, p. 21) as well as teachers with a lack of mathematical subject knowledge and confidence in teaching the subject.
In 2010, the government published the white paper The Importance of Teaching, detailing the new government’s focus on looking to lessons learnt by high-performing countries (DfE, 2010). The OECD PISA survey results from 2006 were cited as evidence for the need for education reform. The White Paper stated that the most important lesson that can be drawn from international educational comparisons “is that no education system can be better than the quality of its teachers” (DfE, 2010, p. 3). It continued to describe the lessons learnt from high-performing education systems of devolving power to the front line, while maintaining high levels of accountability (DfE, 2010).
As part of the educational reform to raise standards, a new knowledge-based National Curriculum was implemented in 2014. The Department for Education conducted an extensive review of mathematics curricula and teaching in a number of South-East Asian countries to inform the new mathematics curriculum (Jerrim and Vignoles, 2015). The new mathematics curriculum stated three aims: fluency, reasoning and problem-solving (DfE, 2013, p. 99). Key changes to the mathematics curriculum were an emphasis on high expectations, less content taught in primary school but in more depth and a focus on building firm knowledge foundations. Assessments were reformed at all ages in line with the new curriculum, increasing expectations and seeking to address issues of grade inflation. For example, in primary schools at the end of Key Stage 1 and Key Stage 2, and in secondary schools, the standardized tests taken at age 16 (GCSEs) and age 18 (A Levels) were reformed. As the new assessments and method of reporting results were only implemented in 2015, it is not possible to compare pre-2015 and post-2015 results. Due to this and the timeframe we evaluate in this paper, we have not used standardized assessment results to measure the impact of the mastery reform.
Mathematics became more prominent on the government’s agenda of education reform following the 2012 PISA rankings, which were cited as a major driver of the need for reform. England was placed 26th in the 2012 PISA rankings (with a score of 494 points), with 22% of 15-year-olds performing at the lowest level of mathematics proficiency (OECD, 2012). While the UK’s performance was average, the contrast between the UK’s score and top-performing South-East Asian countries was significant: Shanghai scored 613, Singapore scored 573 and Hong Kong scored 561 (OECD, 2012). The OECD estimates that the difference of 119 points between the UK and Shanghai is equivalent to nearly three years of schooling (OECD, 2012, p. 4). Moreover, England was identified as having the widest spread in attainment internationally (OECD, 2012). In 2010, only one-fifth of students in England were studying mathematics past age 16, the lowest rate in 24 developed countries (Hodgen & Nuffield Foundation, 2010).
In response to these results, the government reviewed their existing strategies for mathematics. Several issues were identified as areas of concern: children had difficulty in understanding mathematic instruction; instruction was too mechanical and lacking emphasis in thorough comprehension; there was a lack of positive attitudes toward the learning of mathematics; and students performed poorly in the subject (Education Endowment Foundation, 2015, p. 8). Quality of teaching was identified as an ongoing concern. This was not new: in 2006, Ofsted stated that “in promoting a really secure understanding of mathematical ideas, in stimulating students to think for themselves and to apply their knowledge and skills in unfamiliar situations, the picture was less encouraging” (Ofsted, 2006, p. 1). Alongside this, the Department for Education reported concern within the business community that numeracy skills in school and among college leavers were weak (DfE, 2013).
The Department for Education looked upon top-performing South-East Asian countries and cities such as Singapore, Shanghai and Hong Kong, which had demonstrated sustained high achievement in international tests, including PISA and TIMSS. Their curricula, pedagogies and “mastery” approaches were studied. A mastery approach to teaching and learning is not new and was first coined by Benjamin Bloom through his Taxonomy model. However, it is important to note that there are many interpretations and definitions of a mastery pedagogy. Singapore adopted a mastery framework in 1990, building it around five key cornerstones: concepts, skills, processes, attitudes and meta-cognition (Oxford Education Blog, 2018).
2.2 Differences Between South-East Asian Countries and England
There are significant differences between the education systems in top-performing South-East Asian countries and England. These contextual differences are crucial to understanding the different challenges faced in England, the contextual constraints and the adaptations necessary in England’s mastery approach.
Around 30,000 participants enter Initial Teacher Training in England each year, through a number of routes (Foster, 2019). These options fall into two main categories: school-led routes or higher education-led routes. There is currently a teacher recruitment shortfall; in 2018–2019, recruitment at the secondary level was below target by 17% (3,300 places), with 921 of these places being for mathematics trainees (Foster, 2019). As well as addressing continuing recruitment shortfalls, teacher retention is a primary area of current government policy. On the contrary, in Singapore, there is one route into teaching: all teachers are trained by the National Institute of Education (NIE). The NIE partners with the government to oversee this teacher preparation and assists with the placement of teachers in government schools to complete their professional training. To help develop a pipeline of suitable candidates, “prospective teachers are carefully selected from the top one-third of the secondary school graduating class, by panels that include current principals” (OECD, 2010, p. 169).
Another major difference between Singaporean and English approaches is the level of mathematics specialization required. In England, primary school teachers do not specialize and typically teach all subjects. Moreover, they are not required to have any qualification in mathematics apart from a pass in mathematics GCSE. In contrast, both in Singapore and Shanghai, primary schools are subject-specific and have specialist mathematics teachers (OECD, 2011).
Alongside the differences in the composition and training of the teaching workforces, there are some fundamental differences in the requirements of teachers in the job and their opportunities for in-service professional development. As Boylan et al. (2019) point out, teachers in Shanghai have “relatively low levels of pupil contact … typically a teacher has 60–80 min of contact per day, plus one-to-one or small group remediation or extension teaching” (p. 35). Thus, a significant amount of time is set aside for planning and preparing for lessons, reflecting on lessons and assessment to guide teaching, as well as carrying out interventions. In comparison, in England, primary school teachers are entitled to 10% of their teaching time for planning and assessment per week, with the rest of their time typically being devoted to whole-class teaching. In Singapore, teachers are entitled to 100 hours of professional development time a year (OECD, 2011). In Shanghai, teachers “report engaging more regularly in deeper forms of professional collaboration compared to teachers in other TALIS countries” (OECD, 2014). In England, while there are recent government-initiatives to increase funding in certain geographical areas for professional development, teachers do not have entitlement to set number of hours of professional development per year. This remains at the discretion of individual schools.
Lastly, ability grouping and differentiation by content and task has been a common practice in England, especially in mathematics. In contrast, both in Singapore and Shanghai, there is much less emphasis on differentiation, and in-class grouping by ability is avoided (Boyd & Ash, 2018). It is also important to note that compared to top-performing South-East Asian classrooms, on average, there is a much widespread attainment in English classrooms in mathematics (Jerrim & Shure, 2016). Research has shown that “teachers think about, and respond differently towards, pupils according to perceptions of their ability’’ (Francis et al., 2017, p. 5). There is also evidence that teachers of high ability groups have higher expectations for their students than those teaching low ability groups (Boaler, Wiliam, & Brown, 2000).
2.3 Theory of Change
The theory of change of the reform conceives that if teachers have deep mathematical subject knowledge, pedagogical knowledge and skills, and quality teaching resources, then students’ mathematical knowledge, skills, learning and therefore attainment will improve. If lead teachers in schools are trained as Mastery Specialists, with extensive knowledge and expertise in applying the principles of Teaching for Mastery, then they can lead change in practice in their own schools and local schools (Boylan et al., 2019). Mastery Specialists would be trained through a specialist program, with the opportunity to observe practice in Shanghai. Mastery Specialists would then introduce other teachers to the approach by facilitating group observation and analysis of the pedagogical approach. All professional development would be underpinned by the “five big ideas” that inform the approach. If schools, educators and school leaders are connected through networks and given the opportunity to work with educators within and across schools in a collegial manner, then structural change will take place. The outcomes of the activities are observed as the development of the Teaching for Mastery pedagogy, along with professional learning through the school mastery exchange. The results are improved outcomes for students, which are displayed through an increase in conceptual understanding and procedural fluency, which in turn leads to an improvement in pupil outcomes (Boylan et al., 2019).
The program began with Mastery Specialists going on the Maths Teacher Exchange (MTE) for two weeks to Shanghai and a week-long visit to England. The desired outcome was to change practice in the schools of the Mastery Specialists, and improve pupil mathematical outcomes (Boylan et al., 2019). After the first year, the theory of change evolved into large-scale building of capacity, acknowledging that not all teachers would go on the MTE. Thus, the NCETM created a new solution which amalgamated the core concepts of “Mastery”, the creation of Maths Hubs to develop networks of schools and build a professional development program for teachers across England, who in turn trained Mastery Specialists (not all of whom would go on the MTE), who then replicated training for local teachers. This was implemented through Mastery Specialists leading Teacher Research Groups (TRGs), small work groups in which teachers regularly meet to plan, observe and discuss practice over the course of a year, leading to change and impact in other local schools (Boylan et al., 2019).
By 2016, the range of professional development designed to catalyze change had expanded, including working with Mastery Specialists, TRGs, Maths Hubs events, high-quality resources from NCETM and adoption of accredited textbooks. The desired outcomes in schools now included embedding the Mastery approach, improved subject knowledge for mathematics teachers, pedagogical change including whole-class teaching, carefully structured lesson planning, high-quality resources and careful questioning of children. This would, in turn, then lead to improved student outcomes: conceptual understanding and procedural fluency and improved pupil mathematical outcomes.
2.4 Description of the Reform: Pedagogy
The NCETM was contracted by the Department for Education to lead the design and implementation of the reform. The Teaching for Mastery is a pedagogical reform to improve mathematics achievement, through a process of professional development and school-led improvement, based on best practices in Shanghai and Singapore. Its aims align with and support those of the national curriculum, to equip students with mathematical skills and ensure learners themselves have the confidence, meta-cognition and competencies required to succeed (DfE, 2014). In 2013, 46 headteachers and teachers went on a trip organized by the Department for Education to visit schools and observe mathematics teaching in Shanghai. The following year, the Junior Minister for the Department for Education, Liz Truss, led a delegation of experts on a research trip to Shanghai, “to study successful methods and potentially adopt them in schools here” (DfE, 2014, para. 4). The NCETM has called its new approach to teaching mathematics “Teaching for Mastery”.
The Teaching for Mastery approach is underpinned by the rejection of the “idea that a large proportion of people ‘just can’t do maths’” (NCETM, 2016a, para. 1). Central to the approach is the belief that with the appropriate resources, support, teaching and time, all children can succeed mathematically (Boylan et al., 2019, p. 34). The approach aims to develop a uniform expectation of high standards of achievement in mathematics for all students (NCETM, 2014a, 2014b). Thus, the aim is that “the large majority of pupils progress through the curriculum at the same pace. Differentiation is achieved by emphasizing deep knowledge and through individual support and intervention” (NCETM, 2014a, 2014b). All students in the class are introduced to mathematical concepts at the same time, with time given for all learners to master these concepts before moving on. This is a significant shift away from the previous practice of differentiation through content, task, resources and ability groupings.
An important feature of the Teaching for Mastery approach is the focus on methodical, careful curriculum design and lesson planning to “foster deep conceptual and procedural knowledge” (NCETM, 2014a, 2014b). Lesson planning should reflect the five big ideas, which have been influenced by South-East Asian approaches and articulated by the NCETM: variation, fluency, coherence, representation and mathematical thinking. In order to achieve these, teachers need deep subject knowledge to plan lessons in incremental steps, scaffolding concepts, carefully chosen vocabulary, models, representations and tasks. The approach encourages teachers to move away from the traditional three-part lesson structure to one with more parts and an emphasis on increasing direct instruction. This is combined with a focus on the importance of teacher questioning and formative assessment so that teachers can effectively identify students’ conceptual understanding. Lesson planning should pre-empt and plan for misconceptions, as well as provide ample opportunity for teacher–student as well as student-to-student dialogue. An example of this is the use of stem sentences that scaffold how children can articulate concepts and reason mathematically.
The reform consists of three main components to implement this approach to teaching mathematics: the England–Shanghai teacher exchange program, a professional development program led by Maths Hubs and the development of government-approved mastery textbooks.
The principal goal of the NCETM is “to ensure that all teachers of maths … have easy access to high quality, evidence-based, maths-specific continuing professional development” (NCETM, para. 2). Building capacity of teachers is one of the central components of the reform’s implementation and combines a variety of models. There is an acknowledgment that “not all aspects of teacher professional development can be (or should be) addressed in courses” (Villegas-Reimers, 2003, p. 142). The professional development programs address four elements of teaching practice: teachers’ subject knowledge, pedagogy to reflect the mastery approach, practice for student support and differentiation, and lesson planning and assessment.
The MTE was initiated between England and the municipality of Shanghai in 2014. Approximately, 70 Mastery Specialists from England (2 from each Maths Hubs) travel to China in autumn for one week each year, with a reciprocal visit from Chinese counterparts in the spring. During the teachers’ stay in Shanghai, they visit schools, take part in TRGs and learn about the approach to teaching mathematics from Chinese teachers as well as the NCETM delegates. When the Chinese teachers visit England, “Maths Hubs organize ‘showcase’ events so that local teachers can observe and ask questions about the teaching approach” (NCETM, 2018). The principal aim of the exchange is to expose English teachers to the pedagogy that have informed the Mastery approach. This scheme is now in its fifth year and is scheduled to run until the 2019–2020 academic year.
The Maths Hubs play a central role in organizing the promotion of mastery in local schools through recruiting schools for the MTE, recruiting Mastery Specialists, overseeing the work of trained Mastery Specialists and coordinating professional development events (Boylan et al., 2017). Importantly, they facilitate local schools working together on professional development, as well as sharing expertise and resources. Unlike previous efforts to reform mathematics, this reform was designed to be led by schools themselves, rather than a central organizing body such as the NCETM or government. Each Maths Hubs creates a network of schools in its local vicinity and coordinates the training of Mastery Specialists, who then work with Senior Leadership teams in their school to implement a whole-school approach.
Mastery Specialists form an integral part of the reform as they are required to implement training for local schools. Maths leaders have to obtain the support of their headteacher and commit to participating in the program for 2 years to apply to be a Mastery Specialist. To become a Mastery Specialist, teachers undergo a 1-year training program involving three, two-day residential stays. Each year, a group of Mastery Specialists in training (one or two from each Maths Hubs) also have the opportunity to go on the MTE. In their second year they begin training other teachers from local schools in TRGs. However, they do receive continuing support and professional development from the NCETM, as well as resources to carry out the training. Approximately 140 Mastery Specialists are trained each year, with the intention that 700 specialists would be trained by the end of the 2019–2020 school year, and a total of 11,000 primary and secondary schools, approximately one-third of all schools, will be reached by 2023 (Boylan et al., 2017).
The Mastery Specialist training aims at developing deep subject knowledge, pedagogical training in Mastery techniques and teacher educator training to enable specialists to lead and train others. While on the program, specialists are expected to set up a TRG within their own school, introduce their colleagues to the approach and begin to embed it within their own school. The way in which TRGs are run varies depending on the Mastery Specialist, but they are supposed to follow a common format. The Mastery Specialist introduces the session with theoretical background based on the “five big ideas”, using resources provided by the NCETM. They then conduct a “Teaching for Mastery” style lesson to a class of students in their school, observed by the teachers in the TRG, demonstrating the approach. After the lesson, the Mastery Specialist facilitates a discussion with the TRG, analyzing the lesson and approaches seen. In the second year, the Mastery Specialist is expected to create and lead a TRG for teachers from local schools, for which they receive funding. TRGs offer demonstration lessons and professional development in the teaching of mathematics six times throughout the year. Each TRG is asked to work with six schools, which each put forward a Maths leader and one other teacher to be trained (preferably a Key Stage 1 teacher). “The programme promotes collaborative forms of development found in Shanghai such as TRGs, and provides a forum through which teachers share learning and experiences” (Boylan et al., 2017, p. 80). Through this dissemination network, 12 maths teachers, six of whom are Maths Leads, can be trained each year by just one Mastery Specialist, which greatly accelerates the rate of change. The aim is that the Maths Lead will work with the maths teacher to implement the approach in their classrooms in their own schools before scaling the approach across the school.
Each school must opt in to the reform by providing a signed letter of authorization from the headteacher and enter into a contract. The NCETM created a range of videos to explain the approach to headteachers, as well as to advise how to implement it within their school. Maths Hubs host launch events to build on the online resources and explain the program and expectations to school leaders.
Another major area of implementation is the development and introduction of new mastery-style textbooks in the classroom. The drive for implementation of textbooks was a considerable shift in policy away from the National Numeracy Strategy, which had explicitly eschewed the use of textbooks in classroom. Schools initially viewed the reintroduction of textbooks with skepticism. The cost of implementation of the new textbooks has also been seen as a barrier, despite the offer of a £200 grant toward approved primary textbooks. The key importance of textbooks was noted by Tim Oates of Cambridge Assessments:
We’ve missed the fact that we have picked up some bad habits, and failed to notice the emergence, in other nations, of extremely well-theorised, well-designed, and carefully-implemented textbooks. We’ve also missed the fact that high quality textbooks support both teachers and pupils—they free teachers up to concentrate on refining pedagogy and developing engaging, effective learning (Oates, 2014).
The NCETM asked publishers to create a primary textbook series in line with the new pedagogical approach. As of June 2018, two textbook series have been approved by the Department for Education awarding committee (Debbie Morgan, Tim Oates and Bruno Reddy): Maths-No Problem! and Power Maths. The committee’s assessment criteria for the textbooks stated the required features: the need to “integrate understanding of mathematical concepts … with the development of factual and procedural fluency”; use “representations in the form of pictures and diagrams which reveal underlying mathematical structures and help make sense of mathematical ideas”; include “exercises which embody the concept of variation and reinforce the underlying structure of concepts and relationships”; and “ensure that assessment activities allow pupils to review key ideas and concepts and to check their own understanding, ensuring that they are ready for the next stage of teaching” (Maths Hubs, 2017).
Textbooks have been recognized as a core element in the implementation of the mastery approach, as well as a resource to address teacher workload. A good textbook that meets the requirements of this approach needs to be viewed as a “comprehensive tool, providing support for the development of both procedural fluency and conceptual understanding in mathematics as exemplified by proven practice in the high performing Asian jurisdictions” (Maths Hubs, 2017). Mathematical coherence, or explaining concepts in logical steps, along with a focus on the representation of concepts are the key elements in the design of the textbooks. Accurate use of mathematical language within textbooks is seen as a critical factor toward building a sound knowledge in mathematics. Well-designed practical activities and practice are designed with the aim of deepening knowledge and understanding of mathematical concepts for all learners.
According to Vanessa Pittard, Assistant Director for Curriculum and Standards at the DfE, “The textbook doesn’t teach; the teacher does. But having access to an elegant, coherent and comprehensive resource makes it easier. Teachers are liberated to focus on designing and delivering the engaging, interactive lessons that are characteristic of Mastery teaching” (2017, para. 11). A key component of the implementation of textbooks is providing teachers with adequate training on how to most efficiently use the textbooks in the classroom. Tim Oates has commented that the textbook development program “includes vital exchange and development of appropriate professional development to accompany curriculum innovation led by textbook adoption” (Oates, 2014, p. 11).
Nick Gibb, the School Reform Minister, stated in 2015, “mathematics for mastery … is another of the evidence-based approaches we have put at the heart of our education reforms” (Gibb, 2015). By looking to place empirical evidence at the center of the case for change, successive governments have looked to move ahead of some of the ideological criticism that reforms of this nature can produce. An influential supporter of the reform was OFSTED, the schools’ inspectorate. Jane Jones, OFSTED’s National Lead for Mathematics, addressed this concern, “The notion that headteachers might encourage their staff to retain previous ways of working because they fear criticism from an Ofsted inspector is a concern but one that everyone can play a part to dispel” (NCETM, 2015). The high-profile backing from the inspectorate helped secure buy-in from school leaders to engage with the reform.
Securing school leaders’ buy-in was important for the NCETM. As the reform was designed to be opt-in, incentives were structured to encourage schools to take part. Secured funding from the government assisted the NCETM in attracting schools with free professional development. In addition, schools were offered compensation for the time that Mastery Specialists or teachers attending TRGs would be out of class to cover substitute teachers.
Despite the broad support that the reform had from key stakeholders, there were distinct groups of opposition. Some Local Education Authorities (LEAs), who managed state schools, showed reluctance to engage with the NCETM, believing that reforms were being implemented unilaterally between schools and the NCETM, and not in partnership with the LEAs. The media conveyed mixed definitions of the Mastery approach to the general public. Articles in the press which referenced teacher exchanges between China and the United Kingdom were often accompanied with pictures of children rote learning in rows in classrooms. Others took aim at the concept of “borrowing” policy from elsewhere in hopes that it would work in a different context (Roberts, 2018). One reason for this opposition may stem from confusion over the meaning of the term “Mastery”, and what the approach entailed. This policy is not a faithful replication of “Mastery” as it is known in Singapore and China. The NCETM has coined its own definition of the approach.
There have been concerns from teachers and parents that the focus on teaching the whole class at the same place will negatively impact lowest-achieving children who will not be able to access the learning. There is also a critique that due to a lack of personalized support, higher-achieving children will not be adequately challenged.
Misconceptions about the Teaching for Mastery approach can have significant ripple effects into its delivery. The NCETM reinforces that Teaching for Mastery is a pedagogy and that its implementation is to be achieved through schools themselves.
2.6 Twenty-First Century Knowledge and Skills
The goals of the Teaching for Mastery approach in England can be analyzed within the framework for twenty-first century skills created by the National Research Council in the report Education for Life and Work: Developing Transferable Knowledge and Skills in the 21st Century. Three domains of twenty-first century competencies have been identified as being necessary for life and work. As discussed in the first chapter of this book, this framework group skills together under the cognitive, intrapersonal and interpersonal domains can be used to “represent the distinct facets of human thinking” (National Research Council, 2012, p. 21).
2.7 Cognitive Competencies
The majority of the skills promoted by the mastery approach fit within the cognitive domain. Responsibility, perseverance, grit and self-regulation are evident within the goals of how maths is taught according to this reform. The mastery approach challenges the mindset that a large proportion of people “just can’t do maths” (NCETM, 2016b, para. 1). Grouping students according to their abilities, along with other styles of differentiation, had previously been a common practice, contributing to a culture of low expectations. In order to counteract this way of thinking, the approach sets out to develop high expectations and standards of achievement for all students. This serves as a reflection of the important shift in skills identified as being important for the twenty-first century (Reimers & Chung, 2016, p. 11).
One of the main motivators behind the implementation of the mastery approach was to develop students’ fluency, problem-solving and reasoning skills. Variation is one of the key approaches to teaching mathematics within the reform. Students are encouraged to make connections between concepts in mathematics through procedural and conceptual variation. The “variation theory of learning points to variation as a necessary component in teaching for students to notice what is to be learned” (Kullberg, Runesson Kempe, & Marton, 2017, p. 559). This provides students with the ability to discern what concepts are, as well as what concepts are not. At the core of the approach is teaching students to think critically through “intelligent practice” rather than relying on the mechanical repetition of facts.
Fluency extends beyond simply recalling facts and procedures. Students are encouraged to make decisions in an effective manner, which can then be applied to a variety of contexts. The NCETM highlighted how evidence form cognitive science “suggests that learning key facts to automaticity “frees up” working memory to focus on more complex problem solving rather than reaching cognitive overload trying to calculate simple operations” (NCETM, 2018, para. 6).
The emphasis of high standards for all corresponds with Carol Dweck’s statement that “students who believe that intelligence or math and science ability is simply a fixed trait (a fixed mindset) are at a significant disadvantage compared to students who believe that their abilities can be developed (a growth mindset)” (Dweck, 2008, p. 2).
The training of Mastery Specialists focuses on introducing participants to core mastery concepts as well as building deep subject knowledge, developing subject-specific pedagogy, and an approach to assessing pupils within this pedagogical approach. Developing deep subject knowledge and presenting mathematical problems in various ways during classroom instruction, including visual representations, corresponds to the twenty-first century competencies of variation, critical thinking and fluency.
2.8 Intrapersonal Competencies
Within the intrapersonal domain, we can see the strongest alignment to skills that fall under the domains of intellectual openness and work ethic or conscientiousness.
According to the new National Curriculum, confidence has been pinpointed as the base for success in mathematics. As Dweck (2008) explains, “a considerable body of research is emerging from top cognitive psychology and cognitive neuroscience labs demonstrating that fundamental aspects of intelligence, and even intelligence itself, can be altered through training” (p. 3). The NCETM states that in the Teaching for Mastery approach “all pupils are encouraged by the belief that by working hard at maths they can succeed” (NCETM, 2016a, para. 2), a view that is central to the concept of mastery as it is implemented in Singapore (Kaur, 2018).
2.9 Interpersonal Competencies
There is less alignment in the explicit stated goals of the reform to the interpersonal domain. However, some of the pedagogical approaches do promote the development of interpersonal competencies, even though they are not stated explicitly as aims. Building in ample time for effective classroom talk and partner work allows students to develop teamwork and collaboration skills. Removing ability grouping and catering to the learning of all learners encourage students to develop skills such as empathy and perspective taking, especially among high-achieving students who are able to grasp concepts more readily.
2.10 Evaluation and Challenges
The following section evaluates the success of and challenges to implementation. It also provides summary of the current results of the reform, based on existing evaluations.
The implementation of the MTE was evaluated by Boylan et al. (2017). They investigated the first tranche of schools participating in the exchange and found that 30 out of 48 show good levels of implementation, meaning that teaching focused on the core components of the mastery pedagogy: conceptual understanding, procedural fluency, increased use of representations and adoption of textbooks. Only nine schools were identified as actively using textbooks in class, whereas other schools only used them for preparation, if at all. An evaluation of the MTE published in 2018 concluded that the exchange program continues to be one of the most important pillars of the mastery reform (Boylan et al., 2018). Lead primary teachers referenced valuable learning, increased subject knowledge and positive impact on pupil performance as results of the exchange. Many teachers also reported an increase in professional dialogue about maths and how to teach maths in school. While positive impact on subject knowledge, pedagogical approaches and beliefs were reported throughout the cohort of participants, the main effect of the exchange program was an increased confidence in teaching mathematics based on the mastery approach. In addition to their own visit to Shanghai, participants also specifically emphasized the importance of the teachers from Shanghai visiting classrooms in England. One mathematics teacher noted, “the most powerful experience is then bringing those teachers back here so we can see those teachers teach our children and that really supports us in terms of thinking about how can this realistically work in our school in our culture with our curriculum” (Boylan et al., 2019, p. 91). With regards to pupil outcomes, teachers especially recognized a change in attitudes toward math and soft skills. “Most children would say it’s their favourite subject now … that engagement in maths and their belief that they can achieve and that mistakes are valuable” and “I think as a whole the children have absolutely loved doing maths this way … they feel really part of the process of solving problems” (Boylan et al., 2019, p. 118). The teachers also report increased participation and a better feeling of equality within the classroom. “The way children view each other is brilliant now because they feel like they’re on a level playing field and they’re all learning together, whereas before it was very separate” (Boylan et al., 2019, p. 119).
It is difficult to evaluate the implementation and success of the TRG model. Boylan’s evaluation of TRGs run by participants of the MTE showed wide variation in models of TRGs implemented (Boylan et al., 2018, p. 19). While implementation of TRGs has been varied, some evidence was found that cascade models mirroring those used in the previous National Numeracy Strategy were being implemented. Boylan has suggested that Mastery Specialists experience difficulties in impacting non-lead primary teachers due to teachers’ lack of subject knowledge and confidence in applying the Mastery approach (Boylan et al., 2017). More evidence is needed about the quality assurance of the TRGs led by Mastery Specialists, especially as this is the primary way the approach is to be scaled.
To date, there is inconclusive evidence of the impact of the mastery approach being promoted by the NCETM. The latest evaluation published by Boylan et al. (2019) concludes that the different mastery pedagogies developed through the MTE can have a positive impact on student performance. However, quantitative evidence is currently at best mixed or not available at all. A randomized controlled trial conducted by Jerrim and Vignoles (2015) evaluated the impact of the Mathematics Mastery programme, which is based on similar mastery approaches but not designed and implemented by the NCETM. The study, which involved 10,000 pupils in 90 English primary schools and 50 secondary schools, found a small but positive impact of the Mastery reform. However, they also conclude that several other curriculum and pedagogical interventions in the UK have shown very similar results.
Based on these quantitative findings, there is a significant disparity between measurable results on standardized tests and what teachers report in qualitative interviews. There can be several reasons for these findings. A simple reason might be that there has not been sufficient time yet for changes to impact attainment at various testing stages. It might take more than two years of mastery approaches to produce measurable impact. Another factor could be the curriculum and assessment changes since 2014 that led to changes in schemes of work and practices for primary schools in general. Establishing the relationship between innovation and possible impact at a time of national large-scale change is always difficult. Thus, it may be that the finding of no impact could mean that the mastery approach, as implemented by participating schools, was more impactful than the schools’ previous practices, but that it was not more impactful than new practices implemented in comparison schools. Lastly, a plausible explanation for the lack of measurable impact of engagement in the MTE alone lies in the variation of implementation. As reported, not all schools that participated in the first MTE went on to implement the mastery approach, and of those who did, levels of implementation differ.
The NCETM has encountered misconceptions surrounding the mastery approach, especially due to multiple understandings and definitions of mastery, as well as the creation of schemes of work and non-government-approved textbooks by private companies (NAMA, 2015). Another area of concern is that the implementation of this reform requires a significant time and financial commitment from schools. As the NCETM explains, “Schools that have been most successful in introducing teaching for mastery have people in leadership positions putting time, energy and resources into supporting the approach” (NCETM, 2019, para 1). However, the majority of training is given to maths leaders to lead implementation, but the reality is the majority of maths leaders are also class teachers. Without time allocated to middle leaders to implement these changes, work with teachers and develop the approach in school, middle leaders’ capacity to enact change is limited. They often do not have the time or the holistic overview that is available to senior leaders.
It is positive to note that 37 Maths Hubs have already been set up, and by the year 2019–2020, there will be 700 Mastery Specialists working with more than 8,000 primary schools, representing around half of all primary schools in England (Maths Hubs, 2018, para. 1). Furthermore, there are early qualitative indications that the Maths Hub activities are warmly received by schools and teachers and have increased teachers’ confidence in teaching the subject. A school headteacher taking part in the scheme stated, “It is an effective way for mathematics education staff to collaborate with others to achieve best practice, in turn boosting their own personal development” (City of London Academy, 2018, para. 5). There are a few Maths Hubs, such as the White Rose Maths Hub, which are widely successful and have served as valuable resources for maths teachers across England. In addition to providing basic information about professional development opportunities, they also offer curriculum resources and assessment resources for primary and secondary schools, which are being used in schools nationally.
One key issue in the implementation of Maths Hubs has been the regional disparities between coverage and provision. There have also been differing levels of Local Education Authority support for the program. This indicates that Local Education Authorities were not brought into design and implementation planning early enough to build support and involvement. To rectify the regional disparities, the government tried to enhance the reach of Maths Hubs with an additional investment of six million pounds in October 2017 to create Hubs in areas where they will make the biggest difference and are not yet active (Boylan et al., 2017). While this has strengthened the coverage of Maths Hubs, the current distribution of the 37 Maths Hubs across England shows a clear gap along the East Coast. In order to ensure consistency of implementation, these gaps in provision must be addressed. Even if Maths Hubs were evenly distributed throughout England, each Hub would have to serve about 100 state-funded secondary schools and 500 state-funded primary schools. As the scheme grows and the program expands into secondary schools, Maths Hubs will require increased funding to serve more schools.
Another issue is that Maths Hubs are either led by a primary school, secondary school or college. The lead school usually defines the focus of the Hub and thus, if a Hub is led by a secondary school, the focus may tend more toward secondary school professional development than primary, and vice versa. As the number of schools supported by a Maths Hub grows and the program expands into secondary, the leadership capacity of the lead school may be stretched. Maths Hubs were originally asked to build a network of strategic partners and spread the Maths Mastery approach through work groups attached to them. The implementation has been piecemeal to date. Only a few Hubs list their partner organizations and most Maths Hubs do not mention their work groups. If they do, the number varies considerably from Hub to Hub. In general, access to clear and easily accessible information is hard to come by and the amount and quality of published information varies widely. This indicates that there are big differences in effectiveness and involvement between Hubs.
A further important part of the reform is the development of teaching for mastery resources, especially the creation and adoption of textbooks based on those found in East Asia. The NCETM has produced high-quality professional development resources that have been created by groups of maths specialists and teachers. These are freely available on their website and provide a comprehensive guide to teaching mathematical concepts, with carefully selected representations and guides for teachers.
The main argument used by the Department for Education to promote the implementation of textbooks was that they reduce teacher workload and promote “deep and connected knowledge”. However, it is important to recognize that their effectiveness is highly dependent on how they are used by teachers and cannot only be analyzed in terms of content. The only empirical evidence that has been collected on the impact of “mastery” textbooks is a one-year trial of Inspire! Maths, a textbook that did not receive government accreditation (Hall, Lindorff, & Sammons, 2016). Evidence of the effect of textbook use is limited and usually focused on comparing textbooks rather than comparing the use of textbooks to no textbooks at all (Boylan et al., 2018). In a qualitative study conducted a year into the Mastery Program by NCETM Maths Hubs, teachers commented that the implementation of textbooks had contributed to student learning and had helped to foster a more positive attitude toward mathematics. They also reported gaining confidence in their teaching from increased subject knowledge (NCETM, 2018).
Despite the government subsidy, the majority of schools have yet to implement new textbooks. This is reportedly large because of cost and remaining skepticism over whether textbooks are an effective investment. One issue reported by schools that did implement textbooks was that students with lower reading abilities had to be paired with a more skilled partner to assist them; teachers commented that they often had to rewrite material in textbooks to make it more accessible to students, or project the page and explain the content to students step-by-step (NCETM, 2018). This undermines one of the aims of the textbook to streamline teachers’ workload and allow them more time to prepare and teach effectively.
The NCETM has not addressed the role of parents in the Teaching for Mastery approach, both in terms of communicating the approach and offering them ways to support their children at home. This conflicts with research emphasizing the importance of parent involvement for success in primary school (Knowles & Fair Education Alliance, 2017).
There is currently growing buy-in from schools for the teaching for mastery approach. Expansion of government funding and support for the policy, as well as the support from key stakeholders, including Ofsted, means that the reform has become a central part of education policy. To date, 37 Maths Hubs are supporting schools nationally; more than 1,700 schools have already opted-in to the reform and over 280 Mastery Specialists are working in or with these schools. In 2018, the program began expanding into secondary schools with the first cohort of secondary teachers attending the MTE.
However, the impact on student performance is difficult to evaluate at this point in implementation. As the reform is mainly focused on primary schools, we do not yet have PISA results or other standardized maths results of involved pupils. Furthermore, the lack of internal evaluation for the reform means we also do not have solid evidence of the extent to which teachers have implemented these methods. This is also partly due to the varying forms that a mastery approach can take within a school; some schools are adopting mastery schemes created by a variety of publishers, some are using government-approved textbooks, some are using schemes created by White Rose Maths Hub and others are implementing their own take on the mastery approach. With this level of variation in methods of current implementation, it is challenging to evaluate the successes of the reform. The level of implementation has so far only been evaluated for the first cohort of schools participating in the MTE. This evaluation showed that, while there were areas of success, Mastery Specialists have had difficulties in disseminating the approach due to primary math teachers’ lack of subject knowledge and confidence. Additionally, many schools have chosen not to use accredited, government-approved textbooks.
In conclusion, this is a critical point as the reform expands its reach into secondary mathematics. At present, while the reform has been implemented in parts, there is a lack of information about the sustainability and quality of implementation. Importantly, there is a need to put in place rigorous systems of quality assurance in order to assess the impact of the MTE, Mastery Specialist programme and Maths Hubs. Maths Hubs that have shown higher levels of success in recruiting schools and leading the development of sustained whole-school approaches should be studied. These Hubs should share their implementation methods to foster more partnership between Hubs.
- Boaler, J., Wiliam, D., & Brown, M. (2000). Students' experiences of ability grouping—Disaffection, polarisation and the construction of failure. British Educational Research Journal, 26(5), 631–648.Google Scholar
- Boylan, M., Wolstenholme, C., Bronwen, M., Jay, T., Stevens, A., & Demack, S. (2017). Longitudinal evaluation of the Mathematics Teacher Exchange. China, England. Retrieved from: https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/666450/MTE_third_interim_report_121217.pdf
- Boylan, M., Wolstenholme, C., Bronwen, M., Jay, T., Steven, A., & Demack, S. (2018). The Mathematics Teacher Exchange and “Mastery’ in England: The Evidence for the Efficacy of Component Practices. Retrieved from https://www.mdpi.com/2227-7102/8/4/202/pdf
- Boylan, M., Wolstenholme, C., Demack, S., Maxwell, B., Jay, T., Adams, G., Reaney, S. (2019). Longitudinal evaluation of the Mathematics Teacher Exchange. China-England. Final report. Retrieved from https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/773320/MTE_main_report.pdf
- City of London Academy. (2018). Mission Statement. Retrieved from https://www.cityacademy.co.uk/Maths-Hub
- Department for Education. (2010). The Importance of Teaching. Retrieved from https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/175429/CM-7980.pdf
- Department for Education. (2011). The National Strategies 1997–2011—A brief summary of the impact and effectiveness of the National Strategies. Retrieved from https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/175408/DFE-00032-2011.pdf
- Department for Education. (2013, December 4). New maths hubs to raise standards [Press Release]. Retrieved from https://www.gov.uk/government/news/new-maths-hubs-to-raise-standards
- Department for Education. (2014, July 1). Network of 32 maths hubs across England aims to raise standards [Press Release]. Retrieved from https://www.gov.uk/government/news/network-of-32-maths-hubs-across-england-aims-to-raise-standards
- Dweck, C. (2008). Mindsets and Math/Science Achievement. Retrieved from https://www.growthmindsetmaths.com/uploads/2/3/7/7/23776169/mindset_and_math_science_achievement_-_nov_2013.pdf
- Education Endowment Foundation. (2015). Mathematics mastery primary evaluation report. Retrieved from https://v1.educationendowmentfoundation.org.uk/uploads/pdf/Mathematics_Mastery_Primary_(Final)1.pdf
- Foster, D. (2019). Teacher recruitment and retention in England. Retrieved from https://researchbriefings.parliament.uk/ResearchBriefing/Summary/CBP-7222
- Francis, B., Archer, L., Hodgen, J., Pepper, D., Taylor, B., & Travers, M.C. (2017). Exploring the relative lack of impact of research on “Ability Grouping” in England: A discourse analytic account. Cambridge Journal of Education, 47(1), 1–17.Google Scholar
- Gibb, N. (2015). Speech on government’s maths reforms [Transcript]. Retrieved from https://www.gov.uk/government/speeches/nick-gibb-speech-on-governments-maths-reforms
- Hall, J., Lindorff, A., Sammons, P. (2016) Evaluation of the Impact and Implementation of Inspire Maths in Year 1 Classrooms in England: Findings from a Mixed-Method Randomised Control Trial. University of Oxford. Retrieved from https://hdl.handle.net/10871/24265
- Hodgen, J., & Nuffield Foundation. (2010). Is the UK an Outlier: An International Comparison of Upper Secondary Mathematics Education. London: Nuffield FoundationGoogle Scholar
- Jerrim, J., & Shure, N. (2016). Achievement of 15-Year Olds in Wales: PISA 2015 National Report. London: UCL Institute of Education. Retrieved from http://dera.ioe.ac.uk/27969/1/161206-pisa-2015-en.pdf
- Jerrim, J., & Vignoles, A. (2015). The Causal Effect of East Asian “Mastery” Teaching Methods on English Children’s Mathematics Skills. DoQSS working paper series.Google Scholar
- Kaur, B. (2018). Building the Maths house: Singapore’s curriculum framework. Retrieved from https://educationblog.oup.com/secondary/maths/building-the-maths-house-singapores-curriculum-framework
- Knowles, C., & Fair Education Alliance. (2017). Closing the Attainment Gap in Maths: A Study of Good Practice in Early Years and Primary Settings. London: Fair Education AllianceGoogle Scholar
- Kullberg, A., Runesson Kempe, U., & Marton, F. (2017). What is made possible to learn when using the variation theory of learning in teaching mathematics? ZDM, 49(4), 559–569. https://doi.org/10.1007/s11858-017-0858-4
- Maths Hubs. (2017, January 9). Teaching for Mastery of Mathematics—Textbook Assessment Criteria. Retrieved from https://www.mathshubs.org.uk/media/5559/assessment-criteria-final-09012017.pdf
- NAMA. (2015). Five Myths of Mastery in Mathematics. Retrieved from: https://www.nama.org.uk/Downloads/Five%20Myths%20about%20Mathematics%20Mastery.pdf
- National Research Council. (2012). Education for Life and Work: Developing Transferable Knowledge and Skills in the 21st Century. Washington, DC: The National Academies Press. Retrieved from https://doi.org/10.17226/13398
- NCETM. (2014a). Mastery in mathematics: What it is and why we should be doing it. Retrieved from https://www.ncetm.org.uk/resources/45776
- NCETM. (2014b). Charlie’s Angles. Retrieved from https://www.ncetm.org.uk/resources/46034#ItemActions
- NCETM. (2015). Evaluation of the Textbook Project, Year One (January–July 2015). Retrieved from https://www.mathshubs.org.uk/media/5202/ncp2-evaluation-final-sept-21.pdf
- NCETM. (2016a). Teaching for mastery programme expanded with £41 million new funding over four year. Retrieved from https://www.ncetm.org.uk/news/49224
- NCETM. (2016b). The Essence of Maths Teaching for Mastery. NCETM. Retrieved from https://www.ncetm.org.uk/files/37086535/The+Essence+of+Maths+Teaching+for+Mastery+june+2016.pdf
- NCETM. (2018). Teaching for Mastery: Supporting Research, Evidence and Argument. NCETM. Retrieved from https://www.ncetm.org.uk/resources/50819#recent_research_england
- Oates, T. (2014). Why Textbooks Count. Retrieved from https://www.cambridgeassessment.org.uk/Images/181744-why-textbooks-count-tim-oates.pdf
- OECD. (2010). Singapore: Rapid Improvement Followed by Strong Performance. Retrieved from: https://www.oecd.org/countries/singapore/46581101.pdf
- OECD. (2011). Lessons from PISA for the United States: Strong Performers and Successful Reformers in Education. Retrieved from https://www.oecd.org/pisa/46623978.pdf
- OECD. (2012). Programme for International Student Assessment (PISA)—Results from PISA 2012. Retrieved from https://www.oecd.org/pisa/keyfindings/PISA-2012-results-UK.pdf
- OECD. (2014). Key Findings from the Teaching and Learning International Survey. Retrieved from https://www.oecd.org/education/school/TALIS-2014-country-note-Shanghai.pdf
- Ofsted. (2006). Evaluating Mathematics Provision for 14–19-year-olds. Retrieved from https://mei.org.uk/files/pdf/Evaluating_Mathematics_Provision.pdf
- Organisation for Economic Co-operation and Development, National Center on Education and the Economy (U.S.), Programme for International Student Assessment, & OECD Directorate for Education (Eds.). (2011). Strong performers and successful reformers in education: lessons from PISA for the United States. Paris: OECDGoogle Scholar
- Oxford Education Blog. (2018). Mastery: Solving the problem. Retrieved from https://educationblog.oup.com/primary/mastery-solving-the-problem
- Pittard, V. (2017, August 24). Mastery myths: Textbooks constrain creative teaching. Retrieved from: https://educationblog.oup.com/primary/mastery-myths-textbooks-constrain-creative-teaching
- Reimers, F., & Chung, C. K. (2016). Teaching and Learning for The Twenty-First Century: Educational Goals, Policies, and Curricula From Six Nations. Cambridge, Massachusetts: Harvard Education Press.Google Scholar
- Roberts. J. (2018, July 20). Exclusive: England 'wrong to copy Shanghai maths'. Retrieved from https://www.tes.com/news/exclusive-england-wrong-copy-shanghai-maths
- SFR88_2017_TSM_ITT_Allocations_2018_to_2019_Main_Text.pdf. (n.d.). Retrieved from https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/663192/SFR88_2017_TSM_ITT_Allocations_2018_to_2019_Main_Text.pdf
- TALIS-2014-country-note-Shanghai.pdf. (n.d.). Retrieved from https://www.oecd.org/education/school/TALIS-2014-country-note-Shanghai.pdf
- Villegas-Reimers, E. (2003). Teacher Professional Development: An International Review of the Literature. Paris: UNESCO International Institute for Educational Planning.Google Scholar
- Weiss, C. (2001). Theory-Based Evaluation: Theories of Change for Poverty Reduction Programs. In Feinstein, O. N., & Picciotto, R. (Eds.), Evaluation and Poverty Reduction: Proceedings from a World Bank Conference (pp. 103–114). World Bank. Retrieved from https://books.google.com/books?id=e8kiRvx7L-AC
Open Access This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.
The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.