The Punjab Schools Reform Roadmap: A Medium-Term Evaluation

Abstract

In 2010, a whole-system reform was designed and launched in the Punjab province of Pakistan called the Punjab Schools Reform Roadmap (PSRR). This reform was a direct response to the challenges of education in the province at the time, which included scale, capacity to deliver, and political will. Further, 2010 was a time at which the political and administrative landscape of Pakistan was changing: the right to education act had just been formalized and education was simultaneously devolved from a federal matter to a provincial one.

This chapter studies the outcomes of the PSRR a decade after its implementation with an emphasis on three dimensions of the reform: management capacity, teacher capacity and monitoring & information systems. Specifically, we discuss the above with reference to increasing access to and quality of education in the province while also analyzing the outcomes and sustainability of the reform 10 years from its inception.

AEO:

Assistant Education Officer

CEO:

Chief Education Officer

DDEO:

Deputy District Education Officer

DEA:

District Education Authorities

DFID:

Department for International Development (UK)

DMO:

District Monitoring Officer

EVS:

Education Voucher Scheme

FAS:

Foundation Assisted Schools

LND:

Literacy and Numeracy Drive

MEA:

Monitoring and Evaluation Assistants

NLNS:

National Literacy and Numeracy Strategy

NSF:

National Science Foundation (U.S.)

NSP:

New School Program

NTS:

National Testing Service (Pakistan)

OECD:

Organization for Economic Co-operation and Development

PCTB:

Punjab Curriculum and Textbook Board

PEC:

Punjab Examination Commission

PEF:

Punjab Education Foundation

PISA:

Programme for International Student Assessment (OECD)

PITB:

Punjab Information and Technology Board

PMIU:

Programme Monitoring and Implementation Unit

PSRR:

Punjab Schools Reform Roadmap

QAED:

Quaid-e-Azam Academy for Educational Development

SED:

School Education Department

SMU:

Special Monitoring Unit

SSI:

Statewide Systems Initiatives

TALIS:

Teaching and Learning International Survey (OECD)

5.1 Introduction

In 2010, the challenge of education reform in Pakistan was threefold: scale, capacity to deliver, and political will (Barber, 2010, p. 7). To make matters more urgent, an amendment to the Constitution of Pakistan had just been passed that gave all children between the ages of 5 and 16 the right to free education while simultaneously devolving education autonomy, formerly a federal matter, to the provinces (Government of Pakistan, 2010). During this dynamically changing political and administrative landscape, a reform to Pakistan’s most populous province – the Punjab – was launched by then Chief Minister of the Punjab Mian Shahbaz Sharif (Barber, 2013; Pakistan Bureau of Statistics, 1998). This reform came to be known as the Punjab Schools Reform Roadmap (PSRR) and was a direct response to the challenges of education in the province.

The PSRR targeted the entirety of the Punjab and called for the systemic overhaul and transformation of the entire education delivery and monitoring apparatus of the province. The key themes under this reform included the capacity-building of management in the public education system, the capacity-building of teachers, the collection of reliable and timely data, and the expansion of access to education through public private partnerships (Barber, 2013).

This chapter studies the outcomes of the PSRR and analyzes the sustainability of these outcomes 5 years after the conclusion of PSRR (this time period is henceforth referred to as the medium-term) with an emphasis on management capacity, teacher capacity and monitoring & information systems dimensions of the reform. Specifically, we discuss the above with reference to increasing access to and quality of education in the province.

We start with reviewing the literature on whole-system reforms and institutional capacity building to drive change within education systems. Next, we discuss the design elements of the reform, followed by an analysis of the interventions, outcomes and sustainability of the reform in the medium term. Lastly, we conclude with recommendations and a discussion of the way forward.

5.2 What Are Whole System Reforms and Capacity Building?

5.2.1 Whole-System Reforms

A whole-system reform (also known as a large-scale reform) is a reform that focuses on the improvement of every school within a district, every district within a province, and every province within a country. This contrasts to a targeted reform that focuses on improving outcomes for a subset of schools, communities or districts (Fullan, 2009).

“The moral and political purpose of whole-system reform is ensuring that everyone will be affected for the better, starting on day one of implementing the strategy” (Fullan & Levin, 2009). Since before the turn of the twenty-first century, this large-scale reform theory has been applied in several countries and states within individual countries to drive change at scale in their respective educational landscapes. One of the first countries to use this reform theory posited by Michael Fullan was the United Kingdom in 1997. The United Kingdom formulated a National Literacy and Numeracy Strategy (NLNS) that aimed to improve learning levels of 11-year-old students in all primary schools across the country (20,000 schools in total).

One of the key architects of NLNS was Sir Michael Barber, who also played a foundational role in the design and implementation of the PSRR. Barber’s implementation strategy for NLNS rested on six key fundamentals. Namely, (i) ambitious standards, (ii) good data and clear targets, (iii) devolved responsibility, (iv) access to best practice and quality professional development, (v) accountability, and (vi) an increased focus on low performing schools (Fullan, 2009). An evaluation of the NLNS concluded that this strategy led to a significant improvement in student achievements over a relatively short span of time. However, there was concern about its sustainability in the long term due to the strategy being largely centralized and target-driven from the center (Fullan, 2010).

Another major example of a whole-system reform is a series of reforms launched in the United States in the early 1990s by the National Science Foundation (NSF). These systemic reform programs were intended to influence most, if not all aspects of the school system in the U.S. (Supovitz & Taylor, 2005). The NSF first launched the Statewide Systemic Initiatives (SSI) program in 1991, which provided up to $10 million to 25 states and Puerto Rico to assist with beginning whole-systems reform efforts in science, mathematics and technology education at the K-12 grade levels. Two years later, the NSF initiated the Urban Systemic Initiatives program to catalyze changes in policy and resource utilization in 28 of the largest U.S. school systems. The NSF also launched subsequent programs such as the Rural Systemic Initiative and the Local Systemic Change Initiative (Supovitz & Taylor, 2005; Zucker et al., 1998).

The SSIs were born out of a desire to improve student performance in mathematics and the sciences to improve the nation’s economic competitiveness (Earle & Wan, n.d.; Zucker et al., 1998). Over the course of the 3 years following the SSIs’ launch in 1991, the NSF negotiated 5-year cooperative agreements with competitively selected states to undertake such initiatives (Supovitz & Taylor, 2005; Zucker et al., 1998). While results have been varied due to the spread of the reform, the SSI program generally brought overall improvements in the practices of school systems in participating states. Evaluations of the program speak about the legacy of new or improved curriculum frameworks, positive changes in various state policies, new institutions and partnering arrangements, and an increased number of competent state and local leaders of reform. Additionally, evidence from selected states shows that some SSIs had a modest, positive impact in improving student learning outcomes in mathematics and science (Zucker et al., 1998).

5.2.1.1 Implementation and Delivery Chains

5.2.1.1.1 How Are Whole-System Reforms Implemented?

The NLNS serves as a useful case study on the efficacy of whole-systems reform theory and its implementation strategy respectively. Fullan has since directly worked with the Canadian government in Ontario, designing an implementation strategy that draws on lessons learnt from the United Kingdom while adjusting to the context of Canada. However, the fundamentals of the implementation strategy do not drastically differ. The whole-system reform implementation in the eyes of Fullan still rests on the foundation of focusing on a limited number of core policies and strategies, and working on them in tandem while ensuring strong leadership at the top that does not lose sight of the intended focus (Fullan & Levin, 2009). As a result of such an implementation strategy, the impact of well-executed large-scale reforms has been similar (significant impact in a short time period) and with similar potential pitfalls (sustainability concerns due to a top down implementation strategy).

5.2.1.1.2 Whole-System Reform Delivery Chains

Another approach to effectively achieving whole-system reform is by giving prime importance to the implementation strategy (Fullan, 2009). This approach has been pioneered by Michael Barber and is called ‘Deliverology’ (Barber, Moffit, & Kihn, 2011). Deliverology outlines a delivery chain that effectively manages and monitors all steps of the implementation of a reform with the aim of producing the highest impact on desired outcomes.

This approach has three key components. The first step is the establishment of a Delivery Unit that is limited in size but is only comprised of highly skilled and talented individuals who can oversee, inform and improve all aspects of the implementation process. The second foundation is effective and robust data gathering and analysis to ensure evidence-based progression at all steps and stages of the implementation. Third, the establishment of routines in order to assess performance and subsequently ensure a timely delivery of the reform. Relationship building across all levels of the hierarchy of the delivery chain (from top leadership to district officials and teachers working on the ground) is another component that is imperative to achieving optimal implementation of the reform (Barber et al., 2011).

In the next section, we frame this idea of institutional capacity and how it has been built in education systems worldwide to achieve both whole-system reforms or marginal scaling-up reforms.

5.2.2 Strengthening Institutional Capacity

An institution (or system)‘s capacity refers to the capability of the institution to “plan, implement, manage, or evaluate policies, strategies or programs” (Cohen, 1995a, 1995b, p. 409). Institutional capacity has two arms: human resource capacity and management capacity. Thus, strengthening these capacities require the strengthening of the following components (Dill, 2000, p. 214):

• Strengthening human resource capacity requires the strengthening of “training, recruitment, effective utilization and retention of skilled public sector personnel.”

• Strengthening management capacity requires strengthening of the ability of a public organization or agency to anticipate and adapt to change through policy levers and implementation policies, effective management of resources, and evaluation of performance for continuous feedback.

To put it more explicitly, building institutional capacity entails targeted interventions that build human resource capacity and management capacity, the latter of which strengthens the organization and can lead to institutional reform. This section explores how countries have (successfully) attempted to build institutional capacity in their education systems, including best practices and the building of institutional capacity through the use of data.

5.2.2.1 International Best Practices

The Organization of Economic Cooperation and Development (OECD) identifies Canada, Finland, Japan, Shanghai-China and Singapore as having high-performing education systems, formulating lessons from these systems as guidelines for other countries (OECD, 2011, p. 228). Among the lessons from these high-performing education systems is the development of capacity at the point of delivery, which includes attracting and developing high-quality teachers and developing the capacity of school leaders. These systems also place large emphasis on the development of management capacity, giving autonomy to school leaders coupled with strong accountability mechanisms and incentive alignment. Further, the systems embed a culture of developing “ambitious, focused and coherent” standards for education and system coherence that incorporate the alignment of policies and practices, and reform based on continuous feedback (OECD, 2011, p. 233).

In terms of student learning outcomes, Hong Kong, South Korea, Shanghai and Singapore rank among the top five high-performing countries, based on the OECD’s Programme for International Student Assessment (PISA) assessments in 2009 (Jensen, 2012). These countries owe their success to an unrelenting focus on implementation in tandem with policy, using effective education strategies and well-designed programs that target teaching as well as learning. Among their strategies is the provision of high-quality teacher education, mentoring for teachers, treating teachers as researchers in the field and giving them more responsibility and avenues for career growth. These are unsurprisingly similar to the practices identified by OECD (2011) above, emphasizing the importance of common themes of teacher capacity, management capacity and policy reform in high-performing education systems around the world.

5.2.2.2 Capacity Building through the Use of Data

High-performing systems place a strong emphasis on continuous feedback to see what is working and areas in which further investment is required (OECD, 2011). One method to measure teacher and school leader capacity is the OECD’s Teaching and Learning International Survey (TALIS) (OECD, 2019). TALIS brings forth three dimensions of policy recommendations for nations, of which two relate to the professional growth of teachers and school leaders, and attracting quality teachers and school leaders (OECD, 2019, p. 27). These resonate strongly with the practices of high-performing systems that build institutional capacity, providing the data which informs and enables the adjustment of specific policy levers as a result.

Hong Kong is one of the world’s highest performing education systems (Jensen, 2012). From 1999 to 2012, Hong Kong implemented holistic whole-system reforms to its education sector to improve student learning (Jensen, 2012, p. 26). Hong Kong guided its reforms around an in-depth analysis of its education system, which revealed major gaps in student learning and teaching practices (Jensen, 2012, p. 27). Following this, Hong Kong conducted comprehensive consultations with stakeholders and key actors to understand the gaps needing reform in its education system and ultimately improve student learning (Jensen, 2012, p. 27). Hong Kong has significantly reformed its education system, becoming one of the highest-performing education systems in the world, through systematic reforms which were guided significantly by in-depth analyses of the state of its system, feedback loops, and engagement and consultation of stakeholders.

While understanding international best practices is important, it is imperative to remain cognizant of the specific context of Pakistan (and of the Punjab province in particular) and how capacity building (stemming from gaps identified by robust data) relates to this context. India is a country that can illuminate such a situation due to its contextual similarities to Pakistan.

The Indian government established federal and provincial institutes to build the capacity of government officers driven by an analysis of gaps in the system, for example the professional development of government officers to allow them to undertake more challenging and complex tasks (Aijaz, 2010, p. 388). While the (management) capacity building interventions seemed beneficial for those attending the sessions, the study found that many personnel for whom the training was intended would nominate their subordinates to attend instead, not seeing the relevance of the training to themselves (Aijaz, 2010, p. 389). One of the key recommendations from this analysis, which is relevant to the Pakistani context, is that capacity building will fail to reach those it is intended for and thus will fail to achieve its objective of strengthening the capacity of the institution as a whole if the concerns and attitudes of the targeted personnel are not addressed, or perhaps if participation is not incentivized.

5.3 Design Elements of the Reform

The Punjab has an intricate system of interconnected bodies and actors that collectively provide and monitor public education in the province. This system and the interactions within are what led to the formation of the PSRR. Thus, to understand the PSRR we must first understand the system on which it is based.

In 2010, the Eighteenth Amendment to the Constitution of Pakistan was passed (Government of Pakistan, 2010). This amendment had two major implications for education in the country. First, the amendment added Article 25-A to the constitution, which stipulated that all children between 5 and 16 years of age had the right to free and compulsory education. Secondly, the amendment devolved education autonomy to the provinces, making “the curriculum, syllabus, planning, policy, centres of excellence and standards of education” fall within the provincial purview (Dawn, 2010). As a result, the Government of the Punjab was now responsible for providing free and compulsory education (previously a federal matter) to all children 5–16 years old, putting new pressures on the previously limited provincial system of education delivery.

PSRR was born during this period, looking to reform and strengthen the system of public education delivery in the Punjab at a time when such an undertaking was becoming necessary. PSRR was built on a thorough understanding of the system, having been led by then provincial Chief Minister Mian Shahbaz Sharif and the Pakistan Education Taskforce (Barber, 2013). The Pakistan Education Taskforce was developed by then Prime Minister of the United Kingdom Gordon Brown & President of Pakistan Asif Ali Zardari; it was co-chaired by Sir Michael Barber (referenced above) and Shahnaz Wazir Ali.

5.3.1 The System

The Punjab’s public education system is comprised of numerous bodies and agents, each responsible for different components of education delivery. At the provincial level, the network of formal schools falls under the purview of the School Education Department (SED), while non-formal education falls under the Literacy & Non-Formal Basic Education Department and the Technical Education & Vocational Training Agency. Higher education falls under the Higher Education Commission’s Punjab chapter, and standardized assessment at higher levels is the responsibility of the Board for Intermediate and Secondary Education. Since the PSRR was concerned with strengthening the system of public education primarily in relation to Article 25-A (formal education for children 5–16 years old), it focused on the SED and the bodies/actors within.

This system of public education delivery has been extensively mapped out through secondary research and interviews of relevant actors within the system to produce. Figure 5.1 details the other bodies that work concurrently with the SED, the numerous bodies within the SED and the specific channels of public education delivery that the reform targets. Mapping of the system has also been color-coded to display the channels within the system that cater to (a) access, (b) quality, and (c) monitoring & evaluation – the three dimensions of the PSRR.

Fig. 5.1

System map of education in the Punjab

5.3.2 Theory of Change

As stated above, PSRR separated the system of public education into three dimensions: access, quality and monitoring & evaluation. The overarching goal of the reform (the ‘impact’) was to provide quality education to the children in the Punjab through a strengthened and well- functioning provincial education system. It aimed to achieve this by producing three major outcomes:

• 1. Greater access to schools for school-aged children.

• 2. Improved quality of education being imparted in public schools.

• 3. Robust monitoring & evaluation mechanisms for evidence-informed service delivery and improved accountability, embedded in each layer of the system.

A visual representation of this relationship is given in Fig. 5.2 above, following the same color-coded themes as defined in Fig. 5.1.

Fig. 5.2

From outcomes to impact - theory of change of the reform

5.3.3 Interventions

To achieve its ambitious goals under the impact and outcomes statements described in the theory of change above, PSRR developed a portfolio of ambitious yet achievable activities, each targeting a series of outputs that would ultimately and collectively enable the achievement of the outcomes described above. This portfolio of interventions has been re-constructed for this study in consultation with stakeholders and through secondary research. The full portfolio can be found in Table 5.1 in the Annex, while relevant components are elaborated on in the evaluation section that follows.

5.4 A Medium-Term Evaluation: Where Do We Stand?

Given that it has been a decade since the PSRR was launched in 2010, this section studies the outcomes of the reform and whether these outcomes have been sustained over time. The outcomes being studied are those resulting from the institutional capacity building component of the reform. Specifically, we ask whether the reform had a lasting impact on access to, quality of, and monitoring & evaluation mechanisms in the public education system through the building of management capacity, teacher capacity and monitoring systems in the Punjab. Furthermore, we situate our analyses within the five-point framework on education reform and evaluate the sustainability of the outcomes of the PSRR in the medium term according to the five perspectives of educational reform (cultural, psychological, professional, institutional and political).

We addressed our research questions conducting a series of interviews with stakeholders from a wide spectrum of organizations/ positions both within and outside of the public education system in the Punjab.¹ Each stakeholder was carefully selected to represent a specific component of the system and the agencies within, and care was taken to consider potential biases each informant might bring to our research.

5.4.1 Management Capacity

Building the capacity of management in the system was essential to all three dimensions of the reform. Improved management capacity results in a greater ability of management to scale its services (addressing access to education), to improve the quality of services (quality of education) and to utilize feedback from the system to inform the first two components (monitoring & evaluation). This section discusses the activities implemented under the reform to build management capacity and their outcomes, and the sustainability of these moving forward.

5.4.1.1 Activities and Outcomes

The reform adopted a multi-pronged approach to building management capacity. Primarily and most noticeably, it did so via the adoption of frequent accountability meetings driven by data at all levels of management hierarchy in the public education system. Secondly, it directly addressed inefficiencies in the system, such as by placing an emphasis on ensuring all hiring was now done on merit, a process that has historically suffered from political interference and nepotism in Pakistan. Thirdly, it built management capacity by increasing the number of ‘managers’ in the system, realizing that a greater workforce was necessary to implement the new and more-demanding quality standards PSRR aspired towards.

The accountability component was arguably the largest element under the PSRR, based on the ‘Deliverology’ management approach developed by Barber. Deliverology is rooted in the use of reliable, relevant and timely data to inform feedback on performance and hold the system/actors accountable (Barber et al., 2011). Under PSRR, accountability was introduced in an almost identical format to Barber’s Deliverology, employing widespread data dissemination, discussion and decisions based on data gathered under the salient ‘monitoring & evaluation’ dimension of the reform. In practice, the reform did this via the following interventions:

• 1. Generating and widely disseminating monthly data-packs assessing performance at all levels (from district to school) to all actors in the system (from the Chief Minister to head teachers).

• 2. Regular stock-take meetings at all levels of the system hierarchy to discuss progress and areas for growth, and to take action. These levels were:

  • (a) Provincial: Meetings chaired by the Chief Minister and involving all key players in the system. These occurred every month for the first year, and every 2 months thereafter.

  • (b) District Clusters: District Review Committee Meetings chaired by the District Coordination Officer and involving all Chief Education Officers of the District Education Authorities.

  • (c) District: Pre-District Review Committee Meetings chaired by Chief Education Officers and involving all Deputy District Education Officers.

  • (d) Tehsil: Meetings chaired by Deputy District Education Officers and involving all Assistant Education Officers.

  • (e) Moza: Meetings chaired by Assistant Education Officers and involving all Head Teachers in the Moza.

  • (f) School: Meetings chaired by Head Teachers and involving school teachers.

Next, under PSRR all new appointments and hiring were made on the basis of merit in order to ensure the most qualified personnel were being hired into the public education system. This was a result of the Chief Minister’s political will and personal dedication to ensuring merit-based appointments were made, holding strong accountability measures in place for any violations of such a policy.

In addition to these changes, the capacity of the system to manage public education was enhanced by increasing the number of Assistant Education Officers (AEOs) in the system. This meant that there was a greater number of AEOs per Moza, reducing the number of schools each AEO was monitoring. This allowed each AEO to dedicate more energy towards the schools they were monitoring, now also acting as mentors and conducting classroom observations to provide feedback to teachers. Under the reform, AEOs were also hired from the private sector to get a more competitive pool of candidates, whereas they were formerly hired from the teaching cadre alone.

5.4.1.2 Sustainability

This systemic restructuring of education management in the Punjab has managed to take root in the province over the course of the last decade. The Punjab benefits from a period of prolonged political stability that helped in the continuation of policies put in place under the PSRR. Due to the dynamic nature of the reform, it is hard to provide an accurate quantitative causal measure of the impact of the reform on the system in the medium term. Therefore, our understanding and analysis of the impact relies more on qualitative stakeholder interviews.

The stable leadership and political will of the Chief Minister ensured that the accountability embedded within the reform was maintained and that regular stock-taking took place. Expectedly, the continuity of policies is often a prerequisite to sustainable outcomes. Looking at it from the five-point framework on educational reform discussed in the first chapter of this book, a stable positive change in the institutional capacity of the system can be viewed from both a political and institutional perspective. Expanding on the political perspective first, the strong political will of the Chief Minister towards PSRR meant that the interests and positions of the remaining stakeholders of the education system in the Punjab were also aligned with that of the center and this alignment was effectively maintained through a strict process of accountability.

The merits (and ethics) of such a top-heavy approach notwithstanding, the effectiveness of this political approach to the implementation of the reform can also be evaluated from the institutional perspective. Increasing the capacity of the education system while also restructuring district education departments of the entire province represented a fundamental shift in how education was managed in the Punjab. The digitization of data collection with reliability and its speedy dissemination, regular checks on district performance using said data, streamlining of teacher hiring processes with a focus on ensuring meritocracy and quality (measured, as of now, through academic qualifications of applications), restructuring the roles of AEOs and the literal expansion of teaching staff as well as student enrollment drives all serve to highlight the institutional restructuring of the education system in the Punjab. The institutionalization of this reform has been made possible due to sustained political will resulting in a successful cultural change within the entire system.

However, there are still shortcomings that hamper the functioning of the system in the medium term. With a change in government after the elections of 2018, the upward trajectory appears to have plateaued under the new regime. The stock-takes are no longer taking placing (a common theme echoed by our informants) and the original top-down accountability system appears to have lost momentum. The monitoring of the system is now being conducted sporadically, and that too by third parties rather than institutionally by the state.

In terms of accountability through technology, the PSRR has proven very effective in technologically upgrading the education monitoring apparatus of the Punjab with new, efficient and real-time digital data collection. This apparatus continues to enjoy widespread use among the public education delivery chain and the direct accountability benefits (tracking learning levels and school facilities, for example) remain. The systematic use of this evidence by the state is unfortunately less prevalent in the medium term.

5.4.2 Teacher Capacity

The capacity of teachers is essential to any well-functioning education system, especially considering teachers are the system’s primary point of contact with students during their education. Building the capacity of teachers was thus particularly relevant to the quality dimension of the reform in addition to addressing access to education. This section explores the activities and outcomes of the PSRR’s efforts to build teacher capacity (directly addressing teachers under a professional frame of the reform), as well as the sustainability of these activities and outcomes moving forward.

5.4.2.1 Activities and Outcomes

To build teacher capacity, the reform employed five major activities. The first activity was the hiring and training of a significant number of new teachers to increase the strength of the workforce in existing schools, relating to both quality and access. The remaining four activities directly dealt with improving the quality of instruction in order to bring about an improvement in the quality of education being imparted in classrooms. These included the development of teacher training modules, distributing these modules/guides, training teachers to use these guides, and hiring & training teacher coaches.

One of the most visible activities under the reform was the hiring of 80,000 new teachers into the system to ensure that each primary school had at least four teachers (Barber, 2013). This was imperative to improving and maintaining productive student-teacher ratios in public schools. Improving the student-teacher ratio means that teachers are better able to manage and deliver the material to their students and have more time to devote to lesson planning and lesson preparation, improving the quality of instruction. This also means that the school is able to cater to more students due to a greater number of teachers, increasing the access to education.

To further promote an increased quality of instruction, 75 new teacher training modules were developed to facilitate the professional development of teachers and the new emphasis placed on greater quality instruction. 60,000 copies of these modules (collectively known as ‘teacher guides’) were subsequently distributed across the province and 200,000 teachers were trained on using these guides.

As an additional source of support to teachers, 4000 teacher coaches were also hired and trained. This was a new resource for teachers to support improvements in instruction in classrooms and the design of classes, moving towards the greater quality standards set under the reform. These coaches acted as mentors to teachers and also conducted classroom observations to provide timely feedback to teachers, institutionalizing these greater quality standards by expanding the professional frame of teaching to include a new layer of quality assurance and support.

5.4.2.2 Sustainability

The efforts under PSRR have developed the teacher professional frame to one that has a greater emphasis on quality of instruction and one that is becoming more meritocratic. The meritocracy has been institutionalized by introducing a standardized and independent system for the hiring of teachers, where K-12 teachers are now hired through a standardized test conducted by an independent body called the National Testing Service (NTS). This step has made the hiring of teachers more meritocratic, reducing the former political nature of such appointments. Further, the pay scales of public-school teachers have now been increased to allow for the induction of young professionals into the profession of teaching, improving the pool of teachers in this professional frame.

The teacher training facility, Quaid-e-Azam Academy for Educational Development (QAED, formerly known as the Directorate for Staff Development), has also been significantly strengthened. QAED now provides an 8-week induction program for teachers that join the workforce and a 28-day leadership training for head teachers to enable them to lead their schools, signaling that the emphasis on quality under the PSRR is still an important area to the system (Javed & Naveed, 2019).

Despite these efforts, reports of unsatisfactory learning levels of students are still being seen² and rigorous evaluations on the impact of teacher trainings have not been conducted. This means that the actual efficacy and result of these teacher trainings and greater emphasis on quality is still in question.

Moving forward, the government needs to translate its emphasis on accountability under the PSRR into the teacher professional frame as well. Impact evaluations of teaching, for example, should be embedded into the system to diagnose why student learning levels are unsatisfactory and to design specific support mechanisms for teachers as a result. Further, the content-specific knowledge of existing teachers is an area that still needs more dedicated effort, especially via regular in-service trainings (Asian Development Bank, 2019). A final recommendation is to introduce a mechanism within QAED which allows for additional teacher trainings on a per-need basis, rather than a pre-determined arbitrary schedule. This would be a direct next step to the evaluations of teachers, providing additional support to any teacher that is not performing as expected.

5.4.3 Monitoring & Information Systems

Monitoring & information systems were a crucial component of the reform upon which the salient accountability model was built. These relate directly to the education monitoring and evaluation dimension of the reform and play a pivotal role in informing access and quality decisions. The accountability component of monitoring and evaluation has been covered under the management capacity segment above, while the actual monitoring systems and data-collection processes are discussed here.

5.4.3.1 Activities and Outcomes

Strong monitoring and information systems require reliable data. PSRR achieved this through triangulating their data sources to check for accuracy and placing a large emphasis on the accountability of the data collection process itself. This was achieved by digitizing the data collection process used province-wide, conducting monthly assessments of learning, and contracting a reputable independent party to conduct biannual learning assessments with which to check the reliability of their own data.

Monitoring of the school education system in the Punjab is conducted by the Programme Monitoring & Implementation Unit (PMIU) under the SED, which monitors all public schools in the province. Before PSRR, this monitoring was a manual task whereby all data (of school facilities or student/teacher attendance) was collected by hand, manually entered onto a computer (creating a time lag and introducing human error) and then shared onward up the chain. Under PSRR, this mechanism was quickly shifted to one that was completely digitized (requiring all monitoring staff to use digital tablets for entry) and using state-of-the-art technology, such as verifying the location of data entered for a school by referencing the geographic coordinates at the point of entry of the data point against the location of the school.

In addition to routine data monitoring, learning levels of students was another salient component that had to be measured. To this end, monthly Literacy and Numeracy Drives (LNDs) were designed and conducted by AEOs in all schools to spot-check learning levels of random students across the system. With the lens of accountability, this data was then verified by contracting a reputable independent party to conduct their own biannual assessments of child learning to triangulate the state data with and verify reliability, which was crucial.

5.4.3.2 Sustainability

Efficient and real-time data collection and analysis was a major component of the PSRR. Since the data collected on district performance in education metrics also formed the basis of the stock-takes conducted by the Chief Minister, a robust and well-functioning data monitoring system equipped with the requisite technology was a requirement, if not an outright necessity. The province was successfully able to transition to a digitized model, with real time information on education statistics (including but not limited to student performance and teacher attendance) becoming freely available to the SED and the public in a record amount of time. This has persisted because of the sustained initial effort made by the government to structurally integrate this new monitoring mechanism into the operations of the system.

The monthly school performance numbers are still collected by the AEOs and made available on the PMIU website.³ There is now a District Monitoring Officer (DMO) in every district office who is tasked with ensuring continuity of data collection. The Monitoring and Evaluation Assistants (MEAs) report to the DMO who in turn ensures that the data reaches PMIU in time: this is now a regular feature of the Punjab’s education system as originally envisioned by PSRR.

This goal of the PSRR can also be analyzed through the institutional perspective of the five-point framework on educational reform. Framed as a core component of the PSRR, the upgrading of monitoring mechanisms to include robust and timely data collection and dissemination as well as its integration across all levels of the education system, the formulation of policy decisions (pertaining to student learning outcomes, teacher incentives, district management etc.) on the basis of evidence became more than a catchphrase. It has successfully become a fundamental cog in the running of the SED in the Punjab. Thus, this aspect of the reform has brought about an institutional change within the education system of the Punjab.

While data collection is still occurring in the medium term, the stock-takes have stopped taking place, primarily due to the change in government after the elections of 2018. However, the bureaucratic and accountability driven method of collecting data (via the AEOs and MEAs) may also need to be repackaged for teachers and head-teachers of public schools. One of our key informants stated that teachers continue to harbor resentments against the data collection regime, due to the fact that financial (or other) penalties can be imposed on teachers by MEAs if their school does not perform well or if a teacher is absent on the day the MEA visits the school. If the objective in the medium-term has shifted from expanding access to improving quality, then the next step has to involve reducing the trust deficit between the center and the teachers.

5.5 Way Forward

The PSRR was designed keeping in mind the specific challenges of education in the Punjab, looking to reform education by placing its emphasis on three frames that were most relevant to the context and the time. These were the institutional, professional and political. Institutionally, the reform embedded accountability into each layer of the system and built the capacity of those within the system to take action and hold the system accountable. Professionally, the reform placed a large emphasis on the teaching profession, building the capacity of teachers and incorporating support mechanisms for teachers as well as expanding the profession entirely. Politically, the reform was encouragingly cognizant of the political landscape within the province and the political powers that were distorting the system for personal gain. Together, these frames and the vision produced a reform that was highly context specific and had the potential to create real change.

Although an unfortunate learning crisis is still seen in the Punjab today, the education landscape in the province has drastically grown from where it was a decade ago. Large elements of success and structural transformation have taken place over this past decade, many of which appear to be both sustained and sustainable in the foreseeable future. At the core of the reform, we see a push in the system towards a deep accountability in all decisions that is informed by robust, accurate and timely data to provide quality education in the province. Further, we see increased capacity of management in the hierarchy to conduct diagnostics of the performance of their constituencies, which is a powerful tool. Lastly, teachers now being aware that their performance is constantly being monitored, through frequent school visits for example, is a tool that still holds significant power.

While the above is true, the work started under the PSRR now faces significant risks due to the reform being centralized in nature. The PSRR thrived due to strong accountability from the top, specifically due to the Chief Minister being personally invested in the reform. With the 2018 elections and the change in governance, the emphasis on accountability from the top seems to have faded; worryingly, this fading emphasis on accountability currently seems to be trickling down to the rest of the system. Monitoring is still taking place and robust data is still being collected, however the actions taken on this evidence are fewer, which is reducing the accountability within the system that took much effort to build.

This waning accountability from the top is not entirely surprising, given governance changes in a democratic system that are brought on when a new political party assumes power. This is why a reform should be robust to change in governance. In the short term and to keep the system from regressing to its prior state, one of the key pieces of advice we received from our informants was the need for a dedicated person ideally higher up in the hierarchy of the education delivery chain who would continue this unrelenting focus on evidence and accountability. At the same time and for a more long-term solution, a system of accountability must be developed and implemented that is robust to change in government and leadership. Each change in governance should not be a threat to the right to quality education for the Punjab’s children. This means further research must be conducted into embedding accountability (ideally in a developing country context) from the ground-up, instead of limiting it to elements at the top of the management chain.

Annex

Table 5.1 Breakdown of the impact, outcomes, outputs and activities under the PSRR