Measuring Policy Capacity Through Governance Indices

Abstract

While the conceptualization of policy capacity and its application to governance performance have been addressed in the academic literature, existing governance indices appear not to consider policy capacity in its many nuanced forms. This shortcoming may be perpetuating incomplete accounts of governance quality within a diverse and growing group of indices. This chapter surveys five commonly used indices to determine whether and how they measure policy capacity. Two of the indices, the Worldwide Governance Indicators and KPMG Change Readiness Index , address broad measures of governance in a globally comparative context. The remaining indices—the Sustainable Governance Indicators , Global Innovation Policy Index , and Bertelsmann Transformation Index —target particular dimensions of governance. This chapter argues that policy capacity is relevant across many types of indices, and therefore deserves closer attention. In particular, the chapter illustrates how a robust framework measuring policy capacity, proposed by Xun, Ramesh, and Howlett, can be used to identify areas in which governance indices inadequately account for capacity. This chapter is in three parts. After a brief introduction, the first part tabulates measures of capacity within selected indices using elements of the framework as an analytical template. The second part compares tabulation results across all five indices, and the final part advocates a more robust consideration of capacity based on the identified shortcomings of the observed indices.

Appendix

1.1 Abbreviations

ADB:

African Development Bank Country Policy and Institutional Assessments

AFR:

Afrobarometer

ASD:

Asian Development Bank Country Policy and Institutional Assessments

BTI:

Bertelsmann Transformation Index

CCR/FNT:

Freedom House

EIU:

Economist Intelligence Unit

FSI:

Failed States Index

GCS:

Global Competitiveness Report

GE:

Government Effectiveness

GII:

Global Integrity Index

GWP:

Gallup World Poll

IEF:

Index of Economic Freedom

IFD:

IFAD Rural Sector Performance Assessments

ILO:

International Labour Organization

IMF:

International Monetary Fund

IPD:

Institutional Profiles Database

LBO:

Latinobarometro

OE:

Oxford Economics

PIA:

World Bank Country Policy and Institutional Assessments

PRS:

Political Risk Services International Country Risk Guide

RL:

Rule of Law

SGI:

Sustainable Governance Indicators

UNCTAD:

United Nations Conference on Trade and Development

UNDP:

United Nations Development Programme

UNESCO:

United Nations Educational, Scientific and Cultural Organization

UN HDI/A:

United Nations Human Development Index

V&A:

Voice and Accountability (WGI)

VAB:

Vanderbilt University’s Americas Barometer

WB:

World Bank

WB DB:

World Bank Doing Business Index

WCY:

Institute for Management and Development World Competitiveness Yearbook

WEF:

World Economic Forum

WEF GCI:

World Economic Forum Global Competitiveness Index

WGI:

World Governance Indicators

WMO:

Global Insight Business Condition and Risk Indicators

1.2 Indices

WGI

Organizational information capacities
Matrix	WGI (measure; data source)
Storing and disseminating information on client need
Service utilization
Budgeting, human resource management, e-services	Reliability of basic economic and financial statistics (e.g. national accounts, price indices, foreign trade, currency and credit, etc.) (V&A; IPD) Reliability of State-owned banks’ accounts (V&A; IPD) Quality of budgetary and financial management (GE; ADB, ASD and PIA)

Administrative resource capacity
Matrix	WGI
Funding, staffing	Quality of bureaucracy/institutional effectiveness (GE; EIU) Bureaucratic quality (GE; PRS) Quality of the country’s bureaucracy (GE; WMO) Policy consistency and forward planning (GE; WMO) Quality of public administration (GE; ADB, ASD, and PIA) Efficiency of revenue mobilization (GE; ADB, ASD, and PIA) Resource Efficiency (GE; BTI) Policy direction is not consistent (GE; WCY)
Levels of intra- and inter-agency communication, consultation, and coordination	Dialogue between government and rural organizations (V&A; IFD)

Accountability and responsibility system capacity
Matrix	WGI
Presence of rule of law	Do the representative Institutions (e.g. parliament) operate in accordance with the formal rules in force (e.g. Constitution)? (V&A; IPD) Enforceability of contracts (RL; EIU) Speediness of judicial process (RL; EIU) Property rights and rule based governance (RL: ASD) Rule of Law (RL; CCR/FNT) Judicial framework and independence (RL; CCR/FNT) Judicial Accountability (RL; GII) Rule of Law (RL; GII) Law Enforcement (RL; GII)
Transparent adjudicative and career systems	Transparency of government policymaking (V&A; GCS) Is the State economic policy (e.g. budgetary, fiscal, etc.)… communicated? (V&A; IPD) Is the State economic policy (e.g. budgetary, fiscal, etc.)… publicly debated? (V&A; IPD) Degree of transparency in public procurement (V&A; IPD)

Organizational political capacity
Matrix	WGI
Effective civil service bargain	The public service is not independent from political interference (GE; WCY) Judicial Independence (RL; GCS) Independent Judiciary (RL; BTI)
Politicians’ support for the agency programs and projects
Levels of inter-organizational trust and communication	Consensus Building (GE; BTI)

Political-economic system capacity
Matrix	WGI
Presence of public legitimacy and trust	Accountability of public officials (V&A; EIU) Transparency of government policymaking (V&A; GCS) Democratic accountability (V&A; PRS) Institutional permanence: An assessment of how mature and well-established the political system is (V&A; WMO) Representativeness: How well the population and organized interests can make their voices heard in the political system (V&A; WMO) How much do you trust the parliament? (V&A; AFR) Election Integrity (V&A; GII) Trust in Parliament (V&A; LBO and VAB) Trust in Government (GE; LBO) Trust in Judiciary (RL; LBO) Trust in supreme court (RL; VAB) Trust in justice system (RL; VAB) Confidence in honesty of elections (V&A; GWP)
Adequate fiscal system to fund programs and projects	Reliability of State budget (completeness, credibility, performance…) (V&A; IPD) Reliability of State accounts (completeness, audit, review law…) (V&A; IPD)
Access to information	Public Access to Information (V&A; GII)

SGI

Policy analytical capacity
Matrix	SGI
Knowledge of policy substance	Government office expertise (16.67% of interministerial coordination sub-index)
Analytical techniques and communication skills

Organizational information capacities
Matrix	SGI
Storing and disseminating information on client need	Government office gatekeeping and ability to “return items” for policy consideration (16.67% of interministerial coordination sub-index) Quality of regulatory impact assessment process (33.3% of evidence-based instruments sub-index) Inclusion of sustainability checks in regulatory impact assessments (33.3% of evidence-based instruments sub-index)
Service utilization
Budgeting, human resource management, e-services

Knowledge system capacity
Matrix	SGI
Presence of high quality educational and training institutions	Education policy (50% of education sub-index) Upper secondary attainment (10%) Tertiary attainment (10%)
Opportunities for knowledge generation, mobilization and use	Decision-making backed by strategic planning (50% of strategic capacity sub-index) Decision-making backed by scholarly advice (50%) Adaptability of domestic government structures to external developments (50% of adaptability sub-index) Institutional reform in strategic capacity (50% of organizational reform sub-index)

Managerial expertise capacity
Matrix	SGI
Strategic management	Self-monitoring by actors about appropriateness of institutional arrangements (50% of organizational reform sub-index)
Leadership, communication, negotiation and conflict resolution	Coherent communication about ministry activity and policy plans (100% of policy communication sub-index)
Financial management and budgeting

Administrative resource capacity
Matrix	SGI
Funding, staffing	Government efficiency (14.29% of implementation sub-index) Adequate task-related funding delegated by central government (14.29%)
Levels of intra- and inter-agency communication, consultation, and coordination	Line ministry involvement of government offices in policy proposals (16.67% of interministerial coordination sub-index) Effectiveness of coordination between ministries and civil servants in policy proposals (16.67%) Constitutional discretion for sub-national governments (14.29% of implementation sub-index) Sub-national governments held to national standards of public services (14.29%) International coordination (50% of adaptability sub-index)

Accountability and responsibility system capacity
Matrix	SGI
Presence of rule of law	Legal Certainty (25% of rule of law sub-index) Judicial Review (25%) Appointment of Justices (25%) Corruption Prevention (25%)
Transparent adjudicative and career systems

Political acumen‎ capacity
Matrix	SGI
Understanding of the needs and positions of different stakeholders
Judgment of political feasibility	Application of regulatory impact assessment (33.3% of evidence-based instruments sub-index)
Communication skills

Organizational political capacity
Matrix	SGI
Effective civil service bargain	Monitoring of ministries for effective implementation (14.29% of implementation sub-index) Monitoring of agencies and bureaucracies for effective implementation (14.29%)
Politicians’ support for the agency programs and projects	Ministerial compliance and presence of incentives for ministers to implement programs (14.29% of implementation sub-index)
Levels of inter-organizational trust and communication	Informal coordination among government organizations (16.67% of interministerial coordination sub-index)

Political-economic system capacity
Matrix	SGI
Presence of public legitimacy and trust	Popular decision-making (20% of electoral processes sub-index) Civil Rights (33.33% of civil rights and political liberties sub-index) Political Liberties (33.33%) Non-discrimination (33.33%) Societal consultation (100% of societal consultation sub-index)
Adequate fiscal system to fund programs and projects
Access to information	Media access (20% of electoral processes sub-index) Media Freedom (33.33% of access to information sub-index) Media Pluralism (33.33%) Access to Government Information (33.33%)

GIPI

Knowledge system capacity
Matrix	GIPI
Presence of high quality educational and training institutions	Higher education R&D performance (30% of science and R&D policy indicators sub-index; UNESCO)
Opportunities for knowledge generation, mobilization and use

Accountability and responsibility system capacity
Matrix	GIPI
Presence of rule of law	Contract enforcement (10% of domestic market competition and entrepreneurship indicators; World Bank) Efficiency of legal framework in challenging regulations (5%) Integrity of legal system (15% of intellectual property rights sub-index; WEF) Legal environment (10% of digital policy indicators sub-index) Corruption perceptions index (20% of government procurement policy sub-index; Transparency International)
Transparent adjudicative and career systems

CRI

Knowledge system capacity
Matrix	CRI
Presence of high quality educational and training institutions	Quality of Education (WEF GCI) Secondary Enrolment Rate (WB) Tertiary Enrolment Rate (WB) Availability of Research and Training Facilities (WEF GCI) Extent of Staff Training (WEF GCI)
Opportunities for knowledge generation, mobilization and use

Managerial expertise capacity
Matrix	CRI
Strategic management	Quality of Management Schools (WEF GCI)
Leadership, communication, negotiation and conflict resolution
Financial management and budgeting	Quality of Math and Science Education (WEF GCI)

Organizational information capacities
Matrix	CRI
Storing and disseminating information on client need
Service utilization	Civil Service (FSI)
Budgeting, human resource management, e-services

Administrative resource capacity
Matrix	CRI
Funding, staffing	Wastefulness of Government Spending (WEF GCI)
Levels of intra- and inter-agency communication, consultation, and coordination.	Hybrid (Time to Deal With Construction Permits (WBDB) Cost to Deal With Construction Permits (WBDB)) Government Services for Improved Business Performance (WEF GCI)

Accountability and responsibility system capacity
Matrix	CRI
Presence of rule of law	Rule of Law Index (WB) Business Costs of Crime and Violence (WEF GCI) Business Costs of Terrorism (WEF GCI) Business Costs of Organized Crime (WEF GCI) Rule of Law Metric (FSI) Regulatory Quality Index (WB) Efficiency of Legal Framework in Settling Disputes (WEF GCI) Efficiency of Legal Framework in Challenging Regulations (WEF GCI)
Transparent adjudicative and career systems	Transparency of Government Policymaking (WEF GCI) Corruption Index (WB)

Political-economic system capacity
Matrix	CRI
Presence of public legitimacy and trust	State Legitimacy (FSI) Voice and Accountability (WB) Freedom of Expression and Belief (FSI)
Adequate fiscal system to fund programs and projects	Government Budget Balance (% GDP) (OE) Gross Government Debt (% of GDP) (IMF) Resource mobilization
Access to information	Press Freedom Index (Reporters Without Borders)

BTI

Administrative resource capacity
Matrix	BTI
Funding, staffing
Levels of intra- and inter-agency communication, consultation, and coordination.	Civil society traditions Conflict intensity Policy coordination Cleavage/conflict management Civil society participation

Accountability and responsibility system capacity
Matrix	BTI
Presence of rule of law	Monopoly on the use of force No interference of religious dogmas Basic administration Separation of powers Independent judiciary Prosecution of office abuse Civil rights
Transparent adjudicative and career systems	Free and fair elections Anti-corruption policy

Knowledge system capacity
Matrix	BTI
Presence of high quality educational and training institutions
Opportunities for knowledge generation, mobilization and use	Policy learning

Managerial expertise capacity
Matrix	BTI
Strategic management
Leadership, communication, negotiation and conflict resolution
Financial management and budgeting	Efficient use of assets

Organizational political capacity
Matrix	BTI
Effective civil service bargain	Effective power to govern
Politicians’ support for the agency programs and projects	Structural constraints
Levels of inter-organizational trust and communication	Reconciliation

Political acumen‎ capacity
Matrix	BTI
Understanding of the needs and positions of different stakeholders
Judgment of political feasibility	Prioritization Consensus on goals
Communication skills

Political-economic system capacity
Matrix	BTI
Presence of public legitimacy and trust	Agreement about citizenship and acceptance of the nation-state as legitimate Performance of democratic institutions Commitment to democratic institutions
Adequate fiscal system to fund programs and projects
Access to information