Introduction

This chapter explores some of the governmental implications of particular uses of policy relevant data and evidence for health policymaking in Ghana. In particular it looks beyond issues of technical capacity to include issues of political responsibility—namely responsibility to use evidence, to take it into account, and to account for it. It is worth noting that claims that evidence can improve accountability practices and democratic decisions are seen within some current calls for ‘evidence-based policymaking’ (EBPM) (c.f: Weisburd and Neyroud 2011; Clarence 2002), but the way such improvement should occur remains vague and under-investigated. From a general decision-making perspective, the use of evidence serves to inform decisions and to make them more ‘rational’. In addition, explicit use of evidence potentially improves transparency and along with it the accountability of decisions. However, accountability is not an inherent property of decisions. It is a practice put into being by policy agents and procedures.

This chapter uses a case study of Ghana in order to explore how uses of evidence for national planning link with concepts of accountability, and what political effects particular uses of evidence can produce. The analysis focuses on a specific stage of the policy cycle, namely the evaluation process of health policies and programs. Policy evaluation has indeed been assigned an ever important role in policymaking through the idea that decisions are ‘better’ when they can be tested (Weiss 1999). Under this view, policy evaluation combines governance needs for more efficient policy outcomes with legitimacy quests for more democratic decision processes and outcomes. Evidence plays a prominent role in enabling such tests and validating decisions according to their outcomes. Indeed, the mechanisms by which evidence from evaluation systems work to inform future policy choices can be seen as a process by which technical measurements of policy achievements take on political value in shaping policy directions. But questions remain about how such translation occurs in practice; and how it connects concepts of efficient and legitimate policymaking. These are important questions which can be obscured by the apolitical calls for ‘evidence use’ in policymaking. In this chapter, the term ‘evaluation’ principally refers to data related to performance monitoring, but also refers to how such data are used to evaluate policy choices as well.

As a parliamentary democracy and an aid-recipient country,Footnote 1 Ghana provides an interesting arena in which to investigate the democratic implications of evidence use in policymaking, namely by applying the concerns, which have traditionally been applied to more high-income countries, over the expansion of expert-based decision-making structures outside national polities (Barnett and Finnemore 1999) and the consequent balance between governance improvements and legitimacy decisions.

In order to investigate the practical applications of evidence use in policy evaluation in Ghana, and its links to accountability, this paper combines theoretical and empirical considerations as follows: First, we briefly review theoretical arguments drawn from public administration and policy studies to understand how the concepts of accountability and evidence use have been associated in governance and policymaking studies. We then investigate such association through empirical analysis of health policy in Ghana. Finally, we discuss the theoretical contributions in light of both our empirical findings and sociological approaches to policymaking.

Our empirical analysis is principally informed by a set of 24 in-depth interviews following a semi-structured approach, conducted in 2014 with a set of stakeholders in Ghanaian health policy—including representatives of the Ministry of Health (MoH), the Ghana Health Service (GHS), international development partners (DP), local nongovernment organizations (LNGOs), and members of parliament (MP). When specific interviewees are cited in this paper, they are designated by an anonymous number and one of these acronyms. The interviews aimed to understand the institutional role and position of each interviewee with respect to other actors in the health sector, as well as their perception and understanding of evidence use in health policymaking. Data analysis partly benefitted from the use of qualitative tools, such coding using the Nvivo qualitative software package, and from the triangulation with other sources of data including official documents.

Evaluation and Accountability Within a Policy Space

In its simplest form, accountability links the capacity to evaluate decision outcomes to the idea of controlling political agency (Dubnick and Frederickson 2011). As per the principal-agent model, the elected principal confers the delegated administrative ‘agent’ the power to apply directives, while endowing the agent with a margin of discretion. Hence, the need to oversee her decisions by making the agent accountable for them (Pratt and Zeckhauser 1991). In principle, evidence helps operationalize accountability rules: by informing decisions within the range of discretion that authorities have over decisions, and by defining the legitimacy of decisions.

Under this conceptualization of accountability and evidence use, the process of evaluating decisions becomes more than a simple managerial or technical function. Rather, evaluation evidence serves as a powerful tool for testing the achievement of policy objectives, directing policy discussions, validating a particular policy strategy, and rewarding it by allocating more funds or prolonging its life cycle. However, the contribution of evidence to accountability depends on the extent to which evidence also exposes decisions to judgment and contestation (Heidelberg 2015).

Precisely for these reasons of multiple functionality of evidence use, it is critical to analyze how pieces of evidence shift from being used for simple technical measurement, to more normative judging (valuation) of policy. Applying an institutional lens, however, means that we are particularly interested in how this shift can often occur within a specific policy space created by formalized evaluation processes. These insights provide a framework in which we can analyze how the policy evaluation process provides space for participation and contestation among stakeholders over the use of evidence to judge policy value. It further allows reflection on how rules of accountability within those evaluation processes serve to establish power relations and set the spaces through which such contestation takes place.

Evidence and Health Sector Assessment in Ghana

Ghana is a lower-middle income country located in Western Africa. It is often considered one of the more democratic and developed of Sub-Saharan African nations, but it still suffers from significant resource limitations. The structure of the health system in Ghana follows the basics of functional separation between decision making and implementation in policymaking (Cassels 1995). Due to concerns over efficiency, some functions traditionally concentrated in the MoH were delegated to technical agencies benefitting from a certain degree of independence from government and discretion. The GHS is an autonomous Executive Agency of the MoH and represents one of the most important policy implementation bodies in the health sector, responsible for managing and operating all public health facilities and tasked with planning, implementation, monitoring and performance assessment of health programmes and services (Adjei 2003). The GHS has considerable power in the health sector. It was set upon the managerial objective of improving service delivery in Ghana, namely by deconcentrating the vertical structure of programs under the MoH (e.g., HIV, TBA, etc.) into local units of management and implementation (Cassels and Janovsky 1992). In practice, however, the GHS provides a parallel structure of hierarchical governance (Cassels and Janovsky 1992) serving the broader political objective of bringing coherence into the health system.

The Health Information Management Department (HIMD) of the Policy, Planning, Monitoring, and Evaluation (PPME) division was established within the GHS as the focal unit responsible for the collection, analysis, reporting, and presentation of health service information (Adjei 2003). Regional and the District level offices were established with each having their own Health Administrations (RHAs and DHAs) and each should report to the higher hierarchical level (Adjei 2003) (see Fig. 4.1) (Couttolenc 2012). In spite of some problems—e.g. overlapping responsibilities at times emerging across managerial units and local political authorities (Couttolenc 2012),Footnote 2 a fairly well formalized structure of accountability exists (within the GHS and between the GHS and the MoH), integrated with a systematic practice of reporting and reviewing performances of implementation policies, as widely acknowledged by our interviewees at the GHS.

Fig. 4.1
figure 1

The governance structure of health policy in Ghana. (Adapted from Adjei 2003, regional district administration, GHS, http://www.ghanahealthservice.org/ghs-subcategory.php?cid=&scid=44 Others: http://solutionscenter.nethope.org/case_studies/view/district-health-information-management-system-dhims-ii-the-data-challenge-f)

Full size image

The connection between evidence use and accountability in the Ghana health system can be seen in the integration of the Health Information Management Department (HIMD) within this national system of accountability. The HIMD’s specific task is to gather health information such as administrative, demographic, and clinical data—typically collected through desk review, although sometimes accompanied by interviews (Zakariah 2014). This is fed upwards from facility to district to region and, ultimately, to central health management levels in order to inform health sector performances (for more details see Ghana Health Service 2012, p. 30). The Centre for Health Information Management within the HIMD collects the data from the district level through the software called the District Health Information Management Information System, and then sends it to the regional level. The aim of this procedure is to collect information from the district up to the national level in order to support each Ministerial Agency within the health sector—not only the GHS—with the implementation of their respective strategies established in their ‘Programmes of Work’ (POWs). Each Agency assesses its progress in implementing the POWs through an in-house Monitoring & Evaluation (M&E) plan, which relies on the information produced by the HIMD. The results of M&E outcomes should finally converge each year into the Interagency/Health Sector Performance Review.

Therefore, a combined mechanism of information diffusion and evaluation of performance exists in Ghana that ties the health governance structure into a system based on a structured review process: operating internally at each administrative level and vertically between district, regional, and headquarter managers via peer-review meetings.

Accountability, Evaluation and Power in Ghanaian Health Policy

In addition to the process described above combining information and evaluation within the Ghanaian hierarchy, a ‘Holistic Assessment tool’ has been developed to guide interagency performance review. The Holistic Assessment tool was established within the framework of the Common Management Arrangement (CMA), which governs and set the rules for partnership between the MoH and international donors.

As we will discuss, the tool functions as interface between the MoH, responsible for the health sector performance, and international donor agencies, which demand accountability of performances to the MoH.

The CMA was conceived to address the problem of parallel donor systems and increased aid transaction costs. Now in its third iteration (CMA III), the CMA itself was originally introduced in 1997 with the national health sector reform of decentralizing service delivery—the creation of the GHS being one of the main outcomes—under the sponsorship of the World Health Organization (Ghana Health Sector 2012, p. 5). A health-sector-wide approach was established along with a pooled funding account of donor funds managed by the MoH (Pallas et al. 2015). The method to govern this new framework of collaboration was the Holistic Assessment (IHP+ 2003).

The use of sector-wide indicators, milestones, and targets are a key component of the Holistic Assessment tool. These are established at the national level within the four-year Health Sector Medium Term Development Plan (HSMTDP) and are (re-)formulated each year with the programme of work (POW) that the MoH prepares in line with the objectives of the national health strategy as set in the HSMTDP.Footnote 3 Milestones, targets, and indicators at the local level are derived from national equivalents. The data generated by the HIMD from the district to the national level are devoted to fill sector-wide indicators specified in the HSMTDP from which health sector agencies draw their own POWs and implementation strategies (Nyonator et al. 2014). Targets, are negotiated between the GHS and relevant decentralized bodies to administer DPs’ funds across the national, regional and district level.Footnote 4

The Holistic Assessment tool is also used by international partners to assess health sector performances. Thus, international donors are involved in the process of selecting indicators, targets and milestones. Information appears to indicate that performance indicators get established and revised each November of the year during the Business Meeting between the MoH and DP (Ghana Health Sector 2012).Footnote 5 Based on these indicators, the Holistic Assessment reports a score for each health sector objective established within the annual POW, e.g., bridging equity gaps in health care, improving efficiency, and effectiveness in the health system. A score of +1 is attributed if the indicator has attained the set target, 0 if it just show a good trend, −1 if the target has been missed (IHP+ 2003, pp. 37–38).

Interviews conducted with both administrative officials of the MoH and DP confirmed that, besides the senior managers meeting at the GHS, the main venue for research dissemination and discussion of evidence was the Health Summit—the annual meeting in which DP and government discussed the Holistic Assessment of the health sector (Interviews MoH-1, DP-1, DP-2, DP-3). Indeed, according to one MoH official, the Health Summit is “the key policymaking structure within the sector” (MoH-1). The presentation of the Holistic Assessment to the Health Summit is to provide the mechanism for all sector partners to review performance and assess the level of compliance with the CMA. However, this mechanism of data utilisation serves another purpose besides coordination between different levels of health system governance: it makes the system evaluable by external reviewers. However, the CMA clearly states that the use of the Holistic Assessment tool should be made “in line with the principles of mutual accountability” between the MoH and the donors (Ghana Health Sector 2012, p. 7—emphasis added) showing that the Health Summit represents not only an additional venue of evaluation, but also an additional system of accountability in which the MoH is accountable to DP for the overall performance of the health sector.

The Accountability Implications of Uses of Evidence

The creation of a second system of accountability external and parallel to the hierarchical structure of management of the Ghana health system could in theory be managed through the use of common tools for evaluation, which might ensure the alignment and coordination between the two systems of accountability (and evaluation). However, the scores attributed to each target in the Holistic Assessment entails some political judgment of success and failure, raising questions of responsibility and authority to decide on these outcomes (Bovens et al. 2006). Having two accountability systems driven by different stakeholders thus can make it unclear to whom responsibility and liability issues should be referred.

The World Health Organizatoin’s description of Ministries of Health as having key ‘stewardship’ roles in country health decision making (World Health Organization 2000) could imply that the MoH is responsible for the health sector performance and should have overarching authority over relevant stakeholders, active in the health sector including NGOs, international donors, health agencies, academics, health associations, etc. In practice this is not the case, as each stakeholder has its own power to influence the outcome of policy evaluation and, accordingly, influence or bypass accountability structures in place. The ability to do this, in turn, depends on the capacity of each stakeholder to use evidence as a tool for applying its scrutiny to policy performances and defining its discretion in guiding future policy directions; hence, evidence appears as a powerful tool for stakeholders to negotiate their own position with respect to the other participants.

Technical Evaluation and Accountability Mechanisms

Indicators aim to ‘indicate’ (rather than prove) whether some programmatic situation is still relevant to be considered within a certain policy perspective or whether new situations have emerged that affect policy trajectories. As in the case of Ghana, the HIMD has primarily the duty to ‘fill’—rather than create—indicators, as indicators are generally established by international bodies such as the WHO; however, our interviews stated that some margin of discretion over data selection always exists, especially when data are lacking. Also, discretion exists in the very use of indicators to produce reports and draw political attention. So, for instance, one interviewee indicated that the Director General of the GHS can request specific data or indicators that do not fall into the HSTMDP (GHS-1). The discretion over data and indicators could also be seen in the way that particular pieces of data, or particular results of analysis, are promoted by bureaucrats within the HIMD to influence policymakers (GHS-1).

The discretion in filling indicators with data is not a problem per se, neither is selecting specific indicators to promote political awareness over certain issues; on the contrary, discretion is a typical characteristic of technical agencies supposed to simplify very complex situations and enhance the quality and pace of policy decisions. However, discretion raises questions if it is exclusively driven by bureaucracy, in the absence of political engagement to use information in a way that reflects political priorities. As a general consideration, this is a technical problem of managerial accountability relationships: in a typical principal-agent perspective, the ‘principal’ should guide the ‘agent’ in the implementation of policy objectives (Pratt and Zeckhauser 1991). In the case of Ghana, an additional concern relates to the fact that the production of indicators and the political values built into them will be used as a policy tool for negotiation (i.e., the Holistic Assessment tool) during the Health Summit. The holistic assessment of progress is indeed meant to be presented and discussed during the Health Summit and negotiated and agreed upon by the MoH and Partners at the immediately subsequent business meeting in April (Ghana Health Sector 2012, p. 20). Indeed, the outcome of the health sector assessment serves as the basis for discussing the Performance Assessment Framework (PAF) for Multi-Donor Budget Support during the business meeting following the Health Summit (Ghana Health Sector 2012).Footnote 6 Therefore, the CMA sets the framework for both constructing evidence—by specifying how the Holistic Assessment tool should be used—and deciding which evidence should be taken as relevant for future planning.

The case of Ghana shows that the use of evidence does not respond to a purely informative concern of enhancing the quality of decisions and anticipating the consequences of actions; it also responds to the need to justify decisions at the moment of the Health Summit, hence to negotiate the value of the actions that may follow (Boltanski and Thévenot 2006). Further, the power of DP to influence the selection and evaluation of indicators influences the outcome of the negotiating process and the setting of future policy directions. Indeed, the capacity that stakeholders have to influence each other’s views often reflects an adversarial process, in which the construction of policy meaning occurs through negotiation between competing views over policy performances and the subsequent judgments on future policy directions (Bovens et al. 2006). Evidence can be used to arbitrate such adversarial process, but at the same time, where disparity emerges as to the capacity to employ it, evidence can end up determining policy directions.

Excluding coercion, the power that each actor has to influence the process in which policy value get shaped partly rests in the way accountability structures establish common rules for participation and value discussions. These rules, in turn, get operationalized by stakeholders through the selection, activation, and evaluation of policy evaluation tools (Pearce et al. 2014). And indeed these tools create the conditions for setting different types of public spaces of discussion while realistically admitting only those participants with the capacity to provide insights and feedback. For instance, one of our interviewees from the MoH complained about the superior technical capacity of DP to produce evidence of performances (GHS-1). This asymmetry is problematic in two respects: on the one hand, Ghanaian officials have little capacity to enter the technical discussion, hence to raise issues of political relevance connected to them, due to a lack of counteracting arguments.

On the other hand, the absence of clear problem setting and policy directions established by the MoH makes the discussion dominated by technical considerations of policy implementation performance. Interviewees from both DP and NGOs (DP-3 and LNGO-1) recognized such absence as problematic. In the wording of one representative of an international agency (DP-3), health policy in Ghana is only conceived in operational and strategic terms by the government and never in terms of policy objectives; accordingly, indicators are set only in the form of outcomes (e.g., how many new hospitals have been built?) rather than impacts (e.g., how much child mortality has diminished?). On a different level, an NGO representative (LNGO-1) explained that the Ghana Coalition of NGOs in Health (http://www.ghanahealthngos.net/) has recently decided to challenge the government on health priorities by creating a concurrent space of advocacy and evidence use; the objective being to produce an alternative evidence-based report and submit it to the parliament select committee on health in order to influence health financing. However, it has been reported that the Parliament has very little power in influencing actual health policy outcomes (Ayee 2002) as some of our interviewees confirmed. It was explained, for instance, that the Parliament lacks the financial resources to /commission its own inquiries and studies, which could allow it to have greater say in the direction of health policy. This makes Parliament’s influence over domestic policy dependent on external aid provided directly to the legislature, for instance for organizing meetings with the civil society (DP-3, MEP-1 DP-1); at the same time, such dependence renders the Parliament practically impotent to have a say in the approval of sectoral budgets (Ayee 2002).

Therefore, evidence use in this policy space centres mainly on the use of common indicators, which might fail to link evaluation to ‘accountable’ (and potentially more democratic) decisions. The reason draws precisely on the duality of both evidence use and accountability relationships. In the first case, evidence use is both an informative and justificatory policy tool; in the second, accountability relationships envisage at the same time reporting on performances and policy achievements, and exposing performances to some judgments and deliberation.

Evidence Use and Policy Value Judgements

As much as a practical investigation on the use of evidence in policy evaluation has revealed the existence of structures of power, it has also revealed that policy evaluation is not only a technical process of assessment, but a political process of value formation and judgment. In the case of Ghana, the health policymaking process sees the two typical phases of policy evaluation: (i) evidence synthesis and (ii) learning (i.e. evaluation and valuation), disjointed into two separate spaces of accountability. One is structured around a decentralized structure of governance, whereas the other relies on the partnership between donors and the MoH. Whereas the use of evidence—inscribed in the Holistic Assessment tool—is in principle envisaged in bringing these two spaces together, they remain separated. This situation demonstrated that the relation between technical evaluation and political accountability is not always linear, nor can it be fixed by calls for more evidence use. Instead, looking at the systems of accountability in place along with the practices inscribed into them is crucial to understand the democratic implications of evidence use. In conducting policy evaluation, accountability relationships are important in that they help to set, or reconfigure, the power and influence between stakeholders and decision makers. Accountability is a quite elusive but powerful concept that, broadly speaking, indicates the range of responsiveness of policymakers to stakeholders’ interests and values. During—and in contribution to—policy evaluation, interests and ideas of stakeholders become actionable through the way policy outcomes are assessed and policy directions are then selected or altered.

Further, by making political values exposed to public judgment and contestation (Heidelberg 2015), accountability goes beyond simplistic checks of stakeholders’ interests and enables a process of continuous reconstruction of political values. In this sense, accountability structures are important not only to shape authority relationships, but also to activate a social mechanism of participation in which the principles of an ideal relationship between those who govern and those who are governed (Bovens 2010; Flinders 2011; Heidelberg 2015; Koppell 2005; Salminen and Lehto 2012) are continuously recreated each time they are ‘tested’ against legitimacy considerations (Rosanvallon 2011). In turn, the capacity to use knowledge and evidence becomes crucial to (re-)organize such principles through mechanisms of responsiveness and degrees of scrutiny over policymakers’ decisions; hence, crucial to operationalize accountability.

Conclusions

In this chapter, we have explored the implications of the use of evidence for democratic decision making by looking at the accountability structures in Ghana. The Health Summit in Ghana revealed the importance of the policy evaluation stage within the policymaking process for understanding the effects of the use of evidence in policymaking—in terms of how the use of data goes beyond technical measurement, and serves to establish political judgements and affect policy choices. In this process, we found that use of evidence is a means through which power is exercised. Also, the use policy evaluation within the Health Summit reveals the role of evidence use in both informing and justifying decisions and its relevance for understanding how accountability relationships matter in structuring power relations. The structure of the accountability relationship, therefore, provides the basis for discussing issues of democratic decision making connected to the use of evidence in policymaking. Indeed, the involvement of international donors, responsible for funding a significant amount of health services, can challenge national structures of authority and accountability that exist within the existing governance structure of the state.

International donors often champion the language of evidence based policymaking, while simultaneously embracing the language of good governance and democratic representation in aid-recipient countries. As the case of Ghana has showed, however, it is important that donors involved in processes of evidence use to inform policy—either by extracting local data to generate their own assessments or in constructing indicators to serve as evaluation tools—consciously consider the potential implications these practices have over local accountability mechanisms along with possible legitimacy concerns. Data created to evaluate (or monitor the performance of) a health sector’s functioning may often be described as purely technical tools. Yet when such data are used to inform policy and planning, they can have direct political implications, reflecting the political realities of decision making and planning structures in a country. As shown here, their use can create new accountability systems and thus raise questions over governance and influence over local policy decisions.