Iterative Learning: ‘Knowledge for Change’?

Abstract

Chapter 5 applies the knowledge discussed in Chapter 4 to two illustrative case studies. Many interventions tend to represent a simple ‘trial and error’ approach underpinned by intensive grounded research to facilitate our understanding of change processes or change resistance. Tracking the identification of a ‘need’ and our experience of designing and monitoring the evaluation of that process, in the light of the new knowledge gained through ongoing research review, improves our understanding of the complexity of social processes. Chapter 5 redefines the objectives of our action-research project from setting out to capture the ingredients of positive change to pro-actively understanding and learning from failure. It attempts, in the context of this potentially debilitating reality, to take stock and identify the characteristics of least-harm interventions to chart the next stage of our journey.

It concludes with a series of recommendations aimed at policy makers and volunteer deployment agencies.

Introduction: Understanding Failure in Complex Interventions

Chapter 5 takes stock of the material presented in the previous chapters and the cumulative knowledge gained from 8 years’ experience of intensive action-oriented evaluation research. Our learning and ‘knowing’ as activists and researchers has been a continuous process: there was no point at which we went into the field knowledge or value-free and we have and will never reach the point of knowledge saturation (ultimate truth). Research of this nature is a journey. Perhaps the greatest single quality of any researcher (or volunteer) embarking on this type of journey is the honesty and humility to accept that we are always learning and also that, in many respects, we are not simply (or usually) measuring positive outcomes but trying to understand intervention failure and inertia. This may not be the message individual researchers, volunteers or their funding bodies want to hear. However, observing and understanding policy failure does not imply failure on the part of the actors/projects concerned. Far from it. Indeed, a growing body of research documents the specific quality of higher-level entrepreneurial learning that arises from ‘hard knocks’ or ‘discontinuous events’ (Cope 2003, 2011). In a rather different context, Mayer suggests that errors present the learner with robust feedback that can act as scaffolding for future learning (2008) and Somekh identifies ‘episodes of substantial friction’ or ‘knots’ as starting points for deeper collaboration (2006: 23).

On a broader and more ethical note, interventions and associated research are not there to benefit individual actors, be they researchers, volunteers, projects, funding bodies or ‘the industry’ (Valters 2015), but rather to ‘maximise the likelihood that knowledge generated will be ultimately of benefit to humanity’ (Richards 2015: 3).

The Sustainable Volunteering Project (in common with other similar projects and probably most Health Partnership interventions) has very broad and ambitious objectives focused on systemic improvements in maternal and newborn health through professional voluntarism. This type of project is perhaps best characterised as an example of a ‘complex intervention’. If we accept the UK Medical Research Council’s definition of complex interventions as those involving ‘several interacting components’ (MRC 2008: 6) then, as Richards argue, ‘simplicity is probably a chimera¹’ and most interventions should be considered as ‘complex’ (2015: 2). Richards clarifies this definition suggesting that complexity is not so much about the intervention itself but the questions we are posing. Where context is seen to be of great importance and where we are interested not so much in ‘does’ an intervention work but ‘how’ or ‘why’ it works (or perhaps how/why it does not work) then it must qualify as complex.

Furthermore, Hallberg argues that complexity is relevant when we are addressing interventions with several or multiple individual components and when considering how these interact with one another (2015: 17). According to these criteria, it is clear that the SVP qualifies as a ‘complex intervention’. And that intervention comprises numerous constituent interventions that have emerged in a very iterative and reflexive way throughout the duration of the wider programme. The unplanned nature of these may indicate a lack of planning or element of what the MRC refers to somewhat dismissively as ‘pragmatism’ (2008: 9). We would rather contend that this represents a necessary commitment to democratic engagement and subsidiarity supporting grounded policy engagement and development. And, a degree of ‘planned happenchance’ (Malecki 2013: 87) or serendipity (Ackers and Gill 2008: 59) has been identified as a key component of social networking, creativity and entrepreneurialism.

The chapter opens with the presentation of two case studies illustrating the kinds of learning arising from the progressive failure of specific constituent interventions. The two cases are chosen as they represent generic/foundational ‘back-to-basics’ initiatives that lie at the heart of many, if not most, Health Partnership interventions in low-resource settings. For the purposes of this chapter we have adopted the approach advocated by the MRC for the evaluation of complex interventions. This involves identifying ‘a theoretical understanding of the likely process of change’² (2008: 9); identifying outcomes and then, in a break from traditional experimental methods, assessing causal mechanisms and contextual factors.

Case Study 1: ‘Waiting Kills’: Congestion, Disorder and Patient Management

The need for improved patient management is immediately evident on arrival in any large public health facility in Uganda. Stark visible evidence of over-crowding with patients and attendants on floors and spilling over into corridors and outdoor areas confronts and shocks every recently arrived professional volunteer. Similarly, every Ugandan health professional visiting a UK facility is immediately struck by the image of orderliness and effective patient management. Triage³ is a foundational component of basic patient safety and underpins all subsequent clinical interventions (in that facility). It is perhaps no surprise that so many volunteers and HPs visiting Uganda have identified triage as a priority and sought to intervene.

Initial ‘Theory of Change’: The Development of a More Structured System to Prioritise Patients According to Clinical Need Will Improve Clinical Interventions and Patient Outcomes.

Case study 1 reports on a series of constituent interventions that together comprise a more composite and complex intervention. We have reported it here in a way that infers linearity. To some extent, the intervention did emerge over time in one facility becoming more complex and multi-faceted as the interventions progressively failed to impact and our knowledge of the context and failure dynamics deepened. However, similar interventions (with slight differences in approach reflecting the particular qualities of individual partnerships and institutional settings) all met with similar resistance and none has delivered sustained impact. In that respect, we are combining (rather loosely) a longitudinal and comparative case study approach.

In the first instance, and perhaps reflecting presumptions about roles and a profound belief in the value/logic of training (as noted in Chapter 3), a common starting point is typically the co-production/adaptation of protocols and associated short-course training.

Intervention: Skills Deficits

The identification of the ‘problem’ during the early phases of LMP intervention was that the failure to triage patients effectively is immediately and directly linked to skills deficits among Ugandan public health workers. In the language of behavioural science theories, we were focusing attention here on individual (clinical) capabilities. The operational Theory of Change at this point can be described as follows: Engagement of UK professional volunteers alongside Ugandan counterparts in the co-production of appropriate protocols and associated (CME) training and local awareness-raising will encourage effective patient triage.

At this stage, the intervention could be described as alarmingly simple involving the transmission of explicit clinical skills (how to do, record and interpret basic patient observations) in order to prioritise care. So simple, in fact, that our early interventions now look incredibly naive and must have been the source of much amusement on the part of local Ugandan managers.

Evaluation of this intervention is often achieved through straightforward ‘measuring’ tools recording the volume of staff trained, testing skills acquisition before-and-after CMEs and assessing subsequent implementation through (numerical) evidence of completion of observations in patient notes. We combined these (obligatory methods) with ongoing ethnographic fieldwork involving the project evaluator, the professional volunteers (as action-researchers), a dedicated social science volunteer and local collaborators.

Outcomes

Unfortunately, the outcomes associated with this form of intervention have been consistently disappointing. While it is common to recruit large numbers for CME training and to report strong evidence of initial skills acquisition among the cohort trained it is rare to see any evidence of sustained behaviour change. Limited short-term compliance is typically witnessed in defined areas (spaces/units) where volunteers involved in the intervention remain co-present for a period of time. Training in triage showed greater initial success in the 6-bed obstetric High Dependency Unit set up through LMP with funding support from THET. In many of the typical fly-in-fly-out CMEs where volunteer ‘faculty’ unfamiliar with the context leave immediately upon completion of training, implementation failure is unlikely to be recognised. Ongoing ethnographic observation combined with qualitative interviews and continual review of patient notes provided immediate evidence of impact failure and an emerging understanding of implementation gaps.

Identifying and Responding to Failure

In the initial stages of our intervention we were advised by local health workers that they were unable to use their new skills because they lacked resources, including stationary (for recording purposes) and basic equipment to take observations (stethoscopes and blood pressure machines, etc.). A review of parallel interventions in other settings (through literature and policy review work) identified similar outcomes and drew our attention to work on ‘neglected processes’. At this point we held a workshop⁴ in the UK inviting stakeholders from the UK and Ugandan arms of health partnerships (plus a representative from the Ugandan Ministry of Health) to share experiences. The workshop achieved some consensus on problem definition and an ostensible commitment on the part of the MOH to prioritise, integrate and roll-out protocols (such as the African Maternal Early Warning Scoring System⁵ or AMEWS). This failed to happen.

On the basis of expert advice, this led to a refined intervention involving the production of a personalised observation kit to be provided to every health worker in target locations with the aim of achieving a degree of saturation that would empower and motivate individuals to utilise their triage skills.

Intervention: Physical Resource Deficits

The revised Theory of Change was focused on the opportunities that individuals (who have received the above training) have to utilise skills: providing personal observation kits would, we hoped, enable individuals to take observations in a timely fashion and empower/motivate them to do so. And, ensuring continuous supplies of stationary (by external organisations) will facilitate compliance. In the language of behavioural science theories, we were focusing attention here on individual (clinical) opportunities. Further complementary audit tools were co-developed (and discussed with health workers) to capture use of the kits and reduce the risks of theft/misuse.

Outcomes

The intervention was a stark failure: outcomes associated with it were very limited with patchy and short-lived compliance. The co-presence of volunteers enhances compliance for the duration of their stay with almost immediate slippage once the volunteer leaves. Stationary remained a problem, especially in high-volume settings. The provision of personalised equipment failed to empower/motivate health workers and/or promote skills implementation. Personalised kits proved unsustainable even with audit systems in place.

Identifying and Responding to Failure?

The approach proved to lack sustainability as the facility resisted demands to integrate the provision of stationary within the mainstream hospital system. Kits (as valuable commodities) were (very) favourably received and immediately utilised in zones where volunteers were actively present. However, equipment was ‘lost’ and the practice of using equipment suffered (very) rapid decline.⁶

Intervention: Physical Resource Deficits (revised)

A follow-up intervention negotiated with volunteers and senior line managers (who had it has to be said actively resisted the idea of providing personalised kits) involved ‘permanently’ affixing robust equipment and clocks to the walls of a dedicated triage area aimed to overcome the risks of theft and exploit the apparent benefits of establishing discrete and manageable ‘zones’. This was supported by the institution of a simple and clear ‘traffic light’ system using colour-coded boxes affixed to the walls to prioritise patients: cases regarded as urgent were given a simple laminated red card. A US NGO later provided a high-tech electronic notice board to improve patient waiting, which soon after became dysfunctional.

Outcomes

Outcomes were initially strong with some excitement and pride in the new facility and equipment. However, compliance was short-lived. The co-presence of volunteers enhanced compliance for the duration of their stay only. Equipment became damaged and stolen almost immediately. The use of traffic light boxes became redundant as soon as the volunteer left; laminated red cards used to signify emergency cases mysteriously vanished. The continued (high volume) of staff rotations in this highly congested area made it harder to build cultural change and ensure a saturation level of training in the use of protocols.

Identifying and Responding to Failure

There was apparent confusion over the loss of equipment from the walls and disappearance of the laminated red cards (which have no monetary value). Confidential qualitative interviews with mid and senior facility managers (conducted off-site) revealed the presence of endemic and highly organised corruption ‘syndicates’. They explained that triage (as prioritisation according to clinical need) lies in profound tension with organised and systemic corruption that is effectively smoke-screened through the appearance of chaos. The ‘inert’ knowledge deficit model fails to take account of in-depth local tacit knowledge (entrepreneurial destructive/confounding knowledge) impeding implementation. We learnt that chaos is planned and highly functional. It is also systemic: attempts to improve individual behaviour through investments in training (individual capabilities) and personalised equipment (opportunities) stood little chance of altering a system functioning effectively for many health workers and managers on the basis of corruption.

There is insufficient scope here to discuss corruption in more detail (see Ackers 2014). It is important to note however that corruption not only impacts local health workers and their ability to imagine new realities but also has a major impact on volunteer deployment and impact. The following example provided by a Ugandan health worker who had worked alongside SVP volunteers is illustrative:

There is a problem with Ugandan midwives (working with volunteers). I’ve seen it. They think, ‘oh she is white she will know what to do – she can do it by herself’. There is a problem of attitude amongst us – bad attitudes which give off a bad signal. I was in [a health facility] doing a delivery with a white volunteer and there was a retained placenta and oh my goodness the local midwife made a noise in her own language. She wanted money from the patient. She didn’t want the white lady there so I said, ‘you know what, let’s go to another patient.’

[Would the midwife try to get money from a patient even at the point at which she had a retained placenta?]

Yes, of course. They are opportunists. They will look and think, ‘behind that curtain that patient has nice bedsheets’ so she won’t want the volunteer there (UHW).

The very stark conclusion we have come to is that triage will fundamentally fail to work effectively in most (larger and congested) Ugandan public facilities where accountability is absent and efforts by volunteers to support it will be met by marked resistance: triage as a method of sorting patients according to clinical need lies in immediate tension with the sorting of patients according to their ability to pay.

Adherence to basic protocols (and in particular the AMEWS) was initially stronger in the dedicated ‘High Dependency Unit’ set up by the LMP charity with support from THET. This unit had only six beds and (necessarily) higher staffing levels. Equipment in such a confined space was less likely to be lost/stolen as greater accountability was in place. It was also easier to ensure a constant supply of stationary (via project funding). However, even in this more controlled setting, whilst the act of taking and documenting observations was more common, the logic of this process is that once certain critical points in the protocol are reached a doctor must be called. Midwifery staff were failing to ‘escalate’ care and call doctors when critical observations were recorded. Ethnographic work in situ combined with confidential qualitative interviews revealed unwillingness and, in some cases, fear of following this through. In practice, the staff either could not/would not use their own mobile phones for this process (at their own cost⁷) or had grown tired of repeated fruitless attempts to contact on-call doctors. In (many) other cases, midwives had experienced heavy criticism/chastisement from doctors for disturbing them. This is an endemic problem in Ugandan public health facilities.

Our analysis of failure at this stage illustrates the importance of combining explicit clinical skills with tacit knowledge and an emerging understanding of the impact of hierarchy and power (positionality) on relationships and implementation. We are no longer dealing with individuals but with complex organisations and ‘systems within systems’ founded on subversive knowledge rather than a lack of knowledge per se.

Intervention: Human Resource System Deficits

Continued CME training provided by volunteers and local staff was introduced to manage (clinical) skills gaps caused by staff rotations (this is really a restatement of the initial Theory of Change but emphasising the importance of continuity and repeat training). Further engagement with senior managers was pursued to improve accountability, build relationships between staff cadres improving the empowerment of midwives and responsiveness of doctors. The behavioural science model may characterise this as an individual motivation ‘deficit.’ At this stage in the process we tried to combine our earlier interventions targeted at individuals with a firmer focus on advocacy and policy implementation processes. This included more active engagement with high-level stakeholders and managers to remove barriers to the escalation process, reduce the potential for corruption to influence patient prioritisation and ensure that doctors are accountable and accessible.

Outcome: This proved to be a total failure: improving the presence and responsiveness of doctors has proven to be fundamentally resistant to change in spite of managerial interventions. Corruption continues to disrupt processes and resist change. Even the highest-level managers openly admitted that they lacked the power to challenge organised syndicates, the behaviour of senior doctors and corruption dynamics.

Review and Theory Development: At this point the search for relevant research and explanatory theories fans out beyond the initial ‘simple’ intervention spanning multiple disciplines to generate new and highly complex knowledge combinations; clinical knowledge declines in significance as theories of corruption, human resource management and knowledge transfer take on a new significance. The relevance and value of systematic review declines as the search for more composite and complex theories capable of understanding wider systemic and structural dynamics becomes essential to future interventions. Fundamentally, acknowledgement of the complexities of researcher/volunteer positionality and the very limited scope for foreigner engagement in anti-corruption interventions highlights the resilience (functionality) of existing systems and the limited scope for impact from external professional volunteering and Health Partnership intervention.

This case study has shown how the implementation of something as apparently simple as triage with the potential to significantly reduce maternal and neonatal mortality (and the need for emergency intervention) whilst also substantially reducing the costs of public services has proved impossible to implement.

Grounded local knowledge achieved through ongoing ethnographic methods has enabled us to begin to understand the importance of tacit knowledge and the barriers to change. Interventions focused on individual capabilities or opportunities and individual behaviour change will not achieve systems change in public health facilities in Uganda. The powerful impact of subversive (tacit) knowledge and vested interests render the current system fully functional for the status quo. In this environment, those health workers and managers (and there are many) who genuinely hope and strive for systems change inevitably find it impossible to imagine a new reality and engage in action-planning to achieve that. And, sadly, the system will close ranks on those individuals who risk putting their heads above the parapet. Table 5.1 summarises the processes outlined before and the emphasis on deficits at each stage.

Table 5.1 Summary of triage (a ‘complex intervention’)

Case Study 2: Managing Obstetric Emergencies

The majority of UK/Ugandan Health Partnerships focus on hospital–hospital relationships.⁸ And most are headed up by senior clinicians, mainly doctors (Ackers and Porter 2011). Volunteer clinical trainees are also usually keen to be placed in large, congested, hospitals where high patient volumes and clinical complexity meet both altruistic and clinical exposure needs (Tate 2016).⁹ This, coupled with the policy emphasis and associated metrics (embedded in the Millennium Development Goals) on maternal mortality, encourages an emphasis on those facilities where the majority of recorded deaths take place.

The SVP benchmarking process collated facility-held data on maternal mortality. Table 5.2 lists the main ‘causes’ of maternal deaths in Mulago National Referral Hospital in the 12 months from January 2012 to December 2012:

Table 5.2 Causes and frequencies of maternal deaths in Mulago (Jan-12 to Dec-12)

The data presented here should not be regarded as ‘facts.’ Certainly each maternal death recorded here is a sad fact but the process of establishing (single factor) causation is highly problematic. Significant pressure fuelled by SVP volunteers has encouraged a process of maternal mortality review, but compliance across the HUB remains patchy. As such, all facility-generated data must be regarded as a social construction.¹⁰ Nevertheless, it serves one of the most important functions of quantitative data: it indicates trends and raises critical questions. During the early months of the LMP project (and very much playing the role of handmaiden to the obstetrician lead) we took it at face value that these were the causes of maternal deaths rather than the final ‘hit’ on the protracted ‘road to death’ (Filippi et al. 2005).

Faced with the alarming volume of maternal and neonatal deaths occurring in these referral facilities and the earlier ‘evidence’ on causation, it may come as no surprise that the overwhelming initial response was to advocate the introduction of (imported/amended) protocols and associated CME-style short-course training packaged under the generic title of ‘Emergency Obstetric and Neonatal Care (EmONC)’. This was certainly our experience during our early involvement with the Liverpool-Mulago Partnership and, subsequently, in the Ugandan Maternal and Newborn Hub. And, in the first instance, as non-clinicians, we lacked the experiential knowledge and confidence to challenge the dominant medical paradigm. Indeed, in our report on Emergency Obstetric Skills Training conducted by SVP volunteers, we describe the intervention as an ‘intensive two-day course designed to address the main causes of maternal mortality in resource poor settings in a systematic fashion’ (Tate 2014). Several years and experience later we would question this ‘diagnosis’.

The identification of the ‘problem’ at this stage in our learning was twofold: first, that many women are dying from (emergency) conditions that are preventable if staff had the right skills (to treat emergences). And, second, that Uganda lacks human resource capacity (clinicians do not possess the appropriate level of skill). Once again, this is immediately (evidently) apparent on the wards which are manned mainly by intern (junior) doctors.

Initial Theory of Change: The engagement of UK clinical volunteers alongside Ugandan counterparts in the adaptation of established ‘Green Top Guidelines’¹¹ from the UK and associated (CME) training will reduce maternal and neonatal mortality.

Intervention: Skills Deficits (Capabilities)

The intervention could be described as relatively simple¹² involving the transmission of explicit clinical skills. These included training in the value of observations and early warning scores (Case Study 1) and management of conditions such as eclampsia, haemorrhage and sepsis. As in Case Study 1, following the requirements of our funding body (THET), the immediate evaluation involved counting the volume of participants and simple before-and-after testing of skills acquisition (during the 2-day period). Combined with feedback forms, these instruments indicated a very marked improvement in skills and a strong expressed intentionality on the part of participants to utilise them.

In many interventions, this is where evaluation ends and training teams then ‘fly-home.’ Whilst pre- and post-testing measures typically indicate a high level of immediate impact, attempting to assess/project any causal (attributable) impact on mortality rates, etc., is ambitious and highly unlikely to succeed. Ethnographic observational methods combined with interviews indicated the application of some of the skills acquired amongst some of the trainees for a limited period of time and when in co-present relationships with volunteers. In practice, the ripples of good practice extending from these forms of training soon disappear.

Outcomes

Sadly, our research indicates that, although skills such as these can never be entirely wasted in terms of individual learning and potential; the investment in short CME-style interventions does not deliver the kinds of behaviour change capable of impacting public health systems in an effective, sustainable and efficient manner. Staff rotations make it especially difficult to monitor the skills use of those individuals whose skills gain is highest (junior doctors, interns and students). Whilst methods could have been developed to track individual journeys and attempt to assess skills use, the focus of the project on highly contextualised specific public facilities within the HUB discouraged this approach. Whilst we could have expended research resource to track individuals, we also felt quite strongly that implementation depends on achieving a level of critical mass capable of influencing cultural change within organisations supported by managerial buy-in.

Furthermore, many more established staff – and especially doctors – are not present on public wards with any degree of regularity to enable skills utilisation to take place (or be observed). And, many clinicians who are motivated to engage in training do so to facilitate career moves out of clinical work altogether, into the private sector (internal brain drain) or to clinical work in other countries (external brain drain). This proved to be the case with many of the clinicians who came to the UK supported by British Commonwealth Fellowships. Whilst the SVP objectives in bringing people to the UK were initially to enhance capabilities through training, the majority of fellows have used the credentials gained and networks established to exit clinical work in Uganda. In many cases they gained a new appetite for and confidence in international travel, immediately moving onto other courses in Sweden, Australia and Kenya (among others). Others have established their own NGOs or private training enterprises and become trainers themselves in Uganda, positioning themselves to harvest international NGO funding for CMEs. Others have sought direct employment in the private or NGO sector. Certainly the individual motivation in many cases is to use this experience to exit or reduce reliance on public sector clinical work.

When we assess data on human resources in Ugandan facilities, it is immediately apparent that the staff who are visibly present to professional volunteers (and form the basis of the presumption of a major skills shortage) represent the most junior of cadres (intern doctors and students). Large volumes of more experienced and highly trained staff, many of whom have had remarkable opportunities for international training, are simply not present on the wards to implement skills.

Training Protocols in CMEs

In the course of the SVP project, we were made aware of serious problems arising from varieties of foreign organisations coming into Uganda and training to their own national protocols/guidelines. This was resisted by local stakeholders who felt that they had perfectly sound guidelines endorsed by their own professional organisations and that using different systems confused staff and undermined efficacy. Attempts by the SVP to encourage multi-disciplinary and international collaboration, although welcomed, failed to encourage the necessary cooperation required (even within the UK).

The problems associated with selecting appropriate guidelines and protocols extend beyond this concern with imposing imported national systems. One might argue (as we did at one point) that internationally agreed protocols would overcome this tension. Our experience of working with WHO guidelines in our current Hand Hygiene Project¹³ identifies further sources of complexity. In this case, Ugandan infection control specialists accustomed to training according to WHO ‘African Partners for Patient Safety’ (APPS) protocols¹⁴ are also experiencing major implementation gaps. The problem arises when training ignores local contexts and assumes the existence of critical ‘opportunities’. In our recent work, this included an assumption manifest in MOH guidelines (Fig. 5.1) that all facilities have running water, hand soap and single-use hand towels (or paper towels):

Fig. 5.1

Ministry of Health guidance on Hand Hygiene (2014) (Source: Ugandan national infection prevention and control guidelines (MOH 2013))

Such a situation would be unique in Ugandan public health facilities. Indeed, when we sent our project manager to visit the trainer’s own facility, she advised us that recently they had no water due to the fact that bills had not been paid. On this occasion the staff and patients launched a protest, which forced authorities to restore water to the facility. On the basis of this lesson we then proposed that our own training in Fort Portal would be targeted very specifically to local conditions and embrace an element of empowerment of local health workers and patients. Training Ugandan health workers using the APPS tools without careful attention to local conditions achieves nothing in practice and may even contribute to low morale.

This emphasises the importance of the ‘opportunity’ dimension of behaviour change. Certainly in the case of hand hygiene, we would argue that most health workers understand the need to wash their hands both for their own protection and that of their patients. But faced with the lack of running water, soap and single-use towels, they find it impossible to see the benefits of training and imagine the reality of improved patient safety.

The SVP Benchmarking data underlined the importance of context to behaviour and outcomes. In Mulago hospital, between January 2011 and October 2012, only one maternal death was recorded on the private ward compared to 183 on the public ward. The same staff, with the same training, are responsible for both groups of patients. Figures for the same time period evidence marked differences in C-section rates for the private and public ward.

C-section rates on the private ward are around double those on the public ward. Ongoing engagement with colleagues in the hospital made us aware of the ‘policy’ that all c-sections on the private ward should be undertaken by obstetric specialists and must be prioritised. These doctors receive ‘top-up’ pro rata payments for such procedures that dwarf monthly salaries. Given this information the need for EmONC training becomes less obvious.

Intervention: Physical Resource Deficits (Opportunities)

As with Case Study 1 a logical complementary measure responding to claims on the part of healthcare workers that they lack basic physical resources to support emergency obstetric interventions was to provide equipment, infrastructure and consumables – the ‘opportunities’ enabling them to utilise their skills. And, the problems associated with resource management in Ugandan public health facilities underline the legitimacy of this claim. Few, if any, operating theatres in Health Centre IVs are fully functional and available for use.¹⁵

It is with this in mind that the SVP shifted its emphasis mid-project (and following requests by Ugandan stakeholders) away from large referral hospitals to address the efficacy of referral systems. This shift was also informed by evidence on the ground of the failure of CME interventions in congested hospitals to effect any perceptible systems change and further review of the research on maternal delays. These alternative conceptualisations of the causes of maternal deaths involving more longitudinal and holistic analyses and multi-disciplinary insights encouraged us to reconsider the data on the stated causes of maternal deaths (Table 5.2). Informed by research on the ‘three delays’ (Kaye et al. 2011; Pacagnella et al. 2012; Thaddeus and Maine 1990; Thorsen et al. 2012; Tuncalp et al. 2012) and our own research on the ground, our focus then became one of reducing delays in order to avoid preventable problems turning into obstetric emergencies (or deaths).

One of the sites in which this intervention took place was Kisenyi Health Centre IV, which refers patients into Mulago Hospital. At the time of intervention, this purpose-built facility with a full remunerated staffing complement had not been available to delivering mothers since opening 7 years previously.

Our revised Theory of Change at this point indicated a focus on restoring functionality to Health Centre IV facilities in order to increase deliveries, thereby reducing congestion in referral facilities and complications arising from maternal delays.

Intervention

The initial intervention (building on previous experience in two other facilities) involved an intense 1-week multi-disciplinary problem-solving exercise. In practice, most of the infrastructure, consumables and staff were present but small ‘snagging’ problems such as the lack of a sink in the delivery suite, the lack of drainage to wash floors and some basic equipment repair work and a lack of leadership had blocked progress. Within one week the first baby was delivered and, with new local leadership, the facility swung into action.

Outcome: Within 1 year, and following a series of very minor infrastructural and equipment interventions (costing the charity less than £1000 in total) the facility achieved a sustained average monthly delivery rate of over 800 (Ackers 2016). Every mother delivered in Kisenyi is one less mother delivering in Mulago. From that perspective, the impact on congestion and delays is tangible but not directly measureable/attributable¹⁶ as many other factors will intervene to shape admissions (such as the closing of another major referral Health Centre at that time and a significant outbreak of typhoid).

Despite these successes in building Kisenyi into a fully functioning midwifery-led unit, the operating theatre remained under-utilised. Improving functionality and providing associated training (including EmONC) supported by volunteer mentoring failed to reduce referrals for c-sections, which remained high, resulting in preventable deaths (Ackers 2016). At this stage, we could have significantly reduced referrals if we had allowed SVP volunteers to operate on patients in the absence of local doctors. We did not allow this breach of the co-presence principle. Chapter 2 reported on the audit conducted by the then SVP volunteer revealing key factors participating referrals (Fig. 5.2).

Fig. 5.2

Caesarean section rates on the private and public wards, Mulago Hospital (Source: McKay and Ackers 2013: 23)

These included supplies of blood, oxygen and power and the presence of doctors. Each of these components, whilst apparently ‘simple’, is in practice a can of worms. The availability of blood is itself a composite problem reflecting power supply (electricity); the adequacy and efficacy of the blood donation system, elements of corruption and also transportation. Power is also far more complex than meets the eye; shortages may reflect a failure to pay bills (and disappearance of relevant funds), (political) power play and funding shortages related to the purchase of fuel for back-up generators (a common problem in Uganda¹⁷) and the inflated costs of solar power in low-resource settings. On occasions any of these elements may be used and manipulated as excuses for inaction. However, as we have seen, the most common factor precipitating unnecessary referrals concerns physician presence.

Rather than go into detail about any of the aforementioned, we have decided to focus here on a more clinical follow-on intervention; namely, the institution of elective c-sections. Attempts to institute elective c-section lists have been repeatedly made by SVP volunteers across all HUB facilities. As one volunteer explains in her monthly report:

It would help facilities if there was a regular elective day/s for cases that need a caesarean section to prevent mothers having to await labour and all its inherent risks when there are likely time delays in access to theatre. This would possibly reduce transfers, emergency sections and increased mortality/morbidity.

The institution of elective c-sections would further reduce maternal delays and the need for emergency/crisis intervention.

Intervention: Implementing Elective (planned) Caesarean Sections

The overwhelming majority of caesarean sections in Uganda are undertaken as emergencies allowing complications to develop, delays to lengthen, outcomes to worsen and cost to escalate. A recent audit in Mulago Hospital (Acen 2015) found that of 200 C-sections undertaken in September to November 2014, 184 (92 %) were emergency sections. Introducing an electives c-section list in Health Centre IV Facilities would build on the success of SVP intervention to ensure more timely intervention, reducing complications and unnecessary and time-consuming referrals. On the face of it, this is a very simple intervention requiring training/mentoring in the clinical indications for c-section coupled with a staff rota and booking system.

Intervention: Skills Deficits

Most of the short courses in emergency obstetric care discussed in Chapter 3 include the discussion of indications and guidelines for elective sections. And this training has been embedded in the SVP through further short-course skills drills, ongoing mentoring and audit work to support implementation on the ground.

Intervention: Resource Deficits (Stationary and Records Books)

A desk-book was made available to enable staff to book women for elective procedures.

Outcome: Sporadic Minimal Improvement

The provision of training, protocol development and mentoring coupled with the institution of an (unchallenged) booking system failed to improve c-section rates. This failure is in large part due to the persistent resistance of local doctors (and at times anaesthetic assistants) to be present and for local midwives to book patients (anticipating the former and resorting to the cultural ‘if in doubt refer’ logic). At this point the intervention develops a level of complexity implying engagement with highly sensitive power dynamics and generating problems of positionality both within the Ugandan human resource management system and in the relationships that ‘outsiders’ have with local staff and managers.

Figure 5.2 indicated that the presence of doctors was a major factor contributing to over 53 % of referrals. IF we discount referrals made at night over a quarter of referrals made during normal daytime working hours cite the lack of medical presence as a key reason (Fig. 5.3).

Fig. 5.3

Reasons for referrals between the hours of 08:00 and 17:00 Kisenyi Health Centre (Source: Ackers 2016 [CBC: Complete Blood Count])

At this point, the level of complexity involved coupled with the difficulties that Health Partnerships and volunteers face as ‘outsiders’ with no real power to influence policy makers we attempted to offer moral support to brave local managers keen to incentivise and enforce accountability in human resource management systems. The in-charge midwife made considerable efforts to encourage doctors to be present and to expose those who were not present during their allocated working hours. And the City authorities (KCCA) increased doctors’ pay (quite considerably) in Health Centre IV facilities and considered the potential of installing an electronic system to record staff presence. In practice, this resulted in short-lived and minimal behaviour change and significant resistance to change among all doctors.

Somewhat surprisingly, some months later, continued facility benchmarking identified a sudden marked increase in elective c-section rates. Our first inclination was to celebrate the apparent success of ongoing interventions and regret our impatience. Our presence on the ground, however, and localised knowledge raised immediate concerns about this sudden shift. We were aware that one doctor was making a concerted effort to improve practice. However, we became suspicious that other ‘entrepreneurial’ practices were emerging as local doctors began to use the improved premises for private (fee-paying) elective c-sections. SVP volunteers are unable to ‘prove’ this situation empirically, given the subversive quality of corruption and the risks involved in openly asking questions – but they expressed concern when ‘surprise’ elective sections were observed to take priority over emergency cases. This is not the first time that investments made by our programme in improving infrastructure have stimulated an increase in private cases. Rather than indicating a problem caused by skills or knowledge deficits, we would conclude that our interventions have been manipulated through the use of highly entrepreneurial subversive local knowledge. summarises the processes outlined in the Case Studies:

The aforementioned case studies do not reflect the total (zero-sum) failure of interventions; capability-enhancing training combined with a focus on improving the opportunities for skills utilisation are necessary but insufficient constituents of change. Change can be seen to occur in terms of individual skills enhancement and, during volunteer co-presence, some partial skills utilisation and adherence to protocols. But these are key components in far more complex interventions. Understanding why these interventions, which typify the overwhelming majority of Health Partnership-style initiatives internationally, fail to translate into sustainable systems change is essential if we are to avoid wasting public money and undermining health systems.

The Knowledge Deficit Model

Why is it, given the lack of tangible evidence that AID, in any of its guises, works that funding continues to grow exponentially and continue to fund interventions that either do not work or generate damaging externality effects? And why, given the clear failure of existing evaluation metrics to capture the processes responsible for impact failure, is the solution seen as one of honing metrics? Public and political concern at the lack of impact of huge volumes of public funding has led to an increasing obsession with metrics and quantitative outcome measures. The UK Department for International Development’s ‘logframe’ methodology has had a major ‘structuring impact’ generating a common template for the evaluation of international development interventions. This formulaic approach has led to the emergence of a new (and highly confusing) conceptual vocabulary requiring projects to report, in a ‘theory of change’ approach on ‘activities’, ‘inputs’, ‘outcomes’ and ‘outputs’. The THET, almost entirely reliant on DFID funding, has necessarily had to import this approach and apply it to all Health Partnership work. Sadly, this has forced a situation in which the ‘tail has begun to wag the dog’ as the clamour for funds begins to shape interventions around the demands of the funding bodies and their evaluation metrics rather than perceived needs or a grounded understanding of what works or does not. And this has a major impact on the use of evaluation resource; so great is the need to count ‘outcomes’ that there is little time to try to understand and make sense of failure. And failure is and has been the dominant outcome of development interventions for many decades.

The Lancet Commission’s ‘Global Surgery’ report opens with the statement: ‘Remarkable gains have been made in global health in the past 25 years’ (Meara, J.G. et al 2015: 569) but provides no evidence to substantiate this claim. It goes on to present a very holistic and comprehensive case for multi-professional interventions informed by the three delays model and focused on ‘broad-based health-systems solutions’ (p. 570). The concluding section argues that ‘research, monitoring and assessment play a crucial part in the future of global surgery’ together with a ‘commitment to better understand the problems and solutions’ (p. 616). We would agree wholeheartedly with this prognosis. Having said that, we firmly believe that attributing this to the ‘complete absence of globally accepted surgical metrics’ entirely misses the point.

A recent report on International Development Funding by the UK Government’s Public Accounts Committee found that, despite the trebling of public funding for humanitarian crises, ‘the Department for International Development did not have a full understanding of where its money went’ (House of Commons Report 2016). DFID defended its record arguing that it had a system which enabled it to ‘aggregate the success of each intervention into a score which it can track month by month across interventions’ (p. 10). However, it failed to report its portfolio score. What we are doing here is attempting to measure the measurable (as information or ‘data’) rather than the meaningful (as knowledge):

Simplistically conceptualising knowledge as information makes its valuation and trade measurable but loses most of the originality of the empirical phenomenon. By contrast when scholars conceptualise knowledge as complex capabilities embodied in people and organisations, it no longer fits into the concept of an economic good that can be valued, traded, and accumulated, and its exact measurement becomes an impossibility. (Gluckler et al. 2013: 6)

These reports underline the fact that we know so little about the effectiveness of AID and that the solution to this problem is seen to lie in the production of ever-more quantitative metrics. No attempts are made to question the underlying epistemological biases of this logic or, put more simply, the fact that metrics have never worked and never will capture the ‘problems and solutions’ facing health systems in LMICs (or indeed the UK).

Fundamentally, we are suffering from a form of myopia generated by the domination of medical science perspectives or knowledge paradigms which determine the diagnoses, the interventions and epistemological approaches to evaluation. The quest for statistical outcomes (ideally gained through the gold standard of randomised controlled trials) underpinned by the narrowing blinkers of systematic review restricts our ability to understand social processes. There is also a tendency within this paradigm to pathologise or patronise individuals whilst failing to understand the impact of structural constraints. Despite growing recognition of the importance of context, this is often in the form of lip service at best acknowledging it as a cluster of variables that we don’t understand (as external ‘noise’) or, at worst, rather than understanding and capturing its iterative quality, trying to insulate our interventions from it through vain or inappropriate attempts to control it. Trying to cleanse data through what feminist researchers have called a ‘sanitisation process’ (Harding 1987, 1991) will not generate cleaner facts. It will take us further away from the truth as the cleansing process strips data of its real value: of understanding the social processes that shape phenomenon. It may be that what we are swilling away in the effluent is that which is of greatest value.

This narrow conceptualisation of knowledge not only affects the approach to evaluation, but it also reproduces a partial understanding of knowledge mobilisation as an activity. The emphasis on explicit clinical skills and neglect of tacit knowledge leads to the identification of a ‘knowledge deficit’ encouraging uni-directional flows (from the host to the LMIC or from doctors to midwives). We are not adding skills to context (as the COM-B model implies): context lies at the heart of complex fluid and, oftentimes confounding knowledge combinations. Or, as McCormack puts it: ‘nothing exists and can be understood in isolation from its context’ (2015: 3009).

On the basis of our research and learning, we would argue that far from a skills deficit we are in a situation of knowledge saturation; clinical skills/information are not the primary problem and using clinical ‘experts’ as conduits for yet more skills-forcing interventions is neither efficient nor effective. To the contrary, it is both arrogant and wasteful. This arrogance does not derive solely (although this certainly does contribute) to the imposition of ‘expert information’ from the global North: from well-meaning but narrow thinking foreign clinical experts and ‘donors’. It is the consequence of a failure to see the bigger picture – of narrowing disciplinary knowledge paradigms. Although the lack of patient management is recognised as an issue by all UK clinicians working in Uganda: it was a visiting Ugandan obstetrician who proposed the initiative to introduce the African Maternal Early Warning Score (AMEWS) into Mulago Hospital. Enthralled by its apparent success in the UK (visiting foreigners are often not exposed to or fail to see the fundamental weaknesses of UK systems) she quite understandably attempted a ‘policy transfer’ approach: she ‘saw’ a clinical solution to a clinical problem and tried to transfer it. And as a project we sought to support her in this without comprehending the localised knowledge that would ultimately render it unsuccessful.

Knowledge can be both enabling and disabling; it is not only a deficit of skills/knowledge that hampers progress; narrow siloed knowledge (focused on harvesting facts) can limit our ability to see the social world and the truth that lies behind interventions. Resistance to change may be fuelled by knowledge and often quite entrepreneurial tacit localised knowledge. From an individual perspective – how to make ends meet and sustain your family – or, from a systems perspective – how to organise and funnel the rewards from systemic forms of corruption – requires a level of localised tacit knowledge and rational/entrepreneurial decision making that foreign volunteers may fail to ‘see’.

The overall conclusion of this book, based on intense multi-method action-oriented research over a period of 8 years, is that international development initiatives in the form of Health Partnerships and through the mechanism of professional voluntarism (staff exchanges) are failing to bring about significant sustainable systems change in the Ugandan public health sector. Having said that, we believe that Health Partnership activity has the potential to mitigate the forms of systems damage associated with AID. More importantly, we believe it has the capacity to form the basis of evidence-based incremental systems change. Achieving this will require a much more structured approach to professional volunteer deployment based on principles of negotiated conditionality. Whilst we emphasise the importance of understanding the benefits and humility of incrementalism in policy change, from an epistemological perspective we need a paradigm shift.

If we reflect on the three scenarios presented in Chapter 1, we can conclude that the risks associated with Scenario 3 (negative impact through collateral damage) remain very high and most AID activity falls into this category. In terms of long-term sustainability, most of what we have achieved through the SVP would fall under Scenario 2 (neutral long-term impact but with minimal externality effects). More optimistically, and informed by the iterative learning we have experienced ourselves as researchers and project managers, we believe that Scenario 1 is achievable (partial improvement with minimal collateral damage).

The book opened and is peppered with firmly expressed concerns about the externality effects of AID focused on gap filling and service delivery. We have talked a lot about collateral damage, the unintended consequences of well-intentioned interventions and the critical importance of co-presence. When interventions, such as the SVP and the Health Partnership Scheme more generally, are based on sound conditionality principles, externality effects may be positive. Certainly the co-presence principle supports the active clinical engagement of professional volunteers. In this process, the collateral benefits of systems focused intervention are enormous. We have no doubt and are reminded on a daily basis of the impacts of our work and of professional volunteers in particular on patient services and individual patients. SVP volunteers are saving the lives of mothers and babies in Uganda on a daily basis. Finally, the key beneficiaries of the SVP model are those patients who rely on public welfare to meet their healthcare needs, both in the immediate term (through the daily hands-on engagement of volunteers) and through longer-term systems change.

The following section outlines some key elements of structure and conditionality that we have identified through our action-research journey. These concerns and the recommendations that accompany them may not be directly applicable or transportable to other LMIC or disciplinary contexts but when carefully contextualised and translated they could inform policy making and volunteer deployment.

Policy Recommendations

1. Corruption

Health Partnership interventions, as with all AID, generate vast opportunities for corruption. Opportunities for corruption are identified and exploited in every interaction we engage in from the use of disposable gloves to the donation of equipment. This is a highly entrepreneurial and innovative process which is very hard for ‘outsiders’ to see or to challenge. In this context, professional volunteers are often viewed as ‘spies’ and as such are open to forms of harassment. More commonly, they are unable to function effectively as clinicians and knowledge intermediaries.

Recommendation 1: Whistle Blowing

Health Partnerships and Volunteer Deployment Agencies need to work closely with their Ugandan partners on the ground and provide support mechanisms for volunteers to encourage them to recognise and report corruption. Health Partnerships are often loath to acknowledge or take action on corruption for fear of damaging relationships. This, in itself, damages relationships.

2. Labour Substitution

Our research indicates a significant potential for system damage when interventions are framed inappropriately to the context and when the context is itself misunderstood. Labour substitution is a huge risk. It undermines systems, distorts local labour markets and accentuates existing human resource management problems actively encouraging absenteeism and moonlighting (dual working). It is also profoundly arrogant. This ‘style’ of volunteering stems from traditional donor–recipient models and is particularly associated with missionary-style volunteering. The focus on ‘helping’ individual patients and creating parallel institutions for this purpose (mission hospitals etc.), whilst perfectly understandable, in the wider scheme of things undermines universal public health systems.

Recommendation 2.1: Advocacy in Human Resource Management

The efficacy of professional voluntarism will not improve unless key stakeholders in LMICs introduce and enforce accountability in human resource management systems. Measures must be put in place to ensure the timely payment of salaries and to discipline and/or dismiss staff members who fail to comply and present themselves for work. Unless local staff are present in the workplace, the opportunities for knowledge mobilisation and systems change are minimal. Specific attention needs to be paid to staff in leadership and senior positions, especially doctors. If leaders fail to present themselves for work in a timely manner, the lack of effective role modelling reinforces a culture of bad practice. Leadership is seriously lacking in most Ugandan health facilities. We do not believe this represents a need for training, however, but for improved accountability.

Recommendation 2.2: Co-Presence

All efforts should be taken to respect and embed the principle of co-presence when deploying professional volunteers. This should be interpreted as a key component of conditionality and where co-presence is not possible volunteers should be required to withdraw from that context. The only exception to this should be in carefully managed and documented emergency situations. The failure to respect co-presence by volunteer deploying organisations and individual volunteers undermines the position of those who are respecting the principle and generates resentment. Managing and operationalising co-presence requires the development of highly structured programmes for volunteer deployment with clear role descriptions and reporting mechanisms. ‘Lone Ranger’ and Missionary-style volunteering undermines these processes.

The enforcement of co-presence as a key dimension of accountability in human resource management systems requires high-level action at ministerial level and involving consortiums of international NGOs working in collaboration. This is the single most critical element of conditionality.

3. Equality and Ethical Standards in Volunteer Deployment

The THET has played a major role in supporting (and requiring) more structured, risk-assessed and ethical approaches to volunteer deployment. We believe that this has gone a long way to ensuring that the recruitment and deployment of professional volunteers complies with UK equality law and policy. However, we have witnessed behaviour on the part of UK-registered Charities operating in Uganda, particularly those involved in Mission Hospitals (affecting both Ugandan health workers and UK volunteers) which would seriously breach all equality rules in the UK. This includes overt discrimination on grounds of sexuality, gender, race, religious beliefs and breaches of the principles of the Working Time Directive and work–life balance.

Recommendation 3: Equality and Charitable Status

We have noted (earlier) our concern at the role that faith-based NGOs (charities) play in running parallel organisations independent of the Ugandan public health sector. As a general rule, we believe this process undermines the public system and creates opportunities to extend damaging forms of missionary-style colonialism. These exist in a ‘bubble’ with apparent immunity from both Ugandan and UK law and policy and attract religious fanatics. All UK charities operating in LMICs should be required to abide by both Ugandan and UK Employment, Charity and Equality Laws. Failure to comply should result in loss of charitable status.

4. The Commodification of Training

It is inappropriate and wasteful to pay people (per diems) to receive training. The payments attached to training have distorted the whole meaning of training and created a situation where people are training for the wrong reasons and the wrong people are consuming the opportunities on offer. The failure of LMICs and development organisations to develop a common policy on this generates tensions and a competitive environment, which further detracts from knowledge mobilisation objectives. AID organisations have commodified training, creating new opportunities for corruption and serial absenteeism especially among senior staff and ‘leaders’.

Recommendation 4: Per Diems

The practice of providing per diems for training should be immediately stopped. We urge the Ugandan Ministry of Health and District Health Offices to create protocols to regulate these practices so that training interventions reach the people they are designed for and perverse incentives are extinguished. Providing essential assistance with transport and refreshments is less of a problem. Organising training events as close as possible to health workers’ work place and, where possible, in the workplace reduces these costs (and risks) whilst also reducing the amount of time health workers are away from the wards.

5. Internal Brain Drain

Often overlooked, internal brain drain has a far greater impact on the human resource ‘crisis’ in LMICs than external brain drain (emigration). And development interventions create significant opportunities for this through the employment of staff in project management and other roles. This is a complex issue. What is clear from our research is that the clamour for ‘project employment’ is having a hugely distorting effect on national labour markets and career development strategies. The effect of this is to encourage an emphasis on non-clinical postgraduate qualifications (Masters in Business Administration or Public Health for example) as part of a planned exit from clinical work. This is often seen as the alternative to dual working (internal brain drain into the private sector whilst in full-time public employment), which blocks positions for early career health workers.

Recommendation 5: Remunerating Staff in LMICs

There is no simple solution to this complex ethical problem. We are, quite rightly, encouraged by funding bodies to employ local staff where possible and build capacity in leadership and management. However, we believe that local staff should receive a level of remuneration broadly parallel to what they would (or should) receive in their public roles. We have tried to identify the level of pay that delivers a ‘living wage’ as opposed to the (below) subsistence-level pay that health workers are receiving in Uganda. Ultimately the MOH needs to increase health worker pay quite significantly for all cadres and impose a level of accountability to ensure that those who are paid are present and work effectively.

We would propose that a doubling of health worker salaries could be achieved at no cost and with marked efficiencies by dismissing all those staff who fail to report for work. Once this level is achieved and staff members are paid on time, foreign organisations should remunerate at national rates. From a system perspective, it is irresponsible and unethical to remunerate health workers in Uganda at rates common in the UK or the USA.

6. Continuing Professional Development (or Continuing Medical Education)

Health Partnership interventions have focused on the provision of short courses aimed at health workers. Uganda lacks an active CPD programme in most professional areas and there is little evidence of active Professional Development Review Systems. In some areas such as bio-medical engineering they are entirely absent. This leaves a vacuum for foreign organisations. In isolation and without follow-up, one-off short courses do little to change professional behaviour; ongoing mentoring on the job increases the opportunities for health workers to gain confidence and skills; this is a necessary but insufficient condition for individual behaviour change.

Recommendation 6.1: Professional Development Review

CPD/CME training needs to be embedded within a comprehensive PDR process for health workers so that needs are identified by line managers and training is organised to meet those needs.

Recommendation 6.2: Post-Training Mentoring Is Critical to Knowledge Mobilisation

One-off formal CMEs are effective in transmitting information. Operationalisation and application of that learning needs support on the job to build confidence and hone skills. Professional volunteers along with their peers have a potentially valuable role to play in this process. Notwithstanding the pressure on health workers, Uganda does have many experienced clinicians. It is important that these staff members are required to support volunteers in the training and mentoring process as part of their professional role (and not on a top-up fee basis). At the present time, there is very little evidence of supervision and mentoring on the part of experienced Ugandan clinicians and we should take care not to substitute or commodify these roles. Careful thought should be given to the value of taking staff away from their clinical duties for one-off training if there are no mechanisms in place to support post-course co-working and mentoring.

7. Donations

Donations of equipment, consumables or cash distort relationships and generate misunderstandings, which pose problems for subsequent interventions and volunteer activities. Professional volunteers are often seen first and foremost as ‘donors’ or, more crudely but accurately, ‘cash cows’. The SVP and its sister bio-medical engineering project have drawn attention to the serious problems associated with donated equipment, much of which lies unused or unusable creating problems for storage and infection control. It also means that managers often lack awareness of what equipment there is in their facilities. On a more mundane level, it is important to explain to prospective volunteers the importance of discussing donations and requests for cash support, etc., with the project managers.

Recommendation 7: Donations Policies

All organisations, both in the sending country and in the LMIC, should adopt a transparent and joined-up donations policy. However kind and generous the intentions, the act of donation can pollute relations and generate problems. Care should be taken whenever providing infrastructural support to public facilities. The provision of consumables and equipment generates dependency, encourages opportunities for corruption and is entirely unsustainable. Where support is provided, the objective should be to provide the very minimum of support to leverage local systems rather than substituting for them. We consider this to be an example of ‘snagging’ where a whole facility can be out of action because there is no sink, for example, but local staff exist to install the sink. When providing support the SVP and our sister bio-medical engineering project developed a firm policy on donations.¹⁸

8. Evaluation: Efficacy and Meaning

We have described the SVP project as a journey. This has been mainly a research journey. We embarked upon it with an extensive background in multi-methods, comparative, high-impact social research. Throughout this journey we have experienced a creative tension with the evaluation arm of the funding body and with other organisations and stakeholders in the global health field. This discomfort has generated active learning for us as researchers unfamiliar with the foundations of medical research, its dominance across health sciences and the quite similar metrics paradigm framing international development. Our concerns about the epistemological bias of these approaches and their inability to capture social context and process are well rehearsed in the book. But we have another concern about the efficacy of evaluation. The Health Partnership Scheme in common with so many other funders of global health has begun to conceptualise every actor in a funded project as a researcher. On one level we commend that and it resonates with our ethical commitment to co-researching. The problem is that researching such complex interventions requires a very high degree of research expertise and is hugely time-consuming. Partners in HP projects do not have the time or necessarily the expertise to undertake extensive literature reviews or policy analysis work; neither do they possess the skills to develop research plans and operationalise mixed-methods studies. And, they may well not have the experience, time or desire to engage in complex data analysis and writing up. Whilst all actors can become engaged in action-research projects and we have certainly viewed our Ugandan colleagues and professional volunteers as co-researchers, finally a considerable research expertise is required to design, manage and make sense of the data.

On that basis we would question the pressure put on all projects to conduct in-depth, expensive and time-consuming evaluation. Certainly attempts to aggregate core indicators across the diversity of projects will achieve nothing apart from churning out meaningless and potentially misleading statistics.

Recommendation 8: Evaluation Policy

We would recommend that valuable resource expended on evaluation be used more wisely to conduct expert research on a sample of interventions managed by researchers with proven expertise. Many private consultancy organisations lack academic research training and experience and constitute an expensive and poor-quality ‘offer’. The British Academy Report on the role of the Humanities and Social Sciences in Public Policy Making (Wilson 2008) expressed concern at the very high proportion of UK government department’s research budgets that were being ‘allocated to short-term projects to meet current political and administrative demands’ arguing that the government was failing to leverage the academic research base. Identifying the demands of complex inter-departmental collaborations straddling the boundaries of departments, it argues that ‘many challenges require a more sophisticated understanding of human behaviour’ (Executive Summary p. 1). The report concurs with our recommendation that research funding should be focused more into support longitudinal peer-reviewed knowledge development.

Knowledge for Change?

A common response when we present our research findings is to ask us why we carry on with the work when the outcomes are so bleak and depressing. We do not find the work depressing. Indeed, to the contrary, it has been quite liberating. We cannot begin to develop interventions that have the potential to bring about systems change unless we have the knowledge base to understand the context and the processes involved. Trying to understand another country and co-design complex interventions takes time; it also challenges our skills as researchers and activists. This book has been written at a particular juncture in this learning process. We will carry on learning but we feel that we have achieved a level of understanding (and knowledge base) now that can and should be shared with others to reframe interventions.

We have always enjoyed the company of our Ugandan colleagues. We do not regard ourselves as ‘donors’ or them as ‘recipients’: rather we regard each other as fellow professionals. And we would like to move away from the language of helping or volunteering to speak more collegially about international faculty. We know from previous research and common experience that international exposure tends to reinforce rather than challenge stereotypes. Researchers and volunteers new to the field will unwittingly been drawn into absorbing and echoing those stereotypes: of Ugandan health workers as poorly trained and under-skilled, lazy, demotivated, lacking respect for fellow humans and corrupt. And the landscape we ‘see’ may reinforce this perception. Some of the research we have encountered along the way, notably from behavioural science perspectives, has the unintentional tendency to add intellectual credibility to these stereotypes often essentialising human behaviour. The work of evolutionary economics has given us fresh insight and lifted us from the traps of individualism, enabling us to forge genuine human relationships with our Ugandan colleagues as fellow human beings. Finally, there is more that we share than that which distinguishes us: context is indeed everything.