How Does Evidence Matter? Understanding ‘What Works’ for Wellbeing

Abstract

In 2015 the UK launched an independent What Works Centre for Wellbeing, co-funded by government departments and various agencies, which aims to develop a ‘strong and credible evidence base’ to help promote wellbeing in policy. Yet while there is widespread agreement that evidence matters in policy-making, it is far from clear what kinds of evidence matters, in what circumstances and to what extent. In this context, this article presents the findings of research exploring with policy-makers and stakeholders issues in the use of evidence in relation to wellbeing in public policy. In particular, it highlights evidence as a specific form of (research-based) knowledge and considers the importance of this relative to other forms of knowledge; political, professional and experiential. This approach highlights a broader understanding of ‘what works’ beyond the relatively technical sense often employed to describe the work of What Works Centres and in the use of evidence more generally.

1 Introduction

Wellbeing is increasingly important in politics and policy across a range of contexts and not least in the UK. In 2015 the UK launched an independent What Works Centre for Wellbeing (WWCW), co-funded by government departments and various agencies, which aims to develop a ‘strong and credible evidence base’ to help promote wellbeing in policy. In this context, this article presents the findings of research exploring with policy-makers and stakeholders on the use of evidence in relation to wellbeing in public policy. In particular, it highlights as evidence as a specific form of (research-based) knowledge and considers the importance of this relative to other forms of knowledge; political, professional and experiential. This approach highlights a broader understanding of ‘what works’ beyond the relatively technical sense often employed to describe the work of What Works Centres and in the use of evidence more generally.

This paper has six further sections. The next section charts the rise of wellbeing in policy in the UK leading to the creation of the WWCW. Section three considers the use of evidence in policy, highlighting both practical developments in the UK policy arena and important academic contributions to understanding this phenomenon. Section four provides an overview of the research design for this study, while section five presents the main findings. These findings are analysed in the sixth section, referring back to the analytical themes outlined in section two. The concluding section reflects on the key findings and sets out the main contribution of the article.

2 The Rise of Wellbeing in Policy

A pivotal moment in UK developments on wellbeing came in 2010 when the Prime Minister publicly endorsed the Office for National Statistics (ONS) Measuring National Wellbeing programme, which aims to ‘to develop and publish an accepted and trusted set of National Statistics which help people understand and monitor well-being’ (Self et al. 2012). In launching the programme, then Prime Minister David Cameron argued that it would:

…open up a national debate about what really matters, not just in government but amongst people who influence our lives: in the media; in business; the people who develop the products we use, who build the towns we live in, who shape the culture we enjoy. And… this information will help government work out, with evidence, the best ways of trying to help to improve people’s wellbeing (Cameron 2010).

The ONS subsequently conducted a public consultation on what domains and measures of wellbeing should be used for the programme. In response to this consultation, ten domains were decided upon in July 2012: individual wellbeing (later renamed ‘personal wellbeing’¹); our relationships; health; what we do; where we live; personal finance; economy; education and skills; governance; and natural environment. Most of the data for these domains was already being collected by the ONS, and was simply repackaged for the wellbeing framework. However, four new subjective wellbeing questions were added to provide data for the individual wellbeing domain. The four questions were: Overall, how satisfied are you with your life nowadays? Overall, how happy did you feel yesterday? Overall, how anxious did you feel yesterday? Overall, to what extent do you feel the things you do in your life are worthwhile? Each question is measured on a scale from 0 to 10. Data for the programme has since been collected annually. Subjective wellbeing questions were new to national statistics and it was this part of the programme that attracted most attention.

While there had been interest in wellbeing in UK government circles before 2010 (see Bache and Reardon 2016, 71–72), it was the launch of the ONS programme and Cameron’s support for the idea that firmly placed it on the government agenda. In 2011, the Treasury updated its guidance to government departments to include subjective wellbeing in policy appraisal, alongside the long-established market-based approaches of Stated Preference and Revealed Preference (Fujiwara and Campbell 2011, 57–58). However, the government was clear that the ONS subjective wellbeing indicators were still in development and that ‘we should not expect at this stage to have examples of major decisions that have been heavily influenced by wellbeing research’ (HM Government 2013, para. 4). However, it also suggested that it had put in place some ‘new foundations’ for instilling a wellbeing approach (HM Government 2013, para. 4). Among these new policy foundations were a number of new departmental surveys and narratives relating to wellbeing and examples of wellbeing emerging in policy appraisal and evaluations (on these developments see Bache and Reardon 2016, 72–80).

In October 2014 the government announced that funding would be provided for the creation of an independent What Works Centre for Wellbeing (WWCW). The Centre, which has 17 founding partners, including government departments and the Economic and Social Research Council (ESRC), aims to develop a ‘strong and credible evidence base’ which will support organisations ‘to concentrate efforts on interventions that will have the biggest impact’ on wellbeing (Cabinet Office 2015). Funding for the Centre was confirmed after the 2015 general election, which gave a strong signal of the government’s continuing commitment on the issue—and continuing faith in the value of evidence in policy. The creation of the WWCW followed the establishment of What Works Centres in a number of other policy areas: crime reduction, health and social care, education, early intervention, ageing better and local economic growth. The centres aim to ‘help to ensure that policy makers, practitioners and commissioners can make informed decisions based on impact and cost effectiveness’ (Cabinet Office 2015).

In comparison with other policy areas covered by What Works Centres, wellbeing is more recent to the policy agenda and is arguably subject to greater contestation over definition and measurement. An evaluation of the use of wellbeing powers in UK local government found different interpretations and discourses of wellbeing across local authorities and ‘little consistency’ in understanding of the new powers: ‘there was indeed no ‘single’ problem of wellbeing but rather a range of ongoing problematisations, which varied according to the interpretations of different stakeholders’ (Griggs and Howarth 2011, 221). Such conclusions are reflected in the wider literature on wellbeing (e.g. Scott 2012; McGregor 2015) and conflict over the nature of the problem and thus the potential solutions have led to it being described as a quintessential ‘wicked problem’ (Bache et al. 2016).

However, the ESRC (2014, 4) provided a starting point for the WWCW on how wellbeing should be understood, drawing on the work of the ONS:

Wellbeing, put simply, is about ‘how we are doing’ as individuals, communities and as a nation and how sustainable this is for the future. We define wellbeing as having 10 broad dimensions which have been shown to matter most to people in the UK as identified through a national debate…[see above]… Personal wellbeing is a particularly important dimension which we define as how satisfied we are with our lives, our sense that what we do in life is worthwhile, our day to day emotional experiences (happiness and anxiety) and our wider mental wellbeing.

This definition set out common ground for the different programmes in the WWCW but, as the ESRC acknowledged, would not resolve debate over definitional issues. The extent to which personal/subjective wellbeing should be emphasised remains a particularly contentious issue (below).

The newness of wellbeing on to the policy agenda means that much extant research has not been conducted with this definition in mind and, often, relevant research has wellbeing as a secondary or even tertiary outcome (ESRC 2014, 5). As such, part of the task of the WWCW is to address challenges such as ‘as the ability to establish cause and effect from such evidence’ (ESRC 2014, 5). In short, the accumulation and transmission of evidence on how policies can enhance wellbeing is seen as a crucial next step in bringing wellbeing more fully into policy.

3 The Use of Evidence in Policy

The use of evidence in policy has a long history, but has had particular emphasis in UK government since the late 1990s. A key moment in the UK was the publication of the Modernising Government White Paper of 1999, which emphasised the role of evidence-based policy:

This [White Paper] recognised the need for policy making to be more responsive to citizens’ demands; looking forward; evidence-based; properly evaluated and based on “best practice”(GSRU 2007, 9).

The Labour Party’s 1997 manifesto statement ‘what counts is what works’ (Labour Party 1997, 2) was a signal of a post-ideological approach to policy that it sought to bring into government after its election in 1997. The shift towards evidence-based policy occurred in other places around the same time, linked to a broad shift in managerial reforms focusing on effectiveness and efficiency (Marston and Watts 2003, 147). In the UK context, one consequence of this shift was the creation of ‘What Works Centres’, beginning with the National Institute for Health and Care Excellence (NICE) in 1999. According to the ESRC (2017):

What Works centres enable leading social scientists to evaluate the availability and quality of evidence underpinning public policy interventions, compare the effectiveness of interventions, and advise those commissioning and undertaking interventions to ensure that their work can be evaluated effectively. They also identify research and capability gaps, working with partners to fill them.

Evidence can take a variety of forms and these forms are often placed in a hierarchy, based on study design. In such hierarchies, RCTs and systematic reviews are generally placed at the top, with individual (qualitative) case studies at the bottom (see Table 1).

Table 1 Two illustrations of simplified hierarchies of evidence based on study design.

A number of reasons are advanced for using evidence, from helping to understand an issue, to identifying the appropriate policy response and shaping future thinking (Nutley et al. 2013, 10). Thus, it is used at various stages of the policy cycle, from creating, developing and implementing policy to defending and justifying a policy decision.

While it was common in this early period to refer to ‘evidence-based’ policy, growing recognition of the many drivers of policy making (e.g., GSRU 2007; House of Commons Science and Technology Committee 2006; see also below) shifted the discourse more towards evidence-informed policy to acknowledge that policy is rarely a simple and direct response to evidence. Head (2010, 80) developed a more comprehensive account of why evidence ‘can inform policy rather than constitute a systematic foundation for the policy process’. First, that a strong research base is simply not available to policy makers in some areas; second, decision-makers are often motivated or influenced more by factors other than research evidence; third, that where evidence is available it is often poor; fourth, professional knowledge rather than evidence-based is often more important; and fifth, evidence-based policy appears to have less influence in areas that are in flux (Head 2010, 80–81).

Mulgan (2005) suggests that the ways in which evidence can be used varies according to the state of knowledge in a particular field. He identifies three types of field. In stable policy fields, governments know most about ‘what works’ and there is a strong evidence base. Networks are well established and the relevant professional bodies and experts can generally be relied upon to provide good advice. In policy fields in flux, the knowledge base is contested and there is disagreement over even the basic theoretical approaches. So while there is recognition that change is needed, there is no consensus on either the nature of the problem or the solution. In inherently novel policy fields, there is no strong evidence base because of the recent arrival of the issue to the policy arena. The differences between these types of fields highlights the importance of theory:

In the second and third categories, our questions are changing as well as our answers. In such situations, evidence does not exist in the abstract, floating free. It exists in relation to theories and concepts that provide the prisms through which the world is seen. These theories are not alternatives to hard facts and evidence: they are the only ways of making sense of them (Mulgan 2005, 222).

While research-based evidence can provide an appearance of scientific neutrality that holds a particular status and authority in policy-making, its influence is shaped by the nature of the policy context into which it enters. As suggested above, evidence is generally not the only contender for influence and politics and knowledge of different types interact to shape decision-making.

Weiss’s (1979) seminal contribution on research utilisation contrasted political uses of research with decisions directly driven by evidence, suggesting that the latter were particularly rare in relation to social science research. In a later paper she explained further this political dimension, suggesting that ‘When new data or research findings arrive on the conference table in the councils of action they confront four I’s already sitting at the table…ideology, interests, institutional norms and practices, and prior information’ (Weiss 2001, 286).

The difficulty of separating facts from values has been a particular feature of the academic debate. Majone (1989) emphasised the role of persuasion, suggesting that that ‘few arguments are purely rational or purely persuasive’ and thus it is the blend between the two that matters:

An inappropriate choice of data, their placement at a wrong point in the argument, a style of presentation that is unsuitable for the audience to which the argument is directed – any one of these factors can destroy the effectiveness of information as evidence, regardless of its intrinsic cognitive content… the applicability of evidence depends on a number of features peculiar to a given situation, such as the nature of the case, the type of audience, the prevailing “rules of evidence,” and even the persuasiveness of the analyst(Majone 1989, 48).

Other scholars have sought to refine the political dimension further. Boswell (2008) highlighted the symbolic functions of knowledge: one being a legitimizing function, in which an organisation uses knowledge to enhance its legitimacy in a particular policy area; the other a substantiating function, in which knowledge lends authority to help substantiate organisational preferences where there is political contestation.

Head (2010) identified three legitimate types of knowledge in democratic societies, alongside research-based evidence. Political knowledge refers to the strategies, tactics and agenda-setting abilities of political leaders and their organisations, which set the ‘big picture’ of policy priorities and approaches; the professional knowledge of practitioners and managers is essential for understanding feasibility and effectiveness; and the experiential knowledge of service users and stakeholders is central to ‘client focused’ service delivery. Head also identified the particular knowledge challenges faced by complex policy ideas, which resonates with the case of wellbeing:

In relatively simple issues where all the variables can be specified and controlled, methodological rigour is likely to be tight, with some confidence that causal factors can be clarified. But in programs with multiple objectives, or where the clients/stakeholders are subjected to many sources of influence beyond the scope of the program, the challenge of accurate understanding is compounded (Head 2010, 83).

In such cases, the way a problem is framed is important both for the scientific validity and for the political management of complex problems, which ‘are unlikely to be “solvable” through a single policy instrument or “magic bullet”'(Head 2010, 83). It is for these reasons that wellbeing has been described as a ‘wicked problem’ (above).

John Shepherd’s (2014) report to government on the role of What Works Centres,² which pre-dated the creation of the WWCW, provides a useful bridge between these analytical themes and examination of the case of wellbeing. Shepherd’s starting point is that ‘the creation and adoption of effective policies, programmes and interventions depends on a functional evidence ecosystem’ (Shepherd 2014, 5). He suggests that: ‘What Works Centres are an essential part of this ecosystem and need to be concerned not just with evidence synthesis and adoption but with the whole system in their sector so that faults can be identified and put right’ (Shepherd 2014, 5). Shepherd’s approach focused on four features of the evidence ecosystem: evidence sources; transmission lines; problems; and incentives. This conceptualisation provides a structure through which to examine the themes highlighted above relating to the factors that constrain or facilitate the use of evidence and the influence of other factors on this complex issue; in particular, the role of other forms of knowledge.

4 Research Design

Drawing on the literatures discussed above, the following topic guide and interview questions were developed in consultation with WWCW stakeholders close to the policy process:

• 1. Key terms and issues

  • Wellbeing—how do you understand this term? How do they think other people understand it?

  • ‘What works’—what does this mean to you?

• 2. The evidence ecosystem

  • Evidence sources—what sources of evidence do you know about and what are the ones you use?

  • Transmission lines—what are the channels through which your organisation receives evidence?

  • Incentives—What are the main incentives for using evidence?

  • Problems—What are the main challenges around the use of evidence?

• 3. The challenge of wellbeing

  • Does the issue of wellbeing present specific challenges in the use of evidence? If so, of what types?

  • Other than research evidence what other forms of knowledge are important to wellbeing (e.g., political, professional, experiential)?

• 4. Moving forward

  • What specifically do you want from evidence providers/the WWCW?

These themes and questions formed the basis of semi-structured interviews with policy-makers and stakeholders that were undertaken as part of the work of the Community Wellbeing Evidence Programme team’s work for the WWCW.³ Fifteen initial interviews were conducted with individuals from the national civil service (3); local government (3); charities (3); the voluntary sector (3); government agencies (2); and a non-departmental public body (NDPB) (1). The interviews were conducted by telephone and were recorded and transcribed.⁴ Three follow-up interviews took place in order to gain clarification on a number of points and to check for updates on developments.⁵ The research was approved by the Research Ethics Committee of the University of Sheffield and all participants completed a consent form, which allowed data to be used (anonymously) in publications.

The target population consisted of individuals with a close involvement with wellbeing in their work and the sample chosen to reflect a cross-section of this estimated population of interest⁶: one that sought to include all significant elements. The selection of organisations was undertaken jointly with stakeholders from the WWCW who are close to the embryonic wellbeing policy networks and who were thus able to advise on relevant participants for this study. Through these connections I was able to gain access to senior actors who might otherwise have not been available for interview. While it was not the main aim of the research to identify differences in approach or emphasis across organisations, some differences did emerge and are reported below.⁷

Interviews were the preferred choice of data collection in order to provide a deep understanding of the phenomenon of evidence use in complex policy arenas and to provide insight into the meanings of the subject’s experiences. The use of semi-structured interviews ensured that the main themes and questions were addressed in all interviews but also treated the interviewees as ‘active subjects’ able to organise their responses within their own frameworks (Halperin and Heath 2012, 299).

In the first stage of analysing the findings, the interview transcripts were read holistically, looking at patterns (similarities and differences) in the data to facilitate the identification of important themes beyond those that provided the guide to the interview questions. The most important and interesting parts of the text were highlighted to provide a coding frame: all material was coded for relevant themes. The sections of text marked with each code were drawn together and conclusions drawn by analysing the meanings of the data and their implications for the research questions. The data from different respondents was cross checked to validate key findings. To provide verification, the findings were presented to a workshop of WWWC stakeholders from a range of organisations and other policymakers, some of whom were interviewed for the research.⁸

5 Research Findings

5.1 Understanding of Key Terms and Issues

There was consensus among interviewees that wellbeing should be understood as a complex, multidimensional phenomenon: a reflection perhaps of their close connection to the issue and, as such, their knowledge of current thinking within government and other key bodies. This point was made clear when they gave their responses on how they thought others, less closely connected to the issue, see wellbeing, which was quite different. For example:

culturally, some commentators across the UK view wellbeing as something that’s attributed to and in control by individuals… sometimes the media characterise wellbeing as happiness, which is extremely unhelpful… (interviewee #5).

Professionals in different areas will give different definitions - emotional wellbeing, mental wellbeing… You ask ten people, you get 11 different answers, essentially (interviewee #6)

The findings illustrated the complex and contested of nature of wellbeing and the co-existence of both broad and narrow definitions in the policy process. The conflation of wellbeing and happiness is quite common in the UK, not least because of the novelty of subjective wellbeing within the ONS framework and the emphasis often placed on it in policy circles (Austin 2016). A common theme of the interviews was that these definitional issues remain important for understanding the role of evidence in policy and these are discussed further below.

On the meaning of ‘what works’, interviewees tended to give responses that reflected the purpose of the What Works Centres: that it related to the use and standards of evidence. Typical responses were:

Identifying, based on evidence, what works and then doing something with that information so it leads to change (interviewee #7).

…proven interventions and approaches (interviewee #6).

In short, most responses reflected a technical notion of what works, although it was clear from answers to other questions that interviewees were also very clear of the importance of a range of other factors influencing policy (below).

5.2 The Evidence Ecosystem

5.2.1 Evidence Sources Used

Interviewees reported using a wide range of evidence sources for various policy purposes, which are detailed in Table 2. While academic research was acknowledged to inform a number of evidence sources, there was a clear signal across organisations that academic papers are not usually used directly by policy-makers. This is partly because of the volume of academic research, which means it is often received through summaries by think tanks or via seminars or face-to-face presentations. Another common theme across organisations related to the focus of academic research, which was generally viewed as not directly relevant to the questions policy-makers are dealing with.

Table 2 Evidence sources used (in no particular order)

The wide range of sources reported may appear at odds with the discussion of evidence hierarchies above: interviewees tended to reject the idea of a strict hierarchy in principle, with one suggesting that ‘If you have a hierarchy of evidence that puts certain types of evidence at the top, then that immediately narrows the amount of available evidence to you, doesn’t it?’ (interviewee #1). However, alongside this generally-held view was acknowledgement of the need to draw on different types of evidence for different purposes. For example:

I think it’s important that the ecosystem of evidence is equally valued and equally mined but that when one is looking for some direct correlation between an input and an output that might be subject to slightly more rigorous scientific, methodological means (interviewee #5).

There was a clear sense across interviewees that government preferred particular types of evidence that tended to reflect the hierarchies discussed above. However, for other organisations, particularly within the voluntary and community sector and within some local councils, participatory approaches involving their communities and/or stakeholders provided an important source of legitimacy. In some cases, these types of approaches—such as co-production—were privileged over research-based evidence because of the greater legitimacy they were seen to bring to organisations and their chosen course of action.

5.2.2 Transmission Lines

Evidence on wellbeing is drawn through a wide range of channels, which are detailed in Table 3. Here, there is some overlap with responses on evidence sources; seminars and face-to-face presentations, for example, were generally seen as both a source of evidence and a channel through which evidence is received. Again though, there is variation across different types of organisations in relation to some transmission lines. For public health bodies, for example, Public Health England was important in the transmission of evidence, holding a particular authority through its status as an executive agency of government. Similarly, NICE was widely recognised as an established and authoritative source on health-related wellbeing: a ‘trusted brand’. Several organisations paid for sector-specific information bulletins and/or used sector-specific search engines, while most made use of the internet for general searches or social media (particularly Twitter).

Table 3 Transmission lines (in no particular order)

The range of transmission lines indicates something of the breadth and complexity of the idea of wellbeing. Bodies such as NICE and Public Health England were clearly more important to those organisations with a more medicalised conception of wellbeing, while the Legatum Institute’s research (O’Donnell et al. 2014) appealed more to organisations that focused on subjective wellbeing.

Generally, interviewees emphasised the importance of professional networks and personal contacts: ‘You have the right conversations to make sure you’re not missing any tricks’ (interviewee #9). Being well connected in this way meant that individuals often did not have to seek out evidence but found it came to them. However, it was acknowledged that this could skew the evidence received. Such comments illustrate the precariousness of assumptions that policy-making is based on systematic consideration of the best evidence available.

5.2.3 Incentives

A range of internal and external incentives to use evidence were reported (Table 4), again with some variation across organisations. For civil servants, there was an expectation that policies would be based on evidence:

If you’re publishing a policy you’ve got to back it with evidence… [There are] decision-making gates to get through for big policies, like impact assessments and business cases, spending reviews. So big incentives are built into the system in some respects… (interviewee #6)

Civil servants also identified the importance of evidence in justifying policies to the public, which would be a consideration for politicians in particular.

Table 4 Incentives for the use of evidence (in no particular order)

For charitable organisations, evidence was seen as important to their credibility. It might provide a ‘seat at the table’. As one interviewee from this sector put it:

it’s even more important in the current climate in that there is a definite view that charities are kind of ideologically-driven, political mouthpieces… you leave yourself very, very vulnerable to [this criticism] if you’re not evidence-based (interviewee #7).

Generally, interviewees from all organisations seeking to influence government acknowledged that sound evidence is essential. As one local government interviewee stated: ‘We want this evidence to be so strong that the government and the Treasury cannot turn away the findings because they’re methodologically unsound’ (interviewee #9). Concern with demonstrating value for money, which had intensified in the context of economic recession, was a key factor both within all individual organisations and in the wider policy arena.

5.2.4 Problems

Numerous problems with the use of evidence were identified on both the supply and demand side (see Table 5). These ranged from awareness that evidence exists, to being able to understand it. How evidence is presented is an important issue, with a number of interviewees commenting on the difficulty of understanding academic articles and emphasising the importance of ‘plain English’ and the need for ‘a crisp summary and clear recommendations and sound methodology’ (interviewee #8). For most interviewees, not having direct access to academic papers through university libraries put a further barrier in the way: ‘Unless I can download it instantly, print it off, have it there and make sure I’ve given some time to read it, it’s no good to me’ (interviewee #9). This latter quote also highlights the common constraint of the limited time available to policy makers to engage with academic research. In addition to the issue of time were shared challenges of timing and timeframes—for example, the mismatch between research timeframes and political cycles—which often led to sub-optimal outcomes.

Table 5 Problems with the use of evidence

A further issue related to the effects of public spending cuts on strategies for using evidence in policy. As one interviewee put it:

It can take a generation to influence a community… but it is difficult to plan in advance with financial uncertainty. It stifles innovation and planning. You look for options that can give more immediate results (interviewee #2)

The quality of evidence was an important concern across organisations, with one interviewee stating ‘I think there’s a lot of really bad evidence that gives wellbeing a bad name out there… a woolly kind of perspective’. There was also common concern that the relative strength of the evidence available was often insufficiently clear. As one interviewee suggested:

I think it is beholden in publications to say, “Look, this is what we’re basing these findings or statements on and here’s the levels of evidence”. So it’s just that much more honest and open, transparent way of which evidence is being used. And letting people make up their own mind to a certain extent (interviewee #5)

Finally, a number of interviewees pointed to the complexity or messiness of the policy process. As one UK civil servant (interview 2016) put it:

…it’s not [the case of] here’s a problem, let’s explore the problem, let’s engage the public in a problem, let’s look at the evidence, let’s commission new research around it, and have lots of time to do all of this and make the best use of evidence that you can. Then come up with solutions, appraise the options, do all of that type of stuff. I don’t think that happens very often (interviewee #6).

Interviewees across the board pointed to the importance of competing issues and interests, political commitments, public opinion and other forms of knowledge (see below). Such comments highlighted a general awareness of the need to understand ‘what works’ beyond the technical sense of the term.

5.3 Other Forms of Knowledge

While there was general acknowledgement that evidence can play an important role in policy, there was also general awareness that other forms of knowledge—political, professional and experiential—were also important: and in some cases, much more so. As one interviewee put it:

People understand that the evidence ecosystem is pooled and shaped and manipulated in different directions by each of those different interests, whether it’s local advocacy, whether it’s political advocacy, whether it’s economic advocacy (interviewee #5).

Awareness of the importance of political knowledge was most prominent in the highly politicised environments of local and central government. One local government official highlighted the importance of political knowledge in the following way:

… about a third of it you go in with evidence but you’ve also got the political knowledge of - especially talking about councillors - what are their political parties? What wards are they representing? Particular issues in their ward? What sort of things they’ve funded or supported before? It’s putting it through that template of politics with a small ‘p’ at the local level and all [of] your understanding [of] the structures that the council operates within… (interviewee #2).

Another local government official (interviewee #10) made this point even more forcefully: ‘the political decision has much, much more power than everybody else. So, you know, I can do all this work and then if the political people don’t like it, they can just stop it’. She also highlighted the challenge that wellbeing faced in such a politicised context, suggesting that to privilege (subjective) wellbeing measures above other kinds would mean a change in the ‘entire culture of the council’. This point was echoed by a national civil servant:

It’s very difficult for politicians to talk about wellbeing because people feel that politically there’s not demand to express things in that form. They will focus on, for the most part, much more concrete intermediate goals, like you know, how can you reduce unemployment? They tend not to think, fundamentally in terms of wellbeing, at the moment, and particularly in politics, people are afraid that they will be ridiculed for doing so (interviewee #4).

Experiential knowledge emerged as an important theme across organisations but was emphasised most by voluntary sector interviewees. Comments included:

Experiential knowledge is really important… knowing the local market is increasingly important (interviewee #7 – voluntary sector)

Service users… can throw up implementation issues (interview #11 – charitable organisation)

We proceed on a case-by-case basis… stakeholder experience of the problem, frontline workers who are engaged in your policy or who could help to deliver solutions (interviewee #6 – national civil servant)

One local authority official explained how her council emphasised the co-production of decisions with local residents and stakeholders: this was central to its philosophy as a cooperative council. This approach presented tensions with an evidence-based approach:

One of the big problems that I face, at the moment, is that it [the idea of evidence-based policy] is constantly used by people saying to me or my colleagues [that] as long as it’s got an evidence base then it’s bound to work. And that is very, very frustrating because that’s not the case and therefore what it means is that it contradicts [the principle of] co-production (interviewee #10)

One charitable organisation also emphasised participatory approaches to shaping policies more relevant to local needs. One project led to the development of an index to guide policy based on what local people saw as the most important things in their lives. Such approaches bring potential advantages in terms or organisational and policy legitimacy but the interviewee suggested that this project ‘probably has some gaps methodologically’ that would make this approach less attractive in other contexts.

There was consensus among interviewees that much policy-making depends heavily on professional knowledge. As noted above, evidence may not be available or, at least, available at the right time. In relation to a decision over whether to invest in a significant spending programme, one NDPB official stated: ‘we’ve taken a punt on what we understand… we know that the evidence to prove that is limited. If I waited for the evidence base to prove it I wouldn’t have taken the punt’ (interviewee #9). Individuals representing membership organisations highlighted the value of professional knowledge supplied by their constituent organisations, who were often consulted on key issues.

5.4 The Challenge of Wellbeing

As noted earlier, interviewees across all organisations believed the complex nature of wellbeing and contestation over definition and measurement presents a particular challenge for the use of evidence. As one interviewee put it:

It does need to move us into areas of scientific inquiries that are much more integral or integrated across disciplines. And that challenges the way in which academia currently organises itself and scientific funding is currently distributed (interviewee #5).

There were differences between those who believe a broad range of indicators are needed and others who think that subjective wellbeing indicators are a way forward, pragmatically at least. On this issue there was no consistent pattern across organisations. So, on the one hand:

They say we can’t reduce wellbeing down to a couple of questions. No, you can’t, but if you think that it’s practical always putting 20 or 30 wellbeing questions into a survey - it’s very naïve to think that’s the case. You have to sacrifice yourself and basically, reduce or boil down to a few questions (interviewee #6).

And, on the other:

the use of subjective wellbeing as a dominant measure is because we – a lot of people - are looking for simplistic answers to very complex questions… we need to balance that with much greater objectivity (interviewee #5).

However, there was also a common view across organisations that contestation over definition and measurement were to some extent inevitable and that, as one interviewee put it, ‘There’s no point in criticising anyone’s approach on wellbeing because it has to be administration/context-specific’ (interviewee #11).

5.5 What Policy-Makers Want from the WWCW

There were plenty of suggestions on what the What Works Centre for Wellbeing and evidence-providers generally might do to strengthen the role of evidence in policy. Inevitably, many of these responded to the list of problems identified above (see Table 6).

Table 6 What policy makers want from evidence providers

A common theme was the need for complex research to be translated into short and accessible information. A number of interviewees identified a rapid growth in research on wellbeing—an ‘exponential curve’—but suggested that it has ‘tended to be a very academic subject’ that evidence providers need to ‘convert into something that’s very clearly actionable and can influence decisions’ (interviewee #6).

A number of interviewees also raised the issue of whether the WWCW should have a specific focus (e.g., mental wellbeing, subjective wellbeing). One gave the example of how the Joseph Rowntree Foundation had focused specifically on poverty and had developed a respected reputation on this issue. However, there was no clear consensus among interviewees on what this focus should be for the WWCW

There was some consensus on the need to embrace and promote different types of evidence. One interviewee stated that the complexity of wellbeing required:

…a much more modern way of addressing the issues; a much more integrated sense of all of these things together. And that lends itself to, I think, new and potentially very exciting forms of evidence… the What Works Centre for Wellbeing needs to promulgate that ecosystem and multiplicity of what we consider as justifiable evidence (interviewee #5).

Another interviewee suggested that ‘sometimes evidence gets used because it’s the most visible evidence' and the Centre should ‘bring a lot more evidence into play to make that more visible’ (interviewee #5).

Finally, some interviewees believed the Centre might also play a key role over definitional issues and in promoting a ‘common currency’ for comparison of wellbeing interventions. However, while the ONS subjective wellbeing questions were seen as having the widest recognition in this regard, and for some were the best way forward (above), there was no consensus on this issue: other interviewees suggested that the WWCW should focus on setting out different measures for different purposes.

6 Analysis

There was consensus among interviewees that the accumulation of evidence is an important next step for wellbeing in policy. The interviewees indicated considerable demand for evidence of different types and for a range of purposes. While there are issues about definition and measurement outstanding, and an ongoing challenge to persuade key actors that this is an important policy issue, interviewees generally believed that scepticism around wellbeing had receded significantly in recent years. As one put it: ‘I think people do understand that wellbeing is important… They don’t think it’s mad anymore to be measuring this. I think what they want is to do something about it’ (interviewee #6). Central to this purpose is greater understanding of what policies can enhance wellbeing, for whom and by how much. This is where evidence can play an important role.

In Mulgan’s (2005) terms, wellbeing can be considered an inherently novel policy field. As such, less is known about ‘what works’ in policy than in more established fields. The networks are less well established and there are fewer ‘trusted’ brands’ in terms of evidence providers. The creation of the WWCW is aimed at addressing these issues. However, as wellbeing moves from an abstract idea to one that is being defined more precisely for use in policy it also demonstrates characteristics of a policy in flux. What counts as relevant evidence is strongly contested and there are disputes over the basic theoretical approaches. Should, for example, wellbeing be conceived as individual wellbeing measured by subjective wellbeing indicators? Or should wellbeing be conceived as a multidimensional phenomenon in which subjective wellbeing indicators are placed alongside a raft of other, more objective indicators? The answer at this stage is not clear: while the official statistics of the ONS combine 41 subjective and objective indicators across ten domains, there is also a push to privilege subjective wellbeing, which is controversial in some quarters. As noted above, there is no consistent pattern of views across or within organisations on this issue. The interview responses on this issue were informed more by the particular views of individuals, shaped by their different theoretical perspectives and values.

Yet the WWCW has been encouraged to pay particular attention to subjective wellbeing: as a potential ‘common currency’ that would allow the effects on wellbeing of different policy interventions to be compared against the same standard more easily. This may have the effect of privileging certain types of evidence over others and may also discourage some organisations (and/or individuals) from engaging with the issue if this approach is not seen as legitimate. However, while this approach has been encouraged for the purposes of the WWCW, this is by no means the end of the matter. Indeed, while the ESRC identified personal wellbeing as a ‘particularly important dimension’, it also acknowledged that the ONS framework is broad and that some aspects of this framework would be emphasised more in some policy areas than others; and, that other dimensions may also emerge (ESRC 2014, 4). In short, as Mulgan’s arguments would suggest, in the field of wellbeing the questions are changing as well as the answers and evidence—far from floating free—is being generated and accumulated in relation to different theories and concepts that frame wellbeing in different ways. As such, these theoretical issues are inseparable from the evidence and are the only way of making sense of it.

As Majone notes, there is no easy separation of facts and values and, while much of the language of ‘what works’ implies a rational, technical perspective, how evidence is communicated is crucial. This is not simply about clarity and sound methodology but also about understanding the audience and the context. A common theme emerging from the interviews was that evidence serves very different purposes in different contexts and this shapes the nature of the evidence sought. Thus, civil servants and others seeking to demonstrate efficiency and value for money may be drawn more towards quantitative evidence, while organisations seeking different types of legitimacy (see Boswell above) may prefer participatory approaches.

Such findings highlight the usefulness of Head’s distinction between different types of knowledge. Evidence-based knowledge is an important form, holding a particular status through its perceived neutrality; although, as the research here suggests, values and ideology are not only present in the demand-side world of politics but also in the epistemic communities that supply the evidence. As noted above, researchers adopt different theoretical perspectives on wellbeing that are informed by different value structures. These perspectives determine the ‘facts’ and the data collection methods that are seen as relevant. However, explicit consideration of political, professional and experiential forms of knowledge provides nuance to understanding the role of evidence beyond the simple rational versus political dichotomy.

Political knowledge draws attention to a range of factors—values, ideologies, manifesto commitments, electoral competition, party factions, interest group pressure, the media, and public opinion, to name just some—that combine in the minds of political actors to provide a sense of what is politically feasible and what is politically desirable. So, on the one hand, the pursuit of wellbeing might be considered desirable, but not feasible for any of these reasons. On the other hand, it is plausible at some future point that the pursuit of wellbeing through policy may become politically feasible but is seen as undesirable because it conflicts with other goals deemed more desirable. These issues are more pronounced in the more politicised arenas of central and local government.

Professional knowledge is arguably dominant in much routine policy-making. While some public policy decisions are widely debated and various actors have a voice in the process, much policy making takes place in the relatively closed confines of well-established policy communities. Policy communities tend to have embedded norms and practices that constrain the potential for new policy ideas to take hold. Policy specialists know best ‘what works’ in and for their respective communities and this tends to provide a bias against abrupt shifts in direction. This does not mean that new ideas and new evidence cannot lead to significant change, but that they meet with an inbuilt tendency towards inertia when they arrive fresh at the table: a theme that emerged across organisations.

Experiential knowledge draws attention to the importance of end users, communities and residents’ ideas about ‘what works’ for them. This approach to ‘what works’ eschews the search for a ‘one size fits all’ approach to defining, measuring and delivering policy for wellbeing. It may mean a trade-off between generating evidence through means conventionally viewed as more robust methodologically in favour of capturing the views of the specific target audience for policy. As noted above, such an approach offers legitimacy for the delivery organisation and, in theory, for the specific policy that emerges. It is an approach that has acknowledged limitations in terms of scalability, and is thus less attractive to large governmental units than smaller community-based organisations, but is one that nonetheless provides an alternative to established assumptions of what constitutes valid knowledge.

7 Conclusion

Wellbeing in policy has attracted increasing interest in a range of arenas. In the UK, developments have reached the stage where the government has created a What Works Centre to bring together the available evidence to support those seeking to bring wellbeing more fully into policy. This is seen as a crucial ‘next step’. However, as is widely recognised and as this research has illustrated, evidence is just one factor among many in shaping policy developments: and often not the most important. In this context, this article has sought to shed light on the role of evidence in promoting wellbeing in policy. What the research shows is an issue that is complex, relatively new to the policy agenda and one that has contestation at its heart. In such a context, understanding ‘what works’ is a major challenge.

There is widespread agreement that evidence matters, but it is far from clear what evidence matters, why and to what extent. To begin to form answers to these questions requires a broad conception of ‘what works’ that treats evidence as one form of knowledge that interacts with other forms to shape policy. No-one believes there is a ‘magic bullet’ that can resolve deeply held differences over the nature of wellbeing, how it should be defined and measured, and the extent to which governments should seek to promote it through policy. However, this article does highlight a shared understanding across organisations of the importance of different forms of knowledge—political, professional and experiential—that are essential to the notion of ‘what works’ in practice, as well as in a theoretical or technical sense.

Beyond this shared understanding of the importance of other factors are common challenges relating to the use of evidence specifically. Some of these challenges are well documented in research on the issue in other policy fields, such as the difficulty of understanding academic literature and time constraints, which together require evidence to be presented in short, accessible forms. There are also more specific challenges characteristic of a complex and contested issue that is transitioning from being new to the policy arena to one that is in a state of flux. Contestation over definition and measurement mean that different actors seek different types of evidence according to the way in which they frame wellbeing: that is, competing theoretical conceptions of wellbeing raise different questions and require different types of answers. Moreover, there is no obvious resolution to this conflict and, indeed, disagreement over whether resolution is desirable. While it may be possible to move to a common definition in specific contexts for specific purposes, a broad consensus on these issues is not an immediate prospect. Future research might seek to explore the emerging patterns of how wellbeing is conceptualised in different contexts—either by policy area or in different political arenas—to identify any commonalities, but also to explore how different conceptions shape the nature and role of evidence in policy.

However, advocates of a greater role for wellbeing in policy view this as a long term process that competing conceptions should not deter. For many, the central purpose is to shift the focus of public policy away from a narrow conception of ‘what matters’ to people: a conception that is, in the UK at least, dominated by economic concerns and indicators. Movement in this direction has taken place without wide consensus on definitional issues, albeit with limited policy implications to date, but enough to suggest that while resolution of definitional issues might be highly desirable, it might not be essential for further movement in this direction to take place. However, there is agreement that more, high quality and accessible evidence will assist in this task: but most effectively when a broad understanding of the notion of ‘what works’ is more widely embraced.