Keywords

1 Introduction

Crowdsourcing is a broad term that describes activities where a large number of contributions from individuals are used to co-create value [1]. Crowdsourcing may or may not directly entail a problem-solving component, but there are many different ways in which it can be valuable for decision support in organisations (e.g. content production, task competition, voting, crowdfunding) [2]. The importance of crowdsourcing has been evident in public management with popular platforms that invite contributions from the public like Challenge.gov and the Open Government Public Engagement Platform in the USA [3, 4] or the UK government’s Red Tape Challenge that collects feedback on regulations [5].

In parallel to government websites, the principles of crowdsourcing have wider implications for citizen-government relationships e.g. [4, 6]. Less institutionalised forms of crowdsourcing are becoming more widespread in the form of monitoring and aggregating content from open information sources and, more specifically, social media [e.g. 7–9]. It is common that social media users might provide direct feedback on policy topics, broadcast their own information or engage in discussions that can be informative for the work of government. There are increasing signals that social media can be useful as information sources in policy making [e.g. 7, 10], however our knowledge remains much less developed compared to crowdsourcing websites. Studies of social media in the public sector focus on strategic and operational benefits [11] or models of interactions with the public [10]. Understanding the value of social media as information sources can extend our knowledge and inform current practice as social media monitoring tools are being adopted by government organisations [12–14].

This paper explores the value of social media for government crowdsourcing through a series of interviews and workshops with policy teams in the UK Department of Environment, Food and Rural Affairs (DEFRA). As an explanatory lens of how policy makers frame social media input and information flows with the public, we draw from the notion of the imagined audience. This concept has roots in conceptualisations of the public and engagement around science and technology [15–18]. In digitally-mediated environments, the imagined audience indicates how social media users frame communication contexts and navigate through the multiple audiences that they perceive they are engaging with [19, 20].

Following an elaboration of the theoretical background in the next section, the paper describes the study methodology and findings. Policy makers were largely supportive of opportunities to source useful social media content in appropriately summarised forms, but representation of social media users and audience awareness were recognised as major limitations. The paper discusses the implications of these findings with a focus on overcoming audience limitations.

2 Social Media and the Imagined Audience

Social media include online networking (e.g. Facebook, LinkedIn), content sharing (e.g. YouTube, Pinterest) and blogging/micro-blogging platforms (e.g. WordPress, Twitter). The different functionalities of social media allow organisations and individual users to develop their presence, connect with others and share content according to their diverse aims (e.g. both social and professional) [21].

The pluralism of social media inevitably leads to audience fragmentation and distribution of activities across channels. On some occasions, social media users might have obvious motivations to engage on certain platforms (e.g. LinkedIn for professional networking), but on others these boundaries might be blurred both in terms of content (e.g. Twitter updates) and composition of networks (e.g. Facebook friends). It is furthermore common to observe dynamic audiences on social media that form temporarily around events or users’ interests like TV shows [22]. As a result, the relationship between traditional and social media audiences can be challenging for organisations that are seeking to engage with new groups of the public or offer value to those who already engage e.g. [23, 24].

The imagined audience is a concept that refers to how social media users conceptualise the people with whom they are communicating [19, 20, 25]. As Marwick and Boyd emphasise [19], on most social media channels, users experience a collapse of multiple audiences into a single context; audiences that might have otherwise been distinct in the offline world like family, personal and professional contacts. Each social media platform has its own audience-feedback features (e.g. “likes”, “shares” or “followers”) [20], but in many situations it remains unclear how to select audiences or even how many users read each update. Particularly through Twitter’s conversational features (mentions, retweets, hashtags), posted messages can travel through unknown and potentially infinite audiences that are difficult to measure [19]. As social media users make assumptions about their imagined audience, it is not only a case of elevated expectations; in fact, a large study with Facebook users shows that they commonly underestimate how many people view their content [26].

Beyond social media research, the concept of the imagined audience has been relevant to stakeholder engagement studies in science and technology. It originates from observations that discourses within industries, policy-making communities and generally amongst “experts” might rely on assumptions about the “public” or “imagined lay persons” who lack expert knowledge of a topic but have legitimate concerns or expectations (e.g. about chemicals or infrastructure planning) [15, 16, 18]. Whether seen as “stakeholders”, “consumers” or “citizens”, the public is generally perceived as a resource that needs to be managed even if not completely understood. Processes of conceptualising the public by experts usually have high influence on engagement practices and, subsequently, drive reactions from the public about technology trajectories (e.g. investments in renewable energy sources) [17, 18].

The concept of the imagined audience can illustrate important issues about the potential of social media as information sources in government. Related work mainly refers to public input and collaborative actions during emergency events e.g. [14, 27]. A study by Bekkers et al. [7] further suggests that in the Netherlands, organisations with established surveillance mechanisms like the police are more willing to consider social media as sources of information in comparison to policy teams in other departments that prefer the monitoring of closed information spaces (e.g. forums). In the UK government, there is some evidence of crowdsourcing exercises taking place to proactively identify conversations of interest, for example, in incidents of public health or campaigns about food safety and hygiene [12]. More technical approaches to social media crowdsourcing by Charalabidis et al. [8, 9] place emphasis on design and content aggregation elements so that policy makers are able to overview a large amount of information; the authors distinguish this approach as “passive” or “non-moderated” crowdsourcing.

As policy makers are considering the role of social media input, they inevitably have to make assumptions about the imagined audience. In practice, they need to “imagine” who are the people they are listening to or engaging with. At the next level, they need to make assessments about the usefulness of the collective input produced by social media audiences and the extent to which it can influence decisions. Many of these assessments about information flows with the public in policy making are not new, e.g. [28], but the volume, complexity and diversity of social media sources points to the need for a more detailed examination. Starting from the concept of the imagined audience, there is broader scope to examine the different crowdsourcing contexts in which policy makers turn to consider social media input.

3 Study Methodology

The research was organised in the form of a case study [29] and carried out as part of a wider project with the UK Department of Environment, Food and Rural Affairs DEFRA in which the researchers were involved. The selection of DEFRA as the case organisation for this study represents the typical but also the influential case [30], since DEFRA’s work draws heavily on engagement with the public and the use of external stakeholder input in policy decisions. Furthermore, DEFRA hosts a large number of policy teams in different topics where crowdsourcing practices are potentially relevant but the size and composition of involved audiences differs; for example, activities range from farming reform and flood protection to specialised environmental issues like forestry, chemicals and pesticides.

Data collection for the scope of this study took place between November 2014 and February 2015 and involved three sources:

  • Seven semi-structured interviews that lasted for one hour on average and were taped and transcribed following permission from participants. Further to the input from the interviews, the research team had opportunities to follow up with participants or their colleagues on topics of interest.

  • Two workshops with five and six participants respectively. The workshops involved a demonstration of social media monitoring and visualisation tools relevant to a pilot analysis of farming networks on Twitter. Participants were asked to provide feedback on the value of the tools and brainstorm about future requirements. The workshops were not recorded but extensive notes were taken. Although providing full details about this exercise is not possible within the scope of this paper, the case findings include examples relevant to conceptualisations of the audience.

  • A wide range of documentary evidence from a selection of policy topics, including consultations, response to consultations, social media posts and evaluation reports.

Selected participants for the seven interviews came from different levels of the civil service and policy areas mainly related to communications and regulations about the environment, farming and local growth. They are also involved in all the different stages of DEFRA’s policy-making lifecycle (see Fig. 1 in the next section). Participants were first asked about their role within the organisation. Interview discussions then evolved around the following main questions:

Fig. 1.
figure 1

DEFRA policy cycle

Full size image

  • Generally, what type of input from external stakeholders does your role require? What kind of information flows support this input (e.g. consultations, surveys, other stakeholder engagement activities)?

  • What are the different groups within the public that you would like to reach?

  • How could information from social media change the ways in which you understand the needs of external stakeholders and the public?

  • How do you think input from social media could support the work of your policy team and DEFRA in general?

Data analysis was carried out thematically based on the methodology described by Braun and Clarke [31]. This approach to data analysis is suitable for exploratory research as it allows the flexible documentation of main themes from interview data. The focus of the analysis was on the identification of themes related to audience perceptions and information flows with the public. In most interviews, participants drew their narratives around social media audiences using examples from their own knowledge domain or experiences (e.g. following a Twitter hashtag or reading reports from social media analytics tools). The next section provides some more background about DEFRA’s work and presents an overview of the findings.

4 Case Background and Findings

DEFRA is one of the largest government departments in the UK with remit in policy and regulation related to environmental protection, food production and standards, agriculture, fisheries and rural communities. DEFRA’s objectives and priorities include the improvement of technical infrastructure in rural areas, increasing exports and competitiveness in the food chain, simplifying farming regulation and improving water quality [32]. The department employs over 10,000 staff working across 36 agencies and public bodies in England with devolved administrations in Wales, Scotland and Northern Ireland. There is also extensive cooperation with European Union authorities for environmental policies, including the high profile Common Agricultural Policy that involves a system of agricultural subsidies and programmes for farming and rural development.

4.1 Policy Making at DEFRA and the Role of Social Media

DEFRA’s approach is explicitly focused on: (1) developing capabilities for evidence-based policy making (collecting evidence from as many sources as possible) and (2) implementing initiatives that aim at positive behavioural change (e.g. sustainable consumption, energy labelling, reducing food waste) [32, 33]. This approach to policy making is summarised in the form of the Policy Cycle shown in Fig. 1. Compared to more general models, it places emphasis on issue definition and situation understanding. This is necessary for the work of DEFRA due to the high complexity, economic impact and technical nature of environmental issues as well as the wide variety of stakeholders usually involved.

Engagement with the public is also explicitly one of DEFRA’s priorities. The organisation has an overall commitment to evidence-based methods that meet criteria of rigorousness. This involves policy consultations, social science research and public understanding studies (e.g. geographical mapping, experiments, surveys and focus groups). There are also extensive stakeholder management activities involving professional associations, academic research teams and other external experts.

An important hub of DEFRA’s public engagement and dissemination activities are the social media accounts. The department’s presence has been organised on Facebook, LinkedIn, Google+, Storify and Flickr as well as 12 different Twitter accounts, which support diverse policy areas and communication needs (e.g. advice lines for farmers and fisheries, rural news, official statistics, Smarter Guidance and Data, air or water quality feeds). The main Twitter account @DefraGovUK has over 70 K followers and the more specialised accounts might have from fewer than 1 K followers to over 5 K. Some of these feeds are automated while most are managed individually by policy teams following internal guidance and training. Most of the department’s 36 agencies and public bodies also manage their own social media presence on a selection of channels. Some of these accounts are clearly defined as informational in their purpose (news feeds) – others experience varying levels of interaction with the public.

4.2 Sources and Forms of Social Media Input

As many of DEFRA’s traditional stakeholders have developed a digital networking presence, the potential value of social media input in policy decisions per se was rather uncontested by interview participants. There were however diverse opinions about the value of different sources and forms of social media input. Useful sources identified during interviews included a range of blogs, communities of practice, LinkedIn groups and Twitter hashtags/lists; content sharing websites and comments below popular news articles were generally considered as less important. Policy makers were developing their own assessments about the value of online sources based on accumulated experiences.

In terms of using social media input, there was wide agreement that for input to be considered as useful in any form it had to be relevant and appropriately summarised. An important example of this was #AgriChatUK, a national Twitter conversation that takes places weekly to discuss topics around farming. At least on two occasions, summaries from discussions were used as input to the appropriate policy teams – even as official consultation response. #AgriChatUK discussions were useful due to their clear focus, regularity and availability of weekly summaries. For less clearly relevant or more dynamic conversations, the sourcing of potentially useful content was more difficult due to technical and time constraints. Interview participants generally understood that open information channels like Twitter host a large amount of frequent, immediate and potentially relevant content, however the high “noise to signal” ratio made the value of this content not so obvious.

The use of commercial social media management and monitoring tools partially addressed this challenge. The organisation had experience with such tools at the central level in the context of monitoring popular trends and collecting updates for national campaigns or high-profile conversations. For example, a lot content was captured to oversee the reactions of Internet users against DEFRA’s decision to implement a badger cull in 2012 (including content about a popular petition). Monitoring tools either managed internally or with the support of media companies would filter a large incoming flow of content and then produce ad hoc or periodic reports of trending content and influential contributors. This approach focused on popularity measures to select topics and filter content (e.g. retweets).

While this type of central monitoring extended traditional media briefings, some of the department’s agencies had adopted monitoring software for more specialised needs. The most important crowdsourcing context was emergency events where DEFRA and its affiliated agencies need to facilitate timely communication of risks to the public and guide to appropriate actions. For example, the Environment Agency is the body responsible for handling emergencies related to natural disasters. Its main Twitter account has over 250 K followers and is part of Twitter Alerts, the network’s official warning system. Monitoring related to emergencies like floods provided a clearly defined set of keywords and timeframes for sourcing and interpreting content. This was achieved through a combination of flood-related hashtags, direct mentions from the public and scanning content from open sources.

4.3 Conceptualisations of the Social Media Audience

Aligned with DEFRA’s commitment to evidence-based policy making, all participants had a good understanding of traditional stakeholders, important influencers and the value of different public engagement activities. Subjective evaluations of the social media audience proved a quite challenging task that was evidenced in a set of common themes during the interviews and workshops.

Policy makers usually had to identify or make assumptions about how specific groups of professionals have a presence on social media, how they connect to each other, how they create content and whom they represent. Answers to these questions could be more straightforward for social networking groups (e.g. on LinkedIn) but less obvious on channels that support open information flows and dynamic conversations like Twitter, Facebook and blogs. For example, the popular discussions on #AgriChatUK suggested that many farmers and agricultural businesses have a presence on Twitter; however, mapping those networks and absorbing useful content outside specific conversations was challenging. Twitter lists or keyword searches could act as a first step of filtering but still resulted in a large amount of unstructured content that did not include information about the audience.

It is important to emphasise that audience limitations were not simply a matter of lacking demographic information about users but mainly about issues of sampling and representation. Ad hoc feedback suggested that social media users include a variety of domain experts as well as many users who are not experts but have a primary stake in policy topics. Furthermore, it was understood that social media users themselves collapse audiences into a single channel, hence posting content at diverse frequencies and with different intentions about whom they are talking to. As a result, monitoring social media content around keywords only captures the perspective of those users who decide to make a contribution within a specific timeframe, which inevitably leads to a “self-inclusion” perspective.

For policy makers with training in social science and economics research, exploratory analytics methods from large datasets of unstructured content could not be used as “evidence” the same way as traditional methods unless sampling and representation issues could be addressed. For example, our study identified an estimated network of 10 K or more Twitter users from the UK that tweet about issues relevant to farming. Analysis of a sample of large datasets over a period of six months revealed that these users post: (1) information about practical aspects of farming and rural life (including sharing photos), (2) comments and contributions to campaigns about topics like the price of dairy products and (3) to a lesser extent, opinions about the general state of the farming profession with reference to government decisions. Representation issues here were not per se related to the fact that a potential audience of 10 K Twitter users is only a small proportion of an estimated total of 250 K farmers and agricultural businesses in the UK [34]. The issue was that, apart from a general awareness of their professional identity, there was no systematic information in tweets or account metadata about who these users might be and what motivates them to contribute to specific discussions.

Despite limitations of audience awareness, study participants were confident to identify a positive aspect of crowdsourcing from large but unknown audiences. Compared to traditional methods and closed systems approaches to crowdsourcing, social media included more opportunities to source opinions from “real” people or groups of the public that extend beyond stakeholders who make regular contributions to policy consultations. An interviewee with experiences in assessing input from the Red Tape Challenge, the UK government’s crowdsourcing system, highlighted the benefits of reaching more “real” people. Therefore, even if crowdsourced contributions could not be used as hard evidence, they could be valuable to broaden the perspective as information sources complementary to consultations.

5 Discussion and Conclusion

DEFRA provided a stimulating case to look at the value of social media as information sources due to the organisation’s broad remit in environmental policies and commitment to evidence-based policy making. The concept of the imagined audience [19, 20, 25] framed our understanding of how policy makers conceptualise input from social media. It is important to understand how these subjective evaluations emerge because they can highly affect the extent to which contributions from the public are seen as a useful resource in policy decisions [15, 17, 18].

Indeed, our study found that there are important audience awareness issues when considering content from open forms of crowdsourcing on social media. Simply, policy makers find it difficult to understand or even make solid assumptions about whom they are listening to. Broadening the perspective of the audience was nevertheless recognised as an important prospect with opportunities to source opinions from groups that might not otherwise engage. The study also found that social media monitoring tools are used more widely to source contributions in clearly defined contexts like trending discussions, high-profile campaigns or emergency events. While in emergencies there is clear scope and timely monitoring is critical, the attention to popular discussions and campaigns rather follows Mergel’s thoughts on using descriptive insights from popularity content [13].

It is interesting to compare those findings with analyses of policy crowdsourcing websites and particularly the UK government’s Red Tape Challenge. Lodge and Wegrich [5] report that audience awareness was also an issue due to the anonymity of submissions, but the main shortcoming was that there were no explicit intentions or mechanisms to integrate input from the system in decisions. Content from social media sources gives access to a much larger pool of spontaneous, mostly not anonymous, but also less structured contributions. Policy makers were more confident that this type of content can provide useful insights if it is appropriately summarised as long as comes with contextual information that facilitates assumptions about the audience.

After identifying the importance of audience limitations, we need to consider how they can be overcome. A systematic way can involve the concept of an audience or crowd capability that is constructed and managed by an organisation [35]. Findings from our study suggest that developing such a capability could involve several levels of thinking including the following:

  • The need to focus less on content itself and more on the composition of information networks or understanding how different groups of the public interact and engage in discussions (e.g. contributors of #AgricChatUK). This transition can have technical implications for the selection or development of social media monitoring tools that need to enable network-feedback features.

  • Groups of the public can respectively be encouraged to organise and connect on social media so that their contributions can be sourced. This can be a task supported by intermediary organisations like professional associations, trade unions or other representation bodies that could facilitate professional networking and raise the profile of input from their own audiences.

  • From the government perspective, the sourcing of contributions from open information sources can become more explicit to the public. Bekkers et al. [7] place this suggestion mostly in the context of surveillance; we position it as an opportunity to reiterate commitment to listening.

  • Policy processes have to consider subjectivities of the audience as inevitable and understand their methodological implications. Earlier stages of the policy making lifecycle might seem more suitable for exploring social media input but our study indicated high interest for commitment, implementation and evaluation activities as well (Fig. 1).

These suggestions should be taken into account with considerations to the contextual limitations of this research. DEFRA’s Policy Cycle and commitment to evidence-based policy might not be the most fruitful ground for experimental approaches to new data sources like social media. Furthermore, the identification of certain themes about social media and the audience by study participants cannot be seen as a complete overview of perceptions within the organisation or across the UK government. Finally, we need to consider that environmental and agricultural policies tend to attract a large number of views from diverse publics. In other policy topics, there might not be that much potential insight or the audience is more uniform, hence making assumptions about its composition less challenging.

Since the value of crowdsourcing and social media input might differ across policy topics, it would be important to examine activities or stages of the policy making process during which the social media audience is seen as a useful resource. Suggestions from reviews of crowdsourcing studies can provide several starting points and analytical models to be further developed and elaborated on in a public sector context [1, 2, 35]. Research in the area can also involve further case studies and exploratory analyses so that we can learn more about social media audiences and how to facilitate assumptions about their composition.