Keywords

1 Introduction

Crowd-sourcing has been initially developed in the private sector, in order to exploit external information, knowledge and ideas possessed by ‘crowds’ of individuals for problem solving or for the development of innovations [4–6, 17, 18]. It is defined as ‘a new web-based business model that harnesses the creative solutions of a distributed network of individuals, in order to exploit ‘collective wisdom’ and mine fresh ideas from large numbers of individuals’ [4]. There has been extensive research and practice in this area, which has led to the development of efficient and effective crowd-sourcing methods and practices, and also to gaining a deeper understanding of them, their value, advantages a disadvantages, and also the specific contexts and kinds of problems for which each of them is more appropriate for; comprehensive reviews of this research are provided by Rouse [36], Hetmank [15], Tarrell et al. [41], Pedersen et al. [29] and Rechenberger et al. [35].

The public sector, motivated by the multiple ‘success stories’ of this new knowledge sourcing and innovation paradigm, and also by the increasing complexity of social problems and needs, has started making some first steps in this direction, by introducing forms of ‘citizen-sourcing’ in their policy making processes (see Sect. 2.1 for a brief review of relevant literature). However, there is much less knowledge on efficient and effective methods and practices for public sector citizen-sourcing in comparison with private sector crowd-sourcing. It is therefore necessary to conduct extensive research in this area, in order to develop highly efficient ICT-based methods for this purpose, which enable the effective collection of policy relevant information, knowledge and ideas from citizens, and then the advanced processing of them in order to calculate useful policy analytics, which can provide substantial support for public policy making, and addressing its specific needs. In general, it is important in the area of public sector citizen-sourcing to reach a level of knowledge, efficiency, effectiveness and maturity, comparable to those of the private sector crowd-sourcing.

This paper contributes to filling this research gap, by presenting an ICT-based method for ‘passive expert-sourcing’, with the latter term denoting the collection of policy relevant information, knowledge and ideas from experts, in order to support policy making by the European Union by leveraging its large policy community. The first evaluations of citizen-sourcing initiatives [1, 12, 23] have shown that they can provide valuable insights into the perceptions of the general public concerning important social problems and government activities for addressing them, and also existing and proposed public policies for addressing them. However, it is recommended that in order to collect information and knowledge of higher quality, it is necessary to target more knowledgeable communities having strong interest and good expertise on the particular topic/policy under discussion. On this question there has been extensive political sciences research, which has revealed the importance of both democracy (i.e. political consultation with all stakeholder groups) and technocracy (i.e. specialized knowledge of experts) for the development of effective public policies for addressing the complex problems of modern societies [7, 13, 14]. At the same time this research highlights the need of a relationship and balance between them (see Sect. 2.2 for more details on ‘Democracy vs Technocracy in Public Policy Making’). Furthermore, another stream of political sciences research has examined the emergence of policy networks, as a result of the increasing complexity of social problems, in which participate various both governmental actors and non-governmental actors (such as associations of various businesses, professions, labor unions and other interest groups), and their increasing importance for the design and implementation of public policies; each of these actors has developed valuable expertise, usually focused on its particular perspectives and concerns, which can be quite useful for public policy making [33, 34, 40] (see Sect. 2.3 for more details on ‘Public Policy Networks’). So, our research has as theoretical foundation this abovementioned previous work on the relationships between democracy and technocracy, and also on the public policy networks.

The research presented in this paper has been conducted as part of the European research project ‘EU-Community’ (project.eucommunity.eu/), partially funded by the ‘ICT for Governance and Policy Modelling’ research programme of the EU. This paper is structured in six sections. In the following Sect. 2 the background of our research is outlined (on public sector citizen-sourcing, democracy vs technocracy in public policy making, and also public policy networks). Then in Sect. 3 is presented the methodology we adopted for the design of the proposed method and then for a first evaluation of it. In Sect. 4 the proposed ICT-based method of passive expert-sourcing is described, followed by the results of a first evaluation of it in Sect. 5. Finally, in Sect. 6 conclusions are summarized and directions for further research are proposed.

2 Background

2.1 Public Sector Citizen-Sourcing

As mentioned previously in the Introduction, for public sector citizen-sourcing there is a lack of research similar to the one that has been conducted for private sector crowd-sourcing, having similar levels of breadth and depth, probably because the former is a more recent phenomenon than the latter. Limited research has been conducted concerning the application of crowd-sourcing ideas in the public sector, the development of efficient and effective methods and practices for this purpose, and the evaluation of them from various perspectives [12, 16, 22, 24, 27, 28, 37, 39].

Most of the existing literature on ICT-based methods for citizen-sourcing by government agencies is focusing on ‘active citizen-sourcing’, which aim at the use of government agencies’ web-sites or social media accounts, in order to pose a particular social problem or public policy direction, and solicit relevant information, knowledge, opinions and ideas from citizens. In this direction Mergel and Desouza [27] describe and analyze the Challenge.gov initiative the U.S. Office of Management and Budget, which aimed at applying private sector crowdsourcing ideas in the public sector, in order to source from citizens ideas, knowledge, and solutions for specific challenges that government faces. This initiative was based on an ICT platform allowing U.S. federal agencies to launch contests, and at the same time citizens to find appropriate contests and participate in them providing solutions, or reviewing and evaluating solutions provided by others, voting on solutions, and even getting involved in the implementation of solutions and subsequent evaluation of them. Charalabidis and Loukis [8] and Ferro et al. [12] propose a method for the systematic, intensive and centralized exploitation of web 2.0 social media by government agencies on public policies (existing or under development). This method is based is on a central ICT platform, which (a) publishes automatically various types of policy-related content (e.g., short text long text, images, video) in multiple social media accounts of a government agency, using the application programming interfaces (API) of these social media, soliciting citizens’ feedback on them; and (b) similarly collects automatically from them data on citizens’ interactions with this content (e.g., views, comments, ratings, votes, etc.), and makes advanced processing of them. However, more recently, there has been some interest in ‘passive citizen-sourcing’, which aims to exploit political content that has been developed by citizens freely, without any direct stimulation or direction by government, in various external (= not belonging to government agencies) web-sites or social media, such as political fora, news web-sites, political blogs, Facebook, Twitter, etc. accounts [3, 9, 44].

Therefore, extensive further research is required in the area of public sector citizen-sourcing, in order to develop a considerable knowledge base on it, and through it a level of efficiency and effectiveness, comparable to those of the private sector crowd-sourcing area. Our research makes a contribution in this direction, by developing an ICT-based method of passive expert-sourcing.

2.2 Democracy vs Technocracy in Public Policy Making

The increasing complexity of the problems of modern societies, the globalization of the economy and the development of technology have increased the need for and importance of knowledge and expertise for the design and implementation of public policies [7, 13, 14, 21, 30, 31]. This has led to the establishment of various expert bodies (in government agencies competent for the formulation of public policies, and also the other public policy stakeholders, such as associations of professions, labor unions, businesses and other interest groups), which can have various forms, from committees to separate organizations (e.g. economic institutes). These expert bodies have become today highly important for and influential on the formulation of public policies, and this is termed as ‘technocracy’ [13, 21]. So today it is widely recognized that the two fundamental bases of public policy making are democracy (i.e. political consultation with stakeholder groups) and technocracy (i.e. knowledge of experts).

However, political sciences research in this area has highlighted the need of a relationship and balance between them, as each of them needs inputs from the other. In particular, participants in the democratic processes (various stakeholder groups, and even active citizens) need relevant knowledge and expertise, and the lack of them can have quite negative impacts [42]. At the same time the experts dealing with a particular social problem/public policy also need inputs from the political process, concerning diverse values and concerns of different stakeholder groups, and also their diverse perspectives, approaches and ideologies. For the above reasons Brown [7] argues that democracy and technocracy are not in conflict, and their combination is a necessity in todays’ realities of highly complex social problems and needs, and globalization; they generate different kinds of knowledge, which are both necessary for public policy making. The ICT can be very useful for the required exchange of knowledge between democracy and technocracy. Our research contributes in this direction, developing an ICT-based method for transfer of knowledge from the latter to the former.

2.3 Public Policy Networks

The increased complexity of social problems and needs also led governments to realize that their ‘classical’ unilateral modes of governance are insufficient, and they need knowledge resources and cooperation of non-state actors (initially economic actors and later other social actors as well) in order to design and implement effective policies, and this resulted in the development of public policy networks [33, 34, 40]. They are defined as sets of formal and informal institutional linkages between various both governmental actors and non-government actors (such as associations of businesses, professions, labor unions and other interest groups) structured around shared interests in public policy-making and implementation. In public policy networks the non-state actors provide to the state actors on one hand information, knowledge and expertise, and on the other hand support for the design and implementation of public policies, and legitimization of them; in return the former have the opportunity to influence the public policies (e.g. legislation, allocation of government financial resources) towards directions that are beneficial to them [2, 19, 25, 43].

A critical characteristic of a network is the density of interactions among its participants; according to [25] higher density of interactions has positive impact on the timewise stability of the network, the development of shared values and beliefs concerning desirable policy objectives and instruments, and finally the effectiveness and outcomes of the network. Therefore, ICT can be very useful for increasing this critical characteristic at a low cost, and supporting the exchange of diverse expertise and knowledge among participants. Our research makes a contribution in this direction, developing an ICT-based method that supports the exchange of expertise and knowledge between the actors participating in a policy network.

3 Design and Evaluation Methodology

In order to design this ICT-based method of expert-sourcing and its supporting ICT platform thirteen workshops have been organized, with the first five of them aiming to gain a better understanding of the structure of the EU policy community, and then the next eight aiming to collect the requirements of potential users of our method and ICT platform, as part of the preparation and the implementation of the abovementioned EU-Community project. The EurActiv.Com (a leading EU policy online media network (www.euractiv.com), which participates as partner in the EU-Community project, and the Fondation EurActiv Politech (a public service foundation (www.euractiv.com/fondation) having as main mission ‘to bring together individuals and organizations seeking to shape European Union policies’, participating also as partner in this project, were the organizers of these workshops. The participants were representatives of important EU policy stakeholders (such as industry federations), members of the advisory boards of EurActiv.Com and Fondation EurActiv Politech, thematic experts in various EU policies, policy analysts, registered users of EurActiv.Com portals, and also permanent staff of various hierarchical levels from the European Commission.

In order to conduct an initial evaluation of the proposed method and its supporting ICT platform an evaluation session was organized with the participation of similar potential users. During this session the proposed method was introduced to the audience, together with the supporting ICT platform, and some first applications with their results. Then the participants had the opportunity to interact with the ICT platform by executing a set of predefined usage scenarios, under the observation of the organizers who supported them, and recorded any comments or difficulties, and as well feedback on possible improvements. Finally, we collected evaluation data from the participants in this session using mainly quantitative techniques, as they enable condensing and summarizing a large quantity of evidence in a few numbers that enable the easier drawing of conclusions [26, 32]. We developed our evaluation framework based on the models developed in previous technology acceptance research [10, 20, 38], which has concluded that the intention to use a new technology, is determined mainly by two factors: its perceived ‘ease of use’ (= the degree to which potential users believe that using it would require minimal effort) and its perceived ‘usefulness’ (= the degree to which potential users believe that using it will enhance their job performance) [11]. So our evaluation framework has been based on these three factors, which have been elaborated and analyzed into several detailed questions, taking into account the particular objectives and specificities of the proposed method. This elaboration has been made separately for each the two main components that the users of the ICT platform can access: the Euractory (which enables users’ registration and reputation calculation, rating other users and also searching for experts on a topic) and the PolicyLine (which provides a visualization of document search results). Based on the above evaluation framework a questionnaire was designed to be filled by the session participants; the questions of the framework were converted to positive statements, and the respondents were asked to provide the degree of their agreement/disagreement with each of them in a five-levels scale (1 = totally disagree, 2 = disagree, 3 = neutral, 4 = agree, 5 = totally agree). The questionnaire is shown in Table 1. The data collected through the questionnaire were processed using Excel. Furthermore, after filling this questionnaire a qualitative discussion was conducted with the participants on the same questions, in order to get a deeper insight of their perceptions.

Table 1. Evaluation results
Full size table

4 A Passive Expert-Sourcing Method

4.1 Description

From the interviews we conducted (see first paragraph of Sect. 3) it was concluded that the main need of EU policy stakeholders is to be better informed on the most knowledgeable and credible experts on a policy related topic they are interested in, and also the most relevant documents on such a topic; it will also be useful if these documents are associated with the various stages of the E.U. policy processes. Since experts usually do not have time to generate new content on a topic (social problem or public policy – existing or under development) they are interesting in, the use of ‘active citizen-sourcing’ would not be possible. Therefore, a ‘passive citizen-sourcing’ approach should be adopted, based on the retrieval, processing and exploitation of already existing experts’ generated content. So, the proposed method is based on retrieving automatically information from various sources about experts on policy related topics, and then collecting the knowledge and opinions they share online through texts and postings in multiple web-sites and social media they are using. This can be achieved by crawling at regular time intervals the most relevant external sources of knowledgeable and credible experts on EU policies, and also of relevant documents of various types, and update automatically the corresponding databases of a supporting ICT platform. The practical application of this method will lead to the collection of a large amount of information concerning policy experts and content generated by them. So, it is important to apply automated state-of-the-art techniques for processing and classifying this content, in order to extract interesting insights and knowledge from it concerning social problems and public policies. This textual content of documents, articles and social media posts will be processed using opinion mining and sentiment classification methods, in order to identify subjective information, extract opinions, identify the polarity of their orientation (positive, negative or neutral) and assess the relevance of them with relation to a topic (see [10] for more details). Furthermore, for the experts it is necessary to apply digital reputation techniques for assessing their reputation/credibility and provide a ranking of them per topic of interest. By storing the above data in a common database, and enabling search of it by the users and visual presentation of the results, public policy stakeholders will be able to find useful expert knowledge on complex policy debates, e.g. the most reputable/credible experts or the most relevant documents on a specific topic. An overview of the proposed method is shown in Fig. 1.

Fig. 1.
figure 1

An overview of the proposed method of passive expert-sourcing

Full size image

The proposed method aims to foster collaboration and knowledge sharing among the different policy stakeholders on E.U. policy topics. For this purpose, in order to enable focus on a particular policy topic of interest, our method uses the concept of ‘policy process’, under which all relevant information on experts and documents is collected and clustered. In particular, as a policy process can be modelled any ongoing or completed EU legislative procedure, or political debate in general, on a topic, while each topic can be associated with one or more policy processes. Policy processes can be initiated by any policy stakeholder in order to enable the interconnection and presentation of all relevant information and aspects of policy consultations in a structured way.

4.2 ICT Platform

An ICT platform has been designed for supporting the implementation of the above method. It consists of two main components accessible by the users, called ‘EurActory’ and ‘PolicyLine’, with each of them including several sub-components, and also an additional component called ‘CurActory’, which is not directly accessible by the users, and includes the database storing the above information and the functionality for updating it by crawling pre-defined external sources at regular time intervals.

The EurActory component collects and maintains a directory of profiles of people with high levels of knowledge, expertise and credibility in one or more topics related with EU policies, usually having an active role in policy making processes at European level. According to their role they are categorized into the three types, which have been identified though our workshops: influencers, analysts and institutional decision makers. These people are included in the people database of the system in the CurActory component automatically by the crawlers sub-component, or manually by using the self-registration capabilities provided by the EurActory component. The crawlers component crawls at regular time intervals various external sources, which can be pre-defined websites (e.g. Euractiv.com, EUR-Lex, Europa Whoiswho directory, RSS Feeds, blogs and news sites) or social media accounts (e.g. LinkedIn, Twitter), updating the CurActory database, which makes the relevant discovered information available through the EurActory or the PolicyLine.

Furthermore, this EurActory component provides rankings of the expert profiles according to their expertise on a set of topics of interest, through the ‘reputation score’ calculated by the Reputation Management sub-component for each expert per topic, based on the following criteria:

  • Self-evaluation: direct input from the user on his/her own area of expertise.

  • Peer-assessment: based on endorsements from other users made through EurActory

  • Business Card Reputation: based on the reputation ranking of the organization and the user’s position in the organization’s hierarchy

  • Document Assessment: results of authored documents’ assessment by their readers

  • Network Value: level of influence as the sum of network connections

  • Proximity trust: level of connection in social media

  • Past Measurements: taking into account reputation in previous months (its stability means credibility).

  • Offline Reputation: manually added for persons with no online presence

Also, the EurActory provides the following capabilities to registered users:

  • Search for an expert profile, by name, EU policy or topic, which returns experts found in descending reputation score order (i.e. the most reputable first).

  • View an expert profile; the profile pages can also be shared on social media.

  • Create own profile and curate personal information, connect social media accounts, claim expertise topics.

  • Activation of an expert profile that has been already created by the system administrators, after the discovery of it by the crawlers, and also update of profile details.

The ‘PolicyLine’ component provides state-of-the-art visualization of policy relevant documents, which are structured according to policy processes, aiming to provide to the user a better understanding of the multi-actor processes related with the EU decision making procedures and policy debates. Therefore, in the core of PolicyLine functionality is the concept of ‘policy process’ (described previously in Sect. 4.1). We can have documents manually attached by users of the ICT Platform, to the specific ‘policy process’, as well as automatically discovered ones by the abovementioned crawlers sub-component, which searches on regular basis multiple web-sites and social media accounts in order to find significant documents published online (media articles, reports, tweets, policy proposals, legislative documents), authored by the experts’ categories mentioned above.

In particular, PolicyLine provides statistical information for each policy process selected by the user, such as the total number of relevant documents and the number of visits of users on the specific policy process page. For a more detailed view, PolicyLine offers a timeline visualization (see Fig. 2), which structures the main documents (based on relevance as well as author’s reputation) associated with this policy process in a temporal order, and clusters them under a set of user defined stages of the particular policy process. It also provides information with respect to their authorship (colors are used for this purpose to reflect different authors’ categories and sub-categories), and also shapes (such as rectangles and circles) to reflect different types of documents (e.g. rectangles reflect the proposal documents, while general documents are represented by circles). Documents sub-categories are defined concerning the type of organization from which each document is originated (e.g. European Institution, National/Local Governance, Academic Institution, Civil Society Organization, Media, etc.). The sizes of the shapes representing these documents reflect their relevance and author’s reputation (so more relevant documents written by more reputable authors are shown bigger).

Fig. 2.
figure 2

PolicyLine timeline visualization

Full size image

Moreover, a user can select a particular document in order to view more details about it, including the results from the sentiment classification provided by the opinion mining sub-component (a linguistic analysis of the textual content of a document leads to an estimation of the polarity of the underlying text – see [10] for more details) and the relevant input provided by other platform users. In particular, PolicyLine for each document provides an interface where users can rate its accuracy, value, relevance and timeliness, and also enter comments on the document, so that an informal discussion on it can be stimulated.

5 Evaluation Results

In Table 1 are shown the results of the processing of the data collected through the evaluation questionnaire (average ratings for all questions) for the two main components that can be accessed by the users, EurActory and PolicyLine.

We can see that the respondents find the ease of use of the EurActory component high (the average rating of relevant questions is 3.9), and for the PolicyLine component moderate to high, but closer to the latter (the average rating of the relevant questions is 3.67). Slightly lower are their perceptions with respect to usefulness, which it is perceived as moderate to high for the EurActory component (average rating of relevant questions 3.5), and moderate to high, but closer to the former, for the PolicyLine component (average rating of relevant questions 3.3). Finally, high is the intention to use again the PolicyLine component again (average rating of relevant questions 3.9), and slightly lower for the EurActory component (average rating of relevant questions 3.75).

In the qualitative discussion with the participants of the evaluation session the latter agreed that this ICT platform, and the whole method behind it, constitute an easy to use tool for finding quickly high quality information and opinions on important policy related topics and policy formulation processes, authored by knowledgeable experts, and also debate over them with other users. Furthermore, it enables and promotes communication and exchange of knowledge among EU policy stakeholders. It also allows awareness of and also debate and criticism on policy initiatives carried out by the European Institutions. The usefulness of the EurActory component was assessed a bit higher than the PolicyLine; this probably reflects that the former is easier to use and exploit its capabilities than the latter. The participants were in general satisfied with the proposed method and its supporting infrastructure, and expressed interest in using again the functionalities of both components. However, future improvements were suggested, concerning the graphical interface and especially the timeline visualization.

6 Conclusions

In the previous sections of this paper an ICT-based method for ‘passive expert-sourcing’ was presented, which allows the collection of high quality policy relevant information, knowledge and ideas from knowledgeable experts, aiming at supporting policy making in the European Union (EU) by leveraging its large policy community. Its theoretical foundation is previous theoretical work on the relationships and the required balance between democracy and technocracy, and also on policy networks. The proposed method is based on EU policy experts’ profiling and reputation management, relevant documents’ opinion mining and relevance rating, and finally advanced visualized presentation of them. At the political level, its objective is to enable a better interconnection of the two important bases of modern public policy making, the democratic processes and the technocratic expertise (which is of critical importance, as discussed in more detail in Sect. 2.2), by supporting the transfer of knowledge from the latter to the former. In particular, it aims to support the efficient and effective retrieval by various actors of the democratic processes (e.g. representatives of stakeholder groups, journalists, government employees, active citizens, etc.) of diverse expert information, knowledge and ideas on a specific topic/policy, which is included in postings and texts authored by knowledgeable experts and published in various web-sites and social media. Furthermore, the proposed method of passive expert-sourcing aims to increase the density of interactions among the actors participating in public policy networks, which is highly important for their stability, the development of shared values and beliefs, and finally the effectiveness and the outcomes of such networks (as discussed in more detail in Sect. 2.3), by supporting the exchange of expertise and knowledge between network participants.

Also, a first evaluation of the proposed method has been presented, which gave encouraging results, with respect to its ease of use and usefulness. However, further evaluation of this method is required, based on realistic pilot applications of it, in order to assess better its value and potential with respect to its abovementioned ambitious objectives: to what extent it enables and supports the transfer of information, knowledge and proposals from experts to the participants in the democratic processes of modern policy making, and under what conditions? to what extent it can enable and support the exchange of information, knowledge and proposals among the participants in public policy networks, and under what conditions? to what extent can this method can assist the EU institutions to collect high quality information, knowledge, opinions and proposals from their policy networks? Research in this directions is already in progress as part of the abovementioned EU-Community project. Also, further research is required concerning the use of ICT for the transfer of knowledge in the opposite direction, from the democratic processes towards experts/technocracy (e.g. concerning diverse needs, values and concerns of different stakeholder groups on the particular social problem/public policy the experts analyze, and also existing diverse perspectives, approaches and ideologies). This is quite important for the construction of better and more multi-dimensional comprehensive and inclusive expert analyses and plans, which do not miss or neglect important aspects of the social problems or public policies they are dealing with and might be quite important for large social groups, and also do not underestimate existing diverse perspectives, approaches and ideologies.