1 Introduction

E-participation is an interdisciplinary research area dealing with the electronic support of all public activities that enable citizens to participate in processes relevant for society [1]. This often comes with demands to introduce new participation facilities into the traditional processes of decision-making [2]. Tools for citizen decision-making must attract a variety of users with different preferences, literacies and demands for content and processes, especially when it comes to inclusion [3]. Some stress the importance of institutional context and careful design for e-participation performance [4].

Concepts of e-participation, reflecting either the way of participation (top-down or bottom-up), the legal foundation (formal or informal), the goals of a measure or the intensity of e-participation in tiers [5, 6]. In this paper, we draw on structured concepts of citizen involvement and decision-making, where each level of participation increases citizen power or the intensity of participation. In line with the idea of the meta-analytical study of Al-Dalou’ and Abu-Shanab [7] that the most comprehensive model is a five levels schema, a 5-level concept of e-participation levels was chosen in the context of this paper: (1) Information, (2) consultation, (3) cooperation, (4) co-decision, and (5) decision [5].

Previous research of the authors study group has utilized this classification for a model of tool assessment, specifically for investigating which e-ID is appropriate on which particular level of e-participation, with special regard to voting and rating mechanisms, participation threshold and security [5, 8]. This paper focuses on the acceptance of users and potential stakeholders when it comes to choosing appropriate means of identification (e-IDs) for authentication in different e-participation levels. The authors assess potential areas of application from a stakeholder perspective and the acceptance of tools from users’ points of views (micro-level or project perspective, see the evaluation framework below).

For that purpose, an e-participation platform demonstrator-software was developed in a project for assessing acceptance and demand of modern e-participation tools in the Austrian context. While we can draw conclusions from the project in the sense of users’ and stakeholders’ expectations, we are not accessing socio-technical perspectives in the sense of public take up or the greater democratic perspective (macro level in the sense of real-life application). In reference to the domain model of e-participation evaluation [9] and the four layers of this framework, our analysis focuses on participation areas and processes, actors as well as tools and technologies more than the democratic context. We also exclude the socio-economic conditions beyond stakeholder opinion or sustainability of e-participation processes [10] due to the scope of the project. However, we will glimpse at the macro level in the sense of its political situatedness and context. While the project is not tied to a specific planned initiative of citizen involvement, an assessment of acceptance lays the foundations for the implementation of e-participation projects in the future.

2 Project “E-Partizipation”

The Austrian government program of the 2017-elected federal conservative-right government declares in its preamble that citizen participation in political processes shall be increased through the expansion of direct democratic opportunities. Participation shall be improved at the parliamentary level and in the legislative procedures. In addition, direct democratic processes such as popular petitions and consultative referenda are promoted in the government program. However, a single platform comprising various e-participation opportunities and activities for a broader target group is missing.

In general, some efforts towards more citizen decision-making in Austria have not been without controversy in the past. The 2009 e-voting for representation in the Austrian students body [11] was met with criticism regarding its implementation [12] and led to a numerous appeals and a final repeal by the Austrian high court. Given this background, an assessment of the acceptance of e-participation solutions is indicated if Austria wants to foster citizen decision-making within e-participation, and affirm citizens’ trust in digital solutions.

The Austrian Research Promotion Agency (FFG) funded the project “E-Partizipation - Authentifizierung bei demokratischer Online-Beteiligung”Footnote 1 to research means of online participation with use of different authentication and identification methods. The 2 years project ended in Autumn 2016. Its goal was the development of a flexible and modular platform demonstrator-software that uses several means of authentication in the context of citizen e-participation, including the existing state e-ID solution in Austria. The platform enabled an adaptive set up of various e-participation processes. The consortium consisted of actors from public administration, academia and industry:

  • Austrian Institute of Technology as leader of the consortium,

  • Federal Ministry of Internal Affairs, as public agency responsible for election processes,

  • Danube University Krems, Centre for E-Governance,

  • University of Vienna, Legal Informatics,

  • Austrian State Printing House, and

  • Rubicon IT GmbH.

2.1 Applied Evaluation Framework and Research Questions

Scholars have offered a variety of evaluation methodologies for citizen participation, with different criteria for project specific factors [4, 13,14,15]. However, a too general evaluation framework would also miss the goal of accessing context and actor specific aspects [16]. Starting from the idea of a systematic analysis using defined criteria, [17] we followed an evaluation design recommended in the literature. Our starting point were the categories for e-participation evaluation as suggested by Macintosh and Whyte [18] with adaptations towards project goals and the national context. Building on the experiences from other evaluations, [19] we used the following analytical dimensions:

  • Political dimension: attractiveness of the selected cases for users and stakeholders; relevance of the platform (for real-life application); strengthening of e-participation (motivation).

  • Technical dimension: fulfillment of technical requirements; processes and functionality; security aspects.

  • Socio-technical dimension: acceptance of the selected solutions; e-ID variants; trust in the platform; sustainability.

  • Legal dimension: data security compliance; trust of users in data security; protection and privacy.

  • Methodological dimension: Practicability of the model for practitioners; suitability of the tool for the specific areas of application.

Based on the evaluation of the project within that framework, we formulate the following research question for this paper: How do users and stakeholders trust e-participation tools with view to platform specifics and authentication providers? How can these become more attractive to use?

In this paper, we mainly present the results of the user and stakeholder perspective based on the testing simulations with 33 test users as well as the 10 qualitative interviews with stakeholders in governance and public administration (6 interviewees) and politics (4 interviewees stratified by political parties).

2.2 Research and Development Design

The research design used in the project combined qualitative and quantitative methods for the development of the demonstrator platform consisting of a literature review, an internal assessment workshop (within the project consortium), expert interviews, a workshop and questionnaire within the scientific community. The project consortium provided continuous technical evaluation, internal testing and legal feedback throughout the development and scenario set-up period. The demonstrator of the platform was evaluated with a testing simulation, subsequent questionnaires and plenum discussions with user groups. In addition to user testing, stakeholder interviews were conducted. The following Fig. 1 shows the workflow of the entire project starting in autumn 2014.

Fig. 1.
figure 1

Research design and demonstrator development

Full size image

Testing scenarios were conducted by using the demonstrator-software, tested on the 13th and 15th of June, 2016. This was based on usability tests in a lab with 33 participants in two testing groups at university facilities. In addition, feedback was received during the testing sessions via standardized questionnaires and qualitative plenum discussions following the simulations.

2.3 Testing Scenarios with Users

User Test Groups.

In order to assure comparability of results, the two user test groups did not differ with regards to the testing scenarios presented. While the results of the questionnaires are not comparable to the representability required for surveys and follow a predominantly qualitative setup, we still tried to stratify along demographic characteristics, as it is common in user-testing or usability-testing, in order to simulate a realistic mix of potential users. Furthermore, studies have shown that several important aspects of our study (like privacy or data security) can be influenced by demographic features like gender or age. Privacy can be defined as one’s ability to control the release of personally identifiable data in the context of institutional practices, and the level of privacy protection can differ by gender [20]. Gender is a factor in understanding privacy and disclosure practices [21]. Research on social network sites also found associations between gender and disclosure in young adults [22]. Also age and education can relate to how users evaluate aspects of information privacy [23]. User test groups were selected as effectively as possible according to the demographic criteria age, gender and educational level. Further variables (f.i. self-assessment regarding tech-savviness) were included in the questionnaire.

Participants received as compensation € 15 gift vouchers. People were contacted on the basis of a call within the premises of the University of Vienna, leaflets, posters, Facebook groups, within the networks of consortium members and private networks of project workers. Citizenship was not mandatory for participation, but sufficient command of German was a necessary criterion. Registration for the two test events was done online and people had to provide age, education and gender for the composition of the test groups. The final composition of the testing groups was:

  • Age. 16–24: 16 persons, 25–44: 13 persons, 45–99: 5 persons.

  • Gender. Female: 15 persons, male: 18 persons, gender-diverse: 1 person.

  • Education. Compulsory school: 9 persons, Higher School certificate (Matura): 17 persons, university/college: 8 persons.

Additional assessment of user groups showed that participants used the internet on average for 2.8 hours per day professionally, and 3.4 hours per day privately. Regarding mobile usage participants used the internet on average for 2.8 hours per day. The majority of our participants used social media daily. Regarding political activity, about half of participants saw themselves as politically active, however, only a few were politically very active. A bare majority was not very or not politically active. Participants use the internet predominantly for information retrieval purposes (election program or politicians), less so for expressing their political opinion or for discussion. Around a fifth of our participants used the internet for polling or voting. It must be pointed out that we received 34 questionnaire responses despite 33 participants, and this error could not be definitely solved in the analysis thereafter.

Test Scenarios.

The testing was conducted in a PC lab, where two testing scenarios were simulated under the guidance of a moderator. While the platform was modular and adaptive, two testing scenarios had to be selected, that were realistic in the Austrian context. Due to project goals, each scenario needed a voting phase, hence the scenarios had to include the e-participation level of co-decision-making or decision-making. In the testing procedures specific aspects could be tested within the timeframe of 1 h and 15 min (10 min. introduction, 10 min. scenario 1, 25 min. scenario 2, 10 min. questionnaire, 20 min. feedback in a plenum discussion). The testing procedures were pretested with associates of the consortium that were not directly involved in the development of these procedures.

Scenario 1 simulates the election of a works council by employees in a company. Two electable lists provide information about their agendas and candidates can be selected (Table 1).

Table 1. Test scenario 1.
Full size table

Scenario 2 simulates urban planning in a future construction site in Vienna. A part of the new space is to become a public place, and citizens are invited to suggest ideas for a plaza design or similar. The submitted ideas are preselected by the hosts of the participatory process, turned into concepts of urban design, and then discussed and evaluated by the users. Finally, users can select one of final concepts to be presented to the jury that will decide on the development of the public space (Table 2).

Table 2. Test scenario 2.
Full size table

After the simulation, participants in both groups had to fill in a standardized questionnaire. Thereafter they were invited to a moderated qualitative feedback discussion on the basis of semi-structured guidelines. Moderators guided the 20 min long discussion via prompting questions, and the discussion was generally fluent and active among participants. The goal of the plenary discussion was to test the acceptance of e-participation solutions. Moderators observed the users during the discussion and took notes of nonverbal clues of approval or disapproval. The discussion was held structured along the political, socio-technical and legal dimensions and criteria.

2.4 Stakeholder Interviews

To access the stakeholder perspective, interview partners were selected by key-informant sampling [24] and comprised 6 representatives from governance and public administration, including one representative from the data protection authority, and 4 persons from parliamentary political parties. They were selected according to whether they could potentially employ such a platform, and asked whether and how they could imagine its real-life implementation. The data protection authority was asked specifically about aspects of data protection relevant for the implementation of such a platform.

Stakeholders from public administration were selected according to the levels of administration: city, regional and country level were all represented. They were contacted per email, invited to the interview per telephone, and interviewed mostly in their offices. While some interviewees in the stakeholder categories were responsible for citizen participation, others were responsible for elections (thus representing one specific level of e-participation). The interview partners were selected from the following institutions and departments: the Federal Ministry of the Interior (Department III/6), the Federal State Government of Lower Austria (Department for citizenship and elections), the Josefstadt district, the City of Krems (2 interviewees) and the Federal Ministry for Women, Families & Youth (BMFJ). The stakeholders from the domain politics were selected to represent all political parties of the National Council (parties selected their representative after invitation). All parties, except from one party, were available for an interview.

The stakeholder interviews were conducted between the end of April and mid-June 2016. On average, the interviews lasted 45 min each. The interviewer followed semi-structured interview guidelines [25] and a receptive interviewing strategy. [26] They were recorded, thematically coded [27] along the evaluation dimensions, and interpreted via a summary analysis, using paraphrasing methods. At the beginning, the stakeholders were shown the platform from both a user and admin perspective by showcasing the two scenarios with a live presentation. Two times the live demonstration did not work, so the presenter retreated to a PowerPoint-Presentation to show the demonstrator as it would look in the live scenario.

3 Results

According to user feedback, the tested platform was suitable for use in the defined areas of applications. Both users and political stakeholders found the location- and time-independent aspects of e-participation appealing as well the transparency of the processes.

3.1 Technology Application

Multi-ID Design Principle.

Regarding the socio-technical evaluation dimension, user feedback after the test run showed that the option to use various e-IDs was considered as especially user-friendly. Users expect secure, mobile and barrier-free technical solutions, and an adaptive design is advised to meet the diversity of user requirements.

Transparent Operations and Usage Statistics.

Criticism was related to the desire for more information about certain operations and processes on the platform, e.g. how comments and likes influence the ranking of results. While such questions were mostly explained by the moderator during user testing, the use of corresponding information tools (for example a video explaining the ranking and selection processes, or a F.A.Q.) is by all means recommended for such a platform. Users did not see relevant benefits in the transparent presentation of usage statistics of the e-participation phases. This led to some irritation especially in the voting phase, even though the actual voting results were only displayed after the voting phase was closed. It is thus advisable to proactively and clearly communicate information relating data processing, security and data protection (identification and authentication issues), especially when it comes to voting procedures in e-participation, such as the example of work council elections [28].

Perceived Risk of Manipulation.

Users generally rate the perceived risk of manipulation of results as high. A cooperation of public body with NGOs as controlling body seemed to be the most trustworthy solution for operators of e-participation platforms. Security concerns were not only related to the operators, but also to hacker attacks. Thus, transparent and pro-active communication regarding the prevention of manipulation can enhance trust.

3.2 Trust in the Proposed Solutions (e-IDs)

Clear Identification Methods for Sensitive Processes.

If an e-participation platform as the one presented was to be used by political parties for deliberation, they want to get an accurate picture of the participating population, hence clear identification methods are important. To the parties, the more sensitive the process is (e.g. in terms of financial impact or legal implications), the more desirable background information of participants becomes for analysis. In addition, some parties have already internally used electronic tools for voting, opinion forming or project development. While parties want to know background data of the users for analysis, anonymity on the platform remains very relevant to users, but opinions on the use of real names are indifferent.

Acceptance of e-IDs and Strengthening the Attractiveness of the Electronic Citizen Card.

The integration of the electronic citizen card into e-participation is likely to promote the dissemination or increased use of citizen e-IDs according to user feedback. Identification with an e-ID issued by the state provides a sense of security and adds to the trustworthiness of the platform. Yet, only a few people actually used their electronic citizen card (possibly because only a few users had one) in the simulation and the majority used their social security number for authentication. Nonetheless, no participant refused the demand for unique personal identification in the final stage of the testing simulation scenario 2. The use of unique e-IDs increases trust in the results. Social IDs are a first low-threshold entry into e-participation, although users critically evaluated the application of such IDs in e-participation platforms operated by the state and in terms of data protection. At the same time, the use of existing electronic identities seems preferable to the ability to create a new e-ID for yet another platform. Users mainly used their Facebook and Google IDs, some used LinkeIn and Windows Live, but other social IDs were not used. The users rated various options of authentication options on the platform positively. While it is thus recommended to offer a flexible design with different identification options on the different levels of e-participation, platform providers should be aware that in the context of e-participation, users predominantly hold a critical stance towards social IDs they know from other (non-state) contexts.

Reservations Towards Private Platform Providers.

Users expressed general low confidence in all platform operators, but even more negative concern towards private enterprises as platform hosts. The usage of personal data that would be collected and saved by platform providers was emphasized, despite the demonstrator-software of the project using only the required data in a secure way. In particular users were insecure regarding the options of the e-participation platform provider to access the collected data, also from a technical point of view, as users had to accept that the platform exchanges data with their e-ID. These confirmation messages often ask for more data than is actually used by the demonstrator-software.

Practicality of the e-IDs-Model for Practitioners.

Feedback from stakeholders has confirmed the basic principle of the multi-ID approach, in particular with regard to the use of high-quality e-IDs for high-level e-participation processes such as the level of collaboration or co-decision making. At lower levels of e-participation, diverse options of identification are perceived as positive. As the e-participation procedures have to be designed according to the needs of the target group, in relation to the respective topics and expected outcomes, generalization for the whole field of e-participation, with many variations and contexts, is always problematic.

Regulatory Issues.

The platform was tested before the implementation of the General Data Protection Regulation in the European Union, but users were already very well aware of needs for data protection and regulations. There was a clear differentiation between the e-ID provider and the host of the e-participation platform and both need to gain the users trust. The development of the platform was closely monitored by legal advisors within the consortium for legal compliance, and it was also crucial to follow the regulations of voting processes, such as the option to cast a blank vote.

4 Conclusion and Discussion

Research emphasizes that the utilization of ICT does not lead to more participation per se, and that certain measures of inclusion need to be undertaken if that should be the case [29]. However, e-participation can theoretically serve as legitimizing mechanism of democracy if it affords a way of empowerment for citizens [30]. When it comes to sensitive information and different views on privacy and new tools like social identification methods (social IDs), these often sensitive subjects might clash with otherwise well-meant intentions. This study showed that there might be a demand for official sites for participation purposes, as the user feedback in this project has also shown with regard to online authentication and data protection, that platforms hosted by states or NGOs still enjoy greater trust than private platform providers do. Transferring political discussions from semi-public spaces such as social media to official platforms could increase the chances to bring people out of the “echo chambers” of one-sided information, [31] and potentially foster a more direct and open discussion culture, in which several sides actively participate. Accompanying direct democratic processes with digital tools has the potential to reach decisions that are more satisfying. If citizens are provided platforms that meet their needs and enable them to control the ways of their participation, purposefully built digital platforms could support the transparency of the opinion making process, the accountability of the democratic participation and the overall acceptance of the outcomes [30].

The limitations of our methodology lie in the inability to evaluate the impact perspective, as our results are based on a demonstrator-software and not an e-participation project in the field. Thus, a replication of the method in an open phase would be desirable. Further limitations can be seen with view to the creation of trust in modern technological solutions: It is to be expected, that mobile devices will enjoy more trust from people in the future. Thus, future assessments of e-participation solutions should by all means include such dimensions. In our study, participants were not in agreement regarding mobile access: While some took such an option for granted, for others it was not high priority. But overall, users will surely expect mobile, secure and accessible solutions in the future.