Keywords

1 Introduction

We live in a connected and information-driven world. The Internet is widely used across the globe on diverse communication devices and in many different forms. Internet users are able to look for news about their favorite sports, the city they live and about other people. In addition, users may see information about weather forecast, traffic conditions, and the best hotel to stay in a trip. Besides all these tasks, ordinary users become more active on Web. Now, users play another role by generating content to the Web applications. Many applications, sites and social networks have as main information source, data from their users, who is not always conscious about that. [3, 18, 21].

The easy access to the user-generated content brings many benefits, but great challenges, as well. One of those refers to digital producers’ reputation, because the content produced by them become useful information to the final users of an application. As related by [17], reputation is commonly defined as the quantity of trust inspired by a specific community member in a specific environment. Community members with good reputation are more influent, because their contributions to the community. On the other hand, users with bad reputation are gradually excluded from the community.

The reputation of a person or a group of people is totally linked to the judgment background. For instance, a renowned physician can have a great reputation in relation to treatment methods of a disease; however, he could present a bad reputation in computer science community.

The concept of reputation is common in many areas of activity. When the subject is Science or academic area, e.g., reputation represents an important attribute to evaluate the quality of publications and career of scientists, researchers etc. To contract someone to work in a company, it is very common to look for the past actions and activities of that person. Therefore, reputation can be associated to the decision-making process about something or someone that depends on the actions (or content provided by) of someone in a period.

In Computer Science, reputation is a subject that is present in different areas of knowledge, e.g., Recommender Systems, Collaborative Systems, Human-Computer Interaction, Wireless Sensor Networks etc. Mui and co-authors [19] treats reputation in environments as eBayFootnote 1, where reputation is calculated accordingly to the positive or negative classifications about a seller, in a specific period of time (weeks, months, years, etc.). For online auction systems, for example, the reputation of a seller can be considered as a guarantee of product receiving.

Nowadays, there are many applications to solve real-life problems, which have as main characteristic the participation of a group of people congregated to reach a specific goal, namely, crowdsourcing-based applications. From data posted or automatic generated by other users, it is possible to estimate the delay of a bus, to know preferences of people, to help in scientific researches, etc.

However, we must face up to the new challenges arising in this context. For instance, how to attribute/calculate/classify something as reliable? The challenge becomes bigger when there is huge volume of information-generated day-by-day and from a large number of different digital producers. In many cases, the context of use of an app demands a fast response time to make a decision, so, the question around this is: which trust resources are offered to final users to support conscious decisions to reach their objectives?

In crowdsourcing-based apps, a lot of information is provided in a short period of time and in many domains [14]. Crowdsourcing represents an open call to the crowd to solve a problem or to perform some tasks in exchange for payment, social recognition or entertainment. Wexler [25] defines crowdsourcing as a concept related to a group of people who do not necessarily know each other personally and have a weak relationship link, involved in favor of an activity.

Other authors [12] consider an integrated definition of crowdsourcing, based on many works and researches about this topic. So, crowdsourcing refers to a kind of online participatory activity, in which a person, organization or company with sufficient means, proposes to a heterogeneous group, of varied knowledge and experiences, through a flexible open call, the voluntary commitment of a task. With this, the commitment of these tasks always implies mutual benefits.

This paper presents the results of a study to explore and characterize the strategies (adopted by this kind of applications) to communicate reputation mechanisms and how these mechanisms are perceived by users, investigating, thus, their communicability, with the lenses of Semiotic Engineering theory [8]. We present and discuss the results of a qualitative study [5], using a mix of methods: The Semiotic Inspection Method [10] and Observation Method. Our main findings are a common set of strategies to communicate reputation mechanisms and their respective fragilities.

This paper has been divided as follows. In Sect. 2 we present our theoretical foundation. In Sect. 3, we present an overview of some related work in HCI. The Sect. 4 presents more details about our methodology and the Sect. 5 presents the studies and the results founded by them. Finally, we discuss and conclude our results in Sect. 6.

2 Theoretical Foundation

2.1 Semiotic Engineering

In this study, we used Semiotic Engineering [7] as our theoretical framework. Semiotic Engineering is a semiotic theory of HCI, in which the interactive systems and the human-computer interaction are seen as a special case of metacommunication from designers to users. To this theory, the interface represents the main point of communication between designers and users, acting as designer’s deputy, since the designer is not present, physically, next to the user at interaction time [23]. The computer-mediated communication between designers and users communicates the designers’ understanding of who the users are, what they know the users want or need to do, in which preferred ways, and why.

Semiotic is a discipline dedicated to the study of signs and how they are used in communicative processes [8]. By the studies of Charles Sanders Peirce [20], sign is anything that represents something to someone. Thus, it can be an object, symbol, word, draw, icon etc. that represents and transmits some information to someone. As reported by [10], some of the most frequent signs in computer systems are: images, words, colors, dialog structures, layouts, among others. Each sign used in the application’s interface may have different meanings to different users.

In the perspective of Semiotic Engineering, the main quality factor is the communicability, i.e., the property of a system to effectively and efficiently transmit the intentions and principles that guided its design [23]. When the user is not able to comprehend the message intended by the designer, then communication breakdowns may take place at interaction.

2.2 Semiotic Inspection Method

In this research, we used the Semiotic Inspection Method (SIM), a Semiotic Engineering tool for HCI evaluation. With this method, the evaluators can analyze the communicability of the interactive artifacts [10]. The focus is to inspect the designer-to-user metacommunication aiming at identifying the potential communication breakdowns. In the evaluation process, the evaluator examines the interface and classify the signs as metalinguistic, static or dynamic.

Metalinguistic signs are the first one to be analyzed, since they explicitly express and explain other parts of the designer’s metacommunication. This class of signs is usually found throughout the interface in instructions, explanations, warnings and error messages, focusing on online help, user manuals and system propagation materials [10].

Static signs are those that communicate their meaning regardless of cause and effect relationships and can be interpreted from snapshot portraits of the screen. Thus, they express the state of the system at a given moment. They are represented by the elements present in the interface screens (or equivalents in non-visual interfaces), such as labels, images, text boxes, buttons, menus, etc., as well as layout, size, color, font and other characteristics. Its analysis should consider only the interface elements presented in each screen at an instant of time, without examining the behavior of the system, nor the temporal and causal relations between interface elements [10].

As for the dynamic signs, [10] show that in the analysis, the evaluator must inspect the interaction process that the user can experience through the interface. These signs are perceived through changes in the interface that communicate to the user the behavior of the system as a result of user actions (clicking the mouse, pressing enter, changing the focus from a form field to another, etc.), by external events (receiving an email, Internet connection fails, etc.) or over the time. Dynamic signs are usually represented by animations, opening and closing dialogs, transitions between screens or modifications to the elements of a screen (for example, enabling a button, updating a text or image, modifying the layout of some interface elements, etc.).

In order to inspect the interface, the SIM proposes 5 steps to be followed by the evaluator [23]. In the first three steps, the main goal is to rebuild the designer’s metacommunication, by using the following template of the designer’s metacommunication [23] for each category of signs (metalinguistic, static and dynamic): “Here is my understanding, of who you are, what I’ve learned you want or need to do, in which preferred ways, and why. This is the system that I have therefore designed for you, and this is the way you can or should use it in order to fulfill a range of purposes that fall within this version”.

Step 1: Metalinguistic signs inspection. At this stage, the evaluator explores the documentation and help system

Step 2: Static signs inspection. At this stage, the evaluator inspects the static signs of the interface

Step 3: Dynamic signs Inspection. At this stage, the evaluator inspects the signs that emerge from the interaction

Step 4: In this stage, the evaluator contrasts and compares the metacommunication messages from steps 1, 2 and 3. In this step, the evaluator must register the inconsistencies and potential problematic interpretations that may take place at interaction time by the users

Step 5: Appreciating the quality of the metacommunication, in this step, the evaluator produces a report containing the communicability problems found that might frustrate or prevent the user to understand the intended message by the designer, affecting their productivity

In this method, the evaluator is the user’s advocate. Thus, it takes the role of the user and represents it interacting with the system in order to represent it.

The SIM can be applied for technical or scientific purposes. When used for research purposes, a step should be added to its execution: the triangulation of the results [10].

3 Related Works

Tausczik and Pennebaker [24] consider that there are two perspectives on the role of reputation in online collaborative projects. The user’s reputation should be minimized to promote engagement by increasing the number of contributions and the reputation of a digital producer can be a good feature for the quality of his contributions and it may become easier to find high quality content. The study examined how users’ offline and online reputation affect perceived quality in an online community called MathOverFlowFootnote 2 where members post high-level Math questions. For the research, the authors used the following method: information about registered users combined with registered behavior collected in the community. The users could vote about the quality of a question or an answer posted based on interesting or its innovation. For each author of a question or answer, different reputation measures were collected such as offline reputation and number of points in the online community. The study shows that past activities also represents a good measure of reputation which can be correlated with the quality of a post. In our work, the correlation between offline user’s reputation and quality of content was not identified.

De Paoli and co-authors [6] present a design experience to create badges and named levels as mechanisms for representing the reputation in an online community. The badge is a symbol, a representative object of an abstract idea, for example: seals that represent the level of knowledge of a collaborator. Named levels are representations in text format that indicate the level of a participant. In many applications, when the users perform certain tasks, they reach higher levels. The badges and the named levels have been used as reputation mechanisms on various social networks, crowdsourcing-based applications to promote the users’ engagement. Another point reported in the study is that, many users bring experiences about the use of such features from other platforms and sites, being necessary to consider this question on design time. The badges were identified in our study, representing a strategy to communicate reputation.

Bente and co-authors [2], in turn, address the issue of reputation, reliability and use of avatars in e-commerce to reduce uncertainty in online buying and selling relationships. However, the study focuses whether these found mechanisms designed for Western culture also apply to other cultures. For this, the authors compared the buying decisions of Arab and German participants in an experimental game of confidence. The study points out that reputation systems based on profile scoring can influence trust by attributing trustworthiness to the sellers, regardless of culture. Our work does not cover questions related to cultural factors, but we also identified photos and avatars as strategies to communicate reputation.

Woodruff [26] presents a qualitative study of how users manage their online reputation. The study is about users who bother with their online information and how they do it to recover from defamation and damage to their reputation. In this case, it involves the concept of offline reputation, that means how the information in the virtual world affects people’s lives and how to recover from that situation. In the most of results, the users were not able to repair well the problems that affected their reputation. This last contribution indicates that this topic deserves special attention from HCI area, motivating researches to create technological, social and legal mechanisms to prevent such damage.

Coetzee and colleagues [4] studied how design affects the participation and engagement of students in an online course. For this, they created a reputation system where students who created more useful posts earned a score. The study demonstrates how reputation systems lead to faster response times, increasing the number of comments per post and bring differences in the way of students ask questions, even though they don’t have an immediate effect on school performance. The score mechanism is also studied in our paper, being one of the strategies to communicate reputation.

Pinheiro and co-authors [22] address the problem of the huge volume of data generated in social networks, the sharing of unreliable information and the availability of content generated to confuse or deceive users. This study presents a solution to add auditing capacity to social networks, based on a catalog that organizes characteristics, besides suggesting a guide for the engineering of software with the reliability feature. The auditing capacity is a necessary issue to increase the quality of the information available in an environment. Therefore, the reputation systems’ communicability is a very important issue.

Luca and Zervas [18] discuss the problem of fraud by companies, creating false comments for themselves or their competitors, identifying some aspects in common and the motivations behind these actions by analyzing the economic or financial view. For this, the authors analyzed reviews of restaurants that were identified by YelpFootnote 3’s algorithm as suspect or fake. For the accomplishment of the studies the authors made empirical studies creating mathematical models for validation of the results. In the study, the authors identified that 16% of restaurants reviews on Yelp are filtered. With this, tend to be extreme (favorable or unfavorable) and the prevalence of suspicious comments has grown over time. A restaurant is more likely to commit fraud when its reputation is weak, when it receives comments or negative reviews. In which case, such fraud could happen by creating fake profiles to include positive comments about the restaurant or to defame competition. In addition, restaurant chains are less likely to commit fraud, according to the study. Our empirical studies identified strategies to communicate bad reputation and to confirm an information, thus, preventing fraud questions.

Josang and co-authors [17] provide an overview of existing reputation systems and proposals that can be used to generate reliability and reputation measures for Internet transactions. The authors define two types of reputation systems: centralized and distributed. In the centralized system, information about the performance of a particular user is collected as evaluations of other community members who have had direct experience with that participant. So, the central authority collects all the evaluations about a user and derives a reputation score for each participant and makes that score available to the community. In the distributed reputation system, there is no central place for submitting reviews or for scoring other users. This may be done indirectly, when the users receive information from other users and evaluate them individually about the digital producer’s reputation. The work presents mechanisms of reputation used by some sites too.

4 Methodology

The methodology used in this work follows a non-predictive paradigm and makes use of interpretative and qualitative methods [5, 11]. In order to find out the strategies to communicate reputation mechanisms and how users perceive them, i.e., the communicability, we conducted two empirical studies, which contributed to a broader study (see Fig. 1). We selected the mobile applications WazeFootnote 4 and TripAdvisorFootnote 5. Waze addresses traffic-related issues, displaying alerts and other traffic features, reported by its users and TripAdvisor aims to help with travel planning through comments and user reviews on some place, such as hotel, a restaurant etc. Any other application based on crowdsourcing could be used in this study, but it was preferred to adopt smartphone applications, with a context of use where decisions about the information visualized should be taken in a faster way. The versions used in the studies was Waze, 4.7.0.1 version, AndroidFootnote 6 platform and for TripAdvisor application, the 17.8 version and Android.

Fig. 1.
figure 1

Methodology

Full size image

First, we conduct a study (S1), in Waze app to understand how it communicates reputation mechanisms. In the first step (Step 1), we ran the semiotic inspection, where there is no users’ participation. We focused on understanding the emission of such mechanisms. In the second step (Step 2), we ran observations tests with users to identify how the message was understood by them. We focused on understanding the reception of such mechanisms by the users. After these two steps, we arrived at the results founded by contrasting the two parts (Results S1) [1].

Then, we conducted a second Study (S2), using the same steps of S1, but in a different domain. We arrive at results in this second Study (Results S2), by contrasting the emission and reception of reputation mechanisms, following the same methodology.

In the Step 3, we compare both studies and identify common and divergent meaning categories, searching the communicative strategies adopted by them (Comparison between S1 and S2 – Step 3 (S3)). We, thus, drew our conclusions based on the set of categorized meanings that guided the interpretation of findings and help us answer the primary research question. The Fig. 1 illustrate the steps used on this research.

Finally, in order to validate our study, scientifically, we ran a triangulation step. To this, we perform an exogenous triangulation (S4) using a different application, in a different domain to achieve this objective. Following, we present more details about the steps of the studies1 and 2.

The studies were divided in two main steps. The same methodology was adopted in both studies. In Step 1, an HCI evaluator inspected the selected application using the semiotic inspection method. An inspection scenario guided the tasks.

In Step 2, participants were recruited to evaluate the inspection scenarios. Participants received an invitation by e-mail, as well as a pre-test questionnaire in order to know a little more about their habits and knowledge about their use of technologies. This empirical study was conducted with the following phases:

  • Presentation of the study: the researcher presented to the participants the basic concepts needed to carry out the study. In addition, it was explained how the study would be conducted and the application used in the test.

  • Evaluation activity: the researcher started the evaluation activity based on a scenario of use. In the S1, the evaluator accompanied the participants, in a real route, inside a vehicle, using the app Waze. In the S2, the user observation test was conducted in a lab.

  • Post-Test Interview: the researcher conducted this step to investigate how participants perceived reputation issues and to verify if the reputation mechanisms used by the apps were recognized and signified by them.

The main empirical data collected in this stage came from the participant’s discourse through the evaluation activity and the post-test interview. The audio transcript of these empirical data was carried out in order to investigate using the discourse analysis technique.

5 Results

In this section, we present the main strategies founded involving the communication (emission and reception) of reputation mechanisms.

5.1 Strategies from Step 1 (Inspection Studies)

Shortly, we can summarize the main strategies to help users know more about digital content providers and make decisions well informed: (i) offer ways of connecting (following) known people; (ii) show the users’ (digital producers) contributions and/or engagement; (iii) categorize users and show their contributions’ levels; (iv) provide alerts about divergent or incongruent information to prevent frauds; (v) provide ways to end users report/collaborate about the information quality.

About strategy (i), we identify that both apps use some communicative strategies to allow the users to connect with known people. When the people know each other, the meaning of reputation offline is clearer. Thus, if some end users know the digital producer, it is easier to identify his reputation. Both applications have ways of connecting known people. Waze uses mainly phone book contacts, allowing establishing estimated arrival times in the case of meetings with these contacts via the map. It also offers the possibility to login through a FacebookFootnote 7 account. The TripAdvisor, in turn, when connected to Facebook, display ratings and reviews from friends, firstly. This feature enables users to have a criterion for selecting if the information posted is reliable or not, accordingly to the offline reputation of a friend. In the case of Waze, the benefits of this functionality could be perceivable if the application, for example, identify an alert made by a friend on the map. However, this task is not well communicated by the app, because there is no notification about alert issues by known users.

The two applications also present communicative strategies (ii) to show the users’ contributions (digital producers), for example. Waze uses different colors to identify the traffic situation, as well as having avatar resources to promote the engagement of its users. TripAdvisor brings together in the users’ profile the cities already visited and evaluated by them and a seal, that represents the knowledge and participation of a profile. Waze makes use of avatars to promote users’ engagement and uses them to demonstrate user’s moods and levels of influence and engagement in the application. When interacting with the app, completing some tasks, the digital producers earn points, becoming more influential on posting information. Waze also present specific avatars to communicate to end users who edited maps or corrected some failures. Therefore, there is specific signs to communicate reputation in this environment, showing the users who collaborate more frequently, who are more influent in this context. However, this feature is not well communicated, because only through the analysis of the metalinguistic signs (help system) it was possible to understand it. Therefore, many users may consider this feature only as a way of personalizing the avatar, without perceiving the influence communicated by this mechanism. TripAdvisor does not present the avatar resource, but it uses seals – a kind of badges. Seals communicate to end users who are the digital producers whom are more collaborative, highlighting those who have made more evaluations about a specific topic, such as restaurants. Thus, when trying to establish a comparison, such profiles would take advantage of the comments, because they are considered as specialists. However, we found communicability breakdowns in the observation tests. Some users do not understand the meaning of the seals and could not associate it to the reputation of a digital producer.

About the strategy identified to communicate user’s reputation (iii), we found out the profile scoring. In Waze, the scoring is used to rate users who participate more and to promote engagement. Strangely, Waze, for instance, increases user’s score when driving with the application open, so, taxi drivers’ who transit all day long, will probably have a good score in the app, but it is not necessarily correct to assume that they are digital producers’ with a high reputation. So, this resource presents breakdowns in communicating reputation, too. On TripAdvisor, users who include ratings’ content, earn points. Thus, the score is a way to evaluate the reputation of a profile. In the observation test studies, users had doubt about how the calculation of these points were made, which was also not clear in the inspection step. However, this communicative strategy is better communicated in this environment than in Waze.

The apps present communicative strategies (iv) to prevent frauds. Waze presents the “No Exists” option where a user reports a false warning that was posted. It also presents as a strategy to limit the number of alerts per day and per hour of each user. An account may be blocked, being stopped to reporting more warnings or lose points in its score, regressing in the evolution of his avatar. Therefore, these features help prevent malicious users from acting, but it is not reported in a user’s profile that it has been sanctioned, thus, an end user can not verify if a user has bad historical reputation. On TripAdvisor, there is a place to report false reviews. In this case, the moderators will check the denunciation and if the fraud is confirmed, they assign a bad environment/profile seal to establishment/user and it loses points in the ranking. Moreover, because of the analysis of metalinguistic signs, it is clear the prohibition of employees not being able to make evaluations on their own establishment or concurrent. There is also the option of limiting evaluations about a same place/establishment by user for only once. Preventing hotel owners, for example, from commenting positively over time about their establishment. Another feature founded is to limit the evaluations to recent experiences. Therefore, TripAdvisor has as sign to communicate bad reputation.

To avoid situations like the one previously reported, applications do not allow posting made on behalf of others. However, this ends up constituting only as a site policy, not being presented to users, explicitly, at interaction time. Another feature presented in this sense, is about location. By Waze policy, it is only possible to inform alerts near the user is, however, during the inspection it was verified that alerts can be created anywhere on the map.

Another way to prevent fraud is the presence of moderators, one of the ways to control the information provided in the application, increasing the reliability of what is posted. In the Waze is not clear the presence of moderators, although, in the observation of user’s test, the participants believe that all the things posted in the app were its responsibility. On TripAdvisor, the presence of moderators is perceived, however, its presence is more common when denounces are made or about the control of frequency of evaluations policy. The moderators only check to see it the evaluation does not contain offensive content, verifying reputation question only on demand (when a user reports that something is wrong).

Another communicative strategy (v) adopted about reputation is the confirmation of information. The number of “thanks” in Waze keeps the information (alert) longer in the application. However, this feature can be easily confused with the thank function only, without verifying whether it is true, as being difficult to filter useful comments in a context of use of application (in traffic). TripAdvisor presents a sign below the ratings which users have given their opinion informing if the evaluation was useful. However, the problem is that the sign used is very similar to the “Like” sign of Facebook, and can be interpreted only as a way to thank for the information or to enjoy writing.

Those apps also offers mechanisms to digital producers enhance their contributions, such as comments and photos. In Waze, for example, analyzing each comment during the time in which while drives, it becomes a non-viable task. On TripAdvisor, in turn, comments are the main task and objective of the evaluations made.

Photos are also considered as a mechanism. TripAdvisor profiles that showed more photos, whether from places visited or in their own profile were considered in the observation test of users as more reliable, presenting better reputation characteristics than one they do not have. In Waze, alerts that display photos associated with them were considered more reliable too, by end users.

With this two studies, we conclude that the user profile represents the main communicative strategy in terms of reputation. In Waze, the profile displays information such as avatar, name, score, overall rating and the time the user began to use the app. On TripAdvisor, the profile presents information such as score, number of seals and display of the kind of seal, photos, visited and evaluated cities, place of origin of user and the year in which they started to be part of the app.

The recurrent mechanisms used in the applications studied were: integration with social networks or e-mails accounts, use of avatars or user profiles, score and classification features associated with a user profile, use of badges to denote the participation and knowledge of a given profile, number of evaluations or comments made, date that the user became part of application, profile photos and location (can be used as a confirmation of the information) and strategies to prevent fraud. In the case of Waze, in particular, there is still the possibility of emitting alerts by anonymous users, which is a problem in evaluating a user’s reputation.

5.2 Strategies from Step 2 (Users’ Observation Studies)

In this step, we identify potential breakdowns in designer-to-user communication, in relation to our focus. It is worth to mention that the five communicative strategies listed in Sect. 5.1 were not considered efficient by end-users, during our empirical studies. In both apps, participants do not know and do not worry about (self) reputation and they were not able to recognize the reputation mechanisms described in this section. In Waze, participants do not understand the avatars or any other ranking/score mechanisms to classify user-generated content. Most of them, believe that “the system” is trustable and the information is provided by “the system”. Additionally, because the use of Waze is mainly on the road, participants can check the information before make decisions.

With Trip Advisor, in turn, participants recognize the ranking/score mechanisms as important strategies to communicate reputation and they are aware of resources to contribute in reputation, such as comments.

We identified specific signs that potentiate the occurrence of communicability breakdowns. The use of some signs similar to other applications, such as the “Like” sign on Facebook. In Waze, it represents the number of thanks and in TripAdvisor it means the review is “helpful”. Users who have been using Facebook for a long time may not understand the meaning of such signs and contribute differently to the way expected, since this sign on the social networks has nothing about reputation issues. For example, the user behaving in front of the sign presented by Waze, “enjoying” what was posted, may end up keeping the information visible for longer in the application. In case the information is a fake alert, this may end up compromising the decisions of users that will avoid selecting the route with this warning. The same can be considered with the “useful” sign, if any user, activate this option because he liked writing only, without verifying that was posted is true, will contribute to the user profile, giving score to him unconsciously. Other users can take account of this information as if it had been validated and choose that hotel based on that mechanism, at risk of having a poor hosting experience. Therefore, the choice of the signs to be used is a point that must be considered when designing crowdsourcing-based applications and its context of use.

The number of comments is also used as a way to validate information on TripAdvisor. Many users reported rejecting places with few reviews and/or comments. However, the minimum number is a subjective concept, as observed in user observation tests.

The use of avatars to demonstrate the users who collaborate more and do the editing of maps also the feelings of the users in the traffic, in the app Waze, also can generate breakdowns in communication. For a beginner user, the feature can be seen just as a profile customization, since it is not communicating the most influent users on the map and contributions. During the observation test with users, they did not recognize this strategy.

As reported during the study, an alert made is evident on the map until other users confirm via thank option. If the alert is not confirmed, it will no longer be displayed in the app. However, it is no clear how long it will remain, the absence of a sign that explain this verification can also cause problems. Given the context of use, in relation to traffic issues, a user could stop going on a route on account of that warning, suddenly choosing a more dangerous route. Therefore, it might be important to exist some element communicating that the warning has not been validated. There is the option of thanks sign, but as we have already, their meaning is implied in some occasions.

The score mechanism, in some cases, such as that of TripAdvisor, is not clear how points are calculated. Thus, many participants were in doubt whether they could actually use this criterion. It is clear that a user, who has more points, participates more and can be more influential, but, it is also necessary to communicate the way to earn those points, to make a more conscious decision.

Seals and badges are widely used as mechanisms to communicate the reputation of users. However, some seals and their quantity can create doubts for users. In our study, participants were beginners to using crowdsourcing-based applications, and some of them do not understand the meaning of the seals. By the inspection of the signs, the function of the signs becomes clear, so, for a better communicability, there should be a greater harmony between the signs used.

The lack of prior knowledge of end users about the subject of our research, has also proved to be a potential cause of breakdowns in communication and also the need for more dynamic metalinguistic signs, since users don’t understand the meaning of representations used in the interfaces. In the observed cases, participants give to the applications the responsibility for the information they provided. Therefore, they believe that the alerts and evaluations have already been validated by some moderator. As we know, crowdsourcing-based applications do not necessarily have effective moderation, so, to know more information about who is posting information is a very important step.

5.3 Triangulation

In order to have different perspectives of the same object, explored in different contexts, giving plausibility and consistency to the interpretation process, we performed the triangulation stage, a qualitative research procedure [5, 11].

To do this, we perform an exogenous triangulation to confirm the communicative strategies about reputation in a website, that we found out with the two studies. Thus, triangulation was conducted in a new application, in a different domain and in a desktop environment, completely different from the mobile version. We chose the website version of the Project NoahFootnote 8 to search for divergent and convergent categories of the previous studies.

The Project Noah, according to its website, “is a software platform designed to help people to reconnect with the natural world. To do so, it harnesses the power and popularity of new mobile technologies to collect important ecological data and preserve global biodiversity. Through the help of crowdsourcing, organizations around the world are documented, ranked, generating up-to-date information about local and global biodiversity.”

In order to keep the same focus on the context situation, when searching for convergences and divergences during the inspection step in this environment, a context of use was created to guide us.

Just as the strategy (i) to offer ways of connecting (following) known people, the Project Noah presents in the user profile, a website associated with the profile, or an e-mail. However, to create an account, it is necessary to have an account with Google+Footnote 9, Facebook, TwitterFootnote 10, etc. This feature tries to ensure that participants are real, not being fake profiles. In case of this app, a clear strategy for connecting with known people was not identified. There are several types of account, but is unclear if this can be used to select the contacts while browsing the website. However, it is confirmed the need to integrate with another social network, or e-mail, preventing fake profiles to be created.

As in strategy (iii) to categorize users and show their contributions’ levels, we have identified the use of user profiles containing information about users and their activities. The profiles also presents a photo, number of contributions, images, location of origin, site or associated e-mail, number of comments made, number of suggestions made, number of favorites, number of followers and contribution seals (called patches by the website). Thus, we see that the strategies used to communicate the reputation and collaboration of the users identified in early studies are also used in this environment, as well the use of user profiles and its information. Users who have a high index of collaboration receive a ranger seal (expert) and influences the community, as seen in other studies. This is possible through the amount of collaboration seen by the moderators or by indication of other users via e-mail. We did not identify in this study the use of score to communicate the reputation in this website.

The photos are strongly used in the studies, being one of the main features of this website and is directly linked to the participation and collaboration activities. In this case, the photos can be used to help users and moderators make decisions about what is posted, as well as the activities of a digital producer.

The website also presents strategies (iv) to provide alerts about divergent or incongruent information to prevent frauds. User may denunciate by sending messages to more expert users and/or by using a flag sign (this option is only visible to users who have an account on this website). It was not clear what are the consequences of the report of a problem. There is no sign to communicate bad reputation in this environment. However, it confirms the strategies of having a sign to communicate something that is incorrect and also confirms the fact that it is not possible to put information on behalf another person.

As strategies (v) to provide ways to end users report/collaborate/confirm the information quality, the website has a voting scheme. At least 3 votes are required to confirm a post. This information was found in the help of the website, but in practice it did not always appear available. The sign used is similar to Google+, representing the voting validation for what was posted. However, it can also be confused with a thanks icon. Another form is through comments and suggestions placed by other users, which are sorted by date, the most recent being shown first. The amount of comments and suggestions that a profile makes may represent his ability to collaborate on the site, but the same cannot be fully stated as far as reputation is concerned. However, the number of followers is a communicative strategy for the reputation of digital producers. Underneath each post, some signs are shown, communicating that information has already been verified or not. With each post, the location of the post can be added and it can be from anywhere in the world. However, for a particular option, the missions, only may posted near the place it happened.

Finally, it is clear the presence of moderators, being the website organizers or specialist users, that verify all the information posted, in a collaborative way, in order to guarantee the integrity of information.

In conclusion, we can say that the result of this triangulation contributes to the consistency of this research and confirms the scientific validity of the results found in these exploratory studies.

6 Conclusions

In this research, it was possible to identify: (a) five strategies adopted to communicate the reputation of digital producers in examples of applications based on crowdsourcing; and, (b) potential breakdowns in designer-to-user communication, in relation to our focus. This research also indicates that users (end users and digital producers) should be aware of the meaning reputation in order to promote better user use of the applications.

About the communicability, we noticed a huge difference in terms of the application domain. For example, in traffic, reporting of reputational aspects is very fast, being related to making instant decisions, such as selecting a route based on the user who posted the information. In the case of TripAdvisor, it is expected that the information will be available for a long time, in order to give more conditions to decide on the choice of a hotel, for example.

In conclusion, the results reveal the potential risks involved in the use of the application (by users), according to the communicate strategy adopted by the designers, indicating that the design process of such applications should be rethinking for such possibilities of breakdowns and alternative solutions.

The identification of communicative strategies on the reputation of the digital producers and the mapping of resulting problems from the adopted strategies begin to fill a gap in the study of reputation with a focus on HCI.

Additionally, such strategies can serve as the foundation for the study on how to improve the HCI design process for such applications. This work does not provide immediate solutions, but presents challenges and opportunities to improve the design of crowdsourcing-based systems. Thus, some identified strategies may serve as a guide, or may even be avoided. In addition, to emphasizing the importance of reflecting on the context of use in which users will interact with the application created.

In this research, we were faced with the challenge of doing research with innovative systems and profiles of beginner users. For this, we needed a theoretical foundation that would allow the exploration of an innovative environment such as crowdsourcing-based applications and also that would lead to identify possible new problems. For this reason, we adopted Semiotic Engineering and communicability as a criterion of quality, since it allowed us to study the processes of signification and communication, without leaving us limited to heuristics or guidelines, based on concepts prior to the technology studied.

The combination of the SIM with scientific purpose and the observation of the users with a focus on communicability has revealed itself with a good methodology and contributes to the identification of the new problems relation to the reputation and needs of users that we had not even imagined. Therefore, this methodology favors the extension of the knowledge of the problem by the evaluator.

The findings of this research point out to interesting opportunities for future work. In our study, the participants ‘profile of our empirical studies was the digital consumer. The same study could be conducted with expert users in these two apps or digital producers.