Keywords

1 Introduction

The majority of ethnographic studies are related to direct observation. However, interviews, questionnaires, and studying artifacts used in activities also feature in ethnographic studies [1]. The basic tenets of ethnography are the recursive and inductive depth observation of a culture or a community as well as open-ended interviews designed to understand the perspectives of community’s participant [2]. In order to help shape researchers’ participant depth observation, some ethnographic procedures are used, such as making cultural entrée, gathering, and analyzing data, ensuring reliable interpretation, conducting ethical research, and providing an opportunity for member feedback. Furthermore, these procedures are completely known in ethnographies conducted in face-to-face situations [3].

During the last two decades, online environments became rich and vital grounds for ethnographic studies [2]. In the same period, online communities have become one of the most popular forms of online services [4]. Online communities are essentially forums for meeting and communicating with others [5], or in a more detailed definition, online communities are web-based online services with features that make the communication among members possible [4]. Along with online environments, the growth of online communities brought by the Computer-Mediated Communications (CMC) created a solid research field for online ethnography studies [6].

Online ethnography adopts principles of ethnographic research molded in offline environments and applies them to online environments with necessary adjustments [2]. According to Kozinets [7], online ethnography is a generic term for performing any ethnographic research by using some sort of digital or online environment. Methods such as netnography [8] and virtual ethnography [9] are widely adopted for qualitative research. However, it is not clear how Computer Science domain is using online ethnography for empirical studies. Thus, the main goal of this study is to present how online ethnographic studies have been performed in Computer Science.

To achieve this goal, we conducted a systematic mapping study regarding this methodological approach on Computer Science. In the mapping herein presented, we focused on empirical studies using any online ethnography method that were carried out on Computer Science discipline. In order to contextualize our findings, we present the background about online ethnography methods in the next section. Afterwards, we delineate the Research Method, including the research protocol. Next, in the Results section, we present the report and our results’ analysis in order to answer our questions. Finally, we state our discussions and conclusion.

2 Background

Ethnography is a qualitative orientation to research, which emphasizes the detailed observation of people in natural environments. Ethnography seeks to present a picture of life seen and understood by those living and working within the domain in question, through direct involvement of the researcher in the environment under investigation [10]. The emergence of social media on the Internet provides qualitative researchers with a new window into people’s outer and inner worlds, their experiences and their understanding of these [11]. The Internet has created many types of online communities that not only exist in cyberspace, but can also be studied through the internet itself. As collaboration and social activities became online, ethnographers adjusted their strategy to take into account computer-mediated communication [12]. This movement had several names, the most common being: Online ethnography [1], virtual ethnography [9], or netnography [13].

Developed by Robert Kozinets, netnography is a qualitative research methodology which adapts ethnography research processes to study cultures and communities that are emerging through CMC [3]. According to Kozinets [7], online ethnography is a generic term for performing any ethnographic research by using some sort of digital or online environment. As stated by Bengry-Howell et al. [14], the case study of netnography sits within a broader methodological context of online ethnography. Online ethnography encompasses approaches for conducting ethnographic studies of online communities. Normally, online ethnography includes observation of postings and threads within an online forum and interviews with an online community. However, it can implicate in data collection online as well as offline [9].

Another online ethnographic method is virtual ethnography [9]. Virtual ethnography is a form of ethnography for studying online communities based on textual data [15]. However, it appears to allow for a composition of online and offline ethnographic approaches to have an understanding of the online phenomena [16]. Meanwhile, netnography addresses online interactions and differ from other online ethnography methods by offering a more systematic, defined approach to addressing ethical, procedural and methodological issues specific to online research [17]. Nonetheless, both methods have been applied on computer science discipline.

One example of virtual ethnography application is Margaret and Walt’s study [18]. In this research, the authors conducted an extensive virtual ethnography collecting data over a period of four years. Their goals were to deeper understand the ideology and work practices of free and open source software development, which is valuable to software developers and managers who wish to incorporate open source software into their companies. As an example of netnography, Di Guardo and Castriotta [19], applied an exploratory qualitative case study using the netnography method in order to analyze the open innovation experience and crowdsourcing of a large Italian company. Their results imply the effective use of collective knowledge in innovation processes. Besides software engineer applications, some studies also apply online ethnography for human-computer interaction domain, that is the case of Hussein, Mahmud, and Noor [20].The authors conducted netnographic approaches to investigate frustrations among practitioners while incorporating user experience design discipline in software development processes. Their findings provide insights to improve user experience design processes. Therefore, in order to present an overview of how online ethnographic studies have been performed in Computer Science, we conducted a systematic mapping that is detailed in the next section.

3 Research Method

This study was carried out by following the established guidelines for conducting Systematic Mapping Studies suggested by Petersen et al. [21]. A Systematic Mapping Study is a method designed to provide a wide overview of a research field by exploring the research data existence and by providing the amount and classification of such research data [22]. According to Petersen, the mapping process consists of planning, conducting, and reporting. Next sub-sections detail how each phase was performed from planning to conduction, delineating the research questions, search strategy, selection criteria, and data extraction strategy. In addition, the report is presented in the results section.

3.1 Planning

Before conducting the systematic mapping, we had forethought the research questions and establish the research protocol. The protocol was delineated considering the steps of search strategy, selection criteria, and data extraction strategy. As stated by Kitchenham [22], a research protocol is essential for the sake of reducing chances of researcher’s bias.

Research Questions.

The main goal of this mapping is to present how online ethnographic studies have been performed in Computer Science. To accomplish this goal, we defined the following three research questions:

  • RQ1 - Which areas of Computer Science have been using online ethnography research method?

  • RQ2 - Where are online ethnography studies published?

  • RQ3 - How are online ethnography studies performed?

By answering these research questions, this study provides an overview of how online ethnographic studies have been performed in Computer Science and we can understand where these studies are headed.

Search Strategy.

Search strategy comprises the identification of search terms for querying applicable scientific databases. Seven relevant Computer Science databases were selected for the search: ACM Digital Library, EBSCO Host, Elsevier ScienceDirect, IEEE Xplore, ProQuest, Springer Link, and Web of Science. The search string was composed based on well-known online ethnographic research methods such as netnography [13], virtual ethnography [9], webnography [23], and cyber-ethnography [24]. Therefore, in order to automate the search in the selected databases, we defined the following search string with their corresponding logical operators: “online ethnography” OR netnography OR “virtual ethnography” OR webnography OR “cyber-ethnography”.

In addition, Springer and Web of Science databases provide a mechanism to filter by the discipline of Computer Science, which was helpful and returned more accurate results. For all other selected bases, the filter per discipline was performed manually since they do not provide an interface to refine the search considering the discipline. Furthermore, names of the computer science disciplines were not added as part of the search criteria in order to comprehend all possible computer science areas and avoid inaccurate results.

Selection Criteria.

We assessed each publication returned from the automated search after selecting whether or not it should be included by considering the selection criteria. The selection criteria were composed by inclusion and exclusion criteria. In a first filter, we included/excluded papers based only on titles and abstracts. In a second filter, we ensured a full-text reading. Thus, the following inclusion criteria were applied in the first filter:

  • Studies should be published in the computer science area.

  • Studies should present reference(s) of use of online ethnography methods.

Publications that met at least one of the following exclusion criteria were removed:

  • Books.

  • Duplicated papers.

  • Studies written in any other language other than English.

  • Studies presenting summaries of tutorials, panels, poster sessions or workshops.

  • Conference covers and table of content.

During the full-text reading stage, we analyzed all paper content. The goal of this stage was to select the studies according to the following inclusion criteria:

  • Studies should present references of online ethnography methods application, being that a unique method or part of a mixed method.

  • Studies should describe the methodology application.

Data Extraction Strategy.

The data extraction strategy was based on defining a data set that should be collected in order to answer the research questions. RQ1 could be answered by defining the Computer Science area or sub-discipline which the study belongs to, such as User Interfaces and Human Computer Interaction, Software Engineering, and so on. RQ2 and RQ3 data set are composed of a conjunction of data as shown in Table 1.

Table 1. Data extraction for each research question.
Full size table

3.2 Conduction

We searched for papers in the selected databases during April 2017. The first results led us to a set of 853 studies (Table 2). After the results’ compilation, we applied the exclusion criteria, resulting in 762 publications. Afterwards, a total of 62 were selected in accordance with the inclusion criteria from the first stage, where only the title and abstract were considered. Finally, in the full-text reading stage, 36 publications were selected. The selection process is shown in Fig. 1.

Table 2. Number of publications per database
Full size table
Fig. 1.
figure 1

Selection process

Full size image

4 Results

4.1 Computer Science Areas Applying Online Ethnography Methods

The results for question RQ1 – Which areas of Computer Science have been using online ethnography research method? – revealed that 83% of result set studies applying an online ethnographic method are classified in User Interfaces and Human Computer Interaction area. The remaining studies are categorized in Software Engineering, Database Management, and Artificial Intelligence and Robotics areas, as shown in Fig. 2.

Fig. 2.
figure 2

Computer Science areas applying online ethnography methods

Full size image

4.2 Published Online Ethnography Studies

Results for question RQ2 – Where are online ethnography studies published? – revealed that 55.60% of result set studies are published as articles in journals and 44.40% are conference proceedings. All studies’ references are shown in Table 3, which also shows the number of publications per year. The complete list of periodic and conference names is displayed in Table 4.

Table 3. Selected papers
Full size table
Table 4. Periodic and Conference names where studies are published
Full size table

4.3 How Online Ethnography Studies Are Performed

Considering the applied methodology, the results for RQ3 – How are online ethnography studies performed? – exposed that the majority of the studies on Computer Science (86.1%) followed virtual ethnography and netnography methods. Only one study adopted cyber-ethnography method and four studies called specifically online ethnography with no distinction for a specific method. Figure 3 shows the adopted methods on the selected set of studies.

Fig. 3.
figure 3

Adopted methods

Full size image

Another outcome related to studies’ methodology is that 15 studies employed mixed methods, using virtual ethnography or netnography plus interviews, surveys or experiments. Most of these studies applied two methods, except for Rozas’ study [40], which applied virtual ethnography, interviews, and survey, and Bauer, Franke, and Tuertscher’s study [46] which applied netnography, survey, and experiment.

As stated by Bengry-Howell et al. [14] researchers have used online ethnography methods to study a particular online community, which is aligned with our mapping outcome. The majority of selected papers have used only one community to perform their studies. Only eight studies have adopted two or more communities to perform their studies, and one study has not provided this information. Figure 4 details the number of communities per study. In addition, the number of community’s members, for those studies that informed this data, vary from a few members [33] to more than 1 million registered members [40].

Fig. 4.
figure 4

Number of communities per study

Full size image

The period performing an online ethnography study, for those studies that asserted this information, vary from 1 week [36] to 5 years [18]. One year or less is the most common period, stated in 17 studies. Two studies conducted a 2 years research, and other two studies conducted a 4 years research. While running an online ethnography research, the collected and analyzed data is mostly text-based. All studies have collected and analyzed text-based data. However, besides text, some studies also collected and analyzed videos [15, 31, 36] and images [11, 51, 46].

Depending on the participation of the researchers in the community, an online ethnography research can vary from non-participatory (passive) to participatory (active) [17]. The results from our mapping show that 58.7% of the researchers played as passive, while 41.3% participated as active.

5 Discussion

The primary goal of this systematic mapping study is to present how online ethnographic studies have been performed in Computer Science. Based on our analysis of 36 resultant papers, it is evident that Human Computer Interaction is the Computer Science area that most takes advantage of online ethnographic methods. Furthermore, these studies have been published in diverse conferences and periodic. However, the main thoughts and considerations were bounded around the online ethnographic methods approaches that have been used in Computer Science studies.

The majority of reviewed studies have adopted virtual ethnography or netnography methods to achieve their goals. For instance, Sigfridsson and Sheehan [15], used virtual ethnography method for studying free and open source software communities, which contributed assessing multiple and interlinked dimensions and interpreting the context of communities’ activities. Another example is Synnott, Coulias, and Ioannou study [52], which applied virtual ethnography method as part of their multi-method approach to provide a case study analysis of a group of alleged Twitter trolls. In their case, the method provided the research engagement as observational and participatory in a specific online community. Additionally, Teixeira [45], has applied netnography method to delineate how patients use open source disease control software developed by other patients. Despite the fact that netnography has his roots in Marketing discipline, it has been adopted by other disciplines, including Computer Science.

Since online ethnography methods adopt principles of ethnographic research, such as user observation and researcher participation, they can easily be part of a mixed-method approach being used in combination with interviews, surveys or experiments for example. Online ethnography combined with interviews can provide a deep understanding of a specific raised theme.

Findings of this study show that most of the resulted publications focused to study a unique online community, depending on the particular researcher’s interest and mainly on the research goal. However, there is no right or wrong regarding the number of communities included in a study, but it is important to bear in mind the criteria to select the appropriated community to perform the research. In general, as stated by Kozinets [3], online communities should be selected to have a focused topic relevant to the research question, higher number of posts and interactivity, heterogeneity, and rich in data.

The interactivity and number of posts commonly depend on the numbers of community’s members. The number of members in a community vary from study to study and is related to the study goals and the selection criteria used to select the community. Another consideration is the period of time performing this sort of qualitative study. Such methods require researcher’s immersion into the online community long enough to become familiar with the community’s culture [53, 3].

After the researcher becomes familiar with community’s culture, it is possible to begin collecting data. Since online communities data is predominantly text-based, researchers can benefit from the practically automatic transcription of gathered posts [3]. While all resulted studies from this mapping collected and analyzed text-based data, few studies additionally explored videos and images as part of their data collection and analysis. Furthermore, there are two important elements of data collection, which involves the straight gathered data from online communities members’ communication, and the data the researchers address related to their participant observations and interactions with members’ community [3].

Related to researchers’ participation in the online communities, the applied methods can vary from passive participation to active participation. A passive participation means that the researcher is a member of the community but observes the group without interacting with people. On the other hand, an active participation implies that the researcher is actively engaged and involved in community’s activities [54]. To conclude, active researcher participation aid to obtain rich data but it is not always an easy process.

6 Conclusion

In the research herein presented, we focused on how online ethnographic studies have been performed in Computer Science. Through a systematic mapping study about online ethnography methods, we deepen our understanding not only about the domain areas on Computer Science but also about the main processes applying these methods.

The mapping study presented that four Computer Science areas have been using some sort of online ethnography method, being them User Interfaces and Human Computer Interaction, Software Engineering, Database Management, and Artificial Intelligence and Robotics areas (with 30, 3, 2, and 1 citation(s), respectively). In addition, from the mapped online ethnography methods, we can highlight virtual ethnography and netnography, which also are often used in mixed-methods in combination with interviews, surveys, and experiments for example. Furthermore, the community selection is an important stage, where the researcher shall bear in mind its relevance to the research goals, activity, interactivity, heterogeneity, and rich in data. Additionally, the number of members in a community and period of time performing these qualitative methods vary according to each study. To complete, the researcher participation can be passive, when the researcher does not interact with the community, or active when the researcher interacts with the community members. For both researcher participation modes, the data collection and analysis are mostly grounded on text-based data, but it can also be supported by video and images.

The analyzed studies show us that the data analysis can be a challenge due to the large volume of data collected on online communities. Even when the researcher participates as passive, it is important to use a qualitative data analysis software to organize and filter the data. In addition, the use of online communities leads to ethical challenges for qualitative research, which is another perspective to be studied and we shall extend our understanding.