Eye Contact: Gaze as a Connector Between Spectators and Players in Online Games

Abstract

This paper proposes an experimental setting that investigates shared gaze integrations (constant gaze and eye contact) in games and their effects on the social presence perceived by different roles (players and spectators) in a remote scenario. In order to get insights, we conducted a study that is made up of 4 different conditions (2 roles and 2 gaze integrations). Results show, depending on the type of the gaze integration and the role, positive effects of gaze towards an increased awareness and engagement among participants. Through the inclusion of shared gaze information, a new nonverbal communication channel for players and spectators is created that presents an interesting design resource for future approaches of digital play. Designers should receive information on how to design gaze-based interfaces that do not distract players during play, and also give spectators the possibility to experience a game via the player’s eyes.

1 Introduction

Games and playful interactions have the power to foster social connections and establish interpersonal bonds among players. On the one hand social behavior in this context may occur in co-located settings, where the physical presence of players is inherently part of the social interaction, and, on the other hand, remote online settings, where the information flow has to happen via different mediated channels. One of those channels can be seen in the human gaze that forms one of the key factors in interpersonal communication [6, p. 86] and information flow [5, p. 67]. Apart from communication tools such as chat features, human gaze may serve as an additional channel for players to get in contact with the audience and other players. In contrast to standard game controllers, such as a mouse or a keyboard, no explicit input is required, as gaze, and more specifically eye contact, is part of the human repertoire to get in contact with other players. This aspect may also be relevant for audience integration in games as it might enrich the experience via participation. Although the inclusion of gaze may contribute to the social experience among players and spectators, social playful gaze can be rarely found in distributed and remote settings, such as online gaming. This can be explained by the fact that eye tracking technology in games is mainly employed as an input method for single player games with a focus on augmenting or replacing typical game controllers and game interfaces (e.g., using gaze interaction instead of a mouse to select game objects, increasing the opacity of interface elements by looking at them, etc.) [13].

Consequently, gaze-based input as a tool for interpersonal communication in multiplayer games is still an under-explored research area, which offers opportunities for both designers and researchers [14]. First attempts indicate that gaze can be harnessed for onlooker integration resulting in an increased social interaction among participants [10]. Other projects highlight the potential of social gaze in cooperative and competitive game settings (e.g., [7, 11]).

Our contribution should shed some light on how social gaze can be integrated and visualized in remote settings. Typically, the current gaze position of a player is constantly shown to the other players (constant gaze integration). In this paper we want to explore alternative strategies by investigating if event-driven gaze-based interactions (in our case: eye contact between a player and a spectator) can be used as a linking element between spectators and players, and if this strategy leads to a higher social presence experience in remote settings than constant gaze implementations. Via an asymmetrical experimental game setup (player as an active part, spectator as a passive element), we want to research if there is preference for a specific social gaze integration (constant gaze VS eye contact) by spectators and players.

By doing so, readers should gain an understanding of how to use gaze as a way to reduce the gap between the participants in remote settings. Game and interface designers should receive information on how to design gaze-based interfaces that do not distract players during play (i.e., “calm” integration), and also give spectators the possibility to experience a game via the player’s eyes. Furthermore, researchers should get insights on the eye contact integration strategies and its effects on the player experience.

2 Related Work

A range of previous studies (e.g., [9]) identified a relationship between non-verbal communication and the perceived social presence (i.e., a phenomenon when subjects successfully simulate other humans or non-human intelligences in computer-mediated environments [8]).

In that regard, Mansour et al. [9] point out that the complex nature of in-game conversation in multiplayer settings, e.g., clarifying mutual goals or discussing tactics, is based on both, verbal as well as nonverbal communication. Shahid et al. [12] investigated a setting with mutual gaze regarding perceived social presence and game experience, and found out that mutual gaze has a strong effect on the quality of interpersonal interactions between players in a video-mediated gameplay setting. This inclusion of shared gaze-based interaction in both, competitive [7] as well as cooperative gameplay [11], has shown to positively influence social presence, but also drastically changes in how players perceive each other and how they communicate (e.g., via combinations of gaze and verbal communication or by unintended communication via gaze). In a similar direction, Maurer et al. [10] investigated a shared gaze approach as a way to integrate a spectator in a co-located gameplay setting into the gaming experience. In this asynchronous setting (player with controller-based input; spectator with gaze-based input), they explored different levels of onlooker integration and identified a positive impact of this integration on the overall gaming experience for both, spectator as well as player. Based on their results, the researchers propose gaze-based interaction in games as an embodied means to mirror the experience of another player, thus, fostering social couplings and behavioral engagement.

However, research addressing the interpersonal link between audience and spectator via gaze is still very limited. Especially social presence can be an important aspect in such gameplay settings [2]. Hudson & Cairns point out the importance of social relationships between players having an influence on how a game is experienced [4].

Social presence does not only play an important role in co-located play scenarios, but is also relevant in the online games domain. This fact, however, leads to several research challenges. In contrast to co-located play the communication between players in mediated settings is filtered by the media technology [3]. Players cannot make use of all the available cues that are typically employed in face to face communication (verbal, paraverbal, and nonverbal communication). De Kort et al. [3] further note that the level of social presence is influenced by the properties of the media technology and its interface. Via the interface players are enabled to communicate both verbally and non-verbally granting various levels of representations in terms of appearance and behavior.

3 Using Social Gaze as an Interpersonal Link

Until now, research efforts employed gaze interaction without addressing specific in-game events, thus, following an always-on approach (constant gaze), where the gaze of a person is constantly visualized without any triggering mechanic. Studies indicated that, although the integration of gaze was experienced positively, in some situations the constant visualization of the current gaze position distracted players, and also led to ambiguous information. Thus, we argue that design strategies for visualizing gaze in-game are required to address this issue. In this paper, these strategies are considered and explored under the light of spectators integration. Our related research question is: How should gaze be integrated within the game interface without distracting players and giving the spectators a way to participate in a meaningful way?

The aforementioned beneficial aspects of including gaze interaction in games are especially relevant for players and spectators scenarios in remote settings, where people take both active and passive roles. Therefore, we explore the notion of gaze as a motivator for players, as well as gaze input as a way for spectators to participate in the gaming experience. Therefore, we focus both on the player and on the spectator. In our experiment, we wanted to find out if gaze can serve as a connector between players and spectators and how it impacts the perceived social presence between those two sides. We followed the notion of eye contact to drive our design. As gaze doesn’t relate to a required explicit input (in contrast to a mouse input device, a keyboard, a gamepad, etc.), it is part of the human repertoire to get in contact with others (specifically via eye contact). The gaze interaction we use in our experiment aims at creating an additional channel for players and spectators to get in contact with each other (apart from communication tools such as chat features).

The main contribution of this paper lies in researching strategies of integrating game spectators and players via gaze and creating an interpersonal non-verbal link between the player and the spectator. Via our approach we want to give the notion of being a spectator a meaning and function within the game by considering mutual gaze locations in the game as the event-based approach to trigger different gaze-visualization strategies. In doing so, we explore ways of how spectators can be integrated into the game experience, and create a bidirectional interaction.

3.1 Experiment Description

To investigate the impact of gaze-based nonverbal communication on players and spectators in remote settings, we created an experimental setting supported via eye tracking technology. It focuses on the examination regarding the effects of a shared gaze within a game in a remote setting on the perceived social presence among the participants. The main idea behind the experiment was to visualize the counterpart’s gaze on the subject’s screen and vice versa (player or spectator). The gaze information is employed in two different ways: one strategy was to constantly show the current counterpart’s gaze position (abbreviated: ConstGaze), while in the second version the shared gaze information is shown when both, the player and the spectator, look at the same game entity (abbreviated: EyeCon). The experiment requires two subjects per session (a player and a spectator), and consists of four conditions along with two varying gaze-based communication modalities for the participants that are randomized in presentation order (i.e., randomized block factorial design): 1.a ConstGaze.Spectator, 2.a ConstGaze.Player, 1.b EyeCon.Spectator, and 2.b EyeCon.Player.

3.2 Conditions

In condition 1.a ConstGaze.Spectator and condition 2.a ConstGaze.Player, both participants, the player and the spectator, constantly see their partner’s current gaze position (see Fig. 1) — the own gaze is never exposed. On both sides, the gaze is visualized via a circle Fig. 1. In contrast to the player (game control via gamepad), spectators, besides using the eye tracking device, do not have further input possibilities at their disposal.

Fig. 1.

ConstGaze condition 1.a and 1.b: the current gaze position is constantly shown to the spectator (left: blue color) and the spectator (right: yellow color).Only the gaze position of the counterpart is visible. A participant’s own gaze is not displayed. (Color figure online)

In condition 1.b (EyeCon.Spectator) and 2.b (EyeCon.Player) we utilized a different strategy to set up a shared gaze experience (see Fig. 2). Any gaze visualization is triggered by an eye contact between the player and the spectator. Particularly when both subjects are gazing near the same entity, the entity then is marked with a circle (it has the same visual quality as in condition 1.a and 2.a) on the players’ and the spectator’s screen. Additionally, the exact gaze point of the counterpart is displayed as a small dot. As in condition 1.a and 2.a the own gaze spot, however is not exposed.

Fig. 2.

EyeCon Condition 2.a and 2.b: a game entity is visually highlighted via a circle when both participants (left: spectator in blue color and right: player in yellow color) look at it. Also a subtle dot indicates the unfiltered counterpart’s gaze information. (Color figure online)

3.3 Game

To minimize hurdles before entering and to allow the subjects to start playing without spending too much time in learning the game, we chose the very popular game Pac-Man for our experiment. Although the game is easy to learn, it offers several interesting game mechanics. Pac-Man is a single player game, where the player navigates the game character, a yellow puck, through a maze. To complete a level, the player has to collect dots. The player also has to avoid four enemies (ghosts), which are roaming around in the levels. Each time Pac-Man touches an enemy, he loses one of his three lives. The game ends when all lives are lost. Each level also provides four Power Pallets. After collecting one Power Pallet, Pac-Man temporarily is able to eat the enemies.

The overall goal of the game is to reach a high-score. The score can be increased by eating the different game entities. For our experiment we modified an open source version of the game¹ made with the Unity game engine².

Fig. 3.

An illustration of the technical setup consisting of 2 connected PCs and 2 eye trackers.

3.4 Technical Setup

For the experiment two PCs, connected via ethernet, and two screens were set up in two separate rooms. Each PC was connected to an eye tracking device (Tobii EyeX³). While the game was installed on the first PC (player-PC) only, the second one (spectator-PC) exclusively served as a tracking data provider. Both screens were connected to the player-PC (see Fig. 3). The application installed on the spectator-PC connected to the player-PC and started sending eye tracking data. To minimize delays, caused by the network connections, the data was sent unreliable sequenced. This means that not the delivery, bu the order is guaranteed (only the newest packet is accepted).

The main application, the modified Pac-Man game, renders the game two times simultaneously. Each rendering includes one of the two gaze positions. Hereby, it is achieved to visualize only the desired gaze on the respective screen (e.g. player gaze on spectator screen). Thus, the processing of the tracking data exclusively takes place on the main application. To optimize the application and to set it up for the experiment, a few parameters can be changed. The strength of the filtering of the eye tracking data can be altered to avoid jitter artifacts of the gaze information. Also the radii of the different trigger areas, used for the EyeCon conditions (see Sect. 3.2), around the game entities can be changed.

3.5 Hypotheses

Based on the aforementioned considerations, we assume that the four conditions will arouse a different feeling of social presence. As our conditions are based on the very well-known and rather simple Pac-Man game design with a short interaction time in mind, we focused only on two aspects that are associated with social presence: co-presence and psycho-behavioral interaction (sub dimension: attentional engagement) introduced by [1]. The more complex aspects of social presence, subjective and intersubjective symmetry, were not examined in the study (see Sect. 3.7 for a more detailed description of the scales). The following two hypotheses were formulated:

H1 - co-presence: Dependent on the player role (player VS spectator) and on the social gaze-based interaction (ConstGaze VS EyeCon) players will perceive co-presence differently. We expect an interaction effect of the social gaze-based interaction and the player role on co-presence: (a) Among spectators, ConstGaze will result in a higher co-presence as compared with EyeCon; (b) Among players ConstGaze will result in a lower co-presence as compared with EyeCon.

H1 addresses the question how players and spectators experience co-presence if the social gaze-based interaction is either constantly shown or displayed when the participants’ eyes meet. It is hypothesized that condition 1 (ConstGaze.Spectator) will have higher ratings in comparison to condition 3 (EyeCon.Spectators), as this way provides relevant information for spectators about the current players’ behavior (strategies, goals, etc.). On the contrary, we deem that the continuous display of the spectators’ gaze position will lead to low co-presence ratings by the players as it might distract them from game-relevant events leading them to a blend out the gaze visualization (gaze info perceived as background noise). Issues in the context of game-relevant events in Pac-Man may range from occluding enemies via gaze to confusing the animation of the gaze visualization with the movement of the enemies.

H2 - attentional engagement: Dependent on the player role (player VS spectator) and on the social gaze-based interaction (ConstGaze VS EyeCon) players will perceive the attentional engagement differently. We expect an interaction effect of the social gaze-based interaction and player role on the attentional engagement: (a) Among spectators, ConstGaze will result in higher attentional engagement as compared with EyeCon; (b) Among players ConstGaze will result in lower attentional engagement as compared with EyeCon.

Like in H1 we hypothesized that condition 1 (ConstGaze.Spectator) would have higher ratings in comparison to condition 3 (EyeCon.Spectators). Based on the aforementioned considerations, we thought that ConstGaze.Player (condition 2) would receive lower ratings than the EyeCon.Player condition (condition 4).

3.6 Participants and Procedure

The sample consisted of 20 participants, age 19 to 30 years (11 female, mean age = 22.15, SD = 2.65). All participants were either students or research staff of the University of Applied Sciences Upper Austria. Furthermore, subjects represented a variety of disciplines of education having a background in 3D animation, software engineering, or in game design. A majority of the participants (15 people) are keen in playing video games, as they indicated to play games at several times a week or daily.

Each evaluation session was made up of two subjects that either played (role: player) or watched (role: spectator) the game Pac-Man in two separate rooms. By choosing a within-subject design, all participants had to play each of the aforementioned conditions (play time limit for each condition: 3 min). If a subject lost the game earlier, the game was restarted. The time limit resulted in a maximum active testing time of twelve minutes.

The experiment was made up of the following procedure: as a first step, the experimenter welcomed the two participants and provided a short introduction that gave an overview of the overall procedure. After the eye tracking devices were calibrated, subjects were instructed about the game rules, goals and mechanics of the game. When subjects confirmed that every aspect was clear to them, the experiment began. When the interaction with one condition was completed, the experimenter instructed the participants to fill out a questionnaire (Networked Minds Social Presence Inventory by [1]). To minimize distractions caused by adjustments, subjects switched roles only once during the experiment. After participants were finished with the first condition, the remaining three were evaluated. Following, participants had the possibility to give comments on the conditions with a focus on social presence and their experience with the gaze interaction, via a interview carried out by the experimenter. The procedure itself took between 50 to 60 min.

3.7 Measures

All dependent measures were collected by using validated scales. To measure the co-presence and attentional engagement dimensions, the Networked Minds Social Presence Inventory by [1] was employed, which consists of three scales (co-presence, psycho-behavioral interaction, and subjective and inter-subjective symmetry). We only employed the first two scales, co-presence and psycho-behavioral interaction.

Co-presence (4 items) forms a core component of social presence and measures the degree to which the players feel as if they are together in the same space. It emerges when players have the impression that they automatically detect and classify a form as another, which is moderated by the degree to which the user and the agent appear to share an environment together. The second scale, psycho-behavioral interaction, measures the player’s perception of attention, emotional contagion, and mutual understanding with participants. In our study we only made use of the sub-scale perceived attentional engagement (3 items), which measures the degree to which a subject reports attention to the other. Items of the perceived attentional engagement-scale include statements such as “I was easily distracted from the spectator when other things were going on”. Furthermore, data on demographics as well as pre-experience with games was gathered.

3.8 Data Analysis

Data analysis for co-presence and attentional engagement ratings were analyzed using a repeated measures analysis of variance (ANOVA) with the two within-subject factors (CRF-22 design) comprising the player role (player VS spectator), and the type of social gaze interaction (ConstGaze VS EyeCon), as well as the social presence as the dependent variable (co-presence vs. attentional engagement). All statistic tests were carried out with SPSS 22. Significance was set at $\alpha =0.05$.

4 Results

In the following section results of the study are presented (dimensions co-presence and attentional engagement). Insights are provided if the various conditions (with the different roles in the game) have an influence on the perceived social presence (dependent variable: co-presence and attentional engagement). Gender, age, and occupation were not significantly related to social presence ratings.

The results of the ANOVA indicate that co-presence was generally higher in the conditions where participants were in the role of the player than in the condition where participants were in the role of the spectator, F(1, 19) = 25.60, p = 0. The player role in combination with the type of social gaze interaction, F(1, 19) = 21.25, p = 0, indicate that, depending on the player role (spectator VS player), the perceived co-presence was either significantly higher (Const.Gaze Spectator: M = 5.26, SD = 1.4, EyeCon.Player: M = 6.03, SD=0.86) or lower (EyeCon.Spectator: M = 3.84, SD = 1.35, ConstGaze.Player: M = 5.36, SD = 1.08) (see Fig. 4). In general, it has to be noted that co-presence did not differ significantly between the conditions with ConstGaze and EyeCon, F(1, 19) = 2.19, p = .15.

Fig. 4.

Mean values and standard deviation bars for co-presence of all conditions (player roles, social gaze interaction) on a scale from 1 (“strongly disagree”) to 7 (“strongly agree”).

The second dimension, attentional engagement, was rated by subjects in a similar fashion. As in the co-presence dimension the results of the ANOVA revealed a significant main effect of player role, F(1, 19) = 44.19, p = 0, and the combination of the player with the type of the social gaze interaction, F(1, 19) = 26.86, p = 0. Thus, in the case that participants were in the role of the player, social presence was higher with EyeCon (M = 5.85, SD = 0.92) than with ConstGaze (M = 4.89, SD = 1.08); if participants were in the role of the spectator, ConstGaze resulted in higher attentional engagement (M = 4.58, SD = 1.66) than EyeCon (M = 2.82, SD = 1.35) – see Fig. 5). As in co-presence, no significant could be found regarding the type of social gaze integration, F(1, 19) = 1.07, p = .32.

Fig. 5.

Mean values and standard deviation bars for attentional engagement of all conditions (player roles, social gaze interaction) on a scale from 1 (“strongly disagree”) to 7 (“strongly agree”).

5 Discussion

In general, our study showed, dependent on the type of the gaze integration and the player role, the positive effects of a shared gaze towards an increased awareness and engagement among the players and the spectators. It presented a new way of nonverbal interaction for the audience and the players, which in turn, resulted in a range of gaze-based qualities emerging from this interaction channel. As anticipated, condition 1.a (ConstGaze.Spectator) received higher ratings by the audience in comparison to condition 1.b (EyeCon.Spectators) on both scales (co-presence and attentional engagement). Subjects indicated that it was interesting for them to see where the player was currently looking at, as it gave them information about their strategies to evade the ghosts, if the player was aroused, or when they would eat the next Power Pellet.

Additionally, it was revealed that the continuous display of the spectators’ gaze position led to relatively low social ratings by the players. This can be explained by the fact that several players mentioned that they were annoyed and distracted by the current gaze position of the spectators. Although they were aware of the spectators presence in the beginning of the play session, they tended to ignore the visual cue after some time. In some cases the animation and position of the circle (that was driven by the gaze behavior of the spectator) made some players believe that the spectator, although not intended, wanted to give them tips to progress in the game. This effect could be frequently seen in level 1 of the Pac-Man game, where the difficulty was much lower than in level 2. In level 2 the spectators’ was often ignored by the players.

Summarizing, results of the experiment show that the continuous display of the player’s gaze information is more suitable for passive player roles (like spectators). The eye contact approach, on the other hand, is preferred by active game roles (such as players or audiences that can alter game elements). Furthermore, our eye contact design can be integrated into other gaze-based interactions, such as pointing via gaze (example: look at an game element and press a button to reveal the current gaze position). This might be useful in a coop-remote setting, where one of the players uses his/her gaze to highlight an useful item for the other player. Generally, we deem that the gaze point could be incorporated more deeply into the actual mechanics of the game towards deliberately designing for in-game communication mechanics as an integral part of the game design.

6 Conclusion

This paper reported on an experimental setting that investigated two different shared gaze integrations in games (ConstGaze and EyeCon) and their effects on the social presence (co-presence and attentional engagement) perceived by player roles (player VS spectator). Our study findings show a positive effect of the shared gaze integrations on social presence depending on active and passive player roles. In general, we argue that our approach supports social couplings and interpersonal communication in remote gaming and shows promising results to increase the connection between players and spectators. Our findings stress that the inclusion of social and shared gaze information can create a new nonverbal communication channel for players and spectators alike that presents an interesting design resource for future designs of digital play. For future work, we plan to further investigate other genres for their potential to incorporate gaze interaction as well as explore our approach in situations with multiple players and spectators.