Abstract
With few exceptions, the role of user factors in the evaluation of virtual agents has largely been neglected. By taking them into account properly, researchers and virtual agent developers might be able to better understand interindividual differences in virtual agent evaluations. We propose the animacy attribution tendency as a novel user factor that assesses a users individual threshold to accept virtual entities as living and animate beings. Users scoring higher in animacy attribution tendency should accept anomalies in virtual agent behavior more easily and thus provide favorable evaluations. To investigate the impact of this novel concept along with other user factors, we first developed a test to assess interindividual differences of animacy attribution and subsequently carried out an online-study, during which participants had to evaluate video recordings of different virtual agents.
You have full access to this open access chapter, Download conference paper PDF
Similar content being viewed by others
Keywords
1 Introduction
While virtual agent technology advances towards more and more sophisticated patterns of social interaction, users are still reluctant in the adaptation of this new type of technology for everyday practices and react suspiciously when the virtual agents produce unexpected responses [1]. Thus, it is an important endeavor to identify factors contributing to the acceptance barrier and evaluation of virtual agents in order to address these requirements during the design process. Most research efforts focus on technological properties of virtual agents that are supposed to lead to better evaluations. Some of these features are a virtual agent’s ability to produce gestures [2, 3] and authentic facial expressions [4, 5] as well as its ability to follow social motives [6]. At the same time, user factors were considered only a supplement to the technological property in question with only few exceptions [7, 8]. Yet, understanding user factors in the evaluation of virtual agents might prove a valuable asset because (1) they provide additional insight into the evaluation of a virtual agent’s technological aspects and thereby (2) guide designers attention towards aspects of virtual agents that might have received little attention so far. As one positive exception in this context, Rosenthal-von der Pütten, Krämer, and Gratch [8] found that the predictive value of personality traits for agent evaluations even exceeded the actual behavior of the virtual agent. An important challenge in this endeavor is to identify user factors that are both stable over time and closely related to the evaluation of virtual agents. We propose that interpersonal differences in the tendency to attribute animacy to a virtual agent could serve as such a user factor.
2 User Factors in Virtual Agent Evaluation
User factors in virtual agent evaluations can be conceptualized on a continuum according to their temporal stability. Less stable user factors include variables such as prior experience and familiarity with virtual agents or new technologies in general as well as attitudes towards these technologies. A lack of prior experience with new media technologies has been found to invoke feelings of uncertainty about potential benefits and might lead to feelings of unrest about the adoption [9]. As a result, users might be less willing to engage in interactions with virtual agents (see also: technology acceptance model [10]). Additionally, inexperienced users provide evaluations with greater variance after first use, because they have no anchor for comparison. This bias can result from disappointment due to high expectancies that have not been met [11] or surprise by a virtual agents’ unexpected capability. As a consequence, studies investigating virtual agent technology through biased evaluations suffer from lower than expected statistical power compromising judgments about the effectiveness of the investigated technology. Therefore, less stable user factors such as prior experience with the media technology (i.e. virtual agents) or related technologies (e.g. video games) need to be taken into account in the analysis. Yet, although beliefs about virtual agents shape the early interaction process, they can change quickly with more time spent interacting with virtual agents [9]. Thus, the increased variance in virtual agent evaluations should decrease with growing familiarity. After continued interaction with a virtual agent, the influence of less stable user factors should be negligible.
Stable user factors, on the other hand, influence user evaluations during early stages and even after extended periods of interaction. Most studies control for participant sex, which usually brings in-group and out-group favoritism effects into light [12]. However, sex tends to accumulate variance from correlated stable factors [13], when they are not controlled for. Most of those stable factors belong to the realm of personality traits. Agreeableness, conscientiousness, and neuroticism, for example, have been shown to impact usefulness-evaluations of information technology [14]. In another study, however it has been found that the predictive value of rather general traits is limited, while traits more specific to the interaction with virtual agents prove more useful [8]. In this paper, we investigate a potentially contributing factor that is responsible for the perception of a virtual agent as a living entity in the first place: the attribution of animacy.
3 Animacy Attribution Tendency
The impression of animacy can be seen as a perceptual phenomenon, during which we ascribe intention towards an entity that follows specific movement patterns [15–17]. Early work by Heider and Simmel [18] already showed, how observers of rather simplistic movements of geometric shapes infer complex narratives about the motives and actions of the supposed actors. The type of motion carried out by the geometric shapes has repeatedly been shown to have a major impact on the likelihood of animacy attributions [16, 17, 19]. Recently Santos and colleagues [15] investigated different movement parameters and their impact on animacy judgments. They concluded that interruptions of otherwise generic movements, changes of the movement direction towards other entities and interactions between entities reliably lead to a stronger animacy attribution. Thus, even simplistic cues are sufficient to elicit attributions of animacy. When an entity is assumed to be animate, users interpret the capabilities of the respective entities similarly to those of living beings [20]. Virtual agents benefit from these attributions, because even rudimentary indications of animate behavior can lead to complex assumptions about their inner processes rendering them more realistic.
Perceiving other entities as animate is a fundamental human ability [15] and is thus deeply routed within our brains recruiting neural structures with strong ties to social cognition [21]. Entities in our perceptual environment elicit momentary animacy attributions or not. Thus, animacy can be considered as a dichotomous attribution. Human observers, however, might nonetheless differ in their judgments about the animacy of a virtual entity: We know about the artificial nature of a virtual entity and are able to reflect automated reactions towards it. Accordingly, users might either tend to take any deviation from expected behavior as an indicator of the mechanistic nature of a virtual agent or willingly suspend their disbelief by giving less thought to deviant behavior. As a consequence, users might differ in their respective thresholds of perceptual cues that are required to accept virtual entities as being animate. Thus, we argue that possible interpersonal differences in the thresholds of attributing animacy might bridge occurrences in human-agent interaction, in which a virtual agent produces unexpected responses (e.g. repetitious or unrelated responses). Consequently, users with a low threshold for animacy attribution should be less sensitive to odd reactions and thus report less negative evaluation results.
4 Method
Prior research exclusively focused on the influence of behavioral properties of virtual entities on the likelihood of animacy attributions. Thus, we first needed to develop a measure for interindividual differences in animacy attribution. In a second step, we assessed the influence of various user factors on evaluations of virtual agents. We created an online questionnaire containing the newly developed measure and measures for trait empathy, three big-five personality traits, as well as sociodemographic questions. Additionally, we randomly implemented one of four recordings of social interactions with different virtual agents from the internet, for which participants had to evaluate the virtual agent and indicate their subjective impression of animacy. The online questionnaire was filled in by N = 81 students from different disciplines. Because we employed an online questionnaire during which participants had to continually focus their attention on an English video, we had to exclude 25 participants, because they reported insufficient understanding of the language (they were non-native speakers), were distracted during their participation, or reported poor internet connection. As a result we analyzed N = 56 valid data sets (age M = 23.21, SD = 2.96, female = 82,1 %).
4.1 Materials
Animacy Attribution Tendency.
The behavioral cues leading to animacy attributions that are described in the literature include interrupted movement near secondary objects, approach of secondary objects, responses of secondary objects, speed change, direction change, and spatial context [15, 16, 22]. The presence of these characteristics has a strong impact on the perceived animacy and should result in firm attributions of animacy, which is only of little diagnostic value. Therefore, very subtle versions of these behavioral cues had to be employed. We created 165 stimuli depicting a black circle moving across the screen (see Fig. 1), for which we varied a broad range of movement characteristics at different intensities (see Table 1). “The stimuli were created in Adobe Flash (Adobe Systems) and presented to the participants using E-Prime 2 (Psychology Software Tools, Inc.) on PCs with 19” displays and 60 cm seating distance.
A pre-test sample of N = 48 undergraduate students (age M = 21.53, f = 76.6 %) rated the video files on a 4-point scale from “0 – not animate” to “3 – animate”. Thus, participants scoring high in Animacy Attribution Tendency tend to provide higher animacy ratings than participants with lower scores. All stimuli were presented in randomized order with short breaks every few minutes. From the dataset we selected 24 items of different levels of item difficulty and different types of behavioral cues that had sufficient variance to be used as an indicator of the animacy attribution tendency.
For the sake of robustness, the data from all 81 participants of the main study were used to assess descriptive data and scale uniformity. The developed test for animacy attribution formed a highly reliable compound (α = .96) that is normally distributed with a mean score of M = .47 (SD = .19; normed to a range from 0 to 1) and a range of 0.86 of the scale. The results remained stable with the reduced sample size.
Other User Factors.
We used the empathy quotient by Baron-Cohen and Wheelwright [23] to assess trait empathy. It contains 40 items and 20 filler-items that are unrelated to empathy. Participants rated the statements on 4-point likert scales from “strongly disagree” to “strongly agree”. Participants scored 1 point for mildly empathic responses and 2 points for strongly empathic responses (M = 41.52, SD = 10.38, α = .766). The German Big-Five Inventory – Short Form by Rammstedt and John [24] assessed neuroticism (M = .58, SD = .19, α = .73), openness to experiences (M = .76, SD = .02, α = .77), and contentiousness (M = .62, SD = .16, α = .64) using 7-point likert ratings from “strongly disagree” to “strongly agree” and 13 items. We additionally asked for the participants’ age, sex, and prior experience with virtual agents (“How familiar are you with virtual agents?” on a scale from 0 – “not at all” to 10 “very familiar”).
Agent Evaluation.
We employed the Agent Persona Inventory by Baylor and Ryu [25] to assess user evaluations of the virtual agents. We used 15 items with 7-point likert-ratings of the API including the sub-scales credible (α = .76), engaging (α = .86), and human-like (α = .80). Furthermore, we assessed perceived animacy for each virtual agent with a single item for which participants had to rate the agent’s animacy on a scale from “0 – not animate” to “3 – animate”.
4.2 Procedure
Participants were recruited via e-mail. The message contained a link to the online questionnaire. After a short explanation, they first completed the test for animacy attribution tendency. In the next part, they filled in the empathy quotient and the big-five inventory – short form. A video recording of a user interacting with a virtual agent followed the personality measures. The videos were obtained from the internet and represented different virtual agent technologies and scenarios (Obadiah & Spike, chatting, SEMAINE Project; Sgt. Star, virtual guide, ICT Virtual Human Toolkit; Lt. Rocko, virtual patient, ICT Virtual Human Toolkit). We decided to use video recordings from four different interactions to assess the importance of user factors in a more general fashion, while previous research mostly investigated direct interaction with specific agents [e.g. 8]. After watching the complete videos, participants were asked to fill in the Agent Persona Inventory and indicated their perceived level of animacy of the virtual agent. In the remainder of the questionnaire, participants were asked about the above mentioned less stable user factors and sociodemographic variables. We also included several questions to control for attention, Internet connection issues, motivation, and language comprehension.
5 Results
We first wanted to investigate the uniformity of agent evaluations across the four presented video clips. A MANOVA revealed a significant multivariate effect for video clip on the subscales of the Agent Persona Inventory, V = 0.66, F(9,156) = 4.88, p < .001. Although this effect should be considered with caution due to problems with multivariate normality and the assumption of homogeneity of covariance matrices, univariate ANOVAs as follow up analyses confirm this result. We found significant effects for the subscales credible, F(3,52) = 11.23, p < .001, η 2 p = .39, engaging, F(3,52) = 26.96, p < .001, η 2 p = .61, and human-like, F(3,52) = 3.44, p < .05, η 2 p = .17. Closer inspections of the plots for each dependent variable and of post hoc comparisons indicate that these effects are a result of the very positive evaluations of Sgt. Star and to some extent of the negative evaluations of Obadiah.
Table 2 presents first-order correlations between the dependent variables and personality traits across all virtual agents. While situated animacy-judgments correlate highly with agent evaluation scales, .42 < r < .55, p < .01, animacy attribution tendency appears to be unrelated both to perceived animacy, r < .01, and agent evaluation, r < .07. Interestingly, we found a marginally significantFootnote 1 correlation between the animacy attribution tendency and openness to experiences, r = – 0.24, p < .10.
Further qualitative inspection of the correlation pattern for the different virtual agents revealed indicated heterogeneous results. Correlations between animacy attribution tendency and perceived animacy, for example, varied between – .30 and .24. Additionally, the empathy quotient and conscientiousness turned out to correlate significantly for only some of the virtual agents. Yet, the correlational differences between the virtual agents remain statistically insignificant due to low test power for the within-groups analyses (N = 17, 12, 11, 16).
To account for the significant differences in the evaluation ratings between the virtual agents, we z-standardized the evaluations within each virtual agent group. Thus, the difference between the virtual agents is removed in further analyses. Next, we carried out three multiple regression analyses, one for each subscale of the Agent Persona Inventory. Overall, the amount of explained variance was rather low. The included user factors only managed to explain sufficient variance for the engaging-subscale, R 2 adj. = .11. The predictive power of the animacy attribution tendency was low, β < .20, n.s. However, trait empathy explained a substantial amount of variance for the factors credible and engaging. Thus, more empathic participants perceived the virtual agents as less credible and less engaging. Additionally, older participants rated the virtual agents as less engaging (see Table 3).
We again inspected individual regression analyses for each virtual agent. Despite the low statistical power, we found different patterns of significant predictors. These differences appeared to be most pronounced for the engaging-subscale.
6 Discussion
We sought out to assess the differential impact of interindividual differences in animacy attribution on virtual agent evaluation compared to other user factors. Generally, animacy judgments appear to be strongly related to the evaluation of virtual agents. This seems to be true only for situational judgments and not for a general tendency of animacy judgments. Yet, it should be noted that the situational judgments about the video recordings were most likely not solely based on observed movement patterns. Instead, the strong correlation between situational animacy judgments and agent evaluation could have been subject to anchoring, because the question immediately followed the Agent Personal Inventory, which assesses agent evaluation in general. Therefore, participants might have employed a wider interpretation of animacy than intended. This is also supported by the fact that agent evaluation scales tend to correlate highly as evidenced in Table 2.
Despite the fact that our scale for the animacy attribution tendency reached an exalting internal consistency of α = .96, the question remains why we could not observe a correlation with agent evaluations and situational animacy judgments. Especially the latter case raises the question, whether the scale faces issues in validity despite its high reliability. One possible explanation could lie in the nature of animacy attributions: As recognizing animacy in other entities is a basic cognitive ability [15], there might simply be no differences between individuals. However, after an analysis of animacy judgments using fMRI Santos and colleagues [21] concluded that both low- and high-level brain structures are recruited in the process. Thus, while the detection of related movement is part of low-level cognitive abilities, interindividual differences might arise at the level of social cognition in the social neural network. These differences are also reflected in the high range of scores in the test we developed. However, there might have also been other factors such as a general response bias during uncertain decisions that could be partially responsible for the variance within the sample. Lastly, because demonstration videos of virtual agent technology usually do not include unexpected behaviors, the quality of the observed behaviors might have been too high compared to a normal interaction scenario, so that interindividual differences in the threshold of animacy attribution could have played a role. At this point the relevance of interindividual differences in agency attributions for agent perception remains unclear. The marginally significant correlation with openness to experiences suggests that users seeking out interesting experiences are less likely to attribute animacy. This might be related to higher expectancies from virtual entities that are required to perceive it as an interesting experience.
The low predictive value of the other user factors in our study challenges the notion that they are an important element to fully understand agent evaluations when technological properties are manipulated. We were only able to observe effects of trait empathy on two of the three employed evaluation scales. The remaining traits from the big-five personality model had no predictive value. However, this result is in line with the argument that we need to consider personality traits that are more specific to the investigated scenario [8]. The correlational pattern of personality traits and agent evaluation could also depend on the employed measure for agent evaluation, because the Agent Persona Inventory has primarily been developed for learning scenarios with pedagogical agents. Furthermore, the diagnostic properties of the Agent Persona Inventory have not fully been explored, yet. On another note, the rather small sample size of the current study should be considered in the interpretation of the results as well as the fact that participants in our study watched videos of virtual agents rather than directly interacting with them.
In separate analyses for the different virtual agent video clips that were part of our study we found indications that the correlational pattern of user factors and agent evaluation differed depending on the displayed virtual agent. This is unexpected, because we usually assume that our findings generalize to comparable scenarios. The absent correlation between animacy attribution tendency and agent evaluation might also be a result of this inconsistency. On the other hand, studies investigating the link between user factors and agent evaluation [7, 8] so far only included one scenario. Still, the relevance of the investigated scenario for the influence of user factors should—given that this indication is confirmed by a larger sample—be high, because the use cases of current virtual agent technology are rather diverse. As a consequence, we would not be able to include a fixed set of personality inventories into user studies, but would have to identify relevant user factors in the process.
Notes
- 1.
The correlation turns out to be statistically significant due to higher test power, when the complete sample (N = 81) was analyzed, r = – .24, p < .05.
References
Kramer, M., Yaghoubzadeh, R., Kopp, S., Pitsch, K.: A conversational virtual human as autonomous assistant for elderly and cognitively impaired users? Social acceptability and design considerations. Lecture Notes in Informatics (LNI) P-220, pp. 1105–1119 (2013)
Bergmann, K., Kopp, S.: Increasing the expressiveness of virtual agents: autonomous generation of speech and gesture for spatial description tasks. In: Decker, K., Sichman, J., Sierra, C., Castelfranchi, C. (eds.) Proceedings of 8th international Conference on Autonomous Agents and Multiagent Systems (AAMAS 2009), pp. 361–368. IFAAMAS, Richland (2009)
Pelachaud, C.: Studies on gesture expressivity for a virtual agent. Speech Commun. 51, 630–639 (2009)
Liebold, B., Ohler, P.: Multimodal emotion expressions of virtual agents. mimic and vocal emotion expressions and their effects on emotion recognition. In: Pun, T., Pelachaud, C., Sebe, N. (eds.) 2013 Humaine Association Conference on Affective Computing and Intelligent Interaction, ACII 2013, pp. 405–410. IEEE, Los Alamitos (2013)
Pelachaud, C.: Modelling multimodal expression of emotion in a virtual agent. Philos. Trans. R. Soc. Lond. B Biol. Sci. 364, 3539–3548 (2009)
Schönbrodt, F.D., Asendorpf, J.B.: The challenge of constructing psychologically believable agents. J. Media Psychol. 23, 100–107 (2011)
Kang, S.-H., Gratch, J., Wang, N., Watt, J.H.: Agreeable people like agreeable virtual humans. In: Prendinger, H., Lester, J.C., Ishizuka, M. (eds.) IVA 2008. LNCS (LNAI), vol. 5208, pp. 253–261. Springer, Heidelberg (2008)
von der Pütten, A.M., Krämer, N.C., Gratch, J.: How our personality shapes our interactions with virtual characters - implications for research and development. In: Safonova, A. (ed.) IVA 2010. LNCS, vol. 6356, pp. 208–221. Springer, Heidelberg (2010)
Karahana, E., Straub, D.W., Chervany, N.L.: Information technology adoption across time: A cross sectional comparison of pre-adoption and post-adoption beliefs. MIS Q. 23, 183–213 (1999)
Venkatesh, V., Davis, F.D.: A theoretical extension of the technology acceptance model: four longitudinal field studies. Manage. Sci. 46, 186–204 (2000)
Staples, D.S., Wong, I., Seddon, P.B.: Having expectations of information systems benefits that match received benefits: does it really matter? Inf. Manage. 40, 115–131 (2002)
Guadagno, R.E., Blascovich, J., Bailenson, J.N., McCall, C.: Virtual humans and persuasion: The effects of agency and behavioral realism. Media Psychol. 10, 1–22 (2007)
De Bolle, M., De Fruyt, F., McCrae, R.R., Lockenhoff, C.E., Costa, P.T., Aguilar-Vafaie, M.E., Ahn, C.K., Ahn, H.N., Alcalay, L., Allik, J., Avdeyeva, T.V., Bratko, D., Brunner-Sciarra, M., Cain, T.R., Chan, W., Chittcharat, N., Crawford, J.T., Fehr, R., Fickova, E., Gelfand, M.J., Graf, S., Gulgoz, S., Hrebickova, M., Jussim, L., Klinkosz, W., Knezevic, G., Leibovich de Figueroa, N., Lima, M.P., Martin, T.A., Marusic, I., Mastor, K.A., Nakazato, K., Nansubuga, F., Porrata, J., Puric, D., Realo, A., Reategui, N., Rolland, J.P., Schmidt, V., Sekowski, A., Shakespeare-Finch, J., Shimonaka, Y., Simonetti, F., Siuta, J., Szmigielska, B., Vanno, V., Wang, L., Yik, M., Terracciano, A.: The emergence of sex differences in personality traits in early adolescence: a cross-sectional, cross-cultural study. J. Pers. Soc. Psychol. 108, 171–185 (2015)
Devaraj, S., Easley, R.F., Crant, J.M.: Research note—how does personality matter? relating the five-factor model to technology acceptance and use. Inf. Syst. Res. 19, 93–105 (2008)
Santos, N.S., David, N., Bente, G., Vogeley, K.: Parametric induction of animacy experience. Conscious Cogn. 17, 425–427 (2008)
Tremoulet, P.D., Feldman, J.: Perception of animacy from the motion of a single object. Perception 29, 943–951 (2000)
Berry, D.S., Misovich, S.J., Kean, K.J., Baron, R.M.: Effects of disruption of structure and motion on perceptions of social causality. Pers. Soc. Psychol. Bull. 18, 237–244 (1992)
Heider, F., Simmel, M.: An experimental study of apparent behavior. Am. J. Psychol. 57, 243–259 (1944)
Barrett, J., Johnson, A.H.: The role of control in attributing intentional agency to inanimate objects. J. Cogn. Cult. 3, 208–217 (2003)
Hoffmann, L., Krämer, N.C., Lam-chi, A., Kopp, S.: Media equation revisited: do users show polite reactions towards an embodied agent? In: Ruttkay, Z., Kipp, M., Nijholt, A., Vilhjálmsson, H.H. (eds.) IVA 2009. LNCS, vol. 5773, pp. 159–165. Springer, Heidelberg (2009)
Santos, N.S., Kuzmanovic, B., David, N., Rotarska-Jagiela, A., Eickhoff, S.B., Shah, J.N., Fink, G.R., Bente, G., Vogeley, K.: Animated brain: a functional neuroimaging study on animacy experience. Neuroimage. 53, 291–302 (2010)
Tremoulet, P.D., Feldman, J.: The influence of spatial context and the role of intentionality in the interpretation of animacy from motion. Percept. Psychophys. 68, 1047–1058 (2006)
Baron-Cohen, S., Wheelwright, S.: The empathy quotient: an investigation of adults with asperger syndrome or high functioning autism, and normal sex differences. J. Autism Dev. Disord. 34, 163–175 (2004)
Rammstedt, B., John, O.P.: Kurzversion des big five inventory (BFI-K). Diagnostica 51, 195–206 (2005)
Baylor, A., Ryu, J.: The API (Agent Persona Instrument) for assessing pedagogical agent persona. In: Lassner, D., McNaught, C. (eds.) Proceddings of World Conference on Educational Multimedia, Hypermedia and Telecommunications 2003, pp. 448–451. AACE, Chesapeake (2003)
Acknowledgement
This study was partially funded by the German Research Foundation (DFG) under grant 1760 (“CrossWorlds: Connecting Virtual and Real Social Worlds”). We thank Kevin Pfeffel and Yannik Augustin for their help in the creation of the stimuli.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Liebold, B., Pietschmann, D., Ohler, P. (2015). Do We Differ in Our Dispositional Tendency to Perceive Virtual Agents as Animate Beings? . In: Kurosu, M. (eds) Human-Computer Interaction: Users and Contexts. HCI 2015. Lecture Notes in Computer Science(), vol 9171. Springer, Cham. https://doi.org/10.1007/978-3-319-21006-3_43
Download citation
DOI: https://doi.org/10.1007/978-3-319-21006-3_43
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-21005-6
Online ISBN: 978-3-319-21006-3
eBook Packages: Computer ScienceComputer Science (R0)