Educational Technology Research and Development

, Volume 67, Issue 6, pp 1385–1404 | Cite as

The impact of a user’s biases on interactions with virtual humans and learning during virtual emergency management training

  • Sarah A. Zipp
  • Scotty D. CraigEmail author
Research Article


Biases influence the decisions people make in everyday life, even if they are unaware of it. The current study investigates the extent bias behavior transfers into social interactions in virtual worlds by investigating the effect of aversive racism on helping behaviors and learning within a virtual world for medical triage training. In a 2 × 2 × 2 mixed design, two between subjects variables, participant skin tone (light, dark) and avatar skin tone (light, dark), and one within subjects variable, agent skin tone (light, dark), were manipulated. Effects on helping behaviors were observed on three measures: time to initiate help, errors made while helping virtual patients, and learning. Participants, regardless of their skin tone or their avatar’s skin tone, took more time to initiate help and made more errors while triaging dark-skinned agents in comparison to light-skinned agents. The bias against virtual patients with a darker skin tone also served as a mediating factor for learning with lower prior knowledge increasing the errors made for dark skinned virtual patients, which had more of a negative impact on learning than the errors made on light skin virtual patients. This study showed that participants applied general biases against dark-skinned agents regardless of participant’s ethnicity or avatar’s skin-tone. It indicates the importance of considering biases when designing training systems.


Bias Aversive racism Virtual humans Avatars Animated agents Virtual worlds Virtual environments Emergency response training 



This research was partially supported by the Department of Defense [U.S. Army Medical Research Acquisition Activity] under Award Number (W81XWH-11-2-0171). Views and opinions of, and endorsements by the author(s) do not reflect those of the US Army or the Department of Defense. This research was also partially funded by the Fulton Undergraduate Research Initiative at Arizona State University (

Compliance with ethical standards

Conflict of interest

The authors declare that they have no conflict of interest.


  1. Alison, L., van den Heuvel, C., Waring, S., Power, N., Long, A., O’Hara, T., et al. (2013). Immersive simulated learning environments for researching critical incidents: A knowledge synthesis of the literature and experiences of studying high-risk strategic decision making. Journal of Cognitive Engineering and Decision Making,7(3), 255–272.CrossRefGoogle Scholar
  2. Andrews, D. H., & Craig, S. D. (Eds.). (2015). Readings in training and simulation (Vol. 2): Research articles from 2000 to 2014. Santa Monica, CA: Human Factors and Ergonomics Society.Google Scholar
  3. Backlund, P., Engström, H., Hammar, C., Johannessen, M., & Lebram, M. (2007, July). Sidh-a game based firefighter training simulation. In Information Visualization, 2007. IV’07. 11th International Conference (pp. 899–907). IEEE.Google Scholar
  4. Bailenson, J. N., Yee, N., Blascovich, J., Beall, A. C., Lundblad, N., & Jin, M. (2008). The use of immersive virtual reality in the learning sciences: Digital transformations of teachers, students, and social context. The Journal of the Learning Sciences,17(1), 102–141.CrossRefGoogle Scholar
  5. Brewer, M. B. (1999). The psychology of prejudice: Ingroup love or outgroup hate? Journal of Social Issues,55(3), 429–444.CrossRefGoogle Scholar
  6. Burns, M. D., Monteith, M. J., & Parker, L. R. (2017). Training away bias: The differential effects of counterstereotype training and self-regulation on stereotype activation and application. Journal of Experimental Social Psychology,73, 97–110.CrossRefGoogle Scholar
  7. Butler, A. C., Karpicke, J. D., & Roediger, H. L., III. (2008). Correcting a metacognitive error: Feedback increases retention of low confidence correct responses. Journal of Experimental Psychology. Learning, Memory, and Cognition,34, 918–928.CrossRefGoogle Scholar
  8. Cassell, J. (2000). Embodied conversational agents. Cambridge, MA: MIT press.CrossRefGoogle Scholar
  9. Conradi, E., Kavia, S., Burden, D., Rice, A., Woodham, L., Beaumont, C., et al. (2009). Virtual patients in a virtual world: Training paramedic students for practice. Medical Teacher,31(8), 713–720.CrossRefGoogle Scholar
  10. Cooper, T. (2007). Nutrition game. In D. Livingstone & J. Kemp (Eds.), Proceedings of the Second Life Education Workshop 2007 (pp. 47–50). Chicago, IL. Retrieved from
  11. Correll, J., Park, B., Judd, C. M., & Wittenbrink, B. (2002). The police officer’s dilemma: Using ethnicity to disambiguate potentially threatening individuals. Journal of Personality and Social Psychology,83(6), 1314–1329.CrossRefGoogle Scholar
  12. Craig, S. D., Gholson, B., & Driscoll, D. (2002). Animated pedagogical agents in multimedia educational environments: Effects of agent properties, picture features, and redundancy. Journal of Educational Psychology,94, 428–434.CrossRefGoogle Scholar
  13. Craig, S. D., & Schroeder, N. L. (2017). Reconsidering the voice effect when learning from a virtual human. Computers & Education,114, 193–205.CrossRefGoogle Scholar
  14. Craig, S. D., & Schroeder, N. L. (2018). Design principles for virtual humans in educational technology environments. In K. Millis, J. Magliano, D. Long, & K. Wiemer (Eds.), Deep Learning: Multi-disciplinary approaches (pp. 128–139). NY, NY: Routledge.Google Scholar
  15. Craig, S. D., Twyford, J., Irigoyen, N., & Zipp, S. (2015). A test of spatial contiguity for virtual human’s gestures in multimedia learning environments. Journal of Educational Computing Research,53, 3–14.CrossRefGoogle Scholar
  16. Devine, P. G., & Elliot, A. J. (1995). Are racial stereotypes really fading? The Princeton trilogy revisited. Personality and Social Psychology Bulletin,21, 1139–1150.CrossRefGoogle Scholar
  17. Ducheneaut, N., & Moore, R. J. (2005). More than just ‘XP’: Learning social skills in massively multiplayer online games. Interactive Technology & Smart Education,2, 89–100.CrossRefGoogle Scholar
  18. Eastwick, P. W., & Gardner, W. L. (2009). Is it a game? Evidence for social influence in the virtual world. Social Influence,4(1), 18–32.CrossRefGoogle Scholar
  19. Falloon, G. (2010). Using avatars and virtual environments in learning: What do they have to offer? British Journal of Educational Technology,41(1), 108–122.CrossRefGoogle Scholar
  20. Fitzpatrick, T. B. (1975). Soleil et peau. Journal de Médecine Esthétique,2, 33–34.Google Scholar
  21. Folsom-Kovarik, J. T., & Raybourn, E. M. (2016, November). Total Learning Architecture (TLA) Enables Next-generation Learning via Meta-adaptation. In Interservice/Industry Training, Simulation, and Education Conference Proceedings. ITTSEC. Retrieved from
  22. Foronda, C. L., Shubeck, K., Swoboda, S. M., Hudson, K. W., Budhathoki, C., Sullivan, N., et al. (2016). Impact of virtual simulation to teach concepts of disaster triage. Clinical Simulation in Nursing,12(4), 137–144.CrossRefGoogle Scholar
  23. Fox, J., Bailenson, J. N., & Tricase, L. (2013). The embodiment of sexualized virtual selves: The proteus effect and experiences of self-objectification via avatars. Computers in Human Behavior,29, 930–938.CrossRefGoogle Scholar
  24. Gaertner, S. L., & Dovidio, J. F. (1986). The aversive form of racism. San Diego: Academic Press.Google Scholar
  25. Gaertner, S. L., & Dovidio, J. F. (2005). Understanding and addressing contemporary racism: From aversive racism to the common ingroup identity model. Journal of Social Issues,61, 615–639.CrossRefGoogle Scholar
  26. Gallagher, A. G., Seymour, N. E., Jordan-Black, J. A., Bunting, B. P., McGlade, K., & Satava, R. M. (2013). Prospective, randomized assessment of transfer of training (ToT) and transfer effectiveness ratio (TER) of virtual reality simulation training for laparoscopic skill acquisition. Annals of surgery, 257(6), 1025–1031.CrossRefGoogle Scholar
  27. Gamberini, L., Chittaro, L., Spagnolli, A., & Carlesso, C. (2015). Psychological response to an emergency in virtual reality: Effects of victim ethnicity and emergency type on helping behavior and navigation. Computers in Human Behavior,48, 104–113.CrossRefGoogle Scholar
  28. Gerard, H. B., & Hoyt, M. F. (1974). Distinctiveness of social categorization and attitude toward ingroup members. Journal of Personality and Social Psychology,29(6), 836–842.CrossRefGoogle Scholar
  29. Gholson, B., & Craig, S. D. (2006). Promoting constructive activities that support vicarious learning during computer-based instruction. Educational Psychology Review,18, 119–139.CrossRefGoogle Scholar
  30. Graesser, A. C., Li, H., & Forsyth, C. (2014). Learning by communicating in natural language with conversational agents. Current Directions in Psychological Science,23(5), 374–380.CrossRefGoogle Scholar
  31. Heinrichs, W. L., Youngblood, P., Harter, P. M., & Dev, P. (2008). Simulation for team training and assessment: Case studies of online training with virtual worlds. World Journal of Surgery,32, 161–170.CrossRefGoogle Scholar
  32. Hew, K. F., & Cheung, W. S. (2010). Use of three-dimensional (3-D) immersive virtual worlds in K-12 and higher education settings: A review of the research. British Journal of Educational Technology,41(1), 33–55.CrossRefGoogle Scholar
  33. Hu, X., Cai, Z., Han, L., Craig, S. D., Wang, T., & Graesser, A. C. (2009). AutoTutor LITE. In V. Dimitrova, R. Mizoguchi, B. du Boulay, & A. C. Graesser (Eds.), Artificial intelligence in education, building learning systems that care: From knowledge representation to affective modeling (p. 802). Washington, DC: IOS Press.Google Scholar
  34. Johnson, W. L., & Lester, J. C. (2016). Face-to-face interaction with pedagogical agents, twenty years later. International Journal of Artificial Intelligence in Education,26(1), 25–36.CrossRefGoogle Scholar
  35. Kim, Y., & Baylor, A. L. (2016). Research-based design of pedagogical agent roles: A review, progress, and recommendations. International Journal of Artificial Intelligence in Education,26(1), 160–169.CrossRefGoogle Scholar
  36. Lai, C. K., Skinner, A. L., Cooley, E., Murrar, S., Brauer, M., Devos, T., et al. (2016). Reducing implicit racial preferences: II. Intervention effectiveness across time. Journal of Experimental Psychology: General,145(8), 1001–1016.CrossRefGoogle Scholar
  37. Lane, H., Noren, D., Auerbach, D., Birch, M., & Swartout, W. (2011). Intelligent tutoring goes to the museum in the big city: A pedagogical agent for informal science education. In Artificial Intelligence in Education (pp. 155–162). Berlin/Heidelberg: Springer.Google Scholar
  38. Lerner, E. B., Schwartz, R. B., Coule, P. L., Weinstein, E. S., Cone, D. C., Hunt, R. C., et al. (2008). Mass casualty triage: An evaluation of the data and development of a proposed national guideline. Disaster Medicine and Public Health Preparedness,2(S1), S25–S34.CrossRefGoogle Scholar
  39. Levy, M., Koch, R. W., & Royne, M. B. (2013). Self-reported training needs of emergency responders in disasters requiring military interface. Journal of Emergency Management,11(2), 143–150.CrossRefGoogle Scholar
  40. Louwerse, M. M., Graesser, A. C., Lu, S., & Mitchell, H. H. (2005). Social cues in animated conversational agents. Applied Cognitive Psychology,19(6), 693–704.CrossRefGoogle Scholar
  41. Massaguer, D., Balasubramanian, V., Mehrotra, S., & Venkatasubramanian, N. (2006, May). Synthetic humans in emergency response drills. In Proceedings of the fifth international joint conference on Autonomous agents and multiagent systems (pp. 1469–1470). ACM.Google Scholar
  42. McCall, C., Blascovich, J., Young, A., & Persky, S. (2009). Proxemic behaviors as predictors of aggression towards Black (but not White) males in an immersive virtual environment. Social Influence,4(1), 138–154.CrossRefGoogle Scholar
  43. Metcalfe, J., & Kornell, N. (2007). Principles of cognitive science in education: The effects of generation, errors and feedback. Psychonomic Bulletin & Review,14, 225–229.CrossRefGoogle Scholar
  44. Nass, C., & Moon, Y. (2000). Machines and mindlessness: Social responses to computers. Journal of Social Issues,56(1), 81–103.CrossRefGoogle Scholar
  45. Oren, M., Carlson, P., Gilbert, S., & Vance, J. M. (2012). Puzzle assembly training: Real world vs. virtual environment. Virtual Reality Short Papers and Posters (VRW) (pp. 27–30). IEEE.Google Scholar
  46. Pashler, H., Cepeda, N. J., Wixted, J. T., & Rohrer, D. (2005). When does feedback facilitate learning of words? Journal of Experimental Psychology. Learning, Memory, and Cognition,31, 3–8.CrossRefGoogle Scholar
  47. Patterson, R., Pierce, B., Bell, H. H., Andrews, D., & Winterbottom, M. (2009). Training robust decision making in immersive environments. Journal of Cognitive Engineering and Decision Making,3(4), 331–361.CrossRefGoogle Scholar
  48. Peck, T. C., Seinfeld, S., Aglioti, S. M., & Slater, M. (2013). Putting yourself in the skin of a black avatar reduces implicit racial bias. Consciousness and Cognition,22, 779–787.CrossRefGoogle Scholar
  49. Peterson, M. (2005). Learning interaction in an avatar-based virtual environment: A preliminary study. PacCALL Journal,1, 29–40.Google Scholar
  50. Pucher, P. H., Batrick, N., Taylor, D., Chaudery, M., Cohen, D., & Darzi, A. (2014). Virtual-world hospital simulation for real-world disaster response: Design and validation of a virtual reality simulator for mass casualty incident management. Journal of Trauma and Acute Care Surgery,77(2), 315–321.CrossRefGoogle Scholar
  51. Reeves, B., & Nass, C. (1996). The Media Equation: How people treat computers, television, and new media like real people and places. New York, NY: Cambridge University Press.Google Scholar
  52. Rose, F. D., Attree, E. A., Brooks, B. M., Parslow, D. M., & Penn, P. R. (2000). Training in virtual environments: Transfer to real world tasks and equivalence to real task training. Ergonomics,43(4), 494–511.CrossRefGoogle Scholar
  53. Rudman, L. A., Ashmore, R. D., & Gary, M. L. (2001). “Unlearning” automatic biases: The malleability of implicit prejudice and stereotypes. Journal of Personality and Social Psychology,81(5), 856–868.CrossRefGoogle Scholar
  54. Sagar, H. A., & Schofield, J. W. (1980). Racial and behavioral cues in black and white children’s perceptions of ambiguously aggressive acts. Journal of Personality and Social Psychology,39, 590–598.CrossRefGoogle Scholar
  55. Saucier, D. A., Miller, C. T., & Doucet, N. (2005). Differences in helping whites and blacks: A meta-analysis. Personality and Social Psychology Review,9(1), 2–16.CrossRefGoogle Scholar
  56. Schroeder, N. L., Adesope, O. O., & Gilbert, R. (2013). How effective are pedagogical agents for learning? A meta-analytic review. Journal of Educational Computing Research.,49(1), 1–39.CrossRefGoogle Scholar
  57. Schroeder, N. L., & Gotch, C. M. (2015). Persisting issues in pedagogical agent research. Journal of Educational Computing Research,53(2), 183–204.CrossRefGoogle Scholar
  58. Schroeder, N., Romine, W., & Craig, S. D. (2017). Measuring pedagogical agent persona and the influence of agent persona on learning. Computers & Education,109, 176–186.CrossRefGoogle Scholar
  59. Shubeck, K., Craig, S. D., Hu, X., Faghihi, U. Levy, M., & Koch, R. (2012). Incorporating natural language tutoring into a virtual world for emergency response training. In P. M. McCarthy & G. M. Youngblood (Eds.), Proceedings of the 25th International Florida Artificial Intelligence Research Society (FLAIRS) Conference (p. 573). Menlo Park, CA: The AAAI Press.Google Scholar
  60. Shubeck, K. T., Craig, S. D., & Hu, X. (2016). Live-action mass-casualty training and virtual world training: A comparison. In Proceedings of the Human Factors & Ergonomics Society Annual Meeting (pp. 2103–2107). Los Angeles: SAGE.Google Scholar
  61. Sottilare, R. A., Long, R. A., & Goldberg, B. S. (2017, April). Enhancing the Experience Application Program Interface (xAPI) to Improve Domain Competency Modeling for Adaptive Instruction. In Proceedings of the Fourth (2017) ACM Conference on Learning@ Scale (pp. 265–268). ACM.Google Scholar
  62. Steele, C. M., & Aronson, J. (1995). Stereotype threat and the intellectual test performance of African Americans. Journal of Personality and Social Psychology,69, 797–811.CrossRefGoogle Scholar
  63. Stepanikova, I. (2012). Racial-ethnic biases, time pressures, and medical decisions. Journal of Health and Social Behavior,53(3), 329–343.CrossRefGoogle Scholar
  64. Sullins, J., Craig, S. D., & Hu, X. (2015). Exploring the effectiveness of a novel feedback mechanism within an intelligent tutoring system. International Journal of Learning Technology,10, 220–236.CrossRefGoogle Scholar
  65. Toth, E. E. (2016). Analyzing “real-world” anomalous data after experimentation with a virtual laboratory. Educational Technology Research and Development,64(1), 157–173.CrossRefGoogle Scholar
  66. Triona, L. M., & Klahr, D. (2003). Point and click or grab and heft: Comparing the influence of physical and virtual instructional materials on elementary school students’ ability to design experiments. Cognition and Instruction,21(2), 149–173.CrossRefGoogle Scholar
  67. Twyford, J., & Craig, S. D. (2017). Modeling goal setting within a multimedia environment on complex physics content. Journal of Educational Computing Research,55(3), 374–394.CrossRefGoogle Scholar
  68. von der Pütten, A. M., Krämer, N. C., Gratch, J., & Kang, S. H. (2010). “It doesn’t matter what you are!” Explaining social effects of agents and avatars. Computers in Human Behavior,26(6), 1641–1650.CrossRefGoogle Scholar
  69. Wandner, L. D., Heft, M. W., Lok, B. C., Hirsh, A. T., George, S. Z., Horgas, A. L., et al. (2014a). The impact of patients’ gender, race, and age on health care professionals’ pain management decisions: An online survey using virtual human technology. International Journal of Nursing Studies,51(5), 726–733.CrossRefGoogle Scholar
  70. Wandner, L. D., Letzen, J. E., Torres, C. A., Lok, B., & Robinson, M. E. (2014b). Using virtual human technology to provide immediate feedback about participants′ use of demographic cues and knowledge of their cue use. The Journal of Pain,15(11), 1141–1147.CrossRefGoogle Scholar
  71. Yee, N., & Bailenson, J. (2007). The proteus effect: The effect of transformed self-representation on behavior. Human Communication Research,33, 271–290.CrossRefGoogle Scholar

Copyright information

© Association for Educational Communications and Technology 2019

Authors and Affiliations

  1. 1.Human Systems EngineeringArizona State UniversityMesaUSA

Personalised recommendations