User-Centered Evaluation of a Virtual Environment Training System: Utility of User Perception Measures

  • Dawei Jia
  • Asim Bhatti
  • Chris Mawson
  • Saeid Nahavandi
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5622)


This study assessed the utility of measures of Self-efficacy (SelfEfficacy) and Perceived VE efficacy (PVEefficacy) for quantifying how effective VEs are in procedural task training. SelfEfficacy and PVEefficacy have been identified as affective construct potentially underlying VE efficacy that is not evident from user task performance. The motivation for this study is to establish subjective measures of VE efficacy and investigate the relationship between PVEefficacy, SelfEfficacy and User task performance. Results demonstrated different levels of prior experience in manipulating 3D objects in gaming or computer environment (LOE3D) effects on task performance and user perception of VE efficacy. Regression analysis revealed LOE3D, SelfEfficacy, PVEefficacy explain significant portions of the variance in VE efficacy. Results of the study provide further evidence that task performance may share relationships with PVEefficacy and SelfEfficacy, and that affective constructs, such as PVEefficacy, and SelfEfficacy may serve as alternative, subjective measures of task performance that account for VE efficacy.


User-based evaluation Virtual Environment Evaluation methodology 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Bandura, A.: Self-efficacy: the Exercise of Control. W.H. Freeman, New York (1997)Google Scholar
  2. 2.
    Bouffard-Bouchard, T.: Influence of Self-efficacy on Performance in a Cognitive Task. Journal of Social Psychology 130, 153–164 (1990)CrossRefGoogle Scholar
  3. 3.
    Davis, F.D.: User Acceptance of Information Technology: System Characteristics, User perceptions and Behavioral impacts. International Journal of Man-Machine Studies 38, 475–487 (1993)CrossRefGoogle Scholar
  4. 4.
    DeVellis, R.F.: Scale Development: Theory and Applications, 2nd edn. Sage Publications, Thousand Oaks (2003)Google Scholar
  5. 5.
    Hill, T., Smith, N.D., Mann, M.F.: Role of Efficacy Expectations in Predicting the Decision to Use Advanced Technologies: the Case of Computers. Journal of Applied Psychology 72, 307–313 (1987)CrossRefGoogle Scholar
  6. 6.
    Hoffman, B., Schraw, G.: The influence of self-efficacy and working memory capacity on problem-solving efficiency. Journal of Learning and Individual differences 19(1), 91–100 (2009)CrossRefGoogle Scholar
  7. 7.
    Isdale, J.: Introduction to VRTechnology. In: IEEE Virtual Reality Conference, p. 302. IEEE Computer Society Press, Los Alamitos (2003)Google Scholar
  8. 8.
    Jia, D., Bhatti, A., Nahanandi, S.: Computer-simulated Environment (VE) for training: Challenge of Efficacy Evaluation. In: The Simulation Technology and Training Conference (SimTecT 2008), p. 63. Simulation Industry Association of Australia (2008)Google Scholar
  9. 9.
    Lin, J.J.W.: Virtual Guiding Avatar: an Effective Procedure to Reduce Simulator Sickness in Virtual Environments. In: SIGCHI Conference on Human Factors in Computing Systems, pp. 719–726. ACM Press, New York (2004)Google Scholar
  10. 10.
    Mantovani, F.: VR Learning: Potential and Challenges for the Use of 3D Environments in Education and Training. In: Giuseppe, R., Carlo, G. (eds.) Towards CyberPsychology: Mind, Cognitions and Society in the Internet Age, pp. 207–226. IOS Press, Amsterdam (2001)Google Scholar
  11. 11.
    Moreno, R.: Does the Modality Principle Hold for Different Media? A Test of the Method-Affects-Learning Hypothesis. Journal of Computer Assisted Learning 22, 149–158 (2006)CrossRefGoogle Scholar
  12. 12.
    Nash, E.B., Edwards, G.W., Thompson, J.A., Barfield, W.: A Review of Presence and Performance in Virtual Environments. International Journal of Human-Computer Interaction 12(1), 1–41 (2000)CrossRefGoogle Scholar
  13. 13.
    North, M.M., Mathis, J.R., Madajewski, A., Brown, J.T., Cupp, S.M.: Virtual Reality and Transfer of Learning. In: Michael, J.S., Richard, J.K., Gavriel, S., Don, H. (eds.) Usability Evaluation and Interface Design: Cognitive Engineering, Intelligent Agents and Virtual Reality, vol. 1, pp. 634–638. Lawrence Erlbaum Associates, London (2001)Google Scholar
  14. 14.
    Papasratorn, B., Wangpipatwong, T.: The Effects of Self-Efficacy and Attitude on E-learning Outcomes. In: World Conference on E-Learning in Corporate, Government, Healthcare, & Higher Education, pp. 226–2270. Association for the Advancement of Computing in Education (AACE) (2006)Google Scholar
  15. 15.
    Riley, J.M., Kaber, D.B., Draper, J.V.: Situation Awareness and Attention Allocation Measures for Quantifying Telepresence Experiences in Teleoperation. Journal of Human Factors and Ergonomics in Manufacturing 14, 51–67 (2004)CrossRefGoogle Scholar
  16. 16.
    Salzman, M.C., Dede, C., Loftin, R.B., Chen, J.: A Model for Understanding How Virtual Reality Aids Complex Conceptual Learning. Presence: Teleoperators and Virtual Environments, Special Issue on Education 8(3), 293–316 (1999)CrossRefGoogle Scholar
  17. 17.
    Stanney, K.M.: Handbook of Virtual Environments Design, Implementation, and Applications. Lawrence Erlbaum Associates, London (2002)Google Scholar
  18. 18.
    Tesfazgi, S.H.: Survey on Behavioral Observation Methods in Virtual Environments. Research Assignment, Delft University of Technology (2003)Google Scholar
  19. 19.
    Wang, A.Y., Newlin, M.H.: Predictors of Web-student Performance: the Role of Self-efficacy and Reasons or Taking an On-line Class. Journal of Computers in Human Behavior 18, 151–163 (2002)CrossRefGoogle Scholar
  20. 20.
    Witmer, B.G., Singer, M.J.: Measuring Presence in Virtual Environments: A Presence Questionnaire. Journal of Presence: Teleoperators and Virtual Environments 7(3), 225–240 (1998)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Dawei Jia
    • 1
  • Asim Bhatti
    • 1
  • Chris Mawson
    • 1
  • Saeid Nahavandi
    • 1
  1. 1.School of Engineering and Information Technology; Centre for Intelligent System ResearchDeakin UniversityAustralia

Personalised recommendations