A Usability and Workload Investigation: Using Video Games to Compare Two Virtual Reality Systems

  • Crystal S. Maraj
  • Jonathan HurterEmail author
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 973)


Comparison of the effectiveness and user perception of different commercial Virtual Reality (VR) systems for similar locomotion tasks is lacking. This gap is filled in the present paper by comparing two VR systems: the Oculus Rift and the HTC Vive. Three game scenarios of increasing elaboration served as the basis for comparative measures: objective performance and subjective questionnaires (i.e., usability and workload). The between-subjects differences showed the Rift had higher subjective ease of use; an ergonomic recommendation is thus given. The within-subjects differences showed scenario workload differences within both the Rift and Vive. A trend of nonsignificant change in workload between scenarios 1 and scenarios 2 suggests using scenario 1 as a tutorial; a significant change in workload between scenarios 2 and 3 might relate to a lack of competency, signal certainty, scaffolding, and boredom. A higher-workload scenario may be added after scenario 3 in future studies.


Virtual Reality Research game Usability Workload Head-Mounted Displays Locomotion 



This research was sponsored by Gino Fragomeni of the U.S. Army Research Laboratory Human Research Engineering Directorate Advanced Training and Simulation Division (ARL HRED ATSD), under contract W911QX-13-C-0052. However, the views, findings, and conclusions contained in this presentation are solely those of the author and should not be interpreted as representing the official policies, either expressed or implied, of ARL HRED ATSD or the U.S. Government. The U.S. Government is authorized to reproduce and distribute reprints for Government.


  1. 1.
    Milgram, P., Kishino, A.F.: Taxonomy of mixed reality visual displays. IEICE Trans. Inf. Syst. E77-D(12), 1321–1329 (1994)Google Scholar
  2. 2.
    Kwon, C.: Verification of the possibility and effectiveness of experiential learning using HMD-based immersive VR technologies. Virtual Real. 1–18 (2018)Google Scholar
  3. 3.
    Linn, C., Bender, S., Prosser, J., Schmitt, K., Werth, D.: Virtual remote inspection—a new concept for virtual reality enhanced real-time maintenance. In: 2017 23rd International Conference on Virtual System & Multimedia (VSMM), pp. 1–6. IEEE (2017)Google Scholar
  4. 4.
    Millais, P., Jones, S.L., Kelly, R.: Exploring data in virtual reality: comparisons with 2D data visualizations. In: Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems, p. LBW007. ACM (2018)Google Scholar
  5. 5.
    Rose, T., Nam, C.S., Chen, K.B.: Immersion of virtual reality for rehabilitation-review. Appl. Ergon. 69, 153–161 (2018)CrossRefGoogle Scholar
  6. 6.
  7. 7.
    Lum, H.C., Greatbatch, R., Waldfogle, G., Benedict, J.: How immersion, presence, emotion, & workload differ in virtual reality and traditional game mediums. In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting, vol. 62, no. 1, pp. 1474–1478. Sage Publications, Los Angeles (2018)Google Scholar
  8. 8.
    Martel, E., Su, F., Gerroir, J., Hassan, A., Girouard, A., Muldner, K.: Diving head-first into virtual reality: evaluating HMD control schemes for VR games. In: Proceedings of the 10th International Conference on the Foundations of Digital Games. (2015)Google Scholar
  9. 9.
    Frommel, J., Sonntag, S., Weber, M.: Effects of controller-based locomotion on player experience in a virtual reality exploration game. In: Proceedings of the 12th International Conference on the Foundations of Digital Games. ACM (2017)Google Scholar
  10. 10.
    Bozgeyikli, E., Raij, A., Katkoori, S., Dubey, R.: Point & teleport locomotion technique for virtual reality. In: Proceedings of the 2016 Annual Symposium on Computer-Human Interaction in Play, pp. 205–216. ACM (2016)Google Scholar
  11. 11.
    Monteiro, D., Liang, H.N., Xu, W., Brucker, M., Nanjappan, V., Yue, Y.: Evaluating enjoyment, presence, and emulator sickness in VR games based on first-and third-person viewing perspectives. Comput. Animat. Virtual. Worlds 29(3–4), 1–12 (2018)Google Scholar
  12. 12.
    Tcha-Tokey, K., Loup-Escande, E., Christmann, O., Richir, S.: Effects of interaction level, framerate, field of view, 3D content feedback, previous experience on subjective user experience and objective usability in immersive virtual environment. Int. J. Virtual Real. 17(3), 27–51 (2017)Google Scholar
  13. 13.
    Papachristos, N.M., Vrellis, I., Mikropoulos, T.A.: A comparison between Oculus Rift and a low-cost smartphone VR headset: immersive user experience and learning. In: 2017 IEEE 17th International Conference on Advanced Learning Technologies (ICALT), pp. 477–481. IEEE (2017)Google Scholar
  14. 14.
    Rupp, M.A., Odette, K.L., Kozachuk, J., Michaelis, J.R., Smither, J.A., McConnell, D.S.: Investigating learning outcomes and subjective experiences in 360-degree videos. Comput. Educ. 128, 256–268 (2019)CrossRefGoogle Scholar
  15. 15.
    Somrak, A., Humar, I., Hossain, M.S., Alhamid, M.F., Hossain, M.A., Guna, J.: Estimating VR sickness and user experience using different HMD technologies: an evaluation study. Future Gener. Comput. Syst. 94, 302–316 (2019)CrossRefGoogle Scholar
  16. 16.
    Wienrich, C., Döllinger, N., Kock, S., Schindler, K., Traupe, O.: Assessing user experience in virtual reality–a comparison of different measurements. In: International Conference of Design, User Experience, and Usability, pp. 573–589. Springer, Cham (2018)CrossRefGoogle Scholar
  17. 17.
    Dumas, J.S., Redish, J.: A Practical Guide to Usability Testing. Intellect books (1999)Google Scholar
  18. 18.
    Nielsen, J.: Usability Engineering. Elsevier (1994)Google Scholar
  19. 19.
    Organisation Internationale de Normalisation (ISO),
  20. 20.
    Riccio, A., Leotta, F., Bianchi, L., Aloise, F., Zickler, C., Hoogerwerf, E.J., Kübler, A., Mattia, D., Cincotti, F.: Workload measurement in a communication application operated through a P300-based brain–computer interface. J. Neural Eng. 8(2), 1–6 (2011)CrossRefGoogle Scholar
  21. 21.
    Hart, S.G., Staveland, L.E.: Development of NASA-TLX (Task Load Index): results of empirical and theoretical research. In: Hancock, P.A., Meshkati, N. (eds.) Human Mental Workload, pp. 139–183. Elsevier Science, Amsterdam (1988)CrossRefGoogle Scholar
  22. 22.
    Hart, S.G., Wickens, C.D.: Workload assessment and prediction. In: Booher, H.R. (ed.) Manprint, pp. 257–296. Springer, Dordrecht (1990)CrossRefGoogle Scholar
  23. 23.
    Card, S.K., Newell, A., Moran, T.P.: The Psychology of Human-Computer Interaction. Erlbaum Associates, Hillsdale (1983)Google Scholar
  24. 24.
    Wickens, C.D., McCarley, J.S.: Applied Attention Theory. CRC, Boca Raton (2008)Google Scholar
  25. 25.
    Wickens, C.D.: Multiple resources and mental workload. Hum. Factors 50(3), 449–455 (2008)CrossRefGoogle Scholar
  26. 26.
    Sweller, J., Ayres, P., Kalyuga, S.: Cognitive Load Theory. Springer, New York (2011)CrossRefGoogle Scholar
  27. 27.
    Cohen, J.W.: Statistical Power Analysis for the Behavioral Sciences, 2nd edn. Lawrence Erlbaum Associates, Hillsdale (1988)Google Scholar
  28. 28.
    Przybylski, A.K., Deci, E.L., Rigby, C.S., Ryan, R.M.: Competence-impeding electronic games and players’ aggressive feelings, thoughts, and behaviors. J. Pers. Soc. Psychol. 106(3), 441–499 (2014)CrossRefGoogle Scholar
  29. 29.
    Parasuraman, R., Galster, S., Squire, P., Furukawa, H., Miller, C.: A flexible delegation-type interface enhances system performance in human supervision of multiple robots: empirical studies with RoboFlag. IEEE Trans. Syst. Man Cybern. Part A: Syst. Humans 35(4), 481–493 (2005)CrossRefGoogle Scholar
  30. 30.
    Sharek, D., Wiebe, E.: Using flow theory to design video games as experimental stimuli. In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting, vol. 55, no. 1, pp. 1520–1524. SAGE Publications, Los Angeles (2011)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.Institute for Simulation and Training, University of Central FloridaOrlandoUSA

Personalised recommendations