Abstract
To evaluate Virtual Reality (VR) prototypes usability involves a variety of single-perspective or Hybrid methods. The latter has being suggested by literature as offering a more complete sets of requirements highlighting both ‘in-world’ and user interface problems. This paper describes our experiences in using a single-perspective method for gathering user requirements in the REVERIE (Real and Virtual Engagement In Realistic Immersive Environment) project. The study reports results involving nine evaluators who reviewed two hybrid VR prototypes with educational context. It was found that this approach was effective in highlighting a plethora of usability problems covering all aspects of the two VR prototypes. The performance of our approach was similar to the literature. Although additional validation work is required, we can conclude that our approach may provide a viable option to evaluate early design VR prototypes when required (e.g., when the expertise needed to use a hybrid method is not available). Future work aims to compare the performance of our approach with two-stage and multiple stage hybrid methods.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bellwood, P., Neuhaus, P., Juhra, C.: Adapting usability testing techniques to gather user requirements: an illustrative proposal. Stud. Health Technol. Inf. 164, 213–218 (2011)
Standardization, I.O.f.: Ergonomics of human-system interaction, Part 210: Human-Centred Design for Interactive Systems, ed, ISO 9241-210
Bowman, D.A., Gabbard, J.L., Hix, D.: A survey of usability evaluation in virtual environments: classification and comparison of methods, presence: teleoper. Virtual Environ. 11, 404–424 (2002)
Kaur, D.: A Usability Evaluation Method for Virtual Reality User Interfaces (2018)
Sawyerr, W., Brown, E., Hobbs, M.: Using a hybrid method to evaluate the usability of a 3D virtual world user interface. In: 2nd International Conference on Human Computer Interaction & Learning Technologies (ICHCILT 2013) Proceedings, Abu Dhabi (2013). https://goo.gl/TTSbzC. last accessed 14 May 2019
Bremin, S.: Rapid evaluation of TV interaction devices using a Cognitive Walkthrough method (2011)
Fechteler, P., Hilsmann, A., Eisert, P., Broeck, S.V., Stevens, C., Wall, J.: A framework for realistic 3D tele-immersion. In: 6th International Conference on Computer Vision/Computer Graphics Collaboration Techniques and Applications Proceedings, Berlin, Germany (2013)
Wall, J., Izquierdo, E., Argyriou, L., Monaghan, D.S., Connor, N.E.O., Poulakos, S.: REVERIE: Natural human interaction in virtual immersive environments. In: 2014 IEEE International Conference on Image Processing (ICIP), pp. 2165–2167 (2014)
Sutcliffe, A., Kaur, D.K.: Evaluating the usability of virtual reality user interfaces, vol. 19 (2000)
Rusu, C., Muñoz, R., Roncagliolo, S., Rudloff, S., Rusu, V., Figueroa, A.: Usability Heuristics for Virtual Worlds (2011)
Muñoz, R., Barcelos, T., Chalegre, V.: Defining and Validating Virtual Worlds Usability Heuristics. In: 30th International Conference of the Chilean Computer Science Society, pp. 171–178 (2011)
Alencar, M.F.C., Raposo, A.B., Barbosa, S.D.J.: Composition of HCI evaluation methods for hybrid virtual environments. In: 2011 ACM Symposium on Applied Computing, TaiChung, Taiwan (2011)
Gabbard, J., Hix, D.: A Taxonomy of Usability Characteristics in Virtual Environments, PhD thesis (1999)
Anders Ericsson, K., Herber, A.S.: Protocol Analysis: Verbal Reports as Data. MIT Press, Cambridge (2018)
Goodman, E., Kuniavsky, M., Moed, A.: Observing the user experience: a practitioner’s guide to user research (Second Edition). IEEE Trans. Prof. Commun. 56, 260–261 (2013)
deSouza, C.S.: The Semiotic Engineering of Human-Computer Interaction (Acting with Technology). MIT Press, Cambridge (2005)
Apostolakis K.C., Daras, P.: RAAT - The reverie avatar authoring tool. In: 18th International Conference on Digital Signal Processing (DSP), 1–6 (2013)
Kuijk, F., Apostolakis, K.C., Daras, P., Ravenet, B., Wei, H., Monaghan, D.S.: Autonomous agents and avatars in REVERIE’s virtual environment. In: Proceedings of the 20th International Conference on 3D Web Technology, Heraklion, Crete, Greece (2015)
Articulate 360, ed: Articulate Global Inc. (2017)
Vet, J.d., Alfonso, D.: D2.1 Initial version of User Requirements Specification, European Commision (2012)
CTVC, REVERIE Informal Usability Inspection. In: Education Innovation Conference and Exhibition (2014)
Calefato, F., Lanubile, F.: A Planning Poker Tool for Supporting Collaborative Estimation in Distributed Agile Development (2011)
Clegg, D., Barker, R.: Case Method Fast-Track: A Rad Approach. Addison-Wesley Longman Publishing Co., Inc., Boston (1994)
Hollingsed, T., Novick, D.G.: Usability inspection methods after 15 years of research and practice. In: the 25th Annual ACM International Conference on Design of Communication Proceedings, El Paso, Texas, USA (2007)
Acknowledgements
The research that led to this paper was supported in part by the European Commission under the Contract FP7-ICT-287723 REVERIE.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Doumanis, I., Economou, D. (2019). Using Cognitive Walkthrough and Hybrid Prototyping to Gather User Requirements in Early Design Virtual Reality Prototypes. In: Beck, D., et al. Immersive Learning Research Network. iLRN 2019. Communications in Computer and Information Science, vol 1044. Springer, Cham. https://doi.org/10.1007/978-3-030-23089-0_18
Download citation
DOI: https://doi.org/10.1007/978-3-030-23089-0_18
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-23088-3
Online ISBN: 978-3-030-23089-0
eBook Packages: Computer ScienceComputer Science (R0)