Using Cognitive Walkthrough and Hybrid Prototyping to Gather User Requirements in Early Design Virtual Reality Prototypes

  • Ioannis DoumanisEmail author
  • Daphne Economou
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 1044)


To evaluate Virtual Reality (VR) prototypes usability involves a variety of single-perspective or Hybrid methods. The latter has being suggested by literature as offering a more complete sets of requirements highlighting both ‘in-world’ and user interface problems. This paper describes our experiences in using a single-perspective method for gathering user requirements in the REVERIE (Real and Virtual Engagement In Realistic Immersive Environment) project. The study reports results involving nine evaluators who reviewed two hybrid VR prototypes with educational context. It was found that this approach was effective in highlighting a plethora of usability problems covering all aspects of the two VR prototypes. The performance of our approach was similar to the literature. Although additional validation work is required, we can conclude that our approach may provide a viable option to evaluate early design VR prototypes when required (e.g., when the expertise needed to use a hybrid method is not available). Future work aims to compare the performance of our approach with two-stage and multiple stage hybrid methods.


Usability methods Cognitive walkthrough Virtual worlds User interface Interaction 



The research that led to this paper was supported in part by the European Commission under the Contract FP7-ICT-287723 REVERIE.


  1. 1.
    Bellwood, P., Neuhaus, P., Juhra, C.: Adapting usability testing techniques to gather user requirements: an illustrative proposal. Stud. Health Technol. Inf. 164, 213–218 (2011)Google Scholar
  2. 2.
    Standardization, I.O.f.: Ergonomics of human-system interaction, Part 210: Human-Centred Design for Interactive Systems, ed, ISO 9241-210Google Scholar
  3. 3.
    Bowman, D.A., Gabbard, J.L., Hix, D.: A survey of usability evaluation in virtual environments: classification and comparison of methods, presence: teleoper. Virtual Environ. 11, 404–424 (2002)Google Scholar
  4. 4.
    Kaur, D.: A Usability Evaluation Method for Virtual Reality User Interfaces (2018)Google Scholar
  5. 5.
    Sawyerr, W., Brown, E., Hobbs, M.: Using a hybrid method to evaluate the usability of a 3D virtual world user interface. In: 2nd International Conference on Human Computer Interaction & Learning Technologies (ICHCILT 2013) Proceedings, Abu Dhabi (2013). last accessed 14 May 2019
  6. 6.
    Bremin, S.: Rapid evaluation of TV interaction devices using a Cognitive Walkthrough method (2011)Google Scholar
  7. 7.
    Fechteler, P., Hilsmann, A., Eisert, P., Broeck, S.V., Stevens, C., Wall, J.: A framework for realistic 3D tele-immersion. In: 6th International Conference on Computer Vision/Computer Graphics Collaboration Techniques and Applications Proceedings, Berlin, Germany (2013)Google Scholar
  8. 8.
    Wall, J., Izquierdo, E., Argyriou, L., Monaghan, D.S., Connor, N.E.O., Poulakos, S.: REVERIE: Natural human interaction in virtual immersive environments. In: 2014 IEEE International Conference on Image Processing (ICIP), pp. 2165–2167 (2014)Google Scholar
  9. 9.
    Sutcliffe, A., Kaur, D.K.: Evaluating the usability of virtual reality user interfaces, vol. 19 (2000)CrossRefGoogle Scholar
  10. 10.
    Rusu, C., Muñoz, R., Roncagliolo, S., Rudloff, S., Rusu, V., Figueroa, A.: Usability Heuristics for Virtual Worlds (2011)Google Scholar
  11. 11.
    Muñoz, R., Barcelos, T., Chalegre, V.: Defining and Validating Virtual Worlds Usability Heuristics. In: 30th International Conference of the Chilean Computer Science Society, pp. 171–178 (2011)Google Scholar
  12. 12.
    Alencar, M.F.C., Raposo, A.B., Barbosa, S.D.J.: Composition of HCI evaluation methods for hybrid virtual environments. In: 2011 ACM Symposium on Applied Computing, TaiChung, Taiwan (2011)Google Scholar
  13. 13.
    Gabbard, J., Hix, D.: A Taxonomy of Usability Characteristics in Virtual Environments, PhD thesis (1999)Google Scholar
  14. 14.
    Anders Ericsson, K., Herber, A.S.: Protocol Analysis: Verbal Reports as Data. MIT Press, Cambridge (2018)Google Scholar
  15. 15.
    Goodman, E., Kuniavsky, M., Moed, A.: Observing the user experience: a practitioner’s guide to user research (Second Edition). IEEE Trans. Prof. Commun. 56, 260–261 (2013)CrossRefGoogle Scholar
  16. 16.
    deSouza, C.S.: The Semiotic Engineering of Human-Computer Interaction (Acting with Technology). MIT Press, Cambridge (2005)CrossRefGoogle Scholar
  17. 17.
    Apostolakis K.C., Daras, P.: RAAT - The reverie avatar authoring tool. In: 18th International Conference on Digital Signal Processing (DSP), 1–6 (2013)Google Scholar
  18. 18.
    Kuijk, F., Apostolakis, K.C., Daras, P., Ravenet, B., Wei, H., Monaghan, D.S.: Autonomous agents and avatars in REVERIE’s virtual environment. In: Proceedings of the 20th International Conference on 3D Web Technology, Heraklion, Crete, Greece (2015)Google Scholar
  19. 19.
    Articulate 360, ed: Articulate Global Inc. (2017)Google Scholar
  20. 20.
    Vet, J.d., Alfonso, D.: D2.1 Initial version of User Requirements Specification, European Commision (2012)Google Scholar
  21. 21.
    CTVC, REVERIE Informal Usability Inspection. In: Education Innovation Conference and Exhibition (2014)Google Scholar
  22. 22.
    Calefato, F., Lanubile, F.: A Planning Poker Tool for Supporting Collaborative Estimation in Distributed Agile Development (2011)Google Scholar
  23. 23.
    Clegg, D., Barker, R.: Case Method Fast-Track: A Rad Approach. Addison-Wesley Longman Publishing Co., Inc., Boston (1994)Google Scholar
  24. 24.
    Hollingsed, T., Novick, D.G.: Usability inspection methods after 15 years of research and practice. In: the 25th Annual ACM International Conference on Design of Communication Proceedings, El Paso, Texas, USA (2007)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.University of Central LancashirePrestonUK
  2. 2.University of WestminsterLondonUK

Personalised recommendations