Abstract
It has been demonstrated that virtual prototypes can be used productively in usability testing. Virtual prototypes, Augmented Reality (AR) in particular, has received some focus as it combines both the real and virtual worlds and because platforms capable of supporting AR have become more ubiquitous.
This paper aims to explore the challenges of using AR to conduct usability testing in a remote delivery format. It presents a use case examination of the use of an AR tool to gather usability assessment data of a design prototype in a remote (distanced) format. An interview was conducted with those involved in conducting the test to understand issues related to the utilization of hardware provided by the usability test taker; the use of screen sharing to communicate instructions; challenges to efficiently gathering accurate observational data; and issues related to variability in hardware setup, consistency of the AR experience, and other environmental factors.
The results suggest that an approach for utilizing AR for remote usability testing was feasible, but many challenges and frustrations exist. Some of these related to communication with the participants, lack of motivation, difficulties extrapolating the prototype to a real setting, and technical issues. Usability test participants were often more interested in simply completing assigned tasks rather than exploring the tool or product in more detail that often happens in person. Other observations included some limitations in AR technology itself such as level of immersion (ability to accurately visualize the product). Challenges due to internet connectivity between the facilitator and tester at time affected the ability to effectively direct the testing session. Future work should investigate new tools and approaches for mitigating current challenges and experimentally compare in person versus remote testing outcomes.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Bruseberg, A., McDonagh-Philip, D.: Focus groups to support the industrial/product designer: a review based on current literature and designers’ feedback. Appl. Ergon. 33(1), 27–38 (2002)
Moultrie, J., Clarkson, P.J., Probert, D.: Development of a design audit tool for SMEs. J. Prod. Innov. Manag. 24(4), 335–368 (2007)
Creusen, M.E.: Research opportunities related to consumer response to product design. J. Prod. Innov. Manag. 28, 405–408 (2011)
Lewis, J.R.: Usability testing. In: Salvendy, G. (ed.) Handbook of Human Factors and Ergonomics, pp. 1275–1316 (2006)
ISO: ISO 9241-11. Ergonomic Requirements for Office Work with Visual Display Terminals (VDTs)-Part 11, Guidance on Usability (1998)
Srinivasan, V., Lovejoy, W.S., Beach, D.: Sharing user experiences in the product innovation process: Participatory design needs participatory communication. Creat. Innov. Manag. 16, 35–45 (1997)
van Krevelen, D.W.F., Poelman, R.: A survey of augmented reality technologies, applications and limitations. Int. J. Virtual Reality 9(2), 1–20 (2010)
Peddie, J.: Types of Augmented Reality. In: Peddie, J. (ed.) Augmented Reality, pp. 29–46. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-54502-8_2
Folstad, A., Law, E.L.-C., Hornbaek, K.: Analysis in usability evaluations: an exploratory study. In: Proceedings of the 6th Nordic Conference on Human-Computer Interaction: Extending Boundaries, pp. 647–650 (2010). https://doi.org/10.1145/1868914.1868995
McFadden, E., Hager, D.R., Elie, C.J., Blackwell, J.M.: Remote usability evaluation: overview and case studies. Int. J. Hum.-Comput. Interact. 14(3–4), 489–502 (2002). https://doi.org/10.1080/10447318.2002.9669131
Duh, H.B.-L., Tan, G.C.B., Chen, V.H.: Usability evaluation for mobile device: a comparison of laboratory and field tests. In: Proceedings of the 8th Conference on Human-Computer Interaction with Mobile Devices and Services, pp. 181–186 (2006). https://doi.org/10.1145/1152215.1152254
Martin, R., AlShamari, M., Seliaman, M.E., Mayhew, P.: Remote asynchronous testing: a cost-effective alternative for website usability evaluation. Int. J. Comput. Inf. Technol. 03(January), 2279–764 (2014)
Jain, K.: Investigating the efficiency of authoring interactions for augmented reality experience for designers (unpublished master’s thesis). Georgia Institute of Technology, Georgia, United States (2020)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Kim, T., Arconada-Alvarez, S., Choi, Y.M. (2021). Challenges and Workarounds of Conducting Augmented Reality Usability Tests Remotely a Case Study. In: Stephanidis, C., et al. HCI International 2021 - Late Breaking Papers: Design and User Experience. HCII 2021. Lecture Notes in Computer Science(), vol 13094. Springer, Cham. https://doi.org/10.1007/978-3-030-90238-4_6
Download citation
DOI: https://doi.org/10.1007/978-3-030-90238-4_6
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-90237-7
Online ISBN: 978-3-030-90238-4
eBook Packages: Computer ScienceComputer Science (R0)