Skip to main content
SpringerLink
Log in

Character Input in Augmented Reality: An Evaluation of Keyboard Position and Interaction Visualisation for Head-Mounted Displays

  • Conference paper
  • First Online:
Human-Computer Interaction – INTERACT 2021 (INTERACT 2021)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 12932))

Included in the following conference series:

Abstract

Character input in immersive environments is a non trivial task that has attracted much attention in recent years. This paper presents an evaluation of keyboard position, orientation and interaction together with the influence of visual interaction feedback in a controlled character input task with 27 participants in Augmented Reality (AR). It presents 5 different keyboard locations (3 bounded to the headset and 2 bounded to the non-dominant hand of the user) and 3 visual interaction feedback methods (finger raycast, fingertip glow and both combined). Objective (completion time, accuracy, Key per Minute (KPM)) and subjective (After Scenario Questionnaire (ASQ)) metrics are presented. Results showed that keyboard placement had an effect on accuracy, KPM metrics and subjective preference, with keyboard visualisation parallel and bounded to the headset position and orientation outperforming other keyboard locations.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 119.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 159.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://developer.leapmotion.com/, (Last accessed 7th November 2020).

  2. 2.

    https://docs.microsoft.com/en-us/windows/mixed-reality/design/billboarding-and-tag-along, (Last accessed 14th January 2021).

  3. 3.

    https://docs.microsoft.com/en-us/windows/mixed-reality/design/hand-menu, (Last accessed 14th January 2020).

  4. 4.

    https://docs.microsoft.com/en-us/windows/mixed-reality/design/direct-manipulationinteraction, (Last accessed 7th November 2020).

References

  1. Ahn, S., Lee, G.: Gaze-assisted typing for smart glasses. In: Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology, pp. 857–869 (2019)

    Google Scholar 

  2. Bi, X., Smith, B.A., Zhai, S.: Quasi-qwerty soft keyboard optimization. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 283–286 (2010)

    Google Scholar 

  3. Boletsis, C., Kongsvik, S.: Text input in virtual reality: a preliminary evaluation of the drum-like vr keyboard. Technologies 7(2), 31 (2019)

    Google Scholar 

  4. Bovet, S., et al.: Using traditional keyboards in vr: steamvr developer kit and pilot game user study. In: 2018 IEEE Games, Entertainment, Media Conference (GEM), pp. 1–9. IEEE (2018)

    Google Scholar 

  5. Dang, N.-T., Mestre, D.: Effects of menu orientation on pointing behavior in virtual environments. In: Shumaker, R. (ed.) VMR 2011. LNCS, vol. 6773, pp. 144–153. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-22021-0_17

    Chapter  Google Scholar 

  6. Dube, T.J., Arif, A.S.: Text entry in virtual reality: a comprehensive review of the literature. In: Kurosu, M. (ed.) HCII 2019. LNCS, vol. 11567, pp. 419–437. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-22643-5_33

    Chapter  Google Scholar 

  7. Dudley, J., Benko, H., Wigdor, D., Kristensson, P.O.: Performance envelopes of virtual keyboard text input strategies in virtual reality. In: 2019 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), pp. 289–300. IEEE (2019)

    Google Scholar 

  8. Dudley, J.J., Vertanen, K., Kristensson, P.O.: Fast and precise touch-based text entry for head-mounted augmented reality with variable occlusion. ACM Trans. Comput.-Hum. Interact. (TOCHI) 25(6), 1–40 (2018)

    Google Scholar 

  9. Fashimpaur, J., Kin, K., Longest, M.: Pinchtype: text entry for virtual and augmented reality using comfortable thumb to fingertip pinches. In: Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems Extended Abstracts, pp. 1–7 (2020)

    Google Scholar 

  10. Friedman, M.: A comparison of alternative tests of significance for the problem of m rankings. Ann. Math. Stat. 11(1), 86–92 (1940)

    Google Scholar 

  11. Frutos-Pascual, M., Harrison, J.M., Creed, C., Williams, I.: Evaluation of ultrasound haptics as a supplementary feedback cue for grasping in virtual environments. In: 2019 International Conference on Multimodal Interaction, pp. 310–318 (2019)

    Google Scholar 

  12. Grossman, T., Chen, X.A., Fitzmaurice, G.: Typing on glasses: adapting text entry to smart eyewear. In: Proceedings of the 17th International Conference on Human-Computer Interaction with Mobile Devices and Services, pp. 144–152 (2015)

    Google Scholar 

  13. Grubert, J., Witzani, L., Ofek, E., Pahud, M., Kranz, M., Kristensson, P.O.: Effects of hand representations for typing in virtual reality. In: 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), pp. 151–158. IEEE (2018)

    Google Scholar 

  14. Grubert, J., Witzani, L., Ofek, E., Pahud, M., Kranz, M., Kristensson, P.O.: Text entry in immersive head-mounted display-based virtual reality using standard keyboards. In: 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), pp. 159–166. IEEE (2018)

    Google Scholar 

  15. Gupta, A., Ji, C., Yeo, H.S., Quigley, A., Vogel, D.: Rotoswype: word-gesture typing using a ring. In: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, pp. 1–12 (2019)

    Google Scholar 

  16. Gupta, A., Samad, M., Kin, K., Kristensson, P.O., Benko, H.: Investigating remote tactile feedback for mid-air text-entry in virtual reality. In: 2020 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), pp. 350–360. IEEE (2020)

    Google Scholar 

  17. Hincapié-Ramos, J.D., Guo, X., Irani, P.: The consumed endurance workbench: a tool to assess arm fatigue during mid-air interactions. In: Proceedings of the 2014 Companion Publication on Designing Interactive Systems, pp. 109–112 (2014)

    Google Scholar 

  18. Hsieh, Y.T., Jylhä, A., Orso, V., Gamberini, L., Jacucci, G.: Designing a willing-to-use-in-public hand gestural interaction technique for smart glasses. In: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, pp. 4203–4215 (2016)

    Google Scholar 

  19. Jiang, H., Weng, D.: Hipad: text entry for head-mounted displays using circular touchpad. In: 2020 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), pp. 692–703. IEEE (2020)

    Google Scholar 

  20. Kim, J., Delamare, W., Irani, P.: Thumbtext: text entry for wearable devices using a miniature ring. In: Proceedings of Graphics Interface, pp. 18–25 (2018)

    Google Scholar 

  21. Knierim, P., Schwind, V., Feit, A.M., Nieuwenhuizen, F., Henze, N.: Physical keyboards in virtual reality: analysis of typing performance and effects of avatar hands. In: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, pp. 1–9 (2018)

    Google Scholar 

  22. Lee, L.H., Braud, T., Lam, K.Y., Yau, Y.P., Hui, P.: From seen to unseen: designing keyboard-less interfaces for text entry on the constrained screen real estate of augmented reality headsets. Pervasive Mob. Comput. 64, 101148 (2020)

    Google Scholar 

  23. Lewis, J.R.: IBM computer usability satisfaction questionnaires: psychometric evaluation and instructions for use. Int. J. Hum.-Comput. Interact. 7(1), 57–78 (1995)

    Google Scholar 

  24. McGill, M., Boland, D., Murray-Smith, R., Brewster, S.: A dose of reality: overcoming usability challenges in vr head-mounted displays. In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, pp. 2143–2152 (2015)

    Google Scholar 

  25. Mcgill, M., Kehoe, A., Freeman, E., Brewster, S.: Expanding the bounds of seated virtual workspaces. ACM Trans. Comput.-Hum. Interact. (TOCHI) 27(3), 1–40 (2020)

    Google Scholar 

  26. Menzner, T., Otte, A., Gesslein, T., Grubert, J., Gagel, P., Schneider, D.: A capacitive-sensing physical keyboard for vr text entry. In: 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), pp. 1080–1081. IEEE (2019)

    Google Scholar 

  27. Ogitani, T., Arahori, Y., Shinyama, Y., Gondow, K.: Space saving text input method for head mounted display with virtual 12-key keyboard. In: 2018 IEEE 32nd International Conference on Advanced Information Networking and Applications (AINA), pp. 342–349. IEEE (2018)

    Google Scholar 

  28. Otte, A., Menzner, T., Gesslein, T., Gagel, P., Schneider, D., Grubert, J.: Towards utilizing touch-sensitive physical keyboards for text entry in virtual reality. In: 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), pp. 1729–1732. IEEE (2019)

    Google Scholar 

  29. Otte, A., Schneider, D., Menzner, T., Gesslein, T., Gagel, P., Grubert, J.: Evaluating text entry in virtual reality using a touch-sensitive physical keyboard. In: 2019 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct), pp. 387–392. IEEE (2019)

    Google Scholar 

  30. Pham, D.M., Stuerzlinger, W.: Hawkey: efficient and versatile text entry for virtual reality. In: 25th ACM Symposium on Virtual Reality Software and Technology, pp. 1–11 (2019)

    Google Scholar 

  31. Pick, S., Puika, A.S., Kuhlen, T.W.: Swifter: design and evaluation of a speech-based text input metaphor for immersive virtual environments. In: 2016 IEEE Symposium on 3D User Interfaces (3DUI), pp. 109–112. IEEE (2016)

    Google Scholar 

  32. Pietroszek, K.: Raycasting in Virtual Reality, pp. 1–3. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-08234-9_180-1

  33. Rajanna, V., Hansen, J.P.: Gaze typing in virtual reality: impact of keyboard design, selection method, and motion. In: Proceedings of the 2018 ACM Symposium on Eye Tracking Research and Applications, pp. 1–10 (2018)

    Google Scholar 

  34. Schneider, D., et al.: Reconviguration: reconfiguring physical keyboards in virtual reality. IEEE Trans. Visual. Comput. Graph. 25(11), 3190–3201 (2019)

    Google Scholar 

  35. Shapiro, S.S., Wilk, M.B.: An analysis of variance test for normality (complete samples). Biometrika 52(3/4), 591–611 (1965)

    Google Scholar 

  36. Speicher, M., Feit, A.M., Ziegler, P., Krüger, A.: Selection-based text entry in virtual reality. In: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, pp. 1–13 (2018)

    Google Scholar 

  37. Walker, J., Li, B., Vertanen, K., Kuhl, S.: Efficient typing on a visually occluded physical keyboard. In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, pp. 5457–5461 (2017)

    Google Scholar 

  38. Wang, C.Y., Chu, W.C., Chiu, P.T., Hsiu, M.C., Chiang, Y.H., Chen, M.Y.: Palmtype: using palms as keyboards for smart glasses. In: Proceedings of the 17th International Conference on Human-Computer Interaction with Mobile Devices and Services, pp. 153–160 (2015)

    Google Scholar 

  39. Wu, C.M., Hsu, C.W., Lee, T.K., Smith, S.: A virtual reality keyboard with realistic haptic feedback in a fully immersive virtual environment. Virtual Reality 21(1), 19–29 (2017)

    Google Scholar 

  40. Xu, W., Liang, H.N., He, A., Wang, Z.: Pointing and selection methods for text entry in augmented reality head mounted displays. In: 2019 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), pp. 279–288. IEEE (2019)

    Google Scholar 

  41. Yang, Z., Chen, C., Lin, Y., Wang, D., Li, H., Xu, W.: Effect of spatial enhancement technology on input through the keyboard in virtual reality environment. Appl. Ergon. 78, 164–175 (2019)

    Google Scholar 

  42. Yi, X., Yu, C., Zhang, M., Gao, S., Sun, K., Shi, Y.: Atk: enabling ten-finger freehand typing in air based on 3D hand tracking data. In: Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology, pp. 539–548 (2015)

    Google Scholar 

  43. Yu, C., Sun, K., Zhong, M., Li, X., Zhao, P., Shi, Y.: One-dimensional handwriting: inputting letters and words on smart glasses. In: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, pp. 71–82 (2016)

    Google Scholar 

  44. Yu, D., Fan, K., Zhang, H., Monteiro, D., Xu, W., Liang, H.N.: Pizzatext: text entry for virtual reality systems using dual thumbsticks. IEEE Trans. Visual. Comput. Graph. 24(11), 2927–2935 (2018)

    Google Scholar 

  45. Yumura, T., Nakamura, S.: Augmented typing: augmentation of keyboard typing experience by adding visual and sound effects. In: Asian CHI Symposium 2019 (2019)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. DMT Lab, School of Computing and Digital Technology, Birmingham City University, Birmingham, UK

    Maite Frutos-Pascual, Clara Gale, Jake M. Harrison, Chris Creed & Ian Williams

Authors
  1. Maite Frutos-Pascual
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Clara Gale
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jake M. Harrison
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Chris Creed
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ian Williams
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Maite Frutos-Pascual .

Editor information

Editors and Affiliations

  1. Department of Electrical and Information Engineering, Polytechnic University of Bari, Bari, Italy

    Carmelo Ardito

  2. Computer Science Department, University of Bari Aldo Moro, Bari, Italy

    Rosa Lanzilotti

  3. Computer Science Department, University of Pisa, Pisa, Italy

    Alessio Malizia

  4. Department of Computer Science, University of York, York, UK

    Helen Petrie

  5. Computer Science Department, University of Bari Aldo Moro, Bari, Italy

    Antonio Piccinno

  6. Computer Science Department, University of Bari Aldo Moro, Bari, Italy

    Giuseppe Desolda

  7. Microsoft Research, Redmond, WA, USA

    Kori Inkpen

Rights and permissions

Reprints and permissions

Copyright information

© 2021 IFIP International Federation for Information Processing

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Frutos-Pascual, M., Gale, C., Harrison, J.M., Creed, C., Williams, I. (2021). Character Input in Augmented Reality: An Evaluation of Keyboard Position and Interaction Visualisation for Head-Mounted Displays. In: Ardito, C., et al. Human-Computer Interaction – INTERACT 2021. INTERACT 2021. Lecture Notes in Computer Science(), vol 12932. Springer, Cham. https://doi.org/10.1007/978-3-030-85623-6_29

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-85623-6_29

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-85622-9

  • Online ISBN: 978-3-030-85623-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Societies and partnerships

Search

Navigation