Task-Driven Biometric Authentication of Users in Virtual Reality (VR) Environments

  • Alexander Kupin
  • Benjamin Moeller
  • Yijun Jiang
  • Natasha Kholgade Banerjee
  • Sean BanerjeeEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11295)


In this paper, we provide an approach for authenticating users in virtual reality (VR) environments by tracking the behavior of users as they perform goal-oriented tasks, such as throwing a ball at a target. With the pervasion of VR in mission-critical applications such as manufacturing, navigation, military training, education, and therapy, validating the identity of users using VR systems is becoming paramount to prevent tampering of the VR environments, and to ensure user safety. Unlike prior work, which uses PIN and pattern based passwords to authenticate users in VR environments, our approach authenticates users based on their natural interactions within the virtual space by matching the 3D trajectory of the dominant hand gesture controller in a display-based head-mounted VR system to a library of trajectories. To handle natural differences in wait times between multiple parts of an action such as picking a ball and throwing it, our matching approach uses a symmetric sum-squared distance between the nearest neighbors across the query and library trajectories. Our work enables seamless authentication without requiring the user to stop their activity and enter specific credentials, and can be used to continually validate the identity of the user. We conduct a pilot study with 14 subjects throwing a ball at a target in VR using the gesture controller and achieve a maximum accuracy of 92.86% by comparing to a library of 10 trajectories per subject, and 90.00% by comparing to 6 trajectories per subject.



This work was partially supported by the National Science Foundation (NSF) grant #1730183. We acknowledge the support of NVIDIA Corporation with the donation of the Titan Xp GPU used for this research.


  1. 1.
    Ahmed, F., Paul, P.P., Gavrilova, M.L.: DTW-based kernel and rank-level fusion for 3D gait recognition using kinect. Vis. Comput. 31, 915–924 (2015)CrossRefGoogle Scholar
  2. 2.
    Andersson, V., Dutra, R., Araújo, R.: Anthropometric and human gait identification using skeleton data from kinect sensor. In: ACM SAP (2014)Google Scholar
  3. 3.
    Andriotis, P., Tryfonas, T., Oikonomou, G.: Complexity metrics and user strength perceptions of the pattern-lock graphical authentication method. In: Tryfonas, T., Askoxylakis, I. (eds.) HAS 2014. LNCS, vol. 8533, pp. 115–126. Springer, Cham (2014). Scholar
  4. 4.
    Berg, L.P., Vance, J.M.: Industry use of virtual reality in product design and manufacturing: a survey. Virtual Reality 21, 1–17 (2017)CrossRefGoogle Scholar
  5. 5.
    Bhagat, K.K., Liou, W.K., Chang, C.Y.: A cost-effective interactive 3D virtual reality system applied to military live firing training. Virtual Reality 20, 127–140 (2016)CrossRefGoogle Scholar
  6. 6.
    Choi, S., Jung, K., Noh, S.D.: Virtual reality applications in manufacturing industries: past research, present findings, and future directions. Concurrent Eng. 23, 40–63 (2015)CrossRefGoogle Scholar
  7. 7.
    Feng, T., et al.: Continuous mobile authentication using touchscreen gestures. In: IEEE HST (2012)Google Scholar
  8. 8.
    Frank, J., Mannor, S., Precup, D.: Activity and gait recognition with time-delay embeddings. In: AAAI (2010)Google Scholar
  9. 9.
    Freina, L., Ott, M.: A literature review on immersive virtual reality in education: state of the art and perspectives. In: The International Scientific Conference eLearning and Software for Education (2015)Google Scholar
  10. 10.
    George, C., et al.: Seamless and secure VR: Adapting and evaluating established authentication systems for virtual reality. In: NDSS (2017)Google Scholar
  11. 11.
    Haque, A., Alahi, A., Fei-Fei, L.: Recurrent attention models for depth-based person identification. In: IEEE CVPR (2016)Google Scholar
  12. 12.
    John, V., Englebienne, G., Krose, B.: Person re-identification using height-based gait in colour depth camera. In: IEEE ICIP (2013)Google Scholar
  13. 13.
    Kwapisz, J.R., Weiss, G.M., Moore, S.A.: Cell phone-based biometric identification. In: IEEE BTAS (2010)Google Scholar
  14. 14.
    Li, S., Ashok, A., Zhang, Y., Xu, C., Lindqvist, J., Gruteser, M.: Whose move is it anyway? authenticating smart wearable devices using unique head movement patterns. In: IEEE PerCom (2016)Google Scholar
  15. 15.
    Liu, J., Zhong, L., Wickramasuriya, J., Vasudevan, V.: uWave: accelerometer-based personalized gesture recognition and its applications. Pervasive Mobile Comput. 5(6), 657–675 (2009)CrossRefGoogle Scholar
  16. 16.
    Lohse, K.R., Hilderman, C.G., Cheung, K.L., Tatla, S., Van der Loos, H.M.: Virtual reality therapy for adults post-stroke: a systematic review and meta-analysis exploring virtual environments and commercial games in therapy. PloS one 9, e93318 (2014)CrossRefGoogle Scholar
  17. 17.
    Matsuo, K., Okumura, F., Hashimoto, M., Sakazawa, S., Hatori, Y.: Arm swing identification method with template update for long term stability. In: Lee, S.-W., Li, S.Z. (eds.) ICB 2007. LNCS, vol. 4642, pp. 211–221. Springer, Heidelberg (2007). Scholar
  18. 18.
    Mendels, O., Stern, H., Berman, S.: User identification for home entertainment based on free-air hand motion signatures. IEEE Trans. Syst., Man, Cybern. Syst. 44, 1461–1473 (2014)CrossRefGoogle Scholar
  19. 19.
    Merchant, Z., Goetz, E.T., Cifuentes, L., Keeney-Kennicutt, W., Davis, T.J.: Effectiveness of virtual reality-based instruction on students’ learning outcomes in K-12 and higher education: a meta-analysis. Comput. Educ. 70, 29–40 (2014)CrossRefGoogle Scholar
  20. 20.
    Monrose, F., Reiter, M.K., Wetzel, S.: Password hardening based on keystroke dynamics. Int. J. Inf. Secur. 1(2), 69–83 (2002)CrossRefGoogle Scholar
  21. 21.
    Monrose, F., Rubin, A.D.: Keystroke dynamics as a biometric for authentication. Future Gener. Comput. Syst. 16(4), 351–359 (2000)CrossRefGoogle Scholar
  22. 22.
    Muaaz, M., Mayrhofer, R.: Orientation independent cell phone based gait authentication. In: Proceedings of the 12th International Conference on Advances in Mobile Computing and Multimedia (2014)Google Scholar
  23. 23.
    Muaaz, M., Mayrhofer, R.: Smartphone-based gait recognition: from authentication to imitation. IEEE Trans. Mob. Comput. 16, 3209–3221 (2017)CrossRefGoogle Scholar
  24. 24.
    Munsell, B.C., Temlyakov, A., Qu, C., Wang, S.: Person identification using full-body motion and anthropometric biometrics from kinect videos. In: Fusiello, A., Murino, V., Cucchiara, R. (eds.) ECCV 2012. LNCS, vol. 7585, pp. 91–100. Springer, Heidelberg (2012). Scholar
  25. 25.
    North, M.M., North, S.M., Coble, J.R.: Virtual reality therapy: an effective treatment for the fear of public speaking. IJVR 3, 1–6 (2015)Google Scholar
  26. 26.
    Oberhauser, M., Dreyer, D.: A virtual reality flight simulator for human factors engineering. Cogn., Technol. Work 19, 263–277 (2017)CrossRefGoogle Scholar
  27. 27.
    Okumura, F., Kubota, A., Hatori, Y., Matsuo, K., Hashimoto, M., Koike, A.: A study on biometric authentication based on arm sweep action with acceleration sensor. In: ISPACS (2006)Google Scholar
  28. 28.
    Pallavicini, F., Toniazzi, N., Argenton, L., Aceti, L., Mantovani, F.: Developing effective virtual reality training for military forces and emergency operators: from technology to human factors. In: International Conference on Modeling and Applied Simulation, MAS 2015 (2015)Google Scholar
  29. 29.
    Rogers, C.E., Witt, A.W., Solomon, A.D., Venkatasubramanian, K.K.: An approach for user identification for head-mounted displays. In: ACM ISWC (2015)Google Scholar
  30. 30.
    Schneegass, S., Oualil, Y., Bulling, A.: Skullconduct: biometric user identification on eyewear computers using bone conduction through the skull. In: ACM CHI (2016)Google Scholar
  31. 31.
    Serwadda, A., Phoha, V.V., Wang, Z.: Which verifiers work?: a benchmark evaluation of touch-based authentication algorithms. In: IEEE BTAS (2013)Google Scholar
  32. 32.
    Sprager, S., Zazula, D.: A cumulant-based method for gait identification using accelerometer data with principal component analysis and support vector machine. WSEAS Trans. Sig. Process. 5, 369–378 (2009)Google Scholar
  33. 33.
    Syed, Z., Banerjee, S., Cheng, Q., Cukic, B.: Effects of user habituation in keystroke dynamics on password security policy. In: IEEE HASE, pp. 352–359 (2011)Google Scholar
  34. 34.
    Syed, Z., Helmick, J., Banerjee, S., Cukic, B.: Effect of user posture and device size on the performance of touch-based authentication systems. In: IEEE HASE (2015)Google Scholar
  35. 35.
    Wiederhold, B.K., Wiederhold, M.D.: Virtual reality therapy for anxiety disorders: advances in evaluation and treatment. American Psychological Association, Worcester (2005)CrossRefGoogle Scholar
  36. 36.
    Wu, J., Konrad, J., Ishwar, P.: Dynamic time warping for gesture-based user identification and authentication with kinect. In: IEEE ICASSP, pp. 2371–2375 (2013)Google Scholar
  37. 37.
    Yu, Z., Liang, H.N., Fleming, C., Man, K.L.: An exploration of usable authentication mechanisms for virtual reality systems. In: IEEE APCCAS (2016)Google Scholar
  38. 38.
    Zhong, Y., Deng, Y., Meltzner, G.: Pace independent mobile gait biometrics. In: IEEE BTAS (2015)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Alexander Kupin
    • 1
  • Benjamin Moeller
    • 1
  • Yijun Jiang
    • 1
  • Natasha Kholgade Banerjee
    • 1
  • Sean Banerjee
    • 1
    Email author
  1. 1.Clarkson UniversityPotsdamUSA

Personalised recommendations