A Low-Cost Full Body Tracking System in Virtual Reality Based on Microsoft Kinect

  • Nicola Capece
  • Ugo ErraEmail author
  • Giuseppe Romaniello
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10851)


We present an approach based on a natural user interface and virtual reality that allows the user’s body to be visualized and tracked inside a virtual environment. Our aim is to improve the sensation of virtual reality immersion through low-cost technology such as HTC Vive and Microsoft Kinect 2. The system has been developed using the Unity 3D game engine and C# language. Our approach has been validated through the implementation of an application for 3D mesh painting where the user is able to interact through hand gestures to select a color from the 3D color palette, rotate the 3D mesh and paint it.


Mesh painting Virtual reality Natural user interface Human computer interaction 


  1. 1.
    Adam Humanoid 3D model.
  2. 2.
  3. 3.
    Anthes, C., Garca-Hernndez, R.J., Wiedemann, M., Kranzlmller, D.: State of the art of virtual reality technology. In: Proceedings of the 2016 IEEE Aerospace Conference, pp. 1–19, March 2016Google Scholar
  4. 4.
    Biocca, F., Delaney, B.: Immersive Virtual Reality Technology. Communication in the Age of Virtual Reality, pp. 57–124 (1995)Google Scholar
  5. 5.
    Brancati, N., Caggianese, G., Frucci, M., Gallo, L., Neroni, P.: Experiencing touchless interaction with augmented content on wearable head-mounted displays in cultural heritage applications. Pers. Ubiquit. Comput. 21(2), 203–217 (2017)CrossRefGoogle Scholar
  6. 6.
    Capece, N., Erra, U., Romano, S., Scanniello, G.: Visualising a software system as a city through virtual reality. In: De Paolis, L.T., Bourdot, P., Mongelli, A. (eds.) AVR 2017. LNCS, vol. 10325, pp. 319–327. Springer, Cham (2017). Scholar
  7. 7.
    Ceylan, D., Li, W., Mitra, N.J., Agrawala, M., Pauly, M.: Designing and fabricating mechanical automata from mocap sequences. ACM Trans. Graph. 32(6), 186:1–186:11 (2013)CrossRefGoogle Scholar
  8. 8.
    Desai, P.R., Desai, P.N., Ajmera, K.D., Mehta, K.: A review paper on oculus rift-a virtual reality headsetGoogle Scholar
  9. 9.
    Erra, U., Scanniello, G., Capece, N.: Visualizing the evolution of software systems using the forest metaphor. In: 2012 16th International Conference on Information Visualisation, pp. 87–92, July 2012Google Scholar
  10. 10.
    Erra, U., Colonnese, V.: Experiences in the development of an augmented reality dressing room. In: De Paolis, L.T., Mongelli, A. (eds.) AVR 2015. LNCS, vol. 9254, pp. 467–474. Springer, Cham (2015). Scholar
  11. 11.
    Erra, U., Malandrino, D., Pepe, L.: A methodological evaluation of natural user interfaces for immersive 3D graph explorations. J. Vis. Lang. Comput. 44, 13–27 (2018)CrossRefGoogle Scholar
  12. 12.
    Erra, U., Scanniello, G.: Towards the visualization of software systems as 3D forests: the CodeTrees environment. In: Proceedings of the 27th Annual ACM Symposium on Applied Computing, pp. 981–988. SAC 2012. ACM, New York (2012)Google Scholar
  13. 13.
    Filkov, R.: Kinect v2 examples with MS-SDK.
  14. 14.
    Greuter, S., Roberts, D.J.: Spacewalk: movement and interaction in virtual space with commodity hardware. In: Proceedings of the 2014 Conference on Interactive Entertainment, pp. 1–7. ACM (2014)Google Scholar
  15. 15.
    Kaushik, D., Jain, R., et al.: Natural user interfaces: trend in virtual interaction. arXiv preprint arXiv:1405.0101 (2014)
  16. 16.
    Moeslund, T.B., Granum, E.: A survey of computer vision-based human motion capture. Comput. Vis. Image Underst. 81(3), 231–268 (2001)CrossRefGoogle Scholar
  17. 17.
    Moeslund, T.B., Hilton, A., Krüger, V.: A survey of advances in vision-based human motion capture and analysis. Comput. Vis. Image Underst. 104(2–3), 90–126 (2006)CrossRefGoogle Scholar
  18. 18.
    Niehorster, D.C., Li, L., Lappe, M.: The accuracy and precision of position and orientation tracking in the HTC vive virtual reality system for scientific research. i-Perception 8(3), 2041669517708205 (2017)Google Scholar
  19. 19.
    Patsadu, O., Nukoolkit, C., Watanapa, B.: Human gesture recognition using kinect camera. In: 2012 International Joint Conference on Computer Science and Software Engineering (JCSSE), pp. 28–32. IEEE (2012)Google Scholar
  20. 20.
    De la Peña, N., Weil, P., Llobera, J., Giannopoulos, E., Pomés, A., Spanlang, B., Friedman, D., Sanchez-Vives, M.V., Slater, M.: Immersive journalism: immersive virtual reality for the first-person experience of news. Presence Teleoperators Virtual Environ. 19(4), 291–301 (2010)CrossRefGoogle Scholar
  21. 21.
    Preis, J., Kessel, M., Werner, M., Linnhoff-Popien, C.: Gait recognition with kinect. In: 1st International Workshop on Kinect in Pervasive Computing, pp. 1–4. New Castle, UK (2012)Google Scholar
  22. 22.
    Razzaq, A., Wu, Z., Zhou, M., Ali, S., Iqbal, K.: Automatic conversion of human mesh into skeleton animation by using kinect motion. Int. J. Comput. Theory Eng. 7(6), 482 (2015)CrossRefGoogle Scholar
  23. 23.
    Ren, Z., Yuan, J., Meng, J., Zhang, Z.: Robust part-based hand gesture recognition using kinect sensor. IEEE Trans. Multimed. 15(5), 1110–1120 (2013)CrossRefGoogle Scholar
  24. 24.
    Ritschel, T., Botsch, M., Müller, S.: Multiresolution GPU mesh painting. In: Eurographics (Short Presentations), pp. 17–20 (2006)Google Scholar
  25. 25.
    Seibert, J., Shafer, D.M.: Control mapping in virtual reality: effects on spatial presence and controller naturalness. Virtual Real. 22(1), 79–88 (2018)CrossRefGoogle Scholar
  26. 26.
    Sell, J., O’Connor, P.: The xbox one system on a chip and kinect sensor. IEEE Micro 34(2), 44–53 (2014)CrossRefGoogle Scholar
  27. 27.
    Siriborvornratanakul, T.: A study of virtual reality headsets and physiological extension possibilities. In: Gervasi, O., et al. (eds.) ICCSA 2016. LNCS, vol. 9787, pp. 497–508. Springer, Cham (2016). Scholar
  28. 28.
    Slater, M., Linakis, V., Usoh, M., Kooper, R., Street, G.: Immersion, presence, and performance in virtual environments: an experiment with tri-dimensional chess. In: ACM Virtual Reality Software and Technology (VRST), vol. 163, p. 72. ACM Press, New York (1996)Google Scholar
  29. 29.
    Small, D.E.: Immersive virtual reality. Technical report, Sandia National Laboratories (SNL-NM), Albuquerque, NM, United States (2011)Google Scholar
  30. 30.
    Tang, M.: Recognizing hand gestures with microsofts kinect. Department of Electrical Engineering of Stanford University:[sn], Palo Alto (2011)Google Scholar
  31. 31.
    Villaroman, N., Rowe, D., Swan, B.: Teaching natural user interaction using OpenNI and the Microsoft Kinect sensor. In: Proceedings of the 2011 Conference on Information Technology Education, SIGITE 2011, pp. 227–232. ACM, New York (2011)Google Scholar
  32. 32.
    Yeo, H.S., Lee, B.G., Lim, H.: Hand tracking and gesture recognition system for human-computer interaction using low-cost hardware. Multimed. Tools Appl. 74(8), 2687–2715 (2015)CrossRefGoogle Scholar
  33. 33.
    Zennaro, S., Munaro, M., Milani, S., Zanuttigh, P., Bernardi, A., Ghidoni, S., Menegatti, E.: Performance evaluation of the 1st and 2nd generation kinect for multimedia applications. In: 2015 IEEE International Conference on Multimedia and Expo (ICME), pp. 1–6, June 2015Google Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Dipartimento di Matematica, Informatica ed EconomiaUniversità degli Studi della BasilicataPotenzaItaly

Personalised recommendations