Advertisement

Virtual Reality

, Volume 23, Issue 2, pp 155–168 | Cite as

Real-time body tracking in virtual reality using a Vive tracker

  • Polona CasermanEmail author
  • Augusto Garcia-Agundez
  • Robert Konrad
  • Stefan Göbel
  • Ralf Steinmetz
Original Article

Abstract

Due to recent improvements in virtual reality (VR) technology, the number of novel applications for entertainment, education, and rehabilitation has increased. The primary goal of these applications is to enhance the sense of belief that the user is “present” in the virtual environment. By tracking the user’s skeleton in real-time, it is possible to synchronize the avatar’s motions with the user’s motions. Although current common devices implement body tracking to a certain degree, most approaches are limited by either high latency or insufficient accuracy. Due to the lack of positional and rotation data, the current VR applications typically do not represent the user’s motions. In this paper, we present an accurate, low-latency body tracking approach for VR-based applications using Vive Trackers. Using a HTC Vive headset and Vive Trackers, we have been able to create an immersive VR experience, by animating the motions of the avatar as smoothly, rapidly and as accurately as possible. An evaluation showed our solution is capable of tracking both joint rotation and position with reasonable accuracy and a very low end-to-latency of \(6.71 \pm 0.80\hbox { ms}\). Due to this merely imperceptible delay and precise tracking, our solution can show the movements of the user in real-time in order to create deeper immersion.

Keywords

Virtual reality Real-time tracking Full-body avatar Low-latency HTC Vive tracker Inverse kinematics 

Notes

Supplementary material

Supplementary material 1 (mp4 31171 KB)

References

  1. Aristidou A, Lasenby J (2011) FABRIK: a fast, iterative solver for the inverse kinematics problem. Graph Models 73(5):243–260CrossRefGoogle Scholar
  2. Banakou D, Groten R, Slater M (2013) Illusory ownership of a virtual child body causes overestimation of object sizes and implicit attitude changes. Proc Natl Acad Sci 110(31):12846–12851CrossRefGoogle Scholar
  3. Bolton J, Lambert M, Lirette D, Unsworth B (2014) PaperDude: a virtual reality cycling exergame. CHI’14 Extended Abstracts on Human Factors in Computing Systems. CHI EA’14. ACM, New York, NY, USA, pp 475–478Google Scholar
  4. Botev J, Rothkugel S (2017) High-precision gestural input for immersive large-scale distributed virtual environments. In: Proceedings of the 9th workshop on massively multiuser virtual environments, MMVE’17. ACM, New York, NY, USA, pp 7–11Google Scholar
  5. Caserman P, Krabbe P, Wojtusch J, von Stryk O (2016) Real-time step detection using the integrated sensors of a head-mounted display. In: 2016 IEEE international conference on systems, man, and cybernetics (SMC), pp 3510–3515Google Scholar
  6. Chan JCP, Leung H, Tang JKT, Komura T (2011) A virtual reality dance training system using motion capture technology. IEEE Trans Learn Technol 4(2):187–195CrossRefGoogle Scholar
  7. Choi SW, Seo MW, Lee SL, Park JH, Oh EY, Baek JS, Kang SJ (2016) Head position model-based latency measurement system for virtual reality head mounted display. SID Symp Dig Tech Papers 47(1):1381–1384CrossRefGoogle Scholar
  8. Collingwoode-Williams T, Gillies M, McCall C, Pan X (2017) The effect of lip and arm synchronization on embodiment: a pilot study. In: 2017 IEEE virtual reality (VR). IEEE, pp 253–254Google Scholar
  9. Dempsey P (2016) The teardown: HTC Vive VR headset. Eng Technol 11(7–8):80–81Google Scholar
  10. Desai PR, Desai PN, Ajmera KD, Mehta K (2014) A review paper on oculus rift—a virtual reality headset. Int J Eng Trends Technol (IJETT) 13(4):175–179CrossRefGoogle Scholar
  11. Desai K, Raghuraman S, Jin R, Prabhakaran B (2017) QoE studies on interactive 3D tele-immersion. In: 2017 IEEE international symposium on multimedia (ISM), pp 130–137Google Scholar
  12. Farahani N, Post R, Duboy J, Ahmed I, Kolowitz BJ, Krinchai T, Monaco SE, Fine JL, Hartman DJ, Pantanowitz L (2016) Exploring virtual reality technology and the oculus rift for the examination of digital pathology slides. J Pathol Inform 7:22CrossRefGoogle Scholar
  13. Friðriksson FA, Kristjánsson HS, Sigurðsson DA, Thue D, Vilhjálmsson HH (2016) Become your avatar: fast skeletal reconstruction from sparse data for fully-tracked VR. In: Proceedings of the 26th international conference on artificial reality and telexistence and the 21st Eurographics symposium on virtual environments: posters and demos, pp 19–20Google Scholar
  14. Friston S, Steed A (2014) Measuring latency in virtual environments. IEEE Trans Vis Comput Graph 20(4):616–625CrossRefGoogle Scholar
  15. Galna B, Barry G, Jackson D, Mhiripiri D, Olivier P, Rochester L (2014) Accuracy of the microsoft kinect sensor for measuring movement in people with Parkinson’s disease. Gait Posture 39(4):1062–1068CrossRefGoogle Scholar
  16. Goradia I, Doshi J, Kurup L (2014) A review paper on oculus rift & project morpheus. Int J Curr Eng Technol 4(5):3196–3200Google Scholar
  17. Grochow K, Martin SL, Hertzmann A, Popović Z (2004) Style-based inverse kinematics. ACM Trans Graph 23(3):522–531CrossRefGoogle Scholar
  18. Huang J, Wang Q, Fratarcangeli M, Yan K, Pelachaud C (2017) Multi-variate gaussian-based inverse kinematics. Comput Graph Forum 36(8):418–428CrossRefGoogle Scholar
  19. Jain D, Sra M, Guo J, Marques R, Wu R, Chiu J, Schmandt C (2016) Immersive terrestrial scuba diving using virtual reality. In: Proceedings of the 2016 CHI conference extended abstracts on human factors in computing systems. ACM, New York, USA, pp 1563–1569Google Scholar
  20. Jiang F, Yang X, Feng L (2016) Real-time full-body motion reconstruction and recognition for off-the-shelf VR devices. In: Proceedings of the 15th ACM SIGGRAPH conference on virtual-reality continuum and its applications in industry—Volume 1, VRCAI’16. ACM, pp 309–318Google Scholar
  21. Johnson M, Humer I, Zimmerman B, Shallow J, Tahai L, Pietroszek K (2016) Low-cost latency compensation in motion tracking for smartphone-based head mounted display. In: Proceedings of the international working conference on advanced visual interfaces, AVI’16. ACM, New York, NY, USA, pp 316–317Google Scholar
  22. Kasahara S, Konno K, Owaki R, Nishi T, Takeshita A, Ito T, Kasuga S, Ushiba J (2017) Malleable embodiment: changing sense of embodiment by spatial-temporal deformation of virtual human body. In: Proceedings of the 2017 CHI conference on human factors in computing systems, CHI’17. ACM, New York, NY, USA, pp 6438–6448Google Scholar
  23. Kavan L, Sloan PP, O’Sullivan C (2010) Fast and efficient skinning of animated meshes. Comput Graph Forum 29(2):327–336CrossRefGoogle Scholar
  24. Kenwright B (2012) Real-time character inverse kinematics using the Gauss–Seidel iterative approximation method. Int Conf Creat Content Technol 4:63–68Google Scholar
  25. Lange B, Rizzo S, Chang CY, Suma EA, Bolas M (2011) Markerless full body tracking: depth-sensing technology within virtual environments. In: Interservice/industry training, simulation, and education conference (I/ITSEC)Google Scholar
  26. Latoschik ME, Lugrin JL, Habel M, Roth D, Seufert C, Grafe S (2016) Breaking bad behavior: immersive training of class room management. In: Proceedings of the 22nd ACM conference on virtual reality software and technology, VRST’16. ACM, New York, NY, USA, pp 317–318Google Scholar
  27. Latoschik ME, Roth D, Gall D, Achenbach J, Waltemate T, Botsch M (2017) The effect of avatar realism in immersive social virtual realities. In: Proceedings of the 23rd ACM symposium on virtual reality software and technology, VRST’17. ACM, New York, NY, USA, pp 39:1–39:10Google Scholar
  28. Martindale J (2018) Oculus Rift vs. HTC Vive. https://www.digitaltrends.com/virtual-reality/oculus-rift-vs-htc-vive/. Accessed 4 May 2017​
  29. Melo M, Rocha T, Barbosa L, Bessa M (2016) The impact of body position on the usability of multisensory virtual environments: case study of a virtual bicycle. In: Proceedings of the 7th international conference on software development and technologies for enhancing accessibility and fighting info-exclusion, DSAI 2016. ACM, New York, NY, USA, pp 20–24Google Scholar
  30. Nakamura Y, Hanafusa H (1986) Inverse kinematic solutions with singularity robustness for robot manipulator control. J Dyn Syst Meas Control 108(3):163–171CrossRefzbMATHGoogle Scholar
  31. Orin DE, Schrader WW (1984) Efficient computation of the Jacobian for robot manipulators. Int J Robot Res 3(4):66–75CrossRefGoogle Scholar
  32. Peck TC, Seinfeld S, Aglioti SM, Slater M (2013) Putting yourself in the skin of a black avatar reduces implicit racial bias. Conscious Cognit 22(3):779–787CrossRefGoogle Scholar
  33. Raaen K, Kjellmo I (2015) Measuring latency in virtual reality systems. In: Chorianopoulos K, Divitini M, Baalsrud Hauge J, Jaccheri L, Malaka R (eds) Entertainment computing—ICEC 2015. Springer, Cham, pp 457–462Google Scholar
  34. Roberts D, Duckworth T, Moore C, Wolff R, O’Hare J (2009) Comparing the end to end latency of an immersive collaborative environment and a video conference. In: Proceedings of the 2009 13th IEEE/ACM international symposium on distributed simulation and real time applications, DS-RT’09. IEEE Computer Society, Washington, DC, USA, pp 89–94Google Scholar
  35. Schmidt D, Kovacs R, Mehta V, Umapathi U, Köhler S, Cheng LP, Baudisch P (2015) Level-ups: motorized stilts that simulate stair steps in virtual reality. In: Proceedings of the 33rd annual ACM conference extended abstracts on human factors in computing systems, CHI EA’15. ACM, New York, NY, USA, pp 359–362Google Scholar
  36. Seele S, Misztal S, Buhler H, Herpers R, Schild J (2017) Here’s looking at you anyway!: how important is realistic gaze behavior in co-located social virtual reality games? In: Proceedings of the annual symposium on computer-human interaction in play, CHI PLAY’17. ACM, New York, NY, USA, pp 531–540Google Scholar
  37. Shoemake K (1985) Animating rotation with quaternion curves. In: Proceedings of the 12th annual conference on computer graphics and interactive techniques, SIGGRAPH’85. ACM, New York, NY, USA, pp 245–254Google Scholar
  38. Shum H, Ho ES (2012) Real-time physical modelling of character movements with microsoft kinect. In: Proceedings of the 18th ACM symposium on virtual reality software and technology, VRST’12. ACM, pp 17–24Google Scholar
  39. Sra M, Schmandt C (2015) MetaSpace II: object and full-body tracking for interaction and navigation in social VR. CoRR abs/1512.02922Google Scholar
  40. Steed A (2008) A simple method for estimating the latency of interactive, real-time graphics simulations. In: Proceedings of the 2008 ACM symposium on virtual reality software and technology, VRST’08. ACM, New York, NY, USA, pp 123–129Google Scholar
  41. Tao G, Archambault PS, Levin MF (2013) Evaluation of kinect skeletal tracking in a virtual reality rehabilitation system for upper limb hemiparesis. In: 2013 international conference on virtual rehabilitation (ICVR), pp 164–165Google Scholar
  42. Thomas JS, France CR, Leitkam ST, Applegate ME, Pidcoe PE, Walkowski S (2016) Effects of real-world versus virtual environments on joint excursions in full-body reaching tasks. IEEE J Transl Eng Health Med 4:1–8CrossRefGoogle Scholar
  43. Tsai TC, Chen CY, Su GJ (2015) U-art: your art and ubiquitous art. In: Adjunct proceedings of the 2015 ACM international joint conference on pervasive and ubiquitous computing and proceedings of the 2015 ACM international symposium on wearable computers, UbiComp/ISWC’15 Adjunct. ACM, New York, NY, USA, pp 1295–1302Google Scholar

Copyright information

© Springer-Verlag London Ltd., part of Springer Nature 2018

Authors and Affiliations

  1. 1.Multimedia Communications LabTechnische Universität DarmstadtDarmstadtGermany

Personalised recommendations