Skip to main content
SpringerLink
Log in

Real-time body tracking in virtual reality using a Vive tracker

  • Original Article
  • Published:
Virtual Reality Aims and scope Submit manuscript

Abstract

Due to recent improvements in virtual reality (VR) technology, the number of novel applications for entertainment, education, and rehabilitation has increased. The primary goal of these applications is to enhance the sense of belief that the user is “present” in the virtual environment. By tracking the user’s skeleton in real-time, it is possible to synchronize the avatar’s motions with the user’s motions. Although current common devices implement body tracking to a certain degree, most approaches are limited by either high latency or insufficient accuracy. Due to the lack of positional and rotation data, the current VR applications typically do not represent the user’s motions. In this paper, we present an accurate, low-latency body tracking approach for VR-based applications using Vive Trackers. Using a HTC Vive headset and Vive Trackers, we have been able to create an immersive VR experience, by animating the motions of the avatar as smoothly, rapidly and as accurately as possible. An evaluation showed our solution is capable of tracking both joint rotation and position with reasonable accuracy and a very low end-to-latency of \(6.71 \pm 0.80\hbox { ms}\). Due to this merely imperceptible delay and precise tracking, our solution can show the movements of the user in real-time in order to create deeper immersion.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Notes

  1. Vive Tracker: https://www.vive.com/us/vive-tracker/, last visited on April 3rd, 2018.

  2. IKinema Orion: https://ikinema.com/orion, last visited on April 3rd, 2018.

  3. Vive IK Demo: https://github.com/JamesBear/vive_ik_demo, last visited on April 3rd, 2018.

  4. Final IK: https://assetstore.unity.com/packages/tools/animation/final-ik-14290, last visited on April 3rd, 2018.

  5. Body Tracking Demo: https://github.com/CatCuddler/BodyTracking, last visited on April 4th, 2018.

  6. Microsoft Kinect: https://developer.microsoft.com/en-us/windows/kinect, last visited on January 28th, 2018.

  7. Nintendo Wii: https://www.nintendo.co.uk/Wii/Wii-94559.html, last visited on January 28th, 2018.

  8. OptiTrack system: http://www.optitrack.com, last visited on January 17th, 2018.

  9. PrioVR: https://yostlabs.com/priovr/, last visited on July 31st, 2018.

  10. Perception Neuron: https://neuronmocap.com, last visited on July 31st, 2018.

  11. Xsense: https://www.xsens.com/, last visited on 31st July, 2018.

  12. MakeHuman: http://www.makehuman.org, last visited on February 3rd, 2018.

  13. Kore: https://github.com/Kode/Kore, last visited on April 3rd, 2018.

  14. OpenGEX: http://opengex.org, last visited on February 21st, 2018.

  15. KCF Tracker: http://docs.opencv.org/trunk/d2/dff/classcv_1_1TrackerKCF.html, last visited on February 17th, 2017.

  16. Minimum requirements: https://www.vive.com/us/ready/, last visited on February 5th, 2018.

  17. Question: “I find the VR in general exciting”, five-level Likert scale, \(N=13\), \({\hbox {AVR}}=4.92\), \({\hbox {SD}}=\pm \,0.27\), question: “I like the idea of body tracking in VR”, five-level Likert scale, \(N=13\), \({\hbox {AVR}}=4.92\), \({\hbox {SD}}=\pm \,0.27\).

  18. Question: “I felt like I was a part of the VR”, five-level Likert scale, \(N=13\), \({\hbox {AVR}}=4.3\), \({\hbox {SD}}=\pm \,0.48\), question: “I could identify myself with the avatar”, five-level Likert scale, \(N=13\), \({\hbox {AVR}}=4.07\), \({\hbox {SD}}=\pm \,0.49\).

  19. Question: “The movements in the VR have corresponded to the real movements”, five-level Likert scale, \(N=13\), \({\hbox {AVR}}=4.23\), \({\hbox {SD}}=\pm \,0.59\), question: “The tracking was accurate”, five-level Likert scale, \(N=13\), \({\hbox {AVR}}=4.07\), \({\hbox {SD}}=\pm \,0.64\).

  20. Question: “The movements of the avatar were delayed”, five-level Likert scale, \(N=13\), \({\hbox {AVR}}=1.23\), \({\hbox {SD}}=\pm \,0.43\).

  21. Question: “The tracking had some jitter problems”, five-level Likert scale, \(N=13\), \({\hbox {AVR}}=2.37\), \({\hbox {SD}}=\pm \,1.25\).

  22. Question: “I would like body tracking also in other VR games”, five-level Likert scale, \(N=13\), \({\hbox {AVR}}=4.92\), \({\hbox {SD}}=\pm \,0.27\).

  23. Leap Motion: https://www.leapmotion.com, last visited on January 19th, 2018.

  24. Hi5 VR Glove: https://hi5vrglove.com, last visited on January 19th, 2018.

  25. VRgluv: https://vrgluv.com, last visited on January 19th, 2018.

  26. HaptX: https://haptx.com, last visited on January 19th, 2018.

  27. VRtouch: https://www.gotouchvr.com/order_vrtouch/, last visited on January 19th, 2018.

References

  • Aristidou A, Lasenby J (2011) FABRIK: a fast, iterative solver for the inverse kinematics problem. Graph Models 73(5):243–260

    Article  Google Scholar 

  • Banakou D, Groten R, Slater M (2013) Illusory ownership of a virtual child body causes overestimation of object sizes and implicit attitude changes. Proc Natl Acad Sci 110(31):12846–12851

    Article  Google Scholar 

  • Bolton J, Lambert M, Lirette D, Unsworth B (2014) PaperDude: a virtual reality cycling exergame. CHI’14 Extended Abstracts on Human Factors in Computing Systems. CHI EA’14. ACM, New York, NY, USA, pp 475–478

  • Botev J, Rothkugel S (2017) High-precision gestural input for immersive large-scale distributed virtual environments. In: Proceedings of the 9th workshop on massively multiuser virtual environments, MMVE’17. ACM, New York, NY, USA, pp 7–11

  • Caserman P, Krabbe P, Wojtusch J, von Stryk O (2016) Real-time step detection using the integrated sensors of a head-mounted display. In: 2016 IEEE international conference on systems, man, and cybernetics (SMC), pp 3510–3515

  • Chan JCP, Leung H, Tang JKT, Komura T (2011) A virtual reality dance training system using motion capture technology. IEEE Trans Learn Technol 4(2):187–195

    Article  Google Scholar 

  • Choi SW, Seo MW, Lee SL, Park JH, Oh EY, Baek JS, Kang SJ (2016) Head position model-based latency measurement system for virtual reality head mounted display. SID Symp Dig Tech Papers 47(1):1381–1384

    Article  Google Scholar 

  • Collingwoode-Williams T, Gillies M, McCall C, Pan X (2017) The effect of lip and arm synchronization on embodiment: a pilot study. In: 2017 IEEE virtual reality (VR). IEEE, pp 253–254

  • Dempsey P (2016) The teardown: HTC Vive VR headset. Eng Technol 11(7–8):80–81

    Google Scholar 

  • Desai PR, Desai PN, Ajmera KD, Mehta K (2014) A review paper on oculus rift—a virtual reality headset. Int J Eng Trends Technol (IJETT) 13(4):175–179

    Article  Google Scholar 

  • Desai K, Raghuraman S, Jin R, Prabhakaran B (2017) QoE studies on interactive 3D tele-immersion. In: 2017 IEEE international symposium on multimedia (ISM), pp 130–137

  • Farahani N, Post R, Duboy J, Ahmed I, Kolowitz BJ, Krinchai T, Monaco SE, Fine JL, Hartman DJ, Pantanowitz L (2016) Exploring virtual reality technology and the oculus rift for the examination of digital pathology slides. J Pathol Inform 7:22

    Article  Google Scholar 

  • Friðriksson FA, Kristjánsson HS, Sigurðsson DA, Thue D, Vilhjálmsson HH (2016) Become your avatar: fast skeletal reconstruction from sparse data for fully-tracked VR. In: Proceedings of the 26th international conference on artificial reality and telexistence and the 21st Eurographics symposium on virtual environments: posters and demos, pp 19–20

  • Friston S, Steed A (2014) Measuring latency in virtual environments. IEEE Trans Vis Comput Graph 20(4):616–625

    Article  Google Scholar 

  • Galna B, Barry G, Jackson D, Mhiripiri D, Olivier P, Rochester L (2014) Accuracy of the microsoft kinect sensor for measuring movement in people with Parkinson’s disease. Gait Posture 39(4):1062–1068

    Article  Google Scholar 

  • Goradia I, Doshi J, Kurup L (2014) A review paper on oculus rift & project morpheus. Int J Curr Eng Technol 4(5):3196–3200

    Google Scholar 

  • Grochow K, Martin SL, Hertzmann A, Popović Z (2004) Style-based inverse kinematics. ACM Trans Graph 23(3):522–531

    Article  Google Scholar 

  • Huang J, Wang Q, Fratarcangeli M, Yan K, Pelachaud C (2017) Multi-variate gaussian-based inverse kinematics. Comput Graph Forum 36(8):418–428

    Article  Google Scholar 

  • Jain D, Sra M, Guo J, Marques R, Wu R, Chiu J, Schmandt C (2016) Immersive terrestrial scuba diving using virtual reality. In: Proceedings of the 2016 CHI conference extended abstracts on human factors in computing systems. ACM, New York, USA, pp 1563–1569

  • Jiang F, Yang X, Feng L (2016) Real-time full-body motion reconstruction and recognition for off-the-shelf VR devices. In: Proceedings of the 15th ACM SIGGRAPH conference on virtual-reality continuum and its applications in industry—Volume 1, VRCAI’16. ACM, pp 309–318

  • Johnson M, Humer I, Zimmerman B, Shallow J, Tahai L, Pietroszek K (2016) Low-cost latency compensation in motion tracking for smartphone-based head mounted display. In: Proceedings of the international working conference on advanced visual interfaces, AVI’16. ACM, New York, NY, USA, pp 316–317

  • Kasahara S, Konno K, Owaki R, Nishi T, Takeshita A, Ito T, Kasuga S, Ushiba J (2017) Malleable embodiment: changing sense of embodiment by spatial-temporal deformation of virtual human body. In: Proceedings of the 2017 CHI conference on human factors in computing systems, CHI’17. ACM, New York, NY, USA, pp 6438–6448

  • Kavan L, Sloan PP, O’Sullivan C (2010) Fast and efficient skinning of animated meshes. Comput Graph Forum 29(2):327–336

    Article  Google Scholar 

  • Kenwright B (2012) Real-time character inverse kinematics using the Gauss–Seidel iterative approximation method. Int Conf Creat Content Technol 4:63–68

    Google Scholar 

  • Lange B, Rizzo S, Chang CY, Suma EA, Bolas M (2011) Markerless full body tracking: depth-sensing technology within virtual environments. In: Interservice/industry training, simulation, and education conference (I/ITSEC)

  • Latoschik ME, Lugrin JL, Habel M, Roth D, Seufert C, Grafe S (2016) Breaking bad behavior: immersive training of class room management. In: Proceedings of the 22nd ACM conference on virtual reality software and technology, VRST’16. ACM, New York, NY, USA, pp 317–318

  • Latoschik ME, Roth D, Gall D, Achenbach J, Waltemate T, Botsch M (2017) The effect of avatar realism in immersive social virtual realities. In: Proceedings of the 23rd ACM symposium on virtual reality software and technology, VRST’17. ACM, New York, NY, USA, pp 39:1–39:10

  • Martindale J (2018) Oculus Rift vs. HTC Vive. https://www.digitaltrends.com/virtual-reality/oculus-rift-vs-htc-vive/. Accessed 4 May 2017​

  • Melo M, Rocha T, Barbosa L, Bessa M (2016) The impact of body position on the usability of multisensory virtual environments: case study of a virtual bicycle. In: Proceedings of the 7th international conference on software development and technologies for enhancing accessibility and fighting info-exclusion, DSAI 2016. ACM, New York, NY, USA, pp 20–24

  • Nakamura Y, Hanafusa H (1986) Inverse kinematic solutions with singularity robustness for robot manipulator control. J Dyn Syst Meas Control 108(3):163–171

    Article  MATH  Google Scholar 

  • Orin DE, Schrader WW (1984) Efficient computation of the Jacobian for robot manipulators. Int J Robot Res 3(4):66–75

    Article  Google Scholar 

  • Peck TC, Seinfeld S, Aglioti SM, Slater M (2013) Putting yourself in the skin of a black avatar reduces implicit racial bias. Conscious Cognit 22(3):779–787

    Article  Google Scholar 

  • Raaen K, Kjellmo I (2015) Measuring latency in virtual reality systems. In: Chorianopoulos K, Divitini M, Baalsrud Hauge J, Jaccheri L, Malaka R (eds) Entertainment computing—ICEC 2015. Springer, Cham, pp 457–462

  • Roberts D, Duckworth T, Moore C, Wolff R, O’Hare J (2009) Comparing the end to end latency of an immersive collaborative environment and a video conference. In: Proceedings of the 2009 13th IEEE/ACM international symposium on distributed simulation and real time applications, DS-RT’09. IEEE Computer Society, Washington, DC, USA, pp 89–94

  • Schmidt D, Kovacs R, Mehta V, Umapathi U, Köhler S, Cheng LP, Baudisch P (2015) Level-ups: motorized stilts that simulate stair steps in virtual reality. In: Proceedings of the 33rd annual ACM conference extended abstracts on human factors in computing systems, CHI EA’15. ACM, New York, NY, USA, pp 359–362

  • Seele S, Misztal S, Buhler H, Herpers R, Schild J (2017) Here’s looking at you anyway!: how important is realistic gaze behavior in co-located social virtual reality games? In: Proceedings of the annual symposium on computer-human interaction in play, CHI PLAY’17. ACM, New York, NY, USA, pp 531–540

  • Shoemake K (1985) Animating rotation with quaternion curves. In: Proceedings of the 12th annual conference on computer graphics and interactive techniques, SIGGRAPH’85. ACM, New York, NY, USA, pp 245–254

  • Shum H, Ho ES (2012) Real-time physical modelling of character movements with microsoft kinect. In: Proceedings of the 18th ACM symposium on virtual reality software and technology, VRST’12. ACM, pp 17–24

  • Sra M, Schmandt C (2015) MetaSpace II: object and full-body tracking for interaction and navigation in social VR. CoRR abs/1512.02922

  • Steed A (2008) A simple method for estimating the latency of interactive, real-time graphics simulations. In: Proceedings of the 2008 ACM symposium on virtual reality software and technology, VRST’08. ACM, New York, NY, USA, pp 123–129

  • Tao G, Archambault PS, Levin MF (2013) Evaluation of kinect skeletal tracking in a virtual reality rehabilitation system for upper limb hemiparesis. In: 2013 international conference on virtual rehabilitation (ICVR), pp 164–165

  • Thomas JS, France CR, Leitkam ST, Applegate ME, Pidcoe PE, Walkowski S (2016) Effects of real-world versus virtual environments on joint excursions in full-body reaching tasks. IEEE J Transl Eng Health Med 4:1–8

    Article  Google Scholar 

  • Tsai TC, Chen CY, Su GJ (2015) U-art: your art and ubiquitous art. In: Adjunct proceedings of the 2015 ACM international joint conference on pervasive and ubiquitous computing and proceedings of the 2015 ACM international symposium on wearable computers, UbiComp/ISWC’15 Adjunct. ACM, New York, NY, USA, pp 1295–1302

Download references

Author information

Authors and Affiliations

  1. Multimedia Communications Lab, Technische Universität Darmstadt, 64283, Darmstadt, Germany

    Polona Caserman, Augusto Garcia-Agundez, Robert Konrad, Stefan Göbel & Ralf Steinmetz

Authors
  1. Polona Caserman
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Augusto Garcia-Agundez
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Robert Konrad
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Stefan Göbel
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ralf Steinmetz
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Polona Caserman.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (mp4 31171 KB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Caserman, P., Garcia-Agundez, A., Konrad, R. et al. Real-time body tracking in virtual reality using a Vive tracker. Virtual Reality 23, 155–168 (2019). https://doi.org/10.1007/s10055-018-0374-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10055-018-0374-z

Keywords

Search

Navigation