Abstract
This paper describes the devising of a real-time full-body motion-capture system for multi-user Collaborative Virtual Environment (CVE). The idea takes advantage of some Natural User Interface devices such as the Microsoft Kinect and the Leap Motion Controller. The aim of our approach is to allow a rapid and easy access of participants to the tracked area, that is why the described system has been devised to be both wireless and markerless.
The article shows how multiple Kinect units can be used as a whole to both enlarge the tracking area and be tolerant to the shielding effect due to the overlapping of multiple participants seen by the sensors.
Further fusion strategies are presented to combine Kinect-based multi-body tracking along with head-tracking and head-mounted Leap Motion Controllers data in order to get body-tracking with the full hand detail, which enables direct hand manipulation in applications such as first-person virtual maintenance training.
Although preliminary, the shown results are already encouraging. Once data will be analyzed more in depth and after a system tuning, an effective and even more reliable final multi-person tracking system is expected.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Microsoft Kinect for Windows. https://developer.microsoft.com/en-us/windows/Kinect
A State of the Art Report on Kinect Sensor Setups in Computer Vision. http://www.grk1564.uni-siegen.de/sites/www.grk1564.uni-siegen.de/files/inm2013/kinect-star.pdf
Khoshelham, K., Elberink, S.O.: Accuracy and resolution of kinect depth data for indoor mapping applications. Sensors 12, 1437–1454 (2012)
Ye, M., Wang, X., Yang, R., Ren, L., Pollefeys, M.: Accurate 3D pose estimation from a single depth image. In: Proceedings of the IEEE International Conference on Computer Vision, Barcelona, Spain, pp. 731–738, 6–13 November 2011
Weiss, A., Hirshberg, D., Black, M.J.: Home 3D body scans from noisy image and range data. In: Proceedings of the IEEE International Conference on Computer Vision, Barcelona, Spain, pp. 1951–1958, 6–13 November 2011
Shotton, J., Sharp, T., Kipman, A., Fitzgibbon, A., Finocchio, M., Blake, A., Cook, M., Moore, R.: Real-time human pose recognition in parts from single depth images. Commun. ACM 56, 116–124 (2013)
Grest, D., Krüger, V., Koch, R.: Single view motion tracking by depth and silhouette information. In: Ersbøll, B.K., Pedersen, K.S. (eds.) SCIA 2007. LNCS, vol. 4522, pp. 719–729. Springer, Heidelberg (2007). doi:10.1007/978-3-540-73040-8_73
Wei, X., Zhang, P., Chai, J.: Accurate realtime full-body motion capture using a single depth camera. ACM Trans. Graph. 31, 1–12 (2012)
Gao, Z., Yu, Y., Zhou, Y., Du, S.: Leveraging two kinect sensors for accurate full-body motion capture (2015). http://www.mdpi.com/1424-8220/15/9/24297/htm
Liu, Y., Stoll, C., Gall, J., Seidel, H.-P., Theobalt, C.: Markerless motion capture of interacting characters using multi-view image segmentation. http://ieeexplore.ieee.org/abstract/document/5995424/
Leap Motion Controller VR Development. https://developer.leapmotion.com/windows-vr
Marin, G., Dominio, F., Zanuttigh, P.: Hand gesture recognition with leap motion and kinect devices. http://ieeexplore.ieee.org/abstract/document/7025313/
Craig, A., Krishnan, S.: Fusion of leap motion and kinect sensors for improved field of view and accuracy for VR applications
Kowalski, M., Naruniec, J., Daniluk, M.: LiveScan3D: a fast and inexpensive 3D data acquisition system for multiple kinect v2 sensors. In: 2015 International Conference on 3D Vision (3DV), Lyon, France (2015). http://ieeexplore.ieee.org/document/7335499/
Besl, P.J., McKay, N.D.: A method for registration of 3-D shapes. IEEE Trans. Pattern Anal. Mach. Intell. 14(2), 239–256 (1992)
Kopniak, P.: Motion capture using multiple Kinect controllers (2015). http://www.pe.org.pl/articles/2015/8/7.pdf
Morato, C., Kaipa, K.N., Zhao, B., Gupta, S.K.: Toward safe human robot collaboration by using multiple Kinects based real-time human tracking. J. Comput. Inf. Sci. Eng. 14(1), 011005-01–011005-10 (2014). doi:10.1115/1.4025810
Satyavolu, S., Bruder, G., Willemsen, P., Steinicke, F.: Analysis of IR-based virtual reality tracking using multiple Kinects. https://www.researchgate.net/publication/241628899_Analysis_of_IR-based_virtual_reality_tracking_using_multiple_Kinects
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Leoncini, P., Sikorski, B., Baraniello, V., Martone, F., Luongo, C., Guida, M. (2017). Multiple NUI Device Approach to Full Body Tracking for Collaborative Virtual Environments. In: De Paolis, L., Bourdot, P., Mongelli, A. (eds) Augmented Reality, Virtual Reality, and Computer Graphics. AVR 2017. Lecture Notes in Computer Science(), vol 10324. Springer, Cham. https://doi.org/10.1007/978-3-319-60922-5_10
Download citation
DOI: https://doi.org/10.1007/978-3-319-60922-5_10
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-60921-8
Online ISBN: 978-3-319-60922-5
eBook Packages: Computer ScienceComputer Science (R0)