Abstract
In this work, we present a software library which enables the efficient use of the Kinect One, a time-of-flight RGB-D sensor, with the nVidia Jetson TK1, an ARM-based embedded system, for the purpose of people detection. Our software exploits nVidia CUDA to process all data necessary for robust people detection algorithm and other perception algorithms by parallelizing the generation of the 3D point cloud and many pixel-wise operations on both the raw depth and the infrared images coming from the Kinect One sensor. The library developed has been released as open-source and the whole system has been tested as a people detection node in an open source multi-node RGB-D tracking framework (OpenPTrack). The results gathered show that the proposed system can be effectively used as a people detection node, outperforming the state-of-the-art in terms of people detection frame rate not only with the nVidia Jetson, but also with non-embedded computers.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Zennaro, S., Munaro, M., Milani, S., Zanuttigh, P., Bernardi, A., Ghidoni, S., Menegatti, E.: Performance evaluation of the 1st and 2nd generation kinect for multimedia applications. In: 2015 IEEE International Conference on Multimedia and Expo (ICME), pp. 1–6. IEEE (2015)
Gokturk, S.B., Yalcin, H., Bamji, C.: A time-of-flight depth sensor-system description, issues and solutions. In: Conference on Computer Vision and Pattern Recognition Workshop, 2004. CVPRW’04, pp. 35–35. IEEE (2004)
Quigley, M., Conley, K., Gerkey, B., Faust, J., Foote, T., Leibs, J., Wheeler, R., Ng, A.Y.: Ros: an open-source robot operating system. In: ICRA Workshop on Open Source Software, vol. 3, p. 5 (2009)
Munaro, M., Basso, F., Menegatti, E.: Openptrack: open source multi-camera calibration and people tracking for RGB-D camera networks. Robot. Auton. Syst. (2015)
Khoshelham, K., Elberink, S.O.: Accuracy and resolution of kinect depth data for indoor mapping applications. Sensors 12(2), 1437–1454 (2012)
Endres, F., Hess, J., Engelhard, N., Sturm, J., Cremers, D., Burgard, W.: An evaluation of the RGB-D slam system. In: 2012 IEEE International Conference on Robotics and Automation (ICRA), pp. 1691–1696. IEEE (2012)
Munaro, M., Menegatti, E.: Fast RGB-D people tracking for service robots. Auton. Robots 37(3), 227–242 (2014)
Munaro, M., Basso, F., Michieletto, S., Pagello, E., Menegatti, E.: A software architecture for RGB-D people tracking based on ROS framework for a mobile robot. In: Frontiers of Intelligent Autonomous Systems, pp. 53–68. Springer (2013)
Fleuret, F., Shitrit, H.B., Fua, P.: Re-identification for improved people tracking. In: Person Re-identification, pp. 309–330. Springer (2014)
Munaro, M., Ghidoni, S., Dizmen, D.T., Menegatti, E.: A feature-based approach to people re-identification using skeleton keypoints. In: 2014 IEEE International Conference on Robotics and Automation (ICRA), pp. 5644–5651. IEEE (2014)
Nanni, L., Munaro, M., Ghidoni, S., Menegatti, E., Brahnam, S.: Ensemble of different approaches for a reliable person re-identification system. Appl. Comput. Inf. (2015)
Carraro, M., Antonello, M., Tonin, L., Menegatti, E.: An open source robotic platform for ambient assisted living. In: Artificial Intelligence and Robotics (AIRO) (2015)
Fischinger, D., Einramhof, P., Papoutsakis, K., Wohlkinger, W., Mayer, P., Panek, P., Hofmann, S., Koertner, T., Weiss, A., Argyros, A., et al.: Hobbit, a care robot supporting independent living at home: first prototype and lessons learned. Robot. Auton. Syst. (2014)
Ghidoni, S., Anzalone, S.M., Munaro, M., Michieletto, S., Menegatti, E.: A distributed perception infrastructure for robot assisted living. Robot. Auton. Syst. 62(9), 1316–1328 (2014)
Basso, F., Pretto, A., Menegatti, E.: Unsupervised intrinsic and extrinsic calibration of a camera-depth sensor couple. In: 2014 IEEE International Conference on Robotics and Automation (ICRA), pp. 6244–6249. IEEE (2014)
Nickolls, J., Buck, I., Garland, M., Skadron, K.: Scalable parallel programming with CUDA. Queue 6(2), 40–53 (2008)
Xiang, L.: libfreenect2 CUDA library. https://github.com/xlz/libfreenect2 (2015). [Online; accessed 2016-02-03]
Wang, X.: Intelligent multi-camera video surveillance: a review. Pattern Recognit. Lett. 34(1), 3–19 (2013)
Vezzani, R., Baltieri, D., Cucchiara, R.: Pathnodes integration of standalone particle filters for people tracking on distributed surveillance systems. In: Image Analysis and Processing–ICIAP 2009, pp. 404–413. Springer (2009)
Kandhalu, A., Rowe, A., Rajkumar, R., Huang, C., Yeh, C.-C.: Real-time video surveillance over IEEE 802.11 mesh networks. In: Real-Time and Embedded Technology and Applications Symposium, 2009. RTAS 2009. 15th IEEE, pp. 205–214. IEEE (2009)
Jafari, O.H., Mitzel, D., Leibe, B.: Real-time RGB-D based people detection and tracking for mobile robots and head-worn cameras. In: 2014 IEEE International Conference on Robotics and Automation (ICRA), pp. 5636–5643. IEEE (2014)
Munaro, M., Basso, F., Menegatti, E.: Tracking people within groups with RGB-D data. In: 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 2101–2107. IEEE (2012)
Munaro, M., Lewis, C., Chambers, D., Hvass, P., Menegatti, E.: RGB-D human detection and tracking for industrial environments. In: Intelligent Autonomous Systems, vol. 13, pp. 1655–1668. Springer (2016)
Sanders, J., Kandrot, E.: CUDA by example: an introduction to general-purpose GPU programming. Addison-Wesley Professional (2010)
Ryoo, S., Rodrigues, C.I., Baghsorkhi, S.S., Stone, S.S., Kirk, D.B., Hwu, W.-m.W.: Optimization principles and application performance evaluation of a multithreaded GPU using CUDA. In: Proceedings of the 13th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, pp. 73–82. ACM (2008)
Aldoma, A., Tombari, F., Prankl, J., Richtsfeld, A., Di Stefano, L., Vincze, M.: Multimodal cue integration through hypotheses verification for RGB-D object recognition and 6DOF pose estimation. In: 2013 IEEE International Conference on Robotics and Automation (ICRA), pp. 2104–2111. IEEE (2013)
Henry, P., Krainin, M., Herbst, E., Ren, X., Fox, D.: RGB-D mapping: using kinect-style depth cameras for dense 3D modeling of indoor environments. Int. J. Robot. Res. 31(5), 647–663 (2012)
Carraro, M., Munaro, M., Menegatti, E.: Cost-efficient RGB-D smart camera for people detection and tracking. J. Electron. Imaging 25(4), 041007–041007 (2016)
Acknowledgements
Portions of this work have been supported by NVidia, OpenPerception and the REMAP center at UCLA. The authors would like to thank Randy Illum and Jeff Burke at UCLA for the extensive testing of the developed software.
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Carraro, M., Munaro, M., Menegatti, E. (2017). A Powerful and Cost-Efficient Human Perception System for Camera Networks and Mobile Robotics. In: Chen, W., Hosoda, K., Menegatti, E., Shimizu, M., Wang, H. (eds) Intelligent Autonomous Systems 14. IAS 2016. Advances in Intelligent Systems and Computing, vol 531. Springer, Cham. https://doi.org/10.1007/978-3-319-48036-7_35
Download citation
DOI: https://doi.org/10.1007/978-3-319-48036-7_35
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-48035-0
Online ISBN: 978-3-319-48036-7
eBook Packages: EngineeringEngineering (R0)