Skip to main content

Advertisement

SpringerLink
Log in

Robust Tracking Occluded Human in Group by Perception Sensors Network System

  • Published:
Journal of Intelligent & Robotic Systems Aims and scope Submit manuscript

Abstract

Tracking people even being partially or fully occluded in the group situation is studied using a Perception Sensor Network (PSN) system which is composed of multiple Kinects used to detect human 3D locations, and pan tilt zoom (PTZ) cameras used to identify human faces. A method is proposed to fuse multiple detection of human in the PSN system. After associating detected human with corresponding names, the novel grouping and ungrouping algorithms are proposed. When a group of multiple human staying close together is formed, viewpoint and illumination invariant features of group members including human 3D location, height, color and binary robust invariant scalable keypoint (BRISK), retrieved from region of interest (ROI) of both depth and color images, are then stored and updated into the group database. Based on the distance between a group location at previous frame and each member location in the group at current frame, the PSN system decides whether to keep the members in the group or to ungroup them then reassign the right name among the group database by minimizing multiple criterions. The experimental results demonstrate the outperforming of the proposed method on tracking people in group than conventional methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Goodrich, M.A., Schultz, A.C.: Human-robot interaction: a survey. Found Trends Hum-Comput Interact 1, 203–275 (2007)

    Article  MATH  Google Scholar 

  2. Steinfeld, A., Fong, T., Kaber, D., Lewis, M., Scholtz, J., Schultz, A., Goodrich, M.: Common metrics for human-robot interaction. In: Proceedings of the 1St ACM SIGCHI/SIGART conference on human-robot interaction, pp. 33–40. ACM, New York (2006)

  3. Ong, K.S., Hsu, Y.H., Fu, L.C.: Sensor fusion based human detection and tracking system for human-robot interaction. In: 2012 IEEE/RSJ international conference on intelligent robots and systems, pp. 4835–4840 (2012)

  4. Zhang, Z.: Microsoft kinect sensor and its effect. IEEE Multimed 19, 4–10 (2012)

    Article  Google Scholar 

  5. Xia, L., Chen, C.C., Aggarwal, J.K.: Human detection using Depth information by Kinect. In: CVPR 2011 WORKSHOPS, pp. 15–22 (2011)

  6. ROS OpenNI Kinect: ROS OpenNI open source project: http://www.ros.org/wiki/opennikinect (Accessed November 10, 2016) (2016)

  7. Quigley, M., Conley, K., Gerkey, B., Faust, J., Foote, T., Leibs, J., Wheeler, R., Ng, A.: ROS: an open-source Robot Operating System

  8. Schwarz, L.A., Mkhitaryan, A., Mateus, D., Navab, N.: Human skeleton tracking from depth data using geodesic distances and optical flow. Image Vis. Comput. 30, 217–226 (2012)

    Article  Google Scholar 

  9. Zhao, T., Nevatia, R.: Tracking multiple humans in crowded environment. In: Proceedings of the 2004 IEEE computer society conference on computer vision and pattern recognition, 2004. CVPR 2004, vol. 2, pp. II-406-II-413 (2004)

  10. Yun, X., Bachmann, E.R.: Design, implementation, and experimental results of a Quaternion-Based kalman filter for human body motion tracking. IEEE Trans. Robot. 22, 1216–1227 (2006)

    Article  Google Scholar 

  11. Choi, W., Pantofaru, C., Savarese, S.: Detecting and tracking people using an RGB-D camera via multiple detector fusion. In: 2011 IEEE international conference on computer vision workshops (ICCV workshops), pp. 1076–1083 (2011)

  12. Munaro, M., Basso, F., Menegatti, E.: Tracking people within groups with RGB-D data. In: 2012 IEEE/RSJ international conference on intelligent robots and systems, pp. 2101–2107 (2012)

  13. Benfold, B., Reid, I.: Stable multi-target tracking in real-time surveillance video. In: 2011 IEEE conference on computer vision and pattern recognition (CVPR), pp. 3457–3464 (2011)

  14. Yamaguchi, K., Berg, A.C., Ortiz, L.E., Berg, T.L.: Who are You with and Where are You Going?. In: 2011 IEEE conference on computer vision and pattern recognition (CVPR), pp. 1345–1352 (2011)

  15. Pellegrini, S., Ess, A., Gool, L.V.: Improving data association by joint modeling of pedestrian trajectories and groupings. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) Computer vision – ECCV 2010, pp. 452–465. Springer, Berlin (2010)

  16. Zhang, C., Hamid, R., Zhang, Z.: Taylor expansion based classifier adaptation: application to person detection. In: IEEE conference on computer vision and pattern recognition, 2008. CVPR 2008, pp. 1–8 (2008)

  17. Leal-Taixé, L., Pons-Moll, G., Rosenhahn, B.: Everybody needs somebody: modeling social and grouping behavior on a linear programming multiple people tracker. In: 2011 IEEE international conference on computer vision workshops (ICCV workshops), pp. 120–127 (2011)

  18. Izadinia, H., Saleemi, I., Li, W., Shah, M.: (MP)2T: multiple people multiple parts tracker. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) Computer vision – ECCV 2012, pp. 100–114. Springer, Berlin (2012)

  19. Breitenstein, M.D., Reichlin, F., Leibe, B., Koller-Meier, E., Gool, L.V.: Online multiperson tracking-by-detection from a single, uncalibrated camera. IEEE Trans. Pattern Anal. Mach. Intell. 33, 1820–1833 (2011)

    Article  Google Scholar 

  20. Berclaz, J., Fleuret, F., Turetken, E., Fua, P.: Multiple object tracking using K-Shortest paths optimization. IEEE Trans. Pattern Anal. Mach. Intell. 33, 1806–1819 (2011)

    Article  Google Scholar 

  21. Conte, D., Foggia, P., Percannella, G., Vento, M.: Performance evaluation of a people tracking system on PETS2009 database. In: 2010 Seventh IEEE international conference on advanced video and signal based surveillance (AVSS), pp. 119–126 (2010)

  22. Berclaz, J., Fleuret, F., Fua, P.: Multiple object tracking using flow linear programming. In: 2009 Twelfth IEEE international workshop on performance evaluation of tracking and surveillance (PETS-Winter), pp. 1–8 (2009)

  23. Alahi, A., Jacques, L., Boursier, Y., Vandergheynst, P.: Sparsity-Driven People Localization Algorithm: Evaluation in Crowded Scenes Environments. In: 2009 Twelfth IEEE international workshop on performance evaluation of tracking and surveillance (PETS-Winter), pp. 1–8 (2009)

  24. An, K., Park, J., Hoang, M.D., Choi, J.: Dispensing materials of mobile robot cooperating with perception sensor network. In: 2014 11Th international conference on ubiquitous robots and ambient intelligence (URAI), pp. 496–499 (2014)

  25. Park, J., An, K., Choi, J.: Realistic 3D simulation of multiple human recognition over perception sensor network. In: The 23Rd IEEE international symposium on robot and human interactive communication, pp. 507–512 (2014)

  26. Leutenegger, S., Chli, M., Siegwart, R.Y.: BRISK: binary robust invariant scalable keypoints. In: 2011 international conference on computer vision, pp. 2548–2555 (2011)

  27. Lowe, D.G.: Distinctive image features from Scale-Invariant keypoints. Int. J. Comput. Vis. 60, 91–110 (2004)

    Article  Google Scholar 

  28. Bay, H., Ess, A., Tuytelaars, T., Van Gool, L.: Speeded-Up Robust features (SURF). Comput. Vis. Image Underst. 110, 346–359 (2008)

    Article  Google Scholar 

  29. Bernardin, K., Stiefelhagen, R.: Evaluating multiple object tracking performance: The CLEAR MOT metrics. EURASIP J. Image Video Process. 2008, 246309 (2008)

    Article  Google Scholar 

  30. Ferryman, J., Shahrokni, A.: An Overview of the PETS 2009 Challenge. In: Presented at the Eleventh IEEE International Workshop on Performance Evaluation of Tracking and Surveillance, Miami (2009)

Download references

Acknowledgments

The research was partly supported by R&D programs of MOTIE (10041629 [SimonPiC] and 10077468 [DeepTasK]) and by ICT R&D programs of IITP (2015-0-00197 [LISTEN] and 2017-0-00432 [BCI]).

Author information

Authors and Affiliations

  1. Optoelectronics Research Group, Faculty of Electrical and Electronics Engineering, Ton Duc Thang University, Ho Chi Minh City, Vietnam

    Anh Vu Le

  2. Center for Robotics Research, Korea Institute of Science and Technology, Seoul, Korea

    JongSuk Choi

Authors
  1. Anh Vu Le
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. JongSuk Choi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Anh Vu Le.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Le, A.V., Choi, J. Robust Tracking Occluded Human in Group by Perception Sensors Network System. J Intell Robot Syst 90, 349–361 (2018). https://doi.org/10.1007/s10846-017-0667-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10846-017-0667-6

Keywords

Search

Navigation