A Novel 3D Human Action Recognition Method Based on Part Affinity Fields

  • Haipeng Dong
  • Qingqing Meng
  • Tao HuEmail author
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 857)


The analysis of human actions based on 3D skeleton data becomes popular recently due to its succinctness, robustness, and view-invariant representation. Recent attempts on this problem suggested to use human body affinity fields to efficiently detect the 2D pose of multiple people in an image. In this paper, we extend this idea to 3D domains and develop a 3D human action recognition system with ability to understand the cooperative action of several people. To achieve this, we firstly extract the human body affinity fields to robustly represent associate 2D human skeleton with individuals in the image. Inspired by the triangulation techniques in stereo vision analysis, 3D human skeleton data can be obtained. To handle the noise in constructed 3D human skeleton data, we introduce an enhanced light-weight matching algorithm based on Dynamic Time Warping (DTW) to compute the matching cost. The real-life experiments demonstrate the efficiency and applicability of our approach.


Action recognition Binocular stereo camera Part affinity fields Dynamic time warping 


  1. 1.
    Wang, H., Schmid, C.: Action recognition with improved trajectories. In: Proceedings of IEEE International Conference Computer Vision, pp. 3551–3558 (2013)Google Scholar
  2. 2.
    Wang, H., Kläser, A., Schmid, C., Liu, C.L.: Dense trajectories and motion boundary descriptors for action recognition. Int. J. Comput. Vis. 103(1), 60–79 (2013)MathSciNetCrossRefGoogle Scholar
  3. 3.
    Geng-Dai, L., Ming-Liang, X., Ming-min, Z.: Human motion synthesis based on independent spatio-temporal feature space. Chinese J. Comput. 34(3), 464–472 (2011)CrossRefGoogle Scholar
  4. 4.
    Tao, Q., Dexiang, D., Hui, L., Lian, Z., Yifeng, L.: Extracting spatio-temporal features via multi-layer independent subspace analysis for action recognition. Geomatics Inf. Sci. WuHan Univ. 41(4) (2016)Google Scholar
  5. 5.
    Zhang, S., Liu, X., Xiao, J.: On geometric features for skeleton-based action recognition using multilayer LSTM networks. In: Proceedings of 2017 IEEE Winter Conference on Applications of Computer Vision, WACV 2017, pp. 148–157 (2017)Google Scholar
  6. 6.
    Liu, J., Shahroudy, A., Xu, D., Wang, G.: Spatio-temporal LSTM with trust gates for 3D human action recognition. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9907, pp. 816–833. Springer, Cham (2016). Scholar
  7. 7.
    Chen, G., Giuliani, M., Clarke, D., Gaschler, A., Knoll, A.: Action recognition using ensemble weighted multi-instance learning. In: Proceedings of IEEE International Conference on Robotics and Automation, pp. 4520–4525 (2014)Google Scholar
  8. 8.
    Zhu, W., et al.: Co-occurrence feature learning for skeleton based action recognition using regularized deep LSTM networks, no. 2, pp. 3697–3703 (2016)Google Scholar
  9. 9.
    Guohui, T., Jianqin, Y., Xu, H., Jing, Y.: A novel human activity recognition method using joint points information. Robot 36(3), 285–292 (2014)Google Scholar
  10. 10.
    Sempena, S., Maulidevi, N.U., Aryan, P.R.: Human action recognition using dynamic time warping. In: Proceedings of 2011 International Conference on Electrical Engineering Informatics, no. July, pp. 1–5 (2011)Google Scholar
  11. 11.
    Cao, Z., Simon, T., Wei, S.-E., Sheikh, Y.: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields (2016)Google Scholar
  12. 12.
    Galna, B., Barry, G., Jackson, D., Mhiripiri, D., Olivier, P., Rochester, L.: Accuracy of the Microsoft Kinect sensor for measuring movement in people with Parkinson’s disease. Gait Posture 39(4), 1062–1068 (2014)CrossRefGoogle Scholar
  13. 13.
    Chengfeng, W., Hong, C., Ruixuan, Z., Dehai, Z., Qing, W., Shuli, M.: Research on DTW action recognition algorithm with joint weighting. J. Graph. 37(4) (2016)Google Scholar
  14. 14.
    Liu, Z.: Zicheng Liu at Microsoft Research (2017).!projects
  15. 15.
    ZED Stereo Camera:

Copyright information

© Springer Nature Singapore Pte Ltd. 2018

Authors and Affiliations

  1. 1.School of Computer Science and EngineeringNortheastern UniversityShenyangChina

Personalised recommendations