A human pose estimation algorithm based on the integration of improved convolutional neural networks and multi-level graph structure constrained model

  • Yang YouEmail author
  • Yanmin Zhao
Original Article


This work introduces a novel convolutional feature for the task of human pose estimation, which is a framework of fusing a convolutional neural network into a multi-level graph structure model so as to improve the pose estimation results from body-part detection and human spatial structure. In the stage of part detection, the probability vectors corresponding to human body parts in the whole image are analyzed by a convolutional neural network. And then, a novel multi-level graph structure model is designed in this method, which contains the whole body, human body parts and joints, and realizes the coarse-to-fine establishment of a more constrained human spatiTal constraint model from levels of the structures, the edges, to the pixels. The obtained probability vectors are put into the multi-level graph structure model to compute the location coordinates for each joint, successfully achieving a framework of combining the deep learning network and the multi-level graph structure model. A large number of qualitative and quantitative experimental results show that compared with other state-of-art methods, the integration of the deep learning network and the multi-level pictorial structure model can improve the accuracy of human pose estimation to a greater extent.


Multi-level graph structure Pose estimation Convolutional neural network Coarse-to-fine Spatial constraint Combinatorial component 


Funding information

This work was financially supported by the Ministry of Education Project of Humanities and Social Sciences: Research on the Development of High-quality Preschool Physical Education from the Perspective of Body Cognition (No. 19YZC890063).


  1. 1.
    Katircioglu I, Tekin B, Salzmann M et al (2018) Learning latent representations of 3D human pose with deep neural networks. Int J Comput Vis 8:1–16Google Scholar
  2. 2.
    Ge L, Liang H, Yuan J et al (2018) Robust 3D hand pose estimation from single depth images using multi-view CNNs. IEEE Trans Image Process 99:1MathSciNetzbMATHGoogle Scholar
  3. 3.
    Du X, Kurmann T, Chang P L, et al (2018) Articulated multi-instrument 2D pose estimation using fully convolutional networks. IEEE Trans Med Imaging 1Google Scholar
  4. 4.
    Tang Z, Wang S, Huo J et al (2017) Bayesian framework with non-local and low-rank constraint for image reconstruction[C]// journal of physics conference series. J Phys Conf SerGoogle Scholar
  5. 5.
    Marín-Jiménez MJ, Romero-Ramirez FJ, Rafael MS et al (2018) 3D human pose estimation from depth maps using a deep combination of poses. J Vis Commun Image Represent 55:627–639Google Scholar
  6. 6.
    Li M, Zhou Z, Liu X (2019) Multi-person pose estimation using bounding box constraint and LSTM. IEEE Trans Multimedia 99:1Google Scholar
  7. 7.
    Xia K, Wang J, Wu Y (2017) Robust Alzheimer disease classification based on feature integration fusion model for magnetic. J Medicine Imaging and Health Informatics 21(7):1–6Google Scholar
  8. 8.
    Emre D, Gonen E, Christian W et al (2018) Multi-view pose estimation with mixtures of parts and adaptive viewpoint selection. IET Comput Vis 12(4):403–411Google Scholar
  9. 9.
    Liu J, Ding H, Shahroudy A et al Feature boosting network for 3D pose estimation. IEEE Trans Pattern Anal Mach Intell 99:1Google Scholar
  10. 10.
    Nie X, Feng J, Xing J, et al (2018) Hierarchical contextual refinement networks for human pose estimation. IEEE Trans Image Process 1Google Scholar
  11. 11.
    Fang Y, Wusheng C, Yun W et al (2018) Sparse unorganized point cloud based relative pose estimation for uncooperative space target. Sensors 18(4):1009Google Scholar
  12. 12.
    Lebel K, Hamel M, Duval C, Nguyen H, Boissy P (2018) Camera pose estimation to improve accuracy and reliability of joint angles assessed with attitude and heading reference systems. Gait Posture 59:199–205Google Scholar
  13. 13.
    Ning G, Member S (2018) IEEE, et al. knowledge-guided deep fractal neural networks for human pose estimation. IEEE Trans Multimedia 20(5):1246–1259Google Scholar
  14. 14.
    Qian P, Jiang Y, Deng Z, Hu L, Sun S, Wang S, Muzic RF Jr (2016) Cluster prototypes and fuzzy memberships jointly leveraged cross-domain maximum entropy clustering. IEEE Trans Cybern 46(1):181–193Google Scholar
  15. 15.
    Qian P, Jiang Y, Wang S, Su K-H, Wang J, Hu L, Muzic RF Jr (2017) Affinity and penalty jointly constrained spectral clustering with all-compatibility, flexibility, and robustness. IEEE Trans Neural Netw Learn Syst 28(5):1123–1138Google Scholar
  16. 16.
    Jiang Y, Deng Z, Chung F-L, Wang G, Qian P, Choi K-S, Wang S (2017) Recognition of epileptic EEG signals using a novel multi-view TSK fuzzy system. IEEE Trans Fuzzy Syst 25(1):3–20Google Scholar
  17. 17.
    Jiang Y, Chung F-L, Wang S, Deng Z, Wang J, Qian P (2015) Collaborative fuzzy clustering from multiple weighted views. IEEE Trans Cybern 45(4):688–701Google Scholar
  18. 18.
    Jiang Y, Chung F-L, Ishibuchi H et al (2015) Multitask TSK fuzzy system modeling by mining intertask common hidden structure. IEEE Trans Cybern 45(3):548–561Google Scholar
  19. 19.
    Qian P, Zhao K, Jiang Y, Su K-H, Deng Z, Wang S, Muzic RF Jr (2017) Knowledge-leveraged transfer fuzzy c-means for texture image segmentation with self-adaptive cluster prototype matching. Knowl-Based Syst 130:33–50Google Scholar
  20. 20.
    Mohseni SSS, Shadab K, Deniz E, et al (2018) Real-time deep pose estimation with geodesic loss for image-to-template rigid registration. IEEE Trans Med Imaging 1Google Scholar
  21. 21.
    Fu L, Zhang J, Huang K (2017) ORGM: occlusion relational graphical model for human pose estimation. IEEE Trans Image Process 26(2):927–941MathSciNetzbMATHGoogle Scholar
  22. 22.
    Ning G, Zhang Z, He Z (2017) Knowledge-guided deep fractal neural networks for human pose estimation. IEEE Trans Multimedia 99:1Google Scholar
  23. 23.
    Yu J, Hong C (2017) Exemplar-based 3D human pose estimation with sparse spectral embedding. Neurocomputing S0925231217309657Google Scholar
  24. 24.
    Saremi S, Mirjalili S, Lewis A (2018) Vision-based hand posture estimation using a new hand model made of simple components. Optik S0030402618302584Google Scholar
  25. 25.
    Bourdev L, Poselets MJ (2009) Body part detectors trained using 3D human pose annotations// IEEE international conference on computer vision. IEEEGoogle Scholar
  26. 26.
    Gao H, Mao S, Huang W, Yang X (2018) Applying probabilistic model checking to financial production risk evaluation and control: a case study of Alibaba’s Yu’e Bao. IEEE Trans Comput Soc Syst 5(3):785–795Google Scholar
  27. 27.
    Gao H, Duan Y, Miao H, Yin Y (2017) An approach to data consistency checking for the dynamic replacement of service process. IEEE Access 5(1):11700–11711Google Scholar
  28. 28.
    Gao H, Chu D, Duan Y (2017) The probabilistic model checking based service selection method for business process modeling. Int J Softw Eng Knowl Eng 27(6):897–923Google Scholar
  29. 29.
    Toshev A, Szegedy C. (2013) DeepPose: Human pose estimation via deep neural networks. 32(3):1291–1309Google Scholar
  30. 30.
    Guoliang L, Guohui T, Junwei L, et al (2018) Human action recognition using a distributed RGB-depth camera network. IEEE Sensors J 1Google Scholar
  31. 31.
    Jun W, Jing LI, Jun C et al (2018) Face alignment by coarse-to-fine shape estimation. Chin J Electron 27(06):77–85Google Scholar
  32. 32.
    Xia KJ, Yin HS, Wang JQ (2018) A novel improved deep convolutional neural network model for medical image fusion. Clust Comput (3):1–13Google Scholar

Copyright information

© Springer-Verlag London Ltd., part of Springer Nature 2019

Authors and Affiliations

  1. 1.Department of physical educationChina University of Petroleum (East China)QingdaoChina

Personalised recommendations