Skip to main content

Advertisement

SpringerLink
Log in

Generative tracking of 3D human motion in latent space by sequential clonal selection algorithm

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

High dimensional pose state space is the main challenge in articulated human pose tracking which makes pose analysis computationally expensive or even infeasible. In this paper, we propose a novel generative approach in the framework of evolutionary computation, by which we try to widen the bottleneck with effective search strategy embedded in the extracted state subspace. Firstly, we use ISOMAP to learn the low-dimensional latent space of pose state in the aim of both reducing dimensionality and extracting the prior knowledge of human motion simultaneously. Then, we propose a manifold reconstruction method to establish smooth mappings between the latent space and original space, which enables us to perform pose analysis in the latent space. In the search strategy, we adopt a new evolutionary approach, clonal selection algorithm (CSA), for pose optimization. We design a CSA based method to estimate human pose from static image, which can be used for initialization of motion tracking. In order to make CSA suitable for motion tracking, we propose a sequential CSA (S-CSA) algorithm by incorporating the temporal continuity information into the traditional CSA. Actually, in a Bayesian inference view, the sequential CSA algorithm is in essence a multilayer importance sampling based particle filter. Our methods are demonstrated in different motion types and different image sequences. Experimental results show that our CSA based pose estimation method can achieve viewpoint invariant 3D pose reconstruction and the S-CSA based motion tracking method can achieve accurate and stable tracking of 3D human motion.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17

Similar content being viewed by others

References

  1. Agarwal A, Triggs B (2005) Monocular human motion capture with a mixture of regressors. IEEE Workshop on Vision for Human-Computer Interaction

  2. Agarwal A, Triggs B (2006) Recovering 3-D human pose from monocular images. IEEE Trans Pattern Anal Mach Intell 28(1):44–58

    Article  Google Scholar 

  3. CMU database. http://mocap.cs.cmu.edu/

  4. de Castro LN, Von Zuben FJ (2002) Learning and optimization using the clonal selection principle. IEEE Trans Evol Comput 6(3):239–251

    Article  Google Scholar 

  5. Deutscher J, Blake A, Reid I (2000) Articulated body motion capture by annealed particle filtering. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp 126–133

  6. Elgammal A, Lee C (2004) Inferring 3D body pose from silhouettes using activity manifold learning. In: Proceedings of IEEE computer society conference on computer vision and pattern recognition, pp 681–688

  7. El-Nady KE, Abou El-Enien UH, Badr AA (2011) Why are clonal selection algorithms MCMC. Int J Open Probl Comput Math 4(3):37–55

    MathSciNet  Google Scholar 

  8. Gong M, Jiao L, Zhang L (2010) Baldwinian learning in clonal selection algorithm for optimization. Inform Sci 180:1218–1236

    Article  Google Scholar 

  9. Howe NR, Leventon ME, Freeman WT (2000) Bayesian reconstruction of 3D human motion from single-camera video. In: NIPS, pp 820–826

  10. Isard M, Blake A (1998) Condensation: conditional density propagation for visual tracking. Int J Comput Vis 29(1):5–28

    Article  Google Scholar 

  11. John V, Trucco E, Ivekovic S (2010) Markerless human articulated tracking using hierarchical particle swarm optimization. Image Vis Comput 28(11):1530–1547

    Article  Google Scholar 

  12. Krzeszowski T, Kwolek B, Wojciechowski K (2010) Articulated body motion tracking by combined particles swarm optimization and particle filtering. In: Proceedings of international conference on Computer vision and graphics, pp 147–154

  13. Lawrence ND (2003) Gaussian process latent variable models for visualization of high dimensional data. Advances in Neural Information Processing Systems (NIPS), pp 329–336

  14. Lee C-S, Elgammal A (2010) Coupled visual and kinematic manifold models for tracking. Int J Comput Vis 87(1):118–139

    Article  Google Scholar 

  15. Moeslund TB, Hilton A, Kruger V (2006) A survey of advances in vision-based human motion capture and analysis. Comput Vis Image Und 104(2):90–126

    Article  Google Scholar 

  16. Mori G, Malik J (2006) Recovering 3D human body configurations using shape contexts. IEEE Trans Pattern Anal Mach Intell 28(7):1052–1062

    Article  Google Scholar 

  17. Ormoneit D, Sidenbladh H, Black MJ, Hastie T (2001) Learning and tracking cyclic human motion. Advances Neural Info Process Systs 13:894–900

    Google Scholar 

  18. Poppe R (2007) Vision-based human motion analysis: an overview. Comput Vis Image Und 108(1):4–18

    Article  Google Scholar 

  19. Raskin L, Rudzsky M, Rivlin E (2011) Dimensionality reduction using a Gaussian Process Annealed Particle Filter for tracking and classification of articulated body motions. Comput Vis Image Und 115(4):503–519

    Article  Google Scholar 

  20. Sigal L, Black MJ (2010) HumanEva: synchronized video and motion capture dataset for evaluation of articulated human motion. Int J Comput Vis 87(1):4–27

    Article  MathSciNet  Google Scholar 

  21. Sminchisescu C (2007). 3D human motion analysis in monocular video, techniques and challenges. In: Kleete R, Metaxas D, Rosenhahn B (eds) Human motion understanding, modeling, capture and animation. Springer-Verlag

  22. Sminchisescu C, Jepson AD (2004) Generative modeling for continuous non-linearly embedded visual inference. In: Proceedings of the international conference on machine learning, pp 759–766

  23. Sminchisescu C, Triggs B (2003) Estimating articulated human motion with covariance scaled sampling. Int J Robot Res 22(6):371–392

    Article  Google Scholar 

  24. Tenenbaum JB, de Silva V, Langford JC (2000) A global geometric framework for nonlinear dimensionality reduction. Science 290(22):2319–2323

    Article  Google Scholar 

  25. Tian T-P, Li R, Sclaroff S (2005) Tracking human body pose on a learned smooth space. Technical Report BUCS-TR-2005-029, Boston University, Computer Science Department, Boston, MA, July 2005

  26. Urtasun R, Fleet D, Fua P (2005) Monocular 3-D tracking of golf swing. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp 932–93

  27. Zhang X, Hu W, Maybank S, Xi L (2008) Sequential particle swarm optimization for visual tracking. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp 23–28

  28. Zhao X, Liu YC (2008) Generative tracking of 3D human motion by hierarchical annealed genetic algorithm. Pattern Recognit 41(8):2470–2483

    Article  MATH  Google Scholar 

Download references

Acknowledgements

This work is supported by The National High Technology Research and Development Program of China (2007AA01Z334), National Natural Science Foundation of China (61272219, 61021062 and 61100110), Program for New Century Excellent Talents in University of China (NCET-04-04605), Natural Science Foundation of Jiangsu Province (BK2010375), Key Technology R&D Program of Jiangsu Province (BY2012190, BE2010072 and BE2011058).

Author information

Authors and Affiliations

  1. State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China

    Yi Li & Zhengxing Sun

Authors
  1. Yi Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhengxing Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhengxing Sun.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Li, Y., Sun, Z. Generative tracking of 3D human motion in latent space by sequential clonal selection algorithm. Multimed Tools Appl 69, 79–109 (2014). https://doi.org/10.1007/s11042-012-1251-5

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-012-1251-5

Keywords

Search

Navigation