View and Style-Independent Action Manifolds for Human Activity Recognition

  • Michał Lewandowski
  • Dimitrios Makris
  • Jean-Christophe Nebel
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6316)


We introduce a novel approach to automatically learn intuitive and compact descriptors of human body motions for activity recognition. Each action descriptor is produced, first, by applying Temporal Laplacian Eigenmaps to view-dependent videos in order to produce a stylistic invariant embedded manifold for each view separately. Then, all view-dependent manifolds are automatically combined to discover a unified representation which model in a single three dimensional space an action independently from style and viewpoint. In addition, a bidirectional nonlinear mapping function is incorporated to allow projecting actions between original and embedded spaces. The proposed framework is evaluated on a real and challenging dataset (IXMAS), which is composed of a variety of actions seen from arbitrary viewpoints. Experimental results demonstrate robustness against style and view variation and match the most accurate action recognition method.


Action Recognition Human Action Recognition Action Descriptor Visual Hull Nonlinear Dimensionality Reduction 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Ali, S., Basharat, A., Shah, M.: Chaotic invariants for human action recognition. In: ICCV, pp. 1–8 (2007)Google Scholar
  2. 2.
    Belkin, M., Niyogi, P.: Laplacian eigenmaps and spectral techniques for embedding and clustering. NIPS 14, 585–591 (2001)Google Scholar
  3. 3.
    Bobick, A., Davis, J.: The recognition of human movement using temporal templates. TPAMI 23(3), 257–267 (2001)Google Scholar
  4. 4.
    Cutler, R., Davis, L.: Robust real-time periodic motion detection, analysis, and applications. TPAMI 22(8), 781–796 (2000)Google Scholar
  5. 5.
    Elgammal, A., Lee, C.S.: Tracking people on a torus. TPAMI 31(3), 520–538 (2009)Google Scholar
  6. 6.
    Elgammal, A., Lee, C.S.: Separating style and content on a nonlinear manifold. In: CVPR, vol. 1 (2004)Google Scholar
  7. 7.
    Gorelick, L., Blank, M., Shechtman, E., Irani, M., Basri, R.: Actions as space-time shapes. TPAMI 29(12), 2247 (2007)Google Scholar
  8. 8.
    Junejo, I., Dexter, E., Laptev, I., Pérez, P.: Cross-view action recognition from temporal self-similarities. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part II. LNCS, vol. 5303, pp. 293–306. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  9. 9.
    Laptev, I.: On space-time interest points. IJCV 64(2), 107–123 (2005)CrossRefMathSciNetGoogle Scholar
  10. 10.
    Lathauwer, L., Moor, B., Vandewalle, J.: A multilinear singular value decomposition. SIAM J. Matrix Anal. Appl. 21(4), 1253–1278 (2000)zbMATHCrossRefMathSciNetGoogle Scholar
  11. 11.
    Lawrence, N.: Gaussian process latent variable models for visualisation of high dimensional data. In: NIPS, vol. 16 (2004)Google Scholar
  12. 12.
    Lee, C., Elgammal, A.: Homeomorphic manifold analysis: Learning decomposable generative models for human motion analysis. In: Vidal, R., Heyden, A., Ma, Y. (eds.) WDV 2005/2006. LNCS, vol. 4358, pp. 100–114. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  13. 13.
    Lewandowski, M., Makris, D., Nebel, J.C.: Average recognition rates using single views. (2010); supplied as additional material, avgrecrates.tif
  14. 14.
    Lewandowski, M., Martinez-del-Rincon, J., Makris, D., Nebel, J.-C.: Temporal extension of laplacian eigenmaps for unsupervised dimensionality reduction of time series. In: Proc. ICPR (2010)Google Scholar
  15. 15.
    Liu, J., Ali, S., Shah, M.: Recognizing human actions using multiple features. In: CVPR (2008)Google Scholar
  16. 16.
    Liu, J., Shah, M.: Learning human actions via information maximization. In: CVPR (2008)Google Scholar
  17. 17.
    Myronenko, A., Song, X., Carreira-Perpinán, M.: Non-rigid point set registration: Coherent Point Drift. In: NIPS, vol. 19, p. 1009 (2007)Google Scholar
  18. 18.
    Ogale, A., Karapurkar, A., Aloimonos, Y.: View-invariant modeling and recognition of human actions using grammars. In: W. on Dyn. Vis. at ICCV, vol. 5 (2005)Google Scholar
  19. 19.
    Parameswaran, V., Chellappa, R.: View invariance for human action recognition. IJCV 66(1), 83–101 (2006)CrossRefGoogle Scholar
  20. 20.
    Rabiner, L., Juang, B.H.: Fundamentals of speech recognition (1993)Google Scholar
  21. 21.
    Richard, S., Kyle, P.: Viewpoint Manifolds for Action Recognition. EURASIP J. on Img. and Vid. Proc. 2009 (2009)Google Scholar
  22. 22.
    Roweis, S., Saul, L.: Nonlinear dimensionality reduction by locally linear embedding. Science 290(5500), 2323–2326 (2000)CrossRefGoogle Scholar
  23. 23.
    Tenenbaum, J., Silva, V., Langford, J.: A global geometric framework for nonlinear dimensionality reduction. Science 290(5500), 2319–2323 (2000)CrossRefGoogle Scholar
  24. 24.
    Wang, C., Mahadevan, S.: Manifold alignment using Procrustes analysis. In: ICML, pp. 1120–1127 (2008)Google Scholar
  25. 25.
    Weinland, D., Boyer, E., Ronfard, R.: Action recognition from arbitrary views using 3d exemplars. In: ICCV, vol. 5(7), p. 8 (2007)Google Scholar
  26. 26.
    Weinland, D., Ronfard, R., Boyer, E.: Automatic discovery of action taxonomies from multiple views. In: CVPR, vol. 2, pp. 1639–1645 (2006)Google Scholar
  27. 27.
    Weinland, D., Ronfard, R., Boyer, E.: Free viewpoint action recognition using motion history volumes. Computer Vision and Image Understanding 104(2-3), 249–257 (2006)CrossRefGoogle Scholar
  28. 28.
    Yan, P., Khan, S., Shah, M.: Learning 4D action feature models for arbitrary view action recognition. In: CVPR, vol. 12 (2008)Google Scholar
  29. 29.
    Yilmaz, A., Shah, M.: Recognizing human actions in videos acquired by uncalibrated moving cameras. In: ICCV, vol. 1, pp. 150–157 (2005)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Michał Lewandowski
    • 1
  • Dimitrios Makris
    • 1
  • Jean-Christophe Nebel
    • 1
  1. 1.Digital Imaging Research CentreKingston UniversityLondonUnited Kingdom

Personalised recommendations