Human Action Recognition with Hierarchical Growing Neural Gas Learning

  • German Ignacio Parisi
  • Cornelius Weber
  • Stefan Wermter
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8681)


We propose a novel biologically inspired framework for the recognition of human full-body actions. First, we extract body pose and motion features from depth map sequences. We then cluster pose-motion cues with a two-stream hierarchical architecture based on growing neural gas (GNG). Multi-cue trajectories are finally combined to provide prototypical action dynamics in the joint feature space. We extend the unsupervised GNG with two labelling functions for classifying clustered trajectories. Noisy samples are automatically detected and removed from the training and the testing set. Experiments on a set of 10 human actions show that the use of multi-cue learning leads to substantially increased recognition accuracy over the single-cue approach and the learning of joint pose-motion vectors.


human action recognition growing neural gas motion clustering assistive system 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Roy, P.C., Bouzouane, A., Giroux, S., Bouchard, B.: Possibilistic Activity Recognition in Smart Homes for Cognitively Impaired People. Applied Artificial Intelligence: An International Journal 25, 883–926 (2011)CrossRefGoogle Scholar
  2. 2.
    Suarez, J., Murphy, R.: Hand Gesture Recognition with Depth Images: A review. In: IEEE Int. Symposium on Robot and Human Interactive Communication, France, pp. 411–417 (2012)Google Scholar
  3. 3.
    Xu, R., Agarwal, P., Kumar, S., Krovi, V.N., Corso, J.J.: Combining Skeletal Pose with Local Motion for Human Activity Recognition. In: Perales, F.J., Fisher, R.B., Moeslund, T.B. (eds.) AMDO 2012. LNCS, vol. 7378, pp. 114–123. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  4. 4.
    Papadopoulos, G.T., Axenopoulos, A., Daras, P.: Real-Time Skeleton-Tracking-Based Human Action Recognition Using Kinect Data. In: Gurrin, C., Hopfgartner, F., Hurst, W., Johansen, H., Lee, H., O’Connor, N. (eds.) MMM 2014, Part I. LNCS, vol. 8325, pp. 473–483. Springer, Heidelberg (2014)CrossRefGoogle Scholar
  5. 5.
    Giese, M.A., Poggio, T.: Neural Mechanisms for the Recognition of Biological Movements. Nature Reviews Neuroscience 4, 179–192 (2003)CrossRefGoogle Scholar
  6. 6.
    Escobar, M.-J., Kornprobst, P.: Action Recognition with a Bioinspired Feedforward Motion Processing Model: The Richness of Center-Surround Interactions. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part IV. LNCS, vol. 5305, pp. 186–199. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  7. 7.
    Kohonen, T.: Self-organizing Maps. Series in Information Sciences, vol. 30. Springer, Heidelberg (1995)Google Scholar
  8. 8.
    Fritzke, B.: A Growing Neural Gas Network Learns Topologies. In: Advances in Neural Information Processing Systems, vol. 7, pp. 625–632. MIT Press (1995)Google Scholar
  9. 9.
    Martinetz, T., Schluten, K.: A ”neural-gas” network learns topologies. In: Artificial Neural Networks, pp. 397–402. Elsevier (1991)Google Scholar
  10. 10.
    Jiang, Z., Lin, Z., Davis, L.S.: Recognizing Human Actions by Learning and Matching Shape-Motion Prototype Trees. IEEE Transactions on Pattern Analysis and Machine Intelligence 31(3), 533–547 (2012)CrossRefGoogle Scholar
  11. 11.
    Parisi, G.I., Wermter, S.: Hierarchical SOM-based Detection of Novel Behavior for 3D Human Tracking. In: IEEE Int. Joint Conf. on Neural Networks (IJCNN), USA, pp. 1380–1387 (2013)Google Scholar
  12. 12.
    Parisi, G.I., Barros, P., Wermter, S.: FINGeR: Framework for Interactive Neural-based Gesture Recognition. In: European Symposium of Artificial Neural Networks (ESANN), Belgium, pp. 443–447 (2014)Google Scholar
  13. 13.
    Miikkulainen, R., Bednar, J.A., Choe, Y., Sirosh, J.: Computational Maps in the Visual Cortex. Springer, New York (2005)Google Scholar
  14. 14.
    Beyer, O., Cimiano, P.: Online Labelling Strategies for Growing Neural Gas. In: Yin, H., Wang, W., Rayward-Smith, V. (eds.) IDEAL 2011. LNCS, vol. 6936, pp. 76–83. Springer, Heidelberg (2011)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • German Ignacio Parisi
    • 1
  • Cornelius Weber
    • 1
  • Stefan Wermter
    • 1
  1. 1.Department of Computer ScienceUniversity of HamburgHamburgGermany

Personalised recommendations