Comparing Hidden Markov Models and Long Short Term Memory Neural Networks for Learning Action Representations

  • Maximilian PanznerEmail author
  • Philipp Cimiano
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10122)


In this paper we are concerned with learning models of actions and compare a purely generative model based on Hidden Markov Models to a discriminatively trained recurrent LSTM network in terms of their properties and their suitability to learn and represent models of actions. Specifically we compare the performance of the two models regarding the overall classification accuracy, the amount of training sequences required and how early in the progression of a sequence they are able to correctly classify the corresponding sequence. We show that, despite the current trend towards (deep) neural networks, traditional graphical model approaches are still beneficial under conditions where only few data points or limited computing power is available.


HMM LSTM Incremental learning Recurrent network Action classification 



This research/work was supported by the Cluster of Excellence Cognitive Interaction Technology ‘CITEC’ (EXC 277) at Bielefeld University, which is funded by the German Research Foundation (DFG).


  1. 1.
    Sugiura, K., Iwahashi, N., Kashioka, H., Nakamura, S.: Learning, generation and recognition of motions by reference-point-dependent probabilistic models. Adv. Robot. 25(6–7), 825–848 (2011)CrossRefGoogle Scholar
  2. 2.
    Tenorth, M., Beetz, M.: KnowRob - knowledge processing for autonomous personal robots. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 4261–4266 (2009)Google Scholar
  3. 3.
    Panzner, M., Beyer, O., Cimiano, P.: Human activity classification with online growing neural gas. In: Workshop on New Challenges in Neural Computation (NC2) (2013)Google Scholar
  4. 4.
    Veeriah, V., Zhuang, N., Qi, G.-J.: Differential recurrent neural networks for action recognition. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 4041–4049 (2015)Google Scholar
  5. 5.
    Rabiner, L., Juang, B.: An introduction to hidden Markov models. IEEE ASSP Mag. 3(1), 4–16 (1986)CrossRefGoogle Scholar
  6. 6.
    Weghe, N., Kuijpers, B., Bogaert, P., Maeyer, P.: A qualitative trajectory calculus and the composition of its relations. In: Rodríguez, M.A., Cruz, I., Levashkin, S., Egenhofer, M.J. (eds.) GeoS 2005. LNCS, vol. 3799, pp. 60–76. Springer, Heidelberg (2005). doi: 10.1007/11586180_5 CrossRefGoogle Scholar
  7. 7.
    Bruss, T., Rüschendorf, L.: On the perception of time. Gerontology 56(4), 361–370 (2010)CrossRefGoogle Scholar
  8. 8.
    Omohundro, S.: Best-first model merging for dynamic learning and recognition. In: Advances in Neural Information Processing Systems 4, pp. 958–965. Morgan Kaufmann (1992)Google Scholar
  9. 9.
    Stolcke, A., Omohundro, S.: Inducing probabilistic grammars by Bayesian model merging. In: Carrasco, R.C., Oncina, J. (eds.) ICGI 1994. LNCS, vol. 862, pp. 106–118. Springer, Heidelberg (1994). doi: 10.1007/3-540-58473-0_141 CrossRefGoogle Scholar
  10. 10.
    Shepard, R.N.: Toward a universal law of generalization for psychological science. Science 237(4820), 1317–1323 (1987)MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    Panzner, M., Cimiano, P.: Incremental learning of action models as HMMs over qualitative trajectory representations. In: Workshop on New Challenges in Neural Computation (NC2) (2015).
  12. 12.
    Greff, K., Srivastava, R.K., Koutník, J., Steunebrink, B.R., Schmidhuber, J.: LSTM: a search space odyssey. arXiv preprint arXiv:1503.04069 (2015)
  13. 13.
    Gers, F.A., Schmidhuber, J., Cummins, F.: Learning to forget: continual prediction with LSTM. Neural Comput. 12(10), 2451–2471 (2000)CrossRefGoogle Scholar
  14. 14.
    Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)CrossRefGoogle Scholar
  15. 15.
    Tieleman, T., Hinton, G.: Lecture 6.5-rmsprop: divide the gradient by a running average of its recent magnitude. COURSERA: Neural Netw. Mach. Learn. 4, 2 (2012)Google Scholar
  16. 16.
    Panzner, M., Gaspers, J., Cimiano, P.: Learning linguistic constructions grounded in qualitative action models. In: IEEE International Symposium on Robot and Human Interactive Communication (2015).
  17. 17.
    Panzner, M.: TLS Dataset (2016). doi: 10.4119/unibi/2904362

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  1. 1.Semantic Computing Group, CITECBielefeld UniversityBielefeldGermany

Personalised recommendations