STOP: Space-Time Occupancy Patterns for 3D Action Recognition from Depth Map Sequences

  • Antonio W. Vieira
  • Erickson R. Nascimento
  • Gabriel L. Oliveira
  • Zicheng Liu
  • Mario F. M. Campos
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7441)


This paper presents Space-Time Occupancy Patterns (STOP), a new visual representation for 3D action recognition from sequences of depth maps. In this new representation, space and time axes are divided into multiple segments to define a 4D grid for each depth map sequence. The advantage of STOP is that it preserves spatial and temporal contextual information between space-time cells while being flexible enough to accommodate intra-action variations. Our visual representation is validated with experiments on a public 3D human action dataset. For the challenging cross-subject test, we significantly improved the recognition accuracy from the previously reported 74.7% to 84.8%. Furthermore, we present an automatic segmentation and time alignment method for online recognition of depth sequences.


Pattern recognition Machine Learning Human action 


  1. 1.
    Carnegie mellon university motion capture database,
  2. 2.
    Breuer, P., Eckes, C., Müller, S.: Hand Gesture Recognition with a Novel IR Time-of-Flight Range Camera–A Pilot Study. In: Gagalowicz, A., Philips, W. (eds.) MIRAGE 2007. LNCS, vol. 4418, pp. 247–260. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  3. 3.
    Dollar, P., Rabaud, V., Cottrell, G., Belongie, S.: Behavior recognition via sparse spatio-temporal features. In: IEEE Int. Workshop on Visual Surveillance and Performance Evaluation of Tracking and Surveillance, San Diego, CA (2005)Google Scholar
  4. 4.
    Gorelick, L., Blank, M., Shechtman, E., Irani, M., Basri, R.: Actions as space-time shapes. IEEE Trans. PAMI 29(12) (2007)Google Scholar
  5. 5.
    Gu, J., Ding, X., Wang, S., Wu, Y.: Action and gait recognition from recovered 3D human joints. IEEE Trans. on Systems, Man, and Cybernetics-Part B: Cybernetics 40(4) (2010)Google Scholar
  6. 6.
    Han, L., Wu, X., Liang, W., Hou, G., Jia, Y.: Discriminative human action recognition in the learned hierarchical manifold space. Image Vision Comput. 28, 836–849 (2010)CrossRefGoogle Scholar
  7. 7.
    Iddan, G.J., Yahav, G.: 3D imaging in the studio. In: Proc. SPIE 4298 (2001)Google Scholar
  8. 8.
    Li, W., Zhang, Z., Liu, Z.: Expandable data-driven graphical modeling of human actions based on salient postures. IEEE Transactions on Circuits and Systems for Video Technology 18(11) (2008)Google Scholar
  9. 9.
    Li, W., Zhang, Z., Liu, Z.: Action recognition based on a bag of 3D points. In: CVPR Workshop for Human Communicative Behavior Analysis, San Francisco, CA (June 2010)Google Scholar
  10. 10.
    Lv, F., Nevatia, R.: Single view human action recognition using key pose matching and Viterbi path searching. In: Proc. CVPR (2007)Google Scholar
  11. 11.
    Malassiotis, S., Tsalakanidou, F., Mavridis, N., Giagourta, V., Grammalidis, N., Strintzis, M.G.: A face and gesture recognition system based on an active stereo sensor. In: Proc. ICPR, Thessaloniki, Greece, vol. 3 (October 2001)Google Scholar
  12. 12.
    Oliveira, G.L., Nascimento, E.R., Vieira, A.W., Campos, M.F.M.: Sparse spatial coding: A novel approach for efficient and accurate object recognition. In: Proc. ICRA, St. Paul, MN (May 2012)Google Scholar
  13. 13.
    Sun, J., Wu, X., Yan, S., Cheong, L., Chua, T., Li, J.: Hierarchical spatio-temporal context modeling for action recognition. In: Proc. CVPR, Miami, FL (June 2009)Google Scholar
  14. 14.
    Weinland, D., Ronfard, R., Boyer, E.: Free viewpoint action recognition using motion history volumes 104(2) (2006)Google Scholar
  15. 15.
    Yilmaz, A., Shah, M.: Actions sketch: a novel action representation. In: Proc. CVPR, vol. 1 (2005)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Antonio W. Vieira
    • 1
    • 2
  • Erickson R. Nascimento
    • 1
  • Gabriel L. Oliveira
    • 1
  • Zicheng Liu
    • 3
  • Mario F. M. Campos
    • 1
  1. 1.DCC - Universidade Federal de Minas GeraisBelo HorizonteBrazil
  2. 2.CCET - UnimontesMontes ClarosBrazil
  3. 3.Microsoft ResearchRedmondUSA

Personalised recommendations