Histogram-Based Description of Local Space-Time Appearance

  • Karla Brkić
  • Axel Pinz
  • Siniša Šegvić
  • Zoran Kalafatić
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6688)


We introduce a novel local spatio-temporal descriptor intended to model the spatio-temporal behavior of a tracked object of interest in a general manner. The basic idea of the descriptor is the accumulation of histograms of an image function value through time. The histograms are calculated over a regular grid of patches inside the bounding box of the object and normalized to represent empirical probability distributions. The number of grid patches is fixed, so the descriptor is invariant to changes in spatial scale. Depending on the temporal complexity/details at hand, we introduce “first order STA descriptors” that describe the average distribution of a chosen image function over time, and “second order STA descriptors” that model the distribution of each histogram bin over time. We discuss entropy and χ 2 as well-suited similarity and saliency measures for our descriptors. Our experimental validation ranges from the patch- to the object-level. Our results show that STA, this simple, yet powerful novel description of local space-time appearance is well-suited to machine learning and will be useful in video-analysis, including potential applications of object detection, tracking, and background modeling.


Feature Vector Random Forest Video Sequence Interest Point Scale Space Theory 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Brkić, K., Pinz, A., Šegvić, S.: Traffic sign detection as a component of an automated traffic infrastructure inventory system. In: Proc. 33rd ÖAGM Workshop, Stainz, Austria (May 2009)Google Scholar
  2. 2.
    Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: Proc. CVPR (2005)Google Scholar
  3. 3.
    Dollár, P., Rabaud, V., Cottrell, G., Belongie, S.: Behavior recognition via sparse spatio-temporal features. In: VS-PETS (October 2005)Google Scholar
  4. 4.
    Holzer, P., Pinz, A.: Mobile surveillance by 3d-outlier analysis. In: Proc. ACCV Workshop on Visual Surveillance (2010)Google Scholar
  5. 5.
    Kadir, T., Brady, M.: Scale, saliency and image description. Int. J. Computer Vision 45(2), 83–105 (2001)CrossRefzbMATHGoogle Scholar
  6. 6.
    Ke, Y., Sukthankar, R., Hebert, M.: Efficient visual event detection using volumetric features. In: Proc. ICCV, vol. 1, pp. 166–173 (October 2005)Google Scholar
  7. 7.
    Kläser, A., Marszałek, M., Schmid, C.: A spatio-temporal descriptor based on 3d-gradients. In: British Machine Vision Conference, pp. 995–1004 (September 2008)Google Scholar
  8. 8.
    Laptev, I., Lindeberg, T.: Space-time interest points. In: Proc. ICCV (2003)Google Scholar
  9. 9.
    Laptev, I., Perez, P.: Retrieving actions in movies. In: Proc. ICCV, pp. 1–8 (2007)Google Scholar
  10. 10.
    Lindeberg, T.: Scale Space theory in Computer Vision. Kluwer, Dordrecht (1994)CrossRefzbMATHGoogle Scholar
  11. 11.
    Lowe, D.: Distinctive image features from scale-invariant keypoints. Int. J. Computer Vision (2), 91–110 (2004)Google Scholar
  12. 12.
    Luo, Q., Kong, X., Zeng, G., Fan, J.: Human action detection via boosted local motion histograms. Mach. Vision Appl. 21, 377–389 (2010)CrossRefGoogle Scholar
  13. 13.
    Matas, J., Chum, O., Urban, M., Pajdla, T.: Robust wide baseline stereo from maximally stable extremal regions. In: Proc. 13th BMVC, pp. 384–393 (2002)Google Scholar
  14. 14.
    Mikolajczyk, K., Schmid, C.: A performance evaluation of local descriptors. IEEE Transactions on Pattern Analysis & Machine Intelligence 27(10), 1615–1630 (2005)CrossRefGoogle Scholar
  15. 15.
    Obdržálek, S., Matas, J.: Object recognition using local affine frames on distinghuished regions. In: Proc. 13th BMVC, pp. 113–122 (2002)Google Scholar
  16. 16.
    Ozden, K., Schindler, K., van Gool, L.: Multibody structure-from-motion in practice. IEEE PAMI 32(6), 1134–1141 (2010)CrossRefGoogle Scholar
  17. 17.
    Šegvić, S., Remazeilles, A., Chaumette, F.: Enhancing the point feature tracker by adaptive modelling of the feature support. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3952, pp. 112–124. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  18. 18.
    Shi, J., Tomasi, C.: Good features to track. In: Proc. CVPR, pp. 593–600 (1994)Google Scholar
  19. 19.
    Viola, P., Jones, M.J., Snow, D.: Detecting pedestrians using patterns of motion and appearance. Int. J. Computer Vision 63, 153–161 (2005)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Karla Brkić
    • 1
  • Axel Pinz
    • 2
  • Siniša Šegvić
    • 1
  • Zoran Kalafatić
    • 1
  1. 1.Faculty of Electrical Engineering and ComputingUniversity of ZagrebCroatia
  2. 2.Graz University of TechnologyAustria

Personalised recommendations