Anomalous Behaviour Detection Using Spatiotemporal Oriented Energies, Subset Inclusion Histogram Comparison and Event-Driven Processing

  • Andrei Zaharescu
  • Richard Wildes
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6311)


This paper proposes a novel approach to anomalous behaviour detection in video. The approach is comprised of three key components. First, distributions of spatiotemporal oriented energy are used to model behaviour. This representation can capture a wide range of naturally occurring visual spacetime patterns and has not previously been applied to anomaly detection. Second, a novel method is proposed for comparing an automatically acquired model of normal behaviour with new observations. The method accounts for situations when only a subset of the model is present in the new observation, as when multiple activities are acceptable in a region yet only one is likely to be encountered at any given instant. Third, event driven processing is employed to automatically mark portions of the video stream that are most likely to contain deviations from the expected and thereby focus computational efforts. The approach has been implemented with real-time performance. Quantitative and qualitative empirical evaluation on a challenging set of natural image videos demonstrates the approach’s superior performance relative to various alternatives.


Anomaly Detection Anomalous Behaviour Partial Match Dynamic Texture Total Frame 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Supplementary material

978-3-642-15549-9_41_MOESM1_ESM.mp4 (14.1 mb)
Electronic Supplementary Material (52 KB)


  1. 1.
  2. 2.
    Adam, A., Rivlin, E., Shimshoni, I., Reinitz, D.: Robust real-time unusual event detection using multiple fixed-location monitors. PAMI 30, 555–560 (2008)Google Scholar
  3. 3.
    Affifi, A., Azen, S.: Statistical Analysis. Academic (1979)Google Scholar
  4. 4.
    Andrade, E., Blunsden, S., Fisher, R.: Modelling crowd scenes for event detection. In: ICPR, pp. 175–178 (2006)Google Scholar
  5. 5.
    Basharat, A., Gritai, A., Shah, M.: Learning object motion patterns for anomaly detection and improved object detection. In: CVPR (2008)Google Scholar
  6. 6.
    Bebezeth, Y., Jodoin, P., Saligrama, V., Rosenberger, C.: Abnormal events detection based on spatio-temporal co-occurences. In: CVPR, pp. 2458–2465 (2009)Google Scholar
  7. 7.
    Black, M.: Explaining optical flow events with parameterized spatio-temporal models. In: CVPR, pp. 326–332 (1999)Google Scholar
  8. 8.
    Boiman, O., Irani, M.: Detecting irregularities in images and in video. IJCV 74, 17–31 (2007)CrossRefGoogle Scholar
  9. 9.
    Buxton, H.: Learning and understanding dynamic scene activity: A review. IVC 23 (2003)Google Scholar
  10. 10.
    Chomat, O., Crowley, J.: Probabilistic recognition of activity using local appearance. In: CVPR, pp. 104–109 (September 1999)Google Scholar
  11. 11.
    Derpanis, K., Sizintsev, M., Cannons, K., Wildes, R.: Efficient action spotting based on a spacetime oriented structure representation. In: CVPR (2010)Google Scholar
  12. 12.
    Derpanis, K., Wildes, R.: Early spatiotemporal grouping with a distributed oriented energy representation. In: CVPR (June 2009)Google Scholar
  13. 13.
    Dollar, P., Rabaud, V., Cottrell, G., Belongie, S.: Behaviour recognition via sparse spatio-temporal features. In: PETS, pp. 65–72 (2005)Google Scholar
  14. 14.
    Elgammal, A., Durauswami, R., Harwood, D., Davis, L.: Background and foreground modeling using nonparametric kernel density for visual surveillance. Proc. IEEE 90, 1151–1163 (2002)CrossRefGoogle Scholar
  15. 15.
    Freeman, W., Adelson, E.: Design and use of steerable filters. PAMI 13, 891–906 (1991)Google Scholar
  16. 16.
    Granlund, G., Knuttson, H.: Signal Processing for Computer Vision. Kluwer, Dordrecht (1995)Google Scholar
  17. 17.
    Heikkila, M., Pietkainin, M.: A texture-based method for modeling the background and detecting moving objects. PAMI 28, 657–662 (2006)Google Scholar
  18. 18.
    Hu, W., Xian, X., Fu, Z., Xie, D., Tan, T., Maybank, S.: System for learning statistical motion patterns. PAMI 28, 1450–1464 (2006)Google Scholar
  19. 19.
    Jahne, B.: Digital Image Processing. Springer, Heidelberg (2005)Google Scholar
  20. 20.
    Javed, O., Shafique, K., Shah, M.: A hierarchical approach to robust background subtraction using color and gradient information. In: Motion Workshop, pp. 22–27 (2003)Google Scholar
  21. 21.
    Jodoin, P.M., Konrad, J., Saligrama, V.: Modeling background activity for behavior subtraction. In: ICDSC, pp. 1–10 (2008)Google Scholar
  22. 22.
    Johnson, N., Hogg, D.: Learning the distribution of object trajectories for event recognition. IVC 14, 609–615 (1996)Google Scholar
  23. 23.
    Ke, Y., Sukthankar, R., Hebert, M.: Event detection in crowded videos. In: ICCV (2007)Google Scholar
  24. 24.
    Kim, J., Grauman, K.: Observe locally, infer globally: A space-time MRF for detecting abnormal activities with incremental updates. In: CVPR, pp. 2921–2929 (2009)Google Scholar
  25. 25.
    Kratz, L., Nishino, K.: Anomaly detection in extremely crowded scenes using spatio-temporal motion pattern models. In: CVPR, pp. 1446–1453 (2009)Google Scholar
  26. 26.
    Li, L., Gong, S., Xiang, T.: Global behaviour inference using probabilistic latent semantic analysis. In: BMVC (2008)Google Scholar
  27. 27.
    McIvor, A.: Background subtraction techniques. In: Proc. Vid. And Img. Comp., New Zealand (2000)Google Scholar
  28. 28.
    Mehran, R., Oyama, A., Shah, M.: Abnormal crowd behavior detection using a social force model. In: CVPR (2009)Google Scholar
  29. 29.
    Mittal, A., Monnet, A., Paragios, N.: Scene modeling and change detection in dynamic scences: A subspace approach. CVIU 113, 63–79 (2009)Google Scholar
  30. 30.
    Pless, R.: Spatio-temporal background models for outdoor surveillance. In: EURASIP (2005)Google Scholar
  31. 31.
    Stauffer, C., Grimson, E.: Learning patterns of activity using real-time tracking. PAMI 22, 747–757 (2000)Google Scholar
  32. 32.
    Wang, X., Ma, X., Grimson, E.: Unsupervised activity perception by hierarchical bayesian models. In: CVPR (2007)Google Scholar
  33. 33.
    Watson, B., Ahumada, A.: A look at motion in the frequency domain. In: Motion Workshop. pp. 1–10 (1983)Google Scholar
  34. 34.
    Wildes, R., Bergen, J.: Qualitative spatiotemporal analysis using an oriented energy representation. In: Vernon, D. (ed.) ECCV 2000. LNCS, vol. 1843, pp. 768–784. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  35. 35.
    Zhong, H., Shi, J., Visontai, M.: Detecting unusual activity in video. In: CVPR (2004)Google Scholar
  36. 36.
    Zhong, J., Sclaroff, S.: Segmenting foreground objects from a dynamic textured background using a robust Kalman filter. In: ICCV, pp. 44–50 (2003)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Andrei Zaharescu
    • 1
    • 2
  • Richard Wildes
    • 2
  1. 1.Aimetis CorporationWaterlooCanada
  2. 2.Department of Computer Science and EngineeringYork UniversityTorontoCanada

Personalised recommendations