Learning Behavioral Patterns of Time Series for Video-Surveillance

  • Nicoletta Noceti
  • Matteo Santoro
  • Francesca Odone
Part of the Advances in Pattern Recognition book series (ACVPR)


This chapter deals with the problem of learning behaviors of people activities from (possibly big) sets of visual dynamic data, with a specific reference to video-surveillance applications. The study focuses mainly on devising meaningful data abstractions able to capture the intrinsic nature of the available data, and applying similarity measures appropriate to the specific representations. The methods are selected among the most promising techniques available in the literature and include classical curve fitting, string-based approaches, and hidden Markov models. The analysis considers both supervised and unsupervised settings and is based on a set of loosely labeled data acquired by a real video-surveillance system. The experiments highlight different peculiarities of the methods taken into consideration, and the final discussion guides the reader towards the most appropriate choice for a given scenario.


Hide Markov Model Input Space Spectral Cluster Temporal Series Input Representation 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



The authors would like to thank Annalisa Barla and Luca Baldassarre for the code on multi-class RLS and the help given in running the experiments for the supervised case. The low-level processing modules are on going joint work with Augusto Destrero and Alberto Lovato.

Matteo Santoro is supported by Compagnia di San Paolo (Torino) through the Neuroscience Program for the project Action representations and their impairment.


  1. 1.
    Anjum, N., Cavallaro, A.: Multifeature object trajectory clustering for video analysis. IEEE Trans. Circuits Syst. Video Technol. 18(11), 1555–1564 (2008) CrossRefGoogle Scholar
  2. 2.
    Bashir, F., Khokhar, A., Schonfeld, D.: Object trajectory-based activity classification and recognition using hidden Markov model. IEEE Trans. Image Process. 16, 1912–1919 (2007) MathSciNetCrossRefGoogle Scholar
  3. 3.
    Bolshakova, N., Azuaje, F.: Cluster validation techniques for genome expression data. Signal Process. 83(4), 825–833 (2003) MATHCrossRefGoogle Scholar
  4. 4.
    Comaniciu, D., Ramesh, V., Meer, P.: Kernel-based object tracking. IEEE Trans. Pattern Anal. Mach. Intell. 25(5), 564–575 (2003) CrossRefGoogle Scholar
  5. 5.
    Doretto, G., Chiuso, A., Wu, Y., Soatto, S.: Dynamic textures. Int. J. Comput. Vis. 51(2), 91–109 (2003) MATHCrossRefGoogle Scholar
  6. 6.
    Fowlkes, C., Belongie, S., Chung, F.R.K., Malik, J.: Spectral grouping using the Nyström method. IEEE Trans. Pattern Anal. Mach. Intell. 26(2), 214–225 (2004) CrossRefGoogle Scholar
  7. 7.
    Günter, S., Bunke, H.: Validation indices for graph clustering. Pattern Recogn. Lett. 24(8), 1107–1113 (2003) MATHCrossRefGoogle Scholar
  8. 8.
    Halkidi, M., Batistakis, Y., Vazirgiannis, M.: On clustering validation techniques. J. Intell. Inf. Syst. 17, 107–145 (2001) MATHCrossRefGoogle Scholar
  9. 9.
    Hu, W., Xiao, X., Fu, Z., Xie, D., Tan, T., Maybank, S.: A system for learning statistical motion patterns. IEEE Trans. Pattern Anal. Mach. Intell. 28(9), 1450–1464 (2006) CrossRefGoogle Scholar
  10. 10.
    Ivanov, Y., Bobick, A.: Recognition of visual activities and interactions by stochastic parsing. IEEE Trans. Pattern Anal. Mach. Intell. 22(8), 852–872 (2000) CrossRefGoogle Scholar
  11. 11.
    Fei-Fei, L., Niebles, J.C., Wang, H.: Unsupervised learning of human action categories using spatial-temporal words. In: Proc. of BMVC (2006) Google Scholar
  12. 12.
    Jebara, T., Kondor, R.I., Howard, A.: Probability product kernels. J. Mach. Learn. Res. 5, 819–844 (2004) MathSciNetMATHGoogle Scholar
  13. 13.
    Jebara, T., Song, Y., Thadani, K.: Spectral clustering and embedding with hidden Markov models. In: Kok, J.N., Koronacki, J., López de Mántaras, R., Matwin, S., Mladenic, D., Skowron, A. (eds.) ECML. Lecture Notes in Computer Science, vol. 4701, pp. 164–175. Springer, Berlin (2007) Google Scholar
  14. 14.
    Liao, T.W.: Clustering of time series data: a survey. Pattern Recogn. 38(11) (2005) Google Scholar
  15. 15.
    Lin, W., Orgun, M.A., Williams, G.J.: Temporal data mining using multilevel-local polynomial models. In: International Conference IDEAL. Lecture Notes in Computer Science, vol. 1983. Springer, Berlin (2000) Google Scholar
  16. 16.
    Lo Gerfo, L., Rosasco, L., Odone, F., De Vito, E., Verri, A.: Spectral algorithms for supervised learning. Neural Comput. 20, 1873–1897 (2008) MathSciNetMATHCrossRefGoogle Scholar
  17. 17.
    Medioni, G., Cohen, I., Bremond, F., Hongeng, S., Nevatia, R.: Event detection and analysis from video streams. IEEE Trans. Pattern Anal. Mach. Intell. 23, 873–889 (2001) CrossRefGoogle Scholar
  18. 18.
    Morris, B., Trivedi, M.M.: Learning trajectory patterns by clustering: experimental studies and comparative evaluation. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR ’09 (2009) Google Scholar
  19. 19.
    Noceti, N., Santoro, M., Odone, F.: String-based spectral clustering for understanding human behaviours. In: THEMIS-BMVC (2008) Google Scholar
  20. 20.
    Noceti, N., Santoro, M., Odone, F.: Unsupervised learning of behavioural patterns for video-surveillance. In: Workshop MLVMA-ECCV (2008) Google Scholar
  21. 21.
    Noceti, N., Destrero, A., Lovato, A., Odone, F.: Combined motion and appearance models for robust object tracking in real-time. In: AVSS (2009) Google Scholar
  22. 22.
    Piciarelli, C., Micheloni, C., Foresti, G.L.: Trajectory-based anomalous event detection. IEEE Trans. Circuits Syst. Video Technol. 18(11), 1544–1554 (2008) CrossRefGoogle Scholar
  23. 23.
    Pittore, M., Campani, M., Verri, A.: Learning to recognize visual dynamic events from examples. In: IJCV (2000) Google Scholar
  24. 24.
    Rabiner, L.R.: A tutorial on hidden Markov models and selected applications in speech recognition. Proc. IEEE 77, 257–286 (1989) CrossRefGoogle Scholar
  25. 25.
    Rieck, K., Laskov, P.: Linear-time computation of similarity measures for sequential data. J. Mach. Learn. Res. 9, 23–48 (2008) MATHGoogle Scholar
  26. 26.
    Shawe Taylor, J., Cristianini, N.: Kernel Methods for Pattern Analysis. Cambridge University Press, Cambridge (2004) CrossRefGoogle Scholar
  27. 27.
    Shi, J., Malik, J.: Normalized cuts and image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 22(8), 888–905 (2000) CrossRefGoogle Scholar
  28. 28.
    Special issue on event analysis in videos. IEEE Trans. Circuits Syst. Video Technol. 18(11) (2008) Google Scholar
  29. 29.
    Stauffer, C., Grimson, E.: Learning patterns of activity using real-time tracking. IEEE Trans. Pattern Recogn. Mach. Intell. 22(8), 747–757 (2000) CrossRefGoogle Scholar
  30. 30.
    Tommasi, T., Orabona, F., Caputo, B.: Discriminative cue integration for medical image annotation. Pattern Recognit. Lett. 29(15), 1996–2002 (2008) CrossRefGoogle Scholar
  31. 31.
    Vishwanathan, S.V., Smola, A.J., Vidal, R.: Binet–Cauchy kernels on dynamical systems and its application to the analysis of dynamic scenes. Int. J. Comput. Vis. 73(1), 95–119 (2007) CrossRefGoogle Scholar

Copyright information

© Springer-Verlag London Limited 2011

Authors and Affiliations

There are no affiliations available

Personalised recommendations