Unsupervised Behaviour Profiling

  • Shaogang Gong
  • Tao Xiang


Given a large quantity of unprocessed videos of object activities, the goal of automatic behaviour profiling is to learn a model that is capable of detecting unseen abnormal behaviour patterns whilst recognising novel instances of expected normal behaviour patterns. In this context, an anomaly is defined as an atypical behaviour pattern that is not represented by sufficient examples in previous observations. Behaviour profiling is by unsupervised learning and anomaly detection is treated as a binary classification problem. One of the main challenges for a binary classification model is to differentiate a true anomaly from outliers that give false positives. In this chapter, we consider a clustering model that discovers the intrinsic grouping of behaviour patterns. The method does not require manual data labelling for either feature extraction or discovery of grouping. This is crucial because manual labelling of behaviour patterns is often impractical given the vast amount of video data, and is subject to inconsistency and error prone. The method performs incremental learning to cope with changes of behavioural context. It also detects anomalies on-line so that (a decision on whether a behaviour pattern is normal is made as soon as sufficient visual evidence is collected without the completion of the observed pattern.


Behaviour Pattern Mixture Component Anomaly Detection Dynamic Time Warping Affinity Matrix 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. Altman, D.G., Bland, J.M.: Diagnostic tests 1: sensitivity and specificity. Br. Med. J. 308(6943), 1552 (1994) CrossRefGoogle Scholar
  2. Berger, J.: Statistical Decision Theory and Bayesian Analysis. Springer, Berlin (1995) Google Scholar
  3. Bernstein, A., Mannor, S., Shimkin, N.: Online classification with specificity constraints. In: Advances in Neural Information Processing Systems, Vancouver, Canada, December 2010 Google Scholar
  4. Bishop, C.M.: Pattern Recognition and Machine Learning. Springer, Berlin (2006) MATHGoogle Scholar
  5. Dempster, A.P., Laird, N.M., Rubin, D.B.: Maximum likelihood from incomplete data via the EM algorithm. J. R. Stat. Soc. 39(1), 1–38 (1977) MathSciNetMATHGoogle Scholar
  6. Higgins, A., Bahler, L., Porter, J.: Speaker verification using randomized phrase prompting. Digit. Signal Process. 1, 89–106 (1991) CrossRefGoogle Scholar
  7. Kruskal, J.B., Liberman, M.: The Symmetric Time-warping Problem: From Continuous to Discrete. In: Sankoff, D., Kruskal, J. (eds.) Time Warps, String Edits, and Macromolecules: The Theory and Practice of Sequence Comparison, pp. 125–161. CSLI Publications, Standford (1983) Google Scholar
  8. Neal, R., Hinton, G.: A view of the EM algorithm that justifies incremental, sparse, and other variants. In: Jordan, M.I. (ed.) Learning in Graphical Models. Kluwer Academic, Dordrecht (1998) Google Scholar
  9. Ng, A.Y., Jordan, M.I., Weiss, Y.: On spectral clustering: analysis and an algorithm. In: Advances in Neural Information Processing Systems, Vancouver, Canada, December 2001, pp. 849–856 (2001) Google Scholar
  10. Ng, J., Gong, S.: Learning intrinsic video content using Levenshtein distance in graph partition. In: European Conference on Computer Vision, Copenhagen, Denmark, May 2002, pp. 670–684 (2002) Google Scholar
  11. Rabiner, L.R.: A tutorial on hidden Markov models and selected applications in speech recognition. Proc. IEEE 77(2), 257–286 (1989) CrossRefGoogle Scholar
  12. Roberts, S., Husmeier, D., Rezek, I., Penny, W.: Bayesian approaches to Gaussian mixture modelling. IEEE Trans. Pattern Anal. Mach. Intell. 20(11), 1133–1142 (1998) CrossRefGoogle Scholar
  13. Shan, Y., Sawhney, H.S., Pope, A.: Measuring the similarity of two image sequence. In: Asian Conference on Computer Vision, Jeju, Korea, January 2004 Google Scholar
  14. Shi, J., Malik, J.: Normalized cuts and image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 22(8), 888–905 (2000) CrossRefGoogle Scholar
  15. Weiss, Y.: Segmentation using eigenvectors: a unifying view. In: IEEE International Conference on Computer Vision, Corfu, Greece, pp. 975–982 (1999) CrossRefGoogle Scholar
  16. Wilpon, J., Rabiner, L.R., Lee, C., Goldman, E.: Automatic recognition of keywords in unconstrained speech using hidden Markov models. IEEE Trans. Acoust. Speech Signal Process. 38(11), 1870–1878 (1990) CrossRefGoogle Scholar
  17. Xiang, T., Gong, S.: Incremental and adaptive abnormal behaviour detection. Comput. Vis. Image Underst. 111(1), 59–73 (2008a) CrossRefGoogle Scholar
  18. Xiang, T., Gong, S.: Video behaviour profiling for anomaly detection. IEEE Trans. Pattern Anal. Mach. Intell. 30(5), 893–908 (2008b) CrossRefGoogle Scholar
  19. Zweig, M.H., Campbell, G.: Receiver-operating characteristic (ROC) plots: a fundamental evaluation tool in clinical medicine. Clin. Chem. 39(8), 561–577 (1993) Google Scholar

Copyright information

© Springer-Verlag London Limited 2011

Authors and Affiliations

  1. 1.School of Electronic Engineering and Computer ScienceQueen Mary University of LondonLondonUK
  2. 2.School of Electronic Engineering and Computer ScienceQueen Mary University of LondonLondonUK

Personalised recommendations