Advertisement

Spatiotemporal Deformable Prototypes for Motion Anomaly Detection

  • 981 Accesses

  • 1 Citations

Abstract

This paper presents an approach for motion-based anomaly detection, where a prototype pattern is detected and elastically registered against a test sample to detect anomalies in the test sample. The prototype model is learned from multiple sequences to define accepted variations. “Supertrajectories” based on hierarchical clustering of dense point trajectories serve as an efficient and robust representation of motion patterns. An efficient hashing approach provides transformation hypotheses that are refined by a spatiotemporal elastic registration. We propose a new method for elastic registration of 3D+time trajectory patterns that induces spatial elasticity from trajectory affinities. The method is evaluated on a new motion anomaly dataset of juggling patterns and performs well in detecting subtle anomalies. Moreover, we demonstrate the applicability to biological motion patterns.

This is a preview of subscription content, log in to check access.

Access options

Buy single article

Instant unlimited access to the full article PDF.

US$ 39.95

Price includes VAT for USA

Subscribe to journal

Immediate online access to all issues from 2019. Subscription will auto renew annually.

US$ 199

This is the net price. Taxes to be calculated in checkout.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20

Notes

  1. 1.

    Unusual crowd activity dataset made available by the University of Minnesota at: http://mha.cs.umn.edu/Movies/Crowd-Activity-All.avi.

  2. 2.

    We thank the authors for providing essential code pieces to reimplement their method.

References

  1. Adam, A., Rivlin, E., Shimshoni, I., & Reinitz, D. (2008). Robust real-time unusual event detection using multiple fixed-location monitors. IEEE Transactions Pattern Analysis and Machine Intelligence, 30(3), 555–560.

  2. Antic, B. & Ommer, B. (2011). Video parsing for abnormality detection. In D. N. Metaxas, L. Quan, A. Sanfeliu, & L. J. V. Gool (Eds.), IEEE international conference on computer vision (ICCV) (pp. 2415–2422).

  3. Benezeth, Y., Jodoin, P. M., Saligrama, V., & Rosenberger, C. (2009). Abnormal events detection based on spatio-temporal co-occurences. In IEEE international conference on computer vision and pattern recognition (CVPR) (pp. 2458–2465).

  4. Bensch, R., Brox, T., & Ronneberger, O. (2015). Spatiotemporal deformable prototypes for motion anomaly detection. In M. W. J. Xianghua Xie & G. K. L. Tam (Eds.), British machine vision conference (BMVC) (pp. 189.1–189.12). London: BMVA Press.

  5. Besl, P., & McKay, N. D. (1992). A method for registration of 3-D shapes. IEEE Transactions Pattern Analysis and Machine Intelligence, 14(2), 239–256.

  6. Boiman, O. & Irani, M. (2007b). Similarity by composition. In B. Schölkopf, J. Platt, & T. Hoffman (Eds.), Advances in neural information processing systems 19 (NIPS) (pp. 177–184). Cambridge, MA: MIT Press.

  7. Boiman, O., & Irani, M. (2007a). Detecting irregularities in images and in video. International Journal of Computer Vision, 74(1), 17–31.

  8. Brox, T., & Malik, J. (2011). Large displacement optical flow: Descriptor matching in variational motion estimation. IEEE Transactions Pattern Analysis and Machine Intelligence, 33(3), 500–513.

  9. Byrd, R. H., Lu, P., Nocedal, J., & Zhu, C. (1995). A limited memory algorithm for bound constrained optimization. SIAM Journal on Scientific Computing, 16(5), 1190–1208.

  10. Chandola, V., Banerjee, A., & Kumar, V. (2009). Anomaly detection: A survey. ACM Computing Surveys, 41(3), 15:1–15:58.

  11. Cong, Y., Yuan, J., & Liu, J. (2011). Sparse reconstruction cost for abnormal event detection. In IEEE international conference on computer vision and pattern recognition (CVPR) (pp. 3449–3456).

  12. Cong, Y., Yuan, J., & Tang, Y. (2013). Video anomaly search in crowded scenes via spatio-temporal motion context. IEEE Transactions on Information Forensics and Security, 8(10), 1590–1599.

  13. Dee, H. M. & Hogg, D. C. (2004). Detecting inexplicable behaviour. In British machine vision conference (BMVC) (pp. 477–486).

  14. Everitt, B., Landau, S., Leese, M., & Stahl, D. (2011). Cluster analysis. Wiley series in probability and statistics New York: Wiley.

  15. Hu, W., Xiao, X., Fu, Z., Xie, D., Tan, T., & Maybank, S. (2006). A system for learning statistical motion patterns. IEEE Transactions Pattern Analysis and Machine Intelligence, 28(9), 1450–1464.

  16. Kim, J. & Grauman, K. (2009). Observe locally, infer globally: A space-time MRF for detecting abnormal activities with incremental updates. In IEEE international conference on computer vision and pattern recognition (CVPR) (pp. 2921–2928).

  17. Kratz, L. & Nishino, K. (2009). Anomaly detection in extremely crowded scenes using spatio-temporal motion pattern models. In IEEE international conference on computer vision and pattern recognition (CVPR) (pp. 1446–1453).

  18. Li, C., Han, Z., Ye, Q., & Jiao, J. (2013). Visual abnormal behavior detection based on trajectory sparse reconstruction analysis. Neurocomputing, 119,94–100. Intelligent Processing Techniques for Semantic-based Image and Video Retrieval.

  19. Mahadevan, V., Li, W., Bhalodia, V., & Vasconcelos, N. (2010). Anomaly detection in crowded scenes. In IEEE international conference on computer vision and pattern recognition (CVPR) (pp. 1975–1981).

  20. Mehran, R., Oyama, A., & Shah, M. (2009). Abnormal crowd behavior detection using social force model. In IEEE international conference on computer vision and pattern recognition (CVPR) (pp. 935–942).

  21. Nait-Charif, H. & McKenna, S. J. (2004). Activity summarisation and fall detection in a supportive home environment. In IEEE international conference on pattern recognition (ICPR) (Vol. 4, pp. 323–326).

  22. Piciarelli, C., Micheloni, C., & Foresti, G. L. (2008). Trajectory-based anomalous event detection. IEEE Transactions on Circuits and Systems for Video Technology, 18(11), 1544–1554.

  23. Popoola, O. P., & Wang, K. (2012). Video-based abnormal human behavior recognition—A review. IEEE Transactions on Circuits and Systems for Video Technology, 42(6), 865–878.

  24. Roshtkhari, M. J., & Levine, M. D. (2013). An on-line, real-time learning method for detecting anomalies in videos using spatio-temporal compositions. Computer Vision and Image Understanding, 117(10), 1436–1452.

  25. Saligrama, V. & Chen, Z. (2012). Video anomaly detection based on local statistical aggregates. In IEEE international conference on computer vision and pattern recognition (CVPR) (pp. 2112–2119).

  26. Saligrama, V., Konrad, J., & Jodoin, P. M. (2010). Video anomaly identification. IEEE Signal Processing Magazine, 27(5), 18–33.

  27. Schmid, B., Shah, G., Scherf, N., Weber, M., Thierbach, K., Campos, C. P., Roeder, I., Aanstad, P., & Huisken, J. (2013). High-speed panoramic light-sheet microscopy reveals global endodermal cell dynamics. Nature Communications, 4. doi:10.1038/ncomms3207.

  28. Sillito, R. R. & Fisher, R. B. (2008). Semi-supervised learning for anomalous trajectory detection. In British machine vision conference (BMVC) (pp. 1035–1044).

  29. Sundaram, N., Brox, T., & Keutzer, K. (2010). Dense point trajectories by GPU-accelerated large displacement optical flow. In European conference on computer vision (ECCV). Lecture notes in computer science (pp. 438–451). New York: Springer.

  30. Umeyama, S. (1991). Least-squares estimation of transformation parameters between two point patterns. IEEE Signal Processing Magazine, 13(4), 376–380.

  31. Winkelbach, S., Molkenstruck, S., & Wahl, F. M. (2006). Low-cost laser range scanner and fast surface registration approach. In Pattern Recognition (Proc. DAGM). Lecture notes in computer science (pp. 718–728). New York: Springer.

  32. Wu, S., Moore, B. E., & Shah, M. (2010). Chaotic invariants of Lagrangian particle trajectories for anomaly detection in crowded scenes. In IEEE international conference on computer vision and pattern recognition (CVPR) (pp. 2054–2060).

Download references

Acknowledgments

We thank J. Koch, A. Krämer, T. Paxian and D. Mai who contributed their juggling expertise and agreed to perform diverse juggling patterns in front of our Kinect camera. This study was supported by the Excellence Initiative of the German Federal and State Governments (BIOSS Centre for Biological Signalling Studies EXC 294 to R.B., T.B. and O.R.). N.S. and J.H. have received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation program (Grant agreement No. 647885).

Author information

Correspondence to Robert Bensch.

Additional information

Communicated by Xianghua Xie, Mark Jones, Gary Tam.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (m4v 22775 KB)

Supplementary material 1 (m4v 22775 KB)

Appendix: Juggling Patterns Dataset Overview

Appendix: Juggling Patterns Dataset Overview

A tabular overview of the juggling pattern test set used in the experiments in Sect. 7.2 is given in Table 2. It lists all test sequences and their properties, such as the number of frames, distance to the juggler and viewing angle.

Table 2 Juggling patterns test set overview

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Bensch, R., Scherf, N., Huisken, J. et al. Spatiotemporal Deformable Prototypes for Motion Anomaly Detection. Int J Comput Vis 122, 502–523 (2017) doi:10.1007/s11263-016-0934-1

Download citation

Keywords

  • Anomaly detection
  • Motion patterns
  • Point trajectories
  • Elastic registration