This paper presents an approach for motion-based anomaly detection, where a prototype pattern is detected and elastically registered against a test sample to detect anomalies in the test sample. The prototype model is learned from multiple sequences to define accepted variations. “Supertrajectories” based on hierarchical clustering of dense point trajectories serve as an efficient and robust representation of motion patterns. An efficient hashing approach provides transformation hypotheses that are refined by a spatiotemporal elastic registration. We propose a new method for elastic registration of 3D+time trajectory patterns that induces spatial elasticity from trajectory affinities. The method is evaluated on a new motion anomaly dataset of juggling patterns and performs well in detecting subtle anomalies. Moreover, we demonstrate the applicability to biological motion patterns.
This is a preview of subscription content, log in to check access.
Buy single article
Instant unlimited access to the full article PDF.
Price includes VAT for USA
Subscribe to journal
Immediate online access to all issues from 2019. Subscription will auto renew annually.
This is the net price. Taxes to be calculated in checkout.
Unusual crowd activity dataset made available by the University of Minnesota at: http://mha.cs.umn.edu/Movies/Crowd-Activity-All.avi.
We thank the authors for providing essential code pieces to reimplement their method.
Adam, A., Rivlin, E., Shimshoni, I., & Reinitz, D. (2008). Robust real-time unusual event detection using multiple fixed-location monitors. IEEE Transactions Pattern Analysis and Machine Intelligence, 30(3), 555–560.
Antic, B. & Ommer, B. (2011). Video parsing for abnormality detection. In D. N. Metaxas, L. Quan, A. Sanfeliu, & L. J. V. Gool (Eds.), IEEE international conference on computer vision (ICCV) (pp. 2415–2422).
Benezeth, Y., Jodoin, P. M., Saligrama, V., & Rosenberger, C. (2009). Abnormal events detection based on spatio-temporal co-occurences. In IEEE international conference on computer vision and pattern recognition (CVPR) (pp. 2458–2465).
Bensch, R., Brox, T., & Ronneberger, O. (2015). Spatiotemporal deformable prototypes for motion anomaly detection. In M. W. J. Xianghua Xie & G. K. L. Tam (Eds.), British machine vision conference (BMVC) (pp. 189.1–189.12). London: BMVA Press.
Besl, P., & McKay, N. D. (1992). A method for registration of 3-D shapes. IEEE Transactions Pattern Analysis and Machine Intelligence, 14(2), 239–256.
Boiman, O. & Irani, M. (2007b). Similarity by composition. In B. Schölkopf, J. Platt, & T. Hoffman (Eds.), Advances in neural information processing systems 19 (NIPS) (pp. 177–184). Cambridge, MA: MIT Press.
Boiman, O., & Irani, M. (2007a). Detecting irregularities in images and in video. International Journal of Computer Vision, 74(1), 17–31.
Brox, T., & Malik, J. (2011). Large displacement optical flow: Descriptor matching in variational motion estimation. IEEE Transactions Pattern Analysis and Machine Intelligence, 33(3), 500–513.
Byrd, R. H., Lu, P., Nocedal, J., & Zhu, C. (1995). A limited memory algorithm for bound constrained optimization. SIAM Journal on Scientific Computing, 16(5), 1190–1208.
Chandola, V., Banerjee, A., & Kumar, V. (2009). Anomaly detection: A survey. ACM Computing Surveys, 41(3), 15:1–15:58.
Cong, Y., Yuan, J., & Liu, J. (2011). Sparse reconstruction cost for abnormal event detection. In IEEE international conference on computer vision and pattern recognition (CVPR) (pp. 3449–3456).
Cong, Y., Yuan, J., & Tang, Y. (2013). Video anomaly search in crowded scenes via spatio-temporal motion context. IEEE Transactions on Information Forensics and Security, 8(10), 1590–1599.
Dee, H. M. & Hogg, D. C. (2004). Detecting inexplicable behaviour. In British machine vision conference (BMVC) (pp. 477–486).
Everitt, B., Landau, S., Leese, M., & Stahl, D. (2011). Cluster analysis. Wiley series in probability and statistics New York: Wiley.
Hu, W., Xiao, X., Fu, Z., Xie, D., Tan, T., & Maybank, S. (2006). A system for learning statistical motion patterns. IEEE Transactions Pattern Analysis and Machine Intelligence, 28(9), 1450–1464.
Kim, J. & Grauman, K. (2009). Observe locally, infer globally: A space-time MRF for detecting abnormal activities with incremental updates. In IEEE international conference on computer vision and pattern recognition (CVPR) (pp. 2921–2928).
Kratz, L. & Nishino, K. (2009). Anomaly detection in extremely crowded scenes using spatio-temporal motion pattern models. In IEEE international conference on computer vision and pattern recognition (CVPR) (pp. 1446–1453).
Li, C., Han, Z., Ye, Q., & Jiao, J. (2013). Visual abnormal behavior detection based on trajectory sparse reconstruction analysis. Neurocomputing, 119,94–100. Intelligent Processing Techniques for Semantic-based Image and Video Retrieval.
Mahadevan, V., Li, W., Bhalodia, V., & Vasconcelos, N. (2010). Anomaly detection in crowded scenes. In IEEE international conference on computer vision and pattern recognition (CVPR) (pp. 1975–1981).
Mehran, R., Oyama, A., & Shah, M. (2009). Abnormal crowd behavior detection using social force model. In IEEE international conference on computer vision and pattern recognition (CVPR) (pp. 935–942).
Nait-Charif, H. & McKenna, S. J. (2004). Activity summarisation and fall detection in a supportive home environment. In IEEE international conference on pattern recognition (ICPR) (Vol. 4, pp. 323–326).
Piciarelli, C., Micheloni, C., & Foresti, G. L. (2008). Trajectory-based anomalous event detection. IEEE Transactions on Circuits and Systems for Video Technology, 18(11), 1544–1554.
Popoola, O. P., & Wang, K. (2012). Video-based abnormal human behavior recognition—A review. IEEE Transactions on Circuits and Systems for Video Technology, 42(6), 865–878.
Roshtkhari, M. J., & Levine, M. D. (2013). An on-line, real-time learning method for detecting anomalies in videos using spatio-temporal compositions. Computer Vision and Image Understanding, 117(10), 1436–1452.
Saligrama, V. & Chen, Z. (2012). Video anomaly detection based on local statistical aggregates. In IEEE international conference on computer vision and pattern recognition (CVPR) (pp. 2112–2119).
Saligrama, V., Konrad, J., & Jodoin, P. M. (2010). Video anomaly identification. IEEE Signal Processing Magazine, 27(5), 18–33.
Schmid, B., Shah, G., Scherf, N., Weber, M., Thierbach, K., Campos, C. P., Roeder, I., Aanstad, P., & Huisken, J. (2013). High-speed panoramic light-sheet microscopy reveals global endodermal cell dynamics. Nature Communications, 4. doi:10.1038/ncomms3207.
Sillito, R. R. & Fisher, R. B. (2008). Semi-supervised learning for anomalous trajectory detection. In British machine vision conference (BMVC) (pp. 1035–1044).
Sundaram, N., Brox, T., & Keutzer, K. (2010). Dense point trajectories by GPU-accelerated large displacement optical flow. In European conference on computer vision (ECCV). Lecture notes in computer science (pp. 438–451). New York: Springer.
Umeyama, S. (1991). Least-squares estimation of transformation parameters between two point patterns. IEEE Signal Processing Magazine, 13(4), 376–380.
Winkelbach, S., Molkenstruck, S., & Wahl, F. M. (2006). Low-cost laser range scanner and fast surface registration approach. In Pattern Recognition (Proc. DAGM). Lecture notes in computer science (pp. 718–728). New York: Springer.
Wu, S., Moore, B. E., & Shah, M. (2010). Chaotic invariants of Lagrangian particle trajectories for anomaly detection in crowded scenes. In IEEE international conference on computer vision and pattern recognition (CVPR) (pp. 2054–2060).
We thank J. Koch, A. Krämer, T. Paxian and D. Mai who contributed their juggling expertise and agreed to perform diverse juggling patterns in front of our Kinect camera. This study was supported by the Excellence Initiative of the German Federal and State Governments (BIOSS Centre for Biological Signalling Studies EXC 294 to R.B., T.B. and O.R.). N.S. and J.H. have received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation program (Grant agreement No. 647885).
Communicated by Xianghua Xie, Mark Jones, Gary Tam.
Electronic supplementary material
Below is the link to the electronic supplementary material.
Appendix: Juggling Patterns Dataset Overview
About this article
Cite this article
Bensch, R., Scherf, N., Huisken, J. et al. Spatiotemporal Deformable Prototypes for Motion Anomaly Detection. Int J Comput Vis 122, 502–523 (2017) doi:10.1007/s11263-016-0934-1
- Anomaly detection
- Motion patterns
- Point trajectories
- Elastic registration