Abnormal Event Detection and Localization in Visual Surveillance

  • Yonglin Mu
  • Bo ZhangEmail author
Conference paper
Part of the Lecture Notes in Electrical Engineering book series (LNEE, volume 516)


In this paper, we propose a framework for abnormal event detection and analysis in the field of visual surveillance based on the state-of-the-art deep learning techniques. We train a pair of conditional generative adversarial networks (cGANs) using the normal behavior samples, where one cGAN takes video frames as inputs and generates the corresponding optical flow features. While on the other hand, the other cGANs take optical flow features as inputs and generate the corresponding video frames. By analyzing the differences between the generated frames/optical flow features and the realistic samples, abnormal events can be detected and localized effectively. Moreover, for suspected regions, we adopt the faster RCNN to analyze the abnormal events. Experimental results demonstrate that the proposed framework can detect the abnormal events accurately and efficiently.


Conditional GANs Faster RCNN Abnormal event detection Visual surveillance 



This work is partly supported by the National Natural Science Foundation of China (Grant No. 61702073) and the Fundamental Research Funds for the Central Universities (Grant No. 3132018190).


  1. 1.
    Mehran R, Oyama A, Shah M. Abnormal crowd behavior detection using social force model. In: IEEE conference on computer vision and pattern recognition; 2009. p. 935–42.Google Scholar
  2. 2.
    Cong Y, Yuan J, Liu J. Sparse reconstruction cost for abnormal event detection. In: IEEE conference on computer vision and pattern recognition; 2011. p. 3449–3456.Google Scholar
  3. 3.
    Hinami R, Mei T, Shin. Joint detection and recounting of abnormal events by learning deep generic knowledge. In: IEEE international conference on computer vision; 2017. p. 3639–47.Google Scholar
  4. 4.
    Goodfellow IJ, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville AC, Bengio Y. Generative adversarial nets. In: International conference on neural information processing systems; 2014. p. 2672–80.Google Scholar
  5. 5.
    Isola P, Zhu J, Zhou T, Efros AA. Image-to-image translation with conditional adversarial networks. In: IEEE conference on computer vision and pattern recognition; 2016. p. 5967–76.Google Scholar
  6. 6.
    Ren S, Girshick R, Girshick R, Sun J. Faster RCNN: towards real-time object detection with region proposal networks. IEEE Trans Pattern Anal Mach Intell. 2017;39(6):1137–49.CrossRefGoogle Scholar
  7. 7.
    Ionescu RT, Smeureanu S, Popescu M, Alexe B. Detecting abnormal events in video using narrowed motion clusters; 2018.
  8. 8.
    Kim J, Grauman K. Observe locally, infer globally: A space-time MRF for detecting abnormal activities with incremental updates. In: IEEE Conference on Computer Vision and Pattern Recognition; 2015. 2921–7.Google Scholar
  9. 9.
    Choi W, Shahid K, Savarese S. Learning context for collective activity recognition. In: IEEE conference on computer vision and pattern recognition; 2011. p. 3273–80.Google Scholar
  10. 10.
    Mousavi H, Nabi M, Galoogahi HK, Perina A, Murino V. Abnormality detection with improved histogram of oriented tracklets. In: International conference on image analysis and processing; 2015. p. 722–32.CrossRefGoogle Scholar
  11. 11.
    Ravanbakhsh M, Nabi M, Mousavi H, Sangineto E, Sebe N. Plug-and-play CNN for crowd motion analysis: an application in abnormal event detection. In: IEEE winter conference on applications of computer vision; 2018.Google Scholar
  12. 12.
    Sabokrou M, Fayyaz M, Fathy M, Klette R. Deep-anomaly: fully convolutional neural network for fast anomaly detection in crowded scenes; 2018.
  13. 13.
    Radford A, Metz L, Chintala S. Unsupervised representation learning with deep convolutional generative adversarial networks. In: International Conference on Learning Representations; 2016.Google Scholar
  14. 14.
    Ravanbakhsh M, Nabi M, Sangineto E, Marcenaro L, Regazzoni C, Sebe N. Abnormal event detection in videos using generative adversarial nets. In: IEEE international conference on image processing; 2017.Google Scholar
  15. 15.
    Brox T, Malik J. Large displacement optical flow: descriptor matching in variational motion estimation. IEEE Trans Pattern Anal Mach Intell. 2011;33(3):500–13.CrossRefGoogle Scholar
  16. 16.
    Mahadevan V, Li W, Vasconcelos N. Anomaly detection in crowded scenes. In: IEEE conference on computer vision and pattern recognition; 2010. p. 1975–81.Google Scholar
  17. 17.
    Lu C, Shi J, Jia J. Abnormal event detection at 150 FPS in MATLAB. In: IEEE international conference on computer vision; 2014. p. 2720–7.Google Scholar
  18. 18.
    Xu D, Yan Y, Ricci E, Sebe N. Detecting anomalous events in videos by learning deep representations of appearance and motion. Comput Vis Image Underst. 2016;156:117–27.CrossRefGoogle Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2020

Authors and Affiliations

  1. 1.College of Information Science and TechnologyDalian Maritime UniversityDalianChina

Personalised recommendations