Multimedia Tools and Applications

, Volume 75, Issue 14, pp 8799–8826 | Cite as

A thermodynamics-inspired feature for anomaly detection on crowd motions in surveillance videos

  • Xinfeng Zhang
  • Su Yang
  • Yuan Yan Tang
  • Weishan Zhang


Identification of abnormal behaviors in surveillance videos of crowds plays an important role in public security monitoring. However, detecting abnormal crowd behaviors is challenging in that movements of individuals are usually random and unpredictable, and the occlusions caused by over-crowding make the task more difficult. In this paper, we introduce thermodynamic micro-statistics theory to detect and localize abnormal behaviors in crowded scenes based on Boltzmann Entropy. For this purpose, the scene of interest is modeled as moving particles turned out from a general optical flow algorithm. The particles are grouped into a set of prototypes according to their speeds and directions of moving, and a histogram is established to figure out how the particles distribute over the prototypes. Here, Boltzmann Entropy is computed from the histogram for each video clip to characterize the chaos degree of crowd motion. By means of such feature extraction, the crowd motion patterns can be represented as a time series. We find that when most people behave anomaly in an area under surveillance, the corresponding entropy value will increase remarkably in comparison with those of normal cases. This motives us to make use of Boltzmann Entropy to distinguish the collective behaviors of people under emergent circumstances from their normal behaviors by evaluating how significantly the current feature value fits into the Gaussian model of normal cases. We validate our method extensively for anomaly detection and localization. The experimental results show promising performance compared with the state of the art methods.


Boltzmann Entropy Crowd Collective behavior Abnormal event detection Anomaly detection 



This work is supported by NSFC under grant No. 61472087.

Supplementary material

11042_2015_3101_MOESM1_ESM.pdf (646 kb)
ESM 1 (PDF 645 kb)


  1. 1.
    Andrade EL, Blunsden S, Fisher RB Modelling Crowd Scenes for Event Detection. In: Pattern Recognition, 2006. ICPR 2006. 18th International Conference on, 0–0 0 2006. pp 175–178. doi: 10.1109/ICPR.2006.806
  2. 2.
    Basharat A, Gritai A, Shah M Learning object motion patterns for anomaly detection and improved object detection. In: Computer Vision and Pattern Recognition, 2008. CVPR 2008. IEEE Conference on, 23–28 June 2008 2008. pp 1–8. doi: 10.1109/CVPR.2008.4587510
  3. 3.
    Brox T, Bruhn A, Papenberg N, Weickert J (2004) High accuracy optical flow estimation based on a theory for warping. In: Pajdla T, Matas J (eds) Computer vision - ECCV 2004, Lecture Notes in Computer Science, vol 3024. Springer, Berlin Heidelberg, pp. 25–36. doi: 10.1007/978-3-540-24673-2_3 CrossRefGoogle Scholar
  4. 4.
    Cong Y, Yuan J, Liu J (2013) Abnormal event detection in crowded scenes using sparse representation. Pattern Recogn 46(7):1851–1864. doi: 10.1016/j.patcog.2012.11.021 CrossRefGoogle Scholar
  5. 5.
    Dalal N, Triggs B Histograms of oriented gradients for human detection. In: Computer Vision and Pattern Recognition, 2005. CVPR 2005. IEEE Computer Society Conference on, 25–25 June 2005 2005. pp 886–893 vol. 881. doi: 10.1109/CVPR.2005.177
  6. 6.
    Gu X, Cui J, Zhu Q (2014) Abnormal crowd behavior detection by using the particle entropy. Optik – International Journal for Light and Electron Optics 125(14):3428–3433. doi: 10.1016/j.ijleo.2014.01.041 CrossRefGoogle Scholar
  7. 7.
    Haering N, Venetianer P, Lipton A (2008) The evolution of video surveillance: an overview. Mach Vis Appl 19(5–6):279–290. doi: 10.1007/s00138-008-0152-0 CrossRefGoogle Scholar
  8. 8.
    Halliday D, Resnick R, Walker J (2010) Fundamentals of physics extended 9 edition. Wiley:550-560Google Scholar
  9. 9.
    Helbing D, Molnár P (1995) Social force model for pedestrian dynamics. Phys Rev E 51(5):4282–4286CrossRefGoogle Scholar
  10. 10.
    Helbing D, Johansson A, Al-Abideen HZ (2007) Dynamics of crowd disasters: an empirical study. Phys Rev E 75(4). doi: 10.1103/PhysRevE.75.046109
  11. 11.
    Jie F, Chao Z, Pengwei H Online anomaly detection in videos by clustering dynamic exemplars. In: Image Processing (ICIP), 2012 19th IEEE International Conference on, Sept. 30 2012-Oct. 3 2012 2012. pp 3097–3100. doi: 10.1109/ICIP.2012.6467555
  12. 12.
    Kaltsa V, Briassouli A, Kompatsiaris I, Hadjileontiadis LJ, Strintzis MG (2015) Swarm intelligence for detecting interesting events in crowded environments. Image Processing, IEEE Transactions on 24(7):2153–2166. doi: 10.1109/TIP.2015.2409559 MathSciNetCrossRefGoogle Scholar
  13. 13.
    Mahadevan V, Weixin L, Bhalodia V, Vasconcelos N Anomaly detection in crowded scenes. In: Computer Vision and Pattern Recognition (CVPR), 2010 I.E. Conference on, 13–18 June 2010 2010. pp 1975–1981. doi: 10.1109/CVPR.2010.5539872
  14. 14.
    Mehran R, Oyama A, Shah M Abnormal crowd behavior detection using social force model. In: Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on, 20–25 June 2009 2009. pp 935–942. doi: 10.1109/CVPR.2009.5206641
  15. 15.
    Nam Y (2014) Crowd flux analysis and abnormal event detection in unstructured and structured scenes. Multimed Tools Appl 72(3):3001–3029. doi: 10.1007/s11042-013-1593-7 CrossRefGoogle Scholar
  16. 16.
    Pathan S, Al-Hamadi A, Michaelis B (2011) Using conditional random field for crowd behavior analysis. In: Koch R, Huang F (eds) Computer vision – ACCV 2010 workshops, Lecture Notes in Computer Science, vol 6468. Springer, Berlin Heidelberg, pp. 370–379. doi: 10.1007/978-3-642-22822-3_37 CrossRefGoogle Scholar
  17. 17.
    Raghavendra R, Del Bue A, Cristani M, Murino V Optimizing interaction force for global anomaly detection in crowded scenes. In: Computer Vision Workshops (ICCV Workshops), 2011 I.E. International Conference on, 6–13 Nov. 2011 2011. pp 136–143. doi: 10.1109/ICCVW.2011.6130235
  18. 18.
    Reddy V, Sanderson C, Lovell BC Improved anomaly detection in crowded scenes via cell-based analysis of foreground speed, size and texture. In: Computer Vision and Pattern Recognition Workshops (CVPRW), 2011 I.E. Computer Society Conference on, 20–25 June 2011 2011. pp 55–61. doi: 10.1109/CVPRW.2011.5981799
  19. 19.
    Sethi RJ, Roy-Chowdhury AK (2010) Modeling and recognition of complex multi-person interactions in video. Paper presented at the Proceedings of the 1st ACM international workshop on Multimodal pervasive video analysis, Firenze, Italy,Google Scholar
  20. 20.
    Shandong W, Moore BE, Shah M Chaotic invariants of Lagrangian particle trajectories for anomaly detection in crowded scenes. In: Computer Vision and Pattern Recognition (CVPR), 2010 I.E. Conference on, 13–18 June 2010 2010. pp 2054–2060. doi: 10.1109/CVPR.2010.5539882
  21. 21.
    Shannon CE (1948) A mathematical theory of communication. Bell System Technical Journal 27:379–423MathSciNetCrossRefMATHGoogle Scholar
  22. 22.
    Si W, Hau-San W, Zhiwen Y (2014) A Bayesian model for crowd escape behavior detection. Circuits and Systems for Video Technology, IEEE Transactions on 24(1):85–98. doi: 10.1109/TCSVT.2013.2276151 CrossRefGoogle Scholar
  23. 23.
    Susan S, Hanmandlu M (2013) Unsupervised detection of nonlinearity in motion using weighted average of non-extensive entropies. SIViP:1–15. doi:10.1007/s11760-013-0464-zGoogle Scholar
  24. 24.
    Tian C, Xinyu W, Jinnian G, Shiqi Y, Yangsheng X Abnormal crowd motion analysis. In: Robotics and Biomimetics (ROBIO), 2009 I.E. International Conference on, 19–23 Dec. 2009 2009. pp 1709–1714. doi: 10.1109/ROBIO.2009.5420408
  25. 25.
    Tu P, Sebastian T, Doretto G, Krahnstoever N, Rittscher J, Yu T (2008) Unified crowd segmentation. In: Forsyth D, Torr P, Zisserman A (eds) Computer vision – ECCV 2008, Lecture Notes in Computer Science, vol 5305. Springer, Berlin Heidelberg, pp. 691–704. doi: 10.1007/978-3-540-88693-8_51 CrossRefGoogle Scholar
  26. 26.
    UMN Unusual crowd activity dataset of University of Minnesota. available from http://mhacsumnedu/movies/crowd-activity-allaviGoogle Scholar
  27. 27.
    Wang X, Tieu K, Grimson E (2006) Learning semantic scene models by trajectory analysis. In: Leonardis A, Bischof H, Pinz A (eds) Computer vision – ECCV 2006, Lecture Notes in Computer Science, vol 3953. Springer, Berlin Heidelberg, pp. 110–123. doi: 10.1007/11744078_9 CrossRefGoogle Scholar
  28. 28.
    Weixin L, Mahadevan V, Vasconcelos N (2014) Anomaly detection and localization in crowded scenes. Pattern Analysis and Machine Intelligence, IEEE Transactions on 36(1):18–32. doi: 10.1109/TPAMI.2013.111 CrossRefGoogle Scholar
  29. 29.
    Xinyi C, Qingshan L, Mingchen G, Metaxas DN Abnormal detection using interaction energy potentials. In: Computer Vision and Pattern Recognition (CVPR), 2011 I.E. Conference on, 20–25 June 2011 2011. pp 3161–3167. doi: 10.1109/CVPR.2011.5995558
  30. 30.
    Xiong G, Cheng J, Wu X, Chen Y-L, Ou Y, Xu Y (2012) An energy model approach to people counting for abnormal crowd behavior detection. Neurocomputing 83(0):121–135. doi: 10.1016/j.neucom.2011.12.007 CrossRefGoogle Scholar
  31. 31.
    Yang S Gene Ranking Using R-Metric. In: Computational Intelligence and Security, 2006 International Conference on, Nov. 2006 2006. pp 465–469. doi: 10.1109/ICCIAS.2006.294178
  32. 32.
    Yang C, Junsong Y, Ji L Sparse reconstruction cost for abnormal event detection. In: Computer Vision and Pattern Recognition (CVPR), 2011 I.E. Conference on, 20–25 June 2011 2011. pp 3449–3456. doi: 10.1109/CVPR.2011.5995434
  33. 33.
    Zhang X, Yang S, Tang YY, Zhang W (2015) Crowd Motion Monitoring with Thermodynamics-Inspired Feature. North America, mar. 2015., AAAI Conference on Artificial IntelligenceGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2015

Authors and Affiliations

  • Xinfeng Zhang
    • 1
  • Su Yang
    • 1
  • Yuan Yan Tang
    • 2
  • Weishan Zhang
    • 3
  1. 1.Shanghai Key Laboratory of Intelligent Information Processing, College of Computer ScienceFudan UniversityShanghaiChina
  2. 2.Department of Computer and Information ScienceUniversity of MacauMacauChina
  3. 3.Department of Software EngineeringChina University of PetroleumQingdaoChina

Personalised recommendations