Skip to main content
Log in

A review of supervised and unsupervised machine learning techniques for suspicious behavior recognition in intelligent surveillance system

  • Original Research
  • Published:
International Journal of Information Technology Aims and scope Submit manuscript

Abstract

There is a strong demand of smart vision based surveillance system owing to the increase in crime at a frightening rate at various public places like Banks, Airport, Shopping malls and its application in human activity recognition ranges from patient fall detection, irregular pattern recognition or Human computer Interaction. As the crime increases at a disturbing rate, public security violations and high cost of security personals have motivated the author to do the strategic survey of existing vision and image processing based techniques in the past literature. The paper begins with discussing the common approach towards suspicious activity detection and recognition followed by summarizing the supervised and unsupervised machine learning methodologies mainly based on SVM, HMM and ANN classifiers, which were adopted by the researchers previously varying from single human behavior modeling to crowded scenes. Next, this paper discusses system model for human’s normal and abnormal activities recognition along with various feature selectors and detectors used in previous literature. This was followed by conducting a review of benchmark researches which covered a comprehensive state of art methodologies in the related fields, key points owned, feature learning and applications. At last experimental aspects of various papers have been discussed with essential performance matrices like accuracy along with the major issues, common problems, challenges and future scope in the related field.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Gowsikhaa D, Abirami S, Baskaran R (2012) Automated human behavior analysis from video surveillance: a survey. Artif Intell Rev pp 1–19

  2. Lavee G, Khan L, Thuraisingham B (2007) A framework for a video analysis tool for suspicious event detection. Multimed Tools Appl 35:109–123

    Article  Google Scholar 

  3. Wiliem A, Madasu V, Boles W, Yarlagadda P (2008) Detecting uncommon trajectories. In: Proceedings of Digital image computing: techniques and applications, pp 398–404

  4. Benezeth Y, Jodoin P, Saligrama V, Rosenberger C (2009) Abnormal events detection based on spatio-temporal co-occurrences. In: Proceedings of IEEE conference on computer vision pattern recognition workshops, pp 2458–2465

  5. Chen Y, Liang G, Ka KL, Xu Y (2007) Abnormal behavior detection by multi-SVM-based Bayesian network. In: Proceedings of international conference information acquisition, pp 298–303

  6. Duque D, Santos H, Cortez P (2007) Prediction of abnormal behaviors for intelligent video surveillance systems. In: Computational intelligence and data mining, 2007. CIDM 2007. IEEE Symposium on, pp 362–367

  7. Vaswani N, Roy-Chowdhury AK, Chellappa R (2005) Shape activity: a continuous-state HMM for moving/deforming shapes with application to abnormal activity detection. IEEE Trans Image Process 14(10):1603–1616

    Article  Google Scholar 

  8. Duong TV, Bui HH, Phung DQ, Venkatesh S (2005) Activity recognition and abnormality detection with the switching hidden semi-Markov model. In: Proceedings of IEEE computer society conference on computer vision and pattern recognition, vol 1, pp 838–845

  9. Yilmaz A, Javed O, Shah M (2006) Object tracking: a survey. ACM Comput Surv 38

  10. Bakar ZA, Mohemad R, Ahmad A, Deris MM (2006) A comparative study for outlier detection techniques in data mining. In: Proceedings of IEEE conference on cybernetics and intelligent systems, pp 1–6

  11. Pantic M, Pentland A, Nijholt A, Huang TS (2006) Human computing and machine understanding of human behavior: a survey. In: Proceedings of ICMI 2006 and IJCAI 2007 international conference on artificial intelligence human computing (ICMI’06/IJCAI’07). Huang TS, Nijholt A, Pantic M, Pentland A (eds) Springer, Berlin, pp 47–71

  12. Gandhi T, Trivedi MM (2007) Pedestrian protection systems: issues, survey, and challenges. IEEE Trans Intell Transp Syst 8(3):413–430

    Article  Google Scholar 

  13. Volker K (2007) The meaning of action: a review on action recognition and mapping. Adv Rob 21:1473–1501

    Article  Google Scholar 

  14. Poppe R (2007) Vision-based human motion analysis: an over-view. Comput Vis Image Understanding 108:4–18

    Article  Google Scholar 

  15. Zhou Y, Yan S, Huang TS (2007) Detecting anomaly in videos from trajectory similarity analysis. In: Proceedings of the IEEE international conference on multimedia expo, pp 1087–1090

  16. Chen-Chiung H, Shu-Shuo H (2007) A simple and fast surveillance system human tracking and behavior analysis. In: 3rd IEEE international conference on signal-image technologies and internet based system, pp 812–818. https://doi.org/10.1109/sitis.2007.128

  17. Wang Z, Zhang J (2008) Detecting pedestrian abnormal behavior based on fuzzy associative memory. In: 2008, IEEE 4th International conference on natural computation, vol 6, pp 143–147

  18. Zhang J, Liu Z (2008) Detecting abnormal motion of pedestrian in video. In: Information and automation, 2008. ICIA 2008. IEEE, International conference on, pp 81–85

  19. Zhou G, Wu Y (2009) Anomalous event detection based on self-organizing map for supermarket monitoring. In: 2009 International conference on information engineering and computer science, IEEE, pp 1–4

  20. Yin G, Bruckner D (2009) Gaussian models and fast learning algorithm for persistence analysis of tracked video objects. In 2009 2nd conference on human system interactions, IEEE, pp 60–63

  21. Li CL, Hao ZB, Li JJ (2010) Abnormal behavior detection using a novel behavior representation. In Apperceiving computing and intelligence analysis (ICACIA), 2010 International Conference on, IEEE, pp 331–336

  22. Ding N, Chen Y, Zhong Z, Xu Y (2010) Energy-based surveillance systems for ATM machines. In: Intelligent control and automation (WCICA), 2010 8th World Congress on, IEEE, pp 2880–2887

  23. Zin TT, Tin P, Toriu T, Hama H (2010) A Markov random walk model for loitering people detection. In: Intelligent information hiding and multimedia signal processing (IIH-MSP), 2010 6th international conference on, IEEE, pp 680–683

  24. Bouttefroy PLM, Beghdadi A, Bouzerdoum A, Phung SL (2010) Markov random fields for abnormal behavior detection on highways. In: Visual information processing (EUVIP), 2010 2nd Europea Workshop on, IEEE, pp 149–154

  25. Husni M, Suryana N (2010) Crowd event detection in computer vision. In: Proceedings of 2nd international conference on signal processing system, Jul. 5–7, 2010, pp V1444–V1447

  26. Cui L, Li K, Chen J, Li Z (2011) Abnormal event detection in traffic video surveillance based on local features. In: Image and signal processing (CISP), 2011 4th International Congress on, IEEE, Vol 1, pp 362–366

  27. Lizhong L, Zhiguo L, Yubin Z (2012) Research on detection and tracking of moving target in intelligent video surveillance. In: Computer science and electronics engineering (ICCSEE), 2012 International Conference on, IEEE, Vol 3, pp 477–481

  28. Wang C, Wu X, Li N, Chen YL (2012) Abnormal detection based on gait analysis. In: Intelligent control and automation (WCICA), 2012 10th World Congress on, IEEE, pp 4859–4864

  29. Lee JJ, Kim GJ, Kim MH (2012) Trajectory extraction for abnormal behavior detection in public area. In: Emerging technologies for a smarter world (CEWIT), 2012 9th international conference and expo on, IEEE, pp 1–5

  30. Revathi AR, Kumar D (2012) A survey of activity recognition and understanding the behavior. In: Video surveillance, arXiv:1207.6774

  31. Gowsikhaa D, Abirami S, Baskaran R (2014) Automated human behavior analysis from surveillance videos: a survey. Artif Intell Rev Spring 42(4):747–765

    Article  Google Scholar 

  32. Senst T, Eiselein V, Badii A, Einig M, Keller I, Sikora T (2013) A decentralized privacy-sensitive video surveillance framework. In: Digital signal processing (DSP), 2013 18th international conference on, IEEE, pp 1–6

  33. Alqaysi HH, Sasi S (2013) Detection of abnormal behavior in dynamic crowded gatherings. In: 2013 IEEE applied imagery pattern recognition workshop (AIPR), IEEE, pp 1–6

  34. Walha A, Wali A, Alimi AM (2013) A system of abnormal behavior detection in aerial surveillance. In: Information assurance and security (IAS), 2013 9th International Conference on, IEEE, pp. 102–107

  35. Cristani M, Raghavendra R, Del Bue A, Murino V (2013) Human behavior analysis in video surveillance: a social signal processing perspective. Neurocomputing 100:86–97

    Article  Google Scholar 

  36. Zhang Y, Dong L, Li S, Li J (2014) Abnormal crowd behavior detection using interest points. In: 2014 IEEE, international symposium on broadband multimedia systems and broadcasting, IEEE, pp 1–4

  37. Wang J, Xu Z (2014) Texture-based homogeneity analysis for crowd scene modelling and abnormality detection. In: Automation and computing (ICAC), 2014 20th international conference on, IEEE, pp 182–187

  38. Chen C, Shao Y (2014) Anomalous crowd behavior detection and localization in video surveillance. In: Control Science and Systems Engineering (CCSSE), 2014 IEEE International Conference on, IEEE, pp 190–194

  39. Delgado B, Tahboub K, Delp EJ (2014) Automatic detection of abnormal human events on train platforms. In: NAECON 2014-IEEE national aerospace and electronics conference, IEEE, pp 169–173

  40. Divya J. Automatic video based surveillance system for abnormal behavior detection

  41. Chen C, Shao Yu, Bi X (2015) Detection of anomalous crowd behavior based on acceleration feature. IEEE Sens J 15(12):7252–7261

    Article  Google Scholar 

  42. Momin BF, Fanase VM (2015) Detection and localization of abnormal activities in video surveillance system. In: Communications and signal processing (ICCSP), 2015 International Conference on IEEE, 2015 Apr 2, pp 0277–0280

  43. Yavari M, Li M, Li S, Qi M (2015) Parallelizing abnormal event detection in crowded scenes with GPU. In: Fuzzy systems and knowledge discovery (FSKD), 12th international conference on, IEEE, pp 274–277

  44. Tiwari C, Hanmandlu M, Vasikarla S (2015) Suspicious face detection based on eye and other facial features movement monitoring. In: 2015 IEEE applied imagery pattern recognition workshop (AIPR), IEEE, pp 1–8

  45. Ahmed M, Mahmood AN, Islam MR (2016) A survey of anomaly detection techniques in financial domain. Fut Gen Comput Syst 55:278–288

    Article  Google Scholar 

  46. Zhang Y, Lu H, Zhang L, Ruan X (2016) Combining motion and appearance cues for anomaly detection. Pattern Recognit 51:443–452

    Article  Google Scholar 

  47. Farzad A, Asli RN (2015) Recognition and classification of human behavior in Intelligent surveillance systems using Hidden Markov model. Int J Image Graph Signal Process 7(12):31

    Google Scholar 

  48. Dawn DD, Shaikh SH (2016) A comprehensive survey of human action recognition with spatio-temporal interest point (STIP) detector. Vis Comput 32(3):289–306

    Article  Google Scholar 

  49. Al-Nawashi M, Obaida MA-H, Mohamad S (2016) A novel framework for intelligent surveillance system based on abnormal human activity detection in academic environments. In: Neural Computing and Applications, Springer, pp 1–8

  50. Kapoor S, Pateriya A (2016) A study on various methods used in automatic video surveillance. In: 6th IEEE international conference on cloud system and big data engineering (confluence), pp 413–417

  51. Cosar S, Donatiello G, Bogorny V, Garate C, Alvares LO, Bremond F (2016) Towards abnormal trajectory and event detection in video surveillance. IEEE Trans Circ Syst Video Process 1(99):1–14. https://doi.org/10.1109/tcsvt.2016.2589859

    Article  Google Scholar 

  52. Subetha T, Chitrakala S (2016) A survey on human activity recognition from videos. In: Information communication and embedded systems (ICICES), 2016 International Conference on, IEEE, pp 1–7

  53. Ma C, Li W, Gravina R, Fortino G (2016) Activity recognition and monitoring for smart wheelchair users. In: Computer supported cooperative work in design (CSCWD), 2016 IEEE 20th international conference on, IEEE, pp 664–669

  54. Nar R, Singal A, Kumar P (2016) Abnormal activity detection for bank ATM surveillance. In: Advances in computing, communications and informatics (ICACCI), 2016 international conference on, IEEE, pp 2042–2046

  55. Khadse MV, Nijampurkar PP, Pardeshi YD, Kale NS (2016) An effective object detection video surveillance and alert system. In: International journal of computer applications pp 0975–8887

  56. Singh D, Krishna Mohan C (2017) Graph formulation of video activities for abnormal activity recognition. Pattern Recognit 65(2017):265–272

    Article  Google Scholar 

  57. Wu X, Ou Y, Qian H, Xu Y (2005) A detection system for human abnormal behavior. In: 2005 IEEE/RSJ international conference on intelligent robots and systems, pp 1204–1208

  58. Sewaiwar P, Kamal KV (2015) Comparative study of various decision tree classification algorithm using WEKA. In: International Journal of Emerging Research in Management &Technology 4 (2015): 2278-9359

  59. Christodoulou L, Kasparis T, Marques O (2011) Advanced statistical and adaptive threshold techniques for moving object detection and segmentation. In: 2011 17th International conference on digital signal processing (DSP), pp 1–6

  60. Karpathy A, Toderici G, Shetty S, Leung T, Sukthankar R, Fei-Fei L (2014) Large-scale video classification with convolutional neural networks. In: Proceedings of IEEE computer society conference on computer vision and pattern recognition, pp 1725–1732

  61. Simonyan K, Zisserman A (2014) Two-stream convolutional networks for action recognition in videos. In: Advances in Neural information processing systems, pp 568–576

  62. Mahasseni B, Todorovic S (2016) Regularizing long short term memory with 3D human-skeleton sequences for action recognition. In: Proceedings of IEEE computer society conference on computer vision and pattern recognition, pp 3054–3062

  63. Xin M, Zhang H, Wang H, Sun M, Yuan D (2016) ARCH: adaptive recurrent-convolutional hybrid networks for long-term action recognition. Neurocomputing 178:87–102

    Article  Google Scholar 

  64. Murthy OVR, Goecke R (2015) Ordered trajectories for human action recognition with large number of classes. Image Vis Comput 42:22–34

    Article  Google Scholar 

  65. Wang P, Cao Y, Shen C, Liu L, Shen HT (2015) Temporal pyramid pooling based convolutional neural networks for action recognition

  66. Ullah A, Ahmad J, Muhammad K, Sajjad M, Baik SW (2018) Action recognition in video sequences using deep bi-directional LSTM with CNN features. IEEE Access 6:1155–1166

    Article  Google Scholar 

  67. Foroughi H, Aski BS, Pourreza H (2008) Intelligent video surveillance for monitoring fall detection of elderly in home environments. 2008 11th international conference on computer and information technology. IEEE

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Kamal Kant Verma.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Verma, K.K., Singh, B.M. & Dixit, A. A review of supervised and unsupervised machine learning techniques for suspicious behavior recognition in intelligent surveillance system. Int. j. inf. tecnol. 14, 397–410 (2022). https://doi.org/10.1007/s41870-019-00364-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s41870-019-00364-0

Keywords

Navigation