Multi-view recognition system for human activity based on multiple features for video surveillance system

  • Roshan SinghEmail author
  • Alok Kumar Singh Kushwaha
  • Rajeev Srivastava


Recognition of the activities of human, image sequences is most active area of research in computer vision and most of the previous projects, the focus of the activity is to acknowledge the recognition, from a single view and ignored issues of multiple view invariance. In this paper, the proposed framework for the recognition of a view-invariant human activity, solves the above problem. The components of the proposed framework are three consecutive modules: (i) the detection and positioning of the person’s background subtraction, (ii) the function extraction (iii) and the final activity is referenced by using a set of hidden Markov models (HMMs). During features extraction phase in the proposed method for activity representation a combination of contour-based distance signal feature, optical flow-based motion feature and uniform rotation local binary patterns has been used. Due to its rotation invariant nature the uniform LBP provides view-invariant recognition of multi-view human activities. A successful testing of the proposed approach was done on our own viewpoint dataset, KTH action recognition dataset, i3DPost multi-view dataset, and MSR view-point action dataset. From the experimental results and analysis over the chosen datasets, it is observed that the proposed framework is robust, flexible and efficient with respect to multiple views activity recognition, scale and phase variations.


Human activity recognition General framework Features extraction Local binary patterns Optical flow features Hidden Markov models 



  1. 1.
    Ahmadand M, Lee S-W (2008) Human action recognition using shape and CLG-motion flow frommulti-view image sequences. J Pattern Recognit 41:2237–2252CrossRefGoogle Scholar
  2. 2.
    Ahonen T, Matas J, He C, Pietikäinen M (2009) Rotation invariant image description with local binary pattern histogram Fourier features, image analysis, vol 5575. Springer, Berlin, LNCS, pp 61–70Google Scholar
  3. 3.
    Althloothi S, Mahoor MH, Zhang X, Voyles RM (2014) Human activity recognition using multi-features and multiplekernel learning. J Pattern Recognit 47:1800–1812CrossRefGoogle Scholar
  4. 4.
    Ben-Arie J, Wang Z, Pandit P, Rajaram S (2002) Human activity recognition using multidimensional indexing. IEEE Transactions on Pattern Analysis and Machine Intelligence Archive 24(8):1091–1104CrossRefGoogle Scholar
  5. 5.
    Bobick AF, Davis JW (2001) The recognition of human movement using temporal templates. IEEE Trans Pattern Anal Mach Intell 23(3):257–267CrossRefGoogle Scholar
  6. 6.
    Bregler C (1997) Learning and recognizing human dynamics in video sequences. Proc. IEEE Int’l conf. on computer vision and pattern recognition, pp 568–574Google Scholar
  7. 7.
    Bruhn A, Weickert J, Schnörr C (2005) Lucas/Kanade meets horn/Schunck:combining local and global optic flow methods. Int J Comput Vis 61(3):211–231CrossRefGoogle Scholar
  8. 8.
    Carlsson S, Sullivan J (2002) Action recognition by shape matching to key frames. IEEE computer society workshop on models versus exemplars in computer vision, pp 263–270Google Scholar
  9. 9.
    Cohen I, Li H (2003) Inference of human postures by classification of 3D human body shape. IEEE international workshop on analysis and modeling of faces and gestures, pp 74–81Google Scholar
  10. 10.
    Fernández A, Ghita O, González E, Bianconi F, Whelan PF (2011) Evaluation of robustness against rotation of LBP, CCR ILBP features in granite texture classification. Mach Vis Appl 22(6):913–926CrossRefGoogle Scholar
  11. 11.
    Hamid R, Huang Y, Essa I (n.d.) ARGMode –activity recognition using graphical models. Conference on computer vision and pattern recognition workshop, volume 4, pp. 38–45, Madison, Wisconsin, June 16-22, 2003Google Scholar
  12. 12.
    Hannuksela J (2008) Camera based motion estimation recognition for human-computer interaction. Dissertation University of OuluGoogle Scholar
  13. 13.
    Holte MB, Tran C, Trivedi MM, Moeslund TB (2012) "Human pose estimation and activity recognition from multi-view videos: comparative explorations of recent developments". IEEE Journal of Selected Topics in Signal Processing (J-STSP). Volume 6(5):538–552Google Scholar
  14. 14.
    Ikizler-Cinbis N, Sclaroff S (2010) Object, scene and actions: combining multiple features for human action recognition, Proceedings of 11th European Conference Computer Vision–ECCV(2010). Springer, Berlin, pp 494–507Google Scholar
  15. 15.
    Ji X, Liu H (2010) Advances in view-invariant human motion analysis: a review. IEEE Trans Syst Man Cybern Part C Appl Rev 40(1):13–24Google Scholar
  16. 16.
    Kellokumpu V, Zhao G, Pietikäinen M (2010) Dynamic textures for human movement recognition. Proceedings of ACM international conference on image and video retrieval, pp 470–476Google Scholar
  17. 17.
    Kellokumpu V, Zhao G, Pietikäinen M (2011) Recognition of human actions using texture descriptors. Mach Vis Appl 22(5):767–780CrossRefGoogle Scholar
  18. 18.
    Kulathumani V (n.d.) WVU multi-view action recognition dataset. Available on:
  19. 19.
    Laptev I, Caputo B (n.d.) Recognition of human actions, November 2011
  20. 20.
    Liu Y, Nie L, Liu L, Rosenblum DS (2016) Fromactiontoactivity:sensor-basedactivityrecognition. Neurocomputing 181:108–115CrossRefGoogle Scholar
  21. 21.
    Liu L, Cheng L, Liu Y, Jia Y, Rosenblum DS (n.d.) Recognizing complex activities by a probabilistic interval-based model. Proceedings of the thirtieth AAAI conference on artificial intelligence (AAAI-16)Google Scholar
  22. 22.
    Liuy Y, Niey L, Hanx L, Zhangy L, Rosenblum DS (2015) Action2Activity: recognizing complex activities from sensor data. Proceedings of the twenty-fourth international joint conference on artificial intelligence (IJCAI 2015)Google Scholar
  23. 23.
    Dedeoğlu Y, Töreyin B, Güdükbay U, Çetin A (2006) “Silhouette-based method forobject classification and human action recognition in video”, ComputerVision in human-computer interaction, lecture notes in computer science, 3979. Springer, Berlin, pp 64–77Google Scholar
  24. 24.
    Nam Y, Wohn K (1996) Recognition of space-time hand-gestures using hidden Markov model. ACM symposium on virtual reality software and technology, Hong Kong, pp 51–58Google Scholar
  25. 25.
    Ojala T, Pietikainen M, Maenpaa T (2002) Multiresolution gray-scale rotation invariant texture classification with local binary patterns. IEEE Trans Pattern Anal Mach Intell 24(7):971–987CrossRefGoogle Scholar
  26. 26.
    Pietikäinen M (2011) Computer vision using local binary patterns, vol 40. Springer, BerlinGoogle Scholar
  27. 27.
    Qian H, Mao Y, Xiang W, Wang Z (2010) Recognition of human activities using SVM multi-class classifier. Pattern Recogn Lett 31(2):100–111CrossRefGoogle Scholar
  28. 28.
    Sadek S, Al-Hamadi A, Krell G, Michaelis B (2013) Affine-invariant feature extraction for activity recognition. International Scholarly Research Notices- ISRN Machine Vision 2013:215195, 7 pages.
  29. 29.
    Sharma CM, Kushwaha AKS, Nigam S, Khare A (2011) Automatic human activity recognition in video using background modeling and spatio-temporal template matching based technique. In Proc. of international conference on advances in computing and artificial intelligence (ACAI – 2011), pp 97–101Google Scholar
  30. 30.
    Suzuki S, Be K (1985) Topological structural analysis of digitized binary images by border following. Computer Vision, Graphics and Image Processing. volume 30(1):32–46Google Scholar
  31. 31.
    University of Surrey and CERTH-ITI (n.d.) i3dpost multi-view human action datasets, January 2012.
  32. 32.
    Veeraraghavan A, Chellappa R, Roy-Chowdhury AK (2006) The function space of an activity. Proc. of IEEE computer society Conf. On computer vision and pattern recognitionGoogle Scholar
  33. 33.
    Vili K, Guoying Z, Matti P (2008) Texture based description of movements for activity analysis. Proceedings of international conference on computer vision theory and applications (VISAPP 2008), 1(2008), pp 206–213Google Scholar
  34. 34.
    Weinland D, Ronfard R (2011) A survey of vision based methods for action representation, segmentation, and recognition. Comput Vis Image Underst 115(2):529–551CrossRefGoogle Scholar
  35. 35.
    Yang J, Xu Y, Chen CS (2002) Human action learning via hidden Markov model. IEEE Trans Syst Man Cybern Syst Hum 27(1):34–44CrossRefGoogle Scholar
  36. 36.
    Yuan J, Liu Z, Wu Y (n.d.) Discriminative video pattern search for efficient action detection, January 2012.

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Department of Computer Science & EngineeringIIT (BHU), VaranasiVaranasiIndia
  2. 2.Department of Computer Sc. & EngineeringIK Gujral Punjab Technical UniversityKapurthalaIndia

Personalised recommendations