Gravitational search-based optimal deep neural network for occluded face recognition system in videos


Video surveillance is an effective method to improve public safety and privacy. Video surveillance technology has entered a stage where increasing video cameras are inexpensive but requiring open staff to evaluate the videos is expensive. The extensive researches conducted using machine learning for automatic face recognition doesn’t provide accurate results as of human evaluation. To enhance the biometric features of the security applications, automatic face recognition is used. Surveillance records incorporate various challenges for face recognition and face detection. For instance, facial recognition systems can be affected by the size of a face image, occlusion, posture, lighting conditions, and establishment, while recognition accuracy may be affected as a result of low objectives, occlusion, posture, light, and dimness. To conquer these obstacles, an effective face detection and recognition framework is proposed with optimal feature extraction methods. At first, the keyframes with face pictures are extracted using a strategy known as keyframe extraction using wavelet information. After the extraction of keyframes, the multi-angle movement feature, SURF feature, holo-entropy, and appearance features are used for feature extraction. Finally, the recognition can be done using optimal deep neural network based on the gravitational search algorithm. Therefore, the proposed method’s performance is evaluated using various benchmark video dataset. The efficiency of the proposed approach is evaluated by comparing it using sensitivity, specificity, accuracy, keyframe extraction time, etc.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11


  1. Arceda, M., Fernández Fabián, V. E., Laguna Laura, K. M., Rivera Tito, J. J., & Gutiérrez Cáceres, J. C. (2016). Fast face detection in violent video scenes. Electronic Notes in Theoretical Computer Science,329, 5–26.

    Article  Google Scholar 

  2. Atan, O., Andreopoulos, Y., Tekin, C., & van der Schaar, M. (2013). Bandit framework for systematic learning in wireless video-based face recognition. IEEE Journal of Selected Topics in Signal Processing.

    Article  Google Scholar 

  3. Du, M., & Chellappa, R. (2016). Face association for videos using conditional random fields and max-margin markov networks. In IEEE Transactions on Pattern Analysis and Machine Intelligence (Vol. 38, no. 9, pp. 1762–1773).

  4. Ganguly, S., Bhattacharjee, D., & Nasipuri, M. (2015). Wavelet and decision fusion-based 3D face recognition from range image. International Journal of Applied Pattern Recognition,2(4), 306–324.

    Article  Google Scholar 

  5. Hinton, G. (2010). A practical guide to training restricted Boltzmann machines. University of Toronto, UTML TR 2010-003.

  6. Hu, X., Liao, Q., & Peng, S. (2015). Video surveillance face recognition by more virtual training samples based on 3D modeling. In Proceedings of 11th international conference on natural computation (ICNC), 2015.

  7. Huang, Z., Wang, R., Shan, S., & Chen, X. (2013). Face recognition on large-scale video in the wild with hybrid Euclidean-and-Riemannian metric learning. Elsevier, Computer Vision and Image Understanding,117(10), 1384–1399.

    Article  Google Scholar 

  8. Li, H., Hua, G., Shen, X., Lin, Z., & Brandt, J. (2015). Eigen-PEP for video face recognition. In Computer visionACCV 2014, 9005 (pp. 17–33). Springer.

  9. Li, S., Neupane, A., Paul, S., Song, C., Krishnamurthy, S., Roy-Chowdhury, A. K., & Swami, A. (2019). Stealthy adversarial perturbations against real-time video classification systems. In NDSS, 2019.

  10. Liao, S., Jain, A. K., & Li, S. Z. (2013). Partial face recognition: alignment-free approach. IEEE Transactions on Pattern Analysis and Machine Intelligence,35(5), 1–14.

    Article  Google Scholar 

  11. Lu, Y., & Li, Z.-N. (2008). Automatic object extraction and reconstruction in active video. Pattern Recognition,41(3), 1159–1172.

    Article  Google Scholar 

  12. Medioni, G., Choi, J., Kuo, C.-H., & Fidaleo, D. (2019). Identifying noncooperative subjects at a distance using face images and inferred three-dimensional face models. IEEE Transactions on Systems, Man, and Cybernetics—Part A: Systems And Humans,39(1), 12–24.

    Article  Google Scholar 

  13. Mishra, P. K., & Saroha, G. P. (2016). A study on classification for static and moving object in video surveillance system. International Journal of Image Graphics and Signal Processing.

    Article  Google Scholar 

  14. Ngo, T. D., Le, D.-D., Satoh, S., & Duong, D. A. (2008). Robust face track finding in video using tracked points. In Proceedings of IEEE international conference on signal image technology and internet based systems, 2008.

  15. Omaima, N., & AL-Allaf, A. (2014). Review of face detection systems based artificial neural networks algorithms. International Journal of Multimedia and Its Applications,6, 1–16.

    Google Scholar 

  16. Pagano, C., Granger, E., Sabourin, R., & Gorodnichy, D. O. (2012). Detector ensembles for face recognition in video surveillance. In The 2012 international joint conference on neural networks (IJCNN) (pp. 1–8).

  17. Pandey, S. (2014). Review: Face detection and recognition techniques. International Journal of Computer Science and Information Technologies,5, 4111–4117.

    Google Scholar 

  18. Passalis, G., Perakis, P., Theoharis, T., & Kakadiaris, I. A. (2011). Using facial symmetry to handle pose variations in real-world 3D face recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence,33(10), 1938–1951.

    Article  Google Scholar 

  19. Ragashe, M. U., Goswami, M. M., & Raghuwanshi, M. M. (2015). Approach towards real time face recognition in streaming video under partial occlusion. In Proceedings of IEEE sponsored 9th international conference on intelligent systems and control (ISCO), 2015.

  20. Rejeesh, M. R. (2019). Interest point based face recognition using adaptive neuro fuzzy inference system. Multimedia Tools and Applications, 78(16), 22691–22710.

    Article  Google Scholar 

  21. Ramalingam, S. P., & Chandra Mouli, P. V. S. S. R. (2016). Two-level dimensionality reduced local directional pattern for face recognition. International Journal of Biometrics,8(1), 52–64.

    Article  Google Scholar 

  22. Sadeghipour, E., & Sahragard, N. (2016). Face recognition based on improved SIFT algorithm. International Journal of Advanced Computer Science and Applications,7(1), 548–551.

    Article  Google Scholar 

  23. Salman, A., Siddiqui, S. A., Shafait, F., Mian, A., Shortis, M. R., Khurshid, K., et al. (2019). Automatic fish detection in underwater videos by a deep neural network-based hybrid motion learning system. ICES Journal of Marine Science.

    Article  Google Scholar 

  24. Saravanan, D. (2016). Video substance extraction using image future population based techniques. ARPN Journal of Engineering and Applied Science,11(11), 7041–7045.

    Google Scholar 

  25. Sarode, J. P., & Anuse, A. D. (2014). A framework for face classification under pose variations. In International conference on advances in computing, communications and informatics (ICACCI) (pp. 1886–1891).

  26. Shen, H., Zhang, J., & Zhang, H. (2015). Human action recognition by random features and hand-crafted features: A comparative study. In Computer visionECCV 2014 workshops, 8926 (pp. 14–28). Springer.

  27. Shieh, W.-Y., & Huang, J.-C. (2009). Speedup the multi-camera video-surveillance system for elder falling detection. In Proceedings of international conferences on embedded software and systems, 2009.

  28. Shirley, C. P., Lenin Fred, A., & Ram Mohan, N. R. (2016). Video key frame extraction through wavelet information scheme. ARPN Journal of Engineering and Applied Sciences, 11(7).

  29. Smeets, D., Keustermans, J., Hermans, J., Claes, P., Vandermeulen, D., & Suetens, P. (2011). Symmetric surface-feature based 3D face recognition for partial data, biometrics (IJCB). In 2011 International joint conference on IEEE biometrics compendium, IEEE RFIC virtual journal, IEEE RFID virtual journal (pp. 1–6).

  30. Srivastava, G., Yoder, J. A., Park, J., & Kak, A. C. (2013). Using objective ground-truth labels created by multiple annotators for improved video classification: A comparative study. Computer Vision and Image Understanding,117(10), 1384–1399.

    Article  Google Scholar 

  31. Sudha, N., Mohan, A. R., & Meher, P. K. (2011). A self-configurable systolic architecture for face recognition system based on principal component neural network. IEEE Transactions on Circuits and Systems for Video Technology.

    Article  Google Scholar 

  32. Sundararaj, V. (2016). An efficient threshold prediction scheme for wavelet based ECG signal noise reduction using variable step size firefly algorithm. International Journal of Intelligent Engineering and Systems,9(3), 117–126.

    Article  Google Scholar 

  33. Sundararaj, V. (2019a). Optimised denoising scheme via opposition-based self-adaptive learning PSO algorithm for wavelet-based ECG signal noise reduction. International Journal of Biomedical Engineering and Technology,31(4), 325.

    Article  Google Scholar 

  34. Sundararaj, V. (2019b). Optimal task assignment in mobile cloud computing by queue based ant–bee algorithm. Wireless Personal Communications,104(1), 173–197.

    Article  Google Scholar 

  35. Tsagkatakis, G., & Savakis, A. (2009). Random projections for face detection under resource constraints. In Proceedings of 16th IEEE international conference on image processing (ICIP).

  36. Video dataset from

  37. Vinu, S., Selvi, M., & Kumar, R. S. (2018). An optimal cluster formation based energy efficient dynamic scheduling hybrid MAC protocol for heavy traffic load in wireless sensor networks. Computers and Security,77, 277–288.

    Article  Google Scholar 

  38. Wang, S., Zhu, E., Yin, J., & Porikli, F. (2018). Video anomaly detection and localization by local motion based joint video representation and OCELM. Neurocomputing,277(14), 161–175.

    Article  Google Scholar 

  39. Xie, J., Girshick, R., & Farhadi, A. (2016). Deep3d: Fully automatic 2d-to-3d video conversion with deep convolutional neural networks. In European conference on computer vision (pp. 842–857).

  40. Yew, C. T., & Suandi, S. A. (2011). A study on face recognition in video surveillance system using multi-class support vector machines. In Proceedings of IEEE region 10 conference, TENCON, 2011.

  41. Yoder, J., Medeiros, H., Park, J., & Kak, A. C. (2010). Cluster-based distributed face tracking in camera networks. IEEE Transactions on Image Processing,19(10), 2551–2563.

    MathSciNet  Article  Google Scholar 

  42. Yoganand, A., & Aruldoss, C. K. (2015). A region growing and modified neural network classifier based face detection technique from video. International Journal of Applied Engineering Research,10(12), 30231–30248.

    Google Scholar 

  43. Zafeiriou, S., Zhang, C., & Zhang, Z. (2015). A survey on face detection in the wild: Past. Present and Future, Elsevier, Computer Vision and Image Understanding,138, 1–24.

    Article  Google Scholar 

Download references

Author information



Corresponding authors

Correspondence to C. P. Shirley or N. R. Ram Mohan.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Shirley, C.P., Ram Mohan, N.R. & Chitra, B. Gravitational search-based optimal deep neural network for occluded face recognition system in videos. Multidim Syst Sign Process (2020).

Download citation


  • Face recognition
  • Deep neural network
  • Gravitational search algorithm (GSA)
  • Video surveillance
  • Keyframe extraction
  • Discrete wavelet