Skip to main content
SpringerLink
Log in

On Frame Selection for Video Face Recognition

  • Chapter
  • First Online:
Advances in Face Detection and Facial Image Analysis

Abstract

In surveillance applications, a video can be an advantageous alternative to traditional still image face recognition. In order to extract discriminative features from a video, frame based processing is preferred; however, not all frames are suitable for face recognition. While some frames are distorted due to noise, blur, and occlusion, others might be affected by the presence of covariates such as pose, illumination, and expression. Frames affected by imaging artifacts such as noise and blur may not contain reliable facial information and may affect the recognition performance. In an ideal scenario, such frames should not be considered for feature extraction and matching. Furthermore, video contains a large amount of frames and adjacent frames contain largely redundant information and processing all the frames from a video increases the computational complexity. Instead of utilizing all the frames from a video, frame selection can be performed to determine a subset of frames which is best suited for face recognition. Several frame selection algorithms have been proposed in the literature to address these concerns. In this chapter, we discuss the role and importance of frame selection in video face recognition, provide an overview of existing techniques, present an entropy based frame selection algorithm with the results and analysis on Point-and-Shoot-Challenge which is a recent benchmark database, and also propose a new paradigm for frame selection algorithms as a path forward.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
eBook
USD 16.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 54.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Equal contributions by the student authors (Tejas I. Dhamecha and Gaurav Goswami).

References

  1. K. Anantharajah, S. Denman, S. Sridharan, C. Fookes, D. Tjondronegoro, Quality based frame selection for video face recognition, in International Conference on Signal Processing and Communication Systems (2012), pp. 1–5

    Google Scholar 

  2. J.R. Barr, K.W. Bowyer, P.J. Flynn, S. Biswas, Face recognition from video: a review, in International Journal of Pattern Recognition and Artificial Intelligence, 26(5), 1266002 (2012), doi 10.1142/S0218001412660024. http://www.worldscientific.com/doi/abs/10.1142/S0218001412660024

    Google Scholar 

  3. Y. Bengio, P. Lamblin, D. Popovici, H. Larochelle, Greedy layer-wise training of deep networks, in Advances in Neural Information Processing Systems, vol. 19 (MIT Press, Cambridge, 2007), pp. 153–160

    Google Scholar 

  4. S.A. Berrani, C. Garcia, Enhancing face recognition from video sequences using robust statistics, in IEEE Conference on Advanced Video and Signal Based Surveillance (2005), pp. 324–329

    Google Scholar 

  5. L. Best-Rowden, B. Klare, J. Klontz, A.K. Jain, Video-to-video face matching: establishing a baseline for unconstrained face recognition, in International Conference on Biometrics: Theory, Applications and Systems (2013), pp. 1–8

    Google Scholar 

  6. J. Beveridge, P. Phillips, D. Bolme, B. Draper, G. Given, Y.M. Lui, M. Teli, H. Zhang, W. Scruggs, K. Bowyer, P. Flynn, S. Cheng, The challenge of face recognition from digital point-and-shoot cameras, in IEEE Conference on Biometrics: Theory, Applications and Systems (2013), pp. 1–8

    Google Scholar 

  7. S. Bharadwaj, M. Vatsa, R. Singh, Biometric quality: a review of fingerprint, iris, and face. EURASIP J. Image Video Process. 2014(1) (2014), doi 10.1186/1687-5281-2014-34. http://link.springer.com/article/10.1186%2F1687-5281-2014-34

  8. M. De Marsico, M. Nappi, D. Riccio, ES-RU: an entropy based rule to select representative templates in face surveillance. Multimedia Tools Appl. 73(1), 109–128 (2014)

    Article  Google Scholar 

  9. W. Fan, D.Y. Yeung, Face recognition with image sets using hierarchically extracted exemplars from appearance manifolds, in International Conference on Automatic Face and Gesture Recognition (2006), pp. 177–182

    Google Scholar 

  10. G. Goswami, R. Bhardwaj, R. Singh, M. Vatsa, MDLFace: memorability augmented deep learning for video face recognition, in International Joint Conference on Biometrics (2014), pp. 1–7

    Google Scholar 

  11. A. Hadid, M. Pietikainen, Selecting models from videos for appearance-based face recognition, in International Conference on Pattern Recognition (2004), pp. 304–308

    Google Scholar 

  12. M. Hubert, P.J. Rousseeuw, K. Vanden Branden, ROBPCA: a new approach to robust principal component analysis. Technometrics 47(1), 64–79 (2005)

    Article  MathSciNet  Google Scholar 

  13. A.K. Jain, S.Z. Li, Handbook of Face Recognition (Springer, New York, 2005)

    MATH  Google Scholar 

  14. R.R. Jillela, A. Ross, Adaptive frame selection for improved face recognition in low-resolution videos, in International Joint Conference on Neural Networks (2009), pp. 1439–1445

    Google Scholar 

  15. A. Khosla, W.A. Bainbridge, A. Torralba, A. Oliva, Modifying the memorability of face photographs, in IEEE International Conference on Computer Vision (2013), pp. 3200–3207

    Google Scholar 

  16. W. Liu, Z. Li, X. Tang, Spatio-temporal embedding for statistical face recognition from video, in European Conference on Computer Vision (2006), pp. 374–388

    Google Scholar 

  17. B.D. Lucas, T. Kanade, et al., An iterative image registration technique with an application to stereo vision, in International Joint Conference on Artificial Intelligence, vol. 81 (1981), pp. 674–679

    Google Scholar 

  18. E. Meyers, L. Wolf, Using biologically inspired features for face processing. Int. J. Comput. Vis. 76(1), 93–104 (2008)

    Article  Google Scholar 

  19. N. Otsu, A threshold selection method from gray-level histograms. Automatica 11(285–296), 23–27 (1975)

    Google Scholar 

  20. U. Park, A.K. Jain, 3D model-based face recognition in video, in Advances in Biometrics (Springer, Berlin, 2007), pp. 1085–1094

    Google Scholar 

  21. U. Park, A.K. Jain, A. Ross, Face recognition in video: adaptive fusion of multiple matchers, in IEEE Conference on Computer Vision and Pattern Recognition (2007), pp. 1–8

    Google Scholar 

  22. P.J. Phillips, J.R. Beveridge, B.A. Draper, G. Givens, A.J. O’Toole, D.S. Bolme, J. Dunlop, Y.M. Lui, H. Sahibzada, S. Weimer, An introduction to the good, the bad, & the ugly face recognition challenge problem, in IEEE International Conference on Automatic Face & Gesture Recognition Workshops (2011), pp. 346–353

    Google Scholar 

  23. A. Renyi, On measures of entropy and information, in Berkeley Symposium on Mathematical Statistics and Probability (1961), pp. 547–561

    Google Scholar 

  24. E.A. Rúa, J.L.A. Castro, C.G. Mateo, Quality-based score normalization and frame selection for video-based person authentication, in Biometrics and Identity Management (Springer, Berlin, 2008), pp. 1–9

    Google Scholar 

  25. U. Saeed, J.L. Dugelay, Temporally consistent key frame selection from video for face recognition, in European Signal Processing Conference (2010), pp. 1311–1315

    Google Scholar 

  26. R. Salakhutdinov, G.E. Hinton, Deep Boltzmann machines, in International Conference on Artificial Intelligence and Statistics (2009), pp. 448–455

    Google Scholar 

  27. L.K. Saul, S.T. Roweis, Think globally, fit locally: unsupervised learning of low dimensional manifolds. J. Mach. Learn. Res. 4, 119–155 (2003)

    MathSciNet  MATH  Google Scholar 

  28. P. Sinha, B. Balas, Y. Ostrovsky, R. Russell, Face recognition by humans: nineteen results all computer vision researchers should know about. Proc. IEEE 94(11), 1948–1962 (2006)

    Article  Google Scholar 

  29. C. Tomasi, T. Kanade, Shape and motion from image streams under orthography: a factorization method. Int. J. Comput. Vis. 9(2), 137–154 (1992)

    Article  Google Scholar 

  30. P. Vincent, H. Larochelle, I. Lajoie, Y. Bengio, P. Manzagol, Stacked denoising autoencoders: learning useful representations in a deep network with a local denoising criterion. J. Mach. Learn. Res. 11(12), 3371–3408 (2010)

    MathSciNet  MATH  Google Scholar 

  31. D. Yoshor, W.H. Bosking, G.M. Ghose, J.H.R. Maunsell, Receptive fields in human visual cortex mapped with surface electrodes. Cereb. Cortex 17(10), 2293–2302 (2007)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. IIIT Delhi, New Delhi, India

    Tejas I. Dhamecha, Gaurav Goswami, Richa Singh & Mayank Vatsa

Authors
  1. Tejas I. Dhamecha
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Gaurav Goswami
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Richa Singh
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Mayank Vatsa
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tejas I. Dhamecha .

Editor information

Editors and Affiliations

  1. Automatic Control, Silesian University of Technology, Kawulok, Poland

    Michal Kawulok

  2. Computer Science, Louisiana State University in Shreveport, Shreveport, Louisiana, USA

    M. Emre Celebi

  3. Silesian University of Technology, Gliwice, Poland

    Bogdan Smolka

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Dhamecha, T.I., Goswami, G., Singh, R., Vatsa, M. (2016). On Frame Selection for Video Face Recognition. In: Kawulok, M., Celebi, M., Smolka, B. (eds) Advances in Face Detection and Facial Image Analysis. Springer, Cham. https://doi.org/10.1007/978-3-319-25958-1_10

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-25958-1_10

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-25956-7

  • Online ISBN: 978-3-319-25958-1

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation