Skip to main content
SpringerLink
Log in

Face Recognition in Surveillance Video for Criminal Investigations: A Review

  • Conference paper
  • First Online:
Communication, Networks and Computing (CNC 2018)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 839))

Included in the following conference series:

Abstract

Face recognition from surveillance video in a forensic scenario is a very challenging task. Much of the existing work focuses on face recognition in a video where the video frames are of high-resolution, containing faces in frontal pose and in optimal lighting conditions. However, new challenges are encountered as applications of face recognition advance from cooperative and constrained scenarios to uncooperative subjects in unconstrained scenarios such as video surveillance. These challenges are due to low image resolution, variant expressions, face orientations, partial occlusion, complex background and the differences in surrounding illumination. In criminal investigations, the aim is to identify a culprit by collecting the information from the various face input media that includes video tracks, still face images, 3D model of a face, and verbal descriptions of the person presented by eyewitness. Face sketch may be generated from these verbal descriptions which can be matched against mug shot database. This paper reviews various methods and techniques used to identify a person of interest in an unconstrained environment with the above-mentioned challenges.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Tang, X., Wang, X.: Face sketch recognition. IEEE Trans. Circuits Syst. Video Technol. 14(1), 50–57 (2004). https://doi.org/10.1109/tcsvt.2003.818353

    Article  Google Scholar 

  2. Li, Y.-H., Savvides, M., Bhagavatula, V.: Illumination tolerant face recognition using a novel face from sketch synthesis approach and advanced correlation Filters. IEEE (2006). ISBN 1-4244-0469-X, ICASSP

    Google Scholar 

  3. Ouyang, S., et al: A survey on heterogeneous face recognition: sketch, infra-red, 3D and low-resolution. Image Vis. Comput. (2016). http://dx.doi.org/10.1016/.imavis

  4. Klare, B., Li, Z., Jain, A.: Matching forensic sketches to mug shot photos. IEEE Trans. Pattern Anal. Mach. Intell. 33(3), 639–646 (2011)

    Article  Google Scholar 

  5. Zhang, W., Wang, X., Tan, X.: Coupled information theoretic encoding for face photo-sketch recognition. In: Proceedings of IEEE CVPR (2011)

    Google Scholar 

  6. Wang, Z., Miao, Z., Wu, Q.J., Wan, Y., Tang, Z.: Low resolution face recognition: a review. Vis. Comput. 30(4), 359–386 (2014)

    Article  Google Scholar 

  7. Mahajan, K.S., Vaidya, M.B.: Image inpainting techniques: a survey. IOSR J. Comput. Eng. 5(4), 45–49 (2012)

    Article  Google Scholar 

  8. FACES 4.0, IQ Biometrix 2011 (2011). http://www.iqbiometrix.com

  9. Identi-Kit, Identi-Kit Solutions 2011. http://www.identikit.net/

  10. Wells, G., Hasel, L.: Facial composite production by eyewitnesses. Curr. Dir. Psychol. Sci. 16(1), 6–10 (2007)

    Article  Google Scholar 

  11. Suthar, R., Patel, K.R.: A survey on various image inpainting techniques to restore image. Int. J. Eng. Res. Appl. 4(2), 85–88 (2014). ISSN 2248-9622

    Google Scholar 

  12. Klum, S.J., Han, H., Klare, B.F., Jain, A.K.: The FaceSketchID system: matching facial composites to mugshots. IEEE Trans. Inf. Forensics Secur. 9(12), 2248–2263 (2014)

    Article  Google Scholar 

  13. Sharif, M., Adeel, K., Mudassar, R., Sajjad, M.: Face recognition using gabor filters. J. Appl. Comput. Sci. Math. 5(11), 53–57 (2011)

    Google Scholar 

  14. Wang, X., Han, T.X., Yan, S.: An HOG-LBP human detector with partial occlusion handling. In: 12th IEEE International Conference on Computer Vision (ICCV) (2009). ISBN 978-1-4244-4419-9

    Google Scholar 

  15. Hosoi, T., Koichi Ito, S.: Restoration of occluded regions using FW-PCA for face recognition. IEEE (2012). ISBN 978-1-4673-1612-5/12

    Google Scholar 

  16. Criminisi, A., Perez, P., Toyama, K.: Region filling and object removal by exemplar-based image inpainting. IEEE Trans. Image Process. 13(9), 1200–1212 (2004)

    Article  Google Scholar 

  17. Pandya, N., Limbasiya, B.: A survey on image inpainting techniques. Int. J. Curr. Eng. Technol. 3(5), 1828–1831 (2013)

    Google Scholar 

  18. Guillemot, C., Le Meur, O.: Image inpainting. IEEE Signal Process. Mag. January 2014. 1053-5888/14

    Google Scholar 

  19. Banday, M., Sharma, R.: Image inpainting – an inclusive review of the underlying algorithm and comparative study of the associated techniques. Int. J. Comput. Appl. 98(17), 0975–8887 (2014)

    Google Scholar 

  20. Shewale, O.: Restoration of image using SR based image inpainting techniques. IJISET – Int. J. Innov. Sci. Eng. Technol. 2(4) (2015). ISSN 2348–7968

    Google Scholar 

  21. Huang, J., Yuen, P.C., Chen, W.S., Lai, J.H.: Choosing parameters of kernel subspace LDA for recognition of face images under pose and illumination variations. IEEE Trans. Syst. Man Cybern. 37(4), 847–862 (2007). https://doi.org/10.1109/tsmcb.2007.895328

    Article  Google Scholar 

  22. Akhtar, Z., Rattani, A.: A face in any form: new challenges and opportunities for face recognition technology, pp. 0018–9162. IEEE (2017)

    Article  Google Scholar 

  23. Hu, X., Yu, W., Yao, J.: Multi-oriented 2DPCA for face recognition with one training face image per person. J. Comput. Inf. Syst. 6(5), 1563–1570 (2010)

    Google Scholar 

  24. Xu, Y., Li, X., Yang, J., Zhang, D.: Integrate the original face image and its mirror image for face recognition. Neurocomputing 131, 191–199 (2014)

    Article  Google Scholar 

  25. Hu, X., Liao, Q., Peng, S.: Video surveillance face recognition by more virtual training samples based on 3D modeling. In: 11th International Conference on Natural Computation (ICNC). IEEE (2015). ISBN 978-1-4673-7679-2

    Google Scholar 

  26. Zhang, X., Gao, Y., Leung, M.K.H.: Recognizing rotated faces from frontal and side views: an approach toward effective use of mugshot databases. IEEE Trans. Inf. Forensics Secur. 3(4), 684–697 (2008)

    Article  Google Scholar 

  27. Ghinea, G., Kannan, R., Kannaiyan, S.: Gradient-orientation-based PCA subspace for novel face recognition. IEEE Access (2014). https://doi.org/10.1109/access.2014.2348018

    Article  Google Scholar 

  28. Ding, C., Tao, D.: A comprehensive survey on pose-invariant face recognition. ACM Trans. Intell. Syst. Technol. arXiv:1502.04383v3 (2016)

  29. Lin, Y., Medioni, G., Choi, J.: Accurate 3D face reconstruction from weakly calibrated wide baseline images with profile contours. In: Proceedings of CVPR (2010)

    Google Scholar 

  30. Asthana, A., Marks, T.K., Jones, M.J., Tieu, K.H., Rohith, M.: Fully automatic pose-invariant face recognition via 3D pose normalization. In: Proceedings of ICCV (2011)

    Google Scholar 

  31. Matthews, I., Baker, S.: Active appearance models revisited. Int. J. Comput. Vis. 60, 135–164 (2004)

    Article  Google Scholar 

  32. Devi, N.S., Hemachandran, K.: Automatic face recognition system using pattern recognition techniques: a survey. Int. J. Comput. Appl. 83(5), 0975–8887 (2013)

    Google Scholar 

  33. Akhtar, Z., Rattani, A.: A face in any form: new challenges and opportunities for face recognition technology. IEEE Computer Society (2017). ISSN 0018-9162

    Google Scholar 

  34. Jiménez, D.G., Alba-Castro, J.L.: Toward pose-invariant 2-D face recognition through point distribution models and facial symmetry. IEEE Trans. Inf. Forensics Secur. 2(3–1), 413–429 (2007)

    Article  Google Scholar 

  35. Arya, S., Pratap, N., Bhatia, K.: Future of face recognition: a review. In: Second International Symposium on Computer Vision and the Internet (VisionNet 2015) (2015). https://doi.org/10.1016/j.procs.2015.08.076

    Article  Google Scholar 

  36. Grgic, M., Delac, K., Grgic, S.: SCface - surveillance cameras face database. Multimed. Tools Appl. J. 51(3), 863–879 (2011)

    Article  Google Scholar 

  37. Wolf, L., Hassner, T., Maoz, I.: Face recognition in unconstrained videos with matched background similarity. In: IEEE Conference on Computer Vision and Pattern Recognition (2011)

    Google Scholar 

  38. Learned-Miller, E., Huang, G.B., RoyChowdhury, A., Li, H., Hua, G.: Labeled faces in the wild: a survey. In: Kawulok, M., Celebi, M.E., Smolka, B. (eds.) Advances in Face Detection and Facial Image Analysis, pp. 189–248. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-25958-1_8

    Chapter  Google Scholar 

  39. http://www2.ece.ohio-state.edu/~aleix/ARdatabase.html

  40. http://www.iab-rubric.org/resources/sketchDatabase.html

  41. Sim, T., Baker, S., Bsat, M.: Technical report, CMU-RI-TR-01-02, Robotics Institute, Carnegie Mellon University, January 2001

    Google Scholar 

  42. Patil, S., Shubhangi, D.C.: Forensic sketch based face recognition using geometrical face model. In: I2CT (2017). ISBN 978-1-5090-4307-1

    Google Scholar 

  43. Zhang, W., Wang, X., Tang, X.: Coupled information-theoretic encoding for face photo-sketch recognition. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2011)

    Google Scholar 

  44. Sundaram, M., Mani, A.: Face recognition: demystification of multifarious aspect in evaluation metrics. In: Ramakrishnan, S. (ed.) Face Recognition - Semisupervised Classification, Subspace Projection and Evaluation Methods. InTech (2016) https://doi.org/10.5772/62825

    Google Scholar 

  45. Leung, T.K., Burl, M.C., Perona, P.: Finding faces in cluttered scenes using random labeled graph matching. In: Proceedings of Fifth IEEE International Conference on Computer Vision, pp. 637–644 (1995)

    Google Scholar 

  46. Cootes, T.F., Edwards, G.J., Taylor, C.J.: Active appearance models. In: Burkhardt, H., Neumann, B. (eds.) ECCV 1998. LNCS, vol. 1407, pp. 484–498. Springer, Heidelberg (1998). https://doi.org/10.1007/BFb0054760

    Chapter  Google Scholar 

  47. Sung, K.K., Poggio, T.: Example-based learning for view-based human face detection. IEEE Trans. Pattern Anal. Mach. Intell. 20(1), 39–51 (1998)

    Article  Google Scholar 

  48. Fergus, R., Perona, P., Zisserman, A.: Object class recognition by unsupervised scale-invariant learning. In: IEEE Conference on Computer Vision and Pattern Recognition, vol. 2, pp. 264–271 (2003)

    Google Scholar 

  49. Heisele, B., Serre, T., Pontil, M., Poggio, T.: Component-based face detection. In: IEEE Conference Computer Vision and Pattern Recognition, pp. 657–662 (2001)

    Google Scholar 

  50. Wang, X., Tang, X.: Face photo-sketch synthesis and recognition. TPAMI 31, 1955–1967 (2009)

    Article  Google Scholar 

  51. Zhong, J., Gao, X., Tian, C.: Face sketch synthesis using E-HMM and selective ensemble. In: ICASSP, pp. 485–488 (2007)

    Google Scholar 

  52. Liu, Q., Tang, X., Jin, H., Lu, H., Ma, S.: A nonlinear approach for face sketch synthesis and recognition. In: CPVR, pp. 1005–1010 (2005)

    Google Scholar 

  53. Khan, Z., Hu, Y., Mian, A.: Facial self similarity for sketch to photo matching. In: DICTA, pp. 1–7 (2012)

    Google Scholar 

  54. Galea, C., Farrugia, R.: Forensic face photo-sketch recognition using a deep learning based architecture. IEEE Signal Process. Lett. 24(11), 1586–1590 (2017)

    Article  Google Scholar 

  55. Galoogahi, H., Sim, T.: Face photo retrieval by sketch example. ACM, pp. 1–4 (2012)

    Google Scholar 

  56. Bhatt, H., Bharadwaj, S., Singh, R., Vatsa, M.: On matching sketches with digital face images. In: BTAS, pp. 1–7 (2010)

    Google Scholar 

  57. Galoogahi, H., Sim, T.: Inter-modality face sketch recognition. In: ICME, pp. 224–229 (2012)

    Google Scholar 

  58. Galoogahi, H., Sim, T.: Face sketch recognition by local randon binary pattern: LRBP. In: ICIP, pp. 1837–1840 (2012)

    Google Scholar 

  59. Pramanik, S., Bhattacharjee, D.: Geometric feature based face-sketch recognition. In: PRIME, pp. 409–415 (2012)

    Google Scholar 

  60. Peng, C., Gao, X., Wang, N., Tao, D.: Multiple representations-based face sketch photo synthesis. IEEE Trans. Neural Netw. Learn. Syst. 27(11), 2201–2215 (2016)

    Article  Google Scholar 

  61. Mittal, P., Jain, A., Goswami, G., Vatsa, M., Singh, R.: Composite sketch recognition using saliency and attribute feedback. Inf. Fusion 33, 86–99 (2017)

    Article  Google Scholar 

  62. Galea, C., Farrugia, R.: A large-scale software-generated face composite sketch database. In: Proceedings of International Conference Biometrics Special Interest Group, pp. 1–5 (2016)

    Google Scholar 

  63. Masi, I., et al.: Learning pose-aware models for pose-invariant face recognition in the wild (2017). https://doi.org/10.1109/tpami.2018.2792452

  64. Yin, X., Liu, X.: Multi-task convolutional neural network for pose-invariant face recognition, vol. 27, no. 2 (2018). https://doi.org/10.1109/tip.2017.2765830

    Article  MathSciNet  Google Scholar 

  65. Masi, I., Rawls, S., Medioni, G., Natarajan, P.: Pose-aware face recognition in the wild. In: Proceedings of Conference Computer Vision Pattern Recognition (2016)

    Google Scholar 

  66. Klare, B.F., et al.: Pushing the frontiers of unconstrained face detection and recognition: IARPA janus benchmark a. In: CPVR, pp. 1931–1939 (2015)

    Google Scholar 

  67. Peng, X., Yu, X., Sohn, K., Metaxas, D.N., Chandraker, M.: Reconstruction-based disentanglement for pose-invariant face recognition. In: Proceedings of ICCV, pp. 1–10 (2017)

    Google Scholar 

  68. Ranjan, R., Patel, V.M., Chellappa, R.: HyperFace: a deep multi-task learning framework for face detection, landmark localization, pose estimation, and gender recognition (2017). https://doi.org/10.1109/tpami.2017.2781233

Download references

Author information

Authors and Affiliations

  1. Nitte Meenakshi Institute of Technology, Bengaluru, India

    Napa Lakshmi

  2. Ramaiah Institute of Technology, Bengaluru, India

    Megha P. Arakeri

Authors
  1. Napa Lakshmi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Megha P. Arakeri
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Napa Lakshmi .

Editor information

Editors and Affiliations

  1. Indian Institute of Information Technology, Allahabad, India

    Shekhar Verma

  2. ITM University, Gwalior, India

    Ranjeet Singh Tomar

  3. ITM University, Gwalior, India

    Brijesh Kumar Chaurasia

  4. Indian Institute of Information Technology, Allahabad, India

    Vrijendra Singh

  5. School of Information Technology, Deakin University, Burwood, VIC, Australia

    Jemal Abawajy

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Lakshmi, N., Arakeri, M.P. (2019). Face Recognition in Surveillance Video for Criminal Investigations: A Review. In: Verma, S., Tomar, R., Chaurasia, B., Singh, V., Abawajy, J. (eds) Communication, Networks and Computing. CNC 2018. Communications in Computer and Information Science, vol 839. Springer, Singapore. https://doi.org/10.1007/978-981-13-2372-0_31

Download citation

  • DOI: https://doi.org/10.1007/978-981-13-2372-0_31

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-13-2371-3

  • Online ISBN: 978-981-13-2372-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation