Sketch Synthesized Face Recognition with Deep Learning Models

  • Wei Shao
  • Zhicheng Chen
  • Guangben Lu
  • Xiaokang Tu
  • Yuchun FangEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10996)


Sketch face recognition is of great significance in the field of criminal investigation, Internet search and management. In this paper, we explore the feature presentation of sketch synthesized face images with several deep learning models. In order to complete the matching of heterogeneous images, we propose a modified face synthesis technology that combines sketches and face templates into a human face portrait. Through experiments, we investigate the essential problem of the degree of synthetic with respect to face recognition. Several state-of-the-art Deep Neural Network (DNN) models in face recognition are transferred in feature extraction of sketch synthesized face images. Experiments show that the proposed synthetic method is effective working with the DNN models in sketch face recognition.


Heterogeneous face recognition Sketch face recognition CNNs Face synthesis 



The work is funded by the Shanghai Undergraduate Student Innovation Project, the National Natural Science Foundation of China (No. 61170155) and the Shanghai Innovation Action Plan Project (No. 16511101200).


  1. 1.
    Schroff, F., Kalenichenko, D., Philbin, J.: FaceNet: a unified embedding for face recognition and clustering, pp. 815–823 (2015)Google Scholar
  2. 2.
    Galton, F.: Personal identification and description. J. Anthropol. Inst. Great Br. Irel. 18(973), 177–191 (1889)CrossRefGoogle Scholar
  3. 3.
    Wagner, A., et al.: Toward a practical face recognition system: robust alignment and illumination by sparse representation. IEEE Trans. Pattern Anal. Mach. Intell. 34(2), 372–386 (2012)CrossRefGoogle Scholar
  4. 4.
    Tang, X., Wang, X.: Face sketch recognition. IEEE Trans. Circ. Syst. Video Technol. 14(1), 50–57 (2004)CrossRefGoogle Scholar
  5. 5.
    Wang, X., Tang, X.: Face photo-sketch synthesis and recognition. In: IEEE International Conference on Computer Vision, p. 687. IEEE Computer Society (2003)Google Scholar
  6. 6.
    Gao, X., et al.: Face sketch synthesis algorithm based on E-HMM and selective ensemble. IEEE Trans. Circ. Syst. Video Technol. 18(4), 487–496 (2008)CrossRefGoogle Scholar
  7. 7.
    Xiao, B., et al.: A new approach for face recognition by sketches in photos. Signal Process. 89(8), 1576–1588 (2009)CrossRefGoogle Scholar
  8. 8.
    Tang, X., Wang, X.: Face photo recognition using sketch. In: 2002 Proceedings of the IEEE International Conference on Image Processing, vol. 1, pp. I-257–I-260 (2002)Google Scholar
  9. 9.
    Liu, Q., et al.: A nonlinear approach for face sketch synthesis and recognition. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 1005–1010. IEEE Computer Society (2005)Google Scholar
  10. 10.
    Hu, Y.: Research on sketch-based cross-modal face retrieval framework. Beijing University of Posts and Telecommunications (2017)Google Scholar
  11. 11.
    Liu, X., et al.: VIPLFaceNet: an open source deep face recognition SDK. Front. Comput. Sci. 11(2), 208–218 (2017)CrossRefGoogle Scholar
  12. 12.
    Parkhi, O.M., Vedaldi, A., Zisserman, A.: Deep face recognition. In: British Machine Vision Conference, pp. 41.1–41.12 (2015)Google Scholar
  13. 13.
    Wang, N., et al.: A comprehensive survey to face hallucination. Int. J. Comput. Vis. 106(1), 9–30 (2014)CrossRefGoogle Scholar
  14. 14.
    Zhang, W., Wang, X., Tang, X.: Coupled information-theoretic encoding for face photo-sketch recognition. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 513–520. IEEE Computer Society (2011)Google Scholar
  15. 15.
    Liong, V.E., et al.: Deep coupled metric learning for cross-modal matching. IEEE Trans. Multimed. 19(6), 1234–1244 (2017)CrossRefGoogle Scholar
  16. 16.
    Roy, H., Bhattacharjee, D.: Face sketch-photo recognition using local gradient checksum: LGCS. Int. J. Mach. Learn. Cybern. 8, 1–13 (2016)Google Scholar
  17. 17.
    Gao, X., et al.: Face sketch–photo synthesis and retrieval using sparse representation. IEEE Trans. Circ. Syst. Video Technol. 22(8), 1213–1226 (2012)CrossRefGoogle Scholar
  18. 18.
    De Freitas Pereira, T., Marcel, S.: Heterogeneous face recognition using inter-session variability modelling. In: Computer Vision and Pattern Recognition Workshops, pp. 179–186. IEEE (2016)Google Scholar
  19. 19.
    Klare, B.F., Jain, A.K.: Heterogeneous face recognition using kernel prototype similarities. IEEE Trans. Pattern Anal. Mach. Intell. 35(6), 1410–1422 (2013)CrossRefGoogle Scholar
  20. 20.
    Zhang, W., Wang, X., Tang, X.: Lighting and Pose Robust Face Sketch Synthesis. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6316, pp. 420–433. Springer, Heidelberg (2010). Scholar
  21. 21.
    Wang, N., Gao, X., Li, J.: random sampling for fast face sketch synthesis. Pattern Recogn. 76, 215–227 (2017)CrossRefGoogle Scholar
  22. 22.
    Klare, B.F., et al.: Matching composite sketches to face photos: a component-based approach. IEEE Trans. Inf. Forensics Secur. 8(1), 191–204 (2013)CrossRefGoogle Scholar
  23. 23.
    Klare, B., Li, Z., Jain, A.K.: Matching forensic sketches to mug shot photos. IEEE Trans. Pattern Anal. Mach. Intell. 33(3), 639–646 (2011)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Wei Shao
    • 1
  • Zhicheng Chen
    • 1
  • Guangben Lu
    • 1
  • Xiaokang Tu
    • 1
  • Yuchun Fang
    • 1
    Email author
  1. 1.School of Computer Engineering and ScienceShanghai UniversityShanghaiChina

Personalised recommendations