Abstract
We propose an approach for face sketch synthesis by employing deep image transformations using an artistic style transfer algorithm. Face sketch synthesis remains an area of great interest in the research community as well as its applications in law enforcement towards face recognition. Recent methods for this problem typically employ traditional approaches to synthesize face sketches to digital images. However, most approaches are gradually shifting towards convolutional neural networks for robust feature learning and image transformations. In this paper, we propose an approach that uses recent artistic style transfer algorithms for face sketch synthesis. Additionally, we show that poorly synthesized images can be improved with a denoising autoencoder for better facial feature reconstruction. Further, the approach is extended to perform face verification of heterogeneous image samples to assess the effectiveness of the proposed approach and gives a better view into the potential applications for styling algorithms for face image synthesis and transformation problems alike.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Ashwini, B., Prajakta, K.: A survey of face recognition from sketches. Int. J. Latest Trends Eng. Technol. 6(3), 150–158 (2016)
Ouyang, S., et al.: A survey on heterogeneous face recognition: Sketch, infra-red, 3D and low-resolution. arXiv preprint arXiv:1409.5114 (2014)
Wang, X., Tang, X.: Face photo-sketch synthesis and recognition. IEEE Trans. Pattern Anal. Mach. Intell. 31(11), 1955–1967 (2009)
Gao, X., et al.: Face sketch synthesis algorithm based on E-HMM and selective ensemble. IEEE Trans. Circuits Syst. Video Technol. 18(4), 487–496 (2008)
Gao, X., et al.: Face sketch–photo synthesis and retrieval using sparse representation. IEEE Trans. Circuits Syst. Video Technol. 22(8), 1213–1226 (2012)
Gatys, L.A., Ecker, A.S., Bethge, M.: Image style transfer using convolutional neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2016)
Johnson, J., Alahi, A., Fei-Fei, L.: Perceptual losses for real-time style transfer and super-resolution. In: European Conference on Computer Vision. Springer (2016)
Gatys, L.A., Ecker, A.S., Bethge, M.: A neural algorithm of artistic style. arXiv preprint arXiv:1508.06576 (2015)
Ulyanov, D., Vedaldi, A., Lempitsky, V.: Instance normalization: the missing ingredient for fast stylization. arXiv preprint arXiv:1607.08022 (2016)
Jain, A.K., Klare, B., Park, U.: Face matching and retrieval in forensics applications. IEEE Multimed. 19(1), 20 (2012)
Tang, X., Wang, X.: Face photo recognition using sketch. In: Image Proceedings 2002 International Conference on IEEE (2002)
Liu, Q., et al.: A nonlinear approach for face sketch synthesis and recognition. In: 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2005) (2005)
Pramanik, S., Bhattacharjee, D.: An approach: modality reduction and face-sketch recognition (2013)
Saxena, S., Verbeek, J.: Heterogeneous face recognition with CNNs. In: Computer Vision–ECCV 2016 Workshops. Springer (2016)
Güçlütürk, Y., et al.: Convolutional sketch inversion. arXiv preprint arXiv:1606.03073 (2016)
Bigdeli, S.A., Zwicker, M.: Image restoration using autoencoding priors. arXiv preprint arXiv:1703.09964 (2017)
Burger, H.C., Schuler, C.J., Harmeling, S.: Image denoising: can plain neural networks compete with BM3D? In: 2012 IEEE Conference on IEEE Computer Vision and Pattern Recognition (CVPR) (2012)
Dong, C., et al.: Image super-resolution using deep convolutional networks. IEEE Trans. Pattern Anal. Mach. Intell. 38(2), 295–307 (2016)
Zhang, W., Wang, X., Tang, X.: Coupled information-theoretic encoding for face photo-sketch recognition. In: 2011 IEEE Conference on IEEE Computer Vision and Pattern Recognition (CVPR) (2011)
Vincent, P., et al.: Extracting and composing robust features with denoising autoencoders. In: Proceedings of the 25th International Conference on Machine Learning. ACM (2008)
Phillips, P.J., et al.: The feret evaluation, in face recognition, pp. 244–261. Springer (1998)
Phillips, P.J., et al.: The FERET database and evaluation procedure for face-recognition algorithms. Image Vis. Comput. 16(5), 295–306 (1998)
Weston, J., Watkins, C.: Support vector machines for multi-class pattern recognition. In: ESANN (1999)
Hearst, M.A., et al.: Support vector machines. IEEE Intell. Syst. Appl. 13(4), 18–28 (1998)
Acknowledgements
This research was supported by Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education (GR 2016R1D1A3B03931911). This research was supported by the MSIT(Ministry of Science and ICT), Korea, under the ITRC(Information Technology Research Center) support program (IITP-2017-2015-0-00378) supervised by the IITP(Institute for Information & Communications Technology Promotion).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Chikontwe, P., Lee, H.J. (2018). Towards Robust Face Sketch Synthesis with Style Transfer Algorithms. In: Kim, K., Kim, H., Baek, N. (eds) IT Convergence and Security 2017. Lecture Notes in Electrical Engineering, vol 449. Springer, Singapore. https://doi.org/10.1007/978-981-10-6451-7_21
Download citation
DOI: https://doi.org/10.1007/978-981-10-6451-7_21
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-10-6450-0
Online ISBN: 978-981-10-6451-7
eBook Packages: EngineeringEngineering (R0)