Abstract
To narrow the inherent gap between heterogeneous face domains, much efforts have been made in the past decade. Some of these solutions resort to deep generative networks to synthesize proper face images based on inputs from another domain. It is widely agreed that facial topology tends to be closely related to identity information, since facial appearance has been deemed as a crucial feature for identifying individuals. A small topological change on the face may make one person become another. Taking makeup removal as an example, the synthesized output is expected to resemble the input except for the makeup effects, and the facial topology is premier to be preserved during synthesis. We generalize this kind of problem as topology-invariant facial synthesis. In this chapter, we select several representative topology-invariant synthesis tasks and elaborate their latest progress. These tasks include cross-spectral face hallucination, makeup removal and age progression/regression. For each task, we start from its background and characteristics, followed by the illustration of latest methods as well as their experimental outcomes.
Part of this chapter is reprinted from Song et al. [30], Li et al. [18, 20,21,22] with permission from AAAI, Springer and IEEE.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsNotes
- 1.
https://github.com/ly-joy/Cross-Makeup-Face
References
Bulat, A., Tzimiropoulos, G.: How far are we from solving the 2d and 3d face alignment problem? (and a dataset of 230,000 3d facial landmarks). In: IEEE International Conference on Computer Vision, pp. 1021–1030 (2017)
Burlando, B., Verotta, L., Cornara, L., Bottini-Massa, E.: Herbal Principles in Cosmetics: Properties and Mechanisms of Action. CRC Press, Boca Raton (2010)
Chen, C., Dantcheva, A., Ross, A.: An ensemble of patch-based subspaces for makeup-robust face recognition. Inf. Fusion 32, 80–92 (2016)
Dantcheva, A., Chen, C., Ross, A.: Can facial cosmetics affect the matching accuracy of face recognition systems? In: The Fifth International Conference on Biometrics: Theory, Applications and Systems, pp. 391–398. IEEE (2012)
Face++ research toolkit. http://www.faceplusplus.com
Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y.: Generative adversarial nets. In: Advances in Neural Information Processing Systems, pp. 2672–2680 (2014)
Güler, R.A., Trigeorgis, G., Antonakos, E., Snape, P., Zafeiriou, S., Kokkinos, I.: Densereg: fully convolutional dense shape regression in-the-wild. In: The IEEE Conference on Computer Vision and Pattern Recognition, vol. 2, p. 5 (2017)
Güler, R.A., Neverova, N., Kokkinos, I.: Densepose: dense human pose estimation in the wild. In: The IEEE Conference on Computer Vision and Pattern Recognition (2018)
Guo, G., Wen, L., Yan, S.: Face authentication with makeup changes. IEEE Trans. Circuits Syst. Video Technol. 24(5), 814–825 (2014)
Hu, J., Ge, Y., Lu, J., Feng, X.: Makeup-robust face verification. In: International Conference on Acoustics, Speech and Signal Processing, pp. 2342–2346 (2013)
Huang, D.A., Frank Wang, Y.C.: Coupled dictionary and feature space learning with applications to cross-domain image synthesis and recognition. In: IEEE International Conference on Computer Vision (2013)
Huang, R., Zhang, S., Li, T., He, R.: Beyond face rotation: global and local perception GAN for photorealistic and identity preserving frontal view synthesis. In: IEEE International Conference on Computer Vision (2017)
Isola, P., Zhu, J.Y., Zhou, T., Efros, A.A.: Image-to-image translation with conditional adversarial networks. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 5967–5976. IEEE (2017)
Lei, Z., Bai, Q., He, R., Li, S.Z.: Face shape recovery from a single image using CCA mapping between tensor spaces. In: IEEE Conference on Computer Vision and Pattern Recognition (2008)
Lei, Z., Liao, S., Jain, A.K., Li, S.Z.: Coupled discriminant analysis for heterogeneous face recognition. IEEE Trans. Inf. Forensics Secur. 7(6), 1707–1716 (2012)
Lezama, J., Qiu, Q., Sapiro, G.: Not afraid of the dark: NIR-VIS face recognition via cross-spectral hallucination and low-rank embedding. In: IEEE Conference on Computer Vision and Pattern Recognition (2017)
Li, P., Hu, Y., Li, Q., He, R., Sun, Z.: Global and local consistent age generative adversarial networks. In: The 24th International Conference on Pattern Recognition (2018)
Li, P., Hu, Y., He, R., Sun, Z.: Global and local consistent wavelet-domain age synthesis. IEEE Trans. Inf. Forensics Secur. 14(11), 2943–2957 (2019). https://doi.org/10.1109/TIFS.2019.2907973
Li, S.Z., Dong, Y., Zhen, L., Liao, S.: The CASIA NIR-VIS 2.0 face database. In: IEEE Conference on Computer Vision and Pattern Recognition Workshops (2013)
Li, Y., Song, L., Wu, X., He, R., Tan, T.: Anti-makeup: learning a bi-level adversarial network for makeup-invariant face verification. In: The Thirty-Second AAAI Conference on Artificial Intelligence. Copyright \(\copyright \) AAAI (2018). www.aaai.org
Li, Y., Huang, H., Cao, J., He, R., Tan, T.: Disentangled representation learning of makeup portraits in the wild. Int. J. Comput. Vis. (2019)
Li, Y., Huang, H., Yu, J., He, R., Tan, T.: Semantic-aware makeup cleanser. In: The 10th IEEE International Conference on Biometrics: Theory, Applications and Systems (2019)
Li, Y., Song, L., Wu, X., He, R., Tan, T.: Learning a bi-level adversarial network with global and local perception for makeup-invariant face verification. Pattern Recognit. 90, 99–108 (2019)
Li, Z., Hu, Y., Zhang, M., Xu, M., He, R.: Protecting your faces: meshfaces generation and removal via high-order relation-preserving cyclegan. In: The 11th IAPR International Conference on Biometrics, pp. 61–68 (2018)
Radford, A., Metz, L., Chintala, S.: Unsupervised representation learning with deep convolutional generative adversarial networks. Comput. Sci. (2015)
Ricanek, K., Tesafaye, T.: MORPH: a longitudinal image database of normal adult age-progression. In: International Conference on Automatic Face and Gesture Recognition, pp. 341–345. IEEE (2006)
Ruprecht, D., Muller, H.: Image warping with scattered data interpolation. IEEE Comput. Graph. Appl. 15(2), 37–43 (1995)
Shrivastava, A., Pfister, T., Tuzel, O., Susskind, J., Wang, W., Webb, R.: Learning from simulated and unsupervised images through adversarial training. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2242–2251 (2017)
Socolinsky, D.A., Selinger, A.: A comparative analysis of face recognition performance with visible and thermal infrared imagery. Technical report, DTIC Document (2002)
Song, L., Zhang, M., Wu, X., He, R.: Adversarial discriminative heterogeneous face recognition. In: Thirty-Second AAAI Conference on Artificial Intelligence. Copyright \(\copyright \) AAAI (2018). www.aaai.org
Sun, Y., Ren, L., Wei, Z., Liu, B., Zhai, Y., Liu, S.: A weakly supervised method for makeup-invariant face verification. Pattern Recognit. 66, 153–159 (2017)
Tang, X., Wang, X.: Face sketch recognition. IEEE Trans. Circuits Syst. Video Technol. 14(1), 50–57 (2004)
Wei, Z., Sun, Y., Wang, J., Lai, H., Liu, S.: Learning adaptive receptive fields for deep image parsing network. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2434–2442 (2017)
Wu, X., He, R., Sun, Z., Tan, T.: A light CNN for deep face representation with noisy labels. IEEE Trans. Inf. Forensics Secur. 13(11), 2884–2896 (2018)
Yang, H., Huang, D., Wang, Y., Jain, A.K.: Learning face age progression: a pyramid architecture of GANs. In: IEEE Conference on Computer Vision and Pattern Recognition (2018)
Yi, D., Liu, R., Chu, R., Lei, Z., Li, S.Z.: Face matching between near infrared and visible light images. In: International Conference on Biometrics (2007)
Yu, J., Cao, J., Li, Y., Jia, X., He, R.: Pose-preserving cross spectral face hallucination. In: IJCAI (2019)
Zhang, Z., Song, Y., Qi, H.: Age progression/regression by conditional adversarial autoencoder. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 4352–4360 (2017)
Zhu, J.Y., Park, T., Isola, P., Efros, A.A.: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: IEEE International Conference on Computer Vision (2017)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Copyright information
© 2020 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this chapter
Cite this chapter
Li, Y., Huang, H., He, R., Tan, T. (2020). Topology-Invariant Synthesis. In: Heterogeneous Facial Analysis and Synthesis. SpringerBriefs in Computer Science. Springer, Singapore. https://doi.org/10.1007/978-981-13-9148-4_3
Download citation
DOI: https://doi.org/10.1007/978-981-13-9148-4_3
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-13-9147-7
Online ISBN: 978-981-13-9148-4
eBook Packages: Computer ScienceComputer Science (R0)