Advertisement

Introduction

  • Yi LiEmail author
  • Huaibo Huang
  • Ran He
  • Tieniu Tan
Chapter
  • 26 Downloads
Part of the SpringerBriefs in Computer Science book series (BRIEFSCOMPUTER)

Abstract

This chapter gives an overview of the heterogeneous problem in facial analysis as well as its synthesis solution, followed by a brief outline of the rest chapters. We start from the background and challenges in research of heterogeneous facial analysis. Then the heterogeneous facial synthesis is emphasized as one of the promising and effective solutions. The instantiated tasks to be elaborated in the following chapters are finally introduced in a compact manner.

References

  1. 1.
    Bulat, A., Yang, J., Tzimiropoulos, G.: To learn image super-resolution, use a gan to learn how to do image degradation first. In: European Conference on Computer Vision, pp. 185–200 (2018)Google Scholar
  2. 2.
    Cao, J., Hu, Y., Zhang, H., He, R., Sun, Z.: Learning a high fidelity pose invariant model for high-resolution face frontalization. In: Advances in Neural Information Processing Systems, pp. 2867–2877 (2018)Google Scholar
  3. 3.
    Chang, H., Lu, J., Yu, F., Finkelstein, A.: Pairedcyclegan: asymmetric style transfer for applying and removing makeup. In: the IEEE Conference on Computer Vision and Pattern Recognition (2018)Google Scholar
  4. 4.
    Chen, L.: Topological structure in visual perception. Science 218(4573), 699–700 (1982)CrossRefGoogle Scholar
  5. 5.
    Chen, Y., Tai, Y., Liu, X., Shen, C., Yang, J.: FSRNet: End-to-end learning face super-resolution with facial priors. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2492–2501 (2018)Google Scholar
  6. 6.
    Chen, Y.C., Shen, X., Jia, J.: Makeup-go: Blind reversion of portrait edit. In: The IEEE International Conference on Computer Vision, vol. 2 (2017)Google Scholar
  7. 7.
    Dahl, R., Norouzi, M., Shlens, J.: Pixel recursive super resolution. In: IEEE International Conference on Computer Vision, pp. 5439–5448 (2017)Google Scholar
  8. 8.
    Deng, J., Cheng, S., Xue, N., Zhou, Y., Zafeiriou, S.: UV-GAN: adversarial facial UV map completion for pose-invariant face recognition. In: IEEE Conference on Computer Vision and Pattern Recognition (2018)Google Scholar
  9. 9.
    Ganin, Y., Kononenko, D., Sungatullina, D., Lempitsky, V.: Deepwarp: photorealistic image resynthesis for gaze manipulation. In: European Conference on Computer Vision, pp. 311–326. Springer, Berlin (2016)Google Scholar
  10. 10.
    Garrido, P., Valgaerts, L., Rehmsen, O., Thormahlen, T., Perez, P., Theobalt, C.: Automatic face reenactment. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4217–4224 (2014)Google Scholar
  11. 11.
    Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y.: Generative adversarial nets. In: Advances in Neural Information Processing Systems, pp. 2672–2680 (2014)Google Scholar
  12. 12.
    Hu, Y., Wu, X., Yu, B., He, R., Sun, Z.: Pose-guided photorealistic face rotation. In: IEEE Conference on Computer Vision and Pattern Recognition (2018)Google Scholar
  13. 13.
    Huang, H., He, R., Sun, Z., Tan, T.: Wavelet domain generative adversarial network for multi-scale face hallucination. Int. J. Comput. Vis. 127(6–7), 763–784 (2019)CrossRefGoogle Scholar
  14. 14.
    Huang, H., Li, Z., He, R., Sun, Z., Tan, T.: Introvae: Introspective variational autoencoders for photographic image synthesis. In: Advances in Neural Information Processing Systems, pp. 10236–10245 (2018)Google Scholar
  15. 15.
    Huang, R., Zhang, S., Li, T., He, R.: Beyond face rotation: global and local perception GAN for photorealistic and identity preserving frontal view synthesis. In: IEEE International Conference on Computer Vision (2017)Google Scholar
  16. 16.
    Juefei-Xu, F., Pal, D.K., Savvides, M.: Nir-vis heterogeneous face recognition via cross-spectral joint dictionary learning and reconstruction. In: IEEE conference on Computer Vision and Pattern Recognition Workshops (2015)Google Scholar
  17. 17.
    Kemelmacher-Shlizerman, I., Suwajanakorn, S., Seitz, S.M.: Illumination-aware age progression. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3334–3341 (2014)Google Scholar
  18. 18.
    Lei, Z., Bai, Q., He, R., Li, S.Z.: Face shape recovery from a single image using cca mapping between tensor spaces. In: IEEE Conference on Computer Vision and Pattern Recognition (2008)Google Scholar
  19. 19.
    Lezama, J., Qiu, Q., Sapiro, G.: Not afraid of the dark: Nir-vis face recognition via cross-spectral hallucination and low-rank embedding. In: IEEE Conference on Computer Vision and Pattern Recognition (2016)Google Scholar
  20. 20.
    Li, P., Hu, Y., He, R., Sun, Z.: Global and local consistent wavelet-domain age synthesis. IEEE Trans. Inf. Forensics Secur. 14(11), 2943–2957 (2019).  https://doi.org/10.1109/TIFS.2019.2907973CrossRefGoogle Scholar
  21. 21.
    Li, T., Qian, R., Dong, C., Liu, S., Yan, Q., Zhu, W., Lin, L.: Beautygan: instance-level facial makeup transfer with deep generative adversarial network. In: 2018 ACM Multimedia Conference on Multimedia Conference, pp. 645–653. ACM (2018)Google Scholar
  22. 22.
    Li, Y., Huang, H., Cao, J., He, R., Tan, T.: Disentangled representation learning of makeup portraits in the wild. In: International Journal of Computer Vision, pp. 1–19 (2019)Google Scholar
  23. 23.
    Li, Y., Huang, H., Yu, J., He, R., Tan, T.: Semantic-aware makeup cleanser. In: The 10th IEEE International Conference on Biometrics: Theory, Applications and Systems (2019)Google Scholar
  24. 24.
    Li, Y., Liu, S., Yang, J., Yang, M.H.: Generative face completion. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3911–3919 (2017)Google Scholar
  25. 25.
    Li, Y., Song, L., Wu, X., He, R., Tan, T.: Anti-makeup: learning a bi-level adversarial network for makeup-invariant face verification. In: The Thirty-Second AAAI Conference on Artificial Intelligence (2018)Google Scholar
  26. 26.
    Li, Y., Song, L., Wu, X., He, R., Tan, T.: Learning a bi-level adversarial network with global and local perception for makeup-invariant face verification. Pattern Recognit. 90, 99–108 (2019)CrossRefGoogle Scholar
  27. 27.
    Liu, P., Qi, X., He, P., Li, Y., Lyu, M.R., King, I.: Semantically consistent image completion with fine-grained details (2017). arXiv:1711.09345
  28. 28.
    Liu, S., Ou, X., Qian, R., Wang, W., Cao, X.: Makeup like a superstar: deep localized makeup transfer network. In: the Twenty-Fifth International Joint Conference on Artificial Intelligence, pp. 2568–2575. AAAI Press (2016)Google Scholar
  29. 29.
    Lu, Z., Hu, T., Song, L., Zhang, Z., He, R.: Conditional expression synthesis with face parsing transformation. In: 2018 ACM Multimedia Conference on Multimedia Conference, pp. 1083–1091. ACM (2018)Google Scholar
  30. 30.
    van den Oord, A., Kalchbrenner, N., Espeholt, L., kavukcuoglu, k., Vinyals, O., Graves, A.: Conditional image generation with pixelcnn decoders. In: Advances in Neural Information Processing Systems 29, pp. 4790–4798 (2016)Google Scholar
  31. 31.
    Shu, Z., Yumer, E., Hadap, S., Sunkavalli, K., Shechtman, E., Samaras, D.: Neural face editing with intrinsic image disentangling. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5541–5550 (2017)Google Scholar
  32. 32.
    Song, L., Cao, J., Song, L., Hu, Y., He, R.: Geometry-aware face completion and editing. In: AAAI Conference on Artificial Intelligence (2019)Google Scholar
  33. 33.
    Song, L., Lu, Z., He, R., Sun, Z., Tan, T.: Geometry guided adversarial facial expression synthesis. In: 2018 ACM Multimedia Conference on Multimedia Conference, pp. 627–635. ACM (2018)Google Scholar
  34. 34.
    Song, L., Zhang, M., Wu, X., He, R.: Adversarial discriminative heterogeneous face recognition. In: Thirty-Second AAAI Conference on Artificial Intelligence (2018)Google Scholar
  35. 35.
    Susskind, J.M., Hinton, G.E., Movellan, J.R., Anderson, A.K.: Generating facial expressions with deep belief nets. In: Affective Computing. IntechOpen (2008)Google Scholar
  36. 36.
    Tian, Y., Peng, X., Zhao, L., Zhang, S., Metaxas, D.N.: CR-GAN: Learning complete representations for multi-view generation. In: IJCAI (2018)Google Scholar
  37. 37.
    Tran, L., Yin, X., Liu, X.: Disentangled representation learning GAN for pose-invariant face recognition. In: IEEE Conference on Computer Vision and Pattern Recognition (2017)Google Scholar
  38. 38.
    Tsai, M.H., Liao, Y.K., Lin, I.C.: Human face aging with guided prediction and detail synthesis. Multimed. Tools Appl. 72(1), 801–824 (2014)CrossRefGoogle Scholar
  39. 39.
    Wang, R., Yang, J., Yi, D., Li, S.Z.: An analysis-by-synthesis method for heterogeneous face biometrics. In: International Conference on Biometrics (2009)Google Scholar
  40. 40.
    Wang, W., Cui, Z., Yan, Y., Feng, J., Yan, S., Shu, X., Sebe, N.: Recurrent face aging. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2378–2386 (2016)Google Scholar
  41. 41.
    Wang, X., Tang, X.: Hallucinating face by eigentransformation. IEEE Trans. Syst. Man Cybern. Part C (Appl. Rev.) 35(3), 425–434 (2005)Google Scholar
  42. 42.
    Wang, Z., Tang, X., Luo, W., Gao, S.: Face aging with identity-preserved conditional generative adversarial networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7939–7947 (2018)Google Scholar
  43. 43.
    Wu, X., He, R., Sun, Z., Tan, T.: A light cnn for deep face representation with noisy labels. IEEE Trans. Inf. Forensics Secur. 13(11), 2884–2896 (2018)CrossRefGoogle Scholar
  44. 44.
    Yang, F., Bourdev, L., Shechtman, E., Wang, J., Metaxas, D.: Facial expression editing in video using a temporally-smooth factorization. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition, pp. 861–868. IEEE (2012)Google Scholar
  45. 45.
    Yang, H., Huang, D., Wang, Y., Jain, A.K.: Learning face age progression: a pyramid architecture of gans. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 31–39 (2018)Google Scholar
  46. 46.
    Yin, X., Yu, X., Sohn, K., Liu, X., Chandraker, M.: Towards large-pose face frontalization in the wild. In: ICCV (2017)Google Scholar
  47. 47.
    Yu, J., Cao, J., Li, Y., Jia, X., He, R.: Pose-preserving cross spectral face hallucination. In: IJCAI (2019)Google Scholar
  48. 48.
    Yu, X., Fernando, B., Hartley, R., Porikli, F.: Super-resolving very low-resolution face images with supplementary attributes. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 908–917 (2018)Google Scholar
  49. 49.
    Yu, X., Porikli, F.: Ultra-resolving face images by discriminative generative networks. In: European Conference on Computer Vision, pp. 318–333 (2016)Google Scholar
  50. 50.
    Zhang, S., He, R., Sun, Z., Tan, T.: Multi-task convnet for blind face inpainting with application to face verification. In: 2016 International Conference on Biometrics (ICB), pp. 1–8. IEEE (2016)Google Scholar
  51. 51.
    Zhang, S., He, R., Sun, Z., Tan, T.: Demeshnet: Blind face inpainting for deep meshface verification. IEEE Trans. Inf. Forensics Secur. 13(3), 637–647 (2017)CrossRefGoogle Scholar
  52. 52.
    Zhang, Z., Song, Y., Qi, H.: Age progression/regression by conditional adversarial autoencoder. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5810–5818 (2017)Google Scholar
  53. 53.
    Zhao, J., Cheng, Y., Xu, Y., Xiong, L., Li, J., Zhao, F., Jayashree, K., Pranata, S., Shen, S., Xing, J. et al.: Towards pose invariant face recognition in the wild. In: IEEE Conference on Computer Vision and Pattern Recognition (2018)Google Scholar
  54. 54.
    Zhao, J., Xiong, L., Cheng, Y., Cheng, Y., Li, J., Zhou, L., Xu, Y., Karlekar, J., Pranata, S., Shen, S. et al.: 3d-aided deep pose-invariant face recognition. In: IJCAI (2018)Google Scholar
  55. 55.
    Zhao, J., Xiong, L., Jayashree, P.K., Li, J., Zhao, F., Wang, Z., Pranata, P.S., Shen, P.S., Yan, S., Feng, J.: Dual-agent GANs for photorealistic and identity preserving profile face synthesis. In: Advances in Neural Information Processing Systems (2017)Google Scholar
  56. 56.
    Zhu, S., Liu, S., Loy, C.C., Tang, X.: Deep cascaded bi-network for face hallucination. In: European Conference on Computer Vision, pp. 614–630 (2016)Google Scholar

Copyright information

© The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2020

Authors and Affiliations

  1. 1.Institute of Automation, Chinese Academy of SciencesBeijingChina

Personalised recommendations