Abstract
Facial expression synthesis or editing has recently received increasing attention in the field of affective computing and facial expression modeling. However, most existing facial expression synthesis works are limited in paired training data, low resolution, identity information damaging, and so on. To address those limitations, this paper introduces a novel Action Unit (AU) level facial expression synthesis method called Local Attentive Conditional Generative Adversarial Network (LAC-GAN) based on face action units annotations. Given desired AU labels, LAC-GAN utilizes local AU regional rules to control the status of each AU and attentive mechanism to combine several of them into the whole photo-realistic facial expressions or arbitrary facial expressions. In addition, unpaired training data is utilized in our proposed method to train the manipulation module with the corresponding AU labels, which learns a mapping between a facial expression manifold. Extensive qualitative and quantitative evaluations are conducted on commonly used BP4D dataset to verify the effectiveness of our proposed AU synthesis method.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Chen, X., Xu, C., Yang, X.: Attention-GAN for object transfiguration in wild images. arXiv preprint arXiv:1803.06798 (2018)
Cmusatyalab: Openface: facial action units recognition with deep neural networks. http://cmusatyalab.github.io/openface/
Ding, H., Sricharan, K., Chellappa, R.: ExprGAN: facial expression editing with controllable expression intensity. arXiv preprint arXiv:1709.03842 (2017)
Ekman, P., Rosenberg, E.L.: What the Face Reveals: Basic and Applied Studies of Spontaneous Expression using the Facial Action Coding System (FACS). Oxford University Press, USA (1997)
Goodfellow, I.J., Pouget-Abadie, J.: Generative adversarial nets. In: Advances in Neural Information Processing Systems, pp. 2672–2680 (2014)
He, X., Yan, S., Hu, Y.: Face recognition using laplacianfaces. IEEE TPAMI 27(3), 328–340 (2005)
Huang, R., Zhang, S., Li, T., He, R.: Beyond face rotation: global and local perception GAN for photorealistic and identity preserving frontal view synthesis. In: ICCV (2017)
Isola, P., Zhu, J.Y., Zhou, T.: Image-to-image translation with conditional adversarial networks. In: CVPR 2017 (2017)
Kingma, D., Ba, J.: Adam: a method for stochastic optimization. In: ICLR (2014)
Ledig, C., Theis, L., Huszar, F.: Photo-realistic single image super- resolution using a generative adversarial network. In: CVPR 2017 (2017)
Lee, K.C., Ho, J., Yang, M.H.: Video-based face recognition using probabilistic appearance manifolds. In: CVPR, vol. 1(1) (2003)
Li, W., Abtahi, F., Zhu, Z.: EAC-net: a region-based deep enhancing and cropping approach for facial action unit detection. In: IEEE Conference on Computer Vision and Pattern Recognition Workshop, pp. 103–110 (2017)
Liu, Z., Song, G.: Conditional adversarial synthesis of 3D facial action units. arXiv preprint arXiv:1802.07421 (2018)
Mahendran, A., Vedaldi, A.: Understanding deep image representations by inverting them. In: CVPR, pp. 5188–5196 (2015)
Mirza, M., Osindero, S.: Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784 (2014)
Odena, A., Olah, C., Shlens, J.: Conditional image synthesis with auxiliary classifier GANs. In: ICML 2017 (2017)
Pathak, D., Krahenbuhl, P., Donahue, J.: Context encoders: feature learning by inpainting. In: CVPR 2016 (2016)
Pumarola, A., Agudo, A., Martinez, A.M., Sanfeliu, A., Moreno-Noguer, F.: GANimation: anatomically-aware facial animation from a single image. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11214, pp. 835–851. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01249-6_50
Radford, A., Metz, L.: Unsupervised representation learning with deep convolutional generative adversarial networks. In: ICLR 2016 (2016)
Shao, Z., Liu, Z., Cai, J., Ma, L.: Deep adaptive attention for joint facial action unit detection and face alignment. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11217, pp. 725–740. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01261-8_43
Tran, L., Yin, X., Liu, X.: Disentangled representation learning GAN for poseinvariant face recognition. In: CVPR, vol. 4(7) (2017)
Wang, M., Deng, W.: Deep face recognition: a survey. BMVC 1(6) (2015)
Wang, Z., Tang, X.: Face aging with identity-preserved conditional generative adversarial networks. In: CVPR 2018 (2018)
Yeh, R., Liu, Z., Goldman, D.B.: Semantic facial expression editing using autoencoded flow. arXiv preprint arXiv:1611.09961 (2016)
Zhang, X., Yin, L., Cohn, J.F.: Bp4d-spontaneous: a high-resolution spontaneous 3D dynamic facial expression database. Image Vis. Comput. 32(10), 692–706 (2014)
Zhang, Z., Song, Y.: Age progression/regression by conditional adversarial autoencoder. In: CVPR 2017 (2017)
Zhou, Y., Shi, B.E.: Photorealistic facial expression synthesis by the conditional difference adversarial autoencoder. In: ACII 2017 (2017)
Acknowledgements
This work is supported by the National Natural Science Foundation of China under Grants of 41806116 and 61503277. We gratefully acknowledge the support of NVIDIA Corporation with the donation of the Titan V GPU used for this research.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Liu, Z., Liu, D., Wu, Y. (2020). Region Based Adversarial Synthesis of Facial Action Units. In: Ro, Y., et al. MultiMedia Modeling. MMM 2020. Lecture Notes in Computer Science(), vol 11962. Springer, Cham. https://doi.org/10.1007/978-3-030-37734-2_42
Download citation
DOI: https://doi.org/10.1007/978-3-030-37734-2_42
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-37733-5
Online ISBN: 978-3-030-37734-2
eBook Packages: Computer ScienceComputer Science (R0)