Abstract
Facial Action Transfer (FAT) has recently attracted much attention in computer vision due to its diverse applications in the movie industry, computer games, and privacy protection. The goal of FAT is to “clone” the facial actions from the videos of one person (source) to another person (target). In this paper, we will assume that we have a video of the source person but only one frontal image of the target person. Most successful methods for FAT require a training set with annotated correspondence between expressions of different subjects, sometimes including many images of the target subject. However, labeling expressions is time consuming and error prone (i.e., it is difficult to capture the same intensity of the expression across people). Moreover, in many applications it is not realistic to have many labeled images of the target. This paper proposes a method to learn a personalized facial model, that can produce photo-realistic person-specific facial actions (e.g., synthesize wrinkles for smiling), from only a neutral image of the target person. More importantly, our learning method does not need an explicit correspondence of expressions across subjects. Experiments on the Cohn-Kanade and the RU-FACS databases show the effectiveness of our approach to generate video-realistic images of the target person driven by spontaneous facial actions of the source. Moreover, we illustrate applications of FAT to face de-identification.
Chapter PDF
Similar content being viewed by others
References
Gross, R., Sweeney, L., De la Torre, F., Baker, S.: Model-based face de-identification. In: CVPR Workshop on Privacy Research in Vision (2006)
Frome, A., Cheung, G., Abdulkader, A., Zennaro, M.: B. Wu, A.B., Adam, H., Neven, H., Vincent, L.: Largescale privacy protection in google street view. In: ICCV (2009)
Senior, A. (ed.): Protecting Privacy in Video Surveillance. Springer (2009)
Saragih, J., Lucey, S., Cohn, J.: Real-time avatar animation from a single image. In: AFGR (2011)
Huang, D., De la Torre, F.: Bilinear Kernel Reduced Rank Regression for Facial Expression Synthesis. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part II. LNCS, vol. 6312, pp. 364–377. Springer, Heidelberg (2010)
Zhang, Q., Liu, Z., Guo, B., Shum, H.: Geometry-driven photorealistic facial expression synthesis. In: ACM SIGGRAPH/Eurographics Symposium on Computer Animation, pp. 177–186 (2003)
Liu, Z., Shan, Y., Zhang, Z.: Expressive expression mapping with ratio images. In: Ann. Conf. on Computer Graphics and Interactive Techniques (2001)
Noh, J., Neumann, U.: Expression cloning. SIGGRAPH, 277–288 (2001)
Chai, J., Xiao, J., Hodgins, J.: Vision-based control of 3d facial animation. In: Eurographics (2003)
Sumner, R., Popovic, J.: Deformation transfer for triangle meshes. SIGGRAPH 23, 399–405 (2004)
Anguelov, D., Srinivasan, P., Koller, D., Thrun, S., Rodgers, J., Davis, J.: Scape: shape completion and animation of people. SIGGRAPH 24, 408–416 (2005)
Vasilescu, M.A.O., Terzopoulos, D.: Multilinear Analysis of Image Ensembles: TensorFaces. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002, Part I. LNCS, vol. 2350, pp. 447–460. Springer, Heidelberg (2002)
Tenenbaum, J., Freeman, W.: Separating style and content with bilinear models. Neural Computation 12, 1247–1283 (2000)
Wang, H., Ahuja, N.: Facial expression decomposition. In: ICCV (2003)
Chung, K.: Gross Anatomy (Board Review). Lippincott Williams & Wilkins, Hagerstown (2005)
De La Hunty, M., Asthana, A., Goecke, R.: Linear facial expression transfer with active appearance models. In: ICPR (2010)
Abboud, B., Davoine, F.: Appearance factorization for facial expression analysis. In: BMVC (2004)
Vlasic, D., Brand, M., Pfister, H., Popovic, J.: Face transfer with multiliner models. ACM Trans. Graphics 24, 426–433 (2005)
Macedo, I., Brazil, E., Velho, L.: Expression transfer between photographs through multilinear aam’s. In: SIBGRAPI, pp. 239–246 (2006)
Elgammal, A., Lee, C.: Separating style and content on a nonlinear manifold. In: CVPR, pp. 478–485 (2004)
Wang, Y., Huang, X., Lee, C., Zhang, S., Li, Z., Samaras, D., Metaxas, D., Elgammal, A., Huang, P.: High resolution acquisition, learning and transfer of dynamic 3D facial expressions. Computer Graphic Forum 23, 677–686 (2004)
Kanade, T., Cohn, J.F., Tian, Y.: Comprehensive database for facial expression analysis. In: AFGR (2000)
Kim, M., Zhang, Z., De la Torre, F., Zhang, W.: Subspace regression: Predicting a subspace from one sample. In: Asian Conference on Computer Vision, ACCV (2010)
Bartlett, M., Littlewort, G., Frank, M., Lainscsek, C., Fasel, I., Movellan, J.: Automatic recognition of facial actions in spontaneous expressions. Journal of Multimedia 1, 22–35 (2006)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Huang, D., De La Torre, F. (2012). Facial Action Transfer with Personalized Bilinear Regression. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds) Computer Vision – ECCV 2012. ECCV 2012. Lecture Notes in Computer Science, vol 7573. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-33709-3_11
Download citation
DOI: https://doi.org/10.1007/978-3-642-33709-3_11
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-33708-6
Online ISBN: 978-3-642-33709-3
eBook Packages: Computer ScienceComputer Science (R0)