Advertisement

The Visual Computer

, Volume 34, Issue 9, pp 1177–1187 | Cite as

3D cartoon face rigging from sparse examples

  • Jingyong ZhouEmail author
  • Hsiang-Tao Wu
  • Zicheng Liu
  • Xin Tong
  • Baining Guo
Original Article

Abstract

We present a data-driven method for automatically constructing cartoonized 3D blendshapes of a subject’s face. Given a pre-defined blendshape template of the real facial expressions and corresponding cartoonized blendshape template created by an artist, we represent the blendshapes of an identity in the real and cartoon face spaces with the deformations of the blendshape template in each space and learn a mapping between the deformations in the two spaces. To this end, our method decomposes the deformations in each space into two parts: an identity-independent part that is represented with the deformation gradient of the blendshape template, and an identity-dependent part that is modeled by a low-rank linear model. We regress the linear model for the real expressions from a 3D facial expression dataset. An algorithm is then introduced to regress the mapping between the linear models in the two spaces from a small set of real expressions and their cartoonized counterparts. At run time, given the blendshapes of a subject’s real face and her 3D cartoon neutral face, our method automatically constructs the cartoonized blendshapes of the subject with the help of the cartoonized blendshape template and the learned mapping. Our method is user-independent and only requires a small set of 3D cartoonized expressions modeled by the artist for cartoon face rigging. We evaluate our method by creating cartoonized 3D facial animations for variant identities in two different artistic styles. The rigging results demonstrate that our method successfully preserves both artistic styles and personalized expressions of different identities.

Keywords

Cartoon face animation Data-driven method Deformation gradient Blendshape model 

Notes

Acknowledgements

We thank the anonymous reviewers for their constructive comments and suggestions. We also thank the graphics and parallel processing laboratory of Zhejiang University for sharing the FaceWarehouse dataset for our research. All cartoon exemplars used in our system are created by Xing Zhao and Shuitian Yan.

Compliance with ethical standards

Conflict of interest

The authors declare that they have no conflict of interests.

Supplementary material

Supplementary material 1 (mp4 39774 KB)

References

  1. 1.
    Akleman, E.: Making caricatures with morphing. In: ACM SIGGRAPH 97 Visual Proceedings: The Art and Interdisciplinary Programs of SIGGRAPH’97, p. 145. ACM (1997)Google Scholar
  2. 2.
    Akleman, E., Reisch, J.: Modeling expressive 3D caricatures. In: ACM SIGGRAPH 2004 Sketches, p. 61. ACM (2004)Google Scholar
  3. 3.
    Alexander, O., Rogers, M., Lambeth, W., Chiang, M., Debevec, P.: The digital emily project: photoreal facial modeling and animation. In: Acm Siggraph 2009 Courses, p. 12. ACM (2009)Google Scholar
  4. 4.
    Blanz, V., Vetter, T.: A morphable model for the synthesis of 3D faces. In: Proceedings of the 26th Annual Conference on Computer Graphics and Interactive Techniques, pp. 187–194. ACM Press/Addison-Wesley Publishing Co. (1999)Google Scholar
  5. 5.
    Bouaziz, S., Wang, Y., Pauly, M.: Online modeling for realtime facial animation. ACM Trans. Graph. (TOG) 32(4), 40 (2013)CrossRefzbMATHGoogle Scholar
  6. 6.
    Cao, C., Bradley, D., Zhou, K., Beeler, T.: Real-time high-fidelity facial performance capture. ACM Trans. Graph. (ToG) 34(4), 46 (2015)CrossRefGoogle Scholar
  7. 7.
    Cao, C., Hou, Q., Zhou, K.: Displaced dynamic expression regression for real-time facial tracking and animation. ACM Trans. Graph. (TOG) 33(4), 43 (2014)zbMATHGoogle Scholar
  8. 8.
    Cao, C., Weng, Y., Lin, S., Zhou, K.: 3D shape regression for real-time facial animation. ACM Trans. Graph. (TOG) 32(4), 41 (2013)CrossRefzbMATHGoogle Scholar
  9. 9.
    Cao, C., Weng, Y., Zhou, S., Tong, Y., Zhou, K.: Facewarehouse: a 3D facial expression database for visual computing. IEEE Trans. Vis. Comput. Graph. 20(3), 413–425 (2014)CrossRefGoogle Scholar
  10. 10.
    Casas, D., Feng, A., Alexander, O., Fyffe, G., Debevec, P., Ichikari, R., Li, H., Olszewski, K., Suma, E., Shapiro, A.: Rapid photorealistic blendshape modeling from RGB-D sensors. In: Proceedings of the 29th International Conference on Computer Animation and Social Agents, pp. 121–129 (2016)Google Scholar
  11. 11.
    Cosker, D., Krumhuber, E., Hilton, A.: A FACS valid 3D dynamic action unit database with applications to 3D dynamic morphable facial modeling. In: IEEE International Conference on Computer Vision (ICCV), 2011, pp. 2296–2303. IEEE (2011)Google Scholar
  12. 12.
    Fujiwara, T., Koshimizu, H., Fujimura, K., Kihara, H., Noguchi, Y., Ishikawa, N.: 3D modeling system of human face and full 3D facial caricaturing. In: Proceedings of Third International Conference on 3-D Digital Imaging and Modeling, 2001, pp. 385–392. IEEE (2001)Google Scholar
  13. 13.
    Fujiwara, T., Nishihara, T., Tominaga, M., Kato, K., Murakami, K., Koshirnizu, H.: On the detection of feature points of 3D facial image and its application to 3D facial caricature. In: Proceedings of Second International Conference on 3-D Digital Imaging and Modeling, 1999, pp. 490–496. IEEE (1999)Google Scholar
  14. 14.
    Garrido, P., Zollhöfer, M., Casas, D., Valgaerts, L., Varanasi, K., Pérez, P., Theobalt, C.: Reconstruction of personalized 3D face rigs from monocular video. ACM Trans. Graph. 35(3), 28:1–28:15 (2016)CrossRefGoogle Scholar
  15. 15.
    Ichim, A.E., Bouaziz, S., Pauly, M.: Dynamic 3D avatar creation from hand-held video input. ACM Trans. Graph. (SIGGRAPH) 34(4), 45:1–45:14 (2015)CrossRefGoogle Scholar
  16. 16.
    Lewis, J.P., Anjyo, K., Rhee, T., Zhang, M., Pighin, F., Deng, Z.: Practice and theory of blendshape facial models. In: EuroGraphics (2014)Google Scholar
  17. 17.
    Li, H., Weise, T., Pauly, M.: Example-based facial rigging. ACM Trans. Graph. (ToG) 29, 32 (2010)Google Scholar
  18. 18.
    Li, P., Chen, Y., Liu, J., Fu, G.: 3D caricature generation by manifold learning. In: IEEE International Conference on Multimedia and Expo, 2008, pp. 941–944. IEEE (2008)Google Scholar
  19. 19.
    Li, T., Bolkart, T., Black, M.J., Li, H., Romero, J.: Learning a model of facial shape and expression from 4D scans. ACM Trans. Graph. 36(6), 194 (2017).  https://doi.org/10.1145/3130800.3130813. (Proc. SIGGRAPH Asia)Google Scholar
  20. 20.
    Liu, J., Chen, Y., Miao, C., Xie, J., Ling, C.X., Gao, X., Gao, W.: Semi-supervised learning in reconstructed manifold space for 3D caricature generation. In: Computer Graphics Forum, vol. 28, pp. 2104–2116. Wiley Online Library (2009)Google Scholar
  21. 21.
    Sadimon, S.B., Sunar, M.S., Mohamad, D., Haron, H.: Computer generated caricature: a survey. In: International Conference on Cyberworlds (CW), 2010, pp. 383–390. IEEE (2010)Google Scholar
  22. 22.
    Sumner, R.W., Popović, J.: Deformation transfer for triangle meshes. ACM Trans. Graph. (TOG) 23, 399–405 (2004)CrossRefGoogle Scholar
  23. 23.
    Vlasic, D., Brand, M., Pfister, H., Popović, J.: Face transfer with multilinear models. ACM Trans. Graph. (TOG) 24, 426–433 (2005)CrossRefGoogle Scholar
  24. 24.
    Waltz, R.A., Morales, J.L., Nocedal, J., Orban, D.: An interior algorithm for nonlinear optimization that combines line search and trust region steps. Math. Program. 107(3), 391–408 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  25. 25.
    Weise, T., Bouaziz, S., Li, H., Pauly, M.: Realtime performance-based facial animation. ACM Trans. Graph. (TOG) 30, 77 (2011)CrossRefGoogle Scholar
  26. 26.
    Xie, J., Chen, Y., Liu, J., Miao, C., Gao, X.: Interactive 3D caricature generation based on double sampling. In: Proceedings of the 17th ACM International Conference on Multimedia, pp. 745–748. ACM (2009)Google Scholar
  27. 27.
    Zhou, J., Tong, X., Liu, Z., Guo, B.: 3D cartoon face generation by local deformation mapping. Vis. Comput. 32(6–8), 717–727 (2016)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Tsinghua University, Microsoft Research AsiaBeijingChina
  2. 2.Microsoft Research AsiaBeijingChina
  3. 3.Microsoft ResearchRedmondUSA

Personalised recommendations