Advertisement

A Deeply-Initialized Coarse-to-fine Ensemble of Regression Trees for Face Alignment

  • Roberto Valle
  • José M. Buenaposada
  • Antonio Valdés
  • Luis Baumela
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11218)

Abstract

In this paper we present DCFE, a real-time facial landmark regression method based on a coarse-to-fine Ensemble of Regression Trees (ERT). We use a simple Convolutional Neural Network (CNN) to generate probability maps of landmarks location. These are further refined with the ERT regressor, which is initialized by fitting a 3D face model to the landmark maps. The coarse-to-fine structure of the ERT lets us address the combinatorial explosion of parts deformation. With the 3D model we also tackle other key problems such as robust regressor initialization, self occlusions, and simultaneous frontal and profile face analysis. In the experiments DCFE achieves the best reported result in AFLW, COFW, and 300 W private and common public data sets.

Keywords

Face alignment Cascaded shape regression Convolutional neural networks Coarse-to-Fine Occlusions Real-time 

Notes

Acknowledgments

The authors thank Pedro López Maroto for his help implementing the CNN. They also gratefully acknowledge computing resources provided by the Super-computing and Visualization Center of Madrid (CeSViMa) and funding from the Spanish Ministry of Economy and Competitiveness under project TIN2016-75982-C2-2-R. José M. Buenaposada acknowledges the support of Computer Vision and Image Processing research group (CVIP) from Universidad Rey Juan Carlos.

References

  1. 1.
    Alahi, A., Ortiz, R., Vandergheynst, P.: FREAK: fast retina keypoint. In: Proceedings IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2012)Google Scholar
  2. 2.
    Bekios-Calfa, J., Buenaposada, J.M., Baumela, L.: Robust gender recognition by exploiting facial attributes dependencies. Pattern Recognit. Lett. (PRL) 36, 228–234 (2014)CrossRefGoogle Scholar
  3. 3.
    Belhumeur, P.N., Jacobs, D.W., Kriegman, D.J., Kumar, N.: Localizing parts of faces using a consensus of exemplars. In: Proceedings IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2011)Google Scholar
  4. 4.
    Blanz, V., Vetter, T.: Face recognition based on fitting a 3D morphable model. IEEE Trans. Pattern Anal. Mach. Intell. (TPAMI) (2003)Google Scholar
  5. 5.
    Bulat, A., Tzimiropoulos, G.: Binarized convolutional landmark localizers for human pose estimation and face alignment with limited resources. In: Proceedings International Conference on Computer Vision (ICCV) (2017)Google Scholar
  6. 6.
    Burgos-Artizzu, X.P., Perona, P., Dollar, P.: Robust face landmark estimation under occlusion. In: Proceedings International Conference on Computer Vision (ICCV) (2013)Google Scholar
  7. 7.
    Cao, X., Wei, Y., Wen, F., Sun, J.: Face alignment by explicit shape regression. In: Proceedings IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2012)Google Scholar
  8. 8.
    Cootes, T.F., Edwards, G.J., Taylor, C.J.: Active appearance models. In: Burkhardt, H., Neumann, B. (eds.) ECCV 1998. LNCS, vol. 1407, pp. 484–498. Springer, Heidelberg (1998).  https://doi.org/10.1007/BFb0054760CrossRefGoogle Scholar
  9. 9.
    Dantone, M., Gall, J., Fanelli, G., Gool, L.V.: Real-time facial feature detection using conditional regression forests. In: Proceedings IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2012)Google Scholar
  10. 10.
    David, P., DeMenthon, D., Duraiswami, R., Samet, H.: SoftPOSIT: simultaneous pose and correspondence determination. Int. J. Comput. Vis. (IJCV) 59(3), 259–284 (2004)CrossRefGoogle Scholar
  11. 11.
    Deng, J., Liu, Q., Yang, J., Tao, D.: CSR: multi-view, multi-scale and multi-component cascade shape regression. Image Vis. Comput. (IVC) 47, 19–26 (2016)CrossRefGoogle Scholar
  12. 12.
    Dollar, P., Welinder, P., Perona, P.: Cascaded pose regression. In: Proceedings IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2010)Google Scholar
  13. 13.
    Fan, H., Zhou, E.: Approaching human level facial landmark localization by deep learning. Image Vis. Comput. (IVC) 47, 27–35 (2016)CrossRefGoogle Scholar
  14. 14.
    Feng, Z., Kittler, J., Christmas, W.J., Huber, P., Wu, X.: Dynamic attention-controlled cascaded shape regression exploiting training data augmentation and fuzzy-set sample weighting. In: Proceedings IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017)Google Scholar
  15. 15.
    Hastie, T., Tibshirani, R., Friedman, J.H.: The Elements of Statistical Learning. Springer, New York (2009).  https://doi.org/10.1007/978-0-387-84858-7CrossRefzbMATHGoogle Scholar
  16. 16.
    Honari, S., Yosinski, J., Vincent, P., Pal, C.J.: Recombinator networks: Learning coarse-to-fine feature aggregation. In: Proceedings IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016)Google Scholar
  17. 17.
    Jourabloo, A., Ye, M., Liu, X., Ren, L.: Pose-invariant face alignment with a single CNN. In: Proceedings International Conference on Computer Vision (ICCV) (2017)Google Scholar
  18. 18.
    Kazemi, V., Sullivan, J.: One millisecond face alignment with an ensemble of regression trees. In: Proceedings IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2014)Google Scholar
  19. 19.
    Kowalski, M., Naruniec, J.: Face alignment using K-Cluster regression forests with weighted splitting. IEEE Signal Process. Lett. 23(11), 1567–1571 (2016)CrossRefGoogle Scholar
  20. 20.
    Kowalski, M., Naruniec, J., Trzcinski, T.: Deep alignment network: a convolutional neural network for robust face alignment. In: Proceedings IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) (2017)Google Scholar
  21. 21.
    Lee, D., Park, H., Yoo, C.D.: Face alignment using cascade gaussian process regression trees. In: Proceedings IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2015)Google Scholar
  22. 22.
    Liu, W., et al.: SSD: single shot multibox detector. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9905, pp. 21–37. Springer, Cham (2016).  https://doi.org/10.1007/978-3-319-46448-0_2CrossRefGoogle Scholar
  23. 23.
    Lv, J., Shao, X., Xing, J., Cheng, C., Zhou, X.: A deep regression architecture with two-stage re-initialization for high performance facial landmark detection. In: Proceedings IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017)Google Scholar
  24. 24.
    Ren, S., Cao, X., Wei, Y., Sun, J.: Face alignment at 3000 fps via regressing local binary features. In: Proceedings IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2014)Google Scholar
  25. 25.
    Soltanpour, S., Boufama, B., Wu, Q.M.J.: A survey of local feature methods for 3D face recognition. Pattern Recogn. (PR) 72, 391–406 (2017)CrossRefGoogle Scholar
  26. 26.
    Sun, Y., Wang, X., Tang, X.: Deep convolutional network cascade for facial point detection. In: Proceedings IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2013)Google Scholar
  27. 27.
    Trigeorgis, G., Snape, P., Nicolaou, M.A., Antonakos, E., Zafeiriou, S.: Mnemonic descent method: A recurrent process applied for end-to-end face alignment. In: Proceedings IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016)Google Scholar
  28. 28.
    Wu, Y., Ji, Q.: Robust facial landmark detection under significant head poses and occlusion. In: Proceedings International Conference on Computer Vision (ICCV) (2015)Google Scholar
  29. 29.
    Xiao, S., et al.: Recurrent 3D–2D dual learning for large-pose facial landmark detection. In: Proceedings International Conference on Computer Vision (ICCV) (2017)Google Scholar
  30. 30.
    Xiao, S., Feng, J., Xing, J., Lai, H., Yan, S., Kassim, A.: Robust facial landmark detection via recurrent attentive-refinement networks. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9905, pp. 57–72. Springer, Cham (2016).  https://doi.org/10.1007/978-3-319-46448-0_4CrossRefGoogle Scholar
  31. 31.
    Xiong, X., la Torre, F.D.: Supervised descent method and its applications to face alignment. In: Proceedings IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2013)Google Scholar
  32. 32.
    Xiong, X., la Torre, F.D.: Global supervised descent method. In: Proceedings IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2015)Google Scholar
  33. 33.
    Yang, J., Liu, Q., Zhang, K.: Stacked hourglass network for robust facial landmark localisation. In: Proceedings IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) (2017)Google Scholar
  34. 34.
    Yu, X., Zhou, F., Chandraker, M.: Deep deformation network for object landmark localization. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9909, pp. 52–70. Springer, Cham (2016).  https://doi.org/10.1007/978-3-319-46454-1_4CrossRefGoogle Scholar
  35. 35.
    Zafeiriou, S., Trigeorgis, G., Chrysos, G., Deng, J., Shen, J.: The menpo facial landmark localisation challenge: a step towards the solution. In: Proceedings IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) (2017)Google Scholar
  36. 36.
    Zhang, Z., Luo, P., Loy, C.C., Tang, X.: Facial landmark detection by deep multi-task learning. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8694, pp. 94–108. Springer, Cham (2014).  https://doi.org/10.1007/978-3-319-10599-4_7CrossRefGoogle Scholar
  37. 37.
    Zhu, S., Li, C., Change, C., Tang, X.: Unconstrained face alignment via cascaded compositional learning. In: Proceedings IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016)Google Scholar
  38. 38.
    Zhu, S., Li, C., Loy, C.C., Tang, X.: Face alignment by coarse-to-fine shape searching. In: Proceedings IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2015)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  1. 1.Univ. Politécnica de MadridMadridSpain
  2. 2.Univ. Rey Juan CarlosMóstolesSpain
  3. 3.Univ. Complutense de MadridMadridSpain

Personalised recommendations