Skip to main content
SpringerLink
Log in

Sign-Correlation Partition Based on Global Supervised Descent Method for Face Alignment

  • Conference paper
  • First Online:
Computer Vision – ACCV 2016 (ACCV 2016)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 10113))

Included in the following conference series:

Abstract

Face alignment is an essential task for facial performance capture and expression analysis. As a complex nonlinear problem in computer vision, face alignment across poses is still not studied well. Although the state-of-the-art Supervised Descent Method (SDM) has shown good performance, it learns conflict descent direction in the whole complex space due to various poses and expressions. Global SDM has been presented to deal with this case by domain partition in feature and shape PCA spaces for face tracking and pose estimation. However, it is not suitable for the face alignment problem due to unknown ground truth shapes. In this paper we propose a sign-correlation subspace method for the domain partition of global SDM. In our method only one reduced low dimensional subspace is enough for domain partition, thus adjusting the global SDM efficiently for face alignment. Unlike previous methods, we analyze the sign correlation between features and shapes, and project both of them into a mutual sign-correlation subspace. Each pair of projected shape and feature keep sign consistent in each dimension of the subspace, so that each hyperoctant holds the condition that one general descent exists. Then a set of general descent directions are learned from the samples in different hyperoctants. Our sign-correlation partition method is validated in the public face datasets, which includes a range of poses. It indicates that our methods can reveal their latent relationships to poses. The comparison with state-of-the-art methods for face alignment demonstrates that our method outperforms them especially in uncontrolled conditions with various poses, while keeping comparable speed.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Cootes, T.F., Taylor, C.J., Cooper, D.H., Graham, J.: Active shape models—their training and application. Comput. Vis. Image Underst. 61, 38–59 (1995)

    Article  Google Scholar 

  2. Cootes, T.F., Edwards, G.J., Taylor, C.J.: Active appearance models. IEEE Trans. Pattern Anal. Mach. Intell. 23, 681–685 (2001)

    Article  Google Scholar 

  3. Romdhani, S.: A multi-view nonlinear active shape model using kernel PCA. In: British Machine Vision Conference, pp. 483–492 (1999)

    Google Scholar 

  4. Cristinacce, D., Cootes, T.F.: Feature detection and tracking with constrained local models. BMVC 41, 929–938 (2006)

    MATH  Google Scholar 

  5. Gonzalezmora, J., Torre, F.D.L., Murthi, R., Guil, N., Zapata, E.L.: Bilinear active appearance models. In: IEEE International Conference on Computer Vision, pp. 1–8 (2007)

    Google Scholar 

  6. Lee, H.S., Kim, D.: Tensor-based AAM with continuous variation estimation: application to variation-robust face recognition. IEEE Trans. Pattern Anal. Mach. Intell. 31, 1102–1116 (2009)

    Article  Google Scholar 

  7. Saragih, J.M., Lucey, S., Cohn, J.F.: Deformable model fitting by regularized landmark mean-shift. Int. J. Comput. Vision 91, 200–215 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  8. Valstar, M., Martinez, B., Binefa, X., Pantic, M.: Facial point detection using boosted regression and graph models, pp. 2729–2736 (2010)

    Google Scholar 

  9. Dollar, P., Welinder, P., Perona, P.: Cascaded pose regression, vol. 238, pp. 1078–1085. IEEE (2010)

    Google Scholar 

  10. Cao, X., Wei, Y., Wen, F., Sun, J.: Face alignment by explicit shape regression US Patent Application number 13/728,584 (2012)

    Google Scholar 

  11. Xiong, X., De, la Torre, F.: Supervised descent method and its applications to face alignment. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 532–539 (2013)

    Google Scholar 

  12. Burgosartizzu, X.P., Perona, P., Dollar, P.: Robust face landmark estimation under occlusion. In: IEEE International Conference on Computer Vision, pp. 1513–1520 (2013)

    Google Scholar 

  13. Xing, J., Niu, Z., Huang, J., Hu, W., Yan, S.: Towards multi-view and partially-occluded face alignment. In: Computer Vision and Pattern Recognition, pp. 1829–1836 (2014)

    Google Scholar 

  14. Yang, H., He, X., Jia, X., Patras, I.: Robust face alignment under occlusion via regional predictive power estimation. IEEE Trans. Image Process. 24, 2393–2403 (2015)

    Article  MathSciNet  Google Scholar 

  15. Yan, J., Lei, Z., Yi, D., Li, S.Z.: Learn to combine multiple hypotheses for accurate face alignment. In: IEEE International Conference on Computer Vision Workshops, pp. 392–396 (2013)

    Google Scholar 

  16. Zhang, J., Shan, S., Kan, M., Chen, X.: Coarse-to-fine auto-encoder networks (CFAN) for real-time face alignment. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8690, pp. 1–16. Springer, Heidelberg (2014). doi:10.1007/978-3-319-10605-2_1

    Google Scholar 

  17. Fan, X., Wang, H., Luo, Z., Li, Y., Hu, W., Luo, D.: Fiducial facial point extraction using a novel projective invariant. IEEE Trans. Image Process. 24, 1164–1177 (2015)

    Article  MathSciNet  Google Scholar 

  18. Liu, L., Hu, J., Zhang, S., Deng, W.: Extended supervised descent method for robust face alignment. In: Jawahar, C.V., Shan, S. (eds.) ACCV 2014. LNCS, vol. 9010, pp. 71–84. Springer, Cham (2015). doi:10.1007/978-3-319-16634-6_6

    Google Scholar 

  19. Martinez, B., Valstar, M.F.: L 2,1-based regression and prediction accumulation across views for robust facial landmark detection. Image Vis. Comput. 45, 371–382 (2015)

    Google Scholar 

  20. Lee, D., Park, H., Yoo, C.D.: Face alignment using cascade Gaussian process regression trees. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 4204–4212 (2015)

    Google Scholar 

  21. Martinez, B., Pantic, M.: Facial landmarking for in-the-wild images with local inference based on global appearance. Image Vis. Comput. 36, 40–50 (2015)

    Article  Google Scholar 

  22. Lindner, C., Bromiley, P.A., Ionita, M.C., Cootes, T.F.: Robust and accurate shape model matching using random forest regression-voting. IEEE Trans. Pattern Anal. Mach. Intell. 37, 1862–1874 (2015)

    Article  Google Scholar 

  23. Yang, H., Patras, I.: Fine-tuning regression forests votes for object alignment in the wild. IEEE Trans. Image Process. 24, 619–631 (2014). A Publication of the IEEE Signal Processing Society

    Article  MathSciNet  Google Scholar 

  24. Sun, Y., Wang, X., Tang, X.: Deep convolutional network cascade for facial point detection. In: Conference on Computer Vision and Pattern Recognition, pp. 3476–3483 (2013)

    Google Scholar 

  25. Zhou, E., Fan, H., Cao, Z., Jiang, Y., Yin, Q.: Extensive facial landmark localization with coarse-to-fine convolutional network cascade. In: IEEE International Conference on Computer Vision Workshops, pp. 386–391 (2013)

    Google Scholar 

  26. Tzimiropoulos, G.: Project-out cascaded regression with an application to face alignment. In: IEEE Conference on Computer Vision and Pattern Recognition (2015)

    Google Scholar 

  27. Xiong, X., De la Torre, F.: Global supervised descent method. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2664–2673 (2015)

    Google Scholar 

  28. Feng, Z.H., Huber, P., Kittler, J., Christmas, W., Wu, X.J.: Random cascaded-regression copse for robust facial landmark detection. IEEE Sig. Process. Lett. 22, 76–80 (2015)

    Article  Google Scholar 

  29. Yang, H., Jia, X., Patras, I., Chan, K.P.: Random subspace supervised descent method for regression problems in computer vision. IEEE Sig. Process. Lett. 22, 1816–1820 (2015)

    Article  Google Scholar 

  30. Zhu, S., Li, C., Loy, C.C., Tang, X.: Face alignment by coarse-to-fine shape searching. In: CVPR, pp. 4998–5006 (2015)

    Google Scholar 

  31. Feng, Z.H., Hu, G., Kittler, J., Christmas, W., Wu, X.J.: Cascaded collaborative regression for robust facial landmark detection trained using a mixture of synthetic and real images with dynamic weighting. IEEE Trans. Image Process. 24, 3425–3440 (2015)

    Article  MathSciNet  Google Scholar 

  32. Kazemi, V., Sullivan, J.: One millisecond face alignment with an ensemble of regression trees. In: Computer Vision and Pattern Recognition, pp. 1867–1874 (2014)

    Google Scholar 

  33. Ren, S., Cao, X., Wei, Y., Sun, J.: Face alignment at 3000 fps via regressing local binary features. IEEE Trans. Image Process. 25, 1685–1692 (2014)

    Google Scholar 

  34. Zhang, Z., Zhang, W., Ding, H., Liu, J., Tang, X.: Hierarchical facial landmark localization via cascaded random binary patterns. Pattern Recogn. 48, 1277–1288 (2014)

    Article  Google Scholar 

  35. Zhang, Z., Luo, P., Loy, C.C., Tang, X.: Facial landmark detection by deep multi-task learning. In: European Conference on Computer Vision, pp. 94–108 (2014)

    Google Scholar 

  36. Sagonas, C., Tzimiropoulos, G., Zafeiriou, S., Pantic, M.: A semi-automatic methodology for facial landmark annotation. In: IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 896–903 (2013)

    Google Scholar 

  37. Kostinger, M., Wohlhart, P., Roth, P.M., Bischof, H.: Annotated facial landmarks in the wild: a large-scale, real-world database for facial landmark localization. In: IEEE International Conference on Computer Vision Workshops, ICCV 2011 Workshops, Barcelona, Spain, pp. 2144–2151, 6–13 November 2011

    Google Scholar 

  38. Huang, G.B., Mattar, M., Berg, T., Learned-Miller, E.: Labeled faces in the wild: a database for studying face recognition in unconstrained environments (2008)

    Google Scholar 

  39. Belhumeur, P.N., Jacobs, D.W., Kriegman, D.J., Kumar, N.: Localizing parts of faces using a consensus of exemplars. IEEE Trans. Pattern Anal. Mach. Intell. 35, 2930–2940 (2013)

    Article  Google Scholar 

  40. Le, V., Brandt, J., Lin, Z., Bourdev, L., Huang, T.S.: Interactive facial feature localization. In: European Conference on Computer Vision, pp. 679–692 (2012)

    Google Scholar 

  41. Ramanan, D.: Face detection, pose estimation, and landmark localization in the wild. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 31–37 (2015)

    Google Scholar 

Download references

Acknowledgement

This work was supported by Harbin Institute of Technology Scholarship Fund 2016 and National Centre for Computer Animation, Bournemouth University.

Author information

Authors and Affiliations

  1. Harbin Institute of Technology, Harbin, China

    Yongqiang Zhang & Daming Shi

  2. Bournemouth University, Poole, UK

    Shuang Liu, Xiaosong Yang & Jian Jun Zhang

Authors
  1. Yongqiang Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shuang Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiaosong Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Daming Shi
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jian Jun Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yongqiang Zhang .

Editor information

Editors and Affiliations

  1. National Tsing Hua University, Hsinchu, Taiwan

    Shang-Hong Lai

  2. Graz University of Technology, Graz, Austria

    Vincent Lepetit

  3. Drexel University, Philadelphia, Pennsylvania, USA

    Ko Nishino

  4. The University of Tokyo , Tokyo, Japan

    Yoichi Sato

1 Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (zip 18 KB)

Supplementary material 2 (zip 2296 KB)

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper

Zhang, Y., Liu, S., Yang, X., Shi, D., Zhang, J.J. (2017). Sign-Correlation Partition Based on Global Supervised Descent Method for Face Alignment. In: Lai, SH., Lepetit, V., Nishino, K., Sato, Y. (eds) Computer Vision – ACCV 2016. ACCV 2016. Lecture Notes in Computer Science(), vol 10113. Springer, Cham. https://doi.org/10.1007/978-3-319-54187-7_19

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-54187-7_19

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-54186-0

  • Online ISBN: 978-3-319-54187-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation