Sign-correlation cascaded regression for face alignment

  • Dansong Cheng
  • Yongqiang Zhang
  • Ce Liu
  • Xiaofang LiuEmail author


Face alignment plays an important role in many applications such as face recognition and face reconstruction. Current regression based approaches can ease the multi-pose face alignment problem, but they fail to deal with the multiple local minima problem directly. To improve the performance of multi-pose facial landmark localization, in this paper we propose a sign correlation supervised descent method (SC-SDM) based on a nonlinear optimization theory. SC-SDM analyses the sign correlation between features and shapes and project both of them into a mutual sign-correlation subspace. By partitioning the whole multi-pose samples into a series of pose-consistent subsets, a group of models are learned from each subset. The experiments using the public multi-pose datasets has validated the partition and proved that SC-SDM can accurately separate samples into pose-consistent subsets, which reveals their latent relationships to pose. The comparison with state-of-the-art methods demonstrates that SC-SDM outperforms them, especially in uncontrolled conditions with various poses.


Sign correlation Supervised descent method Face alignment cascaded regression 



  1. 1.
    Asthana A, Zafeiriou S, Cheng S, Pantic M (2013) Robust discrim- inative response map fitting with constrained local models. IEEE Conference on Computer Vision and Pattern Recognition:3444–3451Google Scholar
  2. 2.
    Belhumeur PN, Jacobs DW, Kriegman DJ, Kumar N (2013) Localizing parts of faces using a consensus of exemplars. IEEE Transactions on Pattern Analysis & Machine Intelligence 35(12):2930–2940CrossRefGoogle Scholar
  3. 3.
    Cao X, Wei Y, Wen F, Sun J (2012) Face alignment by explicit shape regression. US Patent App. 13/728,584Google Scholar
  4. 4.
    Cheng D, Yang J, Wang J, Shi D, Liu X (2015) Double-noise-dual- problem approach to the augmented lagrange multiplier method for robust principal component analysis. Soft Comput 21:1–10Google Scholar
  5. 5.
    Cootes TF, Edwards GJ, Taylor CJ (2001) Active appearance models. Pattern Analysis & Machine Intelligence IEEE Transactions on 23(6):681–685CrossRefGoogle Scholar
  6. 6.
    Cootes TF, Taylor CJ, Cooper DH, Graham J (1995) Active shape models & their training and application. Computer Vision & Image Understanding 61(1):38–59CrossRefGoogle Scholar
  7. 7.
    Cristinacce D, Cootes TF (2006) Feature detection and tracking with constrained local models. BMVC 41:929–938zbMATHGoogle Scholar
  8. 8.
    Dollar P, Welinder P, Perona P (2010) Cascaded pose regression. CVPR, IEEE Computer Society Conference 238(6):1078–1085Google Scholar
  9. 9.
    Dong S, Luo S (2016) Modified grey-level models for active shape model training. Conf Proc IEEE Eng Med Biol Soc 1:3791–3794Google Scholar
  10. 10.
    Gonzalezmora J, Torre FDL, Murthi R, Guil N, Zapata EL (2007) Bilinear active appearance models. IEEE International Conference on Computer Vision:1–8Google Scholar
  11. 11.
    Gu L, Kanade T (2009) Face Alignment. Springer, Boston, MAGoogle Scholar
  12. 12.
    Huang GB, Mattar M, Berg T, Learned-Miller E (2008) Labeled faces in the wild: A database for studying face recognition in unconstrained environmentsGoogle Scholar
  13. 13.
    Kazemi V, Sullivan J (2014) One millisecond face alignment with an ensemble of regression trees. Computer Vision and Pattern Recognition:1867–1874Google Scholar
  14. 14.
    Kostinger M, Wohlhart P, Roth PM, Bischof H (2011) Annotated facial landmarks in the wild: A large-scale, real-world database for facial landmark localization, vol 6-13. IEEE International Conference on Computer Vision Workshops, ICCV 2011 Workshops, Barcelona, pp 2144–2151Google Scholar
  15. 15.
    Le V, Brandt J, Lin Z, Bourdev L, Huang TS (2012) Interactive facial feature localization. European Conference on Computer Vision:679–692Google Scholar
  16. 16.
    Lee HS, Kim D (2009) Tensor-based AAM with continuous variation estimation: application to variation-robust face recognition. IEEE Transactions on Pattern Analysis & Machine Intelligence 31(6):1102–1116CrossRefGoogle Scholar
  17. 17.
    Ramanan D (2015) Face detection, pose estimation, and landmark localization in the wild. IEEE Conference on Computer Vision and Pattern Recognition:31–37Google Scholar
  18. 18.
    Ren S, Cao X, Wei Y, Sun J (2014) Face alignment at 3000 fps via regressing local binary features. IEEE Trans Image Process:1685–1692Google Scholar
  19. 19.
    Romdhani S (1999) A multi-view nonlinear active shape model using kernel pca. British Machine Vision Conference:483–492Google Scholar
  20. 20.
    Sagonas C, Tzimiropoulos G, Zafeiriou S, Pantic M (2013) A semiautomatic methodology for facial landmark annotation. In: IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 896–903Google Scholar
  21. 21.
    Saragih JM, Lucey S, Cohn JF (2011) Deformable model fitting by regularized landmark mean-shift. Int J Comput Vis 91(2):200–215MathSciNetCrossRefGoogle Scholar
  22. 22.
    Snchez-Lozano E, Martinez B, Valstar MF (2016) Cascaded regression with sparsified feature covariance matrix for facial landmark detection. Pattern Recogn Lett 73(C):19–25CrossRefGoogle Scholar
  23. 23.
    Wilks DS (2011) Canonical correlation analysis (cca). International Geophysics 100:563–582CrossRefGoogle Scholar
  24. 24.
    Xing J, Niu Z, Huang J, Hu W, Yan S (2014) Towards multi-view and partially-occluded face alignment. Computer Vision and Pattern Recognition:1829–1836Google Scholar
  25. 25.
    Xiong X, De, la Torre F (2013) Supervised descent method and its applications to face alignment. IEEE Conference on Computer Vision & Pattern Recognition:532–539Google Scholar
  26. 26.
    Xiong X, la Torre FD (2015) Global supervised descent method. IEEE Conference on Computer Vision and Pattern Recognition:2664–2673Google Scholar
  27. 27.
    Yan J, Lei Z, Yi D, Li SZ (2013) Learn to combine multiple hypotheses for accurate face alignment. IEEE International Conference on Computer Vision Workshops:392–396Google Scholar
  28. 28.
    Yang H, He X, Jia X, Patras I (2015) Robust face alignment under occlusion via regional predictive power estimation. IEEE Trans Image Process 24(8):2393–2403MathSciNetCrossRefGoogle Scholar
  29. 29.
    Zhang Z, Luo P, Loy CC, Tang X (2014) Facial landmark detection by deep multi-task learning. European Conference on Computer Vision:94–108Google Scholar
  30. 30.
    Zhang J, Shan S, Kan M, Chen X (2014) Coarse-to-fine auto-encoder networks (cfan) for real-time face alignment. In: Computer Vision ECCV 2014. Springer, pp. 1–16Google Scholar
  31. 31.
    Zhang Y, Shi D, Gao J, Cheng D (2017) Low-rank-sparse subspace representation for robust regression. IEEE Conference on Computer Vision Pattern RecognitionGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  • Dansong Cheng
    • 1
  • Yongqiang Zhang
    • 1
  • Ce Liu
    • 1
  • Xiaofang Liu
    • 2
    Email author
  1. 1.School of Computer Science and TechnologyHarbin Institute of TechnologyHarbinChina
  2. 2.School of Electrical Engineering and AutomationHarbin Institute of TechnologyHarbinChina

Personalised recommendations