Multi-pose face recognition using Cascade Alignment Network and incremental clustering


The variety of face pose has imposed significant challenges on the existing face recognition. A novel approach has been proposed for multi-pose face recognition using LSTM and CNN-based cascade alignment network (LCCAN) with incremental clustering strategy. LCCAN is used to leverage the memory function of LSTM and explore the spatial contextual information between facial landmarks. The coarse facial landmark locations have been gotten at first in LCCAN. CNNs are utilized to refine facial landmarks as mentioned above. Then, the facial landmarks are used as facial orientation descriptors. In order to fit in with the dynamic updating of diversified facial poses, dynamic adaptive incremental clustering strategy with correntropy-induced metric has been developed to construct facial pose pool. Multi-pose face recognition is implemented by building face recognition classification models on different poses. Experimental results demonstrate that the effectiveness of the proposed method is superior to the investigated state of the arts.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6


  1. 1.

    Ho, H.T., Chellappa, R.: Pose-invariant face recognition using Markov random fields. IEEE Trans. Image Process. 22(4), 1573–1584 (2013)

    MathSciNet  Article  Google Scholar 

  2. 2.

    Ding, C.X., Tao, D.C.: Pose-invariant face recognition with homography-based normalization. Pattern Recogn. 66(1), 144–152 (2017)

    Article  Google Scholar 

  3. 3.

    Su, Y., Gao, X.B., Yin, X.C.: Fast alignment for sparse representation based face recognition. Pattern Recogn. 68(1), 211–221 (2017)

    Article  Google Scholar 

  4. 4.

    Zhao, M.H., Mo, R.Y., Shi, Z.H., Zhang, F.F.: A novel method for recognition of pose invariant face with single image. J. Xi'an Univ. Technol. 33(1), 18–23 (2017)

    Google Scholar 

  5. 5.

    Zhou, E.J., Fan, H.Q., Cao, Z.M., Jiang, Y.N., Yin, Q.: Extensive facial landmark localization with coarse-to-fine convolutional network cascade. In: Proceedings of the IEEE International Conference on Computer Vision Workshops, 2013, pp. 386–391

  6. 6.

    Bian, P., Xie, Z., Jin, Y.: Multi-task feature learning-based improved supervised descent method for facial landmark detection. SIViP 12(1), 17–12 (2018)

    Article  Google Scholar 

  7. 7.

    Sagonas, C., Tzimiropoulos, G., Zafeiriou, S., Pantic, M.: 300 faces in-the-wild challenge: the first facial landmark localization challenge. In: Proceedings of the IEEE International Conference on Computer Vision Workshops, 2013: pp. 397–403

  8. 8.

    Sun, Y., Wang, X.G., Tang, X.: Deep learning face representation from predicting 10,000 classes. In: Proceedings of the IEEE Computer Vision and Pattern Recognition, 2014, pp. 1891–1898

  9. 9.

    Taigman, Y., Yang, M., Ranzato, M.A., Wolf, L.: Deepface: closing the gap to human-level performance in face verification. In: Proceedings of the IEEE Computer Vision and Pattern Recognition, 2014, pp. 1701–1708

  10. 10.

    Schroff, F., Kalenichenko, D., Philbin, J.: FaceNet: a unified embedding for face recognition and clustering. In: Proceedings of the IEEE Computer Vision and Pattern Recognition, 2015, pp. 815–823.

  11. 11.

    Learned-Miller, E., Huang, G.B., RoyChowdhury, A., Li, H., Hua, G.: Labeled faces in the wild: a survey. In: Advances in Face Detection and Facial Image Analysis, Springer, 2016, pp. 189–248.

  12. 12.

    Werner, P., Saxen, F., Al-Hamadi, A.: Landmark based head pose estimation benchmark and method. In: Proceedings of the IEEE International Conference on Image Processing, 2017, pp. 3909–3913

  13. 13.

    Greff, K., Srivastava, R.K., Koutník, J., Steunebrink, B.R., Schmidhuber, J.: LSTM: a search space odyssey. IEEE Trans. Neural Netw. Learn. Syst. 28(10), 2222–2232 (2017)

    MathSciNet  Article  Google Scholar 

  14. 14.

    Chin, W.H., Seera, M., Loo, C.K., Seera, M., Kubota, N., Toda, Y.: Multi-channel Bayesian adaptive resonance associate memory for on-line topological map building. Appl. Soft Comput. 38(1), 269–280 (2016)

    Article  Google Scholar 

  15. 15.

    Ma, W.T., Chen, B.D., Zhao, H.Q., Gui, G., Duan, J.D., Principe, J.C.: Sparse least logarithmic absolute difference algorithm with correntropy-induced metric penalty. Circ. Syst. Signal Process. 35(3), 1077–1089 (2016)

    MathSciNet  Article  Google Scholar 

  16. 16.

    Parkhi, O.M., Vedaldi, A., Zisserman, A.: Deep face recognition. Proc. Br. Mach. Vis. 1(3), 1–12 (2015)

    Google Scholar 

  17. 17.

    Gao, W., Cao, B., Shan, S.G., Chen, X.L., Zhou, D.L., Zhang, X.H., Zhao, D.B.: The CAS-PEAL large-scale Chinese face database and baseline evaluations. IEEE Trans. Syst. Man Cybern. Part A Syst. Hum. 38(1), 149–161 (2008)

    Article  Google Scholar 

  18. 18.

    Sengupta, S., Chen, J.C., Castillo, C., Patel, V.M., Chellappa, R., Jacobs, D.W.: Frontal to profile face verification in the wild. In: Proceedings of the IEEE Winter Conference on Applications of Computer Vision, 2016, pp. 1–9

  19. 19.

    Gross, R., Matthews, I., Cohn, J., Kanade, T., Baker, S.: Multi-PIE. Image Vis. Comput. 28(5), 807–813 (2010)

    Article  Google Scholar 

  20. 20.

    Zhang, Z., Luo, P., Loy, C.C., Tang, X.: Learning deep representation for face alignment with auxiliary attributes. IEEE Trans. Pattern Anal. Mach. Intell. 38(5), 918–930 (2016)

    Article  Google Scholar 

  21. 21.

    Feng, Z., Huber, P., Kittler, J., Christmas, W., Wu, X.: Random cascaded-regression copse for robust facial landmark detection. IEEE Signal Process. Lett. 22(1), 76–80 (2015)

    Article  Google Scholar 

  22. 22.

    Chu, W.B., Guan, Y.P.: Identity verification based on facial pose pool and bag of words model. J. Adv. Intell. Intell. Inf. 21(3), 448–455 (2017)

    Article  Google Scholar 

  23. 23.

    Oji, R.: An automatic algorithm for object recognition and detection based on ASIFT keypoints. Signal Image Process. 3(5), 29–39 (2012)

    Google Scholar 

  24. 24.

    Cebeci, Z., Yildiz, F.: Comparison of K-means and fuzzy C-means algorithms on different cluster structures. J. Agric. Inf. 6(3), 13–23 (2015)

    Google Scholar 

  25. 25.

    Hartigan, J.A., Wong, M.A.: Algorithm AS 136: A K-means clustering algorithm. J. Roy. Stat. Soc. 28(1), 100–108 (1979)

    MATH  Google Scholar 

  26. 26.

    Zhao, L., Lin, J.: Pose-invariant face recognition with two-pathway convolutional neural network. J. East China Univ. Sci. Technol. 6(1), 1–7 (2018)

    Google Scholar 

  27. 27.

    Wen, Y.D., Zhang, K.P., Li, Z.F., Qiao, Y.: A discriminative feature learning approach for deep face recognition. Proc. Eur. Conf. Comput. Vis. 1, 499–515 (2016)

    Google Scholar 

Download references


The funding was received by National Natural Science Foundation of China (Grant nos. 11176016, 60872117) and Specialized Research Fund for the Doctoral Program of Higher Education of China (Grant no. 20123108110014).

Author information



Corresponding author

Correspondence to Yepeng Guan.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Guan, Y., Fang, J. & Wu, X. Multi-pose face recognition using Cascade Alignment Network and incremental clustering. SIViP 15, 63–71 (2021).

Download citation


  • Multi-pose face recognition
  • Face alignment
  • Incremental clustering
  • Correntropy-induced metric
  • Facial landmarks