ClusterFace: Clustering-Driven Deep Face Recognition

  • Lingjiang Xie
  • Cuican Yu
  • Huibin LiEmail author
  • Jihua Zhu
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10996)


Recent years, image-based 2D face recognition has achieved human-level performance with the big breakthrough of deep learning paradigm. However, almost all of the existing deep face recognition methods depend on millions and millions of labeled 2D face images from different individual for supervised deep learning. In this case, face labelling becomes the pain point of deep face recognition. To solve this issue, we propose a novel clustering driven unsupervised deep face recognition framework, namely ClusterFace. In particular, our framework firstly assume that we already have a well-trained deep face model and a large number of face images without any labels. Then, all these face images are represented by this deep face model and then unsupervised clustered into different clusters using a certain clustering algorithm. Finally, these clustering-based face labelling results are employed to train a new deep CNN model for face recognition. Experimental results demonstrated that the proposed framework with a simple Mini-batch K-Means clustering algorithm can achieve surprising state-of-the-art performance (99.41%) on the LFW dataset. We also presented an intuitional explanation the reason of achieving good performance of our framework and also demonstrated its robustness to the choice of the number of clusters and the amount of unlabeled face images.


Deep face recognition Face clustering Mini-batch K-Means 


  1. 1.
    Huang, G.B., Ramesh, M., Berg, T., Learned-Miller, E.: Labeled faces in the wild: A database for studying face recognition in unconstrained environments. Technical report, University of Massachusetts (2007)Google Scholar
  2. 2.
    Learned-Miller, E., Huang G.B.: Labeled faces in the wild: Updates and new reporting procedures. Technical report, University of Massachusetts (2014)Google Scholar
  3. 3.
    Taigman, Y., Yang, M., Ranzato, M., Wolf, L.: Deepface: closing the gap to human-level performance in face verification. In: CVPR (2014)Google Scholar
  4. 4.
    Schroff, F., Kalenichenko, D., Philbin, J.: Facenet: a unified embedding for face recognition and clustering. In: CVPR (2015)Google Scholar
  5. 5.
    Guo, Y., Zhang, L., Hu, Y., He, X., Gao, J.: MS-Celeb-1M: a dataset and benchmark for large-scale face recognition. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9907, pp. 87–102. Springer, Cham (2016). Scholar
  6. 6.
    Sun, Y., Wang, X., Tang, X.: Deep learning face representation from predicting 10,000 classes. In: CVPR (2014)Google Scholar
  7. 7.
    Yi, D., Lei, Z., Liao, S., Li, S.Z.: Learning face representation from scratch. Computer Science (2014)Google Scholar
  8. 8.
    Parkhi, O.M., Vedaldi, A., Zisserman, A.: Deep face recognition. In: BMVC (2015)Google Scholar
  9. 9.
    Kemelmacher, I., Shlizerman, Seitz, S.M., Miller, D., Brossard, E.: The megaface benchmark: 1 million faces for recognition at scale. In: CVPR (2016)Google Scholar
  10. 10.
    Cao, Q., Shen, L., Xie, W., Parkhi, O.M., Zisserman, A.: VGGFace2: A dataset for recognising faces across pose and age. CoRR (2017)Google Scholar
  11. 11.
    Liu, W., Wen, Y., Yu, Z., Li, M., Raj, B., Song, L.: Sphereface: deep hypersphere embedding for face recognition. In: CVPR (2017)Google Scholar
  12. 12.
    Hasnat, M.A., Bohné, J., Milgram, J., Gentric, S., Chen, L.: Deepvisage: making face recognition simple yet with powerful generalization skills. In: IEEE International Conference on Computer Vision Workshops (2017)Google Scholar
  13. 13.
    Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. Computer Science (2014)Google Scholar
  14. 14.
    Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S.E., Anguelov, D., Erhan, D., Vanhoucke, V., Rabinovich, A.: Going deeper with convolutions. In: CVPR (2015)Google Scholar
  15. 15.
    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: CVPR (2016)Google Scholar
  16. 16.
    Wang, H., Wang, Y., Zhou, Z., Ji, X., Li, Z., Gong, D., Zhou, J., Liu, W.: Cosface: large margin cosine loss for deep face recognition. In: CVPR (2018)Google Scholar
  17. 17.
    Hadsell, R., Chopra, S., LeCun, Y.: Dimensionality reduction by learning an invariant mapping. In: CVPR (2006)Google Scholar
  18. 18.
    Wen, Y., Zhang, K., Li, Z., Qiao, Y.: A discriminative feature learning approach for deep face recognition. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9911, pp. 499–515. Springer, Cham (2016). Scholar
  19. 19.
    Liu, W., Wen, Y., Yu, Z., Yang, M.: Large-margin softmax loss for convolutional neural networks. In: ICML (2016)Google Scholar
  20. 20.
    Wu, B., Lyu, S., Hu, B., Ji, Q.: Simultaneous clustering and tracklet linking for multi-face tracking in videos. In: ICCV (2013)Google Scholar
  21. 21.
    Otto, C., Wang, D., Jain, A.K.: Clustering millions of faces by identity. In: IEEE Transactions on Pattern Analysis and Machine Intelligence (2018)Google Scholar
  22. 22.
    Deng, J., Guo, J., Zafeiriou, S.: Arcface: Additive angular margin loss for deep face recognition. CoRR (2018)Google Scholar
  23. 23.
    Sculley, D.: Web-scale k-means clustering. In: International Conference on World Wide Web (2010)Google Scholar
  24. 24.
    Zhang, K., Zhang, Z., Li, Z., Qiao, Y.: Joint face detection and alignment using multitask cascaded convolutional networks. IEEE Signal Process. Lett. 23, 1499–1503 (2016)CrossRefGoogle Scholar
  25. 25.
    Fowlkes, E.B., Mallows, C.L.: A method for comparing two hierarchical clusterings. Publ. Am. Stat. Assoc. 78, 553–569 (1983)CrossRefGoogle Scholar
  26. 26.
    Alex, K., Ilya, S., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: NIPS (2012)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Lingjiang Xie
    • 1
  • Cuican Yu
    • 2
  • Huibin Li
    • 2
    Email author
  • Jihua Zhu
    • 1
  1. 1.School of Software EngineeringXi’an Jiaotong UniversityXi’anChina
  2. 2.School of Mathematics and StatisticsXi’an Jiaotong UniversityXi’anChina

Personalised recommendations