Consensus-Driven Propagation in Massive Unlabeled Data for Face Recognition

  • Xiaohang ZhanEmail author
  • Ziwei Liu
  • Junjie Yan
  • Dahua Lin
  • Chen Change Loy
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11213)


Face recognition has witnessed great progress in recent years, mainly attributed to the high-capacity model designed and the abundant labeled data collected. However, it becomes more and more prohibitive to scale up the current million-level identity annotations. In this work, we show that unlabeled face data can be as effective as the labeled ones. Here, we consider a setting closely mimicking the real-world scenario, where the unlabeled data are collected from unconstrained environments and their identities are exclusive from the labeled ones. Our main insight is that although the class information is not available, we can still faithfully approximate these semantic relationships by constructing a relational graph in a bottom-up manner. We propose Consensus-Driven Propagation (CDP) to tackle this challenging problem with two modules, the “committee” and the “mediator”, which select positive face pairs robustly by carefully aggregating multi-view information. Extensive experiments validate the effectiveness of both modules to discard outliers and mine hard positives. With CDP, we achieve a compelling accuracy of 78.18% on MegaFace identification challenge by using only 9% of the labels, comparing to 61.78% when no unlabeled data are used and 78.52% when all labels are employed.



This work is partially supported by the Big Data Collaboration Research grant from SenseTime Group (CUHK Agreement No. TS1610626), the General Research Fund (GRF) of Hong Kong (No. 14236516, 14241716).

Supplementary material

474192_1_En_35_MOESM1_ESM.pdf (21.3 mb)
Supplementary material 1 (pdf 21847 KB)


  1. 1.
    Argamon-Engelson, S., Dagan, I.: Committee-based sample selection for probabilistic classifiers. J. Artif. Intell. Res. 11, 335–360 (1999)CrossRefGoogle Scholar
  2. 2.
    Blum, A., Mitchell, T.: Combining labeled and unlabeled data with co-training. In: Proceedings of the Eleventh Annual Conference on Computational Learning Theory (1998)Google Scholar
  3. 3.
    Cao, K., Rong, Y., Li, C., Tang, X., Loy, C.C.: Pose-robust face recognition via deep residual equivariant mapping. In: CVPR (2018)Google Scholar
  4. 4.
    Chapelle, O., Scholkopf, B., Zien, A.: Semi-supervised learning. IEEE Trans. Neural Netw. 20(3), 542 (2009). Chapelle, o. et al. (eds.) (2006) [book reviews]CrossRefGoogle Scholar
  5. 5.
    Chen, L., et al.: The devil of face recognition is in the noise. In: ECCV (2018)Google Scholar
  6. 6.
    Dempster, A.P., Laird, N.M., Rubin, D.B.: Maximum likelihood from incomplete data via the EM algorithm. J. Roy. Stat. Soc. Ser. B (methodol.) 39, 1–38 (1977)MathSciNetzbMATHGoogle Scholar
  7. 7.
    Deng, J., Guo, J., Zafeiriou, S.: Arcface: additive angular margin loss for deep face recognition. arXiv preprint arXiv:1801.07698 (2018)
  8. 8.
    Gao, Y., Ma, J., Yuille, A.L.: Semi-supervised sparse representation based classification for face recognition with insufficient labeled samples. TIP 26(5), 2545–2560 (2017)MathSciNetGoogle Scholar
  9. 9.
    Guo, Y., Zhang, L., Hu, Y., He, X., Gao, J.: MS-Celeb-1M: a dataset and benchmark for large-scale face recognition. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9907, pp. 87–102. Springer, Cham (2016). Scholar
  10. 10.
    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: CVPR (2016)Google Scholar
  11. 11.
    Huang, C., Li, Y., Loy, C.C., Tang, X.: Deep imbalanced learning for face recognition and attribute prediction. arXiv preprint arXiv:1806.00194 (2018)
  12. 12.
    Iandola, F., Moskewicz, M., Karayev, S., Girshick, R., Darrell, T., Keutzer, K.: DenseNet: implementing efficient convnet descriptor pyramids. arXiv preprint arXiv:1404.1869 (2014)
  13. 13.
    Kemelmacher-Shlizerman, I., Seitz, S.M., Miller, D., Brossard, E.: The MegaFace benchmark: 1 million faces for recognition at scale. In: CVPR (2016)Google Scholar
  14. 14.
    Liu, Z., Luo, P., Wang, X., Tang, X.: Deep learning face attributes in the wild. In: ICCV (2015)Google Scholar
  15. 15.
    Loy, C.C., Hospedales, T.M., Xiang, T., Gong, S.: Stream-based joint exploration-exploitation active learning. In: CVPR (2012)Google Scholar
  16. 16.
    Mitchell, T.M.: The role of unlabeled data in supervised learning. In: Larrazabal, J.M., Miranda, L.A.P. (eds.) Language, Knowledge, and Representation. PSSP, vol. 99, pp. 103–111. Springer, Dordrecht (2004). Scholar
  17. 17.
    Ng, H.W., Winkler, S.: A data-driven approach to cleaning large face datasets. In: ICIP (2014)Google Scholar
  18. 18.
    Roli, F., Marcialis, G.L.: Semi-supervised PCA-based face recognition using self-training. In: Yeung, D.-Y., Kwok, J.T., Fred, A., Roli, F., de Ridder, D. (eds.) SSPR /SPR 2006. LNCS, vol. 4109, pp. 560–568. Springer, Heidelberg (2006). Scholar
  19. 19.
    Rosenberg, C., Hebert, M., Schneiderman, H.: Semi-supervised self-training of object detection models (2005)Google Scholar
  20. 20.
    de Sa, V.R.: Learning classification with unlabeled data. In: NIPS (1994)Google Scholar
  21. 21.
    Schroff, F., Kalenichenko, D., Philbin, J.: FaceNet: a unified embedding for face recognition and clustering. In: CVPR (2015)Google Scholar
  22. 22.
    Seung, H.S., Opper, M., Sompolinsky, H.: Query by committee. In: Proceedings of the Fifth Annual Workshop on Computational Learning Theory (1992)Google Scholar
  23. 23.
    Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)
  24. 24.
    Sohn, K., Liu, S., Zhong, G., Yu, X., Yang, M.H., Chandraker, M.: Unsupervised domain adaptation for face recognition in unlabeled videos. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3210–3218 (2017)Google Scholar
  25. 25.
    Sun, Y., Chen, Y., Wang, X., Tang, X.: Deep learning face representation by joint identification-verification. In: NIPS (2014)Google Scholar
  26. 26.
    Sun, Y., Wang, X., Tang, X.: Deep learning face representation from predicting 10,000 classes. In: CVPR (2014)Google Scholar
  27. 27.
    Szegedy, C., Ioffe, S., Vanhoucke, V., Alemi, A.A.: Inception-v4, inception-ResNet and the impact of residual connections on learning. In: AAAI, vol. 4 (2017)Google Scholar
  28. 28.
    Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. In: CVPR (2016)Google Scholar
  29. 29.
    Wen, Y., Zhang, K., Li, Z., Qiao, Y.: A discriminative feature learning approach for deep face recognition. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9911, pp. 499–515. Springer, Cham (2016). Scholar
  30. 30.
    Yarowsky, D.: Unsupervised word sense disambiguation rivaling supervised methods. In: ACL (1995)Google Scholar
  31. 31.
    Zhan, X., Liu, Z., Luo, P., Tang, X., Loy, C.C.: Mix-and-match tuning for self-supervised semantic segmentation. In: AAAI (2018)Google Scholar
  32. 32.
    Zhang, X., Yang, L., Yan, J., Lin, D.: Accelerated training for massive classification via dynamic class selection. In: AAAI (2018)Google Scholar
  33. 33.
    Zhao, X., Evans, N., Dugelay, J.L.: Semi-supervised face recognition with LDA self-training. In: ICIP (2011)Google Scholar
  34. 34.
    Zhu, X.: Semi-supervised learning literature survey. Comput. Sci. Univ. Wisconsin-Madison 2(3), 4 (2006)Google Scholar
  35. 35.
    Zhu, X., Ghahramani, Z.: Learning from labeled and unlabeled data with label propagation (2002)Google Scholar
  36. 36.
    Zhu, X., Lafferty, J., Rosenfeld, R.: Semi-supervised learning with graphs. Ph.D. thesis, Carnegie Mellon University, Language Technologies Institute, School Of Computer Science (2005)Google Scholar
  37. 37.
    Zoph, B., Vasudevan, V., Shlens, J., Le, Q.V.: Learning transferable architectures for scalable image recognition. arXiv preprint arXiv:1707.07012 (2017)

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  1. 1.CUHK - SenseTime Joint Lab, The Chinese University of Hong KongShatinHong Kong
  2. 2.SenseTime Group LimitedBeijingChina
  3. 3.Nanyang Technological UniversitySingaporeSingapore

Personalised recommendations