Abstract
The objective of this paper is to learn a compact representation of image sets for template-based face recognition. We make the following contributions: first, we propose a network architecture which aggregates and embeds the face descriptors produced by deep convolutional neural networks into a compact fixed-length representation. This compact representation requires minimal memory storage and enables efficient similarity computation. Second, we propose a novel GhostVLAD layer that includes ghost clusters, that do not contribute to the aggregation. We show that a quality weighting on the input faces emerges automatically such that informative images contribute more than those with low quality, and that the ghost clusters enhance the network’s ability to deal with poor quality images. Third, we explore how input feature dimension, number of clusters and different training techniques affect the recognition performance. Given this analysis, we train a network that far exceeds the state-of-the-art on the IJB-B face recognition dataset. This is currently one of the most challenging public benchmarks, and we surpass the state-of-the-art on both the identification and verification protocols.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Arandjelović, O., Cipolla, R.: An information-theoretic approach to face recognition from face motion manifolds. Image Vis. Comput. 24(6), 639–647 (2006)
Arandjelović, R., Gronat, P., Torii, A., Pajdla, T., Sivic, J.: NetVLAD: CNN architecture for weakly supervised place recognition. In: Proceedings of the CVPR (2016)
Arandjelović, R., Zisserman, A.: Three things everyone should know to improve object retrieval. In: Proceedings of the CVPR (2012)
Cao, Q., Shen, L., Xie, W., Parkhi, O.M., Zisserman, A.: VGGFace2: a dataset for recognising faces across pose and age. In: Proceedings of the International Conference on Automatic Face & Gesture Recognition (2018)
Cevikalp, H., Triggs, B.: Face recognition based on image sets. In: Proceedings of the CVPR (2010)
Crosswhite, N., Byrne, J., Stauffer, C., Parkhi, O.M., Cao, Q., Zisserman, A.: Template adaptation for face verification and identification. In: Proceedings of the International Conference on Automatic Face & Gesture Recognition (2017)
Goswami, G., Bhardwaj, R., Singh, R., Vatsa, M.: MDLFace: memorability augmented deep learning for video face recognition. In: 2014 IEEE International Joint Conference on Biometrics (IJCB), pp. 1–7. IEEE (2014)
Guo, Y., Zhang, L., Hu, Y., He, X., Gao, J.: MS-celeb-1M: a dataset and benchmark for large-scale face recognition. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9907, pp. 87–102. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46487-9_6
Hassner, T., et al.: Pooling faces: template based face recognition with pooled face images. In: CVPR Workshops, pp. 59–67 (2016)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the CVPR (2016)
Hu, J., Shen, L., Sun, G.: Squeeze-and-excitation networks. In: Proceedings of the CVPR (2018)
Huang, Z., Wang, R., Shan, S., Chen, X.: Projection metric learning on Grassmann manifold with application to video based face recognition. In: Proceedings of the CVPR, pp. 140–149 (2015)
Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift. In: Proceedings of the ICML (2015)
Jégou, H., Douze, M., Schmid, C., Pérez, P.: Aggregating local descriptors into a compact image representation. In: Proceedings of the CVPR (2010)
Jégou, H., Zisserman, A.: Triangulation embedding and democratic aggregation for image search. In: Proceedings of the CVPR (2014)
Kemelmacher-Shlizerman, I., Seitz, S., Miller, D., Brossard, E.: The megaface benchmark: 1 million faces for recognition at scale. In: Proceedings of the CVPR, June 2016
Kim, T., Arandjelović, O., Cipolla, R.: Boosted manifold principal angles for image set-based recognition. Pattern Recognit. 40(9), 2475–2484 (2007)
Klare, B., et al.: Pushing the frontiers of unconstrained face detection and recognition: IARPA Janus benchmark-A. In: Proceedings of the CVPR, pp. 1931–1939 (2015)
Lee, K., Ho, J., Yang, M., Kriegman, D.: Video-based face recognition using probabilistic appearance manifolds. In: Proceedings of the CVPR (2003)
Liu, Y., Yan, J., Ouyang, W.: Quality aware network for set to set recognition. In: Proceedings of the CVPR, pp. 5790–5799 (2017)
Parkhi, O.M., Simonyan, K., Vedaldi, A., Zisserman, A.: A compact and discriminative face track descriptor. In: Proceedings of the CVPR. IEEE (2014)
Parkhi, O.M., Vedaldi, A., Zisserman, A.: Deep face recognition. In: Proceedings of the BMVC (2015)
Perronnin, F., Liu, Y., Sánchez, J., Poirier, H.: Large-scale image retrieval with compressed fisher vectors. In: Proceedings of the CVPR (2010)
Rao, Y., Lin, J., Lu, J., Zhou, J.: Learning discriminative aggregation network for video-based face recognition. In: Proceedings of the ICCV, pp. 3781–3790 (2017)
Schroff, F., Kalenichenko, D., Philbin, J.: FaceNet: a unified embedding for face recognition and clustering. In: Proceedings of the CVPR (2015)
Taigman, Y., Yang, M., Ranzato, M., Wolf, L.: Deep-face: closing the gap to human-level performance in face verification. In: IEEE CVPR (2014)
Vedaldi, A., Lenc, K.: MatConvNet: convolutional neural networks for Matlab. In: Proceedings of the ACMM (2015)
Wang, W., Wang, R., Huang, Z., Shan, S., Chen, X.: Discriminant analysis on Riemannian manifold of Gaussian distributions for face recognition with image sets. In: Proceedings of the CVPR, pp. 2048–2057 (2015)
Wang, W., Wang, R., Shan, S., Chen, X.: Discriminative covariance oriented representation learning for face recognition with image sets. In: Proceedings of the CVPR, pp. 5599–5608 (2017)
Whitelam, C., et al.: IARPA Janus benchmark-B face dataset. In: CVPR Workshop on Biometrics (2017)
Xie, W., Shen, L., Zisserman, A.: Comparator networks. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11215, pp. 811–826. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01252-6_48
Xie, W., Zisserman, A.: Multicolumn networks for face recognition. In: Proceedings of the BMVC (2018)
Yang, J., Ren, P., Chen, D., Wen, F., Li, H., Hua, G.: Neural aggregation network for video face recognition. In: Proceedings of the CVPR, pp. 4362–4371 (2017)
Zhang, R., Isola, P., Efros, A.A.: Colorful image colorization. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9907, pp. 649–666. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46487-9_40
Zheng, Y., Pal, D., Savvides, M.: Ring loss: convex feature normalization for face recognition. In: Proceedings of the CVPR, pp. 5089–5097 (2018)
Zhong, Y., Arandjelović, R., Zisserman, A.: GhostVLAD for set-based face recognition. arXiv preprint arXiv:1810.09951 (2018)
Acknowledgements
We thank Weidi Xie for his useful advice, and we thank Li Shen for providing pre-trained networks. This work was funded by an EPSRC studentship and EPSRC Programme Grant Seebibyte EP/M013774/1.
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Zhong, Y., Arandjelović, R., Zisserman, A. (2019). GhostVLAD for Set-Based Face Recognition. In: Jawahar, C., Li, H., Mori, G., Schindler, K. (eds) Computer Vision – ACCV 2018. ACCV 2018. Lecture Notes in Computer Science(), vol 11362. Springer, Cham. https://doi.org/10.1007/978-3-030-20890-5_3
Download citation
DOI: https://doi.org/10.1007/978-3-030-20890-5_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-20889-9
Online ISBN: 978-3-030-20890-5
eBook Packages: Computer ScienceComputer Science (R0)