Advertisement

International Journal of Computer Vision

, Volume 127, Issue 2, pp 181–206 | Cite as

Group Collaborative Representation for Image Set Classification

  • Bo Liu
  • Liping JingEmail author
  • Jia Li
  • Jian Yu
  • Alex Gittens
  • Michael W. Mahoney
Article
  • 319 Downloads

Abstract

With significant advances in imaging technology, multiple images of a person or an object are becoming readily available in a number of real-life scenarios. In contrast to single images, image sets can capture a broad range of variations in the appearance of a single face or object. Recognition from these multiple images (i.e., image set classification) has gained significant attention in the area of computer vision. Unlike many existing approaches, which assume that only the images in the same set affect each other, this work develops a group collaborative representation (GCR) model which makes no such assumption, and which can effectively determine the hidden structure among image sets. Specifically, GCR takes advantage of the relationship between image sets to capture the inter- and intra-set variations, and it determines the characteristic subspaces of all the gallery sets. In these subspaces, individual gallery images and each probe set can be effectively represented via a self-representation learning scheme, which leads to increased discriminative ability and enhances robustness and efficiency of the prediction process. By conducting extensive experiments and comparing with state-of-the-art, we demonstrated the superiority of the proposed method on set-based face recognition and object categorization tasks.

Keywords

Image set classification Group collaborative representation Point-to-sets representation Set-to-sets representation 

Notes

Acknowledgements

Funding was provided by National Natural Science Foundation of China (Grant Nos. 61632004, 61773050) and Opening Project of Beijing Key Lab of Traffic Data Analysis and Mining (Grant No. BKLTDAM2017001).

References

  1. Arandjelovic, O., Shakhnarovich, G., Fisher, J., Cipolla, R., & Darrell, T. (2005). Face recognition with image sets using manifold density divergence. In IEEE conference on computer vision and pattern recognition (Vol. 1, pp. 581–588). IEEE.Google Scholar
  2. Bach, F., Jenatton, R., Mairal, J., Obozinski, G., et al. (2012). Structured sparsity through convex optimization. Statistical Science, 27(4), 450–468.MathSciNetzbMATHGoogle Scholar
  3. Boyd, S., Parikh, N., Chu, E., Peleato, B., & Eckstein, J. (2011). Distributed optimization and statistical learning via the alternating direction method of multipliers. Foundations and Trends in Machine Learning, 3(1), 1–122.zbMATHGoogle Scholar
  4. Cai, D., He, X., Han, J., & Huang, T. S. (2011). Graph regularized nonnegative matrix factorization for data representation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(8), 1548–1560.Google Scholar
  5. Cevikalp, H., & Triggs, B. (2010). Face recognition based on image sets. In IEEE conference on computer vision and pattern recognition (pp. 2567–2573). IEEE.Google Scholar
  6. Chen, S., Sanderson, C., Harandi, M. T., & Lovell, B. C. (2013). Improved image set classification via joint sparse approximated nearest subspaces. In IEEE conference on computer vision and pattern recognition (pp. 452–459). IEEE.Google Scholar
  7. Davies, D. L., & Bouldin, D. W. (1979). A cluster separation measure. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1(2), 224–227.Google Scholar
  8. Deng, W., & Yin, W. (2016). On the global and linear convergence of the generalized alternating direction method of multipliers. Journal of Scientific Computing, 66(3), 889–916.MathSciNetzbMATHGoogle Scholar
  9. Gross, R., & Shi, J. (2001). The CMU motion of body (mobo) database. Technical report.Google Scholar
  10. Harandi, M. T., Sanderson, C., Shirazi, S., & Lovell, B. C. (2011). Graph embedding discriminant analysis on Grassmannian manifolds for improved image set matching. In IEEE conference on computer vision and pattern recognition (pp. 2705–2712). IEEE.Google Scholar
  11. Hayat, M., Bennamoun, M., & An, S. (2014). Reverse training: An efficient approach for image set classification. In European conference on computer vision (pp. 784–799). Springer.Google Scholar
  12. Hayat, M., Bennamoun, M., & An, S. (2015). Deep reconstruction models for image set classification. IEEE Transactions on Pattern Analysis and Machine Intelligence, 37(4), 713–727.Google Scholar
  13. Hu, Y., Mian, A. S., & Owens, R. (2012). Face recognition using sparse approximated nearest points between image sets. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(10), 1992–2004.Google Scholar
  14. Huang, Z., Wang, R., Shan, S., & Chen, X. (2014). Learning Euclidean-to-Riemannian metric for point-to-set classification. In 2014 IEEE conference on computer vision and pattern recognition (CVPR) (pp. 1677–1684). IEEE.Google Scholar
  15. Kim, M., Kumar, S., Pavlovic, V., & Rowley, H. (2008). Face tracking and recognition with visual constraints in real-world videos. In IEEE conference on computer vision and pattern recognition (pp. 1–8). IEEE.Google Scholar
  16. Kim, T. K., Kittler, J., & Cipolla, R. (2007). Discriminative learning and recognition of image set classes using canonical correlations. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29(6), 1005–1018.Google Scholar
  17. Lai, K., Bo, L., Ren, X., & Fox, D. (2011). A large-scale hierarchical multi-view RGB-D object dataset. In IEEE international conference on robotics and automation (pp. 1817–1824). IEEE.Google Scholar
  18. Lee, K. C., Ho, J., Yang, M. H., & Kriegman, D. (2003). Video-based face recognition using probabilistic appearance manifolds. In IEEE conference on computer vision and pattern recognition (Vol. 1, pp 1–313). IEEE.Google Scholar
  19. Lin, Z., Liu, R., & Su, Z. (2011). Linearized alternating direction method with adaptive penalty for low-rank representation. In Advances in neural information processing systems (pp. 612–620).Google Scholar
  20. Lu, C. Y., Min, H., Zhao, Z. Q., Zhu, L., Huang, D. S., & Yan, S. (2012). Robust and efficient subspace segmentation via least squares regression. In Proceedings of the 12th European conference on computer vision (pp. 347–360). Springer.Google Scholar
  21. Lu, J., Wang, G., Deng, W., Moulin, P., & Zhou, J. (2015). Multi-manifold deep metric learning for image set classification. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 1137–1145).Google Scholar
  22. Lu, J., Wang, G., & Moulin, P. (2013). Image set classification using holistic multiple order statistics features and localized multi-kernel metric learning. In IEEE international conference on computer vision (pp. 329–336). IEEE.Google Scholar
  23. Mahmood, A., Mian, A., & Owens, R. (2014). Semi-supervised spectral clustering for image set classification. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 121–128).Google Scholar
  24. Ng, M. K., Wang, F., & Yuan, X. (2011). Inexact alternating direction methods for image recovery. SIAM Journal on Scientific Computing, 33(4), 1643–1668.MathSciNetzbMATHGoogle Scholar
  25. Parkhi, O. M., Vedaldi, A., & Zisserman, A. (2015). Deep face recognition. In British machine vision conference (Vol. 1, p. 6).Google Scholar
  26. Razaviyayn, M., Hong, M., & Luo, Z. Q. (2013). A unified convergence analysis of block successive minimization methods for nonsmooth optimization. SIAM Journal on Optimization, 23(2), 1126–1153.MathSciNetzbMATHGoogle Scholar
  27. Saunders, C., Gammerman, A., & Vovk, V. (1998). Ridge regression learning algorithm in dual variables. In Proceedings of the 15th international conference on machine learning (pp. 515–521). Morgan Kaufmann.Google Scholar
  28. Shakhnarovich, G., Fisher, J. W., & Darrell, T. (2002). Face recognition from long-term observations. In European conference on computer vision (pp. 851–865). Springer.Google Scholar
  29. Simonyan, K., & Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:14091556.
  30. Tao, M., & Yuan, X. (2011). Recovering low-rank and sparse components of matrices from incomplete and noisy observations. SIAM Journal on Optimization, 21(1), 57–81.MathSciNetzbMATHGoogle Scholar
  31. Uzair, M., Mahmood, A., & Mian, A. (2014). Sparse kernel learning for image set classification. In Asian conference on computer vision (pp. 617–631). Springer.Google Scholar
  32. Vidal, R. (2011). Subspace clustering. IEEE Signal Processing Magazine, 28(2), 52–68.Google Scholar
  33. Viola, P., & Jones, M. J. (2004). Robust real-time face detection. International Journal of Computer Vision, 57(2), 137–154.Google Scholar
  34. Wang, R., & Chen, X. (2009). Manifold discriminant analysis. In IEEE conference on computer vision and pattern recognition (pp. 429–436). IEEE.Google Scholar
  35. Wang, R., Guo, H., Davis, L. S., & Dai, Q. (2012). Covariance discriminative learning: A natural and efficient approach to image set classification. In IEEE conference on computer vision and pattern recognition (pp. 2496–2503). IEEE.Google Scholar
  36. Wang, R., Shan, S., Chen, X., & Gao, W. (2008). Manifold–manifold distance with application to face recognition based on image set. In IEEE conference on computer vision and pattern recognition (pp. 1–8). IEEE.Google Scholar
  37. Wang, W., Wang, R., Huang, Z., Shan, S., & Chen, X. (2015). Discriminant analysis on Riemannian manifold of Gaussian distributions for face recognition with image sets. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 2048–2057).Google Scholar
  38. Wolf, L., Hassner, T., & Maoz, I. (2011). Face recognition in unconstrained videos with matched background similarity. In IEEE conference on computer vision and pattern recognition (pp. 529–534). IEEE.Google Scholar
  39. Yang, M., Zhu, P., Van Gool, L., & Zhang, L. (2013). Face recognition based on regularized nearest points between image sets. In 2013 10th IEEE international conference and workshops on automatic face and gesture recognition (FG) (pp. 1–7). IEEE.Google Scholar
  40. Yuan, M., & Lin, Y. (2006). Model selection and estimation in regression with grouped variables. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 68(1), 49–67.MathSciNetzbMATHGoogle Scholar
  41. Zhang, D., Yang, M., & Feng, X. (2011). Sparse representation or collaborative representation: Which helps face recognition? In Proceedings of the 13th international conference on computer vision (pp. 471–478). IEEE.Google Scholar
  42. Zhu, P., Zhang, L., Zuo, W., & Zhang, D. (2013). From point to set: Extend the learning of distance metrics. In 2013 IEEE international conference on computer vision (ICCV) (pp. 2664–2671). IEEE.Google Scholar
  43. Zhu, P., Zuo, W., Zhang, L., & Shiu, S. C. (2014). Image set based collaborative representation for face recognition. IEEE Transactions on Information Forensics and Security, 9(7), 1120–1132.Google Scholar
  44. Zou, H., & Hastie, T. (2005). Regularization and variable selection via the elastic net. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 67(2), 301–320.MathSciNetzbMATHGoogle Scholar
  45. Zou, H., Zhu, J., Hastie, T., et al. (2008). New multicategory boosting algorithms based on multicategory fisher-consistent losses. The Annals of Applied Statistics, 2(4), 1290–1306.MathSciNetzbMATHGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  • Bo Liu
    • 1
    • 2
  • Liping Jing
    • 1
    Email author
  • Jia Li
    • 1
  • Jian Yu
    • 1
  • Alex Gittens
    • 3
  • Michael W. Mahoney
    • 4
    • 5
  1. 1.Beijing Key Lab of Traffic Data Analysis and MiningBeijing Jiaotong UniversityBeijingChina
  2. 2.College of Information Science and TechnologyAgricultural University of HebeiBaodingChina
  3. 3.Computer Science DepartmentRensselaer Polytechnic InstituteTroyUSA
  4. 4.International Computer Science InstituteUniversity of California at BerkeleyBerkeleyUSA
  5. 5.Department of StatisticsUniversity of California at BerkeleyBerkeleyUSA

Personalised recommendations