Advertisement

Efficient k-Support Matrix Pursuit

  • Hanjiang Lai
  • Yan Pan
  • Canyi Lu
  • Yong Tang
  • Shuicheng Yan
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8690)

Abstract

In this paper, we study the k-support norm regularized matrix pursuit problem, which is regarded as the core formulation for several popular computer vision tasks. The k-support matrix norm, a convex relaxation of the matrix sparsity combined with the ℓ2-norm penalty, generalizes the recently proposed k-support vector norm. The contributions of this work are two-fold. First, the proposed k-support matrix norm does not suffer from the disadvantages of existing matrix norms towards sparsity and/or low-rankness: 1) too sparse/dense, and/or 2) column independent. Second, we present an efficient procedure for k-support norm optimization, in which the computation of the key proximity operator is substantially accelerated by binary search. Extensive experiments on subspace segmentation, semi-supervised classification and sparse coding well demonstrate the superiority of the new regularizer over existing matrix-norm regularizers, and also the orders-of-magnitude speedup compared with the existing optimization procedure for the k-support norm.

Keywords

k-support norm subspace segmentation semi-supervised classification sparse coding 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Supplementary material

978-3-319-10605-2_40_MOESM1_ESM.pdf (68 kb)
Electronic Supplementary Material (PDF 68 KB)

References

  1. 1.
    Argyriou, A., Foygel, R., Srebro, N.: Sparse prediction with the k -support norm. In: Advances in Neural Information Processing Systems, pp. 1466–1474 (2012)Google Scholar
  2. 2.
    Beck, A., Teboulle, M.: A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM Journal on Imaging Sciences 2(1), 183–202 (2009)CrossRefzbMATHMathSciNetGoogle Scholar
  3. 3.
    Elhamifar, E., Vidal, R.: Sparse subspace clustering. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2790–2797 (2009)Google Scholar
  4. 4.
    Georghiades, A.S., Belhumeur, P.N., Kriegman, D.J.: From few to many: Illumination cone models for face recognition under variable lighting and pose. IEEE Transactions on Pattern Analysis and Machine Intelligence 23(6), 643–660 (2001)CrossRefGoogle Scholar
  5. 5.
    Grave, E., Obozinski, G., Bach, F.: et al.: Trace lasso: a trace norm regularization for correlated designs. Advances in Neural Information Processing Systems 2, 5 (2011)Google Scholar
  6. 6.
    Hull, J.J.: A database for handwritten text recognition research. IEEE Transactions on Pattern Analysis and Machine Intelligence 16(5), 550–554 (1994)CrossRefGoogle Scholar
  7. 7.
    Lazebnik, S., Schmid, C., Ponce, J.: Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 2, pp. 2169–2178 (2006)Google Scholar
  8. 8.
    Lewis, G.N., Boynton, N.J., Burton, F.W.: Expected complexity of fast search with uniformly distributed data. Information Processing Letters 13(1), 4–7 (1981)CrossRefzbMATHMathSciNetGoogle Scholar
  9. 9.
    Li, L.J., Fei-Fei, L.: What, where and who? classifying events by scene and object recognition. In: IEEE International Conference on Computer Vision, pp. 1–8 (2007)Google Scholar
  10. 10.
    Lin, Z., Liu, R., Su, Z.: Linearized alternating direction method with adaptive penalty for low-rank representation. In: Advances in Neural Information Processing Systems, pp. 612–620 (2011)Google Scholar
  11. 11.
    Liu, G., Lin, Z., Yu, Y.: Robust subspace segmentation by low-rank representation. In: International Conference on Machine Learning, pp. 663–670 (2010)Google Scholar
  12. 12.
    Lowe, D.G.: Object recognition from local scale-invariant features. In: IEEE International Conference on Computer Vision, vol. 2, pp. 1150–1157 (1999)Google Scholar
  13. 13.
    Jia, K., Chan, T.-H., Ma, Y.: Robust and efficient subspace segmentation via least squares regression. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part IV. LNCS, vol. 7575, pp. 331–344. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  14. 14.
    Lu, C., Feng, J., Lin, Z., Yan, S.: Correlation adaptive subspace segmentation by trace lasso. In: IEEE International Conference on Computer Vision (2013)Google Scholar
  15. 15.
    Shalev-Shwartz, S., Shamir, O., Sridharan, K., Srebro, N.: Learnability and stability in the general learning setting. In: Annual Conference of Learning Theory (2009)Google Scholar
  16. 16.
    Szummer, M., Jaakkola, T.: Partially labeled classification with markov random walks. In: Advances in Neural Information Processing Systems, pp. 945–952 (2001)Google Scholar
  17. 17.
    Von Luxburg, U.: A tutorial on spectral clustering. Statistics and Computing 17(4), 395–416 (2007)CrossRefMathSciNetGoogle Scholar
  18. 18.
    Wang, J., Yang, J., Yu, K., Lv, F., Huang, T., Gong, Y.: Locality-constrained linear coding for image classification. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 3360–3367 (2010)Google Scholar
  19. 19.
    Xu, H., Caramanis, C., Mannor, S.: Sparse algorithms are not stable: A no-free-lunch theorem. IEEE Transactions on Pattern Analysis and Machine Intelligence 34(1), 187–193 (2012)CrossRefMathSciNetGoogle Scholar
  20. 20.
    Yang, J., Yu, K., Gong, Y., Huang, T.: Linear spatial pyramid matching using sparse coding for image classification. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1794–1801 (2009)Google Scholar
  21. 21.
    Yuan, X.T., Liu, X., Yan, S.: Visual classification with multitask joint sparse representation. IEEE Transactions on Image Processing 21(10), 4349–4360 (2012)CrossRefMathSciNetGoogle Scholar
  22. 22.
    Zhuang, L., Gao, H., Lin, Z., Ma, Y., Zhang, X., Yu, N.: Non-negative low rank and sparse graph for semi-supervised learning. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2328–2335 (2012)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Hanjiang Lai
    • 1
    • 3
  • Yan Pan
    • 2
  • Canyi Lu
    • 1
  • Yong Tang
    • 4
  • Shuicheng Yan
    • 1
  1. 1.Department of Electrical and Computer EngineeringNational University of SingaporeSingapore
  2. 2.School of SoftwareSun Yat-sen UniversityChina
  3. 3.School of Information Science and TechnologySun Yat-sen UniversityChina
  4. 4.School of Computer ScienceSouth China Normal UniversityChina

Personalised recommendations