Skip to main content
SpringerLink
Log in

Low-Rank and Sparse Dictionary Learning

  • Chapter
  • First Online:
Low-Rank and Sparse Modeling for Visual Analysis

Abstract

Learning an informative dictionary is a critical challenge in sparse representation and low-rank modeling. The quality of dictionary usually affects the performance of learning models significantly. In this chapter, we propose a novel low-rank dictionary learning method, which learns a discriminative dictionary with low-rank constraints. We learn a sub-dictionary for each class separately, and the overall representation ability of the dictionary is also considered. In particular, the Fisher criterion is incorporated in our model to improve the discriminability of dictionary, which maximizes the ratio of the between-class scatter to within-class scatter. In practice, training samples may contain noisy information, which would undermine the quality of the dictionary. Inspired by the recent advances in low-rank matrix recovery, we enforce a low-rank constraint on the sub-dictionary for each class to tackle this problem. Our model is formulated as an \(l_1\) regularized rank-minimization problem, which can be solved by the iterative projection method (IPM) and inexact augmented Lagrange multiplier (ALM) algorithms. The proposed discriminative dictionary learning with low-rank regularization (\(D^2L^2R^2\)) method is evaluated on four public face and digit image datasets, in comparison with existing representative dictionary learning and image classification methods. The experimental results demonstrate that our method outperforms related methods in various settings.

This chapter is reprinted with permission from Elsevier. “Learning Low-Rank and Discriminative Dictionary for Image Classification”, Image and Vision Computing, 2014. © [2014] Elsevier.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    The machine used installs 24 GB RAM and Intel Xeon W3350 CPU.

References

  1. M. Aharon, M. Elad, A. Bruckstein, K-svd: an algorithm for designing overcomplete dictionaries for sparse representation. IEEE Trans. Signal Process. 54(11), 4311–4322 (2006)

    Article  Google Scholar 

  2. T. Bai, Y. Li, Robust visual tracking using flexible structured sparse representation. IEEE Trans. Ind. Inf. 10(1), 538–547 (2014)

    Article  Google Scholar 

  3. S.R. Becker, E.J. Candès, M.C. Grant, Templates for convex cone problems with applications to sparse signal recovery. Math. Program. Comput. 3(3), 165–218 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  4. P.N. Belhumeur, J.P. Hespanha, D.J. Kriegman, Eigenfaces versus fisherfaces: recognition using class specific linear projection. IEEE TPAMI 19(7), 711–720 (1997)

    Article  Google Scholar 

  5. Y. Bengio, A.C. Courville, P. Vincent, Representation learning: a review and new perspectives. IEEE Trans. Pattern Anal. Mach. Intell. 35(8), 1798–1828 (2013)

    Article  Google Scholar 

  6. D.P. Bertsekas, Constrained Optimization and Lagrange Multiplier Methods (Academic Press, New York, 1982)

    MATH  Google Scholar 

  7. E. Candès, B. Recht, Exact matrix completion via convex optimization. Found. Comput. Math. 9(6), 717–772 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  8. E.J. Candès, X.D. Li, Y. Ma, J. Wright, Robust principal component analysis? J. ACM 58(3), 11 (2011)

    Article  MathSciNet  Google Scholar 

  9. E.J. Candès, B. Recht, Exact matrix completion via convex optimization. Found. Comput. Math. 9(6), 717–772 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  10. C.F. Chen, C.P. Wei, Y.F. Wang, Low-rank matrix recovery with structural incoherence for robust face recognition, in CVPR 2012, pp. 2618–2625 (2012)

    Google Scholar 

  11. S. Chen, S.A. Billings, W. Luo, Orthogonal least squares methods and their application to non-linear system identification. Int. J. Control 50(5), 1873–1896 (1989)

    Article  MathSciNet  MATH  Google Scholar 

  12. G. Davis, S. Mallat, M. Avellaneda, Adaptive greedy approximations. Constr. Approximation 13(1), 57–98 (1997)

    Article  MathSciNet  MATH  Google Scholar 

  13. D.L. Donoho, A. Maleki, A. Montanari, Message-passing algorithms for compressed sensing. Proc. Nat. Acad. Sci. 106(45), 18914–18919 (2009)

    Article  Google Scholar 

  14. D.L. Donoho, Y. Tsaig, Fast solution ofwhen the solution may be sparse. IEEE Trans. Inf. Theory 54(11), 4789 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  15. R.O. Duda, P.E. Hart, D.G. Stork, Pattern Classification, 2nd edn. (Wiley, New York, 2001)

    MATH  Google Scholar 

  16. B. Efron, T. Hastie, I. Johnstone, R. Tibshirani, Least angle regression. Ann. Stat. 32(2), 407–499 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  17. M. Elad, B. Matalon, M. Zibulevsky, Coordinate and subspace optimization methods for linear least squares with non-quadratic regularization. Appl. Comput. Harmonic Anal. 23(3), 346–367 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  18. K. Frisch, The logarithmic potential method of convex programming. Memorandum May 13, 1955 (1955)

    Google Scholar 

  19. S. Gao, I.W.-H. Tsang, Y. Ma, Learning category-specific dictionary and shared dictionary for fine-grained image categorization. IEEE Trans. Image Process. 23(2), 623–634 (2014)

    Article  MathSciNet  Google Scholar 

  20. H. Guo, Z. Jiang, L. Davis, Discriminative dictionary learning with pairwise constraints, in ACCV 2012, vol. 7724 (Springer, Berlin, 2013), pp. 328–342

    Google Scholar 

  21. G. Irie, D. Liu, Z. Li, S.-F. Chang, A bayesian approach to multimodal visual dictionary learning, in CVPR, pp. 329–336 (2013)

    Google Scholar 

  22. I.H. Jhuo, D. Liu, D.T. Lee, S.F. Chang, Robust visual domain adaptation with low-rank reconstruction, in CVPR 2012 (2012)

    Google Scholar 

  23. Z. Jiang, Z. Lin, L. Davis, Learning a discriminative dictionary for sparse coding via label consistent k-svd, in CVPR 2011, pp. 1697–1704 (2011)

    Google Scholar 

  24. N. Karmarkar, A new polynomial-time algorithm for linear programming, in Proceedings of the 16th Annual ACM Symposium on Theory of Computing 1984, (ACM, 1984), pp. 302–311

    Google Scholar 

  25. R. Keshavan, A. Montanari, S. Oh, Matrix completion from noisy entries. J. Mach. Learn. Res. 11, 2057–2078 (2010)

    MathSciNet  MATH  Google Scholar 

  26. K. Kreutz-Delgado, J.F. Murray, B.D. Rao, K. Engan, T.-W. Lee, T.J. Sejnowski, Dictionary learning algorithms for sparse representation. Neural Comput. 15(2), 349–396 (2003)

    Article  MATH  Google Scholar 

  27. Y. LeCun, L. Bottou, Y. Bengio, P. Haaffner, Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)

    Article  Google Scholar 

  28. H. Lee, A. Battle, R. Raina, A.Y. Ng, Efficient sparse coding algorithms, in NIPS 2007, ed. by B. Schölkopf, J. Platt, T. Hoffman (MIT Press, Cambridge, 2007), pp. 801–808

    Google Scholar 

  29. K.-C. Lee, J. Ho, D. Kriegman, Acquiring linear subspaces for face recognition under variable lighting. IEEE TPAMI 27(5), 684–698 (2005)

    Article  Google Scholar 

  30. M.S. Lewicki, B.A. Olshausen, Probabilistic framework for the adaptation and comparison of image codes. JOSA A 16(7), 1587–1601 (1999)

    Article  Google Scholar 

  31. L. Li, S. Li, Y. Fu, Discriminative dictionary learning with low-rank regularization for face recognition, in FG 2013 (2013)

    Google Scholar 

  32. L. Li, S. Li, Y. Fu, Learning low-rank and discriminative dictionary for image classification. Image Vis. Comput. (2014)

    Google Scholar 

  33. Z. Lin, M. Chen, Y. Ma, The augmented lagrange multiplier method for exact recovery of corrupted low-rank matrices. arXiv preprint arXiv:1009.5055 (2010)

  34. G.C. Liu, Z.C. Lin, Y. Yu, Robust subspace segmentation by low-rank representation, in ICML 2010, pp. 663–670 (2010)

    Google Scholar 

  35. G.C. Liu, S.C. Yan, Latent low-rank representation for subspace segmentation and feature extraction, in ICCV 2011 (2011)

    Google Scholar 

  36. R.S. Liu, Z.C. Lin, F.D. Torre, Z.X. Su, Fixed-rank representation for unsupervised visual learning, in CVPR 2012 (2012)

    Google Scholar 

  37. L. Ma, C. Wang, B. Xiao, W. Zhou, Sparse representation for face recognition based on discriminative low-rank dictionary learning, in CVPR 2012, pp. 2586–2593 (2012)

    Google Scholar 

  38. D.M. Malioutov, M. Cetin, A.S. Willsky, Homotopy continuation for sparse signal representation, in ICASSP 2005, IEEE, vol. 5, pp. 728–733 (2005)

    Google Scholar 

  39. S.G. Mallat, Z. Zhang, Matching pursuits with time-frequency dictionaries. IEEE Trans. Signal Process. 41(12), 3397–3415 (1993)

    Article  MATH  Google Scholar 

  40. X. Mo, V. Monga, R. Bala, Z. Fan, Adaptive sparse representations for video anomaly detection. IEEE Trans. Circuits Syst. Video Techn. 24(4), 631–645 (2014)

    Article  Google Scholar 

  41. I. Naseem, R. Togneri, M. Bennamoun, Linear regression for face recognition. IEEE TPAMI 32(11), 2106–2112 (2010)

    Article  Google Scholar 

  42. Y. Nesterov, A method of solving a convex programming problem with convergence rate o (1/k2). Sov. Math. Dokl. 27, 372–376 (1983)

    MATH  Google Scholar 

  43. M.R. Osborne, B. Presnell, B.A. Turlach, A new approach to variable selection in least squares problems. IMA J. Numer. Anal. 20(3), 389–403 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  44. T. Poggio, T. Serre, et al., Learning a dictionary of shape-components in visual cortex: comparison with neurons, humans and machines. Ph.D. thesis, Massachusetts Institute of Technology (2006)

    Google Scholar 

  45. L. Rosasco, S. Mosci, S. Santoro, A. Verri, S. Villa, Iterative projection methods for structured sparsity regularization. Technical report, Technical Report MIT-CSAIL-TR-2009-050, MIT (2009)

    Google Scholar 

  46. F. Samaria, A. Harter, Parameterisation of a stochastic model for human face identification, in WACV 1994, pp. 138–142 (1994)

    Google Scholar 

  47. M. Shao, C. Castillo, Z. Gu, Y. Fu, Low-rank transfer subspace learning, in ICDM 2012 (2012)

    Google Scholar 

  48. T. Sim, S. Baker, M. Bsat, The cmu pose, illumination, and expression (pie) database, in FG 2002 (2002)

    Google Scholar 

  49. A. Wagner, J. Wright, A. Ganesh, Z. Zhou, H. Mobahi, Y. Ma, Toward a practical face recognition system: Robust alignment and illumination by sparse representation. IEEE TPAMI 34(2), 372–386 (2012)

    Article  Google Scholar 

  50. J. Wang, J. Yang, K. Yu, F. Lv, T. Huang, Y. Gong, Locality-constrained linear coding for image classification, in CVPR 2010 (2010)

    Google Scholar 

  51. N. Wang, J. Wang, D.-Y. Yeung, Online robust non-negative dictionary learning for visual tracking, in ICCV, pp. 657–664 (2013)

    Google Scholar 

  52. J. Wright, A. Yang, A. Ganesh, S. Sastry, Y. Ma, Robust face recognition via sparse representation. IEEE TPAMI 31(2), 210–227 (2009)

    Article  Google Scholar 

  53. Y. Xie, C. Huang, T. Song, J. Ma, J. Jing, Object co-detection via low-rank and sparse representation dictionary learning, in VCIP, pp. 1–6 (2013)

    Google Scholar 

  54. A.Y. Yang, S. Iyengar, S. Sastry, R. Bajcsy, P. Kuryloski, R. Jafari, Distributed segmentation and classification of human actions using a wearable motion sensor network, in CVPRW 2008, pp. 1–8 (2008)

    Google Scholar 

  55. A.Y. Yang, Z. Zhou, A. Ganesh, S.S. Sastry, Y. Ma, Fast l1-minimization algorithms for robust face recognition. arXiv preprint arXiv:1007.3753 (2010)

  56. J. Yang, J. Wright, T.S. Huang, Y. Ma, Image super-resolution as sparse representation of raw image patches, in CVPR (2008)

    Google Scholar 

  57. J. Yang, K. Yu, Y. Gong, T.S. Huang, Linear spatial pyramid matching using sparse coding for image classification, in CVPR, pp. 1794–1801 (2009)

    Google Scholar 

  58. M. Yang, L. Zhang, X.C. Feng, D. Zhang, Fisher discrimination dictionary learning for sparse representation, in ICCV 2011, pp. 543–550 (2011)

    Google Scholar 

  59. H. Zhang, J. Yang, Y. Zhang, N.M. Nasrabadi, T.S. Huang, Close the loop: joint blind image restoration and recognition with sparse representation prior, in ICCV 2011, pp. 770–777 (2011)

    Google Scholar 

  60. Q. Zhang, B. Li, Discriminative k-svd for dictionary learning in face recognition, in CVPR 2010, pp. 2691–2698 (2010)

    Google Scholar 

  61. Y. Zhang, Z. Jiang, L.S. Davis, Learning structured low-rank representations for image classification, in CVPR, pp. 676–683 (2013)

    Google Scholar 

  62. Z.D. Zhang, A. Ganesh, X. Liang, Y. Ma, Tilt: transform invariant low-rank textures. Int. J. Comput. Vis. 99(1), 1–24 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  63. G.Y. Zhu, S.C. Yan, Y. Ma, Image tag refinement towards low-rank, content-tag prior and error sparsity, in ACM Multimedia 2010, pp. 461–470 (2010)

    Google Scholar 

  64. L.S. Zhuang, H.Y. Gao, Z.C. Lin, Y. Ma, X. Zhang, N.H. Yu, Non-negative low rank and sparse graph for semi-supervised learning, in CVPR 2012, pp. 2328–2335 (2012)

    Google Scholar 

Download references

Acknowledgments

This research is supported in part by the NSF CNS award 1314484, Office of Naval Research award N00014-12-1-1028, Air Force Office of Scientific Research award FA9550-12-1-0201, and U.S. Army Research Office under grant number W911NF-13-1-0160.

Author information

Authors and Affiliations

  1. Department of Electrical and Computer Engineering, Northeastern University, 360 Huntington Avenue, Boston, MA, 02115, USA

    Sheng Li & Liangyue Li

  2. Department of Electrical and Computer Engineering and College of Computer and Information Science, Northeastern University, 360 Huntington Avenue, Boston, MA, 02115, USA

    Yun Fu

Authors
  1. Sheng Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Liangyue Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yun Fu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sheng Li .

Editor information

Editors and Affiliations

  1. Northeastern University, Boston, Massachusetts, USA

    Yun Fu

Rights and permissions

Reprints and permissions

Copyright information

© 2014 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Li, S., Li, L., Fu, Y. (2014). Low-Rank and Sparse Dictionary Learning. In: Fu, Y. (eds) Low-Rank and Sparse Modeling for Visual Analysis. Springer, Cham. https://doi.org/10.1007/978-3-319-12000-3_4

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-12000-3_4

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-11999-1

  • Online ISBN: 978-3-319-12000-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation