Support Vector Guided Dictionary Learning

  • Sijia Cai
  • Wangmeng Zuo
  • Lei Zhang
  • Xiangchu Feng
  • Ping Wang
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8692)


Discriminative dictionary learning aims to learn a dictionary from training samples to enhance the discriminative capability of their coding vectors. Several discrimination terms have been proposed by assessing the prediction loss (e.g., logistic regression) or class separation criterion (e.g., Fisher discrimination criterion) on the coding vectors. In this paper, we provide a new insight on discriminative dictionary learning. Specifically, we formulate the discrimination term as the weighted summation of the squared distances between all pairs of coding vectors. The discrimination term in the state-of-the-art Fisher discrimination dictionary learning (FDDL) method can be explained as a special case of our model, where the weights are simply determined by the numbers of samples of each class. We then propose a parameterization method to adaptively determine the weight of each coding vector pair, which leads to a support vector guided dictionary learning (SVGDL) model. Compared with FDDL, SVGDL can adaptively assign different weights to different pairs of coding vectors. More importantly, SVGDL automatically selects only a few critical pairs to assign non-zero weights, resulting in better generalization ability for pattern recognition tasks. The experimental results on a series of benchmark databases show that SVGDL outperforms many state-of-the-art discriminative dictionary learning methods.


Dictionary learning support vector machine sparse representation Fisher discrimination 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Supplementary material

978-3-319-10593-2_41_MOESM1_ESM.pdf (46 kb)
Electronic Supplementary Material(47 KB)


  1. 1.
    Baraniuk, R.: Compressive sensing. IEEE Signal Processing Magazine (2007)Google Scholar
  2. 2.
    Mairal, J., Elad, M., Sapiro, G.: Sparse representation for color image restoration. IEEE Transactions on Image Processing (2008)Google Scholar
  3. 3.
    Yang, J., Wright, J., Huang, T.S., Ma, Y.: Image super-resolution via sparse representation. IEEE Transactions on Image Processing (2010)Google Scholar
  4. 4.
    Elhamifar, E., Vidal, R.: Sparse subspace clustering: Algorithm, theory, and applications. IEEE Transactions on Pattern Analysis and Machine Intelligence (2013)Google Scholar
  5. 5.
    Yang, J., Yu, K., Gong, Y., Huang, T.: Linear spatial pyramid matching using sparse coding for image classification. In: CVPR (2009)Google Scholar
  6. 6.
    Shabou, A., LeBorgne, H.: Locality-constrained and spatially regularized coding for scene categorization. In: CVPR (2012)Google Scholar
  7. 7.
    Olshausen, B.A., Field, D.J.: Sparse coding with an overcomplete basis set: A strategy employed by v1? Vision research (1997)Google Scholar
  8. 8.
    Candes, E.J.: The restricted isometry property and its implications for compressed sensing. Comptes Rendus Mathematique (2008)Google Scholar
  9. 9.
    Mallat, S.: A wavelet tour of signal processing (1999)Google Scholar
  10. 10.
    Aharon, M., Elad, M., Bruckstein, A.: K-svd: An algorithm for designing overcomplete dictionaries for sparse representation. IEEE Transactions on Signal Processing (2006)Google Scholar
  11. 11.
    Engan, K., Aase, S.O., Husoy, J.: Frame based signal compression using method of optimal directions (mod). In: ISCAS (1999)Google Scholar
  12. 12.
    Rubinstein, R., Peleg, T., Elad, M.: Analysis k-svd: a dictionary-learning algorithm for the analysis sparse model. IEEE Transactions on Signal Processing (2013)Google Scholar
  13. 13.
    Wright, J., Yang, A.Y., Ganesh, A., Sastry, S.S., Ma, Y.: Robust face recognition via sparse representation. IEEE Transactions on Pattern Analysis and Machine Intelligence (2009)Google Scholar
  14. 14.
    Philbin, J., Chum, O., Isard, M., Sivic, J., Zisserman, A.: Lost in quantization: Improving particular object retrieval in large scale image databases. In: CVPR (2008)Google Scholar
  15. 15.
    Sivic, J., Zisserman, A.: Video google: A text retrieval approach to object matching in videos. In: ICCV (2003)Google Scholar
  16. 16.
    Zhang, Q., Li, B.: Discriminative k-svd for dictionary learning in face recognition. In: CVPR (2010)Google Scholar
  17. 17.
    Mairal, J., Bach, F., Ponce, J., Sapiro, G., Zisserman, A., et al.: Supervised dictionary learning. In: NIPS (2008)Google Scholar
  18. 18.
    Yang, M., Zhang, D., Feng, X.: Fisher discrimination dictionary learning for sparse representation. In: ICCV (2011)Google Scholar
  19. 19.
    Jiang, Z., Lin, Z., Davis, L.S.: Learning a discriminative dictionary for sparse coding via label consistent k-svd. In: CVPR (2011)Google Scholar
  20. 20.
    Jiang, Z., Lin, Z., Davis, L.: Label consistent k-svd: learning a discriminative dictionary for recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence (2013)Google Scholar
  21. 21.
    Mairal, J., Bach, F., Ponce, J.: Task-driven dictionary learning. IEEE Transactions on Pattern Analysis and Machine Intelligence (2012)Google Scholar
  22. 22.
    Ramirez, I., Sprechmann, P., Sapiro, G.: Classification and clustering via dictionary learning with structured incoherence and shared features. In: CVPR (2010)Google Scholar
  23. 23.
    Gao, S., Tsang, I., Ma, Y.: Learning category-specific dictionary and shared dictionary for fine-grained image categorization. IEEE Transactions on Image Processing (2013)Google Scholar
  24. 24.
    Zhang, W., Surve, A., Fern, X., Dietterich, T.: Learning non-redundant codebooks for classifying complex objects. In: ICML (2009)Google Scholar
  25. 25.
    Wang, Z., Yang, J., Nasrabadi, N., Huang, T.: Look into sparse representation-based classification: A margin-based perspective. In: ICCV (2013)Google Scholar
  26. 26.
    Zheng, M., Bu, J., Chen, C., Wang, C., Zhang, L., Qiu, G., Cai, D.: Graph regularized sparse coding for image representation. IEEE Transactions on Image Processing (2011)Google Scholar
  27. 27.
    Wang, J., Yang, J., Yu, K., Lv, F., Huang, T., Gong, Y.: Locality-constrained linear coding for image classification. In: CVPR (2010)Google Scholar
  28. 28.
    Gao, S., Tsang, I.W., Chia, L.T., Zhao, P.: Local features are not lonely–laplacian sparse coding for image classification. In: CVPR (2010)Google Scholar
  29. 29.
    Gao, S., Tsang, I.H., Chia, L.T.: Laplacian sparse coding, hypergraph laplacian sparse coding, and applications. IEEE Transactions on Pattern Analysis and Machine Intelligence (2013)Google Scholar
  30. 30.
    Burges, C.J.: A tutorial on support vector machines for pattern recognition. Data Mining and Knowledge Discovery (1998)Google Scholar
  31. 31.
    Rigamonti, R., Brown, M.A., Lepetit, V.: Are sparse representations really relevant for image classification? In: CVPR (2011)Google Scholar
  32. 32.
    Zhang, D., Yang, M., Feng, X.: Sparse representation or collaborative representation: Which helps face recognition? In: ICCV (2011)Google Scholar
  33. 33.
    Mehta, N., Gray, A.G.: Sparsity-based generalization bounds for predictive sparse coding. In: ICML (2013)Google Scholar
  34. 34.
    Platt, J., et al.: Sequential minimal optimization: A fast algorithm for training support vector machines (1998)Google Scholar
  35. 35.
    Lee, H., Battle, A., Raina, R., Ng, A.Y.: Efficient sparse coding algorithms. In: NIPS (2007)Google Scholar
  36. 36.
    Lee, K.C., Ho, J., Kriegman, D.: Acquiring linear subspaces for face recognition under variable lighting. IEEE Transactions on Pattern Analysis and Machine Intelligence (2005)Google Scholar
  37. 37.
    Martınez, A., Benavente, R.: The ar face database. CVC Technical Report (1998)Google Scholar
  38. 38.
    Gross, R., Matthews, I., Cohn, J., Kanade, T., Baker, S.: Multi-pie. Image and Vision Computing (2010)Google Scholar
  39. 39.
    Fei-Fei, L., Fergus, R., Perona, P.: Learning generative visual models from few training examples: An incremental bayesian approach tested on 101 object categories. Computer Vision and Image Understanding (2007)Google Scholar
  40. 40.
    Rodriguez, M., Ahmed, J., Shah, M.: Action mach a spatio-temporal maximum average correlation height filter for action recognition. In: Indian Conference on Computer Vision, Graphics and Image Processing (2008)Google Scholar
  41. 41.
    Qiu, Q., Jiang, Z., Chellappa, R.: Sparse dictionary-based representation and recognition of action attributes. In: ICCV (2011)Google Scholar
  42. 42.
    Yao, A., Gall, J., Van Gool, L.: A hough transform-based voting framework for action recognition. In: CVPR (2010)Google Scholar
  43. 43.
    Sadanand, S., Corso, J.J.: Action bank: A high-level representation of activity in video. In: CVPR (2012)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Sijia Cai
    • 1
    • 3
  • Wangmeng Zuo
    • 2
  • Lei Zhang
    • 3
  • Xiangchu Feng
    • 4
  • Ping Wang
    • 1
  1. 1.School of ScienceTianjin UniversityChina
  2. 2.School of Computer Science and TechnologyHarbin Institute of TechnologyChina
  3. 3.Dept. of ComputingThe Hong Kong Polytechnic UniversityChina
  4. 4.Dept. of Applied MathematicsXidian UniversityChina

Personalised recommendations