Kernel Dictionary Learning

  • Bogdan Dumitrescu
  • Paul Irofti


Sparse representations are linear by construction, a fact that can hinder their use in classification problems. Building vectors of characteristics from the signals to be classified can overcome the difficulties and is automated by employing kernels, which are functions that quantify the similarities between two vectors. DL can be extended to kernel form by assuming a specific form of the dictionary. DL algorithms have the usual form, comprising sparse coding and dictionary update. We present the kernel versions of OMP and of the most common update algorithms: MOD, SGK, AK-SVD, and K-SVD. The kernel methods use many operations involving a square kernel matrix whose size is equal to the number of signals; hence, their complexities are significantly higher than those of the standard methods. We present two ideas for reducing the size of the problem, the most prominent being that based on Nyström sampling. Finally, we show how kernel DL can be adapted to classification methods involving sparse representations, in particular SRC and discriminative DL.


  1. 1.
    V. Abrol, P. Sharma, A.K. Sao, Greedy dictionary learning for kernel sparse representation based classifier. Pattern Recogn. Lett. 78, 64–69 (2016)CrossRefGoogle Scholar
  2. 24.
    E.J. Candès, L. Wakin, An introduction to compressive sampling. IEEE Signal Process. Mag. 25(2), 21–30 (2008)CrossRefGoogle Scholar
  3. 71.
    A.K. Farahat, A. Elgohary, A. Ghodsi, M.S. Kamel, Greedy column subset selection for large-scale data sets. Springer Knowl. Inf. Syst. 45(1), 1–34 (2015)Google Scholar
  4. 75.
    M.J. Gangeh, A. Ghodsi, M.S. Kamel, Kernelized supervised dictionary learning. IEEE Trans. Signal Process. 61(19), 4753–4767 (2013)MathSciNetCrossRefGoogle Scholar
  5. 76.
    S. Gao, I.W.H. Tsang, L.T. Chia, Sparse representation with kernels. IEEE Trans. Image Process. 22(2), 423–434 (2013)Google Scholar
  6. 79.
    A. Golts, M. Elad, Linearized kernel dictionary learning. IEEE J. Sel. Top. Signal Process. 10(4), 726–739 (2016)CrossRefGoogle Scholar
  7. 81.
    M. Gönen, E. Alpaydin, Multiple kernel learning algorithms. J. Mach. Learn. Res. 12, 2211–2268 (2011)Google Scholar
  8. 94.
    T. Hofmann, B. Schölkopf, A.J. Smola, Kernel methods in machine learning. Ann. Stat. 36(3), 1171–1220 (2008)MathSciNetCrossRefGoogle Scholar
  9. 109.
    S.J. Kim, Online kernel dictionary learning, in GlobalSIP, Orlando, December 2015Google Scholar
  10. 114.
    S. Kumar, M. Mohri, A. Talwalkar, Sampling methods for the Nyström method. J. Mach. Learn. Res. 13, 981–1006 (2012)MathSciNetzbMATHGoogle Scholar
  11. 140.
    V.H. Nguyen, V.M. Patel, N.M. Nasrabadi, R. Chellappa, Design of non-linear kernel dictionaries for object recognition. IEEE Trans. Image Process. 22(12), 5123–5135 (2013)MathSciNetCrossRefGoogle Scholar
  12. 175.
    A. Shrivastava, V.M. Patel, R. Chellappa, Multiple kernel learning for sparse representation-based classification. IEEE Trans. Image Process. 23(7), 3013–3024 (2014)MathSciNetCrossRefGoogle Scholar
  13. 186.
    J.J. Thiagarajan, K.N. Ramamurthy, A. Spanias, Multiple kernel sparse representations for supervised and unsupervised learning. IEEE Trans. Image Process. 23(7), 2905–2915 (2014)MathSciNetCrossRefGoogle Scholar
  14. 195.
    P. Vincent, Y. Bengio, Kernel matching pursuit. Mach. Learn. 48, 165–187 (2002)CrossRefGoogle Scholar
  15. 217.
    J. Yin, Z. Liu, Z. Jin, W. Yang, Kernel sparse representation based classification. Neurocomputing 77, 120–128 (2012)CrossRefGoogle Scholar
  16. 222.
    L. Zhang, W.D. Zhou, P.C. Chang, J. Liu, Z. Yan, T. Wang, F.Z. Li, Kernel sparse representation-based classifier. IEEE Trans. Signal Process. 60(4), 1684–1695 (2012)MathSciNetCrossRefGoogle Scholar
  17. 225.
    G. Zhang, H. Sun, G. Xia, Q. Sun, Multiple kernel sparse representation-based orthogonal discriminative projection and its cost-sensitive extension. IEEE Trans. Image Process. 25(9), 4271–4285 (2016)MathSciNetGoogle Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  • Bogdan Dumitrescu
    • 1
  • Paul Irofti
    • 2
  1. 1.Department of Automatic Control and Systems Engineering, Faculty of Automatic Control and ComputersUniversity Politehnica of BucharestBucharestRomania
  2. 2.Department of Computer Science, Faculty of Mathematics and Computer ScienceUniversity of BucharestBucharestRomania

Personalised recommendations