Multiple Kernel Sparse Representation Based Classification

  • Hao Zheng
  • Fan Liu
  • Zhong Jin
Part of the Communications in Computer and Information Science book series (CCIS, volume 321)


Sparse representation based classification (SRC) has been very successful in many pattern recognition problems. Recently, some extended kernel methods have been proposed through mapping the samples from original feature space into a high dimensional feature space, and then performing the SRC in the high dimensional feature space. However they are all simple kernel methods whose kernel is not most suitable one. For addressing this question, we proposed a novel method named multiple kernel sparse representation based classification (MKSRC), which combine several possible kernels and make full of kernel information. More importantly kernel weights of MKSRC can be automatically selected. The experimental results of face databases indicated recognition performance of new method is superior to other state-of-the-art methods.


sparse representation based classification (SRC) multiple kernel face recognition kernel weight 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Duda, R.O., Hart, P.E.: Pattern Classification and Scene Analysis. Wiley, New York (1973)zbMATHGoogle Scholar
  2. 2.
    Hart, P.E.: The condensed nearest neighbor rule. IEEE Trans. Inf. Theory 16, 515–516 (1968)CrossRefGoogle Scholar
  3. 3.
    Wilson, D.L.: Asymptotic properties of nearest neighbor rules using edited data. IEEE Trans. Syst. Man Cybern. SMC-2, 408–421 (1972)zbMATHCrossRefGoogle Scholar
  4. 4.
    Aizerman, M.A., Braverman, E.M., Rozonoer, L.I.: T heoretical foundation of potential function method in pattern recognition learning. Automat. Remote Contr. 25, 821–837 (1964)MathSciNetGoogle Scholar
  5. 5.
    Scholkopf, B., Smola, A., Muller, K.R.: Nonlinear component analysis as a kernel eigenvalue problem. Neural Comput. 10, 1299–1319 (1998)CrossRefGoogle Scholar
  6. 6.
    Mike, S., Ratsch, G., Weston, J., Scholkopf, B., Muller, K.R.: Fisher discriminant analysis with kernels. In: Proceedings of the 1999 IEEE Signal Processing Society Workshop Neural Networks for Signal Processing, vol. IX, pp. 41–48 (1999)Google Scholar
  7. 7.
    Mike, S., Ratsch, G., Scholkopf, B., Smola, A., Weston, J., Muller, K.R.: Invariant feature extraction and classification in kernel spaces. In: Proceedings of the 13th Annual Neural Information Processing Systems Conference, pp. 526–532 (1999)Google Scholar
  8. 8.
    Argyriou, A., Hauser, R., Micchelli, C.A., Pontil, M.: A DC algorithm for kernel selection. In: Proc. 23rd Int. Conf. Mach., Pittsburgh, PA, pp. 41–49 (2006)Google Scholar
  9. 9.
    Argyriou, A., Micchelli, C.A., Pontil, M.: Learning convex combinations of continuously parameterized basic kernels. In: Proc. 18th Annu. Conf. Learn. Theory, Bertinoro, Italy, pp. 338–352 (2005)Google Scholar
  10. 10.
    Ong, C.S., Smola, A.J., Williamson, R.C.: Learning the kernel with hyperkernels. J. Mach. Learn. Res. 6, 1043–1071 (2005)MathSciNetzbMATHGoogle Scholar
  11. 11.
    Rakotomamonjy, A., Bach, F., Canu, S., Grandvalet, Y.: More efficiency in multiple kernel learning. In: Proc. 24th Int. Conf. Mach. Learn., Corvallis, OR, pp. 775–782 (2007)Google Scholar
  12. 12.
    Rakotomamonjy, A., Bach, F.R., Canu, S., Grandvalet, Y.: Grandvalet: Simple MKL. J. Mach. Learn. Res. 9, 2491–2521 (2008)MathSciNetzbMATHGoogle Scholar
  13. 13.
    Sonnenburg, S., Ratsch, G., Schafer, C., Scholkopf, B.: Large scale multiple kernel earning. J. Mach. Learn. Res. 7, 1531–1565 (2006)MathSciNetzbMATHGoogle Scholar
  14. 14.
    Zien, A., Ong, C.S.: Multiclass multiple kernel learning. In: Proc. 24th Int. Conf. Mach. Learn., Corvallis, OR, pp. 1191–1198 (2007)Google Scholar
  15. 15.
    Burges, C.J.C.: Simplified support vector decision rules. In: Proc.13th Int. Conf. Mach. Learn., San Mateo, CA, pp. 71–77 (1996)Google Scholar
  16. 16.
    Scholkopf, B., Smola, A.J.: Learning with Kernels. MIT Press, Cambridge (2002)Google Scholar
  17. 17.
    Nguyen, D., Ho, T.: An efficient method for simplifying support vector machines. In: Proc. 22nd Int. Conf. Mach. Learn., Bonn, Germany, pp. 617–624 (2005)Google Scholar
  18. 18.
    Wu, M., Scholkopf, B., Bakir, B.: A direct method for building sparse kernel learning algorithms. J. Mach. Learn. Res. 7, 603–624 (2006)MathSciNetzbMATHGoogle Scholar
  19. 19.
    Wu, M., Scholkopf, B., Bakir, G.: Building sparse large margin classifiers. In: Proc. 22nd Int. Conf. Mach. Learn., Bonn, Germany, pp. 996–1003 (2005)Google Scholar
  20. 20.
    Wright, J., Yang, A.Y., Ganesh, A., Sastry, S.S., Ma, Y.: Robust face recognition via sparse representation. TPAMI 31(2), 210–227 (2009)CrossRefGoogle Scholar
  21. 21.
    Yin, J., Jin, Z.: Kernel sparse representation based classification. Neurocomputing 77(1), 120–128 (2012)MathSciNetCrossRefGoogle Scholar
  22. 22.
    Gao, S., Tsang, I.W.-H., Chia, L.-T.: Kernel Sparse Representation for Image Classification and Face Recognition. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part IV. LNCS, vol. 6314, pp. 1–14. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  23. 23.
    Zhang, L., Zhou, W.-D.: Kernel sparse representation-based classifier. IEEE Transactions on Signal Processing 60(4), 1684–1695 (2012)MathSciNetCrossRefGoogle Scholar
  24. 24.
    Tibshirani, R.: Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society B 58(1), 267–288 (1996)MathSciNetzbMATHGoogle Scholar
  25. 25.
    Lanckriet, G.R.G., et al.: Learning the Kernel Matrix with Semidefinite Programming. J. Machine Learning Research 5, 27–72 (2004)MathSciNetzbMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Hao Zheng
    • 1
    • 2
  • Fan Liu
    • 1
  • Zhong Jin
    • 1
  1. 1.School of Computer Science and TechnologyNanjing University of Science and TechnologyNanjingP.R. China
  2. 2.School of Mathematics and Information TechnologyNanjing XiaoZhuang UniversityNanjingP.R. China

Personalised recommendations