A Sparse Support Vector Machine Classifier with Nonparametric Discriminants

  • Naimul Mefraz Khan
  • Riadh Ksantini
  • Imran Shafiq Ahmad
  • Ling Guan
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7553)


This paper introduces a novel Sparse Support Vector Machine model with Kernel Nonparametric Discriminants (SSVMKND) which combines data distribution information from two classifiers, namely, the Kernel Support Vector Machine (KSVM) and the Kernel Nonparametric Discriminant (KND). It is a convex quadratic optimization problem with one global solution, so it can be estimated efficiently with the help of numerical methods. It can also be reduced to the classical KSVM model, and existing SVM programs can be used for easy implementation. We show that our method provides a sparse solution through the Bayesian interpretation. This sparsity can be used by existing sparse classification algorithms to obtain better computational efficiency. The experimental results on real-world datasets and face recognition applications show that the proposed SSVMKND model improves the classification accuracy over other classifiers and also provides sparser solution.


Nonparametric Discriminant Analysis Support Vector Machines Sparsity Gaussian Prior kernel face recognition 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Belhumeur, P., Hespanha, J., Kriegman, D.: Eigenfaces vs. fisherfaces: recognition using class specific linear projection. IEEE Transactions on Pattern Analysis and Machine Intelligence 19, 711–720 (1997)CrossRefGoogle Scholar
  2. 2.
    Camps-Valls, G., Bruzzone, L.: Kernel-based methods for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing 43(6), 1351–1362 (2005)CrossRefGoogle Scholar
  3. 3.
    Cristianini, M., Shawe-Taylor, J.: An Introduction to Support Vector Machines. Cambridge University Press, Cambridge (2000)Google Scholar
  4. 4.
    Fukunaga, K.: Introduction to Statistical Pattern Recognition, 2nd edn. Academic Press (2000)Google Scholar
  5. 5.
    Georghiades, A.: Yale face database (1997)Google Scholar
  6. 6.
    Horn, R., Charles, R.: Matrix Analysis. Cambridge University Press (1990)Google Scholar
  7. 7.
    Khan, N., Ksantini, R., Ahmad, I., Boufama, B.: A novel SVM+NDA model for classification with an application to face recognition. Pattern Recognition 45(1), 66–79 (2012)CrossRefzbMATHGoogle Scholar
  8. 8.
    Ksantini, R., Ahmad, I., Boufama, B., Khan, N.: A new combined KSVM and KFD model for classification and recognition. In: Fifth International Conference on Digital Information Management, pp. 188–193 (2010)Google Scholar
  9. 9.
    Lee, C., Landgrebe, D.: Feature Extraction Based on Decision Boundaries. IEEE Transactions on Pattern Analysis and Machine Intelligence 15(4), 388–400 (1993)CrossRefGoogle Scholar
  10. 10.
    Lyons, M., Budynek, J., Akamatsu, S.: Automatic classification of single facial images. IEEE Transactions on Pattern Analysis and Machine Intelligence 21(12), 1357–1362 (1999)CrossRefGoogle Scholar
  11. 11.
    Mika, S., Ratsch, G., Weston, J., Scholkopf, B., Mullers, K.: Fisher Discriminant Analysis with Kernels. In: Neural Networks for Signal Processing, pp. 41–48 (August 1999)Google Scholar
  12. 12.
    Ratsch, G., Onoda, T., Muller, K.: Soft Margins for Adaboost. Machine Learning 42(3), 287–320 (2000)CrossRefGoogle Scholar
  13. 13.
    Sollich, P.: Bayesian Methods for Support Vector Machines: Evidence and Predictive Class Probabilities. Machine Learning 46(1-3), 21–52 (2002)CrossRefzbMATHGoogle Scholar
  14. 14.
    Vapnik, V.: Statistical Learning Theory. John Wiley & Sons (1998)Google Scholar
  15. 15.
    Xiong, T., Cherkassky, V.: A Combined SVM and LDA Approach for Classification. In: International Joint Conference on Neural Networks, pp. 1455–1459 (2005)Google Scholar
  16. 16.
    You, D., Hamsici, O.C., Martinez, A.M.: Kernel Optimization in Discriminant Analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence 33(3), 631–638 (2011)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Naimul Mefraz Khan
    • 1
  • Riadh Ksantini
    • 2
  • Imran Shafiq Ahmad
    • 2
  • Ling Guan
    • 1
  1. 1.Department of Electrical and Computer EngineeringRyerson UniversityCanada
  2. 2.School of Computer ScienceUniversity of WindsorCanada

Personalised recommendations