Abstract
This paper introduces a novel Sparse Support Vector Machine model with Kernel Nonparametric Discriminants (SSVMKND) which combines data distribution information from two classifiers, namely, the Kernel Support Vector Machine (KSVM) and the Kernel Nonparametric Discriminant (KND). It is a convex quadratic optimization problem with one global solution, so it can be estimated efficiently with the help of numerical methods. It can also be reduced to the classical KSVM model, and existing SVM programs can be used for easy implementation. We show that our method provides a sparse solution through the Bayesian interpretation. This sparsity can be used by existing sparse classification algorithms to obtain better computational efficiency. The experimental results on real-world datasets and face recognition applications show that the proposed SSVMKND model improves the classification accuracy over other classifiers and also provides sparser solution.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Belhumeur, P., Hespanha, J., Kriegman, D.: Eigenfaces vs. fisherfaces: recognition using class specific linear projection. IEEE Transactions on Pattern Analysis and Machine Intelligence 19, 711–720 (1997)
Camps-Valls, G., Bruzzone, L.: Kernel-based methods for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing 43(6), 1351–1362 (2005)
Cristianini, M., Shawe-Taylor, J.: An Introduction to Support Vector Machines. Cambridge University Press, Cambridge (2000)
Fukunaga, K.: Introduction to Statistical Pattern Recognition, 2nd edn. Academic Press (2000)
Georghiades, A.: Yale face database (1997)
Horn, R., Charles, R.: Matrix Analysis. Cambridge University Press (1990)
Khan, N., Ksantini, R., Ahmad, I., Boufama, B.: A novel SVM+NDA model for classification with an application to face recognition. Pattern Recognition 45(1), 66–79 (2012)
Ksantini, R., Ahmad, I., Boufama, B., Khan, N.: A new combined KSVM and KFD model for classification and recognition. In: Fifth International Conference on Digital Information Management, pp. 188–193 (2010)
Lee, C., Landgrebe, D.: Feature Extraction Based on Decision Boundaries. IEEE Transactions on Pattern Analysis and Machine Intelligence 15(4), 388–400 (1993)
Lyons, M., Budynek, J., Akamatsu, S.: Automatic classification of single facial images. IEEE Transactions on Pattern Analysis and Machine Intelligence 21(12), 1357–1362 (1999)
Mika, S., Ratsch, G., Weston, J., Scholkopf, B., Mullers, K.: Fisher Discriminant Analysis with Kernels. In: Neural Networks for Signal Processing, pp. 41–48 (August 1999)
Ratsch, G., Onoda, T., Muller, K.: Soft Margins for Adaboost. Machine Learning 42(3), 287–320 (2000)
Sollich, P.: Bayesian Methods for Support Vector Machines: Evidence and Predictive Class Probabilities. Machine Learning 46(1-3), 21–52 (2002)
Vapnik, V.: Statistical Learning Theory. John Wiley & Sons (1998)
Xiong, T., Cherkassky, V.: A Combined SVM and LDA Approach for Classification. In: International Joint Conference on Neural Networks, pp. 1455–1459 (2005)
You, D., Hamsici, O.C., Martinez, A.M.: Kernel Optimization in Discriminant Analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence 33(3), 631–638 (2011)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Khan, N.M., Ksantini, R., Ahmad, I.S., Guan, L. (2012). A Sparse Support Vector Machine Classifier with Nonparametric Discriminants. In: Villa, A.E.P., Duch, W., Érdi, P., Masulli, F., Palm, G. (eds) Artificial Neural Networks and Machine Learning – ICANN 2012. ICANN 2012. Lecture Notes in Computer Science, vol 7553. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-33266-1_41
Download citation
DOI: https://doi.org/10.1007/978-3-642-33266-1_41
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-33265-4
Online ISBN: 978-3-642-33266-1
eBook Packages: Computer ScienceComputer Science (R0)