A Multiple Kernel Support Vector Machine Scheme for Simultaneous Feature Selection and Rule-Based Classification
In many applications such as bioinformatics and medical decision-making, the interpretability is important to make the model acceptable to the user and help the expert discover the novel and perhaps valuable knowledge hidden behind the data. This paper presents a novel feature selection and rule extraction method which is based on multiple kernel support vector machine (MK-SVM). This method has two outstanding properties. Firstly, the multiple kernels are described as the convex combination of the single feature basic kernels. It makes the feature selection problem in the context of SVM transformed into an ordinary multiple parameters learning problem. A 1-norm based linear programming is proposed to carry out the optimization of those parameters. Secondly, the rules are obtained in an easy way: only the support vectors necessary. It is demonstrated in theory that every support vector obtained by this method is just the vertex of the hypercube. Then a tree-like algorithm is proposed to extract the if-then rules. Three UCI datasets are used to demonstrate the effectiveness and efficiency of this approach.
KeywordsSupport Vector Machine Support Vector Feature Selection Coverage Rate Rule Extraction
Unable to display preview. Download preview PDF.
- 1.Liu, Y., Zheng, Y.F.: FS-SFS: a novel feature selection method for support vector machines. In: IEEE International Conference on Acoustics, Speech, Signal Processing, vol. 5, pp. 797–800. IEEE Computer Society Press, Los Alamitos (2004)Google Scholar
- 5.Fung, G., Sandilya, S., Baharat, R.: Rule extraction from linear support vector machines. In: 11th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 32–40. ACM Press, New York (2005)Google Scholar