Abstract
Prior knowledge in the form of multiple polyhedral sets, each belonging to one of two categories, is introduced into a reformulation of a nonlinear kernel support vector machine (SVM) classifier. The resulting formulation leads to a linear program that can be solved efficiently. This extends, in a rather unobvious fashion, previous work [3] that incorporated similar prior knowledge into a linear SVM classifier. Numerical tests on standard-type test problems, such as exclusive-or prior knowledge sets and a checkerboard with 16 points and prior knowledge instead of the usual 1000 points, show the effectiveness of the proposed approach in generating sharp nonlinear classifiers based mostly or totally on prior knowledge.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Bradley, P.S., Mangasarian, O.L.: Feature selection via concave minimization and support vector machines. In: Shavlik, J. (ed.) Machine Learning Proceedings of the 15th International Conference (ICML 1998), San Francisco, California, pp. 82–90. Morgan Kaufmann, San Francisco (1998), ftp://ftp.cs.wisc.edu/mathprog/techreports/98-03.ps
Cherkassky, V., Mulier, F.: Learning from Data - Concepts, Theory and Methods. John Wiley & Sons, New York (1998)
Fung, G., Mangasarian, O.L., Shavlik, J.: Knowledge-based support vector machine classifiers. Technical Report 01-09, Data Mining Institute, Computer Sciences Department, University of Wisconsin, Madison, Wisconsin (November 2001), ftp://ftp.cs.wisc.edu/pub/dmi/tech-reports/01-09.ps ; NIPS 2002 Proceedings (2002) (to appear)
Girosi, F., Chan, N.: Prior knowledge and the creation of “virtual” examples for RBF networks. In: Neural networks for signal processing, Proceedings of the 1995 IEEE-SP Workshop, New York, pp. 201–210 (1995), IEEE Signal Processing Society
Ho, T.K., Kleinberg, E.M.: Building projectable classifiers of arbitrary complexity. In: Proceedings of the 13th International Conference on Pattern Recognition, Vienna, Austria, pp. 880–885 (1996), http://cm.belllabs.com/who/tkh/pubs.html , Checker dataset at ftp://ftp.cs.wisc.edu/mathprog/cpo-dataset/machine-learn/checker
Ho, T.K., Kleinberg, E.M.: Checkerboard dataset (1996), http://www.cs.wisc.edu/math-prog/mpml.html
Lee, Y.-J., Mangasarian, O.L.: RSVM: Reduced support vector machines. Technical Report 00-07, Data Mining Institute, Computer Sciences Department, University of Wisconsin, Madison, Wisconsin (July 2000); Proceedings of the First SIAM International Conference on Data Mining, CDROM Proceedings, Chicago, April 5-7 (2001), ftp://ftp.cs.wisc.edu/pub/dmi/tech-reports/00-07.ps
Lee, Y.-J., Mangasarian, O.L.: SSVM: A smooth support vector machine. Computational Optimization and Applications, 20:5–22, Data Mining Institute, University of Wisconsin, Technical Report 99-03. (2001), ftp://ftp.cs.wisc.edu/pub/dmi/tech-reports/99-03.ps
Mangasarian, O.L.: Nonlinear Programming. SIAM, Philadelphia (1994)
Mangasarian, O.L.: Generalized support vector machines. In: Smola, A., Bartlett, P., Scholkopf, B., Schuurmans, D. (eds.) Advances in Large Margin Classifiers, Cambridge, MA, pp. 135–146. MIT Press, Cambridge (2000), ftp://ftp.cs.wisc.edu/mathprog/tech-reports/98-14.ps
Schölkopf, B., Simard, P., Smola, A., Vapnik, V.: Prior knowledge in support vector kernels. In: Jordan, M., Kearns, M., Solla, S. (eds.) Advances in Neural Information Processing Systems, Cambridge, MA, vol. 10, pp. 640–646. MIT Press, Cambridge (1998)
Smola, A., Scholkopf, B.: Learning with Kernels. MIT Press, Cambridge (2002)
Towell, G.G., Shavlik, J.W.: Knowledge-based artificial neural networks. Artificial Intelligence 70, 119–165 (1994)
Towell, G.G., Shavlik, J.W., Noordewier, M.: Refinement of approximate domain theories by knowledge-based artificial neural networks. In: Proceedings of the 8th National Conference on Artificial Intelligence (AAAI 1990), pp. 861– 866 (1990)
Vapnik, V.N.: The Nature of Statistical Learning Theory, 2nd edn. Springer, New York (2000)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2003 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Fung, G.M., Mangasarian, O.L., Shavlik, J.W. (2003). Knowledge-Based Nonlinear Kernel Classifiers. In: Schölkopf, B., Warmuth, M.K. (eds) Learning Theory and Kernel Machines. Lecture Notes in Computer Science(), vol 2777. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-45167-9_9
Download citation
DOI: https://doi.org/10.1007/978-3-540-45167-9_9
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-40720-1
Online ISBN: 978-3-540-45167-9
eBook Packages: Springer Book Archive