Abstract
In this article we construct a maximal margin classification algorithm for arbitrary metric spaces. At first we show that the Support Vector Machine (SVM) is a maximal margin algorithm for the class of metric spaces where the negative squared distance is conditionally positive definite (CPD). This means that the metric space can be isometrically embedded into a Hilbert space, where one performs linear maximal margin separation. We will show that the solution only depends on the metric, but not on the kernel. Following the framework we develop for the SVM, we construct an algorithm for maximal margin classification in arbitrary metric spaces. The main difference compared with SVM is that we no longer embed isometrically into a Hilbert space, but a Banach space. We further give an estimate of the capacity of the function class involved in this algorithm via Rademacher averages. We recover an algorithm of Graepel et al. [6].
Chapter PDF
References
Bartlett, P.L., Mendelson, S.: Rademacher and Gaussian Complexities: Risk Bounds and Structural Results. JLMR 3, 463–482 (2002)
Bennett, K.P., Bredensteiner, E.J.: Duality and Geometry in SVM classifiers. In: Proceedings of the Seventeenth International Conference on Machine Learning, pp. 57–64 (2000)
Berg, C., Cristensen, J.P.R., Ressel, P.: Harmonic Analysis on Semigroups. Springer, New York (1984)
Cucker, F., Smale, S.: On the Mathematical Foundations of Learning. Bull. Amer. Math. Soc. 39, 1–49 (2002)
Dudley, R.M.: Universal Donsker Classes and Metric Entropy. Ann. Prob. 15, 1306–1326 (1987)
Graepel, T., Herbrich, R., Schölkopf, B., Smola, A., Bartlett, P., Müller, K.R., Obermayer, K., Williamson, R.: Classification on proximity data with LP-machines. In: International Conference on Artificial Neural Networks, pp. 304–309 (1999)
Pekalska, E., Paclik, P., Duin, R.P.W.: A Generalized Kernel Approach to Dissimilarity-based Classification. Journal of Machine Learning Research 2, 175–211 (2001)
Rudin, W.: Functional Analysis. McGraw Hill, New York (1991)
Schoenberg, I.J.: Metric Spaces and Positive Definite Functions. TAMS 44, 522–536 (1938)
Schölkopf, B.: The Kernel Trick for Distances, Neural Information Processing Systems (NIPS), vol. 13 (2000)
Schölkopf, B., Smola, A.J.: Learning with Kernels. MIT Press, Cambridge (2002)
Zhou, D., Xiao, B., Zhou, H., Dai, R.: Global Geometry of SVM Classifiers, Technical Report 30-5-02, AI Lab, Institute of Automation, Chinese Academy of Sciences (2002)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2003 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Hein, M., Bousquet, O. (2003). Maximal Margin Classification for Metric Spaces. In: Schölkopf, B., Warmuth, M.K. (eds) Learning Theory and Kernel Machines. Lecture Notes in Computer Science(), vol 2777. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-45167-9_7
Download citation
DOI: https://doi.org/10.1007/978-3-540-45167-9_7
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-40720-1
Online ISBN: 978-3-540-45167-9
eBook Packages: Springer Book Archive