Abstract
In this paper, we investigate one-class and clustering problems by using statistical learning theory. To establish a universal framework, a unsupervised learning problem with predefined threshold η is formally described and the intuitive margin is introduced. Then, one-class and clustering problems are formulated as two specific η-unsupervised problems. By defining a specific hypothesis space in η-one-class problems, the crucial minimal sphere algorithm for regular one-class problems is proved to be a maximum margin algorithm. Furthermore, some new one-class and clustering marginal algorithms can be achieved in terms of different hypothesis spaces. Since the nature in SVMs is employed successfully, the proposed algorithms have robustness, flexibility and high performance. Since the parameters in SVMs are interpretable, our unsupervised learning framework is clear and natural. To verify the reasonability of our formulation, some synthetic and real experiments are conducted. They demonstrate that the proposed framework is not only of theoretical interest, but they also has a legitimate place in the family of practical unsupervised learning techniques.
This paper is supported by National Basic Research Program of China (2004CB318103). The first author is also supported by the Excellent Youth Science and Technology Foundation of Anhui Province of China (04042069).
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Vapnik, V.: The Nature of Statistical Learning Theory. Springer, New York (1995)
Vapnik, V.: Statistical Learning Theory. Addison-Wiley, Chichester (1998)
Cristianini, N., Schawe-Taylor, J.: An Introduction to Support Vector Machines. Cambridge Univ. Press, Cambridge (2000)
Valiant, L.G.: A Theory of the Learnable. Communications of the ACM 27(11), 1134–1142 (1984)
Duda, R.O., Hart, P.E., Stork, D.G.: Pattern Classification, 2nd edn. John Wiley & Sons, Chichester (2001)
Schapire, R., Freund, Y., Bartlett, P., Sun Lee, W.: Boosting the margin: A new explanation for the effectiveness of voting methods. Ann. Statist. 26(5), 1651–1686 (1998)
Rätsch, G.: Robust boosting via convex optimization. Ph. D. thesis, University of Posdam (2001)
Kégl, B.: Principal curves: learning, design, and applications. Ph. D. Thesis, Concordia University (1999)
Smola, A.J., Mika, S., Schölkopf, B., Williamson, R.C.: Regularized principal manifolds. Journal of Machine Learning Research 1, 179–200 (2001)
Schölkopf, B., Platt, J., Shawe-Taylor, J., Smola, A.J., Williamson, R.C.: Estimating the Support of a High-Dimensional Distribution. Neural Computation 13(7), 1443–1471 (2001)
Rätsch, G., Mika, S., Schölkopf, B., Müller, K.R.: Constructing boosting algorithms from SVMs: an application to one-class classification. IEEE Transactions on Pattern Analysis and Machine Intelligence 9(4), 1184–1199 (2002)
Tax, D., Duin, R.: Support vector domain description. Pattern Recognition Letters 20, 1191–1199 (1999)
Tax, D., Duin, R.: Support vector data description. Machine Learning 54, 45–66 (2004)
Tao, Q., Wu, G., Wang, J.: A new maximum margin algorithm for one-class problems and its boosting implementation. Pattern Recognition (2005) (accepted)
Girolami, M.: Mercer Kernel-Based Clustering in Feature Space. IEEE Trans. Neural Networks 13(3), 780–784 (2002)
Ben-Hur, A., Horn, D., Siegelmann, H.T., Vapnik, V.: Support Vector Clustering. Journal of Machine Learning Research 2, 135–137 (2001)
Schölkopf, B., Smola, A.J., Williamson, R., Bartlett, P.: New support vector algorithms. Neural Computation 12, 1083–1121, 225–254 (2000)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Tao, Q., Wu, GW., Wang, FY., Wang, J. (2005). Some Marginal Learning Algorithms for Unsupervised Problems . In: Kantor, P., et al. Intelligence and Security Informatics. ISI 2005. Lecture Notes in Computer Science, vol 3495. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11427995_34
Download citation
DOI: https://doi.org/10.1007/11427995_34
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-25999-2
Online ISBN: 978-3-540-32063-0
eBook Packages: Computer ScienceComputer Science (R0)