Abstract
In this paper, a fuzzy semi-supervised support vector machines (FSS-SVM) algorithm is proposed. It tries to overcome the need for a large labelled training set to learn accurate classifiers. For this, it uses both labelled and unlabelled data for training. It also modulates the effect of the unlabelled data in the learning process. Empirical evaluations showed that by additionally using unlabelled data, FSS-SVM requires less labelled training data than its supervised version, support vector machines, to achieve the same level of classification performance. Also, the incorporated fuzzy membership values of the unlabelled training patterns in the learning process have positively influenced the classification performance in comparison with its crisp variant.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Aamodt, A., Plaza, E.: Case-Based Reasoning. In: Proc. MLnet Summer School on Machine Learning and Knowledge Acquisition, pp. 1–58 (1994)
Balcan, M.F., Blum, A.: An augmented PAC model for semi-supervised learning (2005) (manuscript)
Baluja, S.: Probabilistic modeling for face orientation discrimination: Learning from labeled and unlabeled data. Advances in Neural Information Processing Systems 11, 854–860 (1998)
Bennett, K., Demiriz, A.: Semi-supervised support vector machines. Advances in Neural Information Processing Systems 11, 368–374 (1998)
Bensaid, A.M., Hall, L.O., et al.: Partially supervised clustering for image segmentation. Pattern Recognition 29(5), 859–871 (1996)
Bezdek, J.C.: Pattern Recognition with Fuzzy Objective Function Algorithms. Kluwer Academic Publishers, Norwell (1981)
Blum, A., Mitchell, T.: Combining labeled and unlabeled data with cotraining. In: Proceedings of the Eleventh Annual Conference on Computational Learning Theory, pp. 92–100 (1998)
Bobrowski, L., Bezdek, J.C.: C-means clustering with the l l and l8 Norms. IEEE Transactions on, Systems, Man and Cybernetics 21(3), 545–554 (1991)
Booker, L.B., Goldberg, D.E., et al.: Classifier systems and genetic algorithms. Artificial Intelligence 40(1-3), 235–282 (1989)
Cao, L.J., Lee, H.P., et al.: Modified support vector novelty detector using training data with outliers. Pattern Recognition Letters 24(14), 2479–2487 (2003)
Chapelle, O., Sindhwani, V., et al.: Branch and Bound for Semi- Supervised Support Vector Machines. In: Advances in Neural Information Processing Systems, (NIPS) (2006)
Cheeseman, P., Freeman, D., et al.: Autoclass: A Bayesian classiciation system. In: Proceedings of the Fifth International Conference on Machine Learning, pp. 54–64 (1988)
Collins, M., Singer, Y.: Unsupervised models for named entity classification. In: Proceedings of the Joint SIGDAT Conference on Empirical Methods in Natural Language Processing and Very Large Corpora, pp. 189–196 (1999)
Cover, T., Hart, P.: Nearest neighbor pattern classification. IEEE Transactions on Information Theory 13(1), 21–27 (1967)
Cozman, F.G., Cohen, I., et al.: Semi-Supervised Learning of Mixture Models and Bayesian Networks. In: Proceedings of the Twentieth International Conference of Machine Learning (2003)
Dagan, I., Engelson, S.P.: Committee-based sampling for training probabilistic classifiers. In: Proceedings of the Twelfth International Conference on Machine Learning, pp. 150–157 (1995)
Day, W.H.E., Edelsbrunner, H.: Efficient algorithms for agglomerative hierarchical clustering methods. Journal of Classification 1(1), 7–24 (1984)
Dempster, A.P., Laird, N.M., et al.: Maximum Likelihood from Incomplete Data via the EM Algorithm. Journal of the Royal Statistical Society. Series B (Methodological) 39(1), 1–38 (1977)
Ester, M., Kriegel, H.P., et al.: A Density-Based Algorithm for Discovering Clusters in Large Spatial Databases with Noise. In: Proc. 2nd Int. Conf. on Knowledge Discovery and Data Mining, pp. 226–231. AAAI Press, Portland (1996)
Freund, Y., Seung, H.S., et al.: Selective Sampling Using the Query by Committee Algorithm. Machine Learning 28(2), 133–168 (1997)
Fung, G., Mangasarian, O.: Semi-supervised support vector machines for unlabeled data classification (Technical Report 99-05). Data mining Institute, University of Wisconsin at Madison, Madison, WI (1999)
Guyon, I., Matic, N., et al.: Discovering informative patterns and data cleaning. Advances in Knowledge Discovery and Data Mining Table of Contents, 181–203 (1996)
Huang, W., Lippmann, R.: Neural net and traditional classifiers. Neural Information Processing Systems, 387–396 (1988)
Joachims, T.: Making large-Scale SVM Learning Practical. In: Scoelkopf, B., Burges, C., Smola, A. (eds.) Advances in Kernel Methods-Support Vector Learning (1999)
Joachims, T.: Transductive inference for text classification using support vector machines. In: Proceedings of the Sixteenth International Conference on Machine Learning, pp. 200–209 (1999)
Jones, R.: Learning to Extract Entities from Labeled and Unlabeled Text, University of Utah (2005)
Kaelbling, L.P., Littman, M.L., et al.: Reinforcement Learning: A Survey. Arxiv preprint cs.AI/9605103 (1996)
Kockelkorn, M., Luneburg, A., et al.: Using transduction and multi-view learning to answer emails. In: Proceedings of the European Conference on Principle and Practice of Knowledge Discovery in Databases, pp. 266–277 (2003)
Kohonen, T.: Self-organized formation of topologically correct feature maps. Biological Cybernetics 43(1), 59–69 (1982)
Koller, D., Sahami, M.: Hierarchically classifying documents using very few words. In: Proceedings of the Fourteenth International Conference on Machine Learning, pp. 170–178 (1997)
Langley, P., Simon, H.A.: Applications of machine learning and rule induction. Communications of the ACM 38(11), 54–64 (1995)
Letters, P.R.: Training algorithms for fuzzy support vector machines with noisy data. Pattern Recognition Letters 25(14), 1647–1656 (2004)
Liere, R., Tadepalli, P.: Active learning with committees for text categorization. In: Proceedings of the Fourteenth National Conference on Artificial Intelligence, pp. 591–596 (1997)
Lin, C.F., Wang, S.D.: Fuzzy support vector machines. IEEE Transactions on Neural Networks 13(2), 464–471 (2002)
Littlestone, N.: Learning quickly when irrelevant attributes abound: A new linear-threshold algorithm. Machine Learning 2(4), 285–318 (1988)
McCallum, A., Nigam, K.: Employing EM in pool-based active learning for text classification. In: Proceedings of ICML 1998, 15th International Conference on Machine Learning, pp. 350–358 (1998)
Mitchell, T.: The role of unlabeled data in supervised learning. In: Proceedings of the Sixth International Colloquium on Cognitive Science (1999)
Mladenic, D.: Learning word normalization using word suffix and context from unlabeled data. In: Proceedings of the Nineteenth International Conference on Machine Learning Table of Contents, pp. 427–434 (2002)
Nigam, K., Ghani, R.: Analyzing the effectiveness and applicability of co-training. In: Proceedings of the Ninth International Conference on Information and Knowledge Management, pp. 86–93 (2000)
Nigam, K., McCallum, A.K., et al.: Text Classification from Labeled and Unlabeled Documents using EM. Machine Learning 39(2), 103–134 (2000)
Quinlan, J.R.: Induction of decision trees. Machine Learning 1(1), 81–106 (1986)
Rosenberg, C., Hebert, M., et al.: Semi-supervised selftraining of object detection models. In: Seventh IEEE Workshop on Applications of Computer Vision, vol. 1, pp. 29–36 (2005)
Schohn, G., Cohn, D.: Less is more: Active learning with support vector machines. In: Proceedings of the Seventeenth International Conference on Machine Learning, vol. 282, pp. 285–286 (2000)
Seung, H.S., Opper, M., et al.: Query by committee. In: Proceedings of the Fifth Annual Workshop on Computational Learning Theory, pp. 287–294 (1992)
Sheng-de Wang, C.L.: Training algorithms for fuzzy support vector machines with noisy data. In: IEEE 13th Workshop on, Neural Networks for Signal Processing NNSP 2003, pp. 517–526 (2003)
Tong, S., Koller, D.: Support vector machine active learning with applications to text classification. The Journal of Machine Learning Research 2, 45–66 (2002)
Vapnik, V., Golowich, S., et al.: Support vector method for function approximation, regression estimation, and signal processing. Advances in Neural Information Processing Systems 9, 281–287 (1997)
Vapnik, V.N.: Statistical learning theory. Wiley, New York (1998)
Wang, W., Yang, J., et al.: STING: A Statistical Information Grid Approach to Spatial Data Mining. In: Proceedings of the 23rd International Conference on Very Large Data Bases, pp. 186–195 (1997)
Xu, L., Schuurmans, D.: Unsupervised and Semi-supervised Multiclass Support Vector Machines. In: AAAI 2005, Pittsburgh, PA, pp. 904–910 (2005)
Yarowsky, D.: Unsupervised word sense disambiguation rivalling supervised methods. In: Proceedings of the 33rd conference on Association for Computational Linguistics, pp. 189–196 (1995)
Zhang, X.: Using class-center vectors to build support vector machines. In: Proceedings of the 1999 IEEE Signal Processing Society Workshop on Neural Networks for Signal Processing IX, pp. 3–11 (1999)
Zhou, Z.-H., Zhan, D.-C., Yang, Q.: Semi-supervised learning with very few labeled training examples. In: Twenty-Second AAAI Conference on Artificial Intelligence, (AAAI 2007) (2007)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Benbrahim, H. (2011). Fuzzy Semi-supervised Support Vector Machines. In: Perner, P. (eds) Machine Learning and Data Mining in Pattern Recognition. MLDM 2011. Lecture Notes in Computer Science(), vol 6871. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-23199-5_10
Download citation
DOI: https://doi.org/10.1007/978-3-642-23199-5_10
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-23198-8
Online ISBN: 978-3-642-23199-5
eBook Packages: Computer ScienceComputer Science (R0)