A Simple Hybrid Method for Semi-Supervised Learning

  • Hernán C. Ahumada
  • Pablo M. Granitto
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7441)


We introduce and describe the Hybrid Semi-Supervised Method (HSSM) for learning. This is the first hybrid method aimed to solve problems with both labeled and unlabeled data. The new method uses an unsupervised stage in order to decompose the full problem into a set of simpler subproblems. HSSM applies simple stopping criteria during the unsupervised stage, which allows the method to concentrate on the difficult portions of the original problem. The new algorithm also makes use of a simple strategy to select at each subproblem a small subset of unlabeled samples that are relevant to modify the decision surface. To this end, HSSM trains a linear SVM on the available labeled samples, and selects the unlabeled samples that lie within the margin of the trained SVM. We evaluated the new method using a previously introduced setup, which includes datasets with very different properties. Overall, the error levels produced by the new HSSM are similar to other SSL methods, but HSSM is shown to be more efficient than all previous methods, using only a small fraction of the available unlabeled data.


Semi-supervised learning Hybrid methods Classification 


  1. 1.
    Ahumada, H.C., Grinblat, G.L., Granitto, P.M.: Unsupervized Data-Driven Partitioning of Multiclass Problems. In: Honkela, T. (ed.) ICANN 2011, Part I. LNCS, vol. 6791, pp. 117–125. Springer, Heidelberg (2011)Google Scholar
  2. 2.
    Blum, A., Chawla, S.: Learning from labeled and unlabeled data using graph mincuts. In: ICML 18, pp. 19–26. Morgan Kaufmann, San Francisco (2001)Google Scholar
  3. 3.
    Chapelle, O., Schölkopf, B., Zien, A. (eds.): Semi-Supervised Learning. MIT Press, Cambridge (2006)Google Scholar
  4. 4.
    Chapelle, O., Sindhwani, V., Keerthi, S.: Branch and bound for semi-supervised support vector machines. In: NIPS 19. MIT Press, Cambridge (2007)Google Scholar
  5. 5.
    Chapelle, O., Zien, A.: Semi-supervised classification by low density separation. In: AISTATS 2005, pp. 57–64 (2005)Google Scholar
  6. 6.
    Cristianini, N., Shawe–Taylor, J.: An Introduction to Support Vector Machines. Cambridge University Press, Cambridge (2000)Google Scholar
  7. 7.
    Delalleau, O., Bengio, Y., Le Roux, N.: Large-scale algorithms. In: Chapelle, O., Schölkopf, B., Zien, A. (eds.) Semi-Supervised Learning, pp. 333–341. MIT Press, Cambridge (2006)Google Scholar
  8. 8.
    Grandvalet, Y., Bengio, Y.: Semi-supervised learning by entropy minimization. In: Actes de CAP 2005, pp. 281–296 (2005)Google Scholar
  9. 9.
    Joachims, T.: Transductive inference for text classification using support vector machines. In: ICML 16, pp. 200–209. Morgan Kaufmann Publishers, San Francisco (1999)Google Scholar
  10. 10.
    Lawrence, N.D., Jordan, M.I.: Semi-supervised learning via gaussian processes. In: NIPS 17, pp. 753–760. MIT Press, Cambridge (2004)Google Scholar
  11. 11.
    Li, Y.-F., Zhou, Z.-H.: Improving semi-supervised support vector machines through unlabeled instances selection. In: Burgard, W., Roth, D. (eds.) AAAI. AAAI Press (2011)Google Scholar
  12. 12.
    Singh, A., Nowak, R.D., Zhu, X.: Unlabeled data: Now it helps, now it doesn’t. In: NIPS 21, pp. 1513–1520 (2008)Google Scholar
  13. 13.
    Sneath, P.H.A., Sokal, R.R.: Numerical Taxonomy. W.H. Freeman and Company, San Francisco (1973)zbMATHGoogle Scholar
  14. 14.
    Zhu, X., Goldberg, A.B.: Introduction to Semi-Supervised Learning. Morgan & Claypool Publishers, California (2009)zbMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Hernán C. Ahumada
    • 1
    • 2
    • 3
  • Pablo M. Granitto
    • 1
    • 2
  1. 1.CIFASIS, French Argentine International Center for Information and Systems SciencesUPCAMFrance
  2. 2.UNR-CONICET, ArgentinaRosarioArgentina
  3. 3.Facultad de Tecnología y Ciencias AplicadasUniversidad Nacional de CatamarcaCatamarcaArgentina

Personalised recommendations