Advertisement

Manifold Coarse Graining for Online Semi-supervised Learning

  • Mehrdad Farajtabar
  • Amirreza Shaban
  • Hamid Reza Rabiee
  • Mohammad Hossein Rohban
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6911)

Abstract

When the number of labeled data is not sufficient, Semi-Supervised Learning (SSL) methods utilize unlabeled data to enhance classification. Recently, many SSL methods have been developed based on the manifold assumption in a batch mode. However, when data arrive sequentially and in large quantities, both computation and storage limitations become a bottleneck. In this paper, we present a new semi-supervised coarse graining (CG) algorithm to reduce the required number of data points for preserving the manifold structure. First, an equivalent formulation of Label Propagation (LP) is derived. Then a novel spectral view of the Harmonic Solution (HS) is proposed. Finally an algorithm to reduce the number of data points while preserving the manifold structure is provided and a theoretical analysis on preservation of the LP properties is presented. Experimental results on real world datasets show that the proposed method outperforms the state of the art coarse graining algorithm in different settings.

Keywords

Semi-Supervised Learning Manifold Assumption Harmonic Solution Label Propagation Spectral Coarse Graining Online Classification 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Zhu, X.: Semi-Supervised Learning Literature Survey. Technical Report 1530, Department of Computer Sciences, University of Wisconsin Madison (2005)Google Scholar
  2. 2.
    Chapelle, O., Scholkopf, B., Zien, A.: Semi-supervised Learning. MIT Press, Cambridge (2006)CrossRefGoogle Scholar
  3. 3.
    Belkin, M., Niyogi, P., Sindhwani, V.: Manifold Regularization: a Geometric Framework for Learning from Labeled and Unlabeled Examples. Journal of Machine Learning Research 7, 2399–2434 (2006)MathSciNetzbMATHGoogle Scholar
  4. 4.
    Duchenne, O., Audibert, J., Keriven, R., Ponce, J., Segonne, F.: Segmentation by Transduction. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2008, pp. 1–8 (2008)Google Scholar
  5. 5.
    Belkin, M., Niyogi, P.: Using Manifold Structure for Partially Labeled Classification. Advances in Neural Information Processing Systems 15, 929–936 (2003)Google Scholar
  6. 6.
    Grabner, H., Leistner, C., Bischof, H.: Semi-supervised On-Line Boosting for Robust Tracking. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part I. LNCS, vol. 5302, pp. 234–247. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  7. 7.
    He., X.: Incremental Semi-Supervised Subspace Learning for Image Retrieval. In: Proceedings of the ACM Conference on Multimedia (2004)Google Scholar
  8. 8.
    Moh, Y., Buhmann, J.M.: Manifold Regularization for Semi-Supervised Sequential Learning. In: ICASSP (2009)Google Scholar
  9. 9.
    Goldberg, A., Li, M., Zhu, X.: Online Manifold Regularization: A New Learning Setting and Empirical Study. In: Proceeding of ECML (2008)Google Scholar
  10. 10.
    Dasgupta, S., Freund, Y.: Random Projection Trees and Low Dimensional Manifolds. Technical Report CS2007-0890, University of California, San Diego (2007)Google Scholar
  11. 11.
    Valko, M., Kveton, B., Ting, D., Huang, L.: Online Semi-Supervised Learning on Quantized Graphs. In: Proceedings of the 26th Conference on Uncertainty in Artificial Intelligence, UAI (2010)Google Scholar
  12. 12.
    Lafon, S., Lee, A.B.: Diffusion Maps and Coarse-Graining: A Unified Framework for Dimensionality Reduction, Graph Partitioning, and Data Set Parameterization. IEEE Transactions on Pattern Analysis and Machine Intelligence 28(9), 1393–1403 (2006)CrossRefGoogle Scholar
  13. 13.
    Zhou, D., Bousquet, O., Lal, T., Weston, J., Scholkopf, B.: Learning with local and global consistency. Neural Information Processing Systems (2004)Google Scholar
  14. 14.
    Zhu, X., Ghahramani, Z., Lafferty, J.: Semi-Supervised Learning Using Gaussian Fields and Harmonic Functions. In: ICML (2003)Google Scholar
  15. 15.
    Zhu, X., Ghahramani, Z.: Learning from Labeled and Unlabeled Data with Label Propagation. Technical Report CMU-CALD-02-107, Carnegie Mellon University (2002)Google Scholar
  16. 16.
    Press, W.H., Teukolsky, S.A., Vetterling, W.T., Flannery, B.P.: Numerical Recipes, The Art of Scientific Computing, 3rd edn. Cambridge University Press, Cambridge (2007)zbMATHGoogle Scholar
  17. 17.
    Gfeller, D., De Los Rios, P.: Spectral Coarse Graining of Complex Networks. Physical Review Letters 99, 3 (2007)CrossRefGoogle Scholar
  18. 18.
    Frank, A., Asuncion, A.: UCI Machine Learning Repository (2010)Google Scholar
  19. 19.
    Fei, L., Fergus, R., Perona, P.: Learning Generative Visual Models From Few Training Examples: An Incremental Bayesian Approach Tested on 101 Object Categories. In: IEEE CVPR 2004, Workshop on Generative Model Based Vision (2004)Google Scholar
  20. 20.
    Chatzichristofis, S.A., Boutalis, Y.S.: CEDD: Color and Edge Directivity Descriptor: A Compact Descriptor for Image Indexing and Retrieval. In: ICVS, pp. 312–322 (2008)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Mehrdad Farajtabar
    • 1
  • Amirreza Shaban
    • 1
  • Hamid Reza Rabiee
    • 1
  • Mohammad Hossein Rohban
    • 1
  1. 1.Digital Media Lab, AICTC Research Center, Department of Computer EngineeringSharif University of TechnologyTehranIran

Personalised recommendations