Semi-supervised Learning Based on Joint Diffusion of Graph Functions and Laplacians

  • Kwang In KimEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9909)


We observe the distances between estimated function outputs on data points to create an anisotropic graph Laplacian which, through an iterative process, can itself be regularized. Our algorithm is instantiated as a discrete regularizer on a graph’s diffusivity operator. This idea is grounded in the theory that regularizing the diffusivity operator corresponds to regularizing the metric on Riemannian manifolds, which further corresponds to regularizing the anisotropic Laplace-Beltrami operator. We show that our discrete regularization framework is consistent in the sense that it converges to (continuous) regularization on underlying data generating manifolds. In semi-supervised learning experiments, across ten standard datasets, our diffusion of Laplacian approach has the lowest average error rate of eight different established and state-of-the-art approaches, which shows the promise of our approach.


Semi-supervised learning Graph Laplacian Diffusion Regularization 



We thank James Tompkin for fruitful discussions and comments, and EPSRC for grant EP/M00533X/1.


  1. 1.
    Zhou, D., Bousquet, O., Lal, T.N., Weston, J., Schölkopf, B.: Learning with local and global consistency. In: NIPS, pp. 321–328 (2003)Google Scholar
  2. 2.
    Kim, K.I., Tompkin, J., Pfister, H., Theobalt, C.: Context-guided diffusion for label propagation on graphs. In: Proceedings of the IEEE ICCV, pp. 2776–2784 (2015)Google Scholar
  3. 3.
    Hein, M., Audibert, J.Y., von Luxburg, U.: Graph Laplacians and their convergence on random neighborhood graphs. JMLR 8, 1325–1368 (2007)MathSciNetzbMATHGoogle Scholar
  4. 4.
    Belkin, M., Niyogi, P.: Towards a theoretical foundation for Laplacian-based manifold methods. J. Comput. Syst. Sci. 74(8), 1289–1308 (2005)MathSciNetCrossRefzbMATHGoogle Scholar
  5. 5.
    Weickert, J.: Anisotropic Diffusion in Image Processing. ECMI Series. Teubner-Verlag, Stuttgart (1998)zbMATHGoogle Scholar
  6. 6.
    Clarenz, U., Diewald, U., Rumpf, M.: Anisotropic geometric diffusion in surface processing. In: Proceedings of the Visualization, pp. 397–405 (2000)Google Scholar
  7. 7.
    Szlam, A.D., Maggioni, M., Coifman, R.R.: Regularization on graphs with function-adapted diffusion processes. JMLR 9, 1711–1739 (2008)MathSciNetzbMATHGoogle Scholar
  8. 8.
    Tschumperlé, D., Deriche, R.: Diffusion tensor regularization with constraints preservation. In: Proceedings of the IEEE CVPR, pp. I948–I953 (2001)Google Scholar
  9. 9.
    Castaño-Moraga, C.A., Lenglet, C., Deriche, R., Ruiz-Alzola, J.: A Riemannian approach to anisotropic filtering of tensor fields. Sig. Process. 87(2), 263–276 (2007)CrossRefzbMATHGoogle Scholar
  10. 10.
    Singer, A., Wu, H.T.: Vector diffusion maps and the connection Laplacian. Commun. Pure Appl. Math. 65(8), 1067–1144 (2012)MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    Zhang, T., Ando, R.: Analysis of spectral kernel design based semi-supervised learning. In: NIPS, pp. 1601–1608 (2006)Google Scholar
  12. 12.
    Johnson, R., Zhang, T.: Graph-based semi-supervised learning and spectral kernel design. IEEE Trans. Inf. Theor. 54(1), 275–288 (2008)MathSciNetCrossRefzbMATHGoogle Scholar
  13. 13.
    Kim, K.I., Tompkin, J., Theobald, M., Kautz, J., Theobalt, C.: Match graph construction for large image databases. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, vol. 7572, pp. 272–285. Springer, Heidelberg (2012). doi: 10.1007/978-3-642-33718-5_20 Google Scholar
  14. 14.
    Szummer, M., Yilmaz, E.: Semi-supervised learning to rank with preference regularization. In: Proceedings of the ACM CIKM, pp. 269–278 (2011)Google Scholar
  15. 15.
    Fan, M.K.H., Nekooie, B.: On minimizing the largest eigenvalue of a symmetric matrix. Linear Algebra Appl. 214, 225–246 (1995)MathSciNetCrossRefzbMATHGoogle Scholar
  16. 16.
    Boyd, S., Yang, Q.: Structured and simultaneous Lyapunov functions for system stability problems. Int. J. Control 49(6), 2215–2240 (1989)MathSciNetCrossRefzbMATHGoogle Scholar
  17. 17.
    Hein, M.: Geometrical Aspects of Statistical Learning Theory. Ph.d. thesis, Fachbereich Informatik, Technische Universität Darmstadt, Germany (2005)Google Scholar
  18. 18.
    Coifman, R.R., Lafon, S.: Diffusion maps. Appl. Comput. Harmonic Anal. 21(1), 5–30 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  19. 19.
    Rosenberg, S.: The Laplacian on a Riemannian Manifold. Cambridge University Press, Cambridge (1997)CrossRefzbMATHGoogle Scholar
  20. 20.
    Hein, M., Audibert, J.-Y., von Luxburg, U.: From graphs to manifolds – weak and strong pointwise consistency of graph Laplacians. In: Auer, P., Meir, R. (eds.) COLT 2005. LNCS (LNAI), vol. 3559, pp. 470–485. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  21. 21.
    Zhou, X., Belkin, M.: Semi-supervised learning by higher order regularization. In: JMLR W&CP (Proceedings of AISTATS), pp. 892–900 (2011)Google Scholar
  22. 22.
    Wang, F., Zhang, C.: Label propagation through linear neighborhoods. In: Proceedings of ICML, pp. 985–992 (2006)Google Scholar
  23. 23.
    Bühler, T., Hein, M.: Spectral clustering based on the graph p-Laplacian. In: Proceedings of ICML, pp. 81–88 (2009)Google Scholar
  24. 24.
    Chapelle, O., Schölkopf, B., Zien, A.: Semi-Supervised Learning. MIT Press, Cambridge (2010). Datasets: Google Scholar
  25. 25.
    LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)CrossRefGoogle Scholar
  26. 26.
    Söderkvist, O.: Computer Vision Classification of Leaves from Swedish Trees. Ph.d. thesis, Master thesis, Linköping University, Sweden (2001). Datasets:
  27. 27.
    Ling, H., Jacobs, D.W.: Shape classification using the inner-distance. IEEE T-PAMI 29(2), 286–299 (2007)CrossRefGoogle Scholar
  28. 28.
    Latecki, L.J., Lakämper, R., Eckhardt, U.: Shape descriptors for non-rigid shapes with a single closed contour. In: Proceedings of the IEEE CVPR, pp. 424–429 (2000)Google Scholar
  29. 29.
    Donoser, M., Bischof, H.: Diffusion processes for retrieval revisited. In: Proceedings of the IEEE CVPR, pp. 1320–1327 (2013)Google Scholar
  30. 30.
    Gopalan, R., Turaga, P., Chellappa, R.: Diffusion processes for retrieval revisited. In: Proceedings of the ECCV, pp. 286–299 (2010). Datasets:
  31. 31.
    Ebert, S., Fritz, M., Schiele, B.: RALF: a reinforced active learning formulation for object class recognition. In: Proceedings of the IEEE CVPR, pp. 3626–3633 (2012)Google Scholar
  32. 32.
    Dai, D., Gool, L.V.: Ensemble projection for semi-supervised image classification. In: Proceedings of the IEEE ICCV, pp. 2072–2079 (2013). Datasets:

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  1. 1.Department of Computer ScienceUniversity of BathBathUK

Personalised recommendations