In this paper, we introduce the approach of graph densification as a means of preconditioning spectral clustering. After motivating the need of densification, we review the fundamentals of graph densifiers based on cut similarity and then analyze their associated optimization problems. In our experiments we analyze the implications of densification in the estimation of commute times.


Graph densification Cut similarity Spectral clustering 


  1. 1.
    Alamgir, M., von Luxburg, U.: Shortest path distance in random k-nearest neighbor graphs. In: Proceedings of ICML 2012 (2012)Google Scholar
  2. 2.
    Arora, S., Karger, D., Karpinski, M.: Polynomial time approximation schemes for dense instances of NP-hard problems. J. Comput. Syst. Sci. 58(1), 193–210 (1999)MathSciNetCrossRefzbMATHGoogle Scholar
  3. 3.
    Batson, J.D., Spielman, D.A., Srivastava, N., Teng, S.: Spectral sparsification of graphs: theory and algorithms. Commun. ACM 56(8), 87–94 (2013)CrossRefGoogle Scholar
  4. 4.
    Benczúr, A.A., Karger, D.R.: Approximating s-t minimum cuts in \(O(n^2)\) time. In: Proceedings of the Twenty-Eighth Annual ACM Symposium on the Theory of Computing, pp. 47–55 (1996)Google Scholar
  5. 5.
    Cai, D., Chen, X.: Large scale spectral clustering via landmark-based sparse representation. IEEE Trans. Cybern. 45(8), 1669–1680 (2015)CrossRefGoogle Scholar
  6. 6.
    Chen, J., Fang, H., Saad, Y.: Fast approximate kNN graph construction for high dimensional data via recursive lanczos bisection. J. Mach. Learn. Res. 10, 1989–2012 (2012)MathSciNetzbMATHGoogle Scholar
  7. 7.
    Frieze, A.M., Kannan, R.: The regularity lemma and approximation schemes for dense problems. In: 37th Annual Symposium on Foundations of Computer Science, FOCS 96, pp. 12–20 (1996)Google Scholar
  8. 8.
    Hardt, M., Srivastava, N., Tulsiani, M.: Graph densification. Innovations Theoret. Comput. Sci. 2012, 380–392 (2012)MathSciNetCrossRefzbMATHGoogle Scholar
  9. 9.
    Khoa, N.L.D., Chawla, S.: Large scale spectral clustering using approximate commute time embedding. CoRR abs/1111.4541 (2011)Google Scholar
  10. 10.
    Vladymyrov, M., Carreira-Perpinan, M.A.: The Variational Nystrom method for large-scale spectral problems. In: ICML 2016, pp. 211–220 (2016)Google Scholar
  11. 11.
    Khoa, N.L.D., Chawla, S.: Large scale spectral clustering using resistance distance and spielman-teng solvers. In: Ganascia, J.-G., Lenca, P., Petit, J.-M. (eds.) DS 2012. LNCS (LNAI), vol. 7569, pp. 7–21. Springer, Heidelberg (2012). doi: 10.1007/978-3-642-33492-4_4 CrossRefGoogle Scholar
  12. 12.
    Komlós, J., Shokoufandeh, A., Simonovits, M., Szemerédi, E.: The regularity lemma and its applications in graph theory. In: Khosrovshahi, G.B., Shokoufandeh, A., Shokrollahi, A. (eds.) TACSci 2000. LNCS, vol. 2292, pp. 84–112. Springer, Heidelberg (2002). doi: 10.1007/3-540-45878-6_3 CrossRefGoogle Scholar
  13. 13.
    Liu, W., He, J., Chang, S.: Large graph construction for scalable semi-supervised learning. In: Proceedings of ICML 2010, pp. 679–686 (2010)Google Scholar
  14. 14.
    Liu, W., Mu, C., Kumar, S., Chang, S.: Discrete graph hashing. In: NIPS 2014, pp. 3419–3427 (2014)Google Scholar
  15. 15.
    Liu, W., Wang, J., Chang, S.: Robust and scalable graph-based semisupervised learning. Proc. IEEE 100(9), 2624–2638 (2012)CrossRefGoogle Scholar
  16. 16.
    Liu, W., Wang, J., Kumar, S., Chang, S.: Hashing with graphs. In: Proceedings of ICML 2011, pp. 1–8 (2011)Google Scholar
  17. 17.
    Luo, Z., Ma, W., So, A.M., Ye, Y., Zhang, S.: Semidefinite relaxation of quadratic optimization problems. IEEE Sig. Process. Mag. 27(3), 20–34 (2010)CrossRefGoogle Scholar
  18. 18.
    Qiu, H., Hancock, E.R.: Clustering and embedding using commute times. IEEE TPAMI 29(11), 1873–1890 (2007)CrossRefGoogle Scholar
  19. 19.
    Spielman, D.A., Srivastava, N.: Graph sparsification by effective resistances. SIAM J. Comput. 40(6), 1913–1926 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
  20. 20.
    von Luxburg, U., Alamgir, M.: Density estimation from unweighted k-nearest neighbor graphs: a roadmap. In: NIPS 2013, pp. 225–233 (2013)Google Scholar
  21. 21.
    von Luxburg, U., Radl, A., Hein, M.: Getting lost in space: large sample analysis of the resistance distance. In: NIPS 2010, pp. 2622–2630 (2010)Google Scholar
  22. 22.
    von Luxburg, U., Radl, A., Hein, M.: Hitting and commute times in large random neighborhood graphs. J. Mach. Learn. Res. 15(1), 1751–1798 (2014)MathSciNetzbMATHGoogle Scholar
  23. 23.
    Toh, K.C., Todd, M., Tutuncu, R.: SDPT3 - A MATLAB software package for semidefinite programming. Optim. Methods Softw. 11, 545–581 (1998)MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  • Francisco Escolano
    • 1
    • 2
    Email author
  • Manuel Curado
    • 1
    • 2
  • Edwin R. Hancock
    • 1
    • 2
  1. 1.Department of Computer Science and AIUniversity of AlicanteAlicanteSpain
  2. 2.Department of Computer ScienceUniversity of YorkYorkUK

Personalised recommendations