Advertisement

Large-Scale Spectral Clustering with Stochastic Nyström Approximation

  • Hongjie JiaEmail author
  • Liangjun Wang
  • Heping Song
Conference paper
  • 15 Downloads
Part of the IFIP Advances in Information and Communication Technology book series (IFIPAICT, volume 581)

Abstract

In spectral clustering, Nyström approximation is a powerful technique to reduce the time and space cost of matrix decomposition. However, in order to ensure the accurate approximation, a sufficient number of samples are needed. In very large datasets, the internal singular value decomposition (SVD) of Nyström will also spend a large amount of calculation and almost impossible. To solve this problem, this paper proposes a large-scale spectral clustering algorithm with stochastic Nyström approximation. This algorithm uses the stochastic low rank matrix approximation technique to decompose the sampled sub-matrix within the Nyström procedure, losing a slight of accuracy in exchange for a significant improvement of the algorithm efficiency. The performance of the proposed algorithm is tested on benchmark data sets and the clustering results demonstrate its effectiveness.

Keywords

Spectral clustering Nyström approximation Stochastic SVD Large dataset 

Notes

Acknowledgement

This work was supported by the National Natural Science Foundations of China (grant numbers 61906077, 61601202), the Natural Science Foundation of Jiangsu Province (grant numbers BK20190838, BK20170558), and the Natural Science Foundation of the Jiangsu Higher Education Institutions of China (grant number 18KJB520009, 16KJB520008).

References

  1. 1.
    Kang, Z., et al.: Multi-graph fusion for multi-view spectral clustering. Knowl.-Based Syst. 189 (2020).  https://doi.org/10.1016/j.knosys.2019.105102
  2. 2.
    Tang, M., Marin, D., Ayed, I.B., Boykov, Y.: Kernel cuts: kernel and spectral clustering meet regularization. Int. J. Comput. Vis. 127(5), 477–511 (2019).  https://doi.org/10.1007/s11263-018-1115-1MathSciNetCrossRefGoogle Scholar
  3. 3.
    Jia, H., Ding, S., Du, M.: A Nyström spectral clustering algorithm based on probability incremental sampling. Soft Comput. 21(19), 5815–5827 (2016).  https://doi.org/10.1007/s00500-016-2160-8CrossRefzbMATHGoogle Scholar
  4. 4.
    Fowlkes, C., Belongie, S., Chung, F., Malik, J.: Spectral grouping using the Nystrom method. IEEE Trans. Pattern Anal. Mach. Intell. 26(2), 214–225 (2004)CrossRefGoogle Scholar
  5. 5.
    Li, M., Bi, W., Kwok, J.T., Lu, B.L.: Large-scale Nyström kernel matrix approximation using randomized SVD. IEEE Trans. Neural Netw. Learn. Syst. 26(1), 152–164 (2014)Google Scholar
  6. 6.
    Halko, N., Martinsson, P.G., Tropp, J.A.: Finding structure with randomness: Probabilistic algorithms for constructing approximate matrix decompositions. SIAM Rev. 53(2), 217–288 (2011)MathSciNetCrossRefGoogle Scholar
  7. 7.
    Drineas, P., Kannan, R., Mahoney, M.W.: Fast Monte Carlo algorithms for matrices II: computing a low-rank approximation to a matrix. SIAM J. Comput. 36(1), 158–183 (2006)MathSciNetCrossRefGoogle Scholar
  8. 8.
    Jia, H., Ding, S., Du, M., Xue, Y.: Approximate normalized cuts without Eigen-decomposition. Inf. Sci. 374, 135–150 (2016)CrossRefGoogle Scholar
  9. 9.
    Wang, S., Gittens, A., Mahoney, M.W.: Scalable kernel K-means clustering with Nyström approximation: relative-error bounds. J. Mach. Learn. Res. 20(1), 431–479 (2019)zbMATHGoogle Scholar
  10. 10.
    Chitta, R., Jin, R., Havens, T.C., Jain, A.K.: Approximate kernel k-means: solution to large scale kernel clustering. In: Proceedings of the 17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 895–903. ACM, San Diego (2011)Google Scholar
  11. 11.
    Chen, W.Y., Song, Y., Bai, H., Lin, C.J., Chang, E.Y.: Parallel spectral clustering in distributed systems. IEEE Trans. Pattern Anal. Mach. Intell. 33(3), 568–586 (2011)CrossRefGoogle Scholar

Copyright information

© IFIP International Federation for Information Processing 2020

Authors and Affiliations

  1. 1.Jiangsu UniversityZhenjiangChina

Personalised recommendations