Accelerated Matrix Factorisation Method for Fuzzy Clustering

  • Mingjun Zhan
  • Bo LiEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10638)


Factorised fuzzy c-means (F-FCM) based on semi nonnegative matrix factorization is a new approach for fuzzy clustering. It does not need the weighting exponent parameter compared with traditional fuzzy c-means, and not sensitive to initial conditions. However, F-FCM does not propose an efficient method to solve the constrained problem, and just suggests to use a lsqlin() function in MATLAB which lead to slow convergence rate and nonconvergence. In this paper, we propose a method to accelerate the convergence rate of F-FCM combining with a non-monotone accelerate proximal gradient (nmAPG) method. We also propose an efficient method to solve the proximal mapping problem when implementing nmAPG. Finally, the experiment results on synthetic and real-world datasets show the performances and feasibility of our method.


Nonnegative matrix factorization Factorised fuzzy c-means Non-monotone accelerate proximal gradient 



This research was supported by the National Natural Science Foundation of China (Grant Nos. 11627802, 51678249), by the Science and Technology Projects of Guangdong (2013A011403003), and by the Science and Technology Projects of Guangzhou (201508010023).


  1. 1.
    Cichocki, A., Zdunek, R., Amari, S.I.: New algorithms for non-negative matrix factorization in applications to blind source separation. In: Proceedings of IEEE International Conference Acoustics, Speech, Signal Processing, vol. 5, pp. 621–624 (2006)Google Scholar
  2. 2.
    Combettes, P.L., Pesquet, J.C.: Proximal splitting methods in signal processing. In: Bauschke, H., Burachik, R., Combettes, P., Elser, V., Luke, D., Wolkowicz, H. (eds.) Fixed-Point Algorithms for Inverse Problems in Science and Engineering, pp. 185–212. Springer, New York (2011). doi: 10.1007/978-1-4419-9569-8_10 CrossRefGoogle Scholar
  3. 3.
    Ding, C., He, X., Simon, H.: On the equivalence of nonnegative matrix factorization and spectral clustering. In: SDM, vol. 5, pp. 606–610 (2005)Google Scholar
  4. 4.
    Ding, C., Li, T., Jordan, M.: Convex and semi-nonnegative matrix factorizations. IEEE Trans. Pattern Anal. Mach. Intell. 32(1), 45–55 (2010)CrossRefGoogle Scholar
  5. 5.
    Gill, P.E., Murray, W., Saunders, M.A., Wright, M.H.: Procedures for optimization problems with a mixture of bounds and general linear constraints. ACM Trans. Math. Softw. (TOMS) 10(3), 282–298 (1984)MathSciNetCrossRefzbMATHGoogle Scholar
  6. 6.
    Gill, P.E., Murray, W., Wright, M.H.: Practical Optimization (1981)Google Scholar
  7. 7.
    Guan, N., Tao, D., Luo, Z., Yuan, B.: NeNMF: an optimal gradient method for nonnegative matrix factorization. IEEE Trans. Sig. Process. 60(6), 2882–2898 (2012)MathSciNetCrossRefGoogle Scholar
  8. 8.
    Lee, D.D., Seung, H.S.: Learning the parts of objects by non-negativ matrix factorization. Nature 401(6755), 788–91 (1999)CrossRefzbMATHGoogle Scholar
  9. 9.
    Lee, D.D., Seung, H.S.: Algorithms for non-negative matrix factorization. In: Advances in Neural Information Processing Systems, pp. 556–562 (2001)Google Scholar
  10. 10.
    Li, H., Lin, Z.: Accelerated proximal gradient methods for nonconvex programming. In: Advances in Neural Information Processing Systems, pp. 379–387 (2015)Google Scholar
  11. 11.
    Ma, W.K., et al.: A signal processing perspective on hyperspectral unmixing: insights from remote sensing. IEEE Sig. Process. Mag. 31(1), 67–81 (2014)MathSciNetCrossRefGoogle Scholar
  12. 12.
    Nicolas, G.: The why and how of nonnegative matrix factorization. In: Regularization, Optimization, Kernels, and Support Vector Machines, vol. 12, no. 257 (2014)Google Scholar
  13. 13.
    Nocedal, J., Wright, S.: Numerical Optimization, pp. 185–212 (2006)Google Scholar
  14. 14.
    Pal, N.R., Bezdek, J.C.: On cluster validity for the fuzzy c-means model. IEEE Trans. Fuzzy Syst. 3(3), 370–379 (1995)CrossRefGoogle Scholar
  15. 15.
    Parikh, N., Boyd, S.: Proximal algorithms. Found. Trends Optim. 1(3), 127–239 (2013)CrossRefGoogle Scholar
  16. 16.
    Pompili, F., Gillis, N., Absil, P., Glineur, F.: Two algorithms for orthogonal nonnegative matrix factorization with application to clustering. Neurocomputing 141, 15–25 (2014)CrossRefGoogle Scholar
  17. 17.
    Shahnaz, F., Berry, M., Pauca, V., Plemmons, R.: Document clustering using nonnegative matrix factorization. Inf. Process. Manag. 42(2), 373–386 (2006)CrossRefzbMATHGoogle Scholar
  18. 18.
    Suleman, A.: A convex semi-nonnegative matrix factorisation approach to fuzzy c-means clustering. Fuzzy Sets Syst. 270, 90–110 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
  19. 19.
    Trigeorgis, G., Bousmalis, K., Zafeiriou, S., Schuller, B.: A deep semi-NMF model for learning hidden representations. In: ICML, pp. 1692–1700 (2014)Google Scholar
  20. 20.
    Zha, H., He, X., Ding, C., Gu, M., Simon, H.D.: Spectral relaxation for k-means clustering. In: Advances in Neural Information Processing Systems, pp. 1057–1064 (2001)Google Scholar
  21. 21.
    Zhou, G., Cichocki, A., Zhang, Y., Mandic, D.P.: Group component analysis for multiblock data: common and individual feature extraction. IEEE Trans. Neural Netw. Learn. Syst. 27(11), 2426–2439 (2015)MathSciNetCrossRefGoogle Scholar
  22. 22.
    Zhou, G., Zhao, Q., Zhang, Y., Adali, T., Xie, S., Cichocki, A.: Linked component analysis from matrices to high-order tensors: applications to biomedical data. Proc. IEEE 104(2), 310–331 (2015)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.School of Electronic and Information EngineeringSouth China University of TechnologyGuangzhouChina

Personalised recommendations