Correntropy-Based Document Clustering via Nonnegative Matrix Factorization

  • Tolga Ensari
  • Jan Chorowski
  • Jacek M. Zurada
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7553)


Nonnegative Matrix Factorization (NMF) is one of the popular techniques to reduce the number of attributes of the data. It has been also widely used for clustering. Several types of the objective functions have been used for NMF in the literature. In this paper, we propose to maximize the correntropy similarity measure to produce the factorization itself. Correntropy is an entropy-based criterion defined as a nonlinear similarity measure. Following the discussion of minimization of the correntropy function, we use it to cluster document data set and compare its clustering performance with the Euclidean Distance (EucD)-based NMF. The comparison is illustrated with 20-Newsgroups data set. The results show that our approach has better clustering compared with other methods which use EucD as an objective function.


Nonnegative Matrix Factorization (NMF) Correntropy Document Clustering 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Lee, D.D., Seung, H.S.: Learning the Parts of Objects with Nonnegative Matrix Factorization. Nature 401, 788–791 (1999)CrossRefGoogle Scholar
  2. 2.
    Berry, M.W., Browne, M., Langville, A.N., Pauca, V.P., Plemmons, R.J.: Algorithms and Applications for Approximate Nonnegative Matrix Factorization. Computational Statistics and Data Analysis 52(1), 155–173 (2007)CrossRefzbMATHMathSciNetGoogle Scholar
  3. 3.
    Hoyer, P.O.: Non-negative Matrix Factorization with Sparseness Constraints. Journal of Machine Learning Research 5, 1457–1469 (2004)zbMATHMathSciNetGoogle Scholar
  4. 4.
    Lee, D.D., Seung, H.S.: Algorithms for Non-negative Matrix Factorization. In: Proc. of Advances in NeuralInformation Processing, vol. 13, pp. 556–562 (2001)Google Scholar
  5. 5.
    Schmidt, M.N., Olsson, R.K.: Single-channel Speech Separation Using Sparse Non-negative Matrix Factorization. In: Proc. of Interspeech, pp. 2614–2617 (2006)Google Scholar
  6. 6.
    Xu, J.W., Bakardjian, H., Cichocki, A., Principe, J.C.: A New Nonlinear Similarity Measure for Multichannel Biological Signals. In: Proc. of Int. Joint Conf. on Neural Networks, Orlando, Florida, USA, August 12-17 (2007)Google Scholar
  7. 7.
    Fevotte, C., Idier, J.: Algorithms for Nonnegative Matrix Factorization with the β-Divergence. Neural Computation 13(3), 1–24 (2010)Google Scholar
  8. 8.
    Choi, S.: Algorithms for Orthogonal Nonnegative Matrix Factorization. In: Proc. of the Int. Joint Conf. on Neural Networks, Hong Kong, June 1-6, pp. 1828–1832 (2008)Google Scholar
  9. 9.
    Zhao, W., Ma, H., Li, N.: A Nonnegative Matrix Factorization Algorithm with Sparseness Constraints. In: Proc. of the Int. Conf. on Machine Learning and Cybernetics, Guilin, ChinaGoogle Scholar
  10. 10.
    Schmidt, M.N., Winther, O., Hansen, L.K.: Bayesian Non-negative Matrix Factorization. In: Adali, T., Jutten, C., Romano, J.M.T., Barros, A.K. (eds.) ICA 2009. LNCS, vol. 5441, pp. 540–547. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  11. 11.
    Fevotte, C., Bertin, N., Durrieu, J.L.: Nonnegative Matrix Factorization with the Itakura-Saito Divergence. Neural Computation 21, 793–830 (2009)CrossRefzbMATHGoogle Scholar
  12. 12.
    Shahnaz, F., Berry, M.W., Pauca, V.P., Plemmons, R.J.: Document Clustering Using Nonnegative Matrix Factorization. Int. Journal of Information Processing and Management 42(2), 373–386 (2006)CrossRefzbMATHGoogle Scholar
  13. 13.
    Guillamet, D., Vitria, J., Schiele, B.: Introducing a Weighted Non-negative Matrix Factorization for Image Classification. Pattern Recognition Letters 24, 2447–2454 (2003)CrossRefzbMATHGoogle Scholar
  14. 14.
    He, R., Zheng, W.S., Hu, B.G.: Maximum Correntropy Criterion for Robust Face Recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence 3(8), 1561–1576 (2011)Google Scholar
  15. 15.
    Liu, W., Pokharel, P.P., Principe, J.C.: Correntropy: Properties and Applications in Non-Gaussian Signal Processing. IEEE Trans. on Signal Processing 55(11), 5286–5298 (2007)CrossRefMathSciNetGoogle Scholar
  16. 16.
    Singh, A., Principe, J.C.: Using Correntropy as a Cost Function in Linear Adaptive Filters. In: Proc. of Int. Joint Conference on Neural Networks, Atlanta, USA, June 14-19 (2009)Google Scholar
  17. 17.
    He, R., Hu, B.G., Zheng, W.S., Kong, X.W.: Robust Principal Component Analysis Based on Maximum Correntropy Criterion. IEEE Trans. on Image Processing 20(6) (2011)Google Scholar
  18. 18.
    Chalasani, R., Principe, J.H.: Self Organizing Maps with Correntropy Induced Metric. In: Proc. of Int. Joint Conf. on Neural Networks, Spain, pp. 1–6 (2010)Google Scholar
  19. 19.
    Jeong, K.H., Principe, J.C.: Enhancing the Correntropy MACE Filter with Random Projections. Neurocomputing 72(1-3), 102–111 (2008)CrossRefGoogle Scholar
  20. 20.
  21. 21.
    Berry, M.W., Gillis, N., Glineur, F.: Document Classification Using Nonnegative Matrix Factorization and Underapproximation. In: Int. Symp on Circuits and Systems, Taiwan (2009)Google Scholar
  22. 22.
    Zhao, W., Ma, H., Li, N.: A New Non-negative Matrix Factorization Algorithm with Sparseness Constraints. In: Proc. of the 2011 Int. Conf. on Machine Learning and Cybernetics, Guilin, July 10-13 (2011)Google Scholar
  23. 23.
    Lin, C.J.: Projected Gradient methods for Non-Negative Matrix Factorization. Neural Computation 19, 2756–2779 (2007)CrossRefzbMATHMathSciNetGoogle Scholar
  24. 24.
    Tan, P., Steinbach, M., Kumar, V.: Introduction to Data Mining. Pearson Addison Wesley (2006)Google Scholar
  25. 25.
    Paatero, P.: Least Squares Formulation of Robust Non-negative Factor Analysis. Chemometricsand Intelligent Laboratory System 37, 23–35 (1997)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Tolga Ensari
    • 1
  • Jan Chorowski
    • 2
  • Jacek M. Zurada
    • 2
  1. 1.Computer Engineering DepartmentIstanbul UniversityIstanbulTurkey
  2. 2.Electrical and Computer EngineeringUniversity of LouisvilleLouisvilleUSA

Personalised recommendations