Row-Action Projections for Nonnegative Matrix Factorization

  • Rafał Zdunek
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8681)


Nonnegative Matrix Factorization (NMF) is more and more frequently used for analyzing large-scale nonnegative data, where the number of samples and/or the number of observed variables is large. In the paper, we discuss two applications of the row-action projections in the context of learning latent factors from large-scale data. First, we show that they can be efficiently used for improving the on-line learning in dynamic NMF. Next, they can also considerably reduce the computational complexity of the optimization algorithms used for factor learning from strongly redundant data. The experiments demonstrate high efficiency of the proposed methods.


NMF Kaczmarz method On-line NMF Row-action projections Feature extraction 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Lee, D.D., Seung, H.S.: Learning the parts of objects by non-negative matrix factorization. Nature 401, 788–791 (1999)CrossRefGoogle Scholar
  2. 2.
    Cichocki, A., Zdunek, R., Phan, A.H., Amari, S.I.: Nonnegative Matrix and Tensor Factorizations: Applications to Exploratory Multi-way Data Analysis and Blind Source Separation. Wiley and Sons (2009)Google Scholar
  3. 3.
    Dong, C., Zhao, H., Wang, W.: Parallel nonnegative matrix factorization algorithm on the distributed memory platform. International Journal of Parallel Programming 38, 117–137 (2010)CrossRefzbMATHGoogle Scholar
  4. 4.
    Wang, Q., Cao, Z., Xu, J., Li, H.: Group matrix factorization for scalable topic modeling. In: Proc. the 35th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2012, pp. 375–384. ACM Press, New York (2012)Google Scholar
  5. 5.
    Liu, C., Yang, H.C., Fan, J., He, L.W., Wang, Y.M.: Distributed nonnegative matrix factorization for web-scale dyadic data analysis on mapreduce. In: Proc. 19th International Conference on World Wide Web, WWW 2010, pp. 681–690. ACM, New York (2010)CrossRefGoogle Scholar
  6. 6.
    Gemulla, R., Nijkamp, E., Haas, P.J., Sismanis, Y.: Large-scale matrix factorization with distributed stochastic gradient descent. In: Proc. the 17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD 2011, pp. 69–77. ACM, New York (2011)Google Scholar
  7. 7.
    Liao, R., Zhang, Y., Guan, J., Zhou, S.: CloudNMF: A mapreduce implementation of nonnegative matrix factorization for large-scale biological datasets. Genomics, Proteomics and Bioinformatics 12(1), 48–51 (2014)CrossRefGoogle Scholar
  8. 8.
    Phan, A.H., Cichocki, A.: PARAFAC algorithms for large-scale problems. Neurocomputing 74(11), 1970–1984 (2011)CrossRefGoogle Scholar
  9. 9.
    Cao, B., Shen, D., Sun, J.T., Wang, X., Yang, Q., Chen, Z.: Detect and track latent factors with online nonnegative matrix factorization. In: Proc. International Joint Conference on Artificial Intelligence (IJCAI), Hyderabad, India, pp. 2689–2694 (2007)Google Scholar
  10. 10.
    Wang, F., Tan, C., Konig, A.C., Li, P.: Efficient document clustering via online nonnegative matrix factorizations. In: Proc. 11-th SIAM Conference on Data Mining, pp. 908–919. SIAM / Omnipress (2011)Google Scholar
  11. 11.
    Lefevre, A., Bach, F., Févotte, C.: Online algorithms for nonnegative matrix factorization with the Itakura-Saito divergence. In: 2011 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA), pp. 313–316. IEEE (2011)Google Scholar
  12. 12.
    Zhou, G., Yang, Z., Xie, S., Yang, J.M.: Online blind source separation using incremental nonnegative matrix factorization with volume constraint. IEEE Transactions on Neural Networks 22(4), 550–560 (2011)CrossRefMathSciNetGoogle Scholar
  13. 13.
    Mairal, J., Bach, F., Ponce, J., Sapiro, G.: Online learning for matrix factorization and sparse coding. Journal of Machine Learning Research 11, 19–60 (2010)zbMATHMathSciNetGoogle Scholar
  14. 14.
    Kaczmarz, S.: Angenaherte Auflosung von Systemen linearer Gleichungen. Bulletin de lAcademie Polonaise des Sciences et Lettres 35, 355–357 (1937)Google Scholar
  15. 15.
    Herman, G.T.: Image Reconstruction from Projections - The fundamentals of computerized tomography. Academic Press, New York (1980)zbMATHGoogle Scholar
  16. 16.
    Popa, C., Zdunek, R.: Kaczmarz extended algorithm for tomographic image reconstruction from limited-data. Mathematics and Computers in Simulation 65(6), 579–598 (2004)CrossRefMathSciNetGoogle Scholar
  17. 17.
    Tanabe, K.: Projection method for solving a singular system of linear equations and its applications. Numer. Math. 17, 203–214 (1971)CrossRefzbMATHMathSciNetGoogle Scholar
  18. 18.
    Popa, C., Zdunek, R.: On some constraining strategies in image reconstructions from projections. In: Proc. of 5th Workshop on Mathematical Modelling of Environmental and Life Sciences Problems, Constantza, Romania, Editura Academiei Romane (2006)Google Scholar
  19. 19.
    Zdunek, R.: Regularized nonnegative matrix factorization: Geometrical interpretation and application to spectral unmixing. International Journal of Applied Mathematics and Computer Science 24(2), 233–247 (2014)CrossRefzbMATHGoogle Scholar
  20. 20.
    Kim, H., Park, H.: Non-negative matrix factorization based on alternating non-negativity constrained least squares and active set method. SIAM Journal in Matrix Analysis and Applications 30(2), 713–730 (2008)zbMATHGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Rafał Zdunek
    • 1
  1. 1.Department of ElectronicsWroclaw University of TechnologyWroclawPoland

Personalised recommendations