Advertisement

Decomposition in Transition: Adaptive Matrix Factorization

  • Alexander Paprotny
  • Michael Thess
Chapter
First Online:
  • 1.7k Downloads
Part of the Applied and Numerical Harmonic Analysis book series (ANHA)

Abstract

We introduce SVD/PCA-based matrix factorization frameworks and present applications to prediction-based recommendation. Furthermore, we devise incremental algorithms that enable to compute the considered factorizations adaptively in a realtime setting. Besides SVD and PCA-based frameworks, we discuss more sophisticated approaches like non-negative matrix factorization and Lanczos-based methods and assess their effectiveness by means of experiments on real-world data. Moreover, we address a compressive sensing-based approach to Netflix-like matrix completion problems and conclude the chapter by proposing a remedy to complexity issues in computing large elements of the low-rank matrices, which, as we shall see, is a recurring problem related to factorization-based prediction methods.

Keywords

Matrix Factorization Singular Vector Collaborative Filter Nonnegative Matrix Factorization Matrix Completion 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
This is a preview of subscription content, log in to check access.

References

  1. [Beb08]
    Bebendorf, M.: Hierarchical Matrices: A Means to Efficiently Solve Elliptic Boundary Value Problems. Lecture Notes in Computational Science and Engineering (LNCSE), vol. 63. Springer-Verlag, Berlin (2008)Google Scholar
  2. [BNS78]
    Bunch, J.R., Nielsen, C.P., Sorensen, D.C.: Rank-one modification of the symmetric eigenproblem. Numer. Math. 31, 31–48 (1978)CrossRefzbMATHMathSciNetGoogle Scholar
  3. [Bra02]
    Brand, M.E.: Incremental singular value decomposition of uncertain data with missing values. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) Computer Vision ECCV. Lecture Notes in Computer Science, vol. 2350, pp. 707–720. Springer, Berlin/Heidelberg (2002)Google Scholar
  4. [Bra03]
    Brand, M.E.: Fast online svd revisions for lightweight recommender systems. In: SIAM International Conference on Data Mining (SDM) (2003)Google Scholar
  5. [Bra06]
    Brand, M.E.: Fast low-rank modifications of the thin singular value decomposition. Linear Algebra Appl. 415, 20–30 (2006)CrossRefzbMATHMathSciNetGoogle Scholar
  6. [CR08]
    Candès, E.J., Recht, B.: Exact matrix completion via convex optimization. Found. Comput. Math. 9, 717–772 (2008)CrossRefGoogle Scholar
  7. [CRT06]
    Candès, E.J., Romberg, J., Tao, T.: Stable signal recovery from incomplete and inaccurate measurements. Comm. Pure Appl. Math. 59, 1207–1223 (2006)CrossRefzbMATHMathSciNetGoogle Scholar
  8. [CS09]
    Chen, J., Saad, Y.: Lanczos vectors versus singular vectors for effective dimension reduction. IEEE Trans. Knowl. Data Eng. 21, 1091–1103 (2009)CrossRefGoogle Scholar
  9. [CT10]
    Candès, E.J., Tao, T.: The power of convex relaxation: near-optimal matrix completion. IEEE Trans. Info. Theor. 56(5), 2053–2080 (2010)CrossRefGoogle Scholar
  10. [DHS05]
    Ding, C., He, X., Simon, H.D.: On the equivalence of nonnegative matrix factorization and spectral clustering. Proc. SIAM Data Mining Conf. 4, 606–610 (2005)Google Scholar
  11. [DLJ10]
    Ding, C., Li, T., Jordan, M.I.: Convex and semi-nonnegative matrix factorizations. IEEE Trans. Pattern Anal. Mach. Intell. 32(1), 45–55 (2010)CrossRefGoogle Scholar
  12. [Don06]
    Donoho, D.L.: Compressed sensing. IEEE Trans. Info.Theor. 52(4), 1289–1306 (2006)CrossRefMathSciNetGoogle Scholar
  13. [Fun06]
    Funk, S.: Netflix update: try this at home. http://sifter.org/~simon/journal/20061211.html
  14. [GNBT92]
    Goldberg, D., Nichols, D., Oki, B.M., Terry, D.: Using collaborative filtering to weave an information tapestry. Commun. ACM 35(12), 61–70 (1992). Special issue on information filteringCrossRefGoogle Scholar
  15. [GH03]
    Grasedyck, L., Hackbusch, W.: Construction and arithmetics of H-matrices. Computing 70, 295–334 (2003)CrossRefzbMATHMathSciNetGoogle Scholar
  16. [GE94]
    Gu, M., Eisenstat, S.C.: A stable and efficient algorithm for the rank-one modification of the symmetric eigenproblem. SIAM J. Matrix Anal. Appl. 15, 1266–1276 (1994)CrossRefzbMATHMathSciNetGoogle Scholar
  17. [Ha99]
    Hackbusch, W.: A sparse matrix arithmetic based on H-matrices. Part I: introduction to H-matrices. Computing 62, 89–108 (1999)CrossRefzbMATHMathSciNetGoogle Scholar
  18. [HK00]
    Hackbusch, W., Khoromskij, B.N.: A sparse H-matrix arithmetic. Part II: application to multi-dimensional problems. Computing 64, 21–47 (2000)zbMATHMathSciNetGoogle Scholar
  19. [HJ85]
    Horn, R.A., Johnson, C.R.: Matrix Analysis. Cambridge University Press, New York (1985)CrossRefzbMATHGoogle Scholar
  20. [LSY03]
    Linden, G., Smith, B., York, J.: Amazon.com recommendations: item-to-item collaborative filtering. IEEE Internet Comput. (2003)Google Scholar
  21. [MHT10]
    Mazumder, R., Hastie, T., Tibshirani, R.: Spectral regularization algorithms for learning large incomplete matrices. J. Mach. Learn. Res. 11, 2287–2322 (2010)zbMATHMathSciNetGoogle Scholar
  22. [Pap09]
    Paprotny A.: Praktikumsbericht zum Fachpraktikum bei der Firma prudsys AG. (in German) Report. TU Hamburg-Harburg (2009)Google Scholar
  23. [RFP10]
    Recht, B., Fazel, M., Parrilo, P.A.: Guaranteed minimum rank solutions of matrix equations via nuclear norm minimization. SIAM Rev. 52(3), 471–501 (2010)CrossRefzbMATHMathSciNetGoogle Scholar
  24. [RS05]
    Rennie J.D.M., Srebro N.: Fast maximum margin matrix factorization for collaborative prediction. In: Proceedings of the 22nd International Conference on Machine Learning, pp. 713–719. ACM (2005)Google Scholar
  25. [SHB05]
    Shani, G., Heckerman, D., Brafman, R.I.: An MDP-based recommender system. J. Mach. Learn. Res. 6, 1265–1295 (2005)zbMATHMathSciNetGoogle Scholar
  26. [VB96]
    Vandenberghe, L., Boyd, S.: Semidefinite programming. SIAM Rev. 38, 49–95 (1996)CrossRefzbMATHMathSciNetGoogle Scholar
  27. [ZWSP08]
    Zhou, Y., Wilkinson, D., Schreiber, R., Pan, R.: Large-scale parallel collaborative filtering for the netflix prize. In: Proceedings of 4th International Conference Algorithmic Aspects in Information and Management, LNCS 5034 (2008)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2013

Authors and Affiliations

  • Alexander Paprotny
    • 1
  • Michael Thess
    • 2
  1. 1.Research and Developmentprudsys AGBerlinGermany
  2. 2.Research and Developmentprudsys AGChemnitzGermany

Personalised recommendations

Cite chapter

Buy options