Abstract
Just as multiplication can be generalized from scalars to matrices, the notion of factorization can also be generalized from scalars to matrices. Exact matrix factorizations need to satisfy the size and rank constraints that are imposed on matrix multiplication. For example, when an n × d matrix A is factorized into two matrices B and C (i.e., A = BC), the matrices B and C must be of sizes n × k and k × d for some constant k. For exact factorization to occur, the value of k must be equal to at least the rank of A. This is because the rank of A is at most equal to the minimum of the ranks of B and C. In practice, it is common to perform approximate factorization with much smaller values of k than the rank of A.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
This will occur under the assumption that the top-k eigenvalues of D TD are distinct. Tied eigenvalues result in a non-unique solution for SVD, which might sometimes result in some differences in the subspace corresponding to the smallest eigenvalue within the rank-k solution.
- 2.
Strictly speaking, the objective function is not defined when p ij is 0 or 1. However, the loss is zero when p ij → x ij in the limit. A logistic function will never yield values of exactly 0 or 1 for p ij.
- 3.
The libraries libFM and libMF are different.
References
C. Aggarwal. Machine learning for text. Springer, 2018.
C. Aggarwal. Recommender systems: The textbook. Springer, 2016.
I. Bayer. Fastfm: a library for factorization machines. arXiv preprint arXiv:1505.00641, 2015. https://arxiv.org/pdf/1505.00641v2.pdf
C. Ding, T. Li, and W. Peng. On the equivalence between non-negative matrix factorization and probabilistic latent semantic indexing. Computational Statistics and Data Analysis, 52(8), pp. 3913–3927, 2008.
C. Freudenthaler, L. Schmidt-Thieme, and S. Rendle. Factorization machines: Factorized polynomial regression models. GPSDAA, 2011.
E. Gaussier and C. Goutte. Relation between PLSA and NMF and implications. ACM SIGIR Conference, pp. 601–602, 2005.
A. Grover and J. Leskovec. node2vec: Scalable feature learning for networks. ACM KDD Conference, pp. 855–864, 2016.
T. Hofmann. Probabilistic latent semantic indexing. ACM SIGIR Conference, pp. 50–57, 1999.
Y. Hu, Y. Koren, and C. Volinsky. Collaborative filtering for implicit feedback datasets. IEEE ICDM, pp. 263–272, 2008.
P. Jain, P. Netrapalli, and S. Sanghavi. Low-rank matrix completion using alternating minimization. ACM Symposium on Theory of Computing, pp. 665–674, 2013.
C. Johnson. Logistic matrix factorization for implicit feedback data. NIPS Conference, 2014.
Y. Koren, R. Bell, and C. Volinsky. Matrix factorization techniques for recommender systems. Computer, 8, pp. 30–37, 2009.
A. Langville, C. Meyer, R. Albright, J. Cox, and D. Duling. Initializations for the nonnegative matrix factorization. ACM KDD Conference, pp. 23–26, 2006.
D. Lay, S. Lay, and J. McDonald. Linear Algebra and its applications, Pearson, 2012.
D. Lee and H. Seung. Algorithms for non-negative matrix factorization. Advances in Neural Information Processing Systems, pp. 556–562, 2001.
P. McCullagh. Regression models for ordinal data. Journal of the royal statistical society. Series B (Methodological), pp. 109–142, 1980.
T. Mikolov, K. Chen, G. Corrado, and J. Dean. Efficient estimation of word representations in vector space. arXiv:1301.3781, 2013. https://arxiv.org/abs/1301.3781
T. Mikolov, I. Sutskever, K. Chen, G. Corrado, and J. Dean. Distributed representations of words and phrases and their compositionality. NIPS Conference, pp. 3111–3119, 2013.
J. Pennington, R. Socher, and C. Manning. Glove: Global Vectors for Word Representation. EMNLP, pp. 1532–1543, 2014.
B. Perozzi, R. Al-Rfou, and S. Skiena. Deepwalk: Online learning of social representations. ACM KDD Conference, pp. 701–710, 2014.
S. Rendle. Factorization machines. IEEE ICDM Conference, pp. 995–100, 2010.
S. Rendle. Factorization machines with libfm. ACM Transactions on Intelligent Systems and Technology, 3(3), 57, 2012.
A. Singh and G. Gordon. A unified view of matrix factorization models. Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 358–373, 2008.
N. Srebro, J. Rennie, and T. Jaakkola. Maximum-margin matrix factorization. Advances in neural information processing systems, pp. 1329–1336, 2004.
G. Strang. An introduction to linear algebra, Fifth Edition. Wellseley-Cambridge Press, 2016.
G. Strang. Linear algebra and its applications, Fourth Edition. Brooks Cole, 2011.
M. Udell, C. Horn, R. Zadeh, and S. Boyd. Generalized low rank models. Foundations and Trends in Machine Learning, 9(1), pp. 1–118, 2016. https://github.com/madeleineudell/LowRankModels.jl
H. Wendland. Numerical linear algebra: An introduction. Cambridge University Press, 2018.
H. Yu, C. Hsieh, S. Si, and I. S. Dhillon. Scalable coordinate descent approaches to parallel matrix factorization for recommender systems. IEEE ICDM, pp. 765–774, 2012.
Y. Zhou, D. Wilkinson, R. Schreiber, and R. Pan. Large-scale parallel collaborative filtering for the Netflix prize. Algorithmic Aspects in Information and Management, pp. 337–348, 2008.
Author information
Authors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this chapter
Cite this chapter
Aggarwal, C.C. (2020). Matrix Factorization. In: Linear Algebra and Optimization for Machine Learning. Springer, Cham. https://doi.org/10.1007/978-3-030-40344-7_8
Download citation
DOI: https://doi.org/10.1007/978-3-030-40344-7_8
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-40343-0
Online ISBN: 978-3-030-40344-7
eBook Packages: Computer ScienceComputer Science (R0)