Abstract
Nonnegative Matrix Factorization (NMF) is a commonly-used unsupervised learning method for extracting parts-based features and dimensionality reduction from nonnegative data. Many computational algorithms exist for updating the latent nonnegative factors in NMF. In this study, we propose an extension of the Hierarchical Alternating Least Squares (HALS) algorithm to a distributed version using the state-of-the-art framework - Apache Spark. Spark gains its popularity among other distributed computational frameworks because of its in-memory approach which works much faster than well-known Apache Hadoop. The scalability and efficiency of the proposed algorithm is confirmed in the numerical experiments, performed on real data as well as synthetic ones.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
- 7.
Whenever we mention the ALS in this study, we refer to the distributed ALS implementation from MLlib in the ML package. This is an important note because there is also an older implementation in the mllib package.
- 8.
- 9.
References
Jolliffe, I.T.: Principal Component Analysis. Springer Series in Statistics, 2nd edn. Springer, New York (2002). https://doi.org/10.1007/978-1-4757-1904-8
Cichocki, A., Zdunek, R., Phan, A.H., Amari, S.I.: Nonnegative Matrix and Tensor Factorizations: Applications to Exploratory Multi-way Data Analysis and Blind Source Separation. Wiley, Hoboken (2009)
Lee, D.D., Seung, H.S.: Learning the parts of objects by non-negative matrix factorization. Nature 401, 788–791 (1999)
Liu, C., Yang, H.C., Fan, J., He, L.W., Wang, Y.M.: Distributed nonnegative matrix factorization for web-scale dyadic data analysis on MapReduce. In: Proceedings of 19th International Conference on World Wide Web. WWW 2010, pp. 681–690. ACM, New York (2010)
Sun, Z., Li, T., Rishe, N.: Large-scale matrix factorization using MapReduce. In: ICDM Workshops, pp. 1242–1248. IEEE Computer Society (2010)
Ding, C., Li, T., Jordan, M.I.: Convex and semi-nonnegative matrix factorizations. IEEE Trans. Pattern Anal. Mach. Intell. 32(1), 45–55 (2010)
Cichocki, A., Zdunek, R., Amari, S.: Hierarchical ALS algorithms for nonnegative matrix and 3D tensor factorization. In: Davies, M.E., James, C.J., Abdallah, S.A., Plumbley, M.D. (eds.) ICA 2007. LNCS, vol. 4666, pp. 169–176. Springer, Heidelberg (2007). https://doi.org/10.1007/978-3-540-74494-8_22
Han, L., Neumann, M., Prasad, U.: Alternating projected Barzilai-Borwein methods for nonnegative matrix factorization. Electron. Trans. Numer. Anal. 36, 54–82 (2009–2010)
Kim, J., Park, H.: Fast nonnegative matrix factorization: an active-set-like method and comparisons. SIAM J. Sci. Comput. 33(6), 3261–3281 (2011)
Gillis, N., Glineur, F.: Accelerated multiplicative updates and hierarchical ALS algorithms for nonnegative matrix factorization. Neural Comput. 24(4), 1085–1105 (2012)
Chen, W., Guillaume, M.: HALS-based NMF with flexible constraints for hyperspectral unmixing. EURASIP J. Adv. Signal Process. 54, 1–14 (2012)
Laudadio, T., Croitor Sava, A.R., Sima, D.M., Wright, A.J., Heerschap, A., Mastronardi, N., Van Huffel, S.: Hierarchical non-negative matrix factorization applied to three-dimensional 3T MRSI data for automatic tissue characterization of the prostate. NMR Biomed. 29(6), 751–758 (2016)
Zdunek, R., Fonal, K.: Distributed nonnegative matrix factorization with HALS algorithm on MapReduce. In: Ibrahim, S., Choo, K.-K.R., Yan, Z., Pedrycz, W. (eds.) ICA3PP 2017. LNCS, vol. 10393, pp. 211–222. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-65482-9_14
Zaharia, M., Chowdhury, M., Das, T., Dave, A., Ma, J., McCauly, M., Franklin, M.J., Shenker, S., Stoica, I.: Resilient distributed datasets: a fault-tolerant abstraction for in-memory cluster computing. In: Gribble, S.D., Katabi, D. (eds.) Proceedings of the 9th USENIX Symposium on Networked Systems Design and Implementation, NSDI 2012, San Jose, CA, USA, 25–27 April 2012, pp. 15–28. USENIX Association (2012)
Cichocki, A., Phan, A.H.: Fast local algorithms for large scale nonnegative matrix and tensor factorizations. IEICE Trans. Fundam. Electron. Commun. Comput. Sci. E92–A(3), 708–721 (2009)
Harper, F.M., Konstan, J.A.: The movielens datasets: history and context. ACM Trans. Interact. Intell. Syst. 5(4), 19:1–19:19 (2015)
Acknowledgment
This work was supported by the grant 2015/17/B/ST6/01865 funded by National Science Center (NCN) in Poland.
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG, part of Springer Nature
About this paper
Cite this paper
Fonał, K., Zdunek, R. (2018). Distributed Nonnegative Matrix Factorization with HALS Algorithm on Apache Spark. In: Rutkowski, L., Scherer, R., Korytkowski, M., Pedrycz, W., Tadeusiewicz, R., Zurada, J. (eds) Artificial Intelligence and Soft Computing. ICAISC 2018. Lecture Notes in Computer Science(), vol 10842. Springer, Cham. https://doi.org/10.1007/978-3-319-91262-2_30
Download citation
DOI: https://doi.org/10.1007/978-3-319-91262-2_30
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-91261-5
Online ISBN: 978-3-319-91262-2
eBook Packages: Computer ScienceComputer Science (R0)