Abstract
The nonnegative matrix factorization (NMF) is a bound-constrained low-rank approximation technique for nonnegative multivariate data. NMF has been studied extensively over the last years, but an important aspect which only has received little attention so far is a proper initialization of the NMF factors in order to achieve a faster error reduction. Since the NMF objective function is usually non-differentiable, discontinuous, and may possess many local minima, heuristic search algorithms are a promising choice as initialization enhancers for NMF.
In this paper we investigate the application of five population based algorithms (genetic algorithms, particle swarm optimization, fish school search, differential evolution, and fireworks algorithm) as new initialization variants for NMF. Experimental evaluation shows that some of them are well suited as initialization enhancers and can reduce the number of NMF iterations needed to achieve a given accuracy. Moreover, we compare the general applicability of these five optimization algorithms for continuous optimization problems, such as the NMF objective function.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Lee, D.D., Seung, H.S.: Learning parts of objects by non-negative matrix factorization. Nature 401(6755), 788–791 (1999)
Berry, M.W., Browne, M., Langville, A.N., Pauca, P.V., Plemmons, R.J.: Algorithms and applications for approximate nonnegative matrix factorization. Computational Statistics & Data Analysis 52(1), 155–173 (2007)
Boutsidis, C., Gallopoulos, E.: SVD based initialization: A head start for nonnegative matrix factorization. Pattern Recogn. 41(4), 1350–1362 (2008)
Wild, S.M., Curry, J.H., Dougherty, A.: Improving non-negative matrix factorizations through structured initialization. Patt. Recog. 37(11), 2217–2232 (2004)
Xue, Y., Tong, C.S., Chen, Y., Chen, W.: Clustering-based initialization for non-negative matrix factorization. Appl. Math. & Comput. 205(2), 525–536 (2008)
Kim, H., Park, H.: Nonnegative matrix factorization based on alternating nonnegativity constrained least squares and active set method. SIAM J. Matrix Anal. Appl. 30, 713–730 (2008)
Janecek, A.G., Gansterer, W.N.: Utilizing nonnegative matrix factorization for e-mail classification problems. In: Berry, M.W., Kogan, J. (eds.) Survey of Text Mining III: Application and Theory. John Wiley & Sons, Inc., Chichester (2010)
Stadlthanner, K., Lutter, D., Theis, F., et al.: Sparse nonnegative matrix factorization with genetic algorithms for microarray analysis. In: IJCNN 2007: Proceedings of the International Joint Conference on Neural Networks, pp. 294–299 (2007)
Snásel, V., Platos, J., Krömer, P.: Developing genetic algorithms for boolean matrix factorization. In: DATESO 2008 (2008)
Lin, C.J.: Projected gradient methods for nonnegative matrix factorization. Neural Comput. 19(10), 2756–2779 (2007)
Schmidt, M.N., Laurberg, H.: Non-negative matrix factorization with Gaussian process priors. Comp. Intelligence and Neuroscience (1), 1–10 (2008)
Goldberg, D.E.: Genetic Algorithms in Search, Optimization and Machine Learning, 1st edn. Addison-Wesley Longman, Amsterdam (1989)
Kennedy, J., Eberhart, R.: Particle swarm optimization. In: Proceedings of IEEE International Conference on Neural Networks, vol. 4, pp. 1942–1948 (1995)
Price, K.V., Storn, R.M., Lampinen, J.A.: Differential Evolution A Practical Approach to Global Optimization. Springer, Heidelberg (2005)
Filho, C.J.A.B., de Lima Neto, F.B., Lins, A.J.C.C., Nascimento, A.I.S., Lima, M.P.: Fish school search. In: Chiong, R. (ed.) Nature-Inspired Algorithms for Optimisation. SCI, vol. 193, pp. 261–277. Springer, Heidelberg (2009)
Janecek, A.G., Tan, Y.: Feeding the fish – weight update strategies for the fish school search algorithm. To appear in Proceedings of ICSI 2011: 2nd International Conference on Swarm Intelligence (2011)
Tan, Y., Zhu, Y.: Fireworks algorithm for optimization. In: Tan, Y., Shi, Y., Tan, K.C. (eds.) ICSI 2010. LNCS, vol. 6145, pp. 355–364. Springer, Heidelberg (2010)
Berry, M.W., Drmac, Z., Jessup, E.R.: Matrices, vector spaces, and information retrieval. SIAM Review 41(2), 335–362 (1999)
Pedersen, M.E.H.: SwarmOps - numerical & heuristic optimization for matlab (2010), http://www.hvass-labs.org/projects/swarmops/matlab
Bratton, D., Kennedy, J.: Defining a standard for particle swarm optimization. In: Swarm Intelligence Symposium, SIS 2007, pp. 120–127. IEEE, Los Alamitos (2007)
Haupt, R.L., Haupt, S.E.: Practical Genetic Algorithms, 2nd edn. John Wiley & Sons, Inc., Chichester (2005)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Janecek, A., Tan, Y. (2011). Using Population Based Algorithms for Initializing Nonnegative Matrix Factorization. In: Tan, Y., Shi, Y., Chai, Y., Wang, G. (eds) Advances in Swarm Intelligence. ICSI 2011. Lecture Notes in Computer Science, vol 6729. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-21524-7_37
Download citation
DOI: https://doi.org/10.1007/978-3-642-21524-7_37
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-21523-0
Online ISBN: 978-3-642-21524-7
eBook Packages: Computer ScienceComputer Science (R0)