Abstract
Nonnegative Matrix Factorization (NMF) is a dimensionality reduction method for representing nonnegative data in a low-dimensional nonnegative space. NMF problems are usually solved with an alternating minimization of a given objective function, using nonnegativity constrained optimization algorithms. This paper is concerned with the projected trust-region algorithm that is adapted to minimize a family of divergences or statistical distances, such as α- or β-divergences that are efficient for solving NMF problems. Using the Cauchy point estimate for the quadratic approximation model, a radius of the trust-region can be estimated efficiently for a symmetric and block-diagonal structure of the corresponding Hessian matrices. The experiments demonstrate a high efficiency of the proposed approach.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Bardsley, J.M.: A nonnegatively constrained trust region algorithm for the restoration of images with an unknown blur. Electronic Transactions on Numerical Analysis 20, 139–153 (2005)
Berry, M., Browne, M., Langville, A.N., Pauca, P., Plemmons, R.J.: Algorithms and applications for approximate nonnegative matrix factorization. Computational Statistics and Data Analysis 52(1), 155–173 (2007)
Cichocki, A., Zdunek, R.: NMFLAB for Signal and Image Processing. Tech. rep., Laboratory for Advanced Brain Signal Processing, BSI, RIKEN, Saitama, Japan (2006), http://www.bsp.brain.riken.jp
Cichocki, A., Zdunek, R., Phan, A.H., Amari, S.I.: Nonnegative Matrix and Tensor Factorizations: Applications to Exploratory Multi-way Data Analysis and Blind Source Separation. Wiley and Sons (2009)
Févotte, C., Bertin, N., Durrieu, J.L.: Nonnegative matrix factorization with the Itakura-Saito divergence: With application to music analysis. Neural Computation 21(3), 793–830 (2009)
Févotte, C., Idier, J.: Algorithms for nonnegative matrix factorization with the beta-divergence. Neural Computation 13(3), 1–24 (2010)
Guillamet, D., Vitrià, J., Schiele, B.: Introducing a weighted nonnegative matrix factorization for image classification. Pattern Recognition Letters 24(14), 2447–2454 (2003)
Heiler, M., Schnoerr, C.: Learning sparse representations by non-negative matrix factorization and sequential cone programming. Journal of Machine Learning Research 7, 1385–1407 (2006)
Lee, D.D., Seung, H.S.: Learning the parts of objects by non-negative matrix factorization. Nature 401, 788–791 (1999)
Li, S.Z., Hou, X.W., Zhang, H.J., Cheng, Q.S.: Learning spatially localized, parts-based representation. In: Proc. of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2001), vol. 1, pp. I–207–I–212 (2001)
Lin, C.J.: Projected gradient methods for non-negative matrix factorization. Neural Computation 19(10), 2756–2779 (2007)
Mauthner, T., Roth, P.M., Bischof, H.: Instant Action Recognition. In: Salberg, A.-B., Hardeberg, J.Y., Jenssen, R. (eds.) SCIA 2009. LNCS, vol. 5575, pp. 1–10. Springer, Heidelberg (2009)
Minka, T.: Divergence measures and message passing. Tech. Rep. MSR-TR-2005-173, Microsoft Research (2005)
Nocedal, J., Wright, S.J.: Numerical Optimization. Springer Series in Operations Research. Springer, New York (1999)
Qin, L., Zheng, Q., Jiang, S., Huang, Q., Gao, W.: Unsupervised texture classification: Automatically discover and classify texture patterns. Image and Vision Computing 26(5), 647–656 (2008)
Rojas, M., Steihaug, T.: An interior-point trust-region-based method for large-scale non-negative regularization. Inverse Problems 18, 1291–1307 (2002)
Wang, F.Y., Chi, C.Y., Chan, T.H., Wang, Y.: Nonnegative least-correlated component analysis for separation of dependent sources by volume maximization. IEEE Transactions Pattern Analysis and Machine Intelligence 32(5), 875–888 (2010)
Zdunek, R., Cichocki, A.: Nonnegative matrix factorization with constrained second-order optimization. Signal Processing 87, 1904–1916 (2007)
Zdunek, R., Cichocki, A.: Nonnegative matrix factorization with quadratic programming. Neurocomputing 71(10-12), 2309–2320 (2008)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Zdunek, R. (2012). Trust-Region Algorithm for Nonnegative Matrix Factorization with Alpha- and Beta-divergences. In: Pinz, A., Pock, T., Bischof, H., Leberl, F. (eds) Pattern Recognition. DAGM/OAGM 2012. Lecture Notes in Computer Science, vol 7476. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-32717-9_23
Download citation
DOI: https://doi.org/10.1007/978-3-642-32717-9_23
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-32716-2
Online ISBN: 978-3-642-32717-9
eBook Packages: Computer ScienceComputer Science (R0)