Skip to main content
SpringerLink
Log in

Trust-Region Strategy with Cauchy Point for Nonnegative Tensor Factorization with Beta-Divergence

  • Conference paper
  • First Online:
Intelligent Decision Technologies (IDT 2020)

Part of the book series: Smart Innovation, Systems and Technologies ((SIST,volume 193))

Included in the following conference series:

Abstract

Nonnegative tensor factorization is a well-known unsupervised learning method for multi-linear feature extraction from a nonnegatively constrained multi-way array. Many computational strategies have been proposed for updating nonnegative factor matrices in this factorization model but they are mostly restricted to minimization of the objective function expressed by the Euclidean distance. Minimization of other functions, such as the beta-divergence, is more challenging and usually leads to higher complexity. In this study, the trust-region (TR) algorithm is used for minimization of the beta-divergence. We noticed that the Cauchy point strategy in the TR algorithm can be simplified for this function, which is profitable for updating the factors in the discussed model. The experiments show high efficiency of the proposed approach.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 169.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 219.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Bardsley, J.M.: A nonnegatively constrained trust region algorithm for the restoration of images with an unknown blur. Electron. Trans. Numer. Anal. 20, 139–153 (2005)

    MathSciNet  MATH  Google Scholar 

  2. Basu, A., Harris, I.R., Hjort, N.L., Jones, M.C.: Robust and efficient estimation by minimising a density power divergence. Biometrika 85(3), 549–559 (1998)

    Article  MathSciNet  Google Scholar 

  3. Carroll, J.D., Chang, J.J.: Analysis of individual differences in multidimensional scaling via an n-way generalization of Eckart-Young decomposition. Psychometrika 35, 283–319 (1970)

    Article  Google Scholar 

  4. Cichocki, A., Zdunek, R., Phan, A.H., Amari, S.I.: Nonnegative Matrix and Tensor Factorizations: Applications to Exploratory Multi-way Data Analysis and Blind Source Separation. Wiley and Sons (2009)

    Google Scholar 

  5. Cichocki, A., Cruces, S., Amari, S.I.: Generalized alpha-beta divergences and their application to robust nonnegative matrix factorization. Entropy 13 (2011)

    Google Scholar 

  6. Cichocki, A., Phan, A.H.: Fast local algorithms for large scale nonnegative matrix and tensor factorizations. IEICE Trans. 92-A(3), 708–721 (2009)

    Google Scholar 

  7. Cichocki, A., Zdunek, R., Choi, S., Plemmons, R.J., Amari, S.: Non-negative tensor factorization using alpha and beta divergences. In: Proceedings of ICASSP 2007, Honolulu, Hawaii, USA, April 15–20, pp. 1393–1396 (2007)

    Google Scholar 

  8. Eguchi, S., Kano, Y.: Robustifying maximum likelihood estimation by psi-divergence. Institute of Statistical Mathematics (2001) (Res. Memorandum 802)

    Google Scholar 

  9. FitzGerald, D., Cranitch, M., Coyle, E.: Extended nonnegative tensor factorisation models for musical sound source separation. Comput. Intell. Neurosci. 872425, 1–15 (2008)

    Article  Google Scholar 

  10. Friedlander, M.P., Hatz, K.: Computing nonnegative tensor factorizations. Comput. Optim. Appl. 23(4), 631–647 (2008)

    MATH  Google Scholar 

  11. Harshman, R.A.: Foundations of the PARAFAC procedure: Models and conditions for an “explanatory” multimodal factor analysis. UCLA Working Papers in Phonetics 16, 1–84 (1970)

    Google Scholar 

  12. Heiler, M., Schnoerr, C.: Controlling sparseness in non-negative tensor factorization. Springer LNCS, vol. 3951, pp. 56–67 (2006)

    Google Scholar 

  13. Kim, H., Park, H., Elden, L.: Non-negative tensor factorization based on alternating large-scale non-negativity-constrained least squares. In: Proceedings of BIBE’2007, pp. 1147–1151 (2007)

    Google Scholar 

  14. Kolda, T.G., Bader, B.W.: Tensor decompositions and applications. SIAM Rev. 51(3), 455–500 (2009)

    Article  MathSciNet  Google Scholar 

  15. Lee, D.D., Seung, H.S.: Learning the parts of objects by non-negative matrix factorization. Nature 401, 788–791 (1999)

    Article  Google Scholar 

  16. Mørup, M., Hansen, L.K., Arnfred, S.M.: Algorithms for sparse nonnegative Tucker decompositions. Neural Comput. 20(8), 2112–2131 (2008)

    Article  Google Scholar 

  17. Nocedal, J., Wright, S.J.: Numerical Optimization. Springer Series in Operations Research. Springer, New York (1999)

    Book  Google Scholar 

  18. Rojas, M., Steihaug, T.: An interior-point trust-region-based method for large-scale non-negative regularization. Inverse Probl. 18, 1291–1307 (2002)

    Article  MathSciNet  Google Scholar 

  19. Shashua, A., Hazan, T.: Non-negative tensor factorization with applications to statistics and computer vision. In: Proceedings of the 22th International Conference on Machine Learning. Bonn, Germany (2005)

    Google Scholar 

  20. Shashua, A., Zass, R., Hazan, T.: Multi-way clustering using super-symmetric non-negative tensor factorization. In: European Conference on Computer Vision (ECCV). Graz, Austria (May 2006)

    Google Scholar 

  21. Simsekli, U., Virtanen, T., Cemgil, A.T.: Non-negative tensor factorization models for bayesian audio processing. Digital Signal Process. 47, 178–191 (2015)

    Article  MathSciNet  Google Scholar 

  22. Xiong, F., Zhou, J., Qian, Y.: Hyperspectral imagery denoising via reweighed sparse low-rank nonnegative tensor factorization. In: 25th IEEE International Conference on Image Processing (ICIP), pp. 3219–3223 (Oct 2018)

    Google Scholar 

  23. Zdunek, R., Cichocki, A.: Nonnegative matrix factorization with quadratic programming. Neurocomputing 71(10–12), 2309–2320 (2008)

    Article  Google Scholar 

  24. Zdunek, R.: Trust-region algorithm for nonnegative matrix factorization with alpha- and beta-divergences. In: Proceedings of DAGM/OAGM, LNCS, vol. 7476, pp. 226–235. Graz, Austria (28–31 Aug 2012)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Faculty of Electronics, Wroclaw University of Science and Technology, Wybrzeze Wyspianskiego 27, 50-370, Wroclaw, Poland

    Rafał Zdunek & Krzysztof Fonał

Authors
  1. Rafał Zdunek
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Krzysztof Fonał
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Rafał Zdunek .

Editor information

Editors and Affiliations

  1. Gdynia Maritime University, Gdynia, Poland

    Ireneusz Czarnowski

  2. KES International Research, UK

    Robert J. Howlett

  3. Faculty of Engineering and Information Technology, Centre for Artificial Intelligence, University of Technology Sydney, Sydney, NSW, Australia

    Lakhmi C. Jain

Rights and permissions

Reprints and permissions

Copyright information

© 2020 The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Zdunek, R., Fonał, K. (2020). Trust-Region Strategy with Cauchy Point for Nonnegative Tensor Factorization with Beta-Divergence. In: Czarnowski, I., Howlett, R., Jain, L. (eds) Intelligent Decision Technologies. IDT 2020. Smart Innovation, Systems and Technologies, vol 193. Springer, Singapore. https://doi.org/10.1007/978-981-15-5925-9_27

Download citation

Publish with us

Policies and ethics

Search

Navigation