Skip to main content
SpringerLink
Log in
Book cover

Compressed Sensing & Sparse Filtering pp 123–148Cite as

Nonnegative Tensor Decomposition

  • Chapter
  • First Online:

Part of the book series: Signals and Communication Technology ((SCT))

Abstract

It is more and more common to encounter applications where the collected data is most naturally stored or represented in a multi-dimensional array, known as a tensor. The goal is often to approximate this tensor as a sum of some type of combination of basic elements, where the notation of what is a basic element is specific to the type of factorization employed. If the number of terms in the combination is few, the tensor factorization gives (implicitly) a sparse (approximate) representation of the data. The terms (e.g. vectors, matrices, tensors) in the combination themselves may also be sparse. This chapter highlights recent developments in the area of non-negative tensor factorization which admit such sparse representations. Specifically, we consider the approximate factorization of third and fourth order tensors into non-negative sums of types of outer-products of objects with one dimension less using the so-called t-product. A demonstration on an application in facial recognition shows the potential promise of the overall approach. We discuss a number of algorithmic options for solving the resulting optimization problems, and modification of such algorithms for increasing the sparsity.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD   169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    R. Harshman, ’70; J. Carroll and J. Chang, ’70.

  2. 2.

    If \({{\fancyscript{A}}}\) is a fourth order tensor, the core tensor will be fourth order and there will be an additional summand and vector outer product.

References

  1. Alex M, Alex OM (2002) Vasilescu, and Demetri Terzopoulos. Multilinear image analysis for facial recognition

    Google Scholar 

  2. Anderson DG (1965) Iterative procedures for nonlinear integral equations. J ACM 12(4):547–560

    Article  MATH  Google Scholar 

  3. Andersson CA, Bro R (1998) Improving the speed of multi-way algorithms: part i. tucker3. Chemometr Intell Lab Syst 42:93–103

    Article  Google Scholar 

  4. Arora S, Ge R, Kannan R, Moitra A (2012) Computing a nonnegative matrix factorization - provably. In: Proceedings of the 44th symposium on theory of computing, STOC ’12 ACM, New York, 145–162

    Google Scholar 

  5. Berry MW, Browne M,Langville AN, Paul Pauca V, Plemmons RJ (2006) Algorithms and applications for approximate nonnegative matrix factorization. In: Computational statistics and data analysis, pp 155–173

    Google Scholar 

  6. Boyd S. Sequential convex programming. Lecture slides

    Google Scholar 

  7. Braman K (2010) Third-order tensors as linear operators on a space of matrices. Linear Algebra Appl 433(7):1241–1253, Dec 2010

    Google Scholar 

  8. Bro R, De Jong S (1997) A fast non-negativity-constrained least squares algorithm. J Chemom 11:393–401

    Article  Google Scholar 

  9. Chi EC, Kolda TG (2012) On tensors, sparsity, and nonnegative factorizations. http://arxiv.org/abs/1112.2414. August 2012 (AarXiv:1112.2414 [math.NA])

    Google Scholar 

  10. Cichocki A, Phan AH, Zdunek R, Amari S (2009) Nonnegative matrix and tensor factorizations: applications to exploratory multiway data analysis and blind source separation . Wiley, NJ (Preprint)

    Google Scholar 

  11. Cichocki A, Zdunek R, Choi S, Plemmons R, Amari S (2007) Nonnegative tensor factorization using alpha and beta divergencies. In: Proceedings of the 32nd international conference on acoustics, speech, and signal processing (ICASSP), Honolulu, April 2007

    Google Scholar 

  12. Cichocki A, Zdunek R, Choi S, Plemmons R, Amari S (2007) Novel multi-layer nonnegative tensor factorization with sparsity constraints. In: 8th international conference on adaptive and natural computing algorithms, Warsaw, MIT Press, Cambridge, April 2007

    Google Scholar 

  13. Donoho D, Stodden V (2003) When does non-negative matrix factorization give correct decomposition into parts?, p 2004. MIT Press, Cambridge

    Google Scholar 

  14. Elden L (2007) Matrix methods in data mining and pattern recognition. Publisher: society for industrial and applied mathematics (April 9 2007)

    Google Scholar 

  15. Friedlander MP, Hatz K (2006) Computing nonnegative tensor factorizations. Technical Report TR-2006-21, Computer Science Department, University of British Columbia

    Google Scholar 

  16. Gillis N, Plemmons RJ (2013) Sparse nonnegative matrix underapproximation and its application to hyperspectral image analysis. Linear Alg Appl 438(10):3991–4007

    Google Scholar 

  17. Gleich DF, Greif C, Varah JM (2012)The power and Arnoldi methods in an algebra of circulants. Numer. Linear Algebra Appl.2012. Published online in Wiley Online Library (www.wileyonlinelibrary.com). doi:10.1002/nla.1845

    Google Scholar 

  18. Hao N, Kilmer ME, Braman K, Hoover RC (2013) Facial recognition using tensor–tensor decompositions, SIAM J Imaging Sci 6(1):437–463

    Google Scholar 

  19. Homer Peng Ni, Walker F (July 2011) Anderson acceleration for fixed-point iterations. SIAM J Numer Anal 49(4):1715–1735

    MATH  Google Scholar 

  20. Kilmer ME, Braman K, Hao N, Hoover RC (2013) Third order tensors as operators on matrices: a theoretical and computational framework with applications in imaging. SIAM J Matrix Anal Appl 34(1):148–172

    Google Scholar 

  21. Kilmer ME, Kolda TG (2011) Approximations of third order tensors as sums of (non-negative) low-rank product cyclic tensors. In: Householder Symposium XVIII Plenary Talk. (there seems to be no live page with the book of abstracts), June 2011

    Google Scholar 

  22. Kilmer ME, MartinCarla D, Perrone L (2008) A third-order generalization of the matrix SVD as a product of third-order tensors. Technical report TR-2008-4, Department of Computer Science, Tufts University

    Google Scholar 

  23. Kilmer ME, Martin CD (2011) Factorization strategies for third-order tensors. Linear Algebra Appl 435(3):641–658. doi:10.1016/j.laa.2010.09.020 (Special Issue in Honor of Stewart’s GW 70th birthday)

    Google Scholar 

  24. Kilmer ME, Martin CD, Perrone L (2008) A third-order generalization of the matrix svd as a product of third-order tensors. In: Tufts computer science technical, report, 10 2008

    Google Scholar 

  25. Kolda TG, Bader BW (2009) Tensor decompositions and applications. SIAM Rev 51(3):455–500

    Article  MathSciNet  MATH  Google Scholar 

  26. Kruskal JB (1989) Rank, decomposition, and uniqueness for 3-way and \(n\)-way arrays. In: Coppi R, Bolasco S (eds) Multiway data analysis. Elsevier, Amsterdam, pp 7–18

    Google Scholar 

  27. Kumar A, Sindhwani V, Kambadur P (2012) Fast conical hull algorithms for near-separable non-negative matrix factorization. arXiv:1210.1190 [stat.ML], Oct 2012

    Google Scholar 

  28. Lee D, Seung H (1999) Learning the parts of objects by non-negative matrix factorization. Nature 401:788791

    Google Scholar 

  29. Lin Chih-Jen (2007) On the convergence of multiplicative update algorithms for nonnegative matrix factorization. IEEE Trans Neural Netw 18(6):1589–1596

    Article  Google Scholar 

  30. Martin CD, Shafer R, LaRue B (2011) A recursive idea for multiplying order-p tensors. SIAM J Sci Comput (submitted July 2011)

    Google Scholar 

  31. Nocedal J, Wright SJ (2006) Numerical optimization, 2nd edn. Springer, New York

    MATH  Google Scholar 

  32. Vasilescu MAO, Terzopoulos D(2002) Multilinear analysis of image ensembles: Tensorfaces. In: Proceedings of the 7th European conference on computer vision ECCV 2002. Lecture notes in computer science, Vol 2350, 447–460

    Google Scholar 

  33. Vasilescu MAO, Terzopoulos D (2002) Multilinear image analysis for face recognition. In: Proceedings of the International conference on patter recognition ICPR 2002, vol 2. Quebec City, pp 511–514

    Google Scholar 

  34. Vasilescu MAO, Terzopoulos D (2003) Multilinear subspace analysis of image ensembles. In: Proceedings of the 2003 IEEE Computer society conference on computer vision and pattern recognition CVPR 2003, 93–99

    Google Scholar 

  35. Sylvestre EA, Lawton WH (1971) Self modeling curve resolution. Technometrics 13(3):617–633

    Article  Google Scholar 

Download references

Acknowledgments

We would like to thank Homer F. Walker from the Mathematical Science Department, Worcester Polytechnic Institute for his assistant with Anderson Acceleration. We would also like to acknowledge Hao and Kilmer’s support from NSF-DMS 0914957.

Author information

Authors and Affiliations

  1. Tufts University, 503 Boston Ave, Medford, MA, 02155, USA

    N. Hao & M. E. Kilmer

  2. T. J. Watson Research Center, Yorktown Heights, NY, 10598, USA

    L. Horesh

Authors
  1. N. Hao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. L. Horesh
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. M. E. Kilmer
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to N. Hao .

Editor information

Editors and Affiliations

  1. Department of Mechanical and Aerospace Engineering, Nanyang Technical University, Singapore, Singapore

    Avishy Y. Carmi

  2. School of Computing and Communications, Lancaster University, Lancaster, United Kingdom

    Lyudmila Mihaylova

  3. Department of Engineering, University of Cambridge, Cambridge, United Kingdom

    Simon J. Godsill

Rights and permissions

Reprints and permissions

Copyright information

© 2014 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Hao, N., Horesh, L., Kilmer, M.E. (2014). Nonnegative Tensor Decomposition. In: Carmi, A., Mihaylova, L., Godsill, S. (eds) Compressed Sensing & Sparse Filtering. Signals and Communication Technology. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-38398-4_5

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-38398-4_5

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-38397-7

  • Online ISBN: 978-3-642-38398-4

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation