Skip to main content
SpringerLink
Log in

Tensors and Latent Variable Models

  • Conference paper
  • First Online:
Latent Variable Analysis and Signal Separation (LVA/ICA 2015)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 9237))

Abstract

In this paper we discuss existing and new connections between latent variable models from machine learning and tensors (multi-way arrays) from multilinear algebra. A few ideas have been developed independently in the two communities. However, there are still many useful but unexplored links and ideas that could be borrowed from one of the communities and used in the other. We will start our discussion from simple concepts such as independent variables and rank-1 matrices and gradually increase the difficulty. The final goal is to connect discrete latent tree graphical models to state of the art tensor decompositions in order to find tractable representations of probability tables of many variables.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    The values of \(P_i(j)\) are actually in the interval [0, 1] but for the purposes of this paper it is easier to think of them as real numbers.

References

  1. Carroll, J., Chang, J.: Analysis of individual differences in multidimensional scaling via an N-way generalization of “Eckart-Young” decomposition. Psychometrika 35(3), 283–319 (1970)

    Article  Google Scholar 

  2. Cichocki, A.: Tensor networks for big data analytics and large-scale optimization problems (2014). arXiv preprint arXiv:1407.3124

  3. Cichocki, A., Mandic, D., De Lathauwer, L., Zhou, G., Zhao, Q., Caiafa, C., Phan, H.A.: Tensor decompositions for signal processing applications: from two-way to multiway component analysis. IEEE Sig. Process. Mag. 32(2), 145–163 (2015)

    Article  Google Scholar 

  4. Cichocki, A., Zdunek, R., Phan, A., Amari, S.: Nonnegative Matrix and Tensor Factorizations. Wiley, Chichester (2009)

    Book  Google Scholar 

  5. Comon, P.: Tensors: a brief introduction. IEEE Signal Process. Mag. 31(3), 44–53 (2014)

    Article  Google Scholar 

  6. Domanov, I., De Lathauwer, L.: On the uniqueness of the canonical polyadic decomposition of third-order tensors – part II: Uniqueness of the overall decomposition. SIAM J. Matrix Anal. Appl. 34(3), 876–903 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  7. Grasedyck, L.: Hierarchical singular value decomposition of tensors. SIAM J. Matrix Anal. Appl. 31(4), 2029–2054 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  8. Grasedyck, L., Kressner, D., Tobler, C.: A literature survey of low-rank tensor approximation techniques. GAMM Mitt. 36(1), 53–78 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  9. Hackbusch, W.: Tensor Spaces and Numerical Tensor Calculus. Springer Series in Computational Mathematics. Springer, Heidelberg (2012). vol. 42

    Book  MATH  Google Scholar 

  10. Harshman, R.A.: Foundations of the PARAFAC procedure: model and conditions for an “explanatory” multi-mode factor analysis. UCLA Working Pap. Phonetics 16(1), 1–84 (1970)

    MATH  Google Scholar 

  11. Ishteva, M., Song, L., Park, H.: Unfolding latent tree structures using \(4\)th order tensors. In: International Conference on Machine Learning (ICML) (2013)

    Google Scholar 

  12. Khoromskij, B.N.: Tensors-structured numerical methods in scientific computing: Survey on recent advances. Chemometr. Intell. Lab. Syst. 110(1), 1–19 (2012)

    Article  MATH  Google Scholar 

  13. Kolda, T.G., Bader, B.W.: Tensor decompositions and applications. SIAM Rev. 51(3), 455–500 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  14. Kroonenberg, P.M.: Applied Multiway Data Analysis. Wiley, New York (2008)

    Book  Google Scholar 

  15. Oseledets, I.V.: Tensor-train decomposition. SIAM J. Sci. Comput. 33, 2295–2317 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  16. Shashua, A.: The applications of tensor factorization in inference, clustering, graph theory, coding and visual representation, 2012. Keynote talk at the 10th International Conference on Latent Variable Analysis and Signal Separation

    Google Scholar 

  17. Smilde, A., Bro, R., Geladi, P.: Multi-way Analysis Applications in the Chemical Sciences. Wiley, Chichester (2004)

    Book  Google Scholar 

  18. Song, L., Ishteva, M., Parikh, A., Xing, E., Park, H.: Hierarchical tensor decomposition of latent tree graphical models. In: International Conference on Machine Learning (ICML) (2013)

    Google Scholar 

  19. Vervliet, N., Debals, O., Sorber, L., De Lathauwer, L.: Breaking the curse of dimensionality using decompositions of incomplete tensors. Sig. Process. Mag. IEEE 31(5), 71–79 (2014)

    Article  Google Scholar 

  20. Yılmaz, Y.K., Cemgil, A.T.: Algorithms for probabilistic latent tensor factorization. Sig. Process. 92(8), 1853–1863 (2012)

    Article  Google Scholar 

Download references

Acknowledgments

This work was supported in part by the Fund for Scientific Research (FWO-Vlaanderen), by FWO project G.0280.15N, by the Flemish Government (Methusalem), by the Belgian Government through the Inter-university Poles of Attraction (IAP VII) Program (DYSCO II, Dynamical systems, control and optimization, 2012–2017), by the ERC Advanced Grant SNLSID under contract 320378, and by the ERC Starting Grant SLRA under contract 258581. Mariya Ishteva was an FWO Pegasus Marie Curie Fellow.

Author information

Authors and Affiliations

  1. Vrije Universiteit Brussel (VUB), 1050, Brussels, Belgium

    Mariya Ishteva

Authors
  1. Mariya Ishteva
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mariya Ishteva .

Editor information

Editors and Affiliations

  1. Inria, Villers-les-Nancy, France

    Emmanuel Vincent

  2. Tel Aviv University, Tel-Aviv, Israel

    Arie Yeredor

  3. Technical University of Libere, Liberec, Czech Republic

    Zbyněk Koldovský

  4. The Czech Academy of Sciences, Prague, Czech Republic

    Petr Tichavský

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this paper

Cite this paper

Ishteva, M. (2015). Tensors and Latent Variable Models. In: Vincent, E., Yeredor, A., Koldovský, Z., Tichavský, P. (eds) Latent Variable Analysis and Signal Separation. LVA/ICA 2015. Lecture Notes in Computer Science(), vol 9237. Springer, Cham. https://doi.org/10.1007/978-3-319-22482-4_6

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-22482-4_6

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-22481-7

  • Online ISBN: 978-3-319-22482-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation