Low Rank Approximation of Multidimensional Data

  • Mejdi AzaïezEmail author
  • Lucas Lestandi
  • Tomás Chacón Rebollo
Part of the CISM International Centre for Mechanical Sciences book series (CISM, volume 592)


In the last decades, numerical simulation has experienced tremendous improvements driven by massive growth of computing power. Exascale computing has been achieved this year and will allow solving ever more complex problems. But such large systems produce colossal amounts of data which leads to its own difficulties. Moreover, many engineering problems such as multiphysics or optimisation and control, require far more power that any computer architecture could achieve within the current scientific computing paradigm. In this chapter, we propose to shift the paradigm in order to break the curse of dimensionality by introducing decomposition to reduced data. We present an extended review of data reduction techniques and intends to bridge between applied mathematics community and the computational mechanics one. The chapter is organized into two parts. In the first one bivariate separation is studied, including discussions on the equivalence of proper orthogonal decomposition (POD, continuous framework) and singular value decomposition (SVD, discrete matrices). Then, in the second part, a wide review of tensor formats and their approximation is proposed. Such work has already been provided in the literature but either on separate papers or into a pure applied mathematics framework. Here, we offer to the data enthusiast scientist a description of Canonical, Tucker, Hierarchical and Tensor train formats including their approximation algorithms. When it is possible, a careful analysis of the link between continuous and discrete methods will be performed.


Data reduction Model Reduction Singular Values Decomposition Data MOR POD HOSVD Low rank approximation tensors Tensor train 


  1. Alexanderian, A. (2015). A brief note on the Karhunen-Loève expansion.Google Scholar
  2. Alimi, J. M., Bouillot, V., Rasera, Y., Reverdy, V., Corasaniti, P., Balmès, I., et al. (2012). First-ever full observable universe simulation. In International Conference for HPC, Networking, Storage and Analysis, SC.Google Scholar
  3. Amsallem, D., & Farhat, C. (2008). Interpolation method for adapting reduced-order models and application to aeroelasticity. AIAA Journal, 46(7), 1803–1813.CrossRefGoogle Scholar
  4. Appellof, C. J., & Davidson, E. R. (1981). Strategies for analyzing data from video fluorometric monitoring of liquid chromatographic effluents. Analytical Chemistry.Google Scholar
  5. Azaïez, M., Ben Belgacem, F., & Chacón Rebollo, T. (2016). Recursive POD expansion for reaction-diffusion equation. Advanced Modeling and Simulation in Engineering Sciences.Google Scholar
  6. Azaïez, M., Ben-Belgacem, F., Casado-Díaz, J., Chacón, T., & Murat, F. (2018). A new algorithm of proper generalized decomposition for parametric symmetric elliptic problems. SIAM Journal of Mathematical Analysis, 50(5), 5426–5445.Google Scholar
  7. Azaiez, M., Chacon Rebollo, T., Perracchione, E., & Vega, J. M. (2018). Recursive POD expansion for the advection-diffusion-reaction equation. Communications in Computational Physics, 24, 1556–1578.Google Scholar
  8. Ballani, J. (2012). Fast evaluation of near-field boundary integrals using tensor approximations. Phd, University of Leipzig.Google Scholar
  9. Ballani, J., & Grasedyck, L. (2014). Hierarchical tensor approximation of output quantities of parameter-dependent PDEs, 3, 1–19.Google Scholar
  10. Ballani, J., Grasedyck, L., & Kluge, M. (2010). Black box approximation of tensors in hierarchical Tucker format. Linear Algebra and its Applications, 438(2), 639–657.MathSciNetCrossRefGoogle Scholar
  11. Bergmann, M. (2004). Optimisation aérodynamique par réduction de modèle POD et contrôle optimal. Application au sillage laminaire d’un cylindre circulaire. PhD thesis, Institut National Polytechnique de Lorraine/LEMTA.Google Scholar
  12. Berkooz, G., Holmes, P., & Lumley, J. L. (1993). The proper orthogonal decomposition in the analysis of turbulent flows. Annual Review of Fluid Mechanics, 25(1971), 539–575.MathSciNetCrossRefGoogle Scholar
  13. Bigoni, D., Engsig-karup, A. P., & Marzouk, Y. M. (2016). Spectral tensor-train decomposition. SIAM Journal on Scientific Computing, 38, 1–32.Google Scholar
  14. Carroll, J. D., & Chang, J. J. (1970). Analysis of individual differences in multidimensional scaling via an n-way generalization of “Eckart-Young” decomposition. Psychometrika, 35(3), 283–319.Google Scholar
  15. Cazemier, W., Verstappen, R. W. C. P., & Veldman, A. E. P. (1998). Proper orthogonal decomposition and low-dimensional models for driven cavity flows. Physics of Fluids, 10(7), 1685–1699.Google Scholar
  16. Chatterjee, A. (2000). An introduction to the proper orthogonal decomposition. Current Science, 78(7), 808–817.Google Scholar
  17. Chinesta, F., Keunings, R., & Leygue, A. (2013). The proper generalized decomposition for advanced numerical simulations. Berlin: Springer.Google Scholar
  18. Chinesta, F., & Ladavèze, P. (2014). Separated representations and PGD-based model reduction (Vol. 554).Google Scholar
  19. Cordier, L., & Bergmann, M. (2003a). Post-processing of experimental and numerical data: POD an overview. von Karman Institute for Fluid Dynamics (pp. 1–46).Google Scholar
  20. Cordier, L., & Bergmann, M. (2003b). Two typical applications of POD: coherent structures eduction and reduced order modelling. Post-Processing of Experimental and Numerical Data.Google Scholar
  21. De Lathauwer, L., De Moor, B., & Vandewalle, J. (2000a). A multilinear singular value decomposition. SIAM Journal on Matrix Analysis and Applications, 21(4), 1253–1278.Google Scholar
  22. De Lathauwer, L., de Moor, B., & Vandewalle, J. (2000b). On the best rank-1 and rank-(R1,R2,...,RN) approximation of higher order tensors. SIAM Journal on Matrix Analysis and Applications, 21(4), 1324–1342.Google Scholar
  23. de Silva, V., & Lim, L.-H. (2008). Tensor rank and the Ill-posedness of the best low-rank approximation problem. SIAM Journal on Matrix Analysis and Applications, 30(3), 1084–1127.MathSciNetCrossRefGoogle Scholar
  24. Deane, A. E., Kevrekidis, I. G., Karniadakis, G. E., & Orszag, S. A. (1991). Low-dimensional models for complex geometry flows: Application to grooved channels and circular cylinders. Physics of Fluids A: Fluid Dynamics, 3(10), 2337.Google Scholar
  25. Doostan, A., & Iaccarino, G. (2009). A least-squares approximation of partial differential equations with high-dimensional random inputs. Journal of Computational Physics.Google Scholar
  26. Eckart, C., & Young, G. (1936). The approximation of one matrix by another of lower rank. Psychometrika, 1(3), 211–218.CrossRefGoogle Scholar
  27. Fahl, M. (2001). Trust-region methods for flow control based on reduced order modelling. PhD thesis.Google Scholar
  28. Falcó, A., Hilario, L., Montés, N., & Mora, M. C. (2013). Numerical strategies for the Galerkin-proper generalized decomposition method. Mathematical and Computer Modelling, 57(7–8), 1694–1702.MathSciNetCrossRefGoogle Scholar
  29. Falco, A., & Nouy, A. (2011). A proper generalized decomposition for the solution of elliptic problems in abstract form by using a functional Eckart-Young approach. Journal of Mathematical Analysis and Applications, 376(2), 469–480.MathSciNetCrossRefGoogle Scholar
  30. Falco, A., Hackbusch, W., & Nouy, A. (2015). Geometric structures in tensor representations (Final release) (pp. 1–50).Google Scholar
  31. Falco, A., & Nouy, A. (2012). Proper generalized decomposition for nonlinear convex problems in tensor Banach spaces. Numerische Mathematik, 121(3), 503–530.MathSciNetCrossRefGoogle Scholar
  32. Gorodetsky, A. (2016). Continuous low-rank tensor decompositions, with applications to stochastic optimal control and data assimilation. PhD thesis, MIT.Google Scholar
  33. Grasedyck, L. (2010). Hierarchical singular value decomposition of tensors. SIAM Journal on Matrix Analysis and Applications, 31(4), 2029–2054.MathSciNetCrossRefGoogle Scholar
  34. Grasedyck, L., Hackbusch, W., & Nr, B. (2011). An introduction to hierachical (H) rank and TT rank of tensors with examples. Computational Methods in Applied Mathematics, 11(3), 291–304.MathSciNetCrossRefGoogle Scholar
  35. Grasedyck, L., Kressner, D., & Tobler, C. (2013). A literature survey of low-rank tensor approximation techniques. GAMM Mitteilungen, 36(1), 53–78.MathSciNetCrossRefGoogle Scholar
  36. Hackbusch, W., & Kühn, S. (2009). A new scheme for the tensor representation. Journal of Fourier Analysis and Applications.Google Scholar
  37. Hackbush, W. (2014). Tensor spaces and numerical Tensor calculus (Vol. 1). Heidelberg: Springer.Google Scholar
  38. Harshman, R., & Lundy, M. (1996). Uniqueness proof for a family of models sharing features of Tucker’s three-mode factor analysis and PARAFAC/candecomp. Psychometrika, 61(1), 133–154.MathSciNetCrossRefGoogle Scholar
  39. Harshman, R. A. (1970). Foundations of the PARAFAC procedure: Models and conditions for an explanatory multimodal factor analysis. UCLA Working Papers in Phonetics, 16(10), 1–84.Google Scholar
  40. Hitchcock, F. L. (1927). Multiple invariants and generalized rank of a p-way matrix or tensor. Journal of Mathematical Physics, 7, 39–79.CrossRefGoogle Scholar
  41. Hotelling, H. (1933). Analysis of a complex of statistical variables into principal components. Journal of Educational Psychology, 24(6), 417–441.CrossRefGoogle Scholar
  42. Iollo, A., Lanteri, S., & Désidéri, J.-A. (2000). Stability properties of POD Galerkin approximations for the compressible Navier Stokes equations. Theoretical and Computational Fluid Dynamics, 13, 377–396.CrossRefGoogle Scholar
  43. Ito, K., & Ravindran, S. S. (1998). A reduced-order method for simulation and control of fluid flows. Journal of Computational Physics.Google Scholar
  44. Khoromskij, B. N. (2011). Introduction to Tensor numerical methods in scientific computing. Lecture Notes.Google Scholar
  45. Kolda, T. G. (2006). Multilinear operators for higher-order decompositions. SANDIA Report (pp. 1–28).Google Scholar
  46. Kolda, T. G., & Bader, B. W. (2009). Tensor decompositions and applications. SIAM Review, 51(3), 455–500.Google Scholar
  47. Kosambi, D. D. (1943). Statistics in function spaces. Journal of the Indian Mathematical Society.Google Scholar
  48. Kressner, D., & Tobler, C. (2011). Low-rank tensor Krylov subspace methods for parametrized linear systems. SIAM Journal on Matrix Analysis and Applications, 32(4), 1288–1316.MathSciNetCrossRefGoogle Scholar
  49. Kressner, D., & Tobler, C. (2013). htucker A Matlab toolbox for tensors in hierarchical Tucker format (pp. 1–28).Google Scholar
  50. Lestandi, L. (2018). Low rank approximation techniques and reduced order modeling applied to some fluid dynamics problems. Phd thesis, Université de Bordeaux.Google Scholar
  51. Lestandi, L., Bhaumik, S., Sengupta, T. K., Krishna Chand Avatar, G. R., & Azaïez, M. (2018). POD applied to numerical study of unsteady flow inside lid-driven cavity. Journal of Mathematical Study, 51(2), 150–176.Google Scholar
  52. Loève, M. (1977). Probability theory (Vol. 9).Google Scholar
  53. Lumley, J. L. (1967). The structure of inhomogeneous turbulence. In Atmospheric turbulence and wave propagation (yagl ed., pp. 166–178). Nauka, Moscow.Google Scholar
  54. Lumley, J. L. (1981). Coherent structures in turbulence. In R. E. Meyer (Ed.), Transition and turbulence (pp. 215–242). Cambridge: Academic.CrossRefGoogle Scholar
  55. Noack, B. R., Afanasiev, K., Morzyński, M., Tadmor, G., & Thiele, F. (2003). A hierarchy of low-dimensional models for the transient and post-transient cylinder wake. Journal of Fluid Mechanics, 497(February 2016), 335–363.Google Scholar
  56. Nouy, A. (2015). Low-rank tensor methods for model order reduction, 1–73.Google Scholar
  57. Oseledets, I. V. (2011). Tensor-train decomposition. SIAM Journal on Scientific Computing, 33(5), 2295–2317.MathSciNetCrossRefGoogle Scholar
  58. Oseledets, I. V. (2013). Constructive representation of functions in low-rank tensor formats, 1–18.MathSciNetCrossRefGoogle Scholar
  59. Oseledets, I. V., Dolgov, S., & Savostyanov, D. (2018). ttpy.Google Scholar
  60. Oseledets, I. V., & Tyrtyshnikov, E. E. (2009). Breaking the curse of dimensionality, or how to use SVD in many dimensions. SIAM Journal on Scientific Computing, 31(5), 3744–3759.MathSciNetCrossRefGoogle Scholar
  61. Oseledets, I., & Tyrtyshnikov, E. (2010). TT-cross approximation for multidimensional arrays. Linear Algebra and Its Applications, 432(1), 70–88.MathSciNetCrossRefGoogle Scholar
  62. Pearson, K. (1901). LIII. On lines and planes of closest fit to systems of points in space. The London, Edinburgh, and Dublin Philosophical Magazine and Journal of Science, 2(11), 559–572.CrossRefGoogle Scholar
  63. Philippe, B., & Saad, Y. (2014). Calcul des valeurs propres. In Techniques de l’ingénieur. Sciences fondamentales, (AF1224).Google Scholar
  64. Savostyanov, D., & Oseledets, I. (2011). Fast adaptive interpolation of multi-dimensional arrays in tensor train format.Google Scholar
  65. Sengupta, T. K., Bhaumik, S., & Bhumkar, Y. G. (2011). Nonlinear receptivity and instability studies by POD. In 6th AIAA Theoretical Fluid Mechanics Conference, Honululu, Hawaii, USA.Google Scholar
  66. Sengupta, T. K., & Gullapalli, A. (2016). Enstrophy-based proper orthogonal decomposition of flow past rotating cylinder at super-critical rotating rate. Physics of Fluids.Google Scholar
  67. Sengupta, T. K., Haider, S. I., Parvathi, M. K., & Pallavi, G. (2015). Enstrophy-based proper orthogonal decomposition for reduced-order modeling of flow past a cylinder. Physical Review E, 91(4), 1–23.MathSciNetCrossRefGoogle Scholar
  68. Sengupta, T. K., Singh, N., & Suman, V. K. (2010). Dynamical system approach to instability of flow past a circular cylinder. Journal of Fluid Mechanics, 656, 82–115.MathSciNetCrossRefGoogle Scholar
  69. Sirovich, L. (1987). Turbulence and the dynamics of coherent structures. I - Coherent structures. II - Symmetries and transformations. III - Dynamics and scaling. Quarterly of Applied Mathematics, 45(July), 561.MathSciNetCrossRefGoogle Scholar
  70. Tucker, L. R. (1966). Some mathematical notes on three-mode factor analysis. Psychometrika, 31(3), 279–311.MathSciNetCrossRefGoogle Scholar
  71. Vannieuwenhoven, N., Vandebril, R., & Meerbergen, K. (2011). On the truncated multilinear singular value decomposition. Department of Computer Science, K. U. Leuven.Google Scholar
  72. Vannieuwenhoven, N., Vandebril, R., & Meerbergen, K. (2012). A new truncation strategy for the higher-order singular value decomposition. SIAM Journal on Scientific Computing, 34(2), A1027–A1052.MathSciNetCrossRefGoogle Scholar

Copyright information

© CISM International Centre for Mechanical Sciences 2019

Authors and Affiliations

  • Mejdi Azaïez
    • 1
    Email author
  • Lucas Lestandi
    • 2
    • 3
  • Tomás Chacón Rebollo
    • 4
  1. 1.Bordeaux Institut National Polytechnique de BordeauxBordeauxFrance
  2. 2.Université de BordeauxBordeauxFrance
  3. 3.Institut de Mécanique et d’IngénierieBordeauxFrance
  4. 4.Instituto de Matemáticas de la Universidad de Sevilla - IMUSSevillaSpain

Personalised recommendations