Abstract
In the last decades, numerical simulation has experienced tremendous improvements driven by massive growth of computing power. Exascale computing has been achieved this year and will allow solving ever more complex problems. But such large systems produce colossal amounts of data which leads to its own difficulties. Moreover, many engineering problems such as multiphysics or optimisation and control, require far more power that any computer architecture could achieve within the current scientific computing paradigm. In this chapter, we propose to shift the paradigm in order to break the curse of dimensionality by introducing decomposition to reduced data. We present an extended review of data reduction techniques and intends to bridge between applied mathematics community and the computational mechanics one. The chapter is organized into two parts. In the first one bivariate separation is studied, including discussions on the equivalence of proper orthogonal decomposition (POD, continuous framework) and singular value decomposition (SVD, discrete matrices). Then, in the second part, a wide review of tensor formats and their approximation is proposed. Such work has already been provided in the literature but either on separate papers or into a pure applied mathematics framework. Here, we offer to the data enthusiast scientist a description of Canonical, Tucker, Hierarchical and Tensor train formats including their approximation algorithms. When it is possible, a careful analysis of the link between continuous and discrete methods will be performed.
The research of Mejdi Azaïez was partially funded by the IV Research and Transfer Plan of the University of Sevilla.
The research of Lucas Lestandi was partially funded by the Institut Carnot ARTZ.
The research of Tomás Chacón was partially funded by Junta de Andalucia - Feder Fund Grant FQM 454, and by the IDEX program of the University of Bordeaux.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsNotes
- 1.
Gordon Moore predicted in 1965 that the density of transistors on chips would double every year. After being slightly downgraded to doubling every 18 month, it has been verified from 1975 to 2012. Current trend shows a slowing pace. Still, this exponential growth amounts to a 20 millions factor. Naturally, it corresponds to the computing power gain.
- 2.
As of June 2018, the largest supercomputer is the Summit at Oak Ridge, USA, with more than 2 million cores it requires 8MW for a peak performance of 122PFlop/s.
- 3.
A typical example in fluid dynamics is Reynolds number \(Re=UL/\mu \) which characterize the relative influence of inertia (U is a typical flow velocity and L a typical length) compared with viscosity (\(\mu \) the kinematic viscosity.).
- 4.
These approaches are conceptually continuous but their implementations requires discrete description of the continuous space including grids, discrete operators.
- 5.
Here, high dimensionality is to be understood as rich phenomenon that require many degrees of freedom to be described properly as opposed to simpler system which are described by few degrees of freedom e.g. simple pendulum.
- 6.
- 7.
The natural choice for fluid dynamics applications \(L^2(\Omega _x)\) scalar product and a time average. The choice of the average operator \( \langle \cdot \rangle \) kind (temporal, spatial,...) determines which kind of POD is used.
- 8.
Actually the choice of the norm has little influence on the numerical results. This is especially true for trapezoidal rule on a Cartesian grid.
- 9.
The reader is advised to follow this description in the PDF version as it allows zooming of the row of small pictures.
- 10.
The order of a tensor is not to be confused with the rank of a tensor.
- 11.
Here we assume without loss of generality that \(\Omega _i\) is a subset of \(\mathbb {R}\) but it could be any domain on which an integral can be defined. e.g. 2D or 3D domains.
- 12.
As long a one only requires a small number of modes as compared to the full representation, PGD can be efficient since it computes only the required information.
- 13.
- 14.
Using a non uniform grid would have little influence on the accuracy given that one uses accurate integration schemes. However it may help to increase the computing speed by using a sparser grid.
- 15.
The norm is not specified here as it can be either a Frobenius norm of tensors or the\(L^2(\Omega )\) norm.
- 16.
Actually, for TT rank of 1 and RPOD rank of 1 i.e. 1 mode only for each dimension, then both algorithms are strictly equivalent, only the data structure is different. Then when the rank grows, the association of modes by explicit summation in Recursive format is less efficient than the implicit summation to the TT format. Finally the truncation strategy used in the software requires that any branch with a weight above truncation limit has at least one leaf kept in the evaluation and all other leaves below the truncation limit are ignored. This results in cumulative loss in precision which means that the rank/epsilon truncation in recursive format is less sharp than in TT format.
- 17.
most efficient methods depends on required accuracy for \(d=4.\)
References
Alexanderian, A. (2015). A brief note on the Karhunen-Loève expansion.
Alimi, J. M., Bouillot, V., Rasera, Y., Reverdy, V., Corasaniti, P., Balmès, I., et al. (2012). First-ever full observable universe simulation. In International Conference for HPC, Networking, Storage and Analysis, SC.
Amsallem, D., & Farhat, C. (2008). Interpolation method for adapting reduced-order models and application to aeroelasticity. AIAA Journal, 46(7), 1803–1813.
Appellof, C. J., & Davidson, E. R. (1981). Strategies for analyzing data from video fluorometric monitoring of liquid chromatographic effluents. Analytical Chemistry.
Azaïez, M., Ben Belgacem, F., & Chacón Rebollo, T. (2016). Recursive POD expansion for reaction-diffusion equation. Advanced Modeling and Simulation in Engineering Sciences.
Azaïez, M., Ben-Belgacem, F., Casado-Díaz, J., Chacón, T., & Murat, F. (2018). A new algorithm of proper generalized decomposition for parametric symmetric elliptic problems. SIAM Journal of Mathematical Analysis, 50(5), 5426–5445.
Azaiez, M., Chacon Rebollo, T., Perracchione, E., & Vega, J. M. (2018). Recursive POD expansion for the advection-diffusion-reaction equation. Communications in Computational Physics, 24, 1556–1578.
Ballani, J. (2012). Fast evaluation of near-field boundary integrals using tensor approximations. Phd, University of Leipzig.
Ballani, J., & Grasedyck, L. (2014). Hierarchical tensor approximation of output quantities of parameter-dependent PDEs, 3, 1–19.
Ballani, J., Grasedyck, L., & Kluge, M. (2010). Black box approximation of tensors in hierarchical Tucker format. Linear Algebra and its Applications, 438(2), 639–657.
Bergmann, M. (2004). Optimisation aérodynamique par réduction de modèle POD et contrôle optimal. Application au sillage laminaire d’un cylindre circulaire. PhD thesis, Institut National Polytechnique de Lorraine/LEMTA.
Berkooz, G., Holmes, P., & Lumley, J. L. (1993). The proper orthogonal decomposition in the analysis of turbulent flows. Annual Review of Fluid Mechanics, 25(1971), 539–575.
Bigoni, D., Engsig-karup, A. P., & Marzouk, Y. M. (2016). Spectral tensor-train decomposition. SIAM Journal on Scientific Computing, 38, 1–32.
Carroll, J. D., & Chang, J. J. (1970). Analysis of individual differences in multidimensional scaling via an n-way generalization of “Eckart-Young” decomposition. Psychometrika, 35(3), 283–319.
Cazemier, W., Verstappen, R. W. C. P., & Veldman, A. E. P. (1998). Proper orthogonal decomposition and low-dimensional models for driven cavity flows. Physics of Fluids, 10(7), 1685–1699.
Chatterjee, A. (2000). An introduction to the proper orthogonal decomposition. Current Science, 78(7), 808–817.
Chinesta, F., Keunings, R., & Leygue, A. (2013). The proper generalized decomposition for advanced numerical simulations. Berlin: Springer.
Chinesta, F., & Ladavèze, P. (2014). Separated representations and PGD-based model reduction (Vol. 554).
Cordier, L., & Bergmann, M. (2003a). Post-processing of experimental and numerical data: POD an overview. von Karman Institute for Fluid Dynamics (pp. 1–46).
Cordier, L., & Bergmann, M. (2003b). Two typical applications of POD: coherent structures eduction and reduced order modelling. Post-Processing of Experimental and Numerical Data.
De Lathauwer, L., De Moor, B., & Vandewalle, J. (2000a). A multilinear singular value decomposition. SIAM Journal on Matrix Analysis and Applications, 21(4), 1253–1278.
De Lathauwer, L., de Moor, B., & Vandewalle, J. (2000b). On the best rank-1 and rank-(R1,R2,...,RN) approximation of higher order tensors. SIAM Journal on Matrix Analysis and Applications, 21(4), 1324–1342.
de Silva, V., & Lim, L.-H. (2008). Tensor rank and the Ill-posedness of the best low-rank approximation problem. SIAM Journal on Matrix Analysis and Applications, 30(3), 1084–1127.
Deane, A. E., Kevrekidis, I. G., Karniadakis, G. E., & Orszag, S. A. (1991). Low-dimensional models for complex geometry flows: Application to grooved channels and circular cylinders. Physics of Fluids A: Fluid Dynamics, 3(10), 2337.
Doostan, A., & Iaccarino, G. (2009). A least-squares approximation of partial differential equations with high-dimensional random inputs. Journal of Computational Physics.
Eckart, C., & Young, G. (1936). The approximation of one matrix by another of lower rank. Psychometrika, 1(3), 211–218.
Fahl, M. (2001). Trust-region methods for flow control based on reduced order modelling. PhD thesis.
Falcó, A., Hilario, L., Montés, N., & Mora, M. C. (2013). Numerical strategies for the Galerkin-proper generalized decomposition method. Mathematical and Computer Modelling, 57(7–8), 1694–1702.
Falco, A., & Nouy, A. (2011). A proper generalized decomposition for the solution of elliptic problems in abstract form by using a functional Eckart-Young approach. Journal of Mathematical Analysis and Applications, 376(2), 469–480.
Falco, A., Hackbusch, W., & Nouy, A. (2015). Geometric structures in tensor representations (Final release) (pp. 1–50).
Falco, A., & Nouy, A. (2012). Proper generalized decomposition for nonlinear convex problems in tensor Banach spaces. Numerische Mathematik, 121(3), 503–530.
Gorodetsky, A. (2016). Continuous low-rank tensor decompositions, with applications to stochastic optimal control and data assimilation. PhD thesis, MIT.
Grasedyck, L. (2010). Hierarchical singular value decomposition of tensors. SIAM Journal on Matrix Analysis and Applications, 31(4), 2029–2054.
Grasedyck, L., Hackbusch, W., & Nr, B. (2011). An introduction to hierachical (H) rank and TT rank of tensors with examples. Computational Methods in Applied Mathematics, 11(3), 291–304.
Grasedyck, L., Kressner, D., & Tobler, C. (2013). A literature survey of low-rank tensor approximation techniques. GAMM Mitteilungen, 36(1), 53–78.
Hackbusch, W., & Kühn, S. (2009). A new scheme for the tensor representation. Journal of Fourier Analysis and Applications.
Hackbush, W. (2014). Tensor spaces and numerical Tensor calculus (Vol. 1). Heidelberg: Springer.
Harshman, R., & Lundy, M. (1996). Uniqueness proof for a family of models sharing features of Tucker’s three-mode factor analysis and PARAFAC/candecomp. Psychometrika, 61(1), 133–154.
Harshman, R. A. (1970). Foundations of the PARAFAC procedure: Models and conditions for an explanatory multimodal factor analysis. UCLA Working Papers in Phonetics, 16(10), 1–84.
Hitchcock, F. L. (1927). Multiple invariants and generalized rank of a p-way matrix or tensor. Journal of Mathematical Physics, 7, 39–79.
Hotelling, H. (1933). Analysis of a complex of statistical variables into principal components. Journal of Educational Psychology, 24(6), 417–441.
Iollo, A., Lanteri, S., & Désidéri, J.-A. (2000). Stability properties of POD Galerkin approximations for the compressible Navier Stokes equations. Theoretical and Computational Fluid Dynamics, 13, 377–396.
Ito, K., & Ravindran, S. S. (1998). A reduced-order method for simulation and control of fluid flows. Journal of Computational Physics.
Khoromskij, B. N. (2011). Introduction to Tensor numerical methods in scientific computing. Lecture Notes.
Kolda, T. G. (2006). Multilinear operators for higher-order decompositions. SANDIA Report (pp. 1–28).
Kolda, T. G., & Bader, B. W. (2009). Tensor decompositions and applications. SIAM Review, 51(3), 455–500.
Kosambi, D. D. (1943). Statistics in function spaces. Journal of the Indian Mathematical Society.
Kressner, D., & Tobler, C. (2011). Low-rank tensor Krylov subspace methods for parametrized linear systems. SIAM Journal on Matrix Analysis and Applications, 32(4), 1288–1316.
Kressner, D., & Tobler, C. (2013). htucker A Matlab toolbox for tensors in hierarchical Tucker format (pp. 1–28).
Lestandi, L. (2018). Low rank approximation techniques and reduced order modeling applied to some fluid dynamics problems. Phd thesis, Université de Bordeaux.
Lestandi, L., Bhaumik, S., Sengupta, T. K., Krishna Chand Avatar, G. R., & Azaïez, M. (2018). POD applied to numerical study of unsteady flow inside lid-driven cavity. Journal of Mathematical Study, 51(2), 150–176.
Loève, M. (1977). Probability theory (Vol. 9).
Lumley, J. L. (1967). The structure of inhomogeneous turbulence. In Atmospheric turbulence and wave propagation (yagl ed., pp. 166–178). Nauka, Moscow.
Lumley, J. L. (1981). Coherent structures in turbulence. In R. E. Meyer (Ed.), Transition and turbulence (pp. 215–242). Cambridge: Academic.
Noack, B. R., Afanasiev, K., Morzyński, M., Tadmor, G., & Thiele, F. (2003). A hierarchy of low-dimensional models for the transient and post-transient cylinder wake. Journal of Fluid Mechanics, 497(February 2016), 335–363.
Nouy, A. (2015). Low-rank tensor methods for model order reduction, 1–73.
Oseledets, I. V. (2011). Tensor-train decomposition. SIAM Journal on Scientific Computing, 33(5), 2295–2317.
Oseledets, I. V. (2013). Constructive representation of functions in low-rank tensor formats, 1–18.
Oseledets, I. V., Dolgov, S., & Savostyanov, D. (2018). ttpy.
Oseledets, I. V., & Tyrtyshnikov, E. E. (2009). Breaking the curse of dimensionality, or how to use SVD in many dimensions. SIAM Journal on Scientific Computing, 31(5), 3744–3759.
Oseledets, I., & Tyrtyshnikov, E. (2010). TT-cross approximation for multidimensional arrays. Linear Algebra and Its Applications, 432(1), 70–88.
Pearson, K. (1901). LIII. On lines and planes of closest fit to systems of points in space. The London, Edinburgh, and Dublin Philosophical Magazine and Journal of Science, 2(11), 559–572.
Philippe, B., & Saad, Y. (2014). Calcul des valeurs propres. In Techniques de l’ingénieur. Sciences fondamentales, (AF1224).
Savostyanov, D., & Oseledets, I. (2011). Fast adaptive interpolation of multi-dimensional arrays in tensor train format.
Sengupta, T. K., Bhaumik, S., & Bhumkar, Y. G. (2011). Nonlinear receptivity and instability studies by POD. In 6th AIAA Theoretical Fluid Mechanics Conference, Honululu, Hawaii, USA.
Sengupta, T. K., & Gullapalli, A. (2016). Enstrophy-based proper orthogonal decomposition of flow past rotating cylinder at super-critical rotating rate. Physics of Fluids.
Sengupta, T. K., Haider, S. I., Parvathi, M. K., & Pallavi, G. (2015). Enstrophy-based proper orthogonal decomposition for reduced-order modeling of flow past a cylinder. Physical Review E, 91(4), 1–23.
Sengupta, T. K., Singh, N., & Suman, V. K. (2010). Dynamical system approach to instability of flow past a circular cylinder. Journal of Fluid Mechanics, 656, 82–115.
Sirovich, L. (1987). Turbulence and the dynamics of coherent structures. I - Coherent structures. II - Symmetries and transformations. III - Dynamics and scaling. Quarterly of Applied Mathematics, 45(July), 561.
Tucker, L. R. (1966). Some mathematical notes on three-mode factor analysis. Psychometrika, 31(3), 279–311.
Vannieuwenhoven, N., Vandebril, R., & Meerbergen, K. (2011). On the truncated multilinear singular value decomposition. Department of Computer Science, K. U. Leuven.
Vannieuwenhoven, N., Vandebril, R., & Meerbergen, K. (2012). A new truncation strategy for the higher-order singular value decomposition. SIAM Journal on Scientific Computing, 34(2), A1027–A1052.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 CISM International Centre for Mechanical Sciences
About this chapter
Cite this chapter
Azaïez, M., Lestandi, L., Chacón Rebollo, T. (2019). Low Rank Approximation of Multidimensional Data. In: Pirozzoli, S., Sengupta, T. (eds) High-Performance Computing of Big Data for Turbulence and Combustion. CISM International Centre for Mechanical Sciences, vol 592. Springer, Cham. https://doi.org/10.1007/978-3-030-17012-7_5
Download citation
DOI: https://doi.org/10.1007/978-3-030-17012-7_5
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-17011-0
Online ISBN: 978-3-030-17012-7
eBook Packages: EngineeringEngineering (R0)