Abstract
The multilinear rank of a tensor is one of the possible generalizations for the concept of matrix rank. In this paper, we are interested in finding the best low multilinear rank approximation of a given tensor. This problem has been formulated as an optimization problem over the Grassmann manifold [14] and it has been shown that the objective function presents multiple minima [15]. In order to investigate the landscape of this cost function, we propose an adaptation of the Particle Swarm Optimization algorithm (PSO). The Guaranteed Convergence PSO, proposed by van den Bergh in [23], is modified, including a gradient component, so as to search for optimal solutions over the Grassmann manifold. The operations involved in the PSO algorithm are redefined using concepts of differential geometry. We present some preliminary numerical experiments and we discuss the ability of the proposed method to address the multimodal aspects of the studied problem.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Absil, P.A., Mahony, R., Sepulchre, R.: Riemannian geometry of Grassmann manifolds with a view on algorithmic computation. Acta Appl. Math. 80(2), 199–220 (2004)
Absil, P.A., Mahony, R., Sepulchre, R.: Optimization Algorithms on Matrix Manifolds. Princeton University Press, Princeton (2008)
Acar, E., Bingol, C.A., Bingol, H., Bro, R., Yener, B.: Multiway analysis of epilepsy tensors. In: ISMB 2007 Conference Proceedings, Bioinformatics, vol. 23(13), pp. i10–i18 (2007)
Audet, C., Dennis Jr., J.E.: Mesh adaptive direct search algorithms for constrained optimization. SIAM J. on Optimization 17(1), 188–217 (2006)
Brits, R., Engelbrecht, A., van den Bergh, F.: A niching particle swarm optimizer. In: Proceedings of the 4th Asia-Pacific Conference on Simulated Evolution and Learning (SEAL 2002), vol. 2, pp. 692–696 (2002)
De Lathauwer, L., De Moor, B., Vandewalle, J.: A multilinear singular value decomposition. SIAM J. Matrix Anal. Appl. 21(4), 1253–1278 (2000)
De Lathauwer, L., De Moor, B., Vandewalle, J.: On the best rank-1 and rank-(R 1,R 2,...,R N ) approximation of higher-order tensors. SIAM J. Matrix Anal. Appl. 21(4), 1324–1342 (2000)
De Lathauwer, L., Vandewalle, J.: Dimensionality reduction in higher-order signal processing and rank-(R 1,R 2,...,R N ) reduction in multilinear algebra. Linear Algebra Appl. 391, 31–55 (2004); Special Issue on Linear Algebra in Signal and Image Processing
Dreisigmeyer, D.W.: Direct search algorithms over Riemannian manifolds (December 2006) (optimization Online 2007-08-1742)
Edelman, A., Arias, T.A., Smith, S.T.: The geometry of algorithms with orthogonality constraints. SIAM J. Matrix Anal. Appl. 20(2), 303–353 (1998)
Engelbrecht, A.P.: Fundamentals of Computational Swarm Intelligence. John Wiley & Sons, Chichester (2006)
Hitchcock, F.L.: The expression of a tensor or a polyadic as a sum of products. Journal of Mathematical Physics 6(1), 164–189 (1927)
Hitchcock, F.L.: Multiple invariants and generalized rank of a p-way matrix or tensor. Journal of Mathematical Physics 7(1), 39–79 (1927)
Ishteva, M.: Numerical methods for the best low multilinear rank approximation of higher-order tensors. Ph.D. thesis, Department of Electrical Engineering, Katholieke Universiteit Leuven (December 2009)
Ishteva, M., Absil, P.-A., Van Huffel, S., De Lathauwer, L.: Tucker compression and local optima. Chemometr. Intell. Lab. Syst. (2010), doi:10.1016/j.chemolab.2010.06.006
Kennedy, J., Eberhart, R.: Particle swarm optimization. In: Proceedings of IEEE International Conference on Neural Networks, vol. 4, pp. 1942–1948 (1995), http://dx.doi.org/10.1109/ICNN.1995.488968
Kroonenberg, P.M.: Applied Multiway Data Analysis. Wiley, Chichester (2008)
Kroonenberg, P.M., de Leeuw, J.: Principal component analysis of three-mode data by means of alternating least squares algorithms. Psychometrika 45(1), 69–97 (1980)
Liu, X., Srivastava, A., Gallivan, K.: Optimal linear representations of images for object recognition. IEEE Pattern Anal. and Mach. Intell. 26(5), 662–666 (2004), http://dx.doi.org/10.1109/TPAMI.2004.1273986
McCullagh, P.: Tensor Methods in Statistics. Chapman and Hall, London (1987)
Tucker, L.R.: The extension of factor analysis to three-dimensional matrices. In: Gulliksen, H., Frederiksen, N. (eds.) Contributions to mathematical psychology, pp. 109–127. Holt, Rinehart & Winston (1964)
Tucker, L.R.: Some mathematical notes on three-mode factor analysis. Psychometrika 31, 279–311 (1966)
van den Bergh, F., Engelbrecht, A.P.: A new locally convergent particle swarm optimiser. In: Proceedings of the IEEE International Conference on Systems, Man and Cybernetics. pp. 96–101 (2002)
Vasilescu, M.A.O., Terzopoulos, D.: Multilinear subspace analysis for image ensembles. In: Proc. Computer Vision and Pattern Recognition Conf. (CVPR 2003), Madison, WI, vol. 2, pp. 93–99 (2003)
Zhang, J., Zhang, J.R., Li, K.: A sequential niching technique for particle swarm optimization. In: Huang, D.-S., Zhang, X.-P., Huang, G.-B. (eds.) ICIC 2005. LNCS, vol. 3644, pp. 390–399. Springer, Heidelberg (2005)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Borckmans, P.B., Ishteva, M., Absil, PA. (2010). A Modified Particle Swarm Optimization Algorithm for the Best Low Multilinear Rank Approximation of Higher-Order Tensors . In: Dorigo, M., et al. Swarm Intelligence. ANTS 2010. Lecture Notes in Computer Science, vol 6234. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-15461-4_2
Download citation
DOI: https://doi.org/10.1007/978-3-642-15461-4_2
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-15460-7
Online ISBN: 978-3-642-15461-4
eBook Packages: Computer ScienceComputer Science (R0)