Abstract
Matrix factorizations and tensor decompositions are now widely used in machine learning and data mining. They decompose input matrix and tensor data into matrix factors by optimizing a least square objective function using iterative updating algorithms, e.g. HOSVD (High Order Singular Value Decomposition) and ParaFac (Parallel Factors). One fundamental problem of these algorithms remains unsolved: are the solutions found by these algorithms global optimal? Surprisingly, we provide a positive answer for HSOVD and negative answer for ParaFac by combining theoretical analysis and experimental evidence. Our discoveries of this intrinsic property of HOSVD assure us that in real world applications HOSVD provides repeatable and reliable results.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Adomavicius, G., Tuzhilin, A.: Toward the next generation of recommender systems: A survey of the state-of-the-art and possible extensions. IEEE Trans. Knowl. Data Eng. 17, 734–749 (2005)
Rendle, S., Marinho, L.B., Nanopoulos, A., Schmidt-Thieme, L.: Learning optimal ranking with tensor factorization for tag recommendation. In: KDD, pp. 727–736 (2009)
Omberg, L., Golub, G.H., Alter, O.: A tensor higher-order singular value decomposition for integrative analysis of dna microarray data from different studies. Proc. Natl. Acad. Sci. of USA 104, 18371–18376 (2007)
Sun, J., Tao, D., Faloutsos, C.: Beyond streams and graphs: Dynamic tensor analysis. In: Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, vol. 12 (2006)
Korn, F., Labrinidis, A., Kotidis, Y., Faloutsos, C.: Quantifiable data mining using ratio rules. VLDB Journal: Very Large Data Bases 8, 254–266 (2000)
Tucker, L.: Some mathematical notes on three-mode factor analysis. Psychometrika 31, 279–311 (1966)
Lathauwer, L.D., Moor, B.D., Vandewalle, J.: A multilinear singular value decomposition. SIAM Journal of Matrix Analysis and Applications 21, 1253–1278 (2000)
Vasilescu, M., Terzopoulos, D.: Multilinear analysis of image ensembles: TensorFaces. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002. LNCS, vol. 2350, pp. 447–460. Springer, Heidelberg (2002)
Shashua, A., Levin, A.: Linear image coding for regression and classification using the tensor-rank principle. In: IEEE Conf. on Computer Vision and Pattern Recognition (2001)
Yang, J., Zhang, D., Frangi, A.F., Yang, J.: Twodimensional pca: A new approach to appearancebased face representation and recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence 26 (2004)
Ye, J.: Generalized low rank approximations of matrices. In: International Conference on Machine Learning (2004)
Ding, C., Ye, J.: Two-dimensional singular value decomposition (2dsvd) for 2d maps and images. In: SIAM Int’l Conf. Data Mining, pp. 32–43 (2005)
Inoue, K., Urahama, K.: Equivalence of non-iterative algorithms for simultaneous low rank approximations of matrices. In: Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR) (2006)
Luo, D., Ding, C., Huang, H.: Symmetric two dimensional linear discriminant analysis (2DLDA). In: CVPR (2009)
Nie, F., Xiang, S., Song, Y., Zhang, C.: Extracting the optimal dimensionality for local tensor discriminant analysis. Pattern Recognition 42, 105–114 (2009)
Ding, C., Huang, H., Luo, D.: Tensor reduction error analysis – applications to video compression and classification. In: CVPR (2008)
Huang, H., Ding, C., Luo, D., Li, T.: Simultaneous tensor subspace selection and clustering: the equivalence of High Order SVD and K-means clustering. In: KDD, pp. 327–335 (2008)
Harshman, R.: Foundations of the parafac procedure: Model and conditions for an ’explanatory’ multi-mode factor analysis. UCLA Working Papers in Phonetics 16, 1–84 (1970)
Carroll, J., Chang, J.: Analysis of individual differences in multidimensional scaling via an n-way generalization of Eckart-Young decomposition. Psychometrika 35, 283–319 (1970)
Zhang, T., Golub, G.H.: Rank-one approximation to high order tensors. SIAM Journal of Matrix Analysis and Applications 23, 534–550 (2001)
Kolda, T.: Orthogonal tensor decompositions. SIAM J. Matrix Analysis and App. 23, 243–255 (2001)
Bader, B., Kolda, T.: Matlab tensor toolbox version 2.2. (January 2007), http://csmr.ca.sandia.gov/~tgkolda/TensorToolbox/
Wang, J.Z., Li, J., Wiederhold, G.: SIMPLIcity: Semantics-sensitive integrated matching for picture LIbraries. IEEE Trans. Pattern Anal. Mach. Intell 23, 947–963 (2001)
Perona, P., Fergus, R., Li, F.F.: Learning generative visual models from few training examples: An incremental bayesian approach tested on 101 object categories. In: Workshop on Generative Model Based Vision, p. 178 (2004)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Luo, D., Ding, C., Huang, H. (2011). Are Tensor Decomposition Solutions Unique? On the Global Convergence HOSVD and ParaFac Algorithms. In: Huang, J.Z., Cao, L., Srivastava, J. (eds) Advances in Knowledge Discovery and Data Mining. PAKDD 2011. Lecture Notes in Computer Science(), vol 6634. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-20841-6_13
Download citation
DOI: https://doi.org/10.1007/978-3-642-20841-6_13
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-20840-9
Online ISBN: 978-3-642-20841-6
eBook Packages: Computer ScienceComputer Science (R0)