Uni-orthogonal Nonnegative Tucker Decomposition for Supervised Image Classification

  • Rafal Zdunek
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6978)


The Tucker model with orthogonality constraints (often referred to as the HOSVD) assumes decomposition of a multi-way array into a core tensor and orthogonal factor matrices corresponding to each mode. Nonnegative Tucker Decomposition (NTD) model imposes nonnegativity constraints onto both core tensor and factor matrices. In this paper, we discuss a mixed version of the models, i.e. where one factor matrix is orthogonal and the remaining factor matrices are nonnegative. Moreover, the nonnegative factor matrices are updated with the modified Barzilai-Borwein gradient projection method that belongs to a class of quasi-Newton methods. The discussed model is efficiently applied to supervised classification of facial images, hand-written digits, and spectrograms of musical instrument sounds.


Facial Image Nonnegative Matrix Factorization Factor Matrice Tensor Factorization Tensor Decomposition 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Tucker, L.R.: Some mathematical notes on three-mode factor analysis. Psychometrika 31, 279–311 (1966)MathSciNetCrossRefGoogle Scholar
  2. 2.
    De Lathauwer, L., de Moor, B., Vandewalle, J.: A multilinear singular value decomposition. SIAM Journal of Matrix Analysis and Applications 21, 1253–1278 (2001)MathSciNetCrossRefzbMATHGoogle Scholar
  3. 3.
    Kiers, H.A.L.: A three-step algorithm for CANDECOMP/PARAFAC analysis of large data sets with multicollinearity. Journal of Chemometrics 12(3), 155–171 (1998)CrossRefGoogle Scholar
  4. 4.
    Smilde, A., Bro, R., Geladi, P.: Multi-way Analysis: Applications in the Chemical Sciences. John Wiley and Sons, New York (2004)CrossRefGoogle Scholar
  5. 5.
    Cichocki, A., Zdunek, R., Phan, A.H., Amari, S.I.: Nonnegative Matrix and Tensor Factorizations: Applications to Exploratory Multi-way Data Analysis and Blind Source Separation. Wiley and Sons, Chichester (2009)CrossRefGoogle Scholar
  6. 6.
    Kolda, T.G., Bader, W.: Tensor decompositions and applications. SIAM Review 51(3), 455–500 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  7. 7.
    Mørup, M., Hansen, L.K., Arnfred, S.M.: Algorithms for sparse nonnegative Tucker decompositions. Neural Computation 20(8), 2112–2131 (2008)CrossRefzbMATHGoogle Scholar
  8. 8.
    Vasilescu, M.A.O., Terzopoulos, D.: Multilinear analysis of image ensembles: TensorFaces. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002. LNCS, vol. 2350, pp. 447–460. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  9. 9.
    Wang, H., Ahuja, N.: Facial expression decomposition. In: Proc. of the Ninth IEEE International Conference on Computer Vision, vol. 2, pp. 958–965 (2003)Google Scholar
  10. 10.
    Vlasic, D., Brand, M., Phister, H., Popovic, J.: Face transfer with multilinear models. ACM Transactions on Graphics 24, 426–433 (2005)CrossRefGoogle Scholar
  11. 11.
    Savas, B., Eldén, L.: Handwritten digit classification using higher order singular value decomposition. Pattern Recognition 40(3), 993–1003 (2007)CrossRefzbMATHGoogle Scholar
  12. 12.
    Kim, Y.D., Choi, S.: Nonnegative tensor decomposition. In: Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2007), Minneapolis, MN, pp. 1–8 (2007)Google Scholar
  13. 13.
    Phan, A.H., Cichocki, A.: Tensor decompositions for feature extraction and classification of high dimensional datasets. IEICE Nonlinear Theory and Its Applications 1(1), 37–68 (2010)CrossRefGoogle Scholar
  14. 14.
    Phan, A.H., Cichocki, A., Vu-Dinh, T.: Classification of scenes based on multiway feature extraction. In: Proc. 2010 International Conference on Advanced Technologies for Communications, Ho Chi Minh City, Vietnam, pp. 142–145 (2010)Google Scholar
  15. 15.
    Lee, D.D., Seung, H.S.: Learning of the parts of objects by non-negative matrix factorization. Nature 401, 788–791 (1999)CrossRefzbMATHGoogle Scholar
  16. 16.
    Kim, H., Park, H., Elden, L.: Non-negative tensor factorization based on alternating large-scale non-negativity-constrained least squares. In: Proc. BIBE 2007, pp. 1147–1151 (2007)Google Scholar
  17. 17.
    Figueiredo, M.A.T., Nowak, R.D., Wright, S.J.: Gradient projection for sparse reconstruction: Application to compressed sensing and other inverse problems. IEEE Journal of Selected Topics in Signal Processing 1(4), 586–597 (2007)CrossRefGoogle Scholar
  18. 18.
    Barzilai, J., Borwein, J.M.: Two-point step size gradient methods. IMA Journal of Numerical Analysis 8(1), 141–148 (1988)MathSciNetCrossRefzbMATHGoogle Scholar
  19. 19.
    Dai, Y.H., Fletcher, R.: Projected Barzilai-Borwein methods for large-scale box-constrained quadratic programming. Numer. Math. 100(1), 21–47 (2005)MathSciNetCrossRefzbMATHGoogle Scholar
  20. 20.
    Zdunek, R., Cichocki, A.: Fast nonnegative matrix factorization algorithms using projected gradient approaches for large-scale problems. In: Computational Intelligence and Neuroscience 2008(939567) (2008)Google Scholar
  21. 21.
    Birgin, E.G., Martínez, J.M., Raydan, M.: Nonmonotone spectral projected gradient methods on convex sets. SIAM Journal on Control and Optimization 10, 1196–1211 (2000)MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Rafal Zdunek
    • 1
  1. 1.Institute of Telecommunications, Teleinformatics and AcousticsWroclaw University of TechnologyWroclawPoland

Personalised recommendations