Trace Norm Regularization and Application to Tensor Based Feature Extraction

  • Yoshikazu Washizawa
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6469)


The trace norm regularization has an interesting property that is rank of a matrix is reduced according to its continuous regularization parameter. We propose a new efficient algorithm for a kind of trace norm regularization problems. Since the algorithm is not gradient-based approach, its computational complexity does not depend on initial states or learning rate. We also apply the proposed algorithm to a tensor based feature extraction method, that is an extension of the trace norm regularized feature extraction.

Computational simulations show that the proposed algorithm provides an accurate solution in less time than conventional methods. The proposed trace based feature extraction method show almost that same performance as Multilinear PCA.


Feature Extraction Singular Value Decomposition Handwritten Digit Alternate Little Square Nuclear Norm 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Washizawa, Y.: Feature extraction using constrained approximation and suppression. IEEE Trans. Neural Networks 21, 201–210 (2010)CrossRefGoogle Scholar
  2. 2.
    Pong, T.K., Tseng, P., Ji, S., Ye, J.: Trace norm regularization: reformulations and multi-task learning (2009),
  3. 3.
    Koren, Y., Bell, R., Volinsky, C.: Matrix factorization techniques for recommender systems. Computer 42, 30–37 (2009)CrossRefGoogle Scholar
  4. 4.
    Cai, J.F., Candeès, E.J., Shen, Z.: A singular value thresholding algorithm for maxrix completion. SIAM J. Optim. 20, 1956–1982 (2010)MathSciNetCrossRefGoogle Scholar
  5. 5.
    Ji, S., Ye, J.: An accelerated gradient method for trace norm minimization. In: Proceedings of the 26th International Conference on Machine Learning (ICML 2009), pp. 457–464 (2009)Google Scholar
  6. 6.
    Jaggi, M., Sulovský, M.: A simple algorithm for nuclear norm regularized problems. In: Proceedings of the 27th International Conference on Machine Learning (ICML 2010), pp. 471–478 (2010)Google Scholar
  7. 7.
    Bach, F.R.: Consistency of trace norm minimization. Journal of Machine Learning Research 8, 1019–1048 (2008)MathSciNetzbMATHGoogle Scholar
  8. 8.
    Lu, H., Plataniotis, K.N., Venetsanopoulos, A.N.: MPCA: multilinear principal component analysis of tensor objects. IEEE Trans. Neural Networks 19, 18–39 (2008)CrossRefGoogle Scholar
  9. 9.
    Turk, M., Pentland, A.: Eigenfaces for recognition. Journal of Cognitive Neuroscience 3, 71–86 (1991)CrossRefGoogle Scholar
  10. 10.
    A & T Laboratories Cambridge: Orl database (2002),

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Yoshikazu Washizawa
    • 1
  1. 1.Brain Science InstituteRiken

Personalised recommendations