Weakly-Paired Maximum Covariance Analysis for Multimodal Dimensionality Reduction and Transfer Learning

  • Christoph H. Lampert
  • Oliver Krömer
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6312)


We study the problem of multimodal dimensionality reduction assuming that data samples can be missing at training time, and not all data modalities may be present at application time. Maximum covariance analysis, as a generalization of PCA, has many desirable properties, but its application to practical problems is limited by its need for perfectly paired data. We overcome this limitation by a latent variable approach that allows working with weakly paired data and is still able to efficiently process large datasets using standard numerical routines. The resulting weakly paired maximum covariance analysis often finds better representations than alternative methods, as we show in two exemplary tasks: texture discrimination and transfer learning.


Dimensionality Reduction Linear Discriminant Analysis Local Binary Pattern Canonical Correlation Analysis Transfer Learning 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Andrews, S., Tsochantaridis, I., Hofmann, T.: Support vector machines for multiple-instance learning. In: NIPS (2003)Google Scholar
  2. 2.
    Bach, F.R., Jordan, M.I.: A probabilistic interpretation of canonical correlation analysis. Technical Report 688, Department of Statistics, University of California, Berkeley (2005)Google Scholar
  3. 3.
    Baudat, G., Anouar, F.: Generalized discriminant analysis using a kernel approach. Neural Computation 12(10) (2000)Google Scholar
  4. 4.
    Blaschko, M., Gretton, A.: Learning taxonomies by dependence maximization. In: NIPS (2009)Google Scholar
  5. 5.
    Blaschko, M., Lampert, C.H.: Correlational spectral clustering. In: CVPR (2008)Google Scholar
  6. 6.
    Fisher, R.A.: The use of multiple measurements in taxonomic problems. Annals of Eugenics 7 (1936)Google Scholar
  7. 7.
    Golub, G.H., Van Loan, C.F.: Matrix computations. Johns Hopkins Univ. Press, Baltinmore (1996)zbMATHGoogle Scholar
  8. 8.
    Hardoon, D., Mourao-Miranda, J., Brammer, M., Shawe-Taylor, J.: Unsupervised analysis of fMRI data using kernel canonical correlation. NeuroImage 37(4) (2007)Google Scholar
  9. 9.
    Hardoon, D., Szedmak, S., Shawe-Taylor, J.: Canonical correlation analysis: an overview with application to learning methods. Neural Computation 16(12) (2004)Google Scholar
  10. 10.
    Hinton, G., Salakhutdinov, R.: Reducing the dimensionality of data with neural networks. Science 313(5786) (2006)Google Scholar
  11. 11.
    Hofmann, T.: Probabilistic latent semantic indexing. In: ACM SIGIR (1999)Google Scholar
  12. 12.
    Hotelling, H.: Relation between two sets of variates. Biometrika 28 (1936)Google Scholar
  13. 13.
    Jonker, R., Volgenant, A.: A shortest augmenting path algorithm for dense and sparse linear assignment problems. Computing 38(4) (1987)Google Scholar
  14. 14.
    Kuhn, H.W.: The hungarian method for the assignment problem. Naval Research Logistics Quarterly 2 (1955)Google Scholar
  15. 15.
    Lampert, C.H., Nickisch, H., Harmeling, S.: Learning to detect unseen object classes by between-class attribute transfer. In: CVPR (2009)Google Scholar
  16. 16.
    Lei, Z., Bai, Q., He, R., Li, S.Z.: Face shape recovery from a single image using CCA mapping between tensor spaces. In: CVPR (2008)Google Scholar
  17. 17.
    Lienhart, R., Romberg, S., Hörster, E.: Multilayer pLSA for multimodal image retrieval. In: CIVR (2009)Google Scholar
  18. 18.
    Livescu, K., Stoehr, M.: Multi-view learning of acoustic features for speaker recognition. In: Automatic Speech Recognition and Understanding (2009)Google Scholar
  19. 19.
    Logan, B.: Mel frequency cepstral coefficients for music modeling. In: International Symposium on Music Information Retrieval (2000)Google Scholar
  20. 20.
    MacQueen, J.: Some methods for classification and analysis of multivariate observations. In: 5th Berkeley Symposium on Mathematics Statistics and Probability (1967)Google Scholar
  21. 21.
    Ojala, T., Pietikainen, M., Maenpaa, T.: Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. PAMI 24(7) (2002)Google Scholar
  22. 22.
    Pan, S.J., Kwok, J.T., Yang, Q.: Transfer learning via dimensionality reduction. In: AAAI (2008)Google Scholar
  23. 23.
    Pan, S.J., Yang, Q.: A survey on transfer learning. Knowledge and Data Engineering (2009)Google Scholar
  24. 24.
    Papadimitriou, C.H., Raghavan, P., Tamaki, H., Vempala, S.: Latent semantic indexing: A probabilistic analysis. Computer and System Sciences 61(2) (2000)Google Scholar
  25. 25.
    Pearson, K.: On lines and planes of closest fit to systems of points in space. Philosophical Magazine Series 6. 2(11) (1901)Google Scholar
  26. 26.
    Rosenberg, A., Hirschberg, J.: V-measure: A conditional entropy-based external cluster evaluation measure. In: EMNLP-CoNLL (2007)Google Scholar
  27. 27.
    Rosipal, R., Trejo, L.J.: Kernel partial least squares regression in reproducing kernel Hilbert space. JMLR 2 (2002)Google Scholar
  28. 28.
    Schölkopf, B., Smola, A.J.: Learning with kernels. MIT Press, Cambridge (2002)Google Scholar
  29. 29.
    Tang, K., Tappen, M., Sukthankar, R., Lampert, C.H.: Optimizing one-shot recognition with micro-set learning. In: CVPR (2010)Google Scholar
  30. 30.
    Tenenbaum, J.B., Silva, V., Langford, J.C.: A global geometric framework for nonlinear dimensionality reduction. Science 290(5500) (2000)Google Scholar
  31. 31.
    Tucker, L.R.: An inter-battery method of factor analysis. Psychometrika 23 (1958)Google Scholar
  32. 32.
    Tuytelaars, T., Lampert, C.H., Blaschko, M.B., Buntine, W.: Unsupervised object discovery: A comparison. IJCV 88(2) (2010)Google Scholar
  33. 33.
    Wold, H.: Estimation of principal components and related models by iterative least squares. Multivariate Analysis 1 (1966)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Christoph H. Lampert
    • 1
  • Oliver Krömer
    • 2
  1. 1.Institute of Science and Technology AustriaKlosterneuburgAustria
  2. 2.Max Planck Institute for Biological CyberneticsTübingenGermany

Personalised recommendations