Skip to main content
SpringerLink
Log in

A Flexible and Effective Linearization Method for Subspace Learning

  • Chapter
  • First Online:
Graph Embedding for Pattern Analysis

Abstract

In the past decades, a large number of subspace learning or dimension reduction methods [2, 16, 20, 32, 34, 37, 44] have been proposed. Principal component analysis (PCA) [32] pursues the directions of maximum variance for optimal reconstruction. Linear discriminant analysis (LDA) [2], as a supervised algorithm, aims to maximize the inter-class scatter and at the same time minimize the intra-class scatter. Due to utilization of label information, LDA is experimentally reported to outperform PCA for face recognition, when sufficient labeled face images are provided [2].

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    An alternative method is to solve \({\min }_{F,W,b,{W}^{\mathrm{T}}W=I}\mathrm{Tr}({F}^{\mathrm{T}}MF) + \gamma \|{X}^{\mathrm{T}}W + \mathbf{1}{b}^{\mathrm{T}} - {F\|}^{2})\), which is equivalent to \({\min }_{F,W,{W}^{\mathrm{T}}W=I}\mathrm{Tr}({F}^{\mathrm{T}}MF) + \gamma \|{X}^{\mathrm{T}}W - {F\|}^{2})\).

  2. 2.

    Available at http://people.csail.mit.edu/jrennie/20Newsgroups/.

References

  1. Abernethy J, Chapelle O, Castillo C (2008) Web spam identification through content and hyperlinks. In: Proceedings of the international workshop on adversarial information retrieval on the web, pp 41–44

    Google Scholar 

  2. Belhumeur P, Hespanha J, Kriegman D (1997) Eigenfaces vs. fisherfaces: Recognition using class specific linear projection. IEEE Trans Pattern Anal Mach Intell 19(7):711–720

    Google Scholar 

  3. Belkin M, Niyogi P (2001) Laplacian eigenmaps and spectral techniques for embedding and clustering. In: Advances in neural information processing systems

    Google Scholar 

  4. Belkin M, Niyogi P, Sindhwani V (2006) Manifold regularization: A geometric framework for learning from labeled and unlabeled examples. J Mach Learn Res 12:2399–2434

    MathSciNet  Google Scholar 

  5. Blum A, Mitchell T (1998) Combining labeled and unlabeled data with co-training. In: Proceedings of the annual conference on learning theory

    Google Scholar 

  6. Cai D (2009) Spectral regression: A regression framework for efficient regularized subspace learning. PhD thesis, Department of Computer Science, University of Illinois at Urbana-Champaign

    Google Scholar 

  7. Cai D, He X, Han J (2007a) Semi-supervised discriminant analysis. In: Proceedings of the IEEE international conference on computer vision

    Google Scholar 

  8. Cai D, He X, Han J (2007b) Spectral regression for efficient regularized subspace learning. In: Proceedings of the IEEE international conference on computer vision

    Google Scholar 

  9. Chu W, Sindhwani V, Ghahramani Z, Keerthi SS (2006) Relational learning with Gaussian processes. In: Advances in neural information processing systems

    Google Scholar 

  10. Georghiades A, Belhumeur P, Kriegman D (2001) From few to many: Illumination cone models for face recognition under variable lighting and pose. IEEE Trans Pattern Anal Mach Intell 23(6):643–660

    Article  Google Scholar 

  11. Graham DB, Allinson NM (1998) Characterizing virtual eigensignatures for general purpose face recognition. NATO ASI Series F, pp 446–456

    Google Scholar 

  12. Guo Z, Zhang Z, Xing E, Faloutsos C (2008) Semi-supervised learning based on semiparametric regularization. In: Proceedings of the SIAM international conference on data mining

    Google Scholar 

  13. He X, Yan S, Hu Y, Niyogi P, Zhang H (2005) Face recognition using laplacianfaces. IEEE Trans Pattern Anal Mach Intell 27(3):328–340

    Article  Google Scholar 

  14. Huang Y, Xu D, Nie F (2012) Semi-supervised dimension reduction using trace ratio criterion. IEEE Trans Neural Networks Learn Syst 23(3):519–526

    Article  Google Scholar 

  15. Jia Y, Nie F, Zhang C (2009) Trace ratio problem revisited. IEEE Trans Neural Networks 20(4):729–735

    Article  Google Scholar 

  16. Li X, Lin S, Yan S, Xu D (2008) Discriminant locally linear embedding with high order tensor data. IEEE Trans Syst Man Cybernet B 38(2):342–352

    Article  Google Scholar 

  17. Liu C, Shum HY, Freeman WT (2007) Face hallucination: Theory and practice. Int J Comput Vision 75(1):115–134

    Article  Google Scholar 

  18. Liu W, Tao D, Liu J (2008) Transductive Component Analysis. In: Proceedings of the IEEE international conference on data mining

    Google Scholar 

  19. Nene SA, Nayar SK, Murase H (1996) Columbia object image library (coil-20). Technical report, Columbia University

    Google Scholar 

  20. Nie F, Huang H, Cai X, Ding C (2010) Efficient and robust feature selection via joint ℓ 2, 1-norms minimization. In: NIPS

    Google Scholar 

  21. Nie FP, Xiang XM, Jia YQ, Zhang CS (2009a) Semi-supervised orthogonal discriminant analysis via label propagation. Pattern Recogn 42(11):2615–2627

    Article  MATH  Google Scholar 

  22. Nie F, Xiang S, Song Y, Zhang C (2009b) Orthogonal locality minimizing globality maximizing projections for feature extraction. Opt Eng 48:017202

    Article  Google Scholar 

  23. Nie F, Xu D, Wai-Hung Tsang I, Zhang C (2010) Flexible manifold embedding: A framework for semi-supervised and unsupervised dimension reduction. IEEE Trans Image Process 19(7):1921–1932

    Article  MathSciNet  Google Scholar 

  24. Nie F, Xu D, Li X, Xiang S (2011) Semisupervised dimensionality reduction and classification through virtual label regression. IEEE Trans Syst Man Cybernet B 41(3):675–685

    Article  Google Scholar 

  25. Roweis S, Saul L (2000) Nonlinear dimensionality reduction by locally linear embedding. Science 290(22):2323–2326

    Article  Google Scholar 

  26. Sim T, Baker S (2003) The cmu pose, illumination, and expression database. IEEE Trans Pattern Anal Mach Intell 25(12):1615–1617

    Article  Google Scholar 

  27. Sindhwani V, Melville P (2008) Document-word co-regularization for semi-supervised sentiment analysis. In: Proceedings of the IEEE international conference on data mining

    Google Scholar 

  28. Sindhwani V, Niyogi P, Belkin M (2005a) Beyond the point cloud: from transductive to semi-supervised learning. In: Proceedings of the international conference on machine learning

    Google Scholar 

  29. Sindhwani V, Niyogi P, Belkin M (2005b) Linear manifold regularization for large scale semi-supervised learning. In: Workshop on learning with partially classified training data, international conference on machine learning

    Google Scholar 

  30. Song Y, Nie F, Zhang C, Xiang S (2008) A unified framework for semi-supervised dimensionality reduction. Pattern Recogn 41(9):2789–2799

    Article  MATH  Google Scholar 

  31. Tenenbaum J, Silva V, Langford J (2000) A global geometric framework for nonlinear dimensionality reduction. Science 290(22):2319–2323

    Article  Google Scholar 

  32. Turk M, Pentland A (1991) Face recognition using eigenfaces. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition

    Google Scholar 

  33. Vapnik V (1998) Statistical learning theory. Wiley-Interscience, New York

    MATH  Google Scholar 

  34. Xiang S, Nie F, Zhang C (2008) Learning a mahalanobis distance metric for data clustering and classification. Pattern Recogn 41(12):3600–3612

    Article  MATH  Google Scholar 

  35. Xiang S, Nie F, Zhang C, Zhang C (2009) Nonlinear dimensionality reduction with local spline embedding. IEEE Trans Knowl Data Eng 21(9):1285–1298

    Article  Google Scholar 

  36. Xiang S, Nie F, Zhang C (2010) Semi-supervised classification via local spline regression. IEEE Trans PAMI 32(11):2039–2053

    Article  MathSciNet  Google Scholar 

  37. Yan S, Xu D, Zhang B, Zhang H, Yang Q, Lin S (2007) Graph embedding and extensions: A general framework for dimensionality reduction. IEEE Trans Pattern Anal Mach Intell 29(1):40–51

    Article  Google Scholar 

  38. Yang X, Fu H, Zha H, Barlow JL (2006) Semi-supervised nonlinear dimensionality reduction. In: Proceedings of the international conference on machine learning, pp 1065–1072

    Google Scholar 

  39. Yang Y, Nie F, Xu D, Luo J, Yunhe Pan YZ (2012) A multimedia retrieval framework based on semi-supervised ranking and relevance feedback. IEEE Trans Pattern Anal Mach Intell, pp 723–742

    Google Scholar 

  40. Zhang C, Nie F, Xiang S (2010) A general kernelization framework for learning algorithms based on kernel PCA. Neurocomputing 73(4–6):959–967

    Article  Google Scholar 

  41. Zhang T, Popescul A, Dom B (2006) Linear prediction models with graph regularization for web-page categorization. In: Proceedings of the ACM SIGKDD international conference on knowledge discovery and data mining, pp 821–826

    Google Scholar 

  42. Zhang T, Tao D, Yang J (2008a) Discriminative locality alignment. In: Proceedings of the European conference on computer vision, pp 725–738

    Google Scholar 

  43. Zhang T, Tao D, Li X, Yang J (2008b) A unifying framework for spectral analysis based dimensionality reduction. In: IEEE international joint conference on neural networks, pp 1671–1678

    Google Scholar 

  44. Zhang T, Tao D, Li X, and Yang J (2009) Patch alignment for dimensionality reduction. IEEE Trans. Knowledge Data Eng, 21(9):1299–1313

    Article  Google Scholar 

  45. Zhang Z, Zha H (2005) Principal manifolds and nonlinear dimension reduction via local tangent space alignment. SIAM J Sci Comput 26:313–338

    Article  MathSciNet  Google Scholar 

  46. Zhao D (2006) Formulating lle using alignment technique. Pattern Recogn 39(11):2233–2235

    Article  MATH  Google Scholar 

  47. Zhou D, Bousquet O, Lal TN, Weston J, Schölkopf B (2004) Learning with local and global consistency. In: Advances in neural information processing systems

    Google Scholar 

  48. Zhu X (2007) Semi-supervised learning literature survey. Technical report, University of Wisconsin-Madison

    Google Scholar 

  49. Zhu X, Ghahramani Z, Lafferty J (2003) Semi-supervised learning using gaussian fields and harmonic functions. In: Proceedings of the international conference on machine learning

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Texas, Arlington, USA

    Feiping Nie

  2. Nanyang Technological University, Singapore, Singapore

    Dong Xu & Ivor W. Tsang

  3. Tsinghua University, Beijing, 100084, China

    Changshui Zhang

Authors
  1. Feiping Nie
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dong Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ivor W. Tsang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Changshui Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Feiping Nie .

Editor information

Editors and Affiliations

  1. , Dept. of ECE, College of Engineering, Northeastern University, 403 Dana Research Center, 360 Huntington Ave, Boston, 02115, Massachusetts, USA

    Yun Fu

  2. Honeywell, Douglas Drive North 1985, Golden Valley, 55422, USA

    Yunqian Ma

Rights and permissions

Reprints and permissions

Copyright information

© 2013 Springer Science+Business Media New York

About this chapter

Cite this chapter

Nie, F., Xu, D., Tsang, I.W., Zhang, C. (2013). A Flexible and Effective Linearization Method for Subspace Learning. In: Fu, Y., Ma, Y. (eds) Graph Embedding for Pattern Analysis. Springer, New York, NY. https://doi.org/10.1007/978-1-4614-4457-2_8

Download citation

  • DOI: https://doi.org/10.1007/978-1-4614-4457-2_8

  • Published:

  • Publisher Name: Springer, New York, NY

  • Print ISBN: 978-1-4614-4456-5

  • Online ISBN: 978-1-4614-4457-2

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation