Abstract
In data mining and machine learning, the embedding methods have commonly been used as a principled way to understand the high-dimensional data. To solve the out-of-sample problem, local preserving projection (LPP) was proposed and applied to many applications. However, LPP suffers two crucial deficiencies: 1) the LPP has no shift-invariant property which is an important property of embedding methods; 2) the rigid linear embedding is used as constraint, which often inhibits the optimal manifold structures finding. To overcome these two important problems, we propose a novel flexible shift-invariant locality and globality preserving projection method, which utilizes a newly defined graph Laplacian and the relaxed embedding constraint. The proposed objective is very challenging to solve, hence we derive a new optimization algorithm with rigorously proved global convergence. More importantly, we prove our optimization algorithm is a Newton method with fast quadratic convergence rate. Extensive experiments have been performed on six benchmark data sets. In all empirical results, our method shows promising results.
Chapter PDF
Similar content being viewed by others
Keywords
- Small Eigenvalue
- Laplacian Matrix
- Preserve Projection
- Locality Preserve Projection
- Nonlinear Dimensionality Reduction
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Belhumeur, P., Hespanha, J., Kriegman, D.: Eigenfaces vs. fisherfaces: Recognition using class specific linear projection. IEEE Transactions on Pattern Analysis and Machine Intelligence 19(7), 711–720 (1997)
Belkin, M., Niyogi, P.: Laplacian eigenmaps for dimensionality reduction and data representation. Neural Computation 15(6), 1373–1396 (2003)
Cai, D.: Spectral Regression: A Regression Framework for Efficient Regularized Subspace Learning. PhD thesis, Department of Computer Science, University of Illinois at Urbana-Champaign (May 2009)
Fisher, R.A.: The use of multiple measurements in taxonomic problems. Annals Eugen. 7, 179–188 (1936)
Georghiades, A.S., Belhumeur, P.N., Kriegman, D.J.: From few to many: Illumination cone models for face recognition under variable lighting and pose. IEEE Trans. Pattern Anal. Mach. Intell. 23(6), 643–660 (2001)
Graham, D.B., Allinson, N.M.: Characterizing virtual eigensignatures for general purpose face recognition. in face recognition: From theory to applications. NATO ASI Series F, Computer and Systems Sciences 163, 446–456 (1998)
He, X., Niyogi, P.: Locality preserving projections. In: NIPS (2003)
http://www.cl.cam.ac.uk/research/dtg/attarchive/facedatabase.html
Huang, Y., Xu, D., Nie, F.: Semi-supervised dimension reduction using trace ratio criterion. IEEE Trans. Neural Netw. Learning Syst. 23(3), 519–526 (2012)
Jia, Y., Nie, F., Zhang, C.: Trace ratio problem revisited. IEEE Transactions on Neural Networks 20(4), 729–735 (2009)
Jolliffe, I.T.: Principal Component Analysis, 2nd edn. Springer, New York (2002)
Martinez, A.: The ar face database. CVC Technical Report, 24 (1998)
Nayar, S., Nene, S., Murase, H.: Real-time 100 object recognition system. In: Proceedings of the 1996 IEEE International Conference on Robotics and Automation, vol. 3, pp. 2321–2325. IEEE (1996)
Nene, S.A., Nayar, S.K., Murase, H.: Columbia object image library (COIL-20), Technical Report CUCS-005-96. Columbia University (1996)
Ng, A.Y., Jordan, M.I., Weiss, Y.: On spectral clustering: Analysis and an algorithm. In: NIPS, pp. 849–856 (2001)
Nie, F., Xiang, S., Song, Y., Zhang, C.: Orthogonal locality minimizing globality maximizing projections for feature extraction. Optical Engineering 48, 017202 (2009)
Nie, F., Xiang, S., Zhang, C.: Neighborhood minmax projections. In: IJCAI, pp. 993–998 (2007)
Nie, F., Xu, D., Tsang, I.W.-H., Zhang, C.: Flexible manifold embedding: A framework for semi-supervised and unsupervised dimension reduction. IEEE Transactions on Image Processing 19(7), 1921–1932 (2010)
Roweis, S.T., Al, E.: Nonlinear dimensionality reduction by locally linear embedding. Science 290, 2323–2326 (2000)
Samaria, F.S., Harter, A.C.: Parameterisation of a stochastic model for human face identification. In: 2nd IEEE Workshop on Applications of Computer Vision, pp. 138–142 (1994)
Shi, J., Malik, J.: Normalized cuts and image segmentation. IEEE Transactions on PAMI 22(8), 888–905 (2000)
Tenenbaum, J.B., de Silva, V., Langford, J.C.: A global geometric framework for nonlinear dimensionality reduction. Science 290(5500), 2319–2323 (2000)
Wang, H., Yan, S., Xu, D., Tang, X., Huang, T.S.: Trace ratio vs. ratio trace for dimensionality reduction. In: CVPR (2007)
Xiang, S., Nie, F., Zhang, C., Zhang, C.: Nonlinear dimensionality reduction with local spline embedding. IEEE Transactions on Knowledge and Data Engineering 21(9), 1285–1298 (2009)
Zhang, Z., Zha, H.: Principal manifolds and nonlinear dimension reduction via local tangent space alignment. SIAM Journal of Scientific Computing 26, 313–338 (2005)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Nie, F., Cai, X., Huang, H. (2014). Flexible Shift-Invariant Locality and Globality Preserving Projections. In: Calders, T., Esposito, F., Hüllermeier, E., Meo, R. (eds) Machine Learning and Knowledge Discovery in Databases. ECML PKDD 2014. Lecture Notes in Computer Science(), vol 8725. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-662-44851-9_31
Download citation
DOI: https://doi.org/10.1007/978-3-662-44851-9_31
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-662-44850-2
Online ISBN: 978-3-662-44851-9
eBook Packages: Computer ScienceComputer Science (R0)