Applied Intelligence

, Volume 48, Issue 12, pp 4923–4936 | Cite as

Supervised data-dependent kernel sparsity preserving projection for image recognition

  • Ao Zhang
  • Xianwen GaoEmail author


Dimensionality reduction methods have commonly been used as principled ways to understand high-dimensional data. In this paper, a novel non-linear method named supervised data-dependent kernel sparsity preserving projection (SDKSPP) is proposed for dimensionality reduction. SDKSPP is a non-linear extension of sparsity preserving projection, it adopts a data-dependent kernel (DK) instead of standard kernels to achieve performance improvements. Different from previous dimensionality reduction methods based on DK, SDKSPP can simultaneously optimize the coefficients in DK and explore the manifold structure, i.e., the sparse reconstructive relationship of data. The manifold structure in the feature space is shared in the label space so that the information of labels can be utilized to help optimizing the coefficients in DK and improving the discriminative ability. After the optimal sparse reconstructive relationship is obtained, a transform matrix that can preserve this relationship is calculated to project the mapped data into a low-dimensional space. The effectiveness of the proposed method is tested and compared on nine popular databases.


Sparsity preserving projection Data-dependent kernel Dimensionality reduction 



This work is partially supported by the National Natural Science Foundation of China (No. 61573088, No. 61573087 and No. 61433004).


  1. 1.
    Turk MA, Pentland AP (1991) Face recognition using eigenfaces. In: International conference on computer research and development, CVPR. IEEE, pp 302–306Google Scholar
  2. 2.
    Belhumeur PN, Hespanha JP, Kriengman DJ (1997) Eigenfaces vs. Fisherfaces: Recognition using class specific linear projection. IEEE Trans Pattern Anal Mach Intell 19(7):711–720CrossRefGoogle Scholar
  3. 3.
    Tenenbaum JB, De SV, Langford JC (2000) A global geometric framework for nonlinear dimensionality reduction. Science 290(5500):2319CrossRefGoogle Scholar
  4. 4.
    Roweis ST, Saul LK (2000) Nonlinear dimensionality reduction by locally linear embedding. Science 290 (5500):2323CrossRefGoogle Scholar
  5. 5.
    Qiao L, Chen S, Tan X (2010) Sparsity preserving projections with applications to face recognition. Pattern Recogn 43(1):331– 341CrossRefGoogle Scholar
  6. 6.
    Zhang J, Wang J, Cai X (2017) Sparse locality preserving discriminative projections for face recognition. Neurocomputing 260:321–330CrossRefGoogle Scholar
  7. 7.
    Lou S, Zhao X, Chuang Y, Zhang S (2016) Graph Regularized Sparsity Discriminant Analysis for face recognition. Neurocomputing 173(P2):290–297CrossRefGoogle Scholar
  8. 8.
    Zhang P, You X, Ou W, Chen CLP, Cheung YM (2016) Sparse discriminative multi-manifold embedding for one-sample face identification. Pattern Recogn 52(C):249–259CrossRefGoogle Scholar
  9. 9.
    Gao S, Tsang WH, Chia LT (2010) Kernel Sparse Representation for Image Classification and Face Recognition. In: European Conference on Computer Vision, ECCV, pp 1–14Google Scholar
  10. 10.
    Lin C, Wang B, Zhao X, Pang M (2013) Optimizing kernel PCA using sparse representation-based classifier for MSTAR SAR image target recognition. Mathematical Problems in Engineering 2013(6)707–724Google Scholar
  11. 11.
    Lee MMS, Keerthi SS, Ong CJ, Decoste D (2004) An efficient method for computing leave-one-out error in support vector machines with Gaussian kernels. IEEE Trans Neural Netw 15(3):750–757CrossRefGoogle Scholar
  12. 12.
    Cristianini N, Kandola J, Elisseeff A, Shawe-Taylor J (2002) On Kernel-Target alignment. Adv Neural Inform Process Syst 179(5):367–373Google Scholar
  13. 13.
    Lanckriet GRG, Cristianini N, Bartlett P, Ghaoui LE, Jordan MI (2004) Learning the kernel matrix with Semi-Definite programming. J Mach Learn Res 5(Jan):27–72zbMATHGoogle Scholar
  14. 14.
    Ong CS, Smola AJ, Williamson RC (2005) Learning the kernel with hyperkernels. J Mach Learn Res 6 (1):1043–1071MathSciNetzbMATHGoogle Scholar
  15. 15.
    Amari S, Wu S (1999) Improving support vector machine classifiers by modifying kernel functions. Neural Netw 12(6):783–789CrossRefGoogle Scholar
  16. 16.
    Xiong H, Zhang Y, Chen XW (2007) Data-Dependent Kernel machines for microarray data classification. IEEE/ACM Trans Comput Biol Bioinform 4(4):583–595CrossRefGoogle Scholar
  17. 17.
    Xiong H, Swamy MN, Ahmad MO (2005) Optimizing the kernel in the empirical feature space. IEEE Trans Neural Netw 16(2):460–474CrossRefGoogle Scholar
  18. 18.
    Luo L, Bao S, Mao J, Tang D (2016) Nonlinear process monitoring based on kernel global-local preserving projections. J Process Control 38:11–21CrossRefGoogle Scholar
  19. 19.
    He Z, Li J (2015) Multiple data-dependent kernel for classification of hyperspectral images. Expert Syst Appl 42(3):1118–1135CrossRefGoogle Scholar
  20. 20.
    Motai Y, Yoshida H (2013) Principal composite kernel feature analysis: Data-dependent kernel approach. IEEE Trans Knowl Data Eng 25(8):1863–1875CrossRefGoogle Scholar
  21. 21.
    Li JB, Wang YH, Chu SC, Roddick JF (2014) Kernel self-optimization learning for kernel-based feature extraction and recognition. Inf Sci 257(2):70–80MathSciNetCrossRefGoogle Scholar
  22. 22.
    Gu N, Wang D, Fan M, Meng D (2014) A kernel-based sparsity preserving method for semi-supervised classification. Neurocomputing 139:345–356CrossRefGoogle Scholar
  23. 23.
    Hou P, Geng X, Zhang ML (2016) Multi-Label Manifold Learning. In: Thirtieth AAAI Conference on Artificial Intelligence, AAAI, pp 1680–1686Google Scholar
  24. 24.
    Smola AJ, Schölkopf B, Müller KR (1998) The connection between regularization operators and support vector kernels. Neural Netw 11(4):637–649CrossRefGoogle Scholar
  25. 25.
    Wang D, Zhang M, Cai Z et al (2016) Combatting nonlinear phase noise in coherent optical systems with an optimized decision processor based on machine learning. Opt Commun 369:199–208CrossRefGoogle Scholar
  26. 26.
    Grant M, Boyd S (2008) Graph implementations for nonsmooth convex programs. Lect Notes Control Inf Sci 371:95–110MathSciNetzbMATHGoogle Scholar
  27. 27.
    Wen J, Fang X, Cui J et al (2018) Robust sparse linear discriminant analysis. IEEE Trans Circ Syst Video Technol PP(99):1–13Google Scholar
  28. 28.
    Martínez A M, Kak AC (2001) PCA versus LDA. IEEE Trans Pattern Anal Mach Intell 23(2):228–233CrossRefGoogle Scholar
  29. 29.
    Samaria FS, Harter AC (1994) Parameterisation of a stochastic model for human face identification. In: Proceedings of the Second IEEE Workshop on Applications of Computer Vision. IEEE, pp 138–142Google Scholar
  30. 30.
    Georghiades A, Belhumeur PN, Kriegman DJ (1997) Yale face database, Center for computational Vision and Control at Yale University,
  31. 31.
    Hull JJ (1998) A database for handwritten text recognition research. IEEE Trans Pattern Anal Mach Intell 16(5):550–554CrossRefGoogle Scholar
  32. 32.
    LeCun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324CrossRefGoogle Scholar
  33. 33.
    Yuan S, Mao X, Chen L (2018) Sparsity regularization discriminant projection for feature extraction. Neural Processing Letters.
  34. 34.
    Zheng J, Qiu H, Wang W et al (2016) Kernel-based discriminative elastic embedding algorithm. Appl Intell 44(2):449–456CrossRefGoogle Scholar
  35. 35.
    Luo T, Yang Y, Yi D et al (2017) Robust discriminative feature learning with calibrated data reconstruction and sparse low-rank model. Applied Intelligence.
  36. 36.
    Wright J, Yang A, Sastry S, Ma Y (2009) Robust face recognition via sparse representation. IEEE Trans Pattern Anal Mach Intell 31(2):210–227CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.College of Information Science and EngineeringNortheastern UniversityShenyangChina

Personalised recommendations