Sparse Discriminant Principle Component Analysis

  • Zhihui LaiEmail author
  • Mangqi Chen
  • Dongmei Mo
  • Xingxing Zou
  • Heng Kong
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 849)


Sparse Principal Component Analysis (SPCA) is a regression-type optimization problem based on PCA. The main advantage of SPCA is that it can get modified PCs with sparse loadings so as to improve the performance of feature extraction. However, SPCA does not consider the label information of the data, which degrades its performance in some practical applications. To address this problem, we integrate the property of Least Squares Regression (LSR) into SPCA to use the prior knowledge to obtain the modified PCs with sparsity as well as discriminative information. Moreover, unlike LSR and its derivatives, the number of the modified PCs of SDPCA is not limited by the number of class, namely, SDPCA can address the small-class problem in LSR based methods. To solve the optimization problem, we also propose a new algorithm. Experimental results on product dataset, face dataset and character dataset demonstrate the effectiveness of SDPCA.


Pattern recognition Feature extraction Sparse principal component analysis Least squares regression 



This work was supported in part by the Natural Science Foundation of China (Grant 61573248, Grant 61773328, Grant 61773328 and Grant 61703283), Research Grant of The Hong Kong Polytechnic University (Project Code:G-UA2B), China Postdoctoral Science Foundation (Project 2016M590812 and Project 2017T100645), the Guangdong Natural Science Foundation (Project 2017A030313367 and Project 2017A030310067), and Shenzhen Municipal Science and Technology Innovation Council (No. JCYJ20170302153434048 and No. JCYJ20160429182058044).


  1. 1.
    Turk, M.: Eigenfaces for recognition. J. Cogn. Neurosci. 3, 71–86 (1991)CrossRefGoogle Scholar
  2. 2.
    Belhumeur, P.N., Hespanha, J.P., Kriegman, D.J.: Eigenfaces vs. fisherfaces: recognition using class specific linear projection. IEEE Trans. Pattern Anal. Mach. Intell. 19, 711–720 (1997)CrossRefGoogle Scholar
  3. 3.
    Tibshirani, R.: Regression shrinkage and selection via the Lasso. J. R. Stat. Soc. Ser. B Stat. Methodol. 58, 267–288 (1996)Google Scholar
  4. 4.
    Zou, H., Hastie, T.: Regularization and variable selection via the elastic net. J. R. Stat. Soc. Ser. B. 67, 301–320 (2005)MathSciNetCrossRefGoogle Scholar
  5. 5.
    Zou, H., Hastie, T., Tibshirani, R.: Sparse principal component analysis. J. Comput. Graph. Stat. 15, 1–30 (2004)MathSciNetGoogle Scholar
  6. 6.
    Qiao, Z., Zhou, L., Huang, J.Z.: Sparse linear discriminant analysis with applications to high dimensional low sample size data. IAENG Int. J. Appl. Math. 39, 48–60 (2009)MathSciNetzbMATHGoogle Scholar
  7. 7.
    Accelerator, S.L.: Regularized Discriminant Analysis. Publ. Am. Stat. Assoc. 84, 165–175 (1989)MathSciNetCrossRefGoogle Scholar
  8. 8.
    Xiang, S., Nie, F., Meng, G., Pan, C., Zhang, C.: Discriminative least squares regression for multiclass classification and feature selection. IEEE Trans. Neural Networks Learn. Syst. 23, 1738–1754 (2012)CrossRefGoogle Scholar
  9. 9.
    Zou, H., Hastie, T.: Regression shrinkage and selection via the elastic net, with applications to microarrays. J. R. Stat. Soc. Ser. B. 67, 301–320 (2003)CrossRefGoogle Scholar
  10. 10.
    Kwak, N.: Principal component analysis based on L1-norm maximization. IEEE Trans. Pattern Anal. Mach. Intell. 30, 1672–1680 (2008)CrossRefGoogle Scholar
  11. 11.
    Xiao, H., Rasul, K., Vollgraf, R.: Fashion-MNIST: a novel image dataset for benchmarking machine learning algorithms. 1–6 (2017)Google Scholar
  12. 12.
    Martinez, A.A., Benavente, R.: The AR face database. CVC Tech. Reptort #24. (1998)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Zhihui Lai
    • 1
    Email author
  • Mangqi Chen
    • 1
  • Dongmei Mo
    • 1
  • Xingxing Zou
    • 2
  • Heng Kong
    • 3
  1. 1.College of Computer Science and Software EngineeringShenzhen UniversityShenzhenChina
  2. 2.Institute of Textiles and ClothingThe Hong Kong Polytechnic UniversityHong KongChina
  3. 3.School of MedicineShenzhen UniversityShenzhenChina

Personalised recommendations