Sparse Discriminant Principle Component Analysis
Sparse Principal Component Analysis (SPCA) is a regression-type optimization problem based on PCA. The main advantage of SPCA is that it can get modified PCs with sparse loadings so as to improve the performance of feature extraction. However, SPCA does not consider the label information of the data, which degrades its performance in some practical applications. To address this problem, we integrate the property of Least Squares Regression (LSR) into SPCA to use the prior knowledge to obtain the modified PCs with sparsity as well as discriminative information. Moreover, unlike LSR and its derivatives, the number of the modified PCs of SDPCA is not limited by the number of class, namely, SDPCA can address the small-class problem in LSR based methods. To solve the optimization problem, we also propose a new algorithm. Experimental results on product dataset, face dataset and character dataset demonstrate the effectiveness of SDPCA.
KeywordsPattern recognition Feature extraction Sparse principal component analysis Least squares regression
This work was supported in part by the Natural Science Foundation of China (Grant 61573248, Grant 61773328, Grant 61773328 and Grant 61703283), Research Grant of The Hong Kong Polytechnic University (Project Code:G-UA2B), China Postdoctoral Science Foundation (Project 2016M590812 and Project 2017T100645), the Guangdong Natural Science Foundation (Project 2017A030313367 and Project 2017A030310067), and Shenzhen Municipal Science and Technology Innovation Council (No. JCYJ20170302153434048 and No. JCYJ20160429182058044).
- 3.Tibshirani, R.: Regression shrinkage and selection via the Lasso. J. R. Stat. Soc. Ser. B Stat. Methodol. 58, 267–288 (1996)Google Scholar
- 11.Xiao, H., Rasul, K., Vollgraf, R.: Fashion-MNIST: a novel image dataset for benchmarking machine learning algorithms. 1–6 (2017)Google Scholar
- 12.Martinez, A.A., Benavente, R.: The AR face database. CVC Tech. Reptort #24. (1998)Google Scholar