Robust Discriminative Principal Component Analysis

  • Xiangxi Xu
  • Zhihui LaiEmail author
  • Yudong Chen
  • Heng Kong
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10996)


Least square regression (LSR) and principal component analysis (PCA) are two representative dimensionality reduction algorithms in the fields of machine learning. In this paper, we propose a novel method to jointly learn projections from the subspaces derived from the modified LSR and PCA. To implement simultaneous feature learning, we design a novel joint regression learning model by imposing two orthogonal constraints. Therefore, the learned projections can preserve the minimum reconstruction error and the discriminative information in the low-dimensional subspaces. Besides, since the traditional LSR and PCA are sensitive to the outliers, we utilize the robust L2,1-norm as the metric of loss function to improve the model’s robustness. A simple iterative algorithm is proposed to solve the proposed framework. Experiments on face databases show the promising performance of our method.


Regression framework Subspace learning Robustness 



This work was supported in part by the Natural Science Foundation of China (Grant 61573248, Grant 61773328, Grant 61773328 and Grant 61703283), Research Grant of The Hong Kong Polytechnic University (Project Code:G-UA2B), China Postdoctoral Science Foundation (Project 2016M590812 and Project 2017T100645), the Guangdong Natural Science Foundation (Project 2017A030313367 and Project 2017A030310067), the Guangdong medical scientific and technological research funding under grant A2017251, Shenzhen Municipal Science and Technology Innovation Council (No. JCYJ20170302153434048, No. JCYJ20160429182058044 and No. JCYJ20160429182058044).


  1. 1.
    Abdi, H., Williams, L.J.: Principal component analysis. Wiley Interdiscip. Rev. Comput. Stat. 2, 433–459 (2010)CrossRefGoogle Scholar
  2. 2.
    Jolliffe, I.T.: Principal Component Analysis, vol. 87, pp. 41–64. Springer, Berlin (2010). Scholar
  3. 3.
    Zhou, Z., Jin, Z.: Double nuclear norm-based robust principal component analysis for image disocclusion and object detection. Neurocomputing 205, 481–489 (2016)CrossRefGoogle Scholar
  4. 4.
    Lai, Z., Xu, Y., Chen, Q., Yang, J., Zhang, D.: Multilinear sparse principal component analysis. IEEE Trans. Neural Netw. Learn. Syst. 25, 1942–1950 (2014)CrossRefGoogle Scholar
  5. 5.
    Zhang, F., Yang, J., Qian, J., Xu, Y.: Nuclear norm-based 2-DPCA for extracting features from images. IEEE Trans. Neural Netw. Learn. Syst. 26, 2247–2260 (2015)MathSciNetCrossRefGoogle Scholar
  6. 6.
    Brooks, J.P., Boone, E.L.: A pure L1-norm principal component analysis. Comput. Stat. Data Anal. 61, 83 (2013)CrossRefGoogle Scholar
  7. 7.
    Nie, F., Huang, H., Ding, C.H.Q., Luo, D., Wang, H.: Robust principal component analysis with non-greedy l1-norm maximization. Presented at the IJCAI Proceedings-International Joint Conference on Artificial Intelligence (2011)Google Scholar
  8. 8.
    Kwak, N.: Principal component analysis based on L1-norm maximization. IEEE Trans. Pattern Anal. Mach. Intell. 30, 1672–1680 (2008)CrossRefGoogle Scholar
  9. 9.
    Ding, C., Zhou, D., He, X., Zha, H.: R1-PCA: rotational invariant L1-norm principal component analysis for robust subspace factorization. In: Proceedings of the 23rd International Conference on Machine Learning, pp. 281–288 (2006)Google Scholar
  10. 10.
    Shi, X., Nie, F., Lai, Z., Guo, Z.: Robust principal component analysis via optimal mean by joint ℓ2,1 and Schatten p-norms minimization. Neurocomputing 283, 205–213 (2018)CrossRefGoogle Scholar
  11. 11.
    Nie, F., Yuan, J., Huang, H.: Optimal mean robust principal component analysis. In: Proceedings of the 31st International Conference on Machine Learning, pp. 1062–1070 (2014)Google Scholar
  12. 12.
    Yi, S., Lai, Z., He, Z., Cheung, Y.M., Liu, Y.: Joint sparse principal component analysis. Pattern Recognit. 61, 524–536 (2016)CrossRefGoogle Scholar
  13. 13.
    Nie, F., Huang, H., Cai, X., Ding, C.: Efficient and robust feature selection via joint L2,1-norms minimization. In: Advances in Neural Information Processing Systems, pp. 1813–1821 (2010)Google Scholar
  14. 14.
    Cai, X., Ding, C., Nie, F., Huang, H.: On the equivalent of low-rank linear regressions and linear discriminant analysis based regressions. In: Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1124–1132. ACM (2013)Google Scholar
  15. 15.
    Zou, H., Hastie, T., Tibshirani, R., Url, S.: Sparse principal component analysis. J. Comput. Graph. Stat. 15, 265–286 (2006)MathSciNetCrossRefGoogle Scholar
  16. 16.
    Yang, J., Yang, J.Y.: Why can LDA be performed in PCA transformed space? Pattern Recognit. 36, 563–566 (2003)CrossRefGoogle Scholar
  17. 17.
    Zheng, W.S., Lai, J.H., Li, S.Z.: 1D-LDA vs. 2D-LDA: when is vector-based linear discriminant analysis better than matrix-based? Pattern Recognit. 41, 2156–2172 (2008)CrossRefGoogle Scholar
  18. 18.
    Fang, X., Xu, Y., Li, X., Lai, Z., Wong, W.K., Fang, B.: Regularized label relaxation linear regression. IEEE Trans. Neural Netw. Learn. Syst. PP, 1–13 (2017)Google Scholar
  19. 19.
    Martinez, A.M.: The AR face database. CVC Technical Report 24 (1998)Google Scholar
  20. 20.
    Jonathon Phillips, P., Moon, H., Rizvi, S.A., Rauss, P.J.: The FERET evaluation methodology for face-recognition algorithms. IEEE Trans. Pattern Anal. Mach. Intell. 22, 1090–1104 (2000)CrossRefGoogle Scholar
  21. 21.
    Sim, T., Baker, S., Bsat, M.: The CMU pose, illumination, and expression database. IEEE Trans. Pattern Anal. Mach. Intell. 25, 1615–1618 (2003)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Xiangxi Xu
    • 1
  • Zhihui Lai
    • 1
    Email author
  • Yudong Chen
    • 1
  • Heng Kong
    • 2
  1. 1.The College of Computer Science and Software EngineeringShenzhen UniversityShenzhenChina
  2. 2.School of MedicineShenzhen UniversityShenzhenChina

Personalised recommendations