Abstract
There is already a great number of highly efficient methods producing components with sparse loadings which significantly facilitates the interpretation of principal component analysis (PCA). However, they produce either only orthonormal loadings, or only uncorrelated components, or, most frequently, neither of them. To overcome this weakness, we introduce a new approach to define sparse PCA similar to the Dantzig selector idea already employed for regression problems. In contrast to the existing methods, the new approach makes it possible to achieve simultaneously nearly uncorrelated sparse components with nearly orthonormal loadings. The performance of the new method is illustrated on real data sets. It is demonstrated that the new method outperforms one of the most popular available methods for sparse PCA in terms of preservation of principal components properties.
M. Genicot—Supported by a FRIA grant from F.R.S.-FNRS, Belgium.
W. Huang—Supported by grant FNRS PDR T.0173.13.
N. T. Trendafilov—Supported by a grant RPG-2013-211 from The Leverhulme Trust, UK.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Absil, P.-A., Mahony, R., Sepulchre, R.: Optimization Algorithms on Matrix Manifolds. Princeton University Press, Princeton (2008)
Boumal, N., Mishra, B., Absil, P.-A., Sepulchre, R.: Manopt, a Matlab toolbox for optimization on manifolds. J. Mach. Learn. Res. 15, 1455–1459 (2014)
Candès, E.J., Tao, T.: The Dantzig selector: statistical estimation when \(p\) is much larger than \(n\). Ann. Stat. 35, 2313–2351 (2007)
Jolliffe, I.T.: Principal Component Analysis, 2nd edn. Springer, New York (2002)
Jolliffe, I.T., Trendafilov, N.T., Uddin, M.: A modified principal component technique based on the LASSO. J. Comput. Graph. Stat. 12, 531–547 (2003)
Journée, M., Nesterov, Y., Richtárik, P., Sepulchre, R.: Generalized power method for sparse principal component analysis. J. Mach. Learn. Res. 11, 517–553 (2010)
Lu, Z., Zhang, Y.: An augmented Lagrangian approach for sparse principal component analysis. Math. Program. Ser. A 135(1–2), 149–193 (2012)
Qi, X., Luo, R., Zhao, H.: Sparse principal component analysis by choice of norm. J. Multivar. Anal. 114, 127–160 (2013)
Tibshirani, R.: Regression shrinkage and selection via the LASSO. J. R. Stat. Soc. 58, 267–288 (1996)
Trendafilov, N.T.: The dynamical system approach to multivariate data analysis, a review. J. Comput. Graph. Stat. 50, 628–650 (2006)
Trendafilov, N.T.: From simple structure to sparse components: a review. Comput. Stat. 29, 431–454 (2014)
Wright, S.: Gradient algorithms for regularized optimization. In: SPARS11, Edinburgh, Scotland (2011). http://pages.cs.wisc.edu/-swright/
Zhuang, J., Jones, A., Lee, S.-H., Ng, E., Fiegl, H., et al.: The dynamics and prognostic potential of DNA methylation changes at stem cell gene loci in women’s cancer. PLoS Genet. 8(2), e1002517 (2012)
Zou, H., Hastie, T., Tibshirani, R.: Sparse principal component analysis. J. Comput. Graph. Stat. 15, 265–286 (2006)
Acknowledgements
This paper presents research results of the Belgian Network DYSCO (Dynamical Systems, Control, and Optimization), funded by the Interuniversity Attraction Poles Programme initiated by the Belgian Science Policy Office. The authors thank Professor Pierre-Antoine Absil for his advices and the reviewers for their pertinent comments.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Genicot, M., Huang, W., Trendafilov, N.T. (2015). Weakly Correlated Sparse Components with Nearly Orthonormal Loadings. In: Nielsen, F., Barbaresco, F. (eds) Geometric Science of Information. GSI 2015. Lecture Notes in Computer Science(), vol 9389. Springer, Cham. https://doi.org/10.1007/978-3-319-25040-3_52
Download citation
DOI: https://doi.org/10.1007/978-3-319-25040-3_52
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-25039-7
Online ISBN: 978-3-319-25040-3
eBook Packages: Computer ScienceComputer Science (R0)