Advertisement

On Closeness Between Factor Analysis and Principal Component Analysis Under High-Dimensional Conditions

  • L. Liang
  • K. HayashiEmail author
  • Ke-Hai Yuan
Part of the Springer Proceedings in Mathematics & Statistics book series (PROMS, volume 140)

Abstract

This article studies the relationship between loadings from factor analysis (FA) and principal component analysis (PCA) when the number of variables p is large. Using the average squared canonical correlation between two matrices as a measure of closeness, results indicate that the average squared canonical correlation between the sample loading matrix from FA and that from PCA approaches 1 as p increases, while the ratio of p/N does not need to approach zero. Thus, the two methods still yield similar results with high-dimensional data. The Fisher-z transformed average canonical correlation between the two loading matrices and the logarithm of p is almost perfectly linearly related.

Keywords

Canonical correlation Factor indeterminacy Fisher-z transformation Guttman condition Large p small N Ridge factor analysis 

Notes

Acknowledgments

Ke-Hai Yuan's work was supported by the National Science Foundation under Grant No. SES-1461355. The authors are grateful to comments from Drs. Sy-Miin Chow and Shin-ichi Mayekawa that led to significant improvements of the article.

References

  1. Anderson, T. W. (2003). An introduction to multivariate statistical analysis (3rd ed.). New York: Wiley.zbMATHGoogle Scholar
  2. Bai, J., & Li, K. (2012). Statistical analysis of factor models of high dimension. The Annals of Statistics, 40, 436–465.MathSciNetCrossRefzbMATHGoogle Scholar
  3. Bentler, P. M., & Kano, Y. (1990). On the equivalence of factors and components. Multivariate Behavioral Research, 25, 67–74.CrossRefGoogle Scholar
  4. Buehlmann, P., & van de Geer, S. (2011). Statistics for high-dimensional data: Method, theory, and applications. Heidelberg: Springer.CrossRefGoogle Scholar
  5. Butcher, J. N., Dahlstrom, W. G., Graham, J. R., Tellegen, A., & Kaemmer, B. (1989). The Minnesota Multiphasic Personality Inventory-2 (MMPI-2): Manual for administration and scoring. Minneapolis, MN: University of Minnesota Press.Google Scholar
  6. Guttman, L. (1956). “Best possible” estimates of communalities. Psychometrika, 21, 273–286.MathSciNetCrossRefzbMATHGoogle Scholar
  7. Hastie, T., Tibshirani, R., & Friedman, J. (2009). The elements of statistical learning (2nd ed.). New York: Springer.CrossRefzbMATHGoogle Scholar
  8. Hayashi, K., & Bentler, P. M. (2000). On the relations among regular, equal unique variances, and image factor analysis models. Psychometrika, 65, 59–72.MathSciNetCrossRefzbMATHGoogle Scholar
  9. Krijnen, W. P. (2006). Convergence of estimates of unique variances in factor analysis, based on the inverse sample covariance matrix. Psychometrika, 71, 193–199.MathSciNetCrossRefGoogle Scholar
  10. Lawley, D. N., & Maxwell, A. E. (1971). Factor analysis as a statistical method (2nd ed.). New York: American Elsevier.zbMATHGoogle Scholar
  11. Pourahmadi, M. (2013). High-dimensional covariance estimation. New York: Wiley.CrossRefzbMATHGoogle Scholar
  12. Schneeweiss, H. (1997). Factors and principal components in the near spherical case. Multivariate Behavioral Research, 32, 375–401.CrossRefGoogle Scholar
  13. Schneeweiss, H., & Mathes, H. (1995). Factor analysis and principal components. Journal of Multivariate Analysis, 55, 105–124.MathSciNetCrossRefzbMATHGoogle Scholar
  14. Tibshirani, R. (1996). Regression shrinkage and selection via the Lasso. Journal of the Royal Statistical Society, Series B, 58, 267–288.MathSciNetzbMATHGoogle Scholar
  15. Velicer, W. F., & Jackson, D. N. (1990). Component analysis versus common factor analysis: Some issues in selecting an appropriate procedure. Multivariate Behavioral Research, 25, 1–28.CrossRefGoogle Scholar
  16. Yuan, K.-H. (2013, July). Ridge structural equation modeling with large p and/or small N. Paper presented at the 78th Annual Meeting of the Psychometric Society (IMPS2013), Arnhem, The Netherlands.Google Scholar
  17. Yuan, K.-H., & Chan, W. (2008). Structural equation modeling with near singular covariance matrices. Computational Statistics and Data Analysis, 52, 4842–4858.MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  1. 1.Department of PsychologyUniversity of Hawaii at ManoaHonoluluUSA
  2. 2.Department of PsychologyUniversity of Notre DameNotre DameUSA

Personalised recommendations