On Closeness Between Factor Analysis and Principal Component Analysis Under High-Dimensional Conditions
This article studies the relationship between loadings from factor analysis (FA) and principal component analysis (PCA) when the number of variables p is large. Using the average squared canonical correlation between two matrices as a measure of closeness, results indicate that the average squared canonical correlation between the sample loading matrix from FA and that from PCA approaches 1 as p increases, while the ratio of p/N does not need to approach zero. Thus, the two methods still yield similar results with high-dimensional data. The Fisher-z transformed average canonical correlation between the two loading matrices and the logarithm of p is almost perfectly linearly related.
KeywordsCanonical correlation Factor indeterminacy Fisher-z transformation Guttman condition Large p small N Ridge factor analysis
Ke-Hai Yuan's work was supported by the National Science Foundation under Grant No. SES-1461355. The authors are grateful to comments from Drs. Sy-Miin Chow and Shin-ichi Mayekawa that led to significant improvements of the article.
- Butcher, J. N., Dahlstrom, W. G., Graham, J. R., Tellegen, A., & Kaemmer, B. (1989). The Minnesota Multiphasic Personality Inventory-2 (MMPI-2): Manual for administration and scoring. Minneapolis, MN: University of Minnesota Press.Google Scholar
- Yuan, K.-H. (2013, July). Ridge structural equation modeling with large p and/or small N. Paper presented at the 78th Annual Meeting of the Psychometric Society (IMPS2013), Arnhem, The Netherlands.Google Scholar