On principal components regression with Hilbertian predictors

  • Ben JonesEmail author
  • Andreas Artemiou


We demonstrate that, in a regression setting with a Hilbertian predictor, a response variable is more likely to be more highly correlated with the leading principal components of the predictor than with trailing ones. This is despite the extraction procedure being unsupervised. Our results are established under the conditional independence model, which includes linear regression and single-index models as special cases, with some assumptions on the regression vector. These results are a generalisation of earlier work which showed that this phenomenon holds for predictors which are real random vectors. A simulation study is used to quantify the phenomenon.


Conditional independence Hilbertian random variables Principal components regression Elliptical distributions Cauchy distribution 



We would like to thank the Editor, Associate Editor and two reviewers for their constructive comments and suggestions which helped improve an earlier version of the manuscript.


  1. Arnold, B. C., Brockett, P. L. (1992). On distributions whose component ratios are cauchy. American Statistician, 46(1), 25–26.MathSciNetGoogle Scholar
  2. Artemiou, A., Li, B. (2009). On principal components regression: A statistical explanation of a natural phenomenon. Statistica Sinica, 19, 1557–1565.MathSciNetzbMATHGoogle Scholar
  3. Artemiou, A., Li, B. (2013). Predictive power of principal components for single-index model and sufficient dimension reduction. Journal of Multivariate Analysis, 119, 176–184.MathSciNetCrossRefGoogle Scholar
  4. Cook, R. (2007). Fisher lecture: Dimension reduction in regression. Statistical Science, 22(1), 1–26.MathSciNetCrossRefGoogle Scholar
  5. Cox, D. R. (1968). Notes on some aspects of regression analysis. Journal of the Royal Statistical Society Series A (General), 131(3), 265–279.CrossRefGoogle Scholar
  6. Dauxois, J., Ferré, L., Yao, A.-F. (2001). Un modèle semi-paramétrique pour variables aléatoires hilbertiennes. Comptes Rendus de l’Académie des Sciences, 333(1), 947–952.MathSciNetzbMATHGoogle Scholar
  7. Ferré, L., Yao, A. F. (2003). Functional sliced inverse regression analysis. Statistics, 37(6), 475–488.MathSciNetCrossRefGoogle Scholar
  8. Hall, P., Yang, Y. J. (2010). Ordering and selecting components in multivariate or functional data linear prediction. Journal of the Royal Statistical Society Series B: Statistical Methodology, 72(1), 93–110.Google Scholar
  9. Hsing, T., Eubank, R. (2015). Theoretical foundations of functional data analysis, with an introduction to linear operators. 1st ed. West Sussex: Wiley.Google Scholar
  10. Kingman, J. F. C. (1972). On random sequences with spherical symmetry. Biometrika, 59(2), 492.MathSciNetCrossRefGoogle Scholar
  11. Li, B. (2007). Comment: Fisher lecture—Dimension reduction in regression. Statistical Science, 22(1), 32–35.MathSciNetCrossRefGoogle Scholar
  12. Li, B. (2018). Sufficient dimension reduction: Methods and applications with R. 1st ed. Boca Raton: CRC Press.Google Scholar
  13. Li, B., Song, J. (2017). Nonlinear sufficient dimension reduction for functional data. The Annals of Statistics, 45(3), 1059–1095.MathSciNetCrossRefGoogle Scholar
  14. Li, Y. (2007). A note on hilbertian elliptically contoured distributions. Unpublished manuscript, Department of Statistics, University of Georgia.Google Scholar
  15. Ni, L. (2011). Principal component regression revisited. Statistica Sinica, 21, 741–747.MathSciNetCrossRefGoogle Scholar
  16. Pinelis, I., Molzon, R. (2016). Optimal-order bounds on the rate of convergence to normality in the multivariate delta method. Electronic Journal of Statistics, 10(1), 1001–1063.MathSciNetCrossRefGoogle Scholar
  17. Ramsay, J., Silverman, B. W. (1997). Functional data analysis. 1st ed. New York: Springer.Google Scholar

Copyright information

© The Institute of Statistical Mathematics, Tokyo 2018

Authors and Affiliations

  1. 1.School of MathematicsCardiff UniversityCardiffUK

Personalised recommendations