A Projection Algorithm for Regression with Collinearity

  • Peter Filzmoser
  • Christophe Croux
Part of the Studies in Classification, Data Analysis, and Knowledge Organization book series (STUDIES CLASS)


Principal component regression (PCR) is often used in regression with multicollinearity. Although this method avoids the problems which can arise in the least squares (LS) approach, it is not optimized with respect to the ability to predict the response variable. We propose a method which combines the two steps in the PCR procedure, namely finding the principal components (PCs) and regression of the response variable on the PCs. The resulting method aims at maximizing the coefficient of determination for a selected number of predictor variables, and therefore the number of predictor variables can be reduced compared to PCR. An important feature of the proposed method is that it can easily be robustified using robust measures of correlation.


Predictor Variable Ridge Regression Stepwise Selection Principal Component Regression Less Square Estimator 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. ARAúJO, M.C.U., SALDANHA, T.C.B., GALVãO, R.K.H., YONEYAMA, T., and CHAME, H.C. (2001): The Successive Projections Algorithm for Variable Selection in Spectroscopic Multicomponent Analysis. Chemometrics and Intelligent Laboratory Systems, 57, 65–73.CrossRefGoogle Scholar
  2. BASILEVSKY, A. (1994): Statistical Factor Analysis and Related Methods: Theory and Applications. Wiley & Sons, New York.zbMATHCrossRefGoogle Scholar
  3. CROUX, C. and DEHON, C. (2001): Estimators of the Multiple Correlation Coefficient: Local Robustness and Confidence Intervals, to appear in Statistical Papers, Scholar
  4. CROUX, C. and RUIZ-GAZEN, A. (1996): A Fast Algorithm for Robust Principal Components Based on Projection Pursuit. In: A. Prat (ed.): Computational Statistics. Physica-Verlag, Heidelberg, 211–216.Google Scholar
  5. FILZMOSER, P. (2001): Robust Principal Compnent Regression. In: S. Aivazian, Yu. Kharin, and H. Rieder (Eds.): Computer Data Analysis and Modeling. Robust and Computer Intensive Methods. Belarusian State University, Minsk, 132–137.Google Scholar
  6. HADI, A.S. and LING, R.F. (1998): Some Cautionary Notes on the Use of Principal Components Regression. The American Statistician, 1, 15–19.Google Scholar
  7. HOERL, A.E. and KENNARD, R.W. (1970): Ridge Regression: Biased Estimation for Nonorthogonal Problems. Technometrics, 12, 55–67.zbMATHCrossRefGoogle Scholar
  8. REIMANN, C., AYRAS, M., CHEKUSHIN, V., BOGATYREV, I., BOYD, R., DE CARITAT, P., DUTTER, R., FINNE, T.E., HALLERAKER, J.H., JIEGER, O., KASHULINA, G., LEHTO, O., NISKAVAARA, H., PAVLOV, V., RäISäNEN, M.L., STRAND, T., and VOLDEN, T. (1998): Environmental Geochemical Atlas of the Central Barents Region. NGU-GTK-CKE special publication. Geological Survey of Norway, Trondheim.Google Scholar
  9. ROUSSEEUW, P.J. (1984): Least Median of Squares Regression. Journal of the American Statistical Association, 79, 871–880.MathSciNetzbMATHCrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2002

Authors and Affiliations

  • Peter Filzmoser
    • 1
  • Christophe Croux
    • 2
  1. 1.Department of Statistics and Probability TheoryVienna University of TechnologyViennaAustria
  2. 2.Department of Applied EconomicsK.U.LeuvenLeuvenBelgium

Personalised recommendations