A Projection Algorithm for Regression with Collinearity
Principal component regression (PCR) is often used in regression with multicollinearity. Although this method avoids the problems which can arise in the least squares (LS) approach, it is not optimized with respect to the ability to predict the response variable. We propose a method which combines the two steps in the PCR procedure, namely finding the principal components (PCs) and regression of the response variable on the PCs. The resulting method aims at maximizing the coefficient of determination for a selected number of predictor variables, and therefore the number of predictor variables can be reduced compared to PCR. An important feature of the proposed method is that it can easily be robustified using robust measures of correlation.
KeywordsPredictor Variable Ridge Regression Stepwise Selection Principal Component Regression Less Square Estimator
Unable to display preview. Download preview PDF.
- CROUX, C. and RUIZ-GAZEN, A. (1996): A Fast Algorithm for Robust Principal Components Based on Projection Pursuit. In: A. Prat (ed.): Computational Statistics. Physica-Verlag, Heidelberg, 211–216.Google Scholar
- FILZMOSER, P. (2001): Robust Principal Compnent Regression. In: S. Aivazian, Yu. Kharin, and H. Rieder (Eds.): Computer Data Analysis and Modeling. Robust and Computer Intensive Methods. Belarusian State University, Minsk, 132–137.Google Scholar
- HADI, A.S. and LING, R.F. (1998): Some Cautionary Notes on the Use of Principal Components Regression. The American Statistician, 1, 15–19.Google Scholar
- REIMANN, C., AYRAS, M., CHEKUSHIN, V., BOGATYREV, I., BOYD, R., DE CARITAT, P., DUTTER, R., FINNE, T.E., HALLERAKER, J.H., JIEGER, O., KASHULINA, G., LEHTO, O., NISKAVAARA, H., PAVLOV, V., RäISäNEN, M.L., STRAND, T., and VOLDEN, T. (1998): Environmental Geochemical Atlas of the Central Barents Region. NGU-GTK-CKE special publication. Geological Survey of Norway, Trondheim.Google Scholar