Boosting Correlation Based Penalization in Generalized Linear Models
Linear models have a long tradition in statistics as nicely summarized in Rao, Toutenburg, Shalabh, Heumann (2008). When the number of covariates is large the estimation of unknown parameters frequently raises problems. Then the interest usually focusses on data driven subset selection of relevant regressors. The sophisticated monitoring equipment which is now routinely used in many data collection processes makes it possible to collect data with a huge amount of regressors, even with considerably more explanatory variables than observations. One example is the analysis of microarray data of gene expressions. Here the typical tasks are to select variables and to classify samples into two or more alternative categories. Binary responses of this type may be handled within the framework of generalized linear models (Neider and Wedderburn (1972)) and are also considered in Rao, Toutenburg, Shalabh, Heumann (2008).
In this paper we propose a new regularization method and a boosted version of it, which explicitly focus on the selection of groups. To reach this target we consider a correlation based penalty which uses correlation between variables as data driven weights for penalization. See also Tutz and Ulbricht (2006) for a similar approach to linear models. This new method and some of its main properties are described in Section 2. A boosted version of it that will be presented in Section 3 allows for variable selection. In Section 4 we use simulated and real data sets to compare our new methods with existing ones.
KeywordsGeneralize Linear Model Maximum Likelihood Estimator Subset Selection Unknown Parameter Vector Fisher Matrix
Unable to display preview. Download preview PDF.
- Meir R, Rätsch G (2003) An introduction to boosting and leveraging. In: Mendelson S, Smola A (eds) Advanced Lectures on Machine Learning, Springer, New York, pp 119-184Google Scholar
- Park MY, Hastie T (2007) An l1 regularization-path algorithm for generalized linear models. JRSSGoogle Scholar
- Rao CR, Toutenburg H, Shalabh, Heumann C (2008) Linear Models - Least Squares and Generalizations (3rd edition). Springer, Berlin Heidelberg New YorkGoogle Scholar
- Tutz G, Binder H (2007) Boosting ridge regression. Computational Statistics & Data Analysis (Appearing)Google Scholar
- Tutz G, Ulbricht J (2006) Penalized regression with correlation based penalty. Discussion Paper 486, SFB 386, Universität MünchenGoogle Scholar