Abstract
In high dimensional setting, componentwise L 2boosting method has been used to construct sparse model of high prediction, but it tends to select many ineffective variables. Several sparse boosting methods, such as, SparseL 2Boosting and Twin Boosting, have been proposed to improve the variable selection of L 2boosting algorithm. In this paper, we propose a new general sparse boosting method (GSBoosting). The relations are established between GSBoosting and other well known regularized variable selection methods in orthogonal linear model, such as adaptive Lasso, hard thresholds etc. Simulations results show that GSBoosting has good performance in both prediction and variable selection.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Breiman, L.: Arcing classifiers (with discussion). Ann. Statist. 26, 801–849 (1998)
Breiman, L.: Prediction games and arcing algorithms. Neural Computation 11, 1493–1517 (1999)
Breiman, L., Friedman, J.: Estimating Optimal Transformations for Multiple Regression and Correlation. J. Am. Statist. Ass. 80, 580–598 (1985)
Bühlmann, P.: Boosting for high-dimensional linear models. Ann. Statist. 34, 559–583 (2006)
Bühlmann, P., Hothorn, T.: Twin Boosting: improved feature selection and prediction. Statistics and Computing 20(2), 119–138 (2010)
Bühlmann, P., Hothorn, T.: Boosting algorithms: regularization, prediction and model fitting. Statistical Science 4(22), 477–505 (2007)
Bühlmann, P., Yu, B.: Boosting with the L 2 loss: Regression and classification. J. Amer. Statist.Assoc. 98, 324–339 (2003)
Bühlmann, P., Yu, B.: Sparse boosting. J. Machine Learning Research 7, 1001–1024 (2006)
Friedman, J., Hastie, T., Tibshirani, R.: Additive logistic regression: a statistical view of boosting (with discussion). Ann. Statist. 28, 337–407 (2000)
Friedman, J.: Greedy function approximation: a gradient boosting machine. Ann. Statist. 29, 1189–1232 (2001)
Frank, I.E., Friedman, J.H.: A statistical view of some chemometrics regression tools. Technometrics 35, 109–148 (1993)
Hurvich, C.M., Tsai, C.L.: Regression and Time Series Model Selection in Small Samples. Biometrika 76, 297–307 (1989)
Tibshirani, R.: Regression shrinkage and selction via the lasso. J. R. Statist. Soc. B 58, 267–288 (1996)
Zou, H.: The adaptive lasso and its oracle properties. J. Am. Statist. Ass. 101, 1418–1429 (2006)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Zhao, J. (2012). Sparse Boosting with Correlation Based Penalty. In: Zhou, S., Zhang, S., Karypis, G. (eds) Advanced Data Mining and Applications. ADMA 2012. Lecture Notes in Computer Science(), vol 7713. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-35527-1_14
Download citation
DOI: https://doi.org/10.1007/978-3-642-35527-1_14
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-35526-4
Online ISBN: 978-3-642-35527-1
eBook Packages: Computer ScienceComputer Science (R0)