Abstract
We consider the problem of accounting for model uncertainty in LASSO models. Bayesian model averaging (BMA) provides a coherent mechanism for accounting for this model uncertainty. In this paper, we propose a method for implementing Bayesian model averaging for LASSO based on regularization path. First, we construct the initial model set using the regularization path, whose inherent piecewise linearity makes the construction easy and effective. Then, we elaborately select the models for BMA from the initial model set through the Occam’s Window method. Finally, we carry out the BMA on the selected models. Experimental results show that BMA has significant advantage over the model selection method based on Bayesian information criterion (BIC).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Tibshirant, R.: Regression shrinkage and selection via the lasso. J. R. Statist. Soc. B 58(1), 267–288 (1996)
Efron, B., Hastie, T., Johnstone, I., Tibshirani, R.: Least angle regression. Annals of Statistics 32, 407–499 (2004)
Zou, H., Hastie, T., Tibshirani, R.: On the ”degrees of freedom” of the lasso. Annals of Statistics 35(5), 2173–2192 (2007)
Fraley, C., Hesterberg, T.: Least angle regression and LASSO for large datasets. Statistical Analysis and Data Mining 1(4), 251–259 (2009)
Liu, H., Zhang, J.: On the L1-Lq regularized regression. Technical report, Technical Report arXiv: 0802.1517 v1, Carnegie Mellon University (2008)
Madigan, D., Raftery, A.E.: Model selection and accounting for model uncertainty in graphical models using Occam’s window. Journal of the American Statistical Association 89(428), 1535–1546 (1994)
Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Springer (2008)
Friedman, J.H.: Multivariate adaptive regression splines. The Annals of Statistics 19(1), 1–67 (1991)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer Berlin Heidelberg
About this paper
Cite this paper
Wang, M., Yang, E. (2012). Bayesian Model Averaging for Lasso Using Regularization Path. In: Jin, D., Lin, S. (eds) Advances in Electronic Engineering, Communication and Management Vol.2. Lecture Notes in Electrical Engineering, vol 140. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-27296-7_43
Download citation
DOI: https://doi.org/10.1007/978-3-642-27296-7_43
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-27295-0
Online ISBN: 978-3-642-27296-7
eBook Packages: EngineeringEngineering (R0)