Abstract
High-dimensional regression/classification is challenging due to the curse of dimensionality. Lasso [18] and its various extensions [10], which can simultaneously perform feature selection and regression/classification, have received increasing attention in this situation. However, in the presence of highly correlated features lasso tends to only select one of those features resulting in suboptimal performance [25]. Several methods have been proposed to address this issue in the literature. Shen and Ye [15] introduce an adaptive model selection procedure that corrects the estimation bias through a data-driven penalty based on generalized degrees of freedom.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bach F, Lanckriet G, Jordan M (2004) Multiple kernel learning, conic duality, and the SMO algorithm. In: ICML ACM New York, NY, USA. DOI 10.1145/1015330.1015424
Bondell H, Reich B (2008) Simultaneous regression shrinkage, variable selection, and supervised clustering of predictors with oscar. Biometrics 64(1):115–123
Boyd S, Parikh N, Chu E, Peleato B, Eckstein J (2011) Distributed optimization and statistical learning via the alternating direction method of multipliers Foundations and Trends ®; in Machine Learning, Now Publishers Inc 3(1):1–122
Chuang H, Lee E, Liu Y, Lee D, Ideker T (2007) Network-based classification of breast cancer metastasis. Mol Syst Biol 3(1)
Fei H, Quanz B, Huan J (2010) Regularization and feature selection for networked features. In: CIKM, ACM New York, NY, USA pp 1893–1896. DOI 10.1145/1871437.1871756
Jacob L, Obozinski G, Vert J (2009) Group lasso with overlap and graph lasso. In: ICML, ACM New York, NY, USA pp 433–440. DOI 10.1145/1553374.1553431
Jenatton R, Mairal J, Obozinski G, Bach F (2010) Proximal methods for sparse hierarchical dictionary learning. In: ICML ACM New York, NY, USA
Kim S, Xing E (2009) Statistical estimation of correlated genome associations to a quantitative trait network. PLoS Genet 5(8):e1000587
Li C, Li H (2008) Network-constrained regularization and variable selection for analysis of genomic data. Bioinformatics 24(9):1175–1182
Liu J, Ji S, Ye J (2009) SLEP: Sparse learning with efficient projections. Arizona State University, http://www.public.asu.edu/~jye02/Software/SLEP/
Liu J, Ye J (2010) Moreau-Yosida regularization for grouped tree structure learning. In: NIPS
Rinaldo A (2009) Properties and refinements of the fused lasso. Ann Stat 37(5B):2922–2952
Shen X, Huang H (2009) Grouping pursuit through a regularization solution surface. J Am Stat Assoc 105(490):727–739
Shen X, Huang H, Pan W (2012) Simultaneous supervised clustering and feature selection over a graph. Biometrika, to appear
Shen X, Ye J (2002) Adaptive model selection. J Am Stat Assoc 97(457):210–221
Tao P, An L (1997) Convex analysis approach to DC programming: Theory, algorithms and applications. Acta Math Vietnam 22(1):289–355
Tao P, El Bernoussi S (1988) Duality in DC (difference of convex functions) optimization. Subgradient methods. Trends Math Optimiz 84:277–293
Tibshirani R (1996) Regression shrinkage and selection via the lasso. J Roy Stat Soc Ser B, 58(1): 267–288
Tibshirani R, Saunders M, Rosset S, Zhu J, Knight K (2005) Sparsity and smoothness via the fused lasso. J Roy Stat Soc Ser B 67(1):91–108
Yuan L, Liu J, Ye J (2011) Efficient methods for overlapping group lasso. In: NIPS
Yuan M, Lin Y (2006) Model selection and estimation in regression with grouped variables. J Roy Stat Soc Ser B 68(1):49–67
Zhao P, Rocha G, Yu B (2009) The composite absolute penalties family for grouped and hierarchical variable selection. Ann Stat 37(6A):3468–3497
Zhong L, Kwok J (2011) Efficient sparse modeling with automatic feature grouping. In: ICML
Zhu Y, Shen X, Pan W (2012) Simultaneous grouping pursuit and feature selection in regression over an undirected graph. Manuscript
Zou H, Hastie T (2005) Regularization and variable selection via the elastic net. J Roy Stat Soc Ser B 67(2):301–320
Acknowledgements
This work was supported in part by NSF (IIS-0953662, MCB-1026710, CCF-1025177, DMS-0906616) and NIH (R01LM010730, 2R01GM081535-01, R01HL105397).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer Science+Business Media New York
About this chapter
Cite this chapter
Yang, S., Yuan, L., Lai, YC., Shen, X., Wonka, P., Ye, J. (2013). Feature Grouping and Selection Over an Undirected Graph. In: Fu, Y., Ma, Y. (eds) Graph Embedding for Pattern Analysis. Springer, New York, NY. https://doi.org/10.1007/978-1-4614-4457-2_2
Download citation
DOI: https://doi.org/10.1007/978-1-4614-4457-2_2
Published:
Publisher Name: Springer, New York, NY
Print ISBN: 978-1-4614-4456-5
Online ISBN: 978-1-4614-4457-2
eBook Packages: EngineeringEngineering (R0)