Abstract
Sparse regression is the problem of selecting a parsimonious subset of all available regressors for an efficient prediction of a target variable. We consider a general setting in which both the target and regressors may be multivariate. The regressors are selected by a forward selection procedure that extends the Least Angle Regression algorithm. Instead of the common practice of estimating each target variable individually, our proposed method chooses sequentially those regressors that allow, on average, the best predictions of all the target variables. We illustrate the procedure by an experiment with artificial data. The method is also applied to the task of selecting relevant pixels from images in multidimensional scaling of handwritten digits.
An erratum to this chapter can be found at http://dx.doi.org/10.1007/11550907_163 .
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Burnham, A.J., MacGregor, J.F., Viveros, R.: Latent Variable Multivariate Regression Modeling. Chemometrics and Intelligent Laboratory Systems 48, 167–180 (1999)
Hoerl, A.E., Kennard, R.W.: Ridge Regression: Biased Estimation for Nonorthogonal Problems. Technometrics 12, 55–67 (1970)
Breiman, L., Friedman, J.H.: Predicting Multivariate Responses in Multivariate Regression. Journal of the Royal Statistical Society. Series B 59, 3–54 (1997)
Abraham, B., Merola, G.: Dimensionality Reduction Approach to Multivariate Prediction. Computational Statistics & Data Analysis 48, 5–16 (2005)
Copas, J.: Regression, Prediction and Shrinkage. Journal of the Royal Statistical Society. Series B 45, 311–354 (1983)
Tibshirani, R.: Regression Shrinkage and Selection via the Lasso. Journal of the Royal Statistical Society. Series B 58, 267–288 (1996)
Efron, B., Hastie, T., Johnstone, I., Tibshirani, R.: Least Angle Regression. Annals of Statistics 32, 407–499 (2004)
Cox, T.F., Cox, M.A.A.: Multidimensional Scaling. In: Monographs on Statistics and Applied Probability, vol. 88. Chapman & Hall, Boca Raton (2001)
Sammon, J.W.: A Nonlinear Mapping for Data Structure Analysis. IEEE Transactions on Computers C-18, 401–409 (1969)
Tipping, M.E., Lowe, D.: Shadow Targets: A Novel Algorithm for Topographic Projections by Radial Basis Functions. Neurocomputing 19, 211–222 (1998)
Zhang, Z.: Learning Metrics via Discriminant Kernels and Multidimensional Scaling: Toward Expected Euclidean Representation. In: International Conference on Machine Learning, pp. 872–879 (2003)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Similä, T., Tikka, J. (2005). Multiresponse Sparse Regression with Application to Multidimensional Scaling. In: Duch, W., Kacprzyk, J., Oja, E., Zadrożny, S. (eds) Artificial Neural Networks: Formal Models and Their Applications – ICANN 2005. ICANN 2005. Lecture Notes in Computer Science, vol 3697. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11550907_16
Download citation
DOI: https://doi.org/10.1007/11550907_16
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-28755-1
Online ISBN: 978-3-540-28756-8
eBook Packages: Computer ScienceComputer Science (R0)