Regression estimates functional dependencies between features to understand dependencies and to systematically control systems. Linear regression models can be efficiently computed from covariances but are restricted to linear dependencies. Substitution allows us to identify specific nonlinear dependencies by linear regression. Robust regression finds models that are robust against outliers. A popular family of nonlinear regression methods are universal approximators.We present two well-known examples for universal approximators from the field of artificial neural networks: the multilayer perceptron and radial basis function networks. Universal approximators can realize arbitrarily small training errors, but cross-validation is required to find models with low validation errors that generalize well on other data sets. Feature selection allows us to include only relevant features in regression models leading to more accurate models.
KeywordsFeature Selection Hide Layer Radial Basis Function Radial Basis Function Network Multilayer Perceptron
Unable to display preview. Download preview PDF.
- 1.H. Akaike. A new look at the statistical model identification. IEEE Transactions on Automatic Control, AC–19:716–723, 1974.Google Scholar
- 3.R. Hecht-Nielsen. Neurocomputing. Addison-Wesley, 1990.Google Scholar
- 11.D. E. Rumelhart, G. E. Hinton, and R. J. Williams. Learning internal representations by error backpropagation. In D. E. Rumelhart and J. L. McClelland, editors, Parallel Distributed Processing. Explorations in the Microstructure of Cognition, volume 1, pages 318–362. MIT Press, Cambridge, 1986.Google Scholar
- 13.S. Vieira, J. M. Sousa, and T. A. Runkler. Multi-criteria ant feature selection using fuzzy classifiers. In C. A. Coello Coello, S. Dehuri, and S. Ghosh, editors, Swarm Intelligence for Multi-objective Problems in Data Mining, pages 19–36. Springer, 2009.Google Scholar
- 14.P. J. Werbos. The Roots of Backpropagation: From Ordered Derivatives to Neural Networks and Political Forecasting (Adaptive and Learning Systems for Signal Processing, Communications and Control Series). Wiley–Interscience, 1994.Google Scholar