Abstract
The residual variance estimation problem is well-known in statistics and machine learning with many applications for example in the field of nonlinear modelling. In this paper, we show that the problem can be formulated in a general supervised learning context. Emphasis is on two widely used non-parametric techniques known as the Delta test and the Gamma test. Under some regularity assumptions, a novel proof of convergence of the two estimators is formulated and subsequently verified and compared on two meaningful study cases.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Müller, U., Schik, A., Wefelmeyer, W.: Estimating the error variance in nonparametric regression by a covariate-matched U-statistic. Statistics 37, 179–188 (2003)
Jones, A.J.: New tools in non-linear modelling and prediction. Computational Management Science 1, 109–149 (2004)
Lendasse, A., Ji, Y., Reyhani, N., Verleysen, M.: LS-SVM Hyperparameter Selection with a Nonparametric Noise Estimator. In: Duch, W., Kacprzyk, J., Oja, E., Zadrożny, S. (eds.) ICANN 2005. LNCS, vol. 3697, pp. 625–630. Springer, Heidelberg (2005)
Kemp, S.E.: Gamma test analysis tools for non-linear time series. PhD thesis, University of Glamorgran (2006)
Reyhani, N., Hao, J., Ji, Y., Lendasse, A.: Mutual information and Gamma test for input selection. In: ESANN’2005 proceedings, Bruges, Belgium, 27-29 April 2005, pp. 503–508 (2005)
Lendasse, A., Corona, F., Hao, J., Reyhani, N., Verleysen, M.: Determination of the Mahalanobis matrix using nonparametric noise estimations. In: ESANN’2006 proceedings, Bruges, Belgium, 26-28 April 2006, pp. 227–237 (2006)
Evans, D.: Data-derived estimates of noise for unknown smooth models using near-neighbour asymptotics. PhD thesis, Cardiff University (2002)
Pi, H., Peterson, C.: Finding the embedding dimension and variable dependencies in time series. Neural Comput. 6, 509–520 (1994)
Shiryaev, A.N.: Probability. Springer, Heidelberg (1995)
Liitiäinen, E., Corona, F., Lendasse, A.: Nearest neighbor distributions and noise variance estimation. In: ESANN 2007, European Symbosium on Artifical Neural Networks (accepted for publication) (2007)
Devroye, L., Wagner, T.J.: Distribution-free probability inequalities for the deleted and holdout estimates. IEEE Transactions on Information Theory, 202–207 (1979)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Liitiäinen, E., Lendasse, A., Corona, F. (2007). Non-parametric Residual Variance Estimation in Supervised Learning. In: Sandoval, F., Prieto, A., Cabestany, J., Graña, M. (eds) Computational and Ambient Intelligence. IWANN 2007. Lecture Notes in Computer Science, vol 4507. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-73007-1_9
Download citation
DOI: https://doi.org/10.1007/978-3-540-73007-1_9
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-73006-4
Online ISBN: 978-3-540-73007-1
eBook Packages: Computer ScienceComputer Science (R0)