Abstract
Smoothness restrictions may be used in linear models, for instance, when the data set does not contain enough information to allow sufficiently accurate estimation of parameters. In this connection the problem of weighting the sample and non-sample information by a smoothing parameter arises. Suppose we want to choose the smoothing parameter in such a way that the mean squared prediction error (MSPE) of the model will be minimized. Starting from this assumption, an equation for the minimizer is derived, and its asymptotic solution is shown to be finite and unique. Next, it is proposed that common model selection criteria such as the AIC be generalized to be used for the estimation of the smoothing parameter. Asymptotically, several generalized criteria, called smoothing criteria in the paper, yield exactly the unique value which minimizes the MSPE. In fact, these are generalizations of the model selection criteria which are optimal according to the definition of Shibata (1981). Application of the above theoretical results to ridge regression is discussed. The effect of autocorrelated errors on the estimation of the smoothing parameter is also considered, and cases where autocorrelation may have little influence on the coefficient estimates of the model are indicated.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Akaike, H. (1969), “Fitting autoregressive models for prediction”. Annals of the Institute of Statistical Mathematics 21, 243–247.
Akaike, H. (1974), “A new look at the statistical model identification”. IEEE Transactions on Automatic Control AC-19, 716–723.
Amemiya T. (1980), “Selection of regressors”. International Economic Review 21, 331–354.
Craven, P., and G. Wahba (1979), “Smoothing noisy data with spline functions. Estimating the correct degree of smoothing by the method of generalized crossvalidation”. Numerische Mathematik 31, 377–403.
Dempster, A. P., M. Schatzoff, and N. Wermuth (1977), “A simulation study of alternatives to ordinary least squares”. Journal of the American Statistical Association 72, 77–106 (with Discussion).
Draper, N. R., and R. C. Van Nostrand (1979), “Ridge regression and James-Stein estimation: review and comments”. Technometrics 21, 451–466.
Engle, R. F., C. W. J. Granger, J. Rice, and A. Weiss (1982), “Non-parametric estimates of the relation between weather and electricity demand”. Department of Economics, UCSD, Discussion Paper 83–17.
Fuller, W. A. (1976), Introduction to Statistical Time Series. NNew York: Wiley and Sons.
Gersovitz, M., and J. G. MacKinnon (1978), “Seasonality in regression: an application of smoothness priors”. Journal of the American Statistical Association 73, 264–273.
Geweke, J., and R. Meese (1981), “Estimating regression models of finite but unknown order”. International Economic Review 22, 55–70.
Gibbons, D. G. (1981), “A simulation study of some ridge estimators”. Journal of the American Statistical Association 76, 131–139.
Golub, G. H., M. Heath, and G. Wahba (1979), “Generalized cross-validation as a method for choosing a good ridge parameter”. Technometrics 21, 215–223.
Hannan, E. J., and B. G. Quinn (1979), “The determination of the order of an autoregression”. Journal of the Royal Statistical Society, Series B 41, 190–195.
Hocking, R. R. (1976), “The analysis and selection of variables in linear regression”. Biometrics 32, 1–49.
Hoerl, A. E., and R. W. Kennard (1970), “Ridge regression: biased estimation for nonorthogonal problems”. Technometrics 12, 55–67.
Hoerl, A. E., R. W. Kennard, and K. F. Baldwin (1975), “Ridge regression: some simulations”. Communications in Statistics 4, 105–123.
Judge, G. G., W. E. Griffiths, R. C. Hill, H. Lütkepohl, and T.-C. Lee (1985), The Theory and Practice of Econometrics, 2nd edition. NNew York: Wiley and Sons.
Lawless, J. F., and P. Wang (1976), “A simulation of ridge and other regression estimators”. Communications in Statistics 5, 307–323.
Mallows, C. L. (1973), “Some comments on C P”. Technometrics 15, 661–675.
Oman, S. D. (1982), “Shrinking towards subspaces in multiple linear regression”. Technometiics 24, 307–311.
Rao, C. R. (1973), Linear Statistical Inference and Its Applications, 2nd edition. NNew York: Wiley and Sons.
Rice, J. (1984), “Bandwidth choice for nonparametric regression”. Annals of Statistics 12, 1215–1230.
Sawa, T. (1978), “Information criteria for discriminating among alternative regression models”. Econometrica 46, 1273–1291.
Schwarz, G. (1978), “Estimating the dimension of a model”. Annals of Statistics 6, 461–464.
Shibata, R. (1981), “An optimal selection of regression variables”. Biometrika 68, 45–54.
Shiller, R. (1973), “A distributed lag estimator derived from smoothness priors”. Econometrica 41, 775–788.
Shiller, R. (1984), “Smoothness priors and nonlinear regression”. Journal of the American Statistical Association 79, 609–615.
Swamy, P. A. V. B., and J. S. Mehta (1983), “Ridge regression estimation of the Rotterdam model”. Journal of Econometrics 22, 365–390.
Teräsvirta, T. (1981), “A comparison of mixed and minimax estimators of linear models”. Communications in Statistics A, Theory and Methods 10, 1765–1778.
Teräsvirta, T., and I. Mellin (1986), “Model selection criteria and model selection tests in regression models”. Scandinavian Journal of Statistics (in press).
Theil, H. (1961), Economic Forecasts and Policy, 2nd edition. Amsterdam: North Holland
Theil, H., and A. S. Goldberger (1961), “On pure and mixed statistical estimation in economics”. International Economic Review 2, 65–78.
Thurman, S. S., P. A. V. B. Swamy, and J. S. Mehta (1984), “An examination of distributed lag model coefficients estimated with smoothness priors”. Federal Reserve Board, Washington, D. C., Special Studies Paper No. 185.
Trivedi, P. K. (1984), “Uncertain prior information and distributed lag analysis”. In Econometrics and Quantitative Economics, ed. D. F. Hendry and K. F. Wallis, pp. 173–210. Oxford: Blackwells.
Ullah, A., and B. Raj (1979), “A distributed lag estimator derived from Shiller’s smoothness priors. An extension”. Economics Letters 2, 219–223.
Wahba, G. (1975), “Smoothing noisy data with spline functions”. Numerische Mathematik 24, 309–317.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 1987 D. Reidel Publishing Company
About this chapter
Cite this chapter
Teräsvirta, T. (1987). Smoothness in Regression: Asymptotic Considerations. In: MacNeill, I.B., Umphrey, G.J., Carter, R.A.L., McLeod, A.I., Ullah, A. (eds) Time Series and Econometric Modelling. The University of Western Ontario Series in Philosophy of Science, vol 36. Springer, Dordrecht. https://doi.org/10.1007/978-94-009-4790-0_4
Download citation
DOI: https://doi.org/10.1007/978-94-009-4790-0_4
Publisher Name: Springer, Dordrecht
Print ISBN: 978-94-010-8624-0
Online ISBN: 978-94-009-4790-0
eBook Packages: Springer Book Archive