Summary
Local regression is an old method for smoothing data, having origins in the graduation of mortality data and the smoothing of time series in the late 19th century and the early 20th century. Still, new work in local regression continues at a rapid pace. We review the history of local regression. We discuss four of its basic components that must be chosen in using local regression in practice — the weight function, the parametric family that is fitted locally, the bandwidth, and the assumptions about the distribution of the response. A major theme of the paper is that these choices represent a modeling of the data; different data sets deserve different choices. We describe polynomial mixing, a method for enlarging polynomial parametric families. We introduce an approach to adaptive fitting,assessment of parametric localization. We describe the use of this approach to design two adaptive procedures: one automatically chooses the mixing degree of mixing polynomials at each x using cross-validation, and the other chooses the bandwidth at each x using C p . Finally, we comment on the efficacy of using asymptotics to provide guidance for methods of local regression.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Brillinger, D. (1977). Discussion of a paper of Stone. Ann. Statist5, 622–623.
Cleveland, W. S. (1979). Robust locally weighted regression and smoothing scatterplots. J. Amer. Statist. Assn. 74, 829–836.
Cleveland, W. S. (1993). Visualizing Data. Hobart Press, Summit, NJ, http://bookshobart.com
Cleveland, W. S. and Devlin, S. J. (1988). Locally weighted regression: an approach to regression analysis by local fitting. J. Amer. Statist. Assn. 83, 596–610.
Cleveland, W. S., Hastie, T., and Loader, C. (1995). Adaptive local regression by automatic selection of the polynomial mixing degree. In preparation.
Cleveland, W. S. and Grosse, E. H. (1991). Computational methods for local regression.Statist, and Computing1, 47–62.
Cleveland, W. S. and Grosse, E. H. and Shyu, M. J. (1992). Local regression models. Statistical Models in S, J. M. Chambers and T. Hastie, editors, pages 309–376. Chapman and Hall, New York.
Cleveland, W. S. and Loader, C. (1995). Computational methods for local regression from the 19th century to the present. In preparation.
Cox, D. R. (1958). Some problems connected with statistical inference. Ann. Math. Statist. 29, 357–372.
Daniel, C. and Wood, F. (1971). Fitting Equations to Data. Wiley, New York.
DeForest, E. L. (1873). On some methods of interpolation applicable to the graduation of irregular series. Annual Report of the Board of Regents of the Smithsonian Institution for 1871, 275–339.
DeForest, E. L. (1874). Additions to a memoir on methods of interpolation applicable to the graduation of irregular series. Annual Report of the Board of Regents of the Smithsonian Institution for 1873, 319–353.
Donoho, D. and Johnstone, I. (1994). Ideal spatial adaptation by wavelet shrinkage. Biometrika81, 425–455.
Fan, J. (1993). Local linear regression smoothers and their minimax efficiencies. Ann. Statist. 21, 196–216.
Fan, J. and Gijbels, I. (1992). Variable bandwidth and local linear regression smoothers. Ann. Statist. 20, 2008–2036.
Fan, J. and Gijbels, I. (1994a). Censored regression: local linear approximations and their applications. J. Amer. Statist. Assn. 89, 560–570.
Fan, J. and Gijbels, I. (1994b). Adaptive order polynomial fitting: bandwidth robustification and bias reduction. Unpublished manuscript.
Fan, J. and Gijbels, I. (1995). Data-driven bandwidth selection in local polynomial fitting: variable bandwidth and spatial adaptation. J. Royal Statist. Soc, Ser. B. 57, 371–394.
Friedman, J. H. Multivariate adaptive regression splines (1991). Ann. Statist. 19, 1–141.
Friedman, J. H. and Stuetzle, W. (1981). Projection pursuit regression. J. Amer. Statist. Assn. 76, 817–823.
Friedman, J. H. and Stuetzle, W. (1982). Smoothing of scatterplots. Technical Report Orion 3, Dept. Statistics, Stanford University.
Gasser, Th. and Jennen-Steinmetz, C. (1988). A unifying approach to nonparametric regression estimation. J. Amer. Statist. Assn. 83, 1084–1089.
Gram, J. P. (1883). Uber Entwickelung reeller Functionen in Reihen mittelst der Methode der kleinsten Quadrate. J. Math. 94, 41–73.
Hall, P., Sheather, S. J., Jones, M. C. and Marron, J. S. (1991). On optimal data-based bandwidth selection in kernel density estimation. Biometrika78, 263–269.
Hardle, W. (1990). Applied Nonparametric Regression. Oxford University Press, Oxford.
Hastie, T. and Loader, C. (1993). Local regression: automatic kernel carpentry (with discussion). Statist. Science8, 120–143.
Hastie, T. J. and Tibshirani, R. J. (1990). Generalized Additive Models. Chapman and Hall, London.
Henderson, R. (1916). Note on graduation by adjusted average.Actuarial Soc. Amer. 17, 43–48.
Henderson, R. (1924). A new method of graduation. Actuarial Soc. Amer. 25, 29–39.
Hoem, J. M. (1983). The reticent trio: some little-known early discoveries in life insurance mathematics by L. H. F. Oppermann, T. N. Thiele and J. P. Gram. Inter. Stat. Rev. 51, 213–221.
Hjort, N. L. (1994). Local Bayesian regression. Unpublished manuscript.
Hjort, N. L. and Jones, M. C. (1994). Locally parametric nonparametric density estimation. Unpublished manuscript.
Katkovnik, V. Ya. (1979). Linear and nonlinear methods of nonparametric regression analysis. Soviet Automatic Control 5, 25–34.
Kendall, M. G. (1973). Time Series. Oxford University Press, Oxford.
Kendall, M. G. and Stuart A. (1976). The Advanced Theory of Statistics, Vol. 3. Hafner, New York.
Lancaster, P. and Salkauskas, K. (1981). Surfaces generated by moving least squares methods. Mathematics of Computation37, 141–158.
Lancaster, P. and Salkauskas, K. (1986). Curve and Surface Fitting: An Introduction. Academic Press: London.
Loader, C. (1993). Change point estimation using local regression. Unpublished manuscript, available by ftp from http://netlib.att.com.
Loader, C.(1994). Computing nonparametric function estimates. Computing Science and Statistics: Proceedings of the 26th Symposium on the interface, 356–361.
Loader, C. (1995). Local likelihood density estimation. Ann. Statist., to appear.
Macaulay, F. R. (1931). The Smoothing of Time Series. National Bureau of Economic Research, New York.
Mallows, C. (1973). Some comments onC p . Technometrics15, 661–675.
Mallows, C. (1974). Discussion of a paper of Beaton and Tukey.Technometrics16, 187–188.
Mcdonald, J. A. and Owen, A. B. (1986). Smoothing with split linear fits. Technometrics28, 195–208.
Müller, H.-G. (1984). Smooth optimum kernel estimators of densities, regression curves and modes. Ann. Statist. 12, 766–774.
Müller, H.-G. (1987). Weighted local regression and kernel methods for nonparametric curve fitting. J. Amer. Statist. Assn. 82, 231–238.
Nadaraya, E. A. (1964). On estimating regression. Theor. Probab. Appl9, 141–142.
Parzen, E. (1962). On estimation of a probability density function and mode. Ann. Math. Statist. 33, 1065–1076.
Rosenblatt, M. (1956). Remarks on some nonparametric estimates of a density function. Ann. Math. Statist. 27, 832–837.
Rosenblatt, M. (1971). Curve estimates. Ann. Math. Statist. 42, 1815–1842.
Ruppert, D. and Wand, M. P. (1992). Multivariate locally weighted least squares regression. Ann. Statist. 22, No. 3.
Shishkin, J, Young, A. H., and Musgrave, J. C. (1967). The X-11 variant of the Census Method II seasonal adjustment program. Technical Paper 15, U.S. Bureau of the Census.
Speckman, P. (1995). Discussion of a paper of Donoho et al. J. Royal Statist. Soc, Ser. B. 57, 337–338.
Spencer, J. (1904a). On the graduation of the rates of sickness and mortality. J. Inst. Act38, 334–347.
Spencer, J. (1904b). Graduation of a sickness table by Makeham’s hypothesis. Biometrika3, 52–57.
Staniswalis, J. (1988). The kernel estimate of a regression function in likelihood-based models. J. Amer. Statist. Assn. 84, 276–283.
Stigler, S. M. (1978). Mathematical statistics in the early States.Ann. Statist6, 239–265.
Stoker, T. M. (1993). Smoothing bias in density derivative estimation. J. Amer. Statist. Assn. 88, 855–863.
Stone, C. J. (1977). Consistent nonparametric regression (with discussion). Ann. Statist. 5, 595–620.
Stone, C. J. (1980). Optimal rates of convergence for nonparametric estimators. Ann. Statist. 8, 1348–1360.
Stone, M. (1974). Cross-validatory choice of assessment of statistical predictions (with discussion). J. R. Statist. Soc. B36, 111–47.
Tibshirani, R. and Hastie, T. (1987). Local likelihood estimation. J. Amer. Statist. Assn. 82, 559–567.
Tsybakov, A. B. (1986). Robust reconstruction of functions by the local-approximation method. Prob. Inform. Trans. 22, 69–84.
Watson, G. S. (1964). Smooth regression analysis. Sankhya Ser. A 26, 359–372.
Wahba, G. (1990). Spline Functions for Observational Data. SIAM, Philadelphia.
Whittaker, E. T. (1923). On a new method of graduation. Proc. Edinburgh Math. Soc. 41, 63–75.
Woolhouse, W. S. B. (1870). Explanation of a new method of adjusting mortality tables, with some observations upon Mr. Makeham’s modification of Gompertz’s theory. J. Inst. Act. 15, 389–410.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 1996 Physica-Verlag Heidelberg
About this paper
Cite this paper
Cleveland, W.S., Loader, C. (1996). Smoothing by Local Regression: Principles and Methods. In: Härdle, W., Schimek, M.G. (eds) Statistical Theory and Computational Aspects of Smoothing. Contributions to Statistics. Physica-Verlag HD. https://doi.org/10.1007/978-3-642-48425-4_2
Download citation
DOI: https://doi.org/10.1007/978-3-642-48425-4_2
Publisher Name: Physica-Verlag HD
Print ISBN: 978-3-7908-0930-5
Online ISBN: 978-3-642-48425-4
eBook Packages: Springer Book Archive