Abstract
In Chap. 3, the data {(Y t , x t ), t = 1, …, n} were assumed to have been generated by model (3.1). In this chapter, we address the subject of Model-Free regression.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Akritas MG, VanKeilegom I (2001) Non-parametric estimation of the residual distribution. Scand J Stat 28(3):549–567
Alonso AM, Peña D, Romo J (2002) Forecasting time series with sieve bootstrap. J Stat Plan Infer 100(1):1–11
Altman NS (1992) An introduction to kernel and nearest-neighbor nonparametric regression. Am Stat 46(3):175–185
Andersen TG, Bollerslev T (1998) Answering the sceptics: yes, standard volatility models do provide accurate forecasts. Int Econ Rev 39(4):885–905
Andersen TG, Bollerslev T, Christoffersen PF, Diebold FX (2006) Volatility and correlation forecasting. In: Elliott G, Granger CWJ, Timmermann A (eds) Handbook of economic forecasting. North-Holland, Amsterdam, pp 778–878
Andersen TG, Bollerslev T, Meddahi N (2004) Analytic evaluation of volatility forecasts. Int Econ Rev 45:1079–1110
Atkinson AC (1985) Plots, transformations and regression. Clarendon Press, Oxford
Antoniadis A, Paparoditis E, Sapatinas T (2006) A functional wavelet-kernel approach for time series prediction. J R Stat Soc Ser B 68(5):837–857
Bachelier L (1900) Theory of speculation. Reprinted in Cootner PH (ed) The random character of stock market prices. MIT Press, Cambridge, MA, pp 17–78, 1964
Barndorff-Nielsen OE, Nielsen B, Shephard N, Ysusi C (1996) Measuring and forecasting financial variability using realized variance with and without a model. In: Harvey AC, Koopman SJ, Shephard N (eds) State space and unobserved components models: theory and applications. Cambridge University Press, Cambridge, pp 205–235
Bartlett MS (1946) On the theoretical specification of sampling properties of autocorrelated time series. J R Stat Soc Suppl 8:27–41
Beran R (1990) Calibrating prediction regions. J Am Stat Assoc 85:715–723
Berkes I, Gombay E, Horvath L, Kokoszka P (2004) Sequential change-point detection in GARCH (p, q) models. Econ Theory 20(6):1140–1167
Bertail P, Clémencon S (2006) Regenerative block bootstrap for Markov chains. Bernoulli 12(4):689–712
Bickel P, Gel YR (2011) Banded regularization of autocovariance matrices in application to parameter estimation and forecasting of time series. J R Stat Soc Ser B 73(5):711–728
Bickel P, Levina E (2008a) Regularized estimation of large covariance matrices. Ann Stat 36:199–227
Bickel P, Levina E (2008b) Covariance regularization via thresholding. Ann Stat 36:2577–2604
Bickel P, Li B (2006) Regularization in statistics. Test 15(2):271–344
Bollerslev T (1986) Generalized autoregressive conditional heteroscedasticity. J Econ 31:307–327
Bollerslev T, Chou R, Kroner K (1992) ARCH modelling in finance: a review of theory and empirical evidence. J Econ 52:5–60
Bose A (1988) Edgeworth correction by bootstrap in autoregressions. Ann Stat 16:1345–1741
Bose A, Chatterjee S (2002) Comparison of bootstrap and jackknife variance estimators in linear regression: second order results. Stat Sin 12:575–598
Box GEP (1976) Science and statistics. J Am Stat Assoc 71(356):791–799
Box GEP, Cox DR (1964) An analysis of transformations. J R Stat Soc Ser B 26:211–252
Box GEP, Draper NR (1987) Empirical model-building and response surfaces. Wiley, New York
Box GEP, Jenkins GM (1976) Time series analysis, control, and forecasting. Holden Day, San Francisco
Breidt FJ, Davis RA, Dunsmuir W (1995) Improved bootstrap prediction intervals for autoregressions. J Time Ser Anal 16(2):177–200
Breiman L (1996) Bagging predictors. Mach Learn 24:123–140
Breiman L, Friedman J (1985) Estimating optimal transformations for multiple regression and correlation. J Am Stat Assoc 80:580–597
Brockwell PJ, Davis RA (1991) Time series: theory and methods, 2nd edn. Springer, New York
Brockwell PJ, Davis RA (1988) Simple consistent estimation of the coefficients of a linear filter. Stoch Process Appl 22:47–59
Bühlmann P, van de Geer S (2011) Statistics for high-dimensional data. Springer, New York
Cai TT, Ren Z, Zhou HH (2013) Optimal rates of convergence for estimating Toeplitz covariance matrices. Probab Theory Relat Fields 156(1–2):101–143
Cao R, Febrero-Bande M, Gonzalez-Manteiga W, Prada-Sanchez JM, GarcIa-Jurado I (1997) Saving computer time in constructing consistent bootstrap prediction intervals for autoregressive processes. Commun Stat Simul Comput 26(3):961–978
Carmack PS, Schucany WR, Spence JS, Gunst RF, Lin Q, Haley RW (2009) Far casting cross-validation. J Comput Graph Stat 18(4):879–893
Carroll RJ, Ruppert D (1988) Transformations and weighting in regression. Chapman and Hall, New York
Caroll RJ, Ruppert D (1991) Prediction and tolerance intervals with transformation and/or weighting. Technometrics 33(2):197–210
Chatterjee S, Bose A (2005) Generalized bootstrap for estimating equations. Ann Stat 33:414–436
Chen X, Xu M, Wu W-B (2013) Covariance and precision matrix estimation for high-dimensional time series. Ann Stat 41(6):2994–3021
Cheng T-ZF, Ing C-K, Yu S-H (2015) Inverse moment bounds for sample autocovariance matrices based on detrended time series and their applications. Linear Algebra Appl (to appear)
Choi B-S (1992) ARMA model identification. Springer, New York
Cox DR (1975) Prediction intervals and empirical Bayes confidence intervals. In: Gani J (eds) Perspectives in probability and statistics. Academic, London, pp 47–55
Dahlhaus R (1997) Fitting time series models to nonstationary processes. Ann Stat 25(1):1–37
Dahlhaus R (2012) Locally stationary processes. In: Handbook of statistics, vol 30. Elsevier, Amsterdam, pp 351–412
Dahlhaus R, Subba Rao S (2006) Statistical inference for time-varying ARCH processes. Ann Stat 34(3):1075–1114
Dai J, Sperlich S (2010) Simple and effective boundary correction for kernel densities and regression with an application to the world income and Engel curve estimation. Comput Stat Data Anal 54(11):2487–2497
DasGupta A (2008) Asymptotic theory of statistics and probability. Springer, New York
Davison AC, Hinkley DV (1997) Bootstrap methods and their applications. Cambridge University Press, Cambridge
Dawid AP (2004) Probability, causality, and the empirical world: a Bayes-de Finetti-Popper-Borel synthesis. Stat Sci 19(1):44–57
Devroye L (1981) Laws of the iterated logarithm for order statistics of uniform spacings. Ann Probab 9(6):860–867
Dowla A, Paparoditis E, Politis DN (2003) Locally stationary processes and the local block bootstrap. In: Akritas MG, Politis DN (eds) Recent advances and trends in nonparametric statistics. Elsevier, Amsterdam, pp 437–444
Dowla A, Paparoditis E, Politis DN (2013) Local block bootstrap inference for trending time series. Metrika 76(6):733–764
Draper NR, Smith H (1998) Applied regression analysis, 3rd edn. Wiley, New York
Efron B (1979) Bootstrap methods: another look at the jackknife. Ann Stat 7:1–26
Efron B (1983) Estimating the error rate of a prediction rule: improvement on cross-validation. J Am Stat Assoc 78:316–331
Efron B (2014) Estimation and accuracy after model selection. J Am Stat Assoc 109:991–1007
Efron B, Tibshirani RJ (1993) An introduction to the bootstrap. Chapman and Hall, New York
Engle R (1982) Autoregressive conditional heteroscedasticity with estimates of the variance of UK inflation. Econometrica 50:987–1008
Fama EF (1965) The behaviour of stock market prices. J Bus 38:34–105
Fan J (1993) Local linear regression smoothers and their minimax efficiencies. Ann Stat 21(1):196–216
Fan J, Gijbels I (1996) Local polynomial modelling and its applications. Chapman and Hall, New York
Fan J, Yao Q (2003) Nonlinear time series: nonparametric and parametric methods. Springer, New York
Ferraty F, Vieu P (2006) Nonparametric functional data analysis. Springer, New York
Francq C, Zakoian JM (2011) GARCH models: structure, statistical inference and financial applications. Wiley, New York
Franke J, Härdle W (1992) On bootstrapping kernel spectral estimates. Ann Stat 20:121–145
Franke J, Kreiss J-P, Mammen E (2002) Bootstrap of kernel smoothing in nonlinear time series. Bernoulli 8(1):1–37
Freedman DA (1981) Bootstrapping regression models. Ann Stat 9:1218–1228
Freedman D (1984) On bootstrapping two-stage least-squares estimates in stationary linear models. Ann Stat 12:827–842
Fryzlewicz P, Van Bellegem S, Von Sachs R (2003) Forecasting non-stationary time series by wavelet process modelling. Ann Inst Stat Math 55(4):737–764
Gangopadhyay AK, Sen PK (1990) Bootstrap confidence intervals for conditional quantile functions. Sankhya Ser A 52(3):346–363
Geisser S (1993) Predictive inference: an introduction. Chapman and Hall, New York
Gijbels I, Pope A, Wand MP (1999) Understanding exponential smoothing via kernel regression. J R Stat Soc Ser B 61:39–50
Ghysels E, Santa-Clara P, Valkanov R (2006) Predicting volatility: getting the most out of return data sampled at different frequencies. J Econ 131(1–2):59–95
Gouriéroux C (1997) ARCH models and financial applications. Springer, New York
Gray RM (2005) Toeplitz and circulant matrices: a review. Commun Inf Theory 2(3):155–239
Grenander U, Szegö G (1958) Toeplitz forms and their applications, vol 321. University of California Press, Berkeley
Hahn J (1995) Bootstrapping quantile regression estimators. Econ Theory 11(1):105–121
Hall P (1992) The bootstrap and edgeworth expansion. Springer, New York
Hall P (1993) On Edgeworth expansion and bootstrap confidence bands in nonparametric curve estimation. J R Stat Soc Ser B 55:291–304
Hall P, Wehrly TE (1991) A geometrical method for removing edge effects from kernel type nonparametric regression estimators. J Am Stat Assoc 86:665–672
Hall P, Wolff RCL, Yao Q (1999) Methods for estimating a conditional distribution function. J Am Stat Assoc 94(445):154–163
Hamilton JD (1994) Time series analysis. Princeton University Press, Princeton, NJ
Hampel FR (1973) Robust estimation, a condensed partial survey. Z Wahrscheinlichkeitstheorie verwandte Gebiete 27:87–104
Hansen BE (2004) Nonparametric estimation of smooth conditional distributions. Working paper, Department of Economics, University of Wisconsin
Hansen PR, Lunde A (2005) A forecast comparison of volatility models: does anything beat a GARCH (1,1)? J Appl Econ 20:873–889
Hansen PR, Lunde A (2006) Consistent ranking of volatility models. J Econ 131:97–121
Hansen PR, Lunde A, Nason JM (2003) Choosing the best volatility models: the model confidence set approach. Oxf Bull Econ Stat 65:839–861
Härdle W (1990) Applied nonparametric regression. Cambridge University Press, Cambridge
Härdle W, Bowman AW (1988) Bootstrapping in nonparametric regression: local adaptive smoothing and confidence bands. J Am Stat Assoc 83:102–110
Härdle W, Marron JS (1991) Bootstrap simultaneous error bars for nonparametric regression. Ann Stat 19:778–796
Härdle W, Vieu P (1992) Kernel regression smoothing of time series. J Time Ser Anal 13:209–232
Hart JD (1997) Nonparametric smoothing and lack-of-fit tests. Springer, New York
Hart JD, Yi S (1998) One-sided cross-validation. J Am Stat Assoc 93(442):620–631
Hastie T, Tibshirani R, Friedman JH (2009) The elements of statistical learning: data mining, inference, and predictions, 2nd edn. Springer, New York
Hocking RR (1976) The analysis and selection of variables in linear regression. Biometrics 31(1):1–49
Hong Y (1999) Hypothesis testing in time series via the empirical characteristic function: a generalized spectral density approach. J Am Stat Assoc 94:1201–1220
Hong Y, White H (2005) Asymptotic distribution theory for nonparametric entropy measures of serial dependence. Econometrica 73(3):837–901
Horowitz J (1998) Bootstrap methods for median regression models. Econometrica 66(6):1327–1351
Huber PJ (1973) Robust regression: asymptotics, conjectures and Monte Carlo. Ann Stat 1:799–821
Hurvich CM, Zeger S (1987) Frequency domain bootstrap methods for time series. Technical Report, New York University, Graduate School of Business Administration
Jentsch C, Politis DN (2015) Covariance matrix estimation and linear process bootstrap for multivariate time series of possibly increasing dimension. Ann Stat 43(3):1117–1140
Kim TY, Cox DD (1996) Bandwidth selection in kernel smoothing of time series. J Time Ser Anal 17:49–63
Kirch C, Politis DN (2011) TFT-Bootstrap: resampling time series in the frequency domain to obtain replicates in the time domain. Ann Stat 39(3):1427–1470
Koenker R (2005) Quantile regression. Cambridge University Press, Cambridge
Kokoszka P, Leipus R (2000) Change-point estimation in ARCH models. Bernoulli 6(3):513–539
Kokoszka P, Politis DN (2011) Nonlinearity of ARCH and stochastic volatility models and Bartlett’s formula. Probab Math Stat 31:47–59
Koopman SJ, Jungbacker B, Hol E (2005) Forecasting daily variability of the S&P 100 stock index using historical, realised and implied volatility measurements. J Empir Finance 12:445–475
Kreiss J-P, Paparoditis E (2011) Bootstrap methods for dependent data: a review. J Korean Stat Soc 40(4):357–378
Kreiss J-P, Paparoditis E (2012) The hybrid wild bootstrap for time series. J Am Stat Assoc 107:1073–1084
Kreiss J-P, Paparoditis E, Politis DN (2011) On the range of validity of the autoregressive sieve bootstrap. Ann Stat 39(4):2103–2130
Kuhn M, Johnson K (2013) Applied predictive modeling. Springer, New York
Künsch H (1989) The jackknife and the bootstrap for general stationary observations. Ann Stat 17:1217–1241
Lahiri SN (2003) A necessary and sufficient condition for asymptotic independence of discrete Fourier transforms under short-and long-range dependence. Ann Stat 31(2):613–641
Lei J, Robins J, Wasserman L (2013) Distribution free prediction sets. J Am Stat Assoc 108:278–287
Li Q, Racine JS (2007) Nonparametric econometrics. Princeton University Press, Princeton
Linton OB, Chen R, Wang N, Härdle W (1997) An analysis of transformations for additive nonparametric regression. J Am Stat Assoc 92:1512–1521
Linton OB, Sperlich S, van Keilegom I (2008) Estimation of a semiparametric transformation model. Ann Stat 36(2):686–718
Loader C (1999) Local regression and likelihood. Springer, New York
Mandelbrot B (1963) The variation of certain speculative prices. J Bus 36:394–419
Maronna RA, Martin RD, Yohai VJ (2006) Robust statistics: theory and methods. Wiley, New York
Masarotto G (1990) Bootstrap prediction intervals for autoregressions. Int J Forecast 6(2):229–239
Masry E, Tjøstheim D (1995) Nonparametric estimation and identification of nonlinear ARCH time series. Econ Theory 11:258–289
McCullagh P, Nelder J (1983) Generalized linear models. Chapman and Hall, London
McMurry T, Politis DN (2008) Bootstrap confidence intervals in nonparametric regression with built-in bias correction. Stat Probab Lett 78:2463–2469
McMurry T, Politis DN (2010) Banded and tapered estimates of autocovariance matrices and the linear process bootstrap. J Time Ser Anal 31:471–482 [Corrigendum: J Time Ser Anal 33, 2012]
McMurry T, Politis DN (2015) High-dimensional autocovariance matrices and optimal linear prediction (with discussion). Electr J Stat 9:753–822
Meddahi N (2001) An eigenfunction approach for volatility modeling. Technical report, CIRANO Working paper 2001s–70, University of Montreal
Mikosch T, Starica C (2004) Changes of structure in financial time series and the GARCH model. Revstat Stat J 2(1):41–73
Nadaraya EA (1964) On estimating regression. Theory Probab Appl 9:141–142
Nelson D (1991) Conditional heteroscedasticity in asset returns: a new approach. Econometrica 59:347–370
Neumann M, Polzehl J (1998) Simultaneous bootstrap confidence bands in nonparametric regression. J Nonparametr Stat 9:307–333
Olive DJ (2007) Prediction intervals for regression models. Comput Stat Data Anal 51:3115–3122
Pagan A, Ullah A (1999) Nonparametric econometrics. Cambridge University Press, Cambridge
Pan L, Politis DN (2014) Bootstrap prediction intervals for Markov processes. Discussion paper, Department of Economics, University of California, San Diego. Retrievable from http://escholarship.org/uc/item/7555757g. Accepted for publication in CSDA Annals of Computational and Financial Econometrics
Pan L, Politis DN (2015) Bootstrap prediction intervals for linear, nonlinear and nonparametric autoregressions (with discussion). J Stat Plan Infer (to appear)
Paparoditis E, Politis DN (1998) The backward local bootstrap for conditional predictive inference in nonlinear time series. In: Lipitakis EA (ed) Proceedings of the 4th Hellenic-European conference on computer mathematics and its applications (HERCMA’98). Lea Publishing, Athens, pp 467–470
Paparoditis E, Politis DN (2001) A Markovian local resampling scheme for nonparametric estimators in time series analysis. Econ Theory 17(3):540–566
Paparoditis E, Politis DN (2002a) The local bootstrap for Markov processes. J Stat Plan Infer 108(1):301–328
Paparoditis E, Politis DN (2002b) Local block bootstrap. C R Acad Sci Paris Ser I 335:959–962
Pascual L, Romo J, Ruiz E (2004) Bootstrap predictive inference for ARIMA processes. J Time Ser Anal 25(4):449–465
Patel JK (1989) Prediction intervals: a review. Commun Stat Theory Methods 18:2393–2465
Patton AJ (2011) Volatility forecast evaluation and comparison using imperfect volatility proxies. J Econ 160(1):246–256
Politis DN (1998) Computer-intensive methods in statistical analysis. IEEE Signal Process Mag 15(1):39–55
Politis DN (2003a) A normalizing and variance-stabilizing transformation for financial time series. In: Akritas MG, Politis DN (eds) Recent advances and trends in nonparametric statistics. Elsevier, Amsterdam, pp 335–347
Politis DN (2003b) Adaptive bandwidth choice. J Nonparametr Stat 15(4–5):517–533
Politis DN (2004) A heavy-tailed distribution for ARCH residuals with application to volatility prediction. Ann Econ Finance 5:283–298
Politis DN (2007a) Model-free vs. model-based volatility prediction. J Financ Econ 5(3):358–389
Politis DN (2007b) Model-free prediction, vol LXII. Bulletin of the International Statistical Institute, Lisbon, pp 1391–1397
Politis DN (2010) Model-free model-fitting and predictive distributions. Discussion paper, Department of Economics, University of California, San Diego. Retrievable from: http://escholarship.org/uc/item/67j6s174
Politis DN (2013) Model-free model-fitting and predictive distributions (with discussion). Test 22(2):183–250
Politis DN (2014) Bootstrap confidence intervals in nonparametric regression without an additive model. In: Akritas MG, Lahiri SN, Politis DN (eds) Proceedings of the first conference of the international society for nonParametric statistics. Springer, New York, pp 271–282
Politis DN, Romano JP (1992) A general resampling scheme for triangular arrays of alpha-mixing random variables with application to the problem of spectral density estimation. Ann Stat 20:1985–2007
Politis DN, Romano JP (1994) The stationary bootstrap. J Am Stat Assoc 89(428):1303–1313
Politis DN, Romano JP, Wolf M (1999) Subsampling. Springer, New York
Politis DN, Thomakos D (2008) Financial time series and volatility prediction using NoVaS transformations. In: Rapach DE, Wohar ME (eds) Forecasting in the presence of structural breaks and model uncertainty. Emerald Group Publishing, Bingley, pp 417–447
Politis DN, Thomakos DD (2012) NoVaS transformations: flexible inference for volatility forecasting. In: Chen X, Swanson N (eds) Recent advances and future directions in causality, prediction, and specification analysis: essays in honor of Halbert L. White Jr. Springer, New York, pp 489–528
Pourahmadi M (1999) Joint mean-covariance models with applications to longitudinal data: unconstrained parameterisation. Biometrika 86(3):677–690
Pourahmadi M (2011) Modeling covariance matrices: the GLM and regularization perspectives. Stat Sci 26(3):369–387
Poon S, Granger C (2003) Forecasting volatility in financial markets: a review. J Econ Lit 41:478–539
Priestley MB (1965) Evolutionary spectra and non-stationary processes. J R Stat Soc Ser B 27:204–237
Priestley MB (1988) Nonlinear and nonstationary time series analysis. Academic, London
Raïs N (1994) Méthodes de rééchantillonnage et de sous-échantillonnage pour des variables aléatoires dépendantes et spatiales. Ph.D. thesis, University of Montreal
Rajarshi MB (1990) Bootstrap in Markov sequences based on estimates of transition density. Ann Inst Stat Math 42:253–268
Resnick S, Samorodnitsky G, Xue F (1999) How misleading can sample ACF’s of stable MA’s be? (Very!) Ann Appl Probab 9(3):797–817
Rissanen J, Barbosa L (1969) Properties of infinite covariance matrices and stability of optimum predictors. Inf Sci 1:221–236
Rosenblatt M (1952) Remarks on a multivariate transformation. Ann Math Stat 23:470–472
Ruppert D, Cline DH (1994) Bias reduction in kernel density estimation by smoothed empirical transformations. Ann Stat 22:185–210
Schmoyer RL (1992) Asymptotically valid prediction intervals for linear models. Technometrics 34:399–408
Schucany WR (2004) Kernel smoothers: an overview of curve estimators for the first graduate course in nonparametric statistics. Stat Sci 19:663–675
Seber GAF, Lee AJ (2003) Linear regression analysis, 2nd edn. Wiley, New York
Shao J, Tu D (1995) The Jackknife and bootstrap. Springer, New York
Shapiro SS, Wilk M (1965) An analysis of variance test for normality (complete samples). Biometrika 52:591–611
Shephard N (1996) Statistical aspects of ARCH and stochastic volatility. In: Cox DR, Hinkley DV, Barndorff-Nielsen OE (eds) Time series models in econometrics, finance and other fields. Chapman and Hall, London, pp 1–67
Shi SG (1991) Local bootstrap. Ann Inst Stat Math 43:667–676
Shmueli G (2010) To explain or to predict? Stat Sci 25:289–310
Starica C, Granger C (2005) Nonstationarities in stock returns. Rev Econ Stat 87(3):503–522
Stine RA (1985) Bootstrap prediction intervals for regression. J Am Stat Assoc 80:1026–1031
Stine RA (1987) Estimating properties of autoregressive forecasts. J Am Stat Assoc 82:1072–1078
Thombs LA, Schucany WR (1990) Bootstrap prediction intervals for autoregression. J Am Stat Assoc 85:486–492
Tibshirani R (1988) Estimating transformations for regression via additivity and variance stabilization. J Am Stat Assoc 83:394–405
Tibshirani R (1996) Regression shrinkage and selection via the Lasso. J R Stat Soc Ser B 58(1):267–288
Thomakos DD, Klepsch J, Politis DN (2015) Multivariate NoVaS and inference on conditional correlations. Working paper, Department of Economics, University of California, San Diego
Wang L, Brown LD, Cai TT, Levine M (2008) Effect of mean on variance function estimation in nonparametric regression. Ann Stat 36:646–664
Wang L, Politis DN (2015) Asymptotic validity of bootstrap confidence intervals in nonparametric regression without an additive model. Working paper, Department of Mathematics, University of California, San Diego
Watson GS (1964) Smooth regression analysis. Sankhya Ser A 26:359–372
Wolf M, Wunderli D (2015) Bootstrap joint prediction regions. J Time Ser Anal 36(3):352–376
Wolfowitz J (1957) The minimum distance method. Ann Math Stat 28:75–88
Wu S, Harris TJ, McAuley KB (2007) The use of simplified or misspecified models: linear case. Can J Chem Eng 75:386–398
Wu W-B, Pourahmadi M (2009) Banding sample autocovariance matrices of stationary processes. Stat Sin 19(4):1755–1768
Xiao H, Wu W-B (2012) Covariance matrix estimation for stationary time series. Ann Stat 40(1):466–493
Zhang T, Wu W-B (2011) Testing parametric assumptions of trends of nonstationary time series. Biometrika 98(3):599–614
Zhou Z, Wu W-B (2009) Local linear quantile estimation for non-stationary time series. Ann Stat 37:2696–2729
Zhou Z, Wu W-B (2010) Simultaneous inference of linear models with time-varying coefficients. J R Stat Soc Ser B 72:513–531
Author information
Authors and Affiliations
Appendix 1: High-Dimensional and/or Functional Regressors
Appendix 1: High-Dimensional and/or Functional Regressors
So far in Chap. 4, it has been assumed for simplicity that the regressors are univariate; we now relax this assumption and show how the Model-free ideas are immediately applicable bearing in mind, of course, the curse of dimensionality. Throughout this Appendix we consider regression data \((Y _{1},x_{1}),\ldots,(Y _{n},x_{n})\) where Y k is the univariate response associated with a regressor value x k that takes values in a linear vector space E equipped with a semi-metric d(⋅ , ⋅ ). The space E can be high-dimensional or even infinite-dimensional, e.g., a function space; see Chap. 5 of Ferraty and Vieu (2006) for details. We will assume that the data adhere to the Model-free regression setup defined in Sect. 4.2. As before, we can estimate \(D_{x}(y) = P\{Y _{j} \leq y\vert X_{j} = x\}\) by the “local” weighted average
where \(\tilde{K}\left (h^{-1}d(x,x_{i})\right ) = K\left (h^{-1}d(x,x_{i})\right )/\sum _{k=1}^{n}K\left (h^{-1}d(x,x_{k})\right )\), the kernel K is a bounded, symmetric probability density with compact support, and h > 0 is the bandwidth parameter. For any fixed y, estimator \(\hat{D}_{x}(y)\) is just a Nadaraya-Watson smoother of the variables 1{Y i ≤ y} for i = 1, …, n. As such, it is discontinuous as a function of y; to come up with a smooth estimator, we replace 1{Y i ≤ y} by \(\varLambda \left (\frac{Y _{i}-y} {h_{0}} \right )\) in Eq. (4.22), leading to the estimator
where h 0 is another bandwidth parameter, and \(\varLambda (y) =\int _{ -\infty }^{y}\lambda (s)ds\) with λ(⋅ ) being a symmetric density function that is continuous and strictly positive over its support. As a result, \(\bar{D}_{x}(y)\) is differentiable and strictly increasing in y. Assuming Eq. (4.2) and additional regularity conditions, e.g., that as n → ∞, max(h, h 0) → 0 but not too fast, Ferraty and Vieu (2006, Theorem 6.4) showed that
for any α ∈ [0, 1] as long as D x (y) is strictly increasing at \(y = D_{x}^{-1}(\alpha )\). It is conjectured that a similar consistency result can be obtained in the case of deterministic regressors that follow a regular design. Conditionally on event \(S_{n} =\{ X_{j} = x_{j}\ \mbox{ for}\ j = 1,\ldots,n\}\), the Y t s are non–i.i.d. but this is only because they do not have identical distributions. Since they are assumed to be continuous random variables, the probability integral transform can again be used to transform them towards “i.i.d.–ness.” Hence, as in Sect. 4.2, our proposed transformation amounts to defining
Equation (4.24) then implies that \(u_{1},\ldots,u_{n}\) should be approximately i.i.d. Uniform (0,1) provided n is large. We can now invoke the Model-Free Prediction Principle in order to construct optimal predictors of g(Y f) where Y f is the out-of-sample response associated with regressor value x f, and g(⋅ ) is a real-valued function; for simplicity, we focus on the case g(x) = x. As usual, the L 2–optimal predictor of Y f is the expected value of Y f given x f that is estimated in the Model-Free paradigm by
Similarly, the Model-Free (MF) L 1–optimal predictor of g(Y f) is the median of the set \(\{\bar{D}_{x_{\mathrm{f}}}^{-1}(u_{i}),\ i = 1,\ldots,n\}\). Under the Limit Model-Free (LMF) paradigm, the L 2– and L 1–optimal predictors are given by \(\int _{0}^{1}\hat{D}_{x_{\mathrm{f}}}^{-1}(u)du\) and \(\hat{D}_{x_{\mathrm{f}}}^{-1}(1/2)\), respectively. Of course, one can also construct traditional estimators of the L 2– and L 1–optimal predictors of Y f; these are respectively given by
Equation (4.24) shows that \(\bar{D}_{x_{\mathrm{f}}}^{-1}(1/2)\) is a consistent estimator of the theoretical L 1–optimal predictor \(D_{x_{\mathrm{f}}}^{-1}(1/2)\). Under some additional regularity conditions, Ferraty and Vieu (2006) also showed that the Nadaraya-Watson smoother \(m_{x_{\mathrm{f}}}\) is consistent for \(E(Y _{\mathrm{f}}\vert X_{\mathrm{f}} = x_{\mathrm{f}})\) under model (4.2). As in Sect. 4.3.2, here as well it is true that the MF, LMF, and traditional predictors are asymptotically equivalent. To elaborate,
and median\(\{\bar{D}_{x_{\mathrm{f}}}^{-1}(u_{i})\} =\) \(\bar{D}_{x_{\mathrm{f}}}^{-1}(\mbox{ median}\{u_{i}\}) \simeq \bar{ D}_{x_{\mathrm{f}}}^{-1}(1/2) \simeq \hat{ D}_{x_{\mathrm{f}}}^{-1}(1/2)\) since the u i s are approximately Uniform (0,1), and \(\bar{D}_{x_{\mathrm{f}}}^{-1}(\cdot )\) is strictly increasing.
Remark 4.7.1
All the aforementioned predictors are based on either the estimator \(\bar{D}_{x_{\mathrm{f}}}(\cdot )\) or \(\hat{D}_{x_{\mathrm{f}}}(\cdot )\) whose finite-sample accuracy crucially depends on the number of data pairs \((Y _{j},X_{j})\) with regressor value that lies in the neighborhood of the point of interest x f. If few (or none) of the regressors are found close to x f, then nonparametric prediction will be highly inaccurate (or just plain impossible); this is where the curse of dimensionality may manifest in practice.
As already mentioned, the main advantage of the Model-Free, transformation-based approach to regression is that it allows us to go beyond point prediction and obtain valid predictive distributions and intervals for Y f. To do this, however, some kind of resampling procedure is necessary in order to also capture the variance due to estimation error. For example, consider the prediction interval
given in Ferraty and Vieu (2006, Eq. (5.10)); this interval is indeed asymptotically valid as it will contain Y f with probability tending to the nominal (1 −α)100%. However, interval (4.27) will be characterized by under-coverage in finite samples since the nontrivial variability in the estimated quantiles \(\hat{D}_{x_{\mathrm{f}}}^{-1}(\alpha /2)\) and \(\hat{D}_{x_{\mathrm{f}}}^{-1}(1 -\alpha /2)\) is ignored. Having mapped the responses \(Y _{1},\ldots,Y _{n}\) onto the approximately i.i.d. variables u 1, …, u n , the premises of the Model-Free Prediction Principle are seen to be satisfied. Hence, the Model-Free bootstrap Algorithm 4.4.1 applies verbatim to the current setup of nonparametric regression with univariate response and functional regressors, and the same is true for the Limit Model-Free resampling Algorithm 4.4.2. Furthermore, the Predictive Model-Free resampling Algorithm 4.5.1 also applies verbatim to the current setup.
Rights and permissions
Copyright information
© 2015 The Author
About this chapter
Cite this chapter
Politis, D.N. (2015). Model-Free Prediction in Regression. In: Model-Free Prediction and Regression. Frontiers in Probability and the Statistical Sciences. Springer, Cham. https://doi.org/10.1007/978-3-319-21347-7_4
Download citation
DOI: https://doi.org/10.1007/978-3-319-21347-7_4
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-21346-0
Online ISBN: 978-3-319-21347-7
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)