Universal output prediction and nonparametric regression for arbitrary data
We construct a class of elementary nonparametric output predictors of an unknown discrete-time nonlinear fading memory system. Our algorithms predict asymptotically well for every bounded input sequence, every disturbance sequence in certain classes, and every linear or nonlinear system that is continuous and asymptotically time-invariant, causal, and with fading memory. The predictor is based on k n -nearest neighbor estimators from nonparametric statistics. It uses only previous input and noisy output data of the system without any knowledge of the structure of the unknown system, the bounds on the input, or the properties of noise. Under additional smoothness conditions we provide rates of convergence for the time-average errors of our scheme. Finally, we apply our results to the special case of stable LTI systems.
KeywordsPrediction Error Input Sequence Near Neighbor Nonparametric Regression Input String
Unable to display preview. Download preview PDF.
- R.M. Dudley, Real Analysis and Probability, Chapman & Hall, 1989.Google Scholar
- J.D. Farmer and J.J. Sidorowich, “Exploiting chaos to predict the future and reduce noise,” Evolution, Learning, and Cognition, pp. 265–289, World Scientific, Singapore, 1988.Google Scholar
- A.J. Helmicki, C.A. Jacobson, C.N. Nett, “Control Oriented System Identification: A Worst-Case/Deterministic Approach in H ∞,” IEEE Trans. Automatic Control, vol. 36, Oct. 1991.Google Scholar
- S.R. Kulkarni, “Data-dependent nearest neighbor and kernel estimators consistent for arbitrary processes,” preprint.Google Scholar
- S.R. Kulkarni and S.E. Posner, “Rates of convergence of nearest neighbor estimation under arbitrary sampling,” IEEE Trans. Information Theory, pp. 1028–1039, July 1995.Google Scholar
- S.R. Kulkarni and S.E. Posner, “Nonparametric output prediction for nonlinear fading memory systems,” to appear IEEE Trans. Automatic Control, Nov., 1998.Google Scholar
- L. Ljung, System Identification: Theory for the User, Prentice-Hall, 1987.Google Scholar
- G. Morvai, S. Yakowitz, and L. Györfi, “Nonparametric inferences for ergodic stationary time series,” preprint.Google Scholar
- S.E. Posner, “Nonparametric estimation, regression, and prediction under minimal regularity conditions,” Ph.D. Thesis, Department of Electrical Engineering, Princeton University, 1995.Google Scholar
- C.J. Stone, “Consistent nonparametric regression,” Ann. Stat., vol. 8, pp. 1348–1360, 1977.Google Scholar