We study a standard method of regularization by projections of the linear inverse problem Y=Ax+ε, where ε is a white Gaussian noise, and A is a known compact operator. It is assumed that the eigenvalues of AA* converge to zero with exponential decay. Such behavior of the spectrum is typical for inverse problems related to elliptic differential operators. As model example we consider recovering of unknown boundary conditions in the Dirichlet problem for the Laplace equation on the unit disk. By using the singular value decomposition of A, we construct a projection estimator of x. The bandwidth of this estimator is chosen by a data-driven procedure based on the principle of minimization of penalized empirical risk. We provide non–asymptotic upper bounds for the mean square risk of this method and we show, in particular, that this approach gives asymptotically minimax estimators in our model example.
This is a preview of subscription content, log in to check access.
Buy single article
Instant access to the full article PDF.
Price includes VAT for USA
Subscribe to journal
Immediate online access to all issues from 2019. Subscription will auto renew annually.
This is the net price. Taxes to be calculated in checkout.
Akaike, H.: Information theory and an extension of the maximum likelihood principle. Proc. 2nd Intern. Symp. Inf. Theory, Petrov P.N. and Csaki F. (eds.), Budapest, 1973, pp. 267–281
Barron, A., Birge, L., Massart, P.: Risk bounds for model selection via penalization. Probab. Theory Relat. Fields 113, 301–413 (1999)
Belitser, E.N., Levit, Ya, B.: On minimax filtering on ellipsoids. Math. Meth. Statist. 4, 259–273 (1995)
Cavalier, L., Golubev, G., Lepski, O., Tsybakov, A: (2003) Block thresholding and sharp adaptive estimation in severely ill-posed inverse problems. Probab. Theory Appl. 3 to appear.
Cavalier, L., Tsybakov, A.: Sharp adaptation for inverse problems with random noise. Probab. Theory Relat. Fields 123, 323–354 (2002)
Cavalier, L., Golubev, G.K., Picard, D., Tsybakov, A.B.: Oracle inequalities for inverse problems. Ann. Stat. 30 (3), 843–874 (2002)
Engl, H., Hanke, M., Nuebauer, A.: Regularization of inverse problems. Kluwer academic publishers, 2000
Golubev, G.K., Khasminski, R.Z.: Statistical approach to some inverse boundary problems for partial differential equations. Probl. Inform. Transm. 35 (2), 51–66 (1999)
Efromovich, S.: Robust and efficient recovery of a signal passed through a filter and then contaminated by non-Gaussian noise. IEEE Trans. Inform. Theory 43, 1184–1191 (1997)
Ermakov, M.S.: On optimal solutions of the deconvolution problem. Inverse Probl. 6 (5), 863–872 (1990)
Johnstone, I.M.: Wavelet shrinkage for correlated data and inverse problems: adaptivity results. Statistica Sinica 9, 51–83 (1999)
Johnstone, I.M., Silverman, B.W.: Wavelet threshold estimators for data with correlated noise. J. Royal Stat. Soc. Ser. B. 59, 319–351 (1997)
Kneip, A.: Ordered linear smoothers. Ann. Stat. 22, 835–866 (1994)
Landau, H.J., Pollak, H.Q.: Prolate spheroidal wave function, Fourier analysis, and uncertainty – II. Bell System Tech. J. 40, 65–84 (1961)
Landau, H.J., Pollak, H.Q.: Prolate spheroidal wave function, Fourier analysis, and uncertainty – III. Bell System Tech. J. 41, 1295–1336 (1962)
Lavrentiev, M.M.: Some improperly posed problems of mathematical physics. Springer Verlag, Berlin Heidelberg New-York, 1967
Mair, B., Ruymgaart, F.H.: Statistical estimation in Hilbert scale. SIAM J. Appl. Math. 56, 1424–1444 (1996)
Mallows, C.L.: Some comments on C p . Technometrics 15, 661–675 (1973)
Pinsker, M.S.: Optimal filtering of square integrable signals in Gaussian white noise. Probl. Inform. Transm. 16, 120–133 (1980)
Shibata, R.: An optimal selection of regression variables. Boimetrika 68, 45–54 (1981)
Slepjan, D., Pollak, H.Q.: Prolate spheroidal wave function, Fourier analysis, and uncertainty – I. Bell System Tech. J. 40, 43–64 (1961)
Slepjan, D.: Some asymptotic expansions for prolate spheroidal wave functions. J. Math. Rhys. 44, 99–140 (1965)
Stein, C.M.: Estimation of the mean of a multivariate normal distribution. Ann. Stat. 9, 1135–1151 (1981)
Sudakov, V.N., Khalfin, L.A.: Statistical approach to ill-posed problems in mathematical physics. Soviet Mathematics Doklady 157, 1094–1096 (1964)
Sullivan, F.O’.: A statistical perspective on ill-posed inverse problems. Stat. Sci. 1, 502–527 (1996)
Tsybakov, A.B.: On the best rate of adaptive estimation in some inverse problems. C.R. Acad. Sci. Paris 330 Série I, 835–840 (2000)
Mathematics Subject Classification (2000):Primary 62G05, 62G20; secondary 62C20
About this article
Cite this article
Golubev, Y. The principle of penalized empirical risk in severely ill-posed problems. Probab. Theory Relat. Fields 130, 18–38 (2004). https://doi.org/10.1007/s00440-004-0362-y
- Ill-posed problem
- Partial differential equation
- Singular value decomposition
- Projection estimator
- Minimax risk
- Empirical risk