Problems of Information Transmission

, Volume 54, Issue 4, pp 351–371 | Cite as

Noise Level Estimation in High-Dimensional Linear Models

  • G. K. GolubevEmail author
  • E. A. Krymova
Methods of Signal Processing


We consider the problem of estimating the noise level σ2 in a Gaussian linear model Y = +σξ, where ξ ∈ ℝn is a standard discrete white Gaussian noise and β ∈ ℝp an unknown nuisance vector. It is assumed that X is a known ill-conditioned n × p matrix with np and with large dimension p. In this situation the vector β is estimated with the help of spectral regularization of the maximum likelihood estimate, and the noise level estimate is computed with the help of adaptive (i.e., data-driven) normalization of the quadratic prediction error. For this estimate, we compute its concentration rate around the pseudo-estimate ||Y||2/n.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Aster, R.C., Borchers, B., and Thurber, C.H., Parameter Estimation and Inverse Problems, Amsterdam: Elsevier/Academic, 2013, 2nd ed.zbMATHGoogle Scholar
  2. 2.
    Kneip, A., Ordered Linear Smoothers, Ann. Statist., 1994, vol. 22, no. 2, pp. 835–866.MathSciNetzbMATHGoogle Scholar
  3. 3.
    Engl, H.W., Hanke, M., and Neubauer, A., Regularization of Inverse Problems, Dordrecht: Kluwer, 1996.zbMATHGoogle Scholar
  4. 4.
    Golubev, G. and Levit, B., On the Second Order Minimax Estimation of Distribution Function, Math. Methods Statist., 1996, vol. 5, no. 1, pp. 1–31.MathSciNetzbMATHGoogle Scholar
  5. 5.
    Dalalyan, A.S., Golubev, G.K., and Tsybakov, A.B., Penalized Maximum Likelihood and Semiparametric Second-Order Efficiency, Ann. Statist., 2006, vol. 34, no. 1, pp. 169–201.MathSciNetzbMATHGoogle Scholar
  6. 6.
    Laurent, B. and Massart, P., Adaptive Estimation of a Quadratic Functional by Model Selection, Ann. Statist., 2000, vol. 28, no. 5, pp. 1302–1338.MathSciNetzbMATHGoogle Scholar
  7. 7.
    Kolmogoroff, A., Über das Gesetz des iterierten Logarithmus, Math. Ann., 1929, vol. 101, no. 1, pp. 126–135.MathSciNetGoogle Scholar
  8. 8.
    Green, P.J. and Silverman, B.W., Nonparametric Regression and Generalized Linear Models: A Roughness Penalty Approach, London: Chapman & Hall, 1994.zbMATHGoogle Scholar
  9. 9.
    Demmler, A. and Reinsch, C., Oscillation Matrices with Spline Smoothing, Numer. Math., 1975, vol. 24, no. 5, pp. 375–382.MathSciNetzbMATHGoogle Scholar
  10. 10.
    Utreras, F., Cross-validation Techniques for Smoothing Spline Functions in One or Two Dimensions, Smoothing Techniques for Curves Estimation (Proc. Workshop Held in Heidelberg, Germany, April 2–4, 1979), Gasser, Th. and Rosenblatt, M., Eds., Lect. Notes Math., vol. 757, New York: Springer, 1979, pp. 196–232.Google Scholar
  11. 11.
    Speckman, P., Spline Smoothing and Optimal Rates of Convergence in Nonparametric Regression Models, Ann. Statist., 1985, vol. 13, no. 3, pp. 970–983.MathSciNetzbMATHGoogle Scholar
  12. 12.
    Kolmogoroff, A., Über die beste Annäherung von Funktionen einer gegebenen Funktionenklasse, Ann. of Math. (2), 1936, vol. 37, no. 1, pp. 107–110. Russian transl. in A.N. Kolmogorov. Izbrannye trudy. Matematika i mekhanika (A.N. Kolmogorov. Selected Works. Mathematics and Mechanics), Moscow: Nauka, 1985, pp. 186–189.MathSciNetGoogle Scholar
  13. 13.
    Efromovich, S. and Low, M., On Optimal Adaptive Estimation of a Quadratic Functional, Ann. Statist., 1996, vol. 24, no. 3, pp. 1106–1125.MathSciNetzbMATHGoogle Scholar
  14. 14.
    Golubev, G.K., On Risk Concentration for Convex Combinations of Linear Estimators, Probl. Peredachi Inf., 2016, vol. 52, no. 4, pp. 31–48 [Probl. Inf. Trans. (Engl. Transl.), 2016, vol. 52, no. 4, pp. 344–358].MathSciNetzbMATHGoogle Scholar

Copyright information

© Pleiades Publishing, Inc. 2018

Authors and Affiliations

  1. 1.Kharkevich Institute for Information Transmission ProblemsRussian Academy of SciencesMoscowRussia
  2. 2.CNRSAix-Marseille Université, I2MMarseilleFrance
  3. 3.Duisburg-Essen UniversityDuisburgGermany

Personalised recommendations