Mathematical Methods of Statistics

, Volume 26, Issue 2, pp 81–110 | Cite as

Moment convergence in regularized estimation under multiple and mixed-rates asymptotics

  • H. Masuda
  • Y. Shimizu


In M-estimation under standard asymptotics, the weak convergence combined with the polynomial type large deviation estimate of the associated statistical random field Yoshida (2011) provides us with not only the asymptotic distribution of the associated M-estimator but also the convergence of its moments, the latter playing an important role in theoretical statistics. In this paper, we study the above program for statistical random fields of multiple and also possibly mixedrates type in the sense of Radchenko (2008) where the associated statistical random fields may be nondifferentiable and may fail to be locally asymptotically quadratic. Consequently, a very strong mode of convergence of a wide range of regularized M-estimators is ensured.Our results are applied to regularized estimation of an ergodic diffusion observed at high frequency.


regularized estimation moment convergence large deviation inequality sparse estimation stochastic differential equation mixed-rates asymptotics 

2000 Mathematics Subject Classification



Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    G. Afendras and M. Markatou, “Optimality of Training/Test Size and Resampling Effectiveness of Cross- Validation Estimators of the Generalization Error”, arXiv:1511.02980 (2015).Google Scholar
  2. 2.
    G. Afendras and M. Markatou, “Uniform Integrability of the OLS Estimators, and the Convergence of Their Moments”, arXiv:1511.02962 (2015).Google Scholar
  3. 3.
    H. Akaike, “A New Look at the Statistical Model Identification”, IEEE Trans. on Autom, Control 19 (6), 716–723 (1974).MathSciNetCrossRefMATHGoogle Scholar
  4. 4.
    B. Antoine and E. Renault, “EfficientMinimum Distance Estimation with Multiple Rates of Convergence”, J. Econometrics 170 (2), 350–367 (2012).MathSciNetCrossRefMATHGoogle Scholar
  5. 5.
    P. J. Bickel and B. Li, “Regularization in Statistics”, Test 15 (2), 271–344 (2006). With comments and a rejoinder by the authors.MathSciNetCrossRefMATHGoogle Scholar
  6. 6.
    K. P. Burnham and D. R. Anderson, Model Selection and Multimodel Inference. (Springer-Verlag, New York, 2002), 2nd ed.MATHGoogle Scholar
  7. 7.
    N. H. Chan and C.-K. Ing, “Uniform Moment Bounds of Fisher’s Information with Applications to Time Series”, Ann. Statist. 39 (3), 1526–1550 (2011).MathSciNetCrossRefMATHGoogle Scholar
  8. 8.
    A. DeGregorio and S. M. Iacus, “Adaptive LASSO-Type Estimation for Multivariate Diffusion Processes”, Econometric Theory 28 (4), 838–860 (2012).MathSciNetCrossRefMATHGoogle Scholar
  9. 9.
    J. Fan and R. Li, “Variable Selection via Nonconcave Penalized Likelihood and Its Oracle Properties”, J. Amer. Statist. Assoc. 96 (456), 1348–1360 (2001).MathSciNetCrossRefMATHGoogle Scholar
  10. 10.
    D. F. Findley and C.-Z. Wei, “AIC, Overfitting Principles, and the Boundedness of Moments of Inverse Matrices for Vector Autoregressions and Related Models”, J. Multivar. Anal. 83 (2), 415–450 (2002).MathSciNetCrossRefMATHGoogle Scholar
  11. 11.
    V. Genon-Catalot and J. Jacod, “On the Estimation of the Diffusion Coefficient for Multidimensional Diffusion Processes”, Ann. Inst. H. PoincaréProbab. Statist. 29 (1), 119–151 (1993).MATHGoogle Scholar
  12. 12.
    T. Hastie, R. Tibshirani, and J. Friedman, The Elements of Statistical Learning, in Springer Ser. in Statist. (Springer, New York, 2009), 2nd ed.CrossRefGoogle Scholar
  13. 13.
    A. E. Hoerl and R. W. Kennard, “Ridge Regression: Biased Estimation for Nonorthogonal Problems”, Technometrics 12 (1), 55–67 (1970).CrossRefMATHGoogle Scholar
  14. 14.
    C.-K. Ing and C.-Y. Yang, “Predictor Selection for Positive Autoregressive Processes”, J. Amer. Statist. Assoc. 109 (505), 243–253 (2014).MathSciNetCrossRefGoogle Scholar
  15. 15.
    J. Jacod and P. Protter, Discretization of Processes, in Stoch. Modelling and Appl. Probab. (Springer, Heidelberg, 2012), Vol. 67.Google Scholar
  16. 16.
    M. Kessler, “Estimation of an Ergodic Diffusion From Discrete Observations”, Scand. J. Statist. 24 (2), 211–229 (1997).MathSciNetCrossRefMATHGoogle Scholar
  17. 17.
    K. Knight and W. Fu, “Asymptotics for Lasso-Type Estimators”, Ann. Statist. 28 (5), 1356–1378 (2000).MathSciNetCrossRefMATHGoogle Scholar
  18. 18.
    H. Masuda, “Simple Estimators for Parametric Markovian Trend of Ergodic Processes Based on Sampled Data”, J. Japan Statist. Soc. 35 (2), 147–170 (2005).MathSciNetCrossRefGoogle Scholar
  19. 19.
    H. Masuda, “Parametric Estimation of Lévy Processes”, in Lecture Notes in Math., Vol. 2128: Lévy matters. IV (Springer, Cham, 2015), pp. 179–286.Google Scholar
  20. 20.
    H. Masuda and Y. Shimizu, “Moment Convergence in Regularized Estimations”, arXiv:1406.6751v2 (2014).Google Scholar
  21. 21.
    Y. Ninomiya and S. Kawano, “AIC for the Lasso in Generalized Linear Models”, Electron. J. Statist. 10, 2537–2560 (2016).MathSciNetCrossRefMATHGoogle Scholar
  22. 22.
    B. M. Pötscher and H. Leeb, “On the Distribution of Penalized Maximum Likelihood Estimators: The LASSO SCAD and Thresholding”, J. Multivar. Anal. 100 (9), 2065–2082 (2009).MathSciNetCrossRefMATHGoogle Scholar
  23. 23.
    P. Radchenko, “Mixed-Rates Asymptotics”, Ann. Statist. 36 (1), 287–309 (2008).MathSciNetCrossRefMATHGoogle Scholar
  24. 24.
    Y. Sakamoto and N. Yoshida, “Asymptotic Expansion Formulas for Functionals of ϵ-Markov Processes with a Mixing Property”, Ann. Inst. Statist. Math. 56 (3), 545–597 (2004).MathSciNetCrossRefMATHGoogle Scholar
  25. 25.
    Y. Shimizu, “Moment Convergence of Regularized Least-Squares Estimator for Linear Regression Model”, Ann. Inst. Statist. Math. (2016).Google Scholar
  26. 26.
    M. Uchida and N. Yoshida, “Information Criteria in Model Selection for Mixing Processes”, Statist. Inference Stochast. Process. 4 (1), 73–98 (2001).MathSciNetCrossRefMATHGoogle Scholar
  27. 27.
    M. Uchida and N. Yoshida, “Asymptotic Expansion and Information Criteria”, SUT J. Math. 42 (1), 31–58 (2006).MathSciNetMATHGoogle Scholar
  28. 28.
    M. Uchida and N. Yoshida, “Adaptive Estimation of an Ergodic Diffusion Process Based on Sampled Data”, Stochast. Process. Appl. 122 (8), 2885–2924 (2012).MathSciNetCrossRefMATHGoogle Scholar
  29. 29.
    Y. Umezu, Y. Shimizu, H. Masuda, and Y. Ninomiya, “AIC for Nonconcave Penalized Likelihood Method”, arXiv:1509.01688v2 2015.Google Scholar
  30. 30.
    A. W. van der Vaart, Asymptotic Statistics, in Cambridge Ser. in Statist. and Probab. Math. (Cambridge Univ. Press, Cambridge, 1998), Vol. 3.Google Scholar
  31. 31.
    H. Wang, G. Li, and G. Jiang, “Robust Regression Shrinkage and Consistent Variable Selection through the LAD-Lasso”, J. Bus. Econom. Statist. 25 (3), 347–355 (2007).MathSciNetCrossRefGoogle Scholar
  32. 32.
    N. Yoshida, “Estimation for Diffusion Processes from Discrete Observation”, J. Multivar. Anal. 41 (2), 220–242 (1992).MathSciNetCrossRefMATHGoogle Scholar
  33. 33.
    N. Yoshida, “Polynomial Type Large Deviation Inequalities and Quasi-Likelihood Analysis for Stochastic Differential Equations”, Ann. Inst. Statist. Math. 63 (3), 431–479 (2011).MathSciNetCrossRefMATHGoogle Scholar

Copyright information

© Allerton Press, Inc. 2017

Authors and Affiliations

  1. 1.Faculty of Math.Kyushu Univ.FukuokaJapan
  2. 2.Dept.Math.Josai Univ.SakadoJapan

Personalised recommendations