Advertisement

The de-biased group Lasso estimation for varying coefficient models

  • Toshio HondaEmail author
Article
  • 4 Downloads

Abstract

There has been much attention on the de-biased or de-sparsified Lasso. The Lasso is very useful in high-dimensional settings. However, it is well known that the Lasso produces biased estimators. Therefore, several authors proposed the de-biased Lasso to fix this drawback and carry out statistical inferences based on the de-biased Lasso estimators. The de-biased Lasso needs desirable estimators of high-dimensional precision matrices. Thus, the research is almost limited to linear regression models with some restrictive assumptions, generalized linear models with stringent assumptions, and the like. To our knowledge, there are a few papers on linear regression models with group structure, but no result on structured nonparametric regression models such as varying coefficient models. We apply the de-biased group Lasso to varying coefficient models and examine the theoretical properties and the effects of approximation errors involved in nonparametric regression. The results of numerical studies are also presented.

Keywords

High-dimensional data B-spline Varying coefficient models Group Lasso Bias correction 

Notes

Acknowledgements

The author appreciates comments from the AE and two reviewers very much. He also thanks Akira Shinkyu for research assistance. This research is supported by JSPS KAKENHI Grant Number JP 16K05268.

Supplementary material

10463_2019_740_MOESM1_ESM.pdf (332 kb)
Supplementary material 1 (pdf 332 KB)

References

  1. Bickel, P. J., Ritov, Y., Tsybakov, A. B. (2009). Simultaneous analysis of Lasso and Dantzig selector. Annals of Statistics, 37, 1705–1732.MathSciNetCrossRefGoogle Scholar
  2. Bühlmann, P., van de Geer, S. (2011). Statistics for high-dimensional data: Methods theory and applications. New York: Springer.CrossRefGoogle Scholar
  3. Caner, M., Kock, A. B. (2018). Asymptotically honest confidence regions for high dimensional parameters by the desparsified conservative lasso. Journal of Econometrics, 203, 143–168.MathSciNetCrossRefGoogle Scholar
  4. Cheng, M.-Y., Honda, T., Li, J., Peng, H. (2014). Nonparametric independence screening and structure identification for ultra-high dimensional longitudinal data. Annals of Statistics, 42, 1819–1849.MathSciNetCrossRefGoogle Scholar
  5. Cheng, M.-Y., Honda, T., Zhang, J.-T. (2016). Forward variable selection for sparse ultra-high dimensional varying coefficient models. Journal of the American Statistical Association, 111, 1201–1221.MathSciNetCrossRefGoogle Scholar
  6. Fan, J., Li, R. (2001). Variable selection via nonconcave penalized likelihood and its oracle properties. Journal of the American Statistical Association, 96, 1348–1360.MathSciNetCrossRefGoogle Scholar
  7. Fan, J., Ma, Y., Dai, W. (2014). Nonparametric independence screening in sparse ultra-high dimensional varying coefficient models. Journal of the American Statistical Association, 109, 1270–1284.MathSciNetCrossRefGoogle Scholar
  8. Fan, J., Song, R. (2010). Sure independence screening in generalized linear models with NP-dimensionality. Annals of Statistics, 38, 3567–3604.MathSciNetCrossRefGoogle Scholar
  9. Fan, J., Xue, L., Zou, H. (2014). Strong oracle optimality of folded concave penalized estimation. Annals of Statistics, 42, 819–849.MathSciNetCrossRefGoogle Scholar
  10. Fan, J., Zhang, W. (2008). Statistical methods with varying coefficient models. Statistics and Its Interface, 1, 179–195.MathSciNetCrossRefGoogle Scholar
  11. Greene, W. H. (2012). Econometric analysis 7th ed. Harlow: Pearson Education.Google Scholar
  12. Hastie, T., Tibshirani, R., Wainwright, M. (2015). Statistical learning with sparsity. Boca Raton: CRC Press.CrossRefGoogle Scholar
  13. Honda, T., Härdle, W. K. (2014). Variable selection in Cox regression models with varying coefficients. Journal of Statistical Planning and Inference, 148, 67–81.MathSciNetCrossRefGoogle Scholar
  14. Honda, T., Yabe, R. (2017). Variable selection and structure identification for varying coefficient Cox models. Journal of Multivariate Analysis, 161, 103–122.MathSciNetCrossRefGoogle Scholar
  15. Huang, J. Z., Wu, C. O., Zhou, L. (2004). Polynomial spline estimation and inference for varying coefficient models with longitudinal data. Statistica Sinica, 14, 763–788.MathSciNetzbMATHGoogle Scholar
  16. Ing, C.-K., Lai, T. L. (2011). A stepwise regression method and consistent model selection for high-dimensional sparse linear models. Statistica Sinica, 22, 1473–1513.MathSciNetzbMATHGoogle Scholar
  17. Javanmard, A., Montanari, A. (2014). Confidence intervals and hypothesis testing for high-dimensional regression. Journal of Machine Learning Research, 15, 2869–2909.MathSciNetzbMATHGoogle Scholar
  18. Javanmard, A., Montanari, A. (2018). Debiasing the lasso: Optimal sample size for gaussian designs. Annals of Statistics, 46, 2593–2622.MathSciNetCrossRefGoogle Scholar
  19. Liu, J., Li, R., Wu, R. (2014). Feature selection for varying coefficient models with ultrahigh dimensional covariates. Journal of the American Statistical Association, 109, 266–274.MathSciNetCrossRefGoogle Scholar
  20. Liu, J., Zhong, W., Li, R. (2015). A selective overview of feature screening for ultrahigh-dimensional data. Science China Mathematics, 58, 1–22.MathSciNetCrossRefGoogle Scholar
  21. Lounici, K., van de Pontil, M., Geer, S., Tsybakov, A. B. (2011). Oracle inequalities and optimal inference under group sparsity. Annals of Statistics, 39, 2164–2204.MathSciNetCrossRefGoogle Scholar
  22. Mitra, R., Zhang, C.-H. (2016). The benefit of group sparsity in group inference with de-biased scaled group Lasso. Electronic Journal of Statistics, 10, 1829–1873.MathSciNetCrossRefGoogle Scholar
  23. Schumaker, L. L. (2007). Spline functions: Basic theory 3rd ed. Cambridge: Cambridge University Press.Google Scholar
  24. Stucky, B., van de Geer, S. (2018). Asymptotic confidence regions for high-dimensional structured sparsity. IEEE Transactions on Signal Processing, 66, 2178–2190.MathSciNetCrossRefGoogle Scholar
  25. Tibshirani, R. J. (1996). Regression shrinkage and selection via the Lasso. Journal of the Royal Statistical Society, Series B, 58, 267–288.MathSciNetzbMATHGoogle Scholar
  26. van de Geer, S. (2016). Estimation and testing under sparsity. Dordrecht: Springer.CrossRefGoogle Scholar
  27. van de Geer, S., Bühlmann, P., Ritov, Y., Dezeure, R. (2014). On asymptotically optimal confidence regions and tests for high-dimensional models. Annals of Statistics, 42, 1166–1202.MathSciNetCrossRefGoogle Scholar
  28. Wang, H. (2009). Forward regression for ultra-high dimensional variable screening. Journal of the American Statistical Association, 104, 1512–1524.MathSciNetCrossRefGoogle Scholar
  29. Wei, F., Huang, J., Li, H. (2011). Variable selection and estimation in high-dimensional varying-coefficient models. Statistica Sinica, 21, 1515–1540.MathSciNetCrossRefGoogle Scholar
  30. Yang, Y., Zou, H. (2017). gglasso: Group Lasso penalized learning using a unified BMD algorithm. R Package Version, 1, 4.Google Scholar
  31. Yuan, M., Lin, Y. (2006). Model selection and estimation in regression with grouped variables. Journal of the Royal Statistical Society, Series B, 68, 49–67.MathSciNetCrossRefGoogle Scholar
  32. Zhang, C.-H., Zhang, S. S. (2014). Confidence intervals for low dimensional parameters in high dimensional linear models. Journal of the Royal Statistical Society, Series B, 76, 217–242.MathSciNetCrossRefGoogle Scholar
  33. Zou, H. (2006). The adaptive Lasso and its oracle properties. Journal of the American Statistical Association, 101, 1418–1429.MathSciNetCrossRefGoogle Scholar

Copyright information

© The Institute of Statistical Mathematics, Tokyo 2019

Authors and Affiliations

  1. 1.Graduate School of EconomicsHitotsubashi UniversityKunitachiJapan

Personalised recommendations