Skip to main content
SpringerLink
Log in

Local Walsh-average-based estimation and variable selection for single-index models

  • Articles
  • Published:
Science China Mathematics Aims and scope Submit manuscript

Abstract

We propose a robust estimation procedure based on local Walsh-average regression (LWR) for single-index models. Our novel method provides a root-n consistent estimate of the single-index parameter under some mild regularity conditions; the estimate of the unknown link function converges at the usual rate for the nonparametric estimation of a univariate covariate. We theoretically demonstrate that the new estimators show significant efficiency gain across a wide spectrum of non-normal error distributions and have almost no loss of efficiency for the normal error. Even in the worst case, the asymptotic relative efficiency (ARE) has a lower bound compared with the least squares (LS) estimates; the lower bounds of the AREs are 0.864 and 0.8896 for the single-index parameter and nonparametric function, respectively. Moreover, the ARE of the proposed LWR-based approach versus the ARE of the LS-based method has an expression that is closely related to the ARE of the signed-rank Wilcoxon test as compared with the t-test. In addition, to obtain a sparse estimate of the single-index parameter, we develop a variable selection procedure by combining the estimation method with smoothly clipped absolute deviation penalty; this procedure is shown to possess the oracle property. We also propose a Bayes information criterion (BIC)-type criterion for selecting the tuning parameter and further prove its ability to consistently identify the true model. We conduct some Monte Carlo simulations and a real data analysis to illustrate the finite sample performance of the proposed methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Carroll R J, Fan J, Gijbels I, et al. Generalized partially linear single-index models. J Amer Statist Assoc, 1997, 92: 477–489

    Article  MathSciNet  Google Scholar 

  2. Fan J, Li R. Variable selection via nonconcave penalized likelihood and its oracle properties. J Amer Statist Assoc, 2001, 96: 1348–1360

    Article  MathSciNet  Google Scholar 

  3. Fan Y, Härdle W, Wang W, et al. Composite quantile regression for the single-index model. SFB 649 Discussion paper 2013–010. Sonderforschungsbereich 649. Berlin: Humboldt University, 2013, https://doi.org/10.2139/ssrn.2892647

    Google Scholar 

  4. Feng L, Zou C, Wang Z. Local Walsh-average regression. J Multivariate Anal, 2012, 106: 36–48

    Article  MathSciNet  Google Scholar 

  5. Feng L, Zou C, Wang Z. Rank-based inference for the single-index model. Statist Probab Lett, 2012, 82: 535–541

    Article  MathSciNet  Google Scholar 

  6. Feng S, Xue L. Model detection and estimation for single-index varying coefficient model. J Multivariate Anal, 2015, 139: 227–244

    Article  MathSciNet  Google Scholar 

  7. Härdle W, Hall P, Ichimura H. Optimal smoothing in single-index models. Ann Statist, 1993, 21: 157–178

    Article  MathSciNet  Google Scholar 

  8. Härdle W, Stoker T M. Investigating smooth multiple regression by method of average derivatives. J Amer Statist Assoc, 1989, 84: 986–995

    MathSciNet  MATH  Google Scholar 

  9. Hettmansperger T P, McKean J W. Robust Nonparametric Statistical Methods, 2nd ed. New York: Chapman-Hall, 2011

    MATH  Google Scholar 

  10. Hodges J L, Lehmann E L. The efficiency of some nonparametric competitors of the t-test. Ann Math Statist, 1956, 27: 324–335

    Article  MathSciNet  Google Scholar 

  11. Hristache M, Juditski A, Spokoiny V. Direct estimation of the index coefficients in a single-index model. Ann Statist, 2001, 29: 595–623

    Article  MathSciNet  Google Scholar 

  12. Kai B, Li R, Zou H. Local composite quantile regression smoothing: An efficient and safe alternative to local polynomial regression. J R Stat Soc Ser B Stat Methodol, 2010, 72: 49–69

    Article  MathSciNet  Google Scholar 

  13. Kong E, Xia Y. Variable selection for the single-index model. Biometrika, 2007, 94: 217–229

    Article  MathSciNet  Google Scholar 

  14. Kong E, Xia Y. A single-index quantile regression model and its estimation. Econom Theory, 2012, 28: 730–768

    Article  MathSciNet  Google Scholar 

  15. Luo S, Ghosal S. Forward selection and estimation in high dimensional single index models. Stat Methodol, 2016, 33: 172–179

    Article  MathSciNet  Google Scholar 

  16. Neykov M, Liu S, Cai T. Li-regularized least squares for support recovery of high dimensional single index models with Gaussian designs. J Mach Learn Res, 2016, 17: 1–37

    MATH  Google Scholar 

  17. Peng H, Huang T. Penalized least squares for single index models. J Statist Plann Infer, 2011, 141: 1362–1379

    Article  MathSciNet  Google Scholar 

  18. Radchenko P. High dimensional single index models. J Multivariate Anal, 2015, 139: 266–282

    Article  MathSciNet  Google Scholar 

  19. Shang S, Zou C, Wang Z. Local Walsh-average regression for semiparametric varying-coefficient models. Statist Probab Lett, 2012, 82: 1815–1822

    Article  MathSciNet  Google Scholar 

  20. Terpstra J, McKean J. Rank-based analysis of linear models using R. J Statist Soft, 2005, 14: 1–26

    Article  Google Scholar 

  21. Tian G, Wang M, Song L. Variable selection in the high-dimensional continuous generalized linear model with current status data. J Appl Stat, 2014, 41: 467–483

    Article  MathSciNet  Google Scholar 

  22. Wang G, Wang L. Spline estimation and variable selection for single-index prediction models with diverging number of index parameters. J Statist Plann Infer, 2015, 162: 1–19

    Article  MathSciNet  Google Scholar 

  23. Wang H. Bayesian estimation and variable selection for single index models. Comput Statist Data Anal, 2009, 53: 2617–2627

    Article  MathSciNet  Google Scholar 

  24. Wang H, Leng C. Unified lasso estimation via least squares approximation. J Amer Statist Assoc, 2007, 101: 1418–1429

    MATH  Google Scholar 

  25. Wang K. Robust direction identification and variable selection in high dimensional general single-index models. J Korean Statist Soc, 2015, 44: 606–618

    Article  MathSciNet  Google Scholar 

  26. Wang L. Wilcoxon-type generalized Bayesian information criterion. Biometrika, 2009, 96: 163–173

    Article  MathSciNet  Google Scholar 

  27. Wang L, Kai B, Li R. Local rank inference for varying coefficient models. J Amer Statist Assoc, 2009, 488: 1631–1645

    Article  MathSciNet  Google Scholar 

  28. Wang M, Song L, Tian G. SCAD-penalized least absolute deviation regression in high dimensional models. Comm Statist Theory Meth, 2015, 44: 2452–2472

    Article  MathSciNet  Google Scholar 

  29. Wang M, Song L, Wang X. Quadratic approximation via the SCAD penalty with a diverging number of parameters. Comm Statist Simulation Comput, 2016, 45: 1–16

    Article  MathSciNet  Google Scholar 

  30. Wang M, Tian G. Robust group non-convex estimations for high-dimensional partially linear models. J Nonparametr Stat, 2016, 28: 49–67

    Article  MathSciNet  Google Scholar 

  31. Wang M, Wang X. Adaptive LASSO estimators for ultrahigh dimensional generalized linear models. Statist Probab Lett, 2014, 89: 41–50

    Article  MathSciNet  Google Scholar 

  32. Wang T, Xu P, Zhu L. Non-convex penalized estimation in high-dimensional models with single-index structure. J Multivariate Anal, 2012, 109: 221–235

    Article  MathSciNet  Google Scholar 

  33. Wang T, Zhu L. A distribution-based LASSO for a general single-index model. Sci China Math, 2015, 58: 109–130

    Article  MathSciNet  Google Scholar 

  34. Wang X, Zhao S, Wang M. Restricted profile estimation for partially linear models with large-dimensional covariates. Statist Probab Lett, 2017, 128: 71–76

    Article  MathSciNet  Google Scholar 

  35. Wu T Z, Yu K, Yu Y. Single-index quantile regression. J Multivariate Anal, 2010, 101: 1607–1621

    Article  MathSciNet  Google Scholar 

  36. Xia Y. A constructive approach to the estimation of dimension reduction directions. Ann Statist, 2007, 35: 2654–2690

    Article  MathSciNet  Google Scholar 

  37. Xia Y, Härdle W. Semi-parametric estimation of partially linear single-index models. J Multivariate Anal, 2006, 97: 1162–1184

    Article  MathSciNet  Google Scholar 

  38. Xia Y, Tong H, Li W K, et al. An adaptive estimation of dimension reduction space. J R Stat Soc Ser B Stat Methodol, 2002, 64: 363–410

    Article  MathSciNet  Google Scholar 

  39. Zeng P, He T, Zhu Y. An LASSO-type approach for estimation and variable selection in single index models. J Comput Graph Statist, 2012, 21: 92–109

    Article  MathSciNet  Google Scholar 

  40. Zhu L, Huang M, Li R. Semiparametric quantile regression with high-dimensional covariates. Statist Sinica, 2012, 22: 1379–1401

    MathSciNet  MATH  Google Scholar 

  41. Zhu L, Qian L, Lin J. Variable selection in a class of single-index models. Ann Inst Statist Math, 2011, 63: 1277–1293

    Article  MathSciNet  Google Scholar 

  42. Zhu L, Zhu L. Nonconcave penalized inverse regression in single-index models with high dimensional predictors. J Multivariate Anal, 2009, 100: 862–875

    Article  MathSciNet  Google Scholar 

  43. Zou H, Li R. One-step sparse estimates in nonconcave penalized likelihood models. Ann Statist, 2008, 36: 1509–1533

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgements

This work was partially supported by National Natural Science Foundation of China (Grant Nos. 11801168, 11801169, 11571055 and 11671059) and the Natural Science Foundation of Hunan Province (Grant No. 2018JJ3322). The authors are grateful to the two anonymous referees whose comments led to a significant improvement of the paper.

Author information

Authors and Affiliations

  1. Key Laboratory of High Performance Computing and Stochastic Information Processing (Ministry of Education of China), College of Mathematics and Computer Science, Hunan Normal University, Changsha, 410081, China

    Jing Yang & Fang Lu

  2. College of Mathematics and Statistics, Chongqing University, Chongqing, 401331, China

    Hu Yang

Authors
  1. Jing Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Fang Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hu Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jing Yang.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yang, J., Lu, F. & Yang, H. Local Walsh-average-based estimation and variable selection for single-index models. Sci. China Math. 62, 1977–1996 (2019). https://doi.org/10.1007/s11425-017-9262-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11425-017-9262-3

Keywords

MSC(2010)

Search

Navigation