Skip to main content
SpringerLink
Log in

Bias-Reduced Moment Estimators of Population Spectral Distribution and Their Applications

  • Chapter
  • First Online:
Big and Complex Data Analysis

Part of the book series: Contributions to Statistics ((CONTRIB.STAT.))

Abstract

In this paper, we propose a series of bias-reduced moment estimators for the Population Spectral Distribution (PSD) of large covariance matrices, which are fundamentally important for modern high-dimensional statistics. In addition, we derive the limiting distributions of these moment estimators, which are then adopted to test the order of PSDs. The simulation study demonstrates the desirable performance of the order test in conjunction with the proposed moment estimators for the PSD of large covariance matrices.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 139.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Bai, Z.D., Silverstein, J.W.: CLT for linear spectral statistics of large-dimensional sample covariance matrices. Ann. Probab. 32, 553–605 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  2. Bai, Z. D., Silverstein, J. W. (2010). Spectral Analysis of Large Dimensional Random Matrices (2nd ed.). New York: Springer.

    Book  MATH  Google Scholar 

  3. Bai, Z.D., Chen, J.Q., Yao, J.F.: On estimation of the population spectral distribution from a high-dimensional sample covariance matrix. Aust. N. Z. J. Stat. 52, 423–437 (2010)

    Article  MathSciNet  Google Scholar 

  4. Berthet, Q., Rigollet, P.: Optimal detection of sparse principal components in high dimension. Ann. Statist. 41, 1780–1815 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  5. Birnbaum, A., Johnstone, I.M., Nadler, B., Paul, D.: Minimax bounds for sparse PCA with noisy high-dimensional data. Ann. Statist. 41, 1055–1084 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  6. Cai, T.T., Ma, Z.M., Wu, Y.H.: Sparse PCA: Optimal rates and adaptive estimation. Ann. Statist. 41, 3074–3110 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  7. Chen, J.Q., Delyon, B., Yao, J.F.: On a model selection problem from high-dimensional sample covariance matrices. J. Multivar. Anal. 102, 1388–1398 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  8. Dudoit, S., Fridlyand, J., Speed, T.P.: Comparison of discrimination methods for the classification of tumors using gene expression data. J. Am. Stat. Assoc. 97, 77–87 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  9. El Karoui, N.: Spectrum estimation for large dimensional covariance matrices using random matrix theory. Ann. Statist. 36, 2757–2790 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  10. Fisher, T.J., Sun, X., Gallagher, C.M.: A new test for sphericity of the covariance matrix for high dimensional data. J. Multivar. Anal. 101, 2554–2570 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  11. Johnstone, I.M.: On the distribution of the largest eigenvalue in principal components analysis. Ann. Statist. 29, 295–327 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  12. Li, W.M., Yao, J.F.: A local moment estimator of the spectrum of a large dimensional covariance matrix. Stat. Sin. 24, 919–936 (2014)

    MathSciNet  MATH  Google Scholar 

  13. Li, W.M., Yao, J.F.: On generalized expectation-based estimation of a population spectral distribution from high-dimensional data. Ann. Inst. Stat. Math. 67, 359–373 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  14. Li, W.M., Chen, J.Q., Qin, Y.L., Yao, J.F., Bai, Z.D.: Estimation of the population spectral distribution from a large dimensional sample covariance matrix. J. Stat. Plan. Inference 143, 1887–1897 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  15. Mestre, X.: Improved estimation of eigenvalues and eigenvectors of covariance matrices using their sample estimates. IEEE Trans. Inf. Theory 54, 5113–5129 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  16. Nica, A., Speicher, R.: Lectures on the Combinatorics of Free Probability. Cambridge University Press, New York (2006)

    Book  MATH  Google Scholar 

  17. Pan, G.M., Zhou, W.: Central limit theorem for signal-to-interference ratio of reduced rank linear receiver. Ann. Appl. Probab. 18, 1232–1270 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  18. Qin, Y.L., Li, W.M.: Testing the order of a population spectral distribution for high-dimensional data. Comput. Stat. Data Anal. 95, 75–82 (2016)

    Article  MathSciNet  Google Scholar 

  19. Rao, N.R., Mingo, J.A., Speicher, R., Edelman, A.: Statistical eigen-inference from large Wishart matrices. Ann. Statist. 36, 2850–2885 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  20. Silverstein, J.W.: Strong convergence of the empirical distribution of eigenvalues of large-dimensional random matrices. J. Multivar. Anal. 55, 331–339 (1995)

    Article  MathSciNet  MATH  Google Scholar 

  21. Srivastava, M.S.: Some tests concerning the covariance matrix in high dimensional data. J. Japan Stat. Soc. 35, 251–272 (2005)

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgements

We would like to thank Dr. S. Ejaz Ahmed for organizing this Springer refereed volume. We appreciate his tremendous efforts. Comments by the anonymous referees led to substantial improvement of the manuscript. Yingli Qin’s research is partly supported by Research Incentive Fund grant No. 115953 and Natural Sciences and Engineering Research Council of Canada (NSERC) grant No. RGPIN-2016-03890. Weiming Li’s research is supported by National Natural Science Foundation of China, No. 11401037 and Program for IRTSHUFE.

Author information

Authors and Affiliations

  1. Department of Statistics and Actuarial Science, University of Waterloo, Waterloo, ON, Canada, N2L 3G1

    Yingli Qin

  2. Shanghai University of Finance and Economics, Shanghai, China

    Weiming Li

Authors
  1. Yingli Qin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Weiming Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Weiming Li .

Editor information

Editors and Affiliations

  1. Department of Mathematics & Statistics, Brock University, St. Catherines, Ontario, Canada

    S. Ejaz Ahmed

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this chapter

Cite this chapter

Qin, Y., Li, W. (2017). Bias-Reduced Moment Estimators of Population Spectral Distribution and Their Applications. In: Ahmed, S. (eds) Big and Complex Data Analysis. Contributions to Statistics. Springer, Cham. https://doi.org/10.1007/978-3-319-41573-4_6

Download citation

Publish with us

Policies and ethics

Search

Navigation