Abstract
In many practical applications, the structure of covariance matrix is often blurred due to random errors, making the estimation of covariance matrix very difficult particularly for high-dimensional data. In this article, we propose a regularization method for finding a possible banded Toeplitz structure for a given covariance matrix A (e.g., sample covariance matrix), which is usually an estimator of the unknown population covariance matrix Σ. We aim to find a matrix, say B, which is of banded Toeplitz structure, such that the Frobenius-norm discrepancy between B and A achieves the smallest in the whole class of banded Toeplitz structure matrices. As a result, the obtained Toeplitz structured matrix B recoveries the underlying structure behind Σ. Our simulation studies show that B is also more accurate than the sample covariance matrix A when estimating the covariance matrix Σ that has a banded Toeplitz structure. The studies also show that the proposed method works very well in regularization of covariance structure.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Belloni, A., Chernozhukov, V., & Wang, L. (2012). Square-root lasso: Pivotal recovery of sparse signals via conic programming. Biometrika, 98, 791–806.
Bickel, P., & Levina, E. (2008). Covariance regularization by thresholding. Annals of Statistics, 36, 2577–2604.
Burg, J. P., Luenberger, D. G., & Wenger, D. L. (1982). Estimation of structured covariance matrices. Proceedings of the IEEE, 70(9), 963–974.
Cai, T., & Liu, W. (2011). Adaptive thresholding for sparse covariance estimation. Journal of the American Statistical Association, 106, 672–684.
Chakraborty, M. (1998). An efficient algorithm for solving general periodic Toeplitz systems. IEEE Transactions on Signal Processing, 46(3), 784–787.
Cui, X., Li, C., Zhao, J., Zeng, L., Zhang, D. & Pan, J. (2016). Regularization for high-dimensional covariance matrix. Special Matrices, 4, 189–201.
Cui, X., Li, C., Zhao, J., Zeng, L., Zhang, D., & Pan, J. (2016). Covariance structure regularization via Frobenius-norm discrepancy. Linear Algebra and Its Applications, 510, 124–145.
Deng, X., & Tsui, K. (2013). Penalized covariance matrix estimation using a matrix-logarithm transformation. Journal of Computational and Graphical Statistics, 22, 494–512.
El Karoui, N. (2008). Operator norm consistent estimation of large dimensional sparse covariance matrices. Annals of Statistics, 36, 2712–2756.
Fan, J., Liao, Y., & Mincheva, M. (2013). Large covariance estimation by thresholding principal orthogonal complements. Journal of Royal Statistical Society Series B, 75, 656–658.
Johnstone, I. M. (2001). On the distribution of the largest eigenvalue in principal components analysis. Annals of Statistics, 29, 295–327.
Lin, L., Higham, N. J., & Pan, J. (2014). Covariance structure regularization via entropy loss function. Computational Statistics & Data Analysis, 72, 315–327.
Moura, J. M. F., & Balram, N. (1992). Recursive structure of noncausal gauss-markov random fields. IEEE Transactions on Information Theory, 38(2), 334–354.
Pan, J., & Mackenzie, G. (2003). On modelling mean-covariance structures in longitudinal studies. Biometrika, 90(1), 239–244.
Parter, S. V. (1962). An observation on the numerical solution of difference equations and a theorem of Szegö. Numerische Mathematik, 4, 293–295.
Pourahmadi, M. (1999). Joint mean-covariance models with applications to longitudinal data: Unconstrained parameterisation. Biometrika, 86(3), 677–690.
Rothman, A. (2012). Positive definite estimators of large covariance matrices. Biometrika, 99, 733–740.
Rothman, A., Levina, E., & Zhu, J. (2009). Generalized thresholding of large covariance matrices. Journal of the American Statistical Association, 104, 177–186.
Woods, J. W. (1972). Two-dimensional discrete markovian fields. IEEE Transactions on Information Theory, 18(2), 232–240.
Xue, L., Ma, S., & Zou, H. (2012). Positive definite ℓ 1 penalized estimation of large covariance matrices (2012). Journal of the American Statistical Association, 107, 1480–1491.
Acknowledgements
We gratefully acknowledge very constructive comments and suggestions by the Editors and one anonymous reviewer, which lead to significant improvements to the paper.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Cui, X., Li, Z., Zhao, J., Zhang, D., Pan, J. (2019). Covariance Matrix Regularization for Banded Toeplitz Structure via Frobenius-Norm Discrepancy. In: Ahmed, S., Carvalho, F., Puntanen, S. (eds) Matrices, Statistics and Big Data. IWMS 2016. Contributions to Statistics. Springer, Cham. https://doi.org/10.1007/978-3-030-17519-1_9
Download citation
DOI: https://doi.org/10.1007/978-3-030-17519-1_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-17518-4
Online ISBN: 978-3-030-17519-1
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)