# Sparse Kalman filtering approaches to realized covariance estimation from high frequency financial data

- 151 Downloads

## Abstract

Estimation of the covariance matrix of asset returns from high frequency data is complicated by asynchronous returns, market microstructure noise and jumps. One technique for addressing both asynchronous returns and market microstructure is the Kalman-Expectation-Maximization (KEM) algorithm. However the KEM approach assumes log-normal prices and does not address jumps in the return process which can corrupt estimation of the covariance matrix. In this paper we extend the KEM algorithm to price models that include jumps. We propose a sparse Kalman filtering approach to this problem. In particular we develop a Kalman Expectation Conditional Maximization algorithm to determine the unknown covariance as well as detecting the jumps. In order to promote a sparse estimate of the jumps,we consider both Laplace and the spike and slab jump priors. Numerical results using simulated data show that each of these approaches provide for improved covariance estimation relative to the KEM method in a variety of settings where jumps occur.

## Keywords

Spike and slab ECM Kalman filtering \(\ell _{1}\) regularization## Mathematics Subject Classification

90C26 62P05## Notes

## Supplementary material

## References

- 1.Aravkin, A., Bell, B., Burke, J., Pilonetto, G.: An \(\ell _{1}\) Laplace robust Kalman smoother. IEEE Trans. Autom. Control
**56**(12), 2898–2911 (2011)MathSciNetCrossRefzbMATHGoogle Scholar - 2.Aït-Sahalia, Y., Fan, J., Xiu, D.: High-frequency covariance estimates with noisy and asynchronous financial data. J. Am. Stat. Assoc.
**105**(492), 1504–1517 (2010)MathSciNetCrossRefzbMATHGoogle Scholar - 3.Aït-Sahalia, Y., Myklank, P., Zhang, L.: How often to sample a continuous-time process in the presence of market microstructure noise. Rev. Financ. Stud.
**100**, 1394–1411 (2005)Google Scholar - 4.Bandi, F., Russell, J.: Separating microstructure noise from volatility. J. Financ. Econ.
**79**, 655–692 (2006)CrossRefGoogle Scholar - 5.Barndorff-Nielsen, O., Hansen, P., Lunde, A., Shephard, N.: Multivariate realised kernels: consistent positive semi-definite estimators of the covariation of equity prices with noise and non-synchronous trading. J. Econom.
**162**, 149–169 (2011)MathSciNetCrossRefzbMATHGoogle Scholar - 6.Barry, C.B.: Portfolio analysis under uncertain means, variances and covariances. J. Finance
**29**, 515–522 (1974)CrossRefGoogle Scholar - 7.Beck, A., Teboulle, M.: A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM Imaging Sci.
**2**(1), 183–202 (2009)MathSciNetCrossRefzbMATHGoogle Scholar - 8.Bollerslev, T.: Modeling the coherence in short-run nominal exchange rates: a multivariate generalized ARCH approach. Rev. Econ. Stat.
**72**, 498–505 (1990)CrossRefGoogle Scholar - 9.Boudt, K., Croux, C., Laurent, S.: Outlyingness weighted covariation. J. Financ. Econom.
**9**(4), 657–684 (2011)CrossRefGoogle Scholar - 10.Boudt, K., Zhang, J.: Jump robust two time scale covariance estimation and realized volatility budgets. Quant. Finance
**15**(6), 1041–1054 (2015)MathSciNetCrossRefzbMATHGoogle Scholar - 11.Boyd, S., Parikh, N., Chu, E., Peleato, B., Eckstein, J.: Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends Mach. Learn.
**3**(1), 1–122 (2011)CrossRefzbMATHGoogle Scholar - 12.Campbell, J., Lo, A., MacKinlay, A.C.: The Econometrics of Financial Markets. Princeton University Press, Princeton (1996)zbMATHGoogle Scholar
- 13.Candès, E., Wakin, M., Boyd, S.: Enhancing sparsity by reweighted \(\ell _{1}\) minimzation. J. Fourier Anal. Appl.
**14**, 877–905 (2008)MathSciNetCrossRefzbMATHGoogle Scholar - 14.Chan, W., Maheu, J.: Conditional jump dynamics in stock market returns. J. Bus. Econ. Stat.
**20**(3), 377–389 (2002)MathSciNetCrossRefGoogle Scholar - 15.Corsi, F., Peluso, S., Audrino, F.: Missing in asynchronicity: a Kalman-EM approach for multivariate realized covariance estimation. J. Appl. Econom.
**30**(3), 377–397 (2015)MathSciNetCrossRefGoogle Scholar - 16.DeMiguel, V., Garlappi, L., Uppal, R.: Optimal versus Naive diversification: how inefficient is the 1/n portfolio strategy? Rev. Financ. Stud.
**22**(5), 1915–1953 (2009)CrossRefGoogle Scholar - 17.Dempster, A., Laird, N., Rubin, D.: Maximum likelihood from incomplete data via the EM algorithm. J. R. Stat. Soc. Ser. B
**39**(1), 1–38 (1977)MathSciNetzbMATHGoogle Scholar - 18.Fan, J., Li, Y., Yu, K.: Vast volatility matrix estimation using high frequency data for portfolio selection. J. Am. Stat. Assoc.
**107**(497), 412–428 (2012)MathSciNetCrossRefzbMATHGoogle Scholar - 19.Fan, J., Wang, Y.: Multi-scale jump and volatility analysis for high-frequency financial data. J. Am. Stat. Assoc.
**102**(480), 1349–1362 (2007)MathSciNetCrossRefzbMATHGoogle Scholar - 20.Fink, D.: A compendium of conjugate priors. Technical Report, Montana State Univeristy (1997)Google Scholar
- 21.Ghahramani, Z., Hinton, G.E.: Variational learning for switching state-space models. Neural Comput.
**12**(4), 963–996 (2000)CrossRefGoogle Scholar - 22.Goldstein, T., Osher, S.: The split Bregman method for \(\ell _{1}\) regularized problems. SIAM J. Imaging Sci.
**2**(2), 323–343 (2009)MathSciNetCrossRefzbMATHGoogle Scholar - 23.Jobson, J.D., Korkie, B.: Estimation for Markowitz efficient portfolios. J. Am. Stat. Assoc.
**75**, 544–554 (1980)MathSciNetCrossRefzbMATHGoogle Scholar - 24.Karpoff, J.: The relation between price changes and trading volume: a survey. J. Financ. Quant. Anal.
**22**, 109–126 (1987)CrossRefGoogle Scholar - 25.Liu, C., Tang, C.Y.: A quasi-maximum likelihood approach for integrated covariance matrix estimation with high frequency data. J. Econom.
**180**, 217–232 (2014)MathSciNetCrossRefzbMATHGoogle Scholar - 26.Lo, A., MacKinlay, A.C.: An econometric analysis of nonsynchronous trading. J. Econom.
**45**, 181–211 (1990)MathSciNetCrossRefzbMATHGoogle Scholar - 27.Lunenberger, D., Ye, Y.: Linear and Nonlinear Programming. Addison-Wesley, New York (2008)CrossRefGoogle Scholar
- 28.Maheu, J.M., McCurdy, T.H.: News arrival, jump dynamics, and volatility components for individual stock returns. J. Finance
**59**(2), 755–793 (2004)CrossRefGoogle Scholar - 29.Mattingley, J., Boyd, S.: Real-time convex optimization in signal processing. IEEE Signal Process. Mag.
**27**, 50–61 (2010)CrossRefzbMATHGoogle Scholar - 30.Meng, X.L., Rubin, D.: Maximum likelihood estimation via the ECM algorithm: a general framework. Biometrika
**80**, 267–278 (1993)MathSciNetCrossRefzbMATHGoogle Scholar - 31.Peluso, S., Corsi, F., Mira, A.: A Bayesian high-frequency estimator of the multivariate covariance of noisy and asynchronous returns. J. Financ. Econom.
**13**(3), 665–697 (2015)CrossRefGoogle Scholar - 32.Roll, R.: A simple implicit measure of the effective bid-ask spread in an efficient market. J. Finance
**39**(4), 1127–1139 (1984)CrossRefGoogle Scholar - 33.Seeger, M.W.: Bayesian inference and optimal design for the sparse linear model. J. Mach. Learn. Res.
**9**, 759–813 (2008)MathSciNetzbMATHGoogle Scholar - 34.Shumway, R., Stoffer, D.: Time Series Analysis and its Applications with R Examples. Springer, Berlin (2011)CrossRefzbMATHGoogle Scholar
- 35.Shumway, R.H., Stoffer, D.S.: An approach to time series smoothing and forecasting using the EM algorithm. J. Time Ser. Anal.
**3**(4), 253–264 (1982)CrossRefzbMATHGoogle Scholar - 36.Wu, C.F.: On the convergence properties of the EM algorithm. Ann. Stat.
**11**(1), 95–103 (1983)MathSciNetCrossRefzbMATHGoogle Scholar - 37.Zangwill, W.: Nonlinear Programming: A Unified Approach. Prentice-Hall, Upper Saddle River (1969)zbMATHGoogle Scholar
- 38.Zhang, L.: Estimating covariation: Epps effect and microstructure noises. J. Econ.
**160**(1), 33–47 (2011)MathSciNetCrossRefzbMATHGoogle Scholar - 39.Zhang, M., Russel, J., Tsay, R.: Determinants of bid and ask quotes and implications for the cost of trading. J. Empir. Finance
**15**(4), 656–678 (2008)CrossRefGoogle Scholar