Abstract
We consider on-line density estimation with the multivariate Gaussian distribution. In each of a sequence of trials, the learner must posit a mean μ and covariance Σ; the learner then receives an instance x and incurs loss equal to the negative log-likelihood of x under the Gaussian density parameterized by (μ,Σ). We prove bounds on the regret for the follow-the-leader strategy, which amounts to choosing the sample mean and covariance of the previously seen data.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Azoury, K., Warmuth, M.: Relative loss bounds for on-line density estimation with the exponential family of distributions. Journal of Machine Learning 43(3), 211–246 (2001)
Cesa-Bianchi, N., Lugosi, G.: Prediction, Learning, and Games. Cambridge University Press, Cambridge (2006)
Crammer, K.: Online tracking of linear subspaces. 19th Annual Conference on Learning Theory (2006)
Freund, Y.: Predicting a binary sequence almost as well as the optimal biased coin. 9th Annual Conference on Computational Learning Theory (1996)
Hannan, J.: Approximation to Bayes risk in repeated play. In: M. Dresher, A. Tucker, P. Wolfe (Eds.), Contributions to the Theory of Games, vol. III, pp. 97–139 (1957)
Hazan, E., Kalai, A., Kale, S., Agarwal, A.: Logarithmic regret algorithms for online convex optimization. 19th Annual Conference on Learning Theory (2006)
Kalai, A., Vempala, S.: Efficient algorithms for the online decision problem. 16th Annual Conference on Learning Theory (2005)
Shalev-Shwartz, S., Singer, Y.: Convex repeated games and Fenchel duality. Advances in Neural Information Processing Systems 19 (2006)
Takimoto, E., Warmuth, M.: The last-step minimax algorithm. 11th International Conference on Algorithmic Learning Theory (2000a)
Takimoto, E., Warmuth, M.: The minimax strategy for Gaussian density estimation. 13th Annual Conference on Computational Learning Theory (2000b)
Warmuth, M., Kuzmin, D.: Randomized PCA algorithms with regret bounds that are logarithmic in the dimension. Advances in Neural Information Processing Systems 19 (2006)
Zinkevich, M.: Online convex programming and generalized infinitesimal gradient ascent. In: 20th International Conference on Machine Learning (2003)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer Berlin Heidelberg
About this paper
Cite this paper
Dasgupta, S., Hsu, D. (2007). On-Line Estimation with the Multivariate Gaussian Distribution. In: Bshouty, N.H., Gentile, C. (eds) Learning Theory. COLT 2007. Lecture Notes in Computer Science(), vol 4539. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-72927-3_21
Download citation
DOI: https://doi.org/10.1007/978-3-540-72927-3_21
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-72925-9
Online ISBN: 978-3-540-72927-3
eBook Packages: Computer ScienceComputer Science (R0)