Abstract
We study the prediction with expert advice problem, where in each round, the player selects one of N actions and incurs the corresponding loss according to an N-dimensional linear loss vector, and aim to minimize the regret. In this paper, we consider a new measure of the loss functions, which we call L ∞ -variation. Consider the loss functions with small L ∞ -variation, if the player is allowed to have some information related to the variation in each round, we can obtain an online bandit algorithm for the problem without using the self-concordance methodology, which conditionally answers an open problem in [8]. Another related problem is the combinatorial prediction game, in which the set of actions is a subset of {0,1}d, and the loss function is in [–1,1]d. We provide an online algorithm in the semi-bandit setting when the loss functions have small L ∞ -variation.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Abernethy, J., Hazan, E., Rakhlin, A.: Competing in the dark: An efficient algorithm for bandit linear optimization. In: COLT, pp. 263–274 (2008)
Audibert, J.-Y., Bubeck, S.: Regret Bounds and Minimax Policies under Partial Monitoring. Journal of Machine Learning Research 11, 2635–2686 (2010)
Audibert, J.-Y., Bubeck, S., Lugosi, G.: Minimax Policies for Combinatorial Prediction Games. In: COLT, pp. 107–132 (2011)
Chiang, C.-K., Yang, T., Lee, C.-J., Mahdavi, M., Lu, C.-J., Jin, R., Zhu, S.: Online optimization with gradual variations. In: COLT, pp. 6.1–6.20 (2012)
Dani, V., Hayes, T., Kakade, S.M.: The Price of Bandit Information for Online Optimization. In: NIPS, pp. 345–352 (2008)
Freund, Y., Schapire, R.E.: A Decision-Theoretic Generalization of On-Line Learning and an Application to Boosting. J. Comput. Syst. Sci. 55(1), 119–139 (1997)
Hazan, E., Kale, S.: Extracting certainty from uncertainty: Regret bounded by variation in costs. Machine Learning 80(2-3), 165–188 (2010)
Hazan, E., Kale, S.: Better Algorithms for Benign Bandits. Journal of Machine Learning Research 12, 1287–1311 (2011)
Littlestone, N., Warmuth, M.K.: The Weighted Majority Algorithm. Inf. Comput. 108(2), 212–261 (1994)
Vitter, J.S.: Random sampling with a reservoir. ACM Trans. Math. Softw. 11(1), 37–57 (1985)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Lee, CJ., Tsai, SC., Yang, MC. (2014). Online Prediction Problems with Variation. In: Cai, Z., Zelikovsky, A., Bourgeois, A. (eds) Computing and Combinatorics. COCOON 2014. Lecture Notes in Computer Science, vol 8591. Springer, Cham. https://doi.org/10.1007/978-3-319-08783-2_5
Download citation
DOI: https://doi.org/10.1007/978-3-319-08783-2_5
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-08782-5
Online ISBN: 978-3-319-08783-2
eBook Packages: Computer ScienceComputer Science (R0)