Full Information Game with Gains and Losses

Abstract

In the Full Information Game the player sequentially selects one out of K actions. After the player has made his choice, the K payoffs of the actions become known and the player receives the payoff of the action he selected. The Gain-Loss game is the variant of this game, where both gains from [0,1] and losses from [0,1] are possible payoffs. This game has two well studied special cases: the Full Loss game where only losses are allowed, and the Full Gain game where only gains are allowed. For each of these cases the appropriate variant of Freund and Schapire’s algorithm Hedge [7,3] can be used to obtain nearly optimal regrets. Both of these variants have an immediate adaptations to the Full Gain-Loss game. However these solutions are not always optimal.

The first result of this paper is a new variant of algorithm Hedge that achieves a regret of $O(\sqrt{ \ln K} \sqrt{G_j + L_j})$ for the Full Gain-Loss game, where j is the index of one of the actions in the game, G _j , the total gain of j, is the sum of all the positive payoffs that the jth action had in the game, and L _j is the absolute value of the sum of all its negative payoffs. In addition, the new algorithm achieves matches the performance of the known Hedge algorithms in the special cases of gains only and losses only.

The second result is an application of the new algorithm that achieves new upper bounds on the regrets of the original Full Gain game and Full Loss game. The new upper bounds are a function of a new parameter.

The third result is a method for combining online learning algorithms online. This method yields an $O\big(\min \big(\sqrt{L_{opt}\ {\ln K}}\ , \sqrt{(T-L_{opt})\ {\ln K}}\big) \big)$ upper bound on the regret of the the Full Loss game, and an $O\big(\min \big(\sqrt{G_{opt}\ {\ln K}}\ , \sqrt{(T-G_{opt})\ {\ln K}}\big) \big)$ upper bound on the regret of the the Full Gain game.