Markov Chain Solution to the 3-Tower Problem
The 3-tower problem is a 3-player gambler’s ruin model where two players are involved in a zero information, even-money bet during each round. The probabilities that each player accumulates all the money has a trivial solution. However, the probability of each player getting ruined first is an open problem. In this paper, the 3-tower problem recursions are modeled as a directed multigraph with loops, which is used to construct a Markov chain. The solution leads to exact values, and results show that, unlike in other models where the first ruin probabilities depend only on the proportion of chips of each player, the probabilities obtained by this model depend on the number of chips each player holds.
KeywordsMarkov chains Graph theory Discrete mathematics 3-dimensional gambler’s ruin Applied probability Tower of Hanoi
In the 3-tower model, the probability of finishing 1st is easily solved by recursion. The result is exactly the same as (3). A method of calculating 3rd place probabilities (and hence, 2nd place probabilities) for the 3-tower problem is presented here using Markov chains constructed using directed multigraphs with loops.
Consider a 3-player model where the players are involved in even money bets. We define a state as an ordered triple (x, y, z) with \(x \ge y \ge z \ge 0\). Because the games in each round are all fair and random, the probabilities of becoming ruined first of each player depends only on the amount of money each player has in a given state. We also define chip position (or simply, position) to be the number of money (or chips) a player has in a given state. Let us also define the function p(u|v, w) to be the probability that a player with a given chip position u in the state (u, v, w) will finish 3rd, or become ruined first. If the state is understood from context, we will simply write this as p(u). If v and w are positions in the same state such that \(v = w\), then it is assumed that \(p(v) = p(w)\).
A terminal state is one wherein the probabilities of placing 3rd are known. There are two types of terminal states.
If one of the three positions is zero, i.e. \(z = 0\)
If all three positions are equal, i.e. \(x = y = z\).
Note that for the terminal state (x, y, 0), then \(p(x) = p(y) = 0\) and \(p(0) = 1\). For the terminal state (x, x, x), \(p(x) = 1/3\) using the previous assumption.
A state (x, y, z) is adjacent to a state (u, v, w) if the former state can move to the latter state in one round. A state that is adjacent to a terminal state is called a near-terminal state.
\(z = 1\)
\(x = y + 1\) and \(z = y - 1\).
In constructing the multigraph, all possible states of S are represented by nodes. The transitions between adjacent states are given by directed edges. The states are arranged such that all states (x, y, z) with a fixed value of z are aligned vertically, with the highest value of x in the topmost position, in decreasing order going down (i.e. from North to South), while y is increasing at the same time. All states (x, y, z) with fixed x are aligned horizontally, with y in decreasing order from left to right (i.e. West to East) and z increasing at the same time. Consequently, all states with fixed y are aligned diagonally, with x decreasing and z increasing as the states move from Northwest to Southeast. An example of the resulting multigraph is given in Figs. 1 and 2. From the construction, it is clear that for a given node, its adjacent nodes are the ones located to its immediate top, bottom, left, right, top left and bottom right positions (i.e. North, South, East, West, Northwest and Southeast). A state may be adjacent to itself if the following holds:
A state (x, y, z) is adjacent to itself if \(y = x - 1\) and/or \(z = y - 1\).
In the “and” case in Lemma 2, the state is doubly adjacent to itself. The state is also doubly adjacent to itself for states of the form \((x, x, x-1)\) and \((x, x-1, x-1)\). This and the following Lemma can be proved using the definition of adjacent nodes and (1) with \(b = 1\).
A state of the form (x, y, y) or (x, x, z) is doubly adjacent to its adjacent nodes.
There is always at least one edge from a non-terminal state to its adjacent state. A state that is doubly adjacent to another state has two edges going to that other state. If a state A is doubly adjacent to a non-terminal state B, it does not follow that B is doubly adjacent to A.
3 Results and Discussion
Based on the multigraph, we construct the Markov chain. Let \(\tau \) be a relation that maps a position from one state to a position in another non-terminal state. We can think of \(\tau \) as directed edges that connect specific positions within states to other positions in other states (or possibly within the same state). Let the function \(\phi \) denote the ruin probability of the position in one move. Note that the unique non-terminal positions are exactly the transient states in the Markov chain, while \(\phi \) gives the probability of absorption to the first-ruined state. The Theorem below follows from the previous Lemmas.
Let the state corresponding to a non-terminal vertex be denoted by (x, y, z) where \(x \ge y \ge z\) such that \(z \ge 1\). Let \(S = x + y + z\) and let (u, v, w) be an adjacent non-terminal vertex.
If \(z = y - 1\), then \(\tau (x) \rightarrow u, \tau (y) \rightarrow w, \tau (z) \rightarrow v\)
If \(y = x - 1\), then \(\tau (x) \rightarrow v, \tau (y) \rightarrow u, \tau (z) \rightarrow w\)
If \(z = y\), then \(\tau (x) \rightarrow u\) twice, \(\tau (y) \rightarrow v\) and \(\tau (y) \rightarrow w\)
If \(y = x\), then \(\tau (z) \rightarrow w\) twice, \(\tau (x) \rightarrow u\) and \(\tau (x) \rightarrow v\).
In all other cases, the transitions are \(\tau (x) \rightarrow u\), \(\tau (y) \rightarrow v\) and \(\tau (z) \rightarrow w\).
\(\tau (x) \rightarrow x, \tau (y) \rightarrow z, \tau (z) \rightarrow y\)
\(\tau (x) \rightarrow y, \tau (y) \rightarrow x, \tau (z) \rightarrow z\).
Each mapping of positions by \(\tau \) gives a transition probability of 1 / 6, except when the mapping occurs twice, as in the 3rd and 4th cases in the Theorem, then the transition probability is 2 / 6. These values are then used to generate the transient matrix \(\mathbf Q \) in the Markov chain. For the absorption probabilities, we use the following:
Given a position u, then \(\phi (u) = 1/3\) when \(u = 1\). If \(u > 1\), then \(\phi (u) = 0\), the only exception is for the state \((u+1, u, u-1)\), wherein \(\phi (u+1) = \phi (u) = \phi (u-1) = 1/18\).
Probabilities of placing 3rd for given states (x, y, z), where x : y : z are in the ratio 3 : 2 : 1. The ICM values are shown for comparison.
(x, y, z)
(3, 2, 1)
(6, 4, 2)
(12, 8, 4)
(24, 16, 8)
(48, 32, 16)
(96, 64, 32)
In this paper, a method of solving players’ first-ruin probabilities in the 3-tower problem or gambler’s ruin with three players was presented. The assumptions were that each player started with some nonzero number of chips and during each round, two players were randomly selected in an even-money bet with a randomly chosen winner winning one chip from the loser. One application of this is computing equities in a partial information game (e.g. poker tournaments) modeled as a random game. A multigraph of the various states given S total chips was constructed. The method specified how to obtain the state transitions and absorption probabilities, as given by Theorems 1 and 2. The resulting linear system of the Markov chain were then used to solve the 3rd place (and thus 2nd place) probabilities of any state in S. Although a closed form formula was not derived for the probabilities, the method produces exact solutions instead of numerical approximations. This made it possible to show subtle differences in probabilities of first ruin as S was increased, while preserving the relative chip ratios. In contrast, other methods, such as ICM or Brownian models, only depend on the proportion of chips each player has, thus are independent of any scaling factor. The calculated results were similar to previous numerical approximations using Brownian motion, but differed from ICM by up to \(15\,\%\), although as mentioned, ICM and the 3-tower problem do not use the same assumptions.
The 3-tower model may be extended to one wherein bet sizes are not fixed. The multigraph form of such a model would be much more complex because the number of edges and adjacent nodes is not limited to six. The model may also be applied to other forms of the three-player gambler’s ruin such as player-centric and symmetric games. An extension to an N-tower problem may be done but the increase in complexity of the graph is expected to be significant.
This project was supported by the National Sciences Research Institute of the University of the Philippines, Project Reference No. 2008.149. A special thanks to Ramon Marfil of the Institute of Mathematics, University of the Philippines, for his assistance in the project.
- 3.Ferguson, T.S.: Gambler’s Ruin in Three Dimensions (1995). http://www.math.ucla.edu/gamblers
- 4.Finch, S.: Gambler’s Ruin (2008). http://www.people.fas.harvard.edu/sfinch/csolve/ruin.pdf
- 6.Kim, M.S.: Gambler’s ruin in many dimensions and optimal strategy in repeated multi-player games with application to poker. Master’s Thesis, UCLA (2005)Google Scholar
- 7.Roberts, B.: Ben Roberts ICM Model (2012). http://www.pokericmcalculator.com/en-us/articles/ben-roberts-icm-model