# Markov Chain Solution to the 3-Tower Problem

## Abstract

The 3-tower problem is a 3-player gambler’s ruin model where two players are involved in a zero information, even-money bet during each round. The probabilities that each player accumulates all the money has a trivial solution. However, the probability of each player getting ruined first is an open problem. In this paper, the 3-tower problem recursions are modeled as a directed multigraph with loops, which is used to construct a Markov chain. The solution leads to exact values, and results show that, unlike in other models where the first ruin probabilities depend only on the proportion of chips of each player, the probabilities obtained by this model depend on the number of chips each player holds.

## Keywords

Markov chains Graph theory Discrete mathematics 3-dimensional gambler’s ruin Applied probability Tower of Hanoi## 1 Introduction

*n*be the number of players, \(X_i\) be the random variable denoting the placing of Player

*i*, and \(S_i\) be the current stack size of Player

*i*. Then the probability of player

*i*placing 1st is

*i*’s stack to the total stack not including the stack of the conditional 1st place finisher. Thus,

*i*finishing 3rd is

In the 3-tower model, the probability of finishing 1st is easily solved by recursion. The result is exactly the same as (3). A method of calculating 3rd place probabilities (and hence, 2nd place probabilities) for the 3-tower problem is presented here using Markov chains constructed using directed multigraphs with loops.

## 2 Methods

Consider a 3-player model where the players are involved in even money bets. We define a *state* as an ordered triple (*x*, *y*, *z*) with \(x \ge y \ge z \ge 0\). Because the games in each round are all fair and random, the probabilities of becoming ruined first of each player depends only on the amount of money each player has in a given state. We also define *chip position* (or simply, *position*) to be the number of money (or chips) a player has in a given state. Let us also define the function *p*(*u*|*v*, *w*) to be the probability that a player with a given chip position *u* in the state (*u*, *v*, *w*) will finish 3rd, or become ruined first. If the state is understood from context, we will simply write this as *p*(*u*). If *v* and *w* are positions in the same state such that \(v = w\), then it is assumed that \(p(v) = p(w)\).

A *terminal state* is one wherein the probabilities of placing 3rd are known. There are two types of terminal states.

- 1.
If one of the three positions is zero, i.e. \(z = 0\)

- 2.
If all three positions are equal, i.e. \(x = y = z\).

Note that for the terminal state (*x*, *y*, 0), then \(p(x) = p(y) = 0\) and \(p(0) = 1\). For the terminal state (*x*, *x*, *x*), \(p(x) = 1/3\) using the previous assumption.

A state (*x*, *y*, *z*) is adjacent to a state (*u*, *v*, *w*) if the former state can move to the latter state in one round. A state that is adjacent to a terminal state is called a *near-terminal state*.

### **Lemma 1**

*x*,

*y*,

*z*) with \(x \ge y \ge z\) that satisfies one of the following is a near-terminal state:

- (i)
\(z = 1\)

- (ii)
\(x = y + 1\) and \(z = y - 1\).

In constructing the multigraph, all possible states of *S* are represented by nodes. The transitions between adjacent states are given by directed edges. The states are arranged such that all states (*x*, *y*, *z*) with a fixed value of *z* are aligned vertically, with the highest value of *x* in the topmost position, in decreasing order going down (i.e. from North to South), while *y* is increasing at the same time. All states (*x*, *y*, *z*) with fixed *x* are aligned horizontally, with *y* in decreasing order from left to right (i.e. West to East) and *z* increasing at the same time. Consequently, all states with fixed *y* are aligned diagonally, with *x* decreasing and *z* increasing as the states move from Northwest to Southeast. An example of the resulting multigraph is given in Figs. 1 and 2. From the construction, it is clear that for a given node, its adjacent nodes are the ones located to its immediate top, bottom, left, right, top left and bottom right positions (i.e. North, South, East, West, Northwest and Southeast). A state may be adjacent to itself if the following holds:

### **Lemma 2**

A state (*x*, *y*, *z*) is adjacent to itself if \(y = x - 1\) and/or \(z = y - 1\).

In the “and” case in Lemma 2, the state is doubly adjacent to itself. The state is also doubly adjacent to itself for states of the form \((x, x, x-1)\) and \((x, x-1, x-1)\). This and the following Lemma can be proved using the definition of adjacent nodes and (1) with \(b = 1\).

### **Lemma 3**

A state of the form (*x*, *y*, *y*) or (*x*, *x*, *z*) is doubly adjacent to its adjacent nodes.

There is always at least one edge from a non-terminal state to its adjacent state. A state that is doubly adjacent to another state has two edges going to that other state. If a state *A* is doubly adjacent to a non-terminal state *B*, it does not follow that *B* is doubly adjacent to *A*.

## 3 Results and Discussion

Based on the multigraph, we construct the Markov chain. Let \(\tau \) be a relation that maps a position from one state to a position in another non-terminal state. We can think of \(\tau \) as directed edges that connect specific positions within states to other positions in other states (or possibly within the same state). Let the function \(\phi \) denote the ruin probability of the position in one move. Note that the unique non-terminal positions are exactly the transient states in the Markov chain, while \(\phi \) gives the probability of absorption to the first-ruined state. The Theorem below follows from the previous Lemmas.

### **Theorem 1**

Let the state corresponding to a non-terminal vertex be denoted by (*x*, *y*, *z*) where \(x \ge y \ge z\) such that \(z \ge 1\). Let \(S = x + y + z\) and let (*u*, *v*, *w*) be an adjacent non-terminal vertex.

- (i)
If \(z = y - 1\), then \(\tau (x) \rightarrow u, \tau (y) \rightarrow w, \tau (z) \rightarrow v\)

- (ii)
If \(y = x - 1\), then \(\tau (x) \rightarrow v, \tau (y) \rightarrow u, \tau (z) \rightarrow w\)

- (iii)
If \(z = y\), then \(\tau (x) \rightarrow u\) twice, \(\tau (y) \rightarrow v\) and \(\tau (y) \rightarrow w\)

- (iv)
If \(y = x\), then \(\tau (z) \rightarrow w\) twice, \(\tau (x) \rightarrow u\) and \(\tau (x) \rightarrow v\).

In all other cases, the transitions are \(\tau (x) \rightarrow u\), \(\tau (y) \rightarrow v\) and \(\tau (z) \rightarrow w\).

### *Remark 1*

*i*),(

*ii*) are given by

- (i)
\(\tau (x) \rightarrow x, \tau (y) \rightarrow z, \tau (z) \rightarrow y\)

- (ii)
\(\tau (x) \rightarrow y, \tau (y) \rightarrow x, \tau (z) \rightarrow z\).

Each mapping of positions by \(\tau \) gives a transition probability of 1 / 6, except when the mapping occurs twice, as in the 3rd and 4th cases in the Theorem, then the transition probability is 2 / 6. These values are then used to generate the transient matrix \(\mathbf Q \) in the Markov chain. For the absorption probabilities, we use the following:

### **Theorem 2**

Given a position *u*, then \(\phi (u) = 1/3\) when \(u = 1\). If \(u > 1\), then \(\phi (u) = 0\), the only exception is for the state \((u+1, u, u-1)\), wherein \(\phi (u+1) = \phi (u) = \phi (u-1) = 1/18\).

**r**. Finally, we solve the system

**p**gives the probabilities of first ruin for each position and

**I**is the identity matrix with the same dimension as

**Q**. It is easy to show the transient matrix \(\mathbf I -\mathbf Q \) is invertible [5].

### *Example 1*

*iii*) to get \(\tau (2) \rightarrow 1\) twice, \(\tau (1) \rightarrow \tau (2)\) once and \(\tau (1) \rightarrow \tau (1)\) once. This generates our transient matrix

**Q**. Using Theorem 2, \(\phi (2) = 0\) and \(\phi (1) = 1/3\). This produces the vector

**r**. Thus

### *Example 2*

*a*, 1

*a*), (6, 2

*a*, 1

*b*), (5

*a*, 3

*a*, 1

*c*), (4

*a*, 1

*d*, 1

*d*), (5

*b*, 2

*b*, 2

*b*), and (4

*b*, 3

*b*, 2

*c*). The unique positions are then arranged starting from the

*x*positions in each of the states above, then the

*y*positions, and then the

*z*positions, skipping the non-unique positions as needed. Thus our indices correspond to 7, 6, 5

*a*, 4

*a*, 5

*b*, 4

*b*, 1

*a*, 2

*a*, 3

*a*, 1

*d*, 2

*b*, 3

*b*, 1

*b*, 1

*c*and 2

*c*, respectively. For example, index \(i = 1\) of

**Q**,

**r**and

**p**corresponds to values for position 7 from the state (7, 1, 1), while index \(i=15\) corresponds to the state-position 2

*c*from the state (4, 3, 2). In constructing

**Q**, note that by Theorem 1(

*iii*), \(\tau (7) \rightarrow 6\) twice, and using the 1 and 2-index for positions 7 and 6, respectively, we have \(Q_{12} = 2/6\). Because all other transitions of 7 are towards terminal states, then \(Q_{1j} = 0\) for \(j \ne 2\). From Theorem 2, \(r_1 = \phi (7) = 0\) because 7 is not a near-terminal position. For position 6, we have \(\tau (6) \rightarrow \{7, 6, 5a, 5b\}\) using Theorem 1(

*i*), hence \(Q_{21} = Q_{22} = Q_{23} = Q_{25} = 1/6\). For position 5

*a*in the state (5

*a*, 3

*a*, 1

*c*), the ’regular’ case of Theorem 1 applies, hence \(\tau (5a) \rightarrow \{6, 5b, 4a, 4b\}\). The rest of the entries are computed similarly, using Theorems 1 and 2. The transient matrix

**Q**and absorption vector

**r**in (6) are obtained as follows:

*x*,

*y*and

*z*increase, the ruin probabilities approach a limiting value. The ICM values are shown for comparison.

Probabilities of placing 3rd for given states (*x*, *y*, *z*), where *x* : *y* : *z* are in the ratio 3 : 2 : 1. The ICM values are shown for comparison.

( | | | |
---|---|---|---|

(3, 2, 1) | 0.12690355 | 0.25888325 | 0.61421320 |

(6, 4, 2) | 0.12672895 | 0.25857077 | 0.61470028 |

(12, 8, 4) | 0.12671616 | 0.25854223 | 0.61474162 |

(24, 16, 8) | 0.12671533 | 0.25854029 | 0.61474439 |

(48, 32, 16) | 0.12671528 | 0.25854016 | 0.61474456 |

(96, 64, 32) | 0.12671527 | 0.25854016 | 0.61474457 |

ICM | 0.1500 | 0.2667 | 0.5833 |

**Q**when \(S\text { mod } 3 = 0\) by regarding the state (

*S*/ 3,

*S*/ 3,

*S*/ 3) as non-terminal. The time until ruin is calculated from the row sum of \(\mathbf N = (\mathbf I - \mathbf Q )^{-1}\). In Example 1, for the state (2, 1, 1),

**Q**given by (7) and

**I**the \(2 \times 2\) identity, we obtain

## 4 Conclusion

In this paper, a method of solving players’ first-ruin probabilities in the 3-tower problem or gambler’s ruin with three players was presented. The assumptions were that each player started with some nonzero number of chips and during each round, two players were randomly selected in an even-money bet with a randomly chosen winner winning one chip from the loser. One application of this is computing equities in a partial information game (e.g. poker tournaments) modeled as a random game. A multigraph of the various states given *S* total chips was constructed. The method specified how to obtain the state transitions and absorption probabilities, as given by Theorems 1 and 2. The resulting linear system of the Markov chain were then used to solve the 3rd place (and thus 2nd place) probabilities of any state in *S*. Although a closed form formula was not derived for the probabilities, the method produces exact solutions instead of numerical approximations. This made it possible to show subtle differences in probabilities of first ruin as *S* was increased, while preserving the relative chip ratios. In contrast, other methods, such as ICM or Brownian models, only depend on the proportion of chips each player has, thus are independent of any scaling factor. The calculated results were similar to previous numerical approximations using Brownian motion, but differed from ICM by up to \(15\,\%\), although as mentioned, ICM and the 3-tower problem do not use the same assumptions.

The 3-tower model may be extended to one wherein bet sizes are not fixed. The multigraph form of such a model would be much more complex because the number of edges and adjacent nodes is not limited to six. The model may also be applied to other forms of the three-player gambler’s ruin such as player-centric and symmetric games. An extension to an *N*-tower problem may be done but the increase in complexity of the graph is expected to be significant.

## Notes

### Acknowledgments

This project was supported by the National Sciences Research Institute of the University of the Philippines, Project Reference No. 2008.149. A special thanks to Ramon Marfil of the Institute of Mathematics, University of the Philippines, for his assistance in the project.

## References

- 1.Bruss, F.T., Louchard, G., Turner, J.W.: On the \(N\)-tower-problem and related problems. Adv. Appl. Probab.
**35**, 278–294 (2003)MathSciNetzbMATHGoogle Scholar - 2.Engel, A.: The computer solves the three tower problem. Am. Math. Mon.
**100**, 62–64 (1993)CrossRefzbMATHGoogle Scholar - 3.Ferguson, T.S.: Gambler’s Ruin in Three Dimensions (1995). http://www.math.ucla.edu/gamblers
- 4.Finch, S.: Gambler’s Ruin (2008). http://www.people.fas.harvard.edu/sfinch/csolve/ruin.pdf
- 5.Grinstead, C.M., Snell, J.L.: Introduction to Probability, 2nd edn. American Mathematical Society, Providence (2006)zbMATHGoogle Scholar
- 6.Kim, M.S.: Gambler’s ruin in many dimensions and optimal strategy in repeated multi-player games with application to poker. Master’s Thesis, UCLA (2005)Google Scholar
- 7.Roberts, B.: Ben Roberts ICM Model (2012). http://www.pokericmcalculator.com/en-us/articles/ben-roberts-icm-model
- 8.Rocha, A., Stern, F.: The gambler’s ruin problem with \(n\) players and asymmetric play. Stat. Probabil. Lett.
**44**, 87–95 (1998)MathSciNetCrossRefzbMATHGoogle Scholar - 9.Rocha, A., Stern, F.: The asymmetric \(n\)-player gambler’s ruin problem with equal initial fortunes. Adv. Appl. Math.
**33**, 512–530 (2004)MathSciNetCrossRefzbMATHGoogle Scholar - 10.Stirzaker, D.: Tower problems and martingales. Math. Sci.
**19**, 52–59 (1994)MathSciNetzbMATHGoogle Scholar - 11.Swan, Y., Bruss, F.T.: A matrix-analytic approach to the \(n\)-player ruin problem. J. Appl. Probab.
**43**, 755–766 (2006)MathSciNetzbMATHGoogle Scholar