This post is part of the ppmmc series. Please make sure to have read the previous posts in the ppmmc series so that the context is understood.

A Very Introductory Introduction to Markov Chains — Part 2

In the previous post we considered a very simple game wherein a roll of a \(1,3,\) or \(5\) resulted in a loss, a roll of \(6\) resulted in a win, and a roll of \(2\) or \(4\) entitled the player to another roll. We saw that our simulation seemed to be inline with what probability theory would dictate (although we didn’t actually show this rigorously) — three in six rolls resulted in a loss, one in six rolls resulted in a win, and two in six rolls gave us a reroll.

Now, what we want to do is actually play this game and record some game specific statistics. Before we write any more code, let’s try to answer these questions: what is the probability of winning a game? Is it different from the probability of winning on a single roll? If it is different, why? If it isn’t different, also, why?

For this game, it turns out that the probability of winning is different than the probability of winning on a single roll. Why? There are a lot of ways to reason this out, but here is one way.

On the first roll, we have a one in six chance of winning and a two in six change of “staying alive.”

If we stay alive, then we have a one in six chance of winning again. So, if we asked the question, “what is the probability of winning on the first roll?”, the answer would be “one in six”. If we asked the question, “what is the probability of winning exactly on the second roll?”, then we would reason that it would be two in thirty-six since we need to first stay alive (two in six chance) \textit{and} then on the second roll we need to win (one in six chance). Therefore, the product means that we have a two in thirty-six chance of winning on exactly the second roll.

What about winning on exactly the third roll? That means we have to stay alive two times in a row and then on the third roll we have to win. That works out to \(\frac{2}{6}\cdot \frac{2}{6}\cdot \frac{1}{6} = \frac{4}{216}\)

Now, convince yourself that winning on exactly the \(n\)th roll occurs with probability \(\Big(\frac{2}{6}\Big)^{n-1}\cdot \frac{1}{6}\).

So, now that we have figured out the probability of winning on a given roll, we can figure out the probability of winning. This sentence is of vital important for you to understand!!! This sentence is the math!! There is a huge difference between winning on a given roll and just winning. The conditions matter! If you can understand this, you can understand probability theory, since in a nutshell that’s all that probability theory is!

So the probability of winning is simply the sum of the probabilities of all the winning events. Specifically, that’s $$P(winning) = \sum_{k = 1}^{\infty}\Big(\frac{2}{6}\Big)^{k-1}\frac{1}{6}$$

And if we remember how infinite series work (don’t know? don’t worry! we’ll show you in a second), then this resolves to \(\frac{1}{4}\) or 25%.

Now, we arrived at 25% via some calculation. While that is important, it is just as important to check our intuition. Does this make sense? The probability of winning on a single roll is \(\frac{1}{6}\), so why did the probability of winning (see the difference in the statement?) go up? Think about this intuitively. Forget about the numbers for a minute. Ask yourself if it makes sense that the probability should increase. This part you have to come to terms on your own. It is important!

A Little Diversion

Earlier we wrote: $$P(winning) = \sum_{k = 1}^{\infty}\Big(\frac{2}{6}\Big)^{k-1}\frac{1}{6}$$ and we asserted that this was equal to 25%.

But what if we didn’t know how to solve this analytically? We can’t add numbers all the way to infinity. But, can we get a ball park estimate for this quantity? We sure can! We’re learning to program! So here’s the exercise, write a program that computes the above sum, up to some fixed upper bound, and monitor the output. You should be able to see that it is stabilizing near 0.25.

Back To the Game

Ok, now for the exercise. We’re going to make this a game. Mr. Conner Tist comes along and makes you an offer you can’t refuse. You must pick one of the following two games to play.

In the first game, Mr. Conner Tist will pay you $1 if you roll a six. But you must pay him $1 if you roll an odd number. If you roll a two or a four, no money is exchanged. The game ends after one roll.

In the second game, the payout is the same, except if you roll a two or a four, you roll again. The game ends once you roll a one, three, five, or six.

So which game do you play since you have to play one?

Answer in the next part.

Don’t hesitate to get in touch if you have questions or would like more information about setting up a workshop or training in your school.