problem 4

4.1.4 problem 4

Let’s start defining some convenient r.v.s for this problem:

$D_{i}$ : value of the i-th roll.
$w_{1}$ : winning if one keeps playing after the first roll.
$w_{2}$ : winning if one keeps playing after the second roll.

The optimal strategy is to stop if the value of the last roll is greater than the expected winning if one keeps playing. In other words, keep rolling if doing so brings winnings that are, on average, greater than the last roll:

1.: If $D_{1} > E (w_{1})$ , STOP after 1 roll.
2.: Else if $D_{2} > E (w_{2})$ , STOP after 2 rolls.

Since the rolls are independent, we can calculate the expectations of $w_{1}$ and $w_{2}$ in reverse order.

The winning $w_{2}$ is equal to the value of the third roll:

E (w_{2}) = E (D_{3}) = 6 \sum x = 1 x P (D_{3} = x) = \frac{1}{6} * (1 + 2 + 3 + 4 + 5 + 6)

E (w_{2}) = 3.50 dollars

This reveals the second part of the optimal strategy: stop after 2 rolls if $D_{2} \geq 4$ .

The PMF of $w_{1}$ is given by

\begin{matrix} \begin{matrix} P (w_{1} = x) & = P (D_{2} < 4, D_{3} = x) = \frac{3}{6} * \frac{1}{6} = \frac{3}{36} for x = 1, 2, 3 \end{matrix} \end{matrix}

\begin{matrix} \begin{matrix} P (w_{1} = x) & = P (D_{2} = x \cup D_{2} < 4, D_{3} = x) = \frac{1}{6} + \frac{3}{6} * \frac{1}{6} = \frac{9}{36} for x = 4, 5, 6 \end{matrix} \end{matrix}

Using those probabilities to calculate the expected winning

w_{1}

from the definition of expectation:

E (w_{1}) = 6 \sum x = 1 x P (w_{1} = x) = 4.25 dollars

Now we can fully describe the optimal strategy, which maximizes the expected winnings:

1.: If the value of the first roll is $\geq$ 5, STOP after 1 roll.
2.: Else if the value of the second roll is $\geq$ 4, STOP after 2 rolls.

Finally, let’s calculate the expected winning $W^{*}$ of the optimal strategy. The PMF of $W^{*}$ is calculated below

\begin{matrix} \begin{matrix} P (W^{*} = x) & = P (D_{1} < 5, D_{2} < 4, D_{3} = x) = \frac{4}{6} * \frac{3}{6} * \frac{1}{6} = \frac{12}{6^{3}} for x = 1, 2, 3 \end{matrix} \end{matrix}

\begin{matrix} \begin{matrix} P (W^{*} = 4) & = P (D_{1} < 5, D_{2} = 4 \cup D_{1} < 5, D_{2} < 4, D_{3} = 4) = \frac{4}{6} * \frac{1}{6} + \frac{4}{6} * \frac{3}{6} * \frac{1}{6} = \frac{36}{6^{3}} \end{matrix} \end{matrix}

\begin{matrix} \begin{matrix} P (W^{*} = x) & = P (D_{1} = x \cup D_{1} < 5, D_{2} = x \cup D_{1} < 5, D_{2} < 4, D_{3} = x) = \frac{1}{6} + \frac{4}{6} * \frac{1}{6} + \frac{4}{6} * \frac{3}{6} * \frac{1}{6} = \frac{72}{6^{3}} for x = 5, 6 \end{matrix} \end{matrix}

Plugging these probabilities into the definition of expectation:

E (W^{*}) = 6 \sum x = 1 x P (W^{*} = x) = 4.67 dollars

[next] [prev] [prev-tail] [front] [up]