- 02 Jul '07 19:42 / 3 edits

Let's start by stating Bayes' Theorem in plain English.*Originally posted by Nemesio***I've just read the Bayesian Theorem article on Wikipedia and don't even understand the chocolate**

cookie example.

Hold my hand a bit and explain it to me and then explain what it means to be a Bayesian atheist.

I thank you in advance and am,

Appreciatively yours,

Nemesio

Bayes' Theorem asserts that these two products have an equal value, for any propositions A and B:

1) The probability that A is true whenever B is true, times the probability that B is in fact true.

2) The probability that B is true whenever A is true, times the probability that A is in fact true.

So, there are four things in play in Bayes' Theorem:

a) The probability that A is true, given no information about B, denoted P(A)

b) The probability that B is true, given no information about A, denoted P(B)

c) The probability that A is true, given that B is true, denoted P(A|B)

d) The probability that B is true, given that A is true, denoted P(B|A)

Bayes' Theorem can be applied whenever three of these quantities are known and you wish to determine the fourth. This is why it's usually written as an equation with the unknown quantity on the left and the ratio on the right.

If that framework is clear, then we can go on to apply it to the cookie problem. - 03 Jul '07 04:12

Sorry I took so long to respond. I wanted to print out the post and look it over a few times and*Originally posted by DoctorScribbles***Let's start by stating Bayes' Theorem in plain English.**

Bayes' Theorem asserts that these two products have an equal value, for any propositions A and B:

1) The probability that A is true whenever B is true, times the probability that B is in fact true.

2) The probability that B is true whenever A is true, times the probability that A is i ...[text shortened]... right.

If that framework is clear, then we can go on to apply it to the cookie problem.

get it comfortable in my head.

I think I understand what the theorem asserts and what it is used for discovering, but the As and Bs

make it a little hard for the concept to crystalize for me (being an exemplar-based thinker). Let's

move on to the cookie example (or a simpler one, perhaps?) and see if it starts to take root.

Nemesio - 03 Jul '07 06:03 / 17 editsSuppose you are playing Hold'em against an odd opponent whom you

know with certainty raises before the flop only with pocket Aces,

pocket Queens, or Ace-Queen.

You are playing a hand and lo and behold, he raises before the flop.

Then, as he's shuffling his two cards he happens to expose one at

random - an Ace.

You hold pocket Kings and might have the best hand -- as long as he

doesn't hold pocket Aces.

Problem: Given that you've seen his flashed Ace, what is the probability

that his other card is a Queen, leaving you the favorite?

Bayes' Theorem comes to the rescue.

Define two propositions:

Proposition A: Your opponent was dealt Queens or Ace-Queen.

Proposition B: A random exposed card of this opponent is an Ace.

State the three known quantities:

a) P(A): We can compute the probability that the opponent holds

Queens or Ace-Queen by counting the ways he could hold Queens

or Ace-Queen and dividing it by the total possible hands he could

have (disregarding the flashed card). He could have Ace-Queen

16 ways (4 Aces, times 4 Queens), and he could have Queens 6

ways (4*3/2), and he could have a total of 28 (8*7/2) different

hands. Thus, P(A)=22/28.

b) P(B): Since the opponent's hand can contain only Aces or

Queens, and since he is equally likely to have recieved either

(still disregarding the flashed card), since there are four of each

in the deck, P(B)=1/2.

[*Note that P(A) and P(B) must always be computed from a*]

position of ignorance with regard to any reavealed truth of A and B.

They are typically called prior probabilities, to be computed from a

set of information available prior to the truth of either A or B

being revealed (as B is revealed to be true in this scenario).

c) P(B|A): Here we compute the probability that he flashes an

Ace given that he holds Queens or Ace-Queen. If he has an Ace

and a Queen, an Ace is exposed 50% of the time. If he holds

Queens, an Ace is exposed 0% of the time. Weighting these by

the number of ways each type of hand could be dealt, we get

P(B|A)=0*(6/28)+.5*(16/28)=8/28.

Finally, solve for the unknown:

d) P(A|B): This is the unknown that we are now ready to use

Bayes' Theorem to compute. According to the theorem:

P(A|B) = P(B|A)*P(A)/P(B)

= (8/28)*(22/28)/(1/2)

= 45%

Bayes' Theorem tells us that given all of our information, the

opponent has Queens or Ace-Queen with likelihood 45%,

which means that he holds Aces with probability 55%, which

means folding your Kings is the correct play.

It is the extra information, the flashed Ace, the revelation that

B is in fact true, that allows you to make the correct play in this

case, since pocket Kings would otherwise be on average a

favorite to his possible holdings in the absence of the extra

information.

Bayes' Theorem is all about revising probability estimates in

light of new information. The Monty Hall problem is another

good problem that you could try to work through using this method,

although that has other, more straightforward methods of solution.

The thing about being introduced to Bayes' Theorem is that at first

it seems like more trouble than it's worth when using it on toy

problems, since the final probability of interest can be solved

directly without too much trouble using non-Bayesian analysis.

The value of Bayes' is seen when the prior probabilities are much

easier to compute than the conditional probabilities, which is often

the case in real problems. - 03 Jul '07 06:39

I'm too tired to review the problem's solution right now, but let me give a first blush impression*Originally posted by DoctorScribbles***Suppose you are playing Hold'em against an odd opponent whom you**

know with certainty raises before the flop only with pocket Aces,

pocket Queens, or Ace-Queen.

You are playing a hand and lo and behold, he raises before the flop.

Then, as he's shuffling his two cards he happens to expose one at

random - an Ace.

You hold pocket Kings and ...[text shortened]... shed Ace, what is the probability

that his other card is a Queen, leaving you the favorite?

and you can tell me why it's wrong.

He's flashed an Ace, so that means he has either Aces or Ace-Queen. So, here are the hands he

could have:

1) A-S + A-C

2) A-S + A-D

3) A-S + A-H

4) A-C + A-D

5) A-C + A-H

6) A-D + A-C

7) A-S + Q-S

8) A-S + Q-C

9) A-S + Q-D

10) A-S + Q-H

11) A-C + Q-S

12) A-C + Q-C

13) A-C + Q-D

14) A-C + Q-H

15) A-D + Q-S

16) A-D + Q-C

17) A-D + Q-D

18) A-D + Q-H

19) A-H + Q-S

20) A-H + Q-C

21) A-H + Q-D

22) A-H + Q-H

22 hands, of which I can beat 16. 16/22 tells me I'm just shy of a 3 to 1 favorite. If I don't see

the Ace, I would suppose that I was a slightly better than 3 to 1 favorite.

What's wrong with this analysis?

Nemesio - 03 Jul '07 06:50 / 4 edits

One problem with your analysis is that it does not account for all of the available information. Namely, it does not account for the fact that you saw a particular Ace.*Originally posted by Nemesio***I'm too tired to review the problem's solution right now, but let me give a first blush impression**

and you can tell me why it's wrong.

He's flashed an Ace, so that means he has either Aces or Ace-Queen. So, here are the hands he

could have:

1) A-S + A-C

2) A-S + A-D

3) A-S + A-H

4) A-C + A-D

5) A-C + A-H

6) A-D + A-C

7) A-S + Q-S

8) A-S + Q-C as a slightly better than 3 to 1 favorite.

What's wrong with this analysis?

Nemesio

Your distribution would really look something like this, after seeing the Ace of, say, Spades.

As, Ah

As, Ac

As, Ad

As, Qh

As, Qs

As, Qd

As, Qc

Seven hands, of which you can beat four.

Now, I have to figure out what else is missing to reconcile our analyses, because that doesn't quite cut it...

We might have to go back to cookies if I can't fix it. I can do cookies. - 03 Jul '07 06:56

I should be asleep, but is this where the mistake is?*Originally posted by DoctorScribbles***c) P(B|A): Here we compute the probability that he flashes an**

Ace given that he holds Queens or Ace-Queen. If he has an Ace

and a Queen, an Ace is exposed 50% of the time. If he holds

Queens, an Ace is exposed 0% of the time. Weighting these by

the number of ways each type of hand could be dealt, we get

P(B|A)=0*(6/28)+.5*(16/28)=8/28.

We're not accounting for how often he flashes an Ace given that he has Ace-Ace.

Or am I totally lost? - 03 Jul '07 07:02

If I have chosen the correct A, then that is not the problem, because A is assumed to be true with certainty when computing P(B|A). The problem might be in my choice of A.*Originally posted by Nemesio***I should be asleep, but is this where the mistake is?**

We're not accounting for how often he flashes an Ace given that he has Ace-Ace.

Or am I totally lost? - 03 Jul '07 07:10 / 1 editWell, I'm a bit stumped for the moment, but my gut feeling is that as an empirical matter, Kings is definitely not a 3-to-1 favorite in this scenario, and before doing the Bayesian analysis, I would have guessed Kings to be a slight dog, so I think that analysis is right. I think there is something subtle in your distribution regarding your assumption that each holding is equally likely after the Ace flashes. I'll think it over a bit.
- 03 Jul '07 07:16 / 6 editsYes. For one, he's more likely to have any one of your (1) through (6) as any one of your (7) through (22).

(This is the same idea as in the Monty Hall problem; Monty can't open the door with the prize; and you can't ever see a Queen exposed, but there are two cards you could have seen in each hand when he has Aces. Your equally-likely assumption about your remaining distribution is akin to the faulty Monty Hall analysis of listing the remaining doors after one is opened, and concluding that you have a 50% chance of picking the right one.)

I'll work out the numbers and see if that reconciles it. - 03 Jul '07 08:23 / 6 edits

Yes, that's correct!*Originally posted by Palynka***Your mistake is in P(B|A). You use 28 in the denominator but you should have used 22 because it's the probabilities given A.**

With the correct arithmetic, P(A|B)=57%, which matches the number given by the revised seven-way distribution, leaving Kings likely to be a favorite.

After the correction, this result was initially very unintuitive and troubling to me, but then I realized that I was confounding the likelihood of being the favorite with the likelihood of winning the hand. Kings is only a slight favorite to the Ace-Queen hands, but a big dog to Aces, so taking the likelihood of actually winning the hand into consideration, Kings is a dog against the possible holdings and folding is in fact the correct play, even though Kings is most frequently a small favorite.