Bayes Theorem, with Applications

A simple form of Bayes' theorem is

P(A|B) = P(A) * P(B|A) * / P(B)

(1)

This can be read as "the conditional probability of A given B is equal to the conditional probability of B given A, times the probability of A divided by the probability of B."

Derivation and Application No. 0

A conditional probability is the probability that a condition A is true, given that you already know that another condition B is true. For example, if you have no access to the weather forcast (or a calendar) in Washington, D.C., the probability of cloudy skies tonight might be 0.18, or 18 percent. If Al Joker says to expect cloudy skies, then the probability could be 89 percent. This is a conditional probability; the forecast is the independent event, and the actual cloudiness is the dependent, or conditioned, event. If you let C = "there are cloudy skies tonight" and A = "Al Joker predicts cloudy skies tonight", these two probabilities can be expressed symbolically as:

P(C) = 0.18
P(C|A) = 0.89

It should be apparent that if you know a conditional probability of an event, based on the condition of another event, then the probability of both events events happening can be calculated. Simply multiply the conditional probability by the probability of the independent event. Suppose that 15 percent of the time, Al Joker predicts clouds. Then, the probability that he predicts clouds, and clouds actually appear is:

P(CA) = P(C|A) * P(A)

or

P(CA) = 0.89 * 0.15 = 0.1335

However, the following is also true:

P(CA) = P(AC) = P(A|C) * P(C)

This leads to:

P(C|A) * P(A) = P(A|C) * P(C)

Which can be rearranged into the form of Bayes' theorem as shown above:

P(A|C) = P(C|A) * P(A) / P(C)

(2)

Substituting our numbers, we find another measure of Al Joker's reliability:

P(A|C) = 0.89 * 0.15 / 0.18 = 0.742

This tells us that Al successfully predicts only 74 percent of the cloudy nights which actually happen.

Application No. 1

This can be used to determine whether or not to panic when your HIV test comes out positive.

We know that even the best test can give false results. Using made-up numbers, suppose that the test is known to be 98 percent accurate, giving only two percent false negatives and two percent false positives. Suppose further that, without knowing anything more about you except that you do not engage in risky behavior, the probability of having an HIV infection is one-half percent.

We can define:

A = a person is infected,
B = the test is positive.

So we have:

P(A) = 0.005
P(~B|A) = 0.02
P(B|A) = 0.98
P(B|~A) = 0.02

We don't yet have P(B), the probability that the test will be positive, so we calculate it by

P(B) = P(B|A) * P(A) + P(B|~A) * P(~A)
= 0.98 * 0.005 + 0.02 * 0.995 = 0.0049 + .0199 = 0.0248

So

P(A|B) = 0.005 * 0.98 / 0.0248 = 0.197

This is almost exactly one chance in five, and the test remains 98 percent accurate! Here is a web site that allows you to try your own figures.

Application No. 2

Bayes' theorem can be used as a normative model to assess how well people use empirical information to update the probability that a hypothesis is true. In other words, how well they learn from experience.

Note No. 1

References

Here are some web pages with lists of references on Thomas Bayes.

Bayes' theorem and the prosecutor's fallacy.

Bayesian links.

© Copyright 1999 by John C. Stewart