Thanks David
20 Nov 2008
Feb
26
Fido helps explain Bayes’ Formula
by David Harper, CFA, FRM, CIPM
Quant | |
We sometimes enlist Fido’s help with statistics. This time, he helps explain Bayes’ Formula.
Bayes’ Formula updates the odds of an event given a piece of helpful information. So, we need an event and we need information. To illustrate, the event is whether Fido received a dinner-bone. The information is Fido’s mood: he is either happy or not happy. Bayes’ Formula is about prior probabilities and posterior probabilities. Prior probabilities are probabilities in a vacuum, before we have information. Posterior probabilities are conditional (or updated) probabilities: the odds if we do have some additional information.
Take Fido as an example. When we come home from work, before we open the door, experience tells us there is a 60% chance that Fido already ate his dinner-bone. That’s a prior probability of 60% that Fido ate his bone. But that’s without prior information, before we open the door. Now compare this to the moment after we open the door, when Fido greets us. And assume Fido is wearing a happy dog face. Now we have additional information. Armed with additional information (“Fido is happy”), we can ask, what is the posterior probability that Fido already ate his dinner-bone?
Because Fido’s happiness correlates with dinner, the odds are now greater than 60% that he already ate his dinner-bone. We need two other conditional probabilities, which Fido supplies (since he is versed in statistics). According to Fido, if he gets a bone, there is a 90% chance he will be happy. On the other hand, if he does not get his bone, there is only a 20% chance that he will be happy.
Note these two statistics, P(happy|dinner) and P(happy|no dinner), are conditional probabilities. Respectively, they read “what is the probability Fido is happy given he had dinner?” and “what is the probability Fido is happy given he did not have dinner?” (Note: they do not need to sum to 100%).
But our question is “What are the odds Fido ate dinner, given we see that Fido is happy?” We already noted that the prior probability is 60%; that is simply the proportion over an entire historical sample, regardless of Fido’s happiness.
But the posterior probability is denoted by P(Event|Info). Translated this means, “the probability of the event given the information.” (on the keyboard, the ’|’ is called a ‘pipe’).
Bayes’ Formula tells us that the P(Event|Info) is given by:
In our example, dinner (or not dinner) is the event and happy (or not happy) dog is the information. So we have:
The only term we don’t already have is the P(Happy). But we can multiply that out. The probability that Fido is happy, P(Happy), is the probability Fido is happy given he had dinner plus the probability Fido is happy given he did not have dinner. Specifically,
P(Happy) = P(Happy | Dinner) x P(Dinner) + P(Happy | No Dinner) x P(No Dinner)
See how this is just the weighted expectation that Fido is happy? The probability he is happy if he did have dinner plus the probability he is happy that he didn’t have dinner. That covers all scenarios. Now we can calculate the posterior probability using Bayes’ Formula:
If we abstract from the specifics, we can denote the event as (E) and the information as (I). Please note, you will see different notation. Instead of (E) for event and (I) for information, some use, respectively, (C) for “cause” and (E) for “effect.”
But generically, given (E) for the event and (I) for the information, the probability of the event given prior information, P(E|I) is given by:
The denominator is the total probability of the information. That’s because it sums up all the probabilities of the information given all possible events (E1, E2, ...). Therefore, the denominator contains as many terms as events. In its full abstraction, therefore, Bayes’ Formula is given by:
Comments
Be the first to leave a comment!
Leave a Comment