Bayes Theorem | CommonLounge

NaN.

Bayes Theorem[ Edit ]

Till now, we have calculated the probability of events based on the empirical( experimental ) evidence or on probabilities of other events. As we had remarked in our earlier tutorials that statistics enables us to tell a story about data, Bayesian inference is another powerful form of such story telling. It helps us to understand how our belief about the occurrence of an event can affect the probability of the event.

Generally, Bayes' theorem can be defined as:

P(A|B) = \frac{P(B|A) P(A)}{P(B)}

where A and B are two events.

P(A|B) := likelihood of the occurrence of event A given that event B has occurred.
P(A) := prior probability of the occurrence of event A
P(B) := prior probability of the occurrence of event B
P(B|A) := likelihood of the occurrence of event B given that event A has occurred.

In calculating the probability of occurrence of A given B has occurred, we have also incorporated the evidence of A and B occurring. Prior probability is the probability of the occurrence of an event without the presence of test evidence while Posterior probability accounts for the change in probability after incorporating test evidence.

Bayesian Rule

Q) Let us do a simple example on spam classification. Let A denote the probability that mail is "spam", B denote that it is of "low priority", and C denote that it is of "high priority". The prior probabilities are:

P(A) = 0.7, P(B) = 0.2, P(C) = 0.1. Let K denote the likelihood that the mail contains the word "free". The posterior probabilities accounting the prior experience of the mails containing "free" can be outlined as follows:

P(K|A) = 0.9, P(K|B) = 0.01, P(K|C) = 0.01. Given that I have received a mail containing "free", what is the probability that it is spam?

Ans. We have to calculate the probability of mail being spam given that it contains the word free. Applying the Bayesian theorem, we get:

P(A|K) = \frac{P(K|A) P(A)}{P(K)}

Now, to calculate P(K), we need to account our experience too:

P(K) = P(K|A)P(A) + P(K|B)P(B) + P(K|C)P(C)

Calculating both the above equations, we get: P(A|K) = 0.995.