Bayes’ Rule

Synonyms

Bayes’ Theorem

Definition

Bayes’ rule provides a decomposition of a conditional probability that is frequently used in a family of learning techniques collectively called Bayesian learning. Bayes’ rule is the equality

$$ P(A|B)=\frac{P(A)P(B|A)}{P(B)} $$

(1)

P(A) is called the prior probability and P(A | B) is called the posterior probability.

Discussion

Bayes’ rule is used for two primary purposes in machine learning. The first is Bayesian update. In this context, B represents some new information that has become available since an estimate P(A) was formed of some hypothesis A. The application of Bayes’ rule enables a new estimate of the probability of A (the posterior probability) to be calculated from estimates of the prior probability, P(B | A) and P(B).

The second common application of Bayes’ rule is for estimating posterior probabilities in probabilistic learning, where it is the core of Bayesian networks, naïve Bayes, and semi-naïve Bayesian techniques.

While Bayes’...