Bayesian Learning

Publication Mitchell/97b: Machine Learning

Name Bayesian Learning

Description
Bayesian Learning constitutes a probabilistic view of learning, based on Bayes Theorem. The underlying assumption is, that there is a set of hypotheses, each having a certain probability of being correct. Receiving more information changes the probabilities from a learner's point of view. For instance an observation might contradict a hypothesis, or strengthen the belief in it. The aim in this setting is to be able to find a hypothesis with highest probability of being correct, given a specific set of data / piece of information.
Bayes Theorem:
Let

... P be a probability distribution.

... h_i, i ∈ {1, ..., n} denote a set of hypotheses.

... P(h_i) denote the probability of h_i being correct in the general case, that is without being given any additional information. P(h_i) is called the prior probability of h_i.

... D denote a set of data.

... P(D) be the probability of the information denoted by D being correct in the general case.

... P(D | h_i) denote the probability of the information D being correct, given the correctness of hypothesis h_i.

... P(h_i | D) denote the probability of the correctness of hypothesis h_i, given the additional information D. This is called the posterior probability of hypothesis h_i.

Bayes Theorem:
P(h | D) = P(D | h) * P(h) / P(D)

This theorem allows us to find a hypothesis h'with maximum posterior probability, given the prior probabilities of all hypotheses and the probabilities of D being correct under the assumption of each single hypothesis being correct:

h' := max [ P(D | h_i) * P(h_i) ]

^h_i

Dm Step Concept Learning

Methods Naive Bayes