|
|
Information Gain
| Publication |
Mitchell/97b: Machine Learning
|
| Name |
Information Gain |
| Description |
The Information Gain is a measure based on Entropy.
Given
- a set E of classified examples and
- a partition P = {E1, ..., En} of E.
The Information Gain is defined as
| ig(E, P) := entropy(E) - |
∑ |
entropy(Ei) * |Ei| / |E| |
| i=1,...,n |
|
Intuitively spoken the Information Gain measures the decrease of the weighted
average impurity of the partitions E1, ..., En,
compared with the impurity of the complete set of examples E.
|
| Algorithm |
ID3
|
|
|
