Overfitting

Overfitting is the phenomenum that a learning algorithm adapts so well to a training set, that the random disturbances in the training set are included in the model as being meaningful. Consequently (as these disturbances do not reflect the underlying distribution), the performance on the testset (with its own, but definitively other, disturbances) will suffer from techiques that learn to well.

Train and test, cross validation and bootstrap approaches have been developed to cope with this problem.