Generalization error

In supervised learning applications in machine learning and statistical learning theory, generalization error (also known as the out-of-sample error) is a measure of how accurately an algorithm is able to predict outcome values for previously unseen data. Because learning algorithms are evaluated on finite samples, the evaluation of a learning algorithm may be sensitive to sampling error. As a result, measurements of prediction error on the current data may not provide much information about predictive ability on new data. Generalization error can be minimized by avoiding overfitting in the learning algorithm. The performance of a machine learning algorithm is measured by plots of the generalization error values through the learning process, which are called learning curves.

In a learning problem, the goal is to develop a function ${\displaystyle f(x)}$ that predicts output values ${\displaystyle y}$ based on some input data ${\displaystyle x}$. The expected error, ${\displaystyle I[f_{n}]}$ of a particular function ${\displaystyle f_{n}}$ over all possible values of ${\displaystyle x}$ and ${\displaystyle y}$ is:

...
Wikipedia