Margin classifier

In machine learning, a margin classifier is a classifier which is able to give an associated distance from the decision boundary for each example. For instance, if a linear classifier (e.g. perceptron or linear discriminant analysis) is used, the distance (typically euclidean distance, though others may be used) of an example from the separating hyperplane is the margin of that example.

The notion of margin is important in several machine learning classification algorithms, as it can be used to bound the generalization error of the classifier. These bounds are frequently shown using the VC dimension. Of particular prominence is the generalization error bound on boosting algorithms and support vector machines.

See support vector machines and maximum-margin hyperplane for details.

The margin for an iterative boosting algorithm given a set of examples with two classes can be defined as follows. The classifier is given an example pair ${\displaystyle (x,y)}$ where ${\displaystyle x\in X}$ is a domain space and ${\displaystyle y\in Y=\{-1,+1\}}$ is the label of the example. The iterative boosting algorithm then selects a classifier ${\displaystyle h_{j}\in C}$ at each iteration ${\displaystyle j}$ where ${\displaystyle C}$ is a space of possible classifiers that predict real values. This hypothesis is then weighted by ${\displaystyle \alpha _{j}\in R}$ as selected by the boosting algorithm. At iteration ${\displaystyle t}$, the margin of an example ${\displaystyle x}$ can thus be defined as

...
Wikipedia