Regularization (mathematics)

Regularization, in mathematics and statistics and particularly in the fields of machine learning and inverse problems, refers to a process of introducing additional information in order to solve an ill-posed problem or to prevent overfitting.

In general, a regularization term ${\displaystyle R(f)}$ is introduced to a general loss function:

for a loss function ${\displaystyle V}$ that describes the cost of predicting ${\displaystyle f(x)}$ when the label is ${\displaystyle y}$, such as the square loss or hinge loss, and for the term ${\displaystyle \lambda }$ which controls the importance of the regularization term. ${\displaystyle R(f)}$ is typically a penalty on the complexity of ${\displaystyle f}$, such as restrictions for smoothness or bounds on the vector space norm.

...
Wikipedia