Maximum-likelihood

In statistics, maximum likelihood estimation (MLE) is a method of estimating the parameters of a statistical model given observations, by finding the parameter values that maximize the likelihood of making the observations given the parameters. MLE can be seen as a special case of the maximum a posteriori estimation (MAP) that assumes a uniform prior distribution of the parameters, or as a variant of the MAP that ignores the prior and which therefore is unregularized.

The method of maximum likelihood corresponds to many well-known estimation methods in statistics. For example, one may be interested in the heights of adult female penguins, but is unable to measure the height of every single penguin in a population due to cost or time constraints. Assuming that the heights are normally distributed with some unknown mean and variance, the mean and variance can be estimated with MLE while only knowing the heights of some sample of the overall population. MLE would accomplish this by taking the mean and variance as parameters and finding particular parametric values that make the observed results the most probable given the model.

In general, for a fixed set of data and underlying statistical model, the method of maximum likelihood selects the set of values of the model parameters that maximizes the likelihood function. Intuitively, this maximizes the "agreement" of the selected model with the observed data, and for discrete random variables it indeed maximizes the probability of the observed data under the resulting distribution. Maximum likelihood estimation gives a unified approach to estimation, which is well-defined in the case of the normal distribution and many other problems.

Suppose there is a sample x₁, x₂, …, x_n of n independent and identically distributed observations, coming from a distribution with an unknown probability density function f₀(·). It is however surmised that the function f₀ belongs to a certain family of distributions { f(·| θ), θ ∈ Θ } (where θ is a vector of parameters for this family), called the parametric model, so that f₀ = f(·| θ₀). The value θ₀ is unknown and is referred to as the true value of the parameter vector. It is desirable to find an estimator ${\displaystyle \scriptstyle {\hat {\theta }}}$ which would be as close to the true value θ₀ as possible. Either or both the observed variables x_i and the parameter θ can be vectors.

...
Wikipedia