Rao–Blackwell theorem
In statistics, the Rao–Blackwell theorem, sometimes referred to as the Rao–Blackwell–Kolmogorov theorem, is a result which characterizes the transformation of an arbitrarily crude estimator into an estimator that is optimal by the mean-squared-error criterion or any of a variety of similar criteria.
The Rao–Blackwell theorem states that if g(X) is any kind of estimator of a parameter θ, then the conditional expectation of g(X) given T(X), where T is a sufficient statistic, is typically a better estimator of θ, and is never worse. Sometimes one can very easily construct a very crude estimator g(X), and then evaluate that conditional expected value to get an estimator that is in various senses optimal.
The theorem is named after Calyampudi Radhakrishna Rao and David Blackwell. The process of transforming an estimator using the Rao–Blackwell theorem is sometimes called Rao–Blackwellization. The transformed estimator is called the Rao–Blackwell estimator.
One case of Rao–Blackwell theorem states:
In other words
The essential tools of the proof besides the definition above are the law of total expectation and the fact that for any random variable Y, E(Y2) cannot be less than [E(Y)]2. That inequality is a case of Jensen's inequality, although it may also be shown to follow instantly from the frequently mentioned fact that
The more general version of the Rao–Blackwell theorem speaks of the "expected loss" or risk function:
where the "loss function" L may be any convex function. For the proof of the more general version, Jensen's inequality cannot be dispensed with.
The improved estimator is unbiased if and only if the original estimator is unbiased, as may be seen at once by using the law of total expectation. The theorem holds regardless of whether biased or unbiased estimators are used.
...
Wikipedia