Convergence of random variables

Examples of convergence in distribution
Dice factory
Suppose a new dice factory has just been built. The first few dice come out quite biased, due to imperfections in the production process. The outcome from tossing any of them will follow a distribution markedly different from the desired uniform distribution. As the factory is improved, the dice become less and less loaded, and the outcomes from tossing a newly produced die will follow the uniform distribution more and more closely.
Tossing coins
Let $X n$ be the fraction of heads after tossing up an unbiased coin $n$ times. Then $X 1$ has the Bernoulli distribution with expected value $μ = 0.5$ and variance $σ 2 = 0.25$ . The subsequent random variables $X 2, X 3, ...$ will all be distributed binomially. As $n$ grows larger, this distribution will gradually start to take shape more and more similar to the bell curve of the normal distribution. If we shift and rescale $X n$ appropriately, then $\scriptstyle Z_{n}={\frac {\sqrt {n}}{\sigma }}(X_{n}-\mu )$ $\scriptstyle Z_{n}={\frac {\sqrt {n}}{\sigma }}(X_{n}-\mu )$ will be converging in distribution to the standard normal, the result that follows from the celebrated central limit theorem.
Graphic example
Suppose ${X i}$ is an iid sequence of uniform $U (-1, 1)$ random variables. Let $\scriptstyle Z_{n}={\scriptscriptstyle {\frac {1}{\sqrt {n}}}}\sum _{i=1}^{n}X_{i}$ $\scriptstyle Z_{n}={\scriptscriptstyle {\frac {1}{\sqrt {n}}}}\sum _{i=1}^{n}X_{i}$ be their (normalized) sums. Then according to the central limit theorem, the distribution of $Z n$ approaches the normal $N (0, 1 / 3)$ distribution. This convergence is shown in the picture: as $n$ grows larger, the shape of the pdf function gets closer and closer to the Gaussian curve.

Examples of convergence in probability
Height of a person
This example should not be taken literally. Consider the following experiment. First, pick a random person in the street. Let X be his/her height, which is ex ante a random variable. Then ask other people to estimate this height by eye. Let $X n$ be the average of the first n responses. Then (provided there is no systematic error) by the law of large numbers, the sequence X_n will converge in probability to the random variable X.
Archer
Suppose a person takes a bow and starts shooting arrows at a target. Let X_n be his score in n-th shot. Initially he will be very likely to score zeros, but as the time goes and his archery skill increases, he will become more and more likely to hit the bullseye and score 10 points. After years of practice the probability that he hit anything but 10 will be getting increasingly smaller and smaller and will converge to 0. Thus, the sequence $X n$ converges in probability to X = 10. Note that $X n$ does not converge almost surely however. No matter how professional the archer becomes, there will always be a small probability of making an error. Thus the sequence {X_n} will never turn stationary: there will always be non-perfect scores in it, even if they are becoming increasingly less frequent.

Examples of almost sure convergence
Example 1
Consider an animal of some short-lived species. We record the amount of food that this animal consumes per day. This sequence of numbers will be unpredictable, but we may be quite certain that one day the number will become zero, and will stay zero forever after.
Example 2
Consider a man who tosses seven coins every morning. Each afternoon, he donates one pound to a charity for each head that appeared. The first time the result is all tails, however, he will stop permanently. Let X₁, X₂, … be the daily amounts the charity received from him. We may be almost sure that one day this amount will be zero, and stay zero forever after that. However, when we consider any finite number of days, there is a nonzero probability the terminating condition will not occur.

In probability theory, there exist several different notions of convergence of random variables. The convergence of sequences of random variables to some limit random variable is an important concept in probability theory, and its applications to statistics and . The same concepts are known in more general mathematics as stochastic convergence and they formalize the idea that a sequence of essentially random or unpredictable events can sometimes be expected to settle down into a behaviour that is essentially unchanging when items far enough into the sequence are studied. The different possible notions of convergence relate to how such a behaviour can be characterised: two readily understood behaviours are that the sequence eventually takes a constant value, and that values in the sequence continue to change but can be described by an unchanging probability distribution.

"Stochastic convergence" formalizes the idea that a sequence of essentially random or unpredictable events can sometimes be expected to settle into a pattern. The pattern may for instance be

Some less obvious, more theoretical patterns could be

These other types of patterns that may arise are reflected in the different types of stochastic convergence that have been studied.

While the above discussion has related to the convergence of a single series to a limiting value, the notion of the convergence of two series towards each other is also important, but this is easily handled by studying the sequence defined as either the difference or the ratio of the two series.

For example, if the average of n independent random variables Y_i, i = 1, ..., n, all having the same finite mean and variance, is given by

then as n tends to infinity, $X n$ converges in probability (see below) to the common mean, μ, of the random variables Y_i. This result is known as the weak law of large numbers. Other forms of convergence are important in other useful theorems, including the central limit theorem.

...
Wikipedia