Information geometry

Information geometry is a branch of mathematics that applies the techniques of differential geometry to the field of probability theory. This is done by taking probability distributions for a statistical model as the points of a Riemannian manifold, forming a statistical manifold. The Fisher information metric provides the Riemannian metric.

Information geometry reached maturity through the work of Shun'ichi Amari and other Japanese mathematicians in the 1980s. Amari and Nagaoka's book, Methods of Information Geometry, is cited by most works of the relatively young field due to its broad coverage of significant developments attained using the methods of information geometry up to the year 2000. Many of these developments were previously only available in Japanese-language publications.

The following introduction is based on Methods of Information Geometry.

Define an n-set to be a set V with cardinality $|V|=n$ $|V|=n$ . To choose an element v (value, state, point, outcome) from an n-set V, one needs to specify $\log _{b}n$ $\log _{b}n$ b-sets (default b=2), if one disregards all but the cardinality. That is, $I(v)=\log n$ $I(v)=\log n$ nats of information are required to specify v; equivalently, $I(v)=\log _{2}n$ $I(v)=\log _{2}n$ bits are needed.

...
Wikipedia