Free energy principle

The free energy principle tries to explain how (biological) systems maintain their order (non-equilibrium steady-state) by restricting themselves to a limited number of states. It says that biological systems minimise a free energy functional of their internal states, which entail beliefs about hidden states in their environment. The implicit minimisation of variational free energy is formally related to variational Bayesian methods and was originally introduced by Karl Friston as an explanation for embodied perception in neuroscience, where it is also known as active inference.

The notion that self-organising biological systems – like a cell or brain – can be understood as minimising variational free energy is based upon Helmholtz’s observations on unconscious inference and subsequent treatments in psychology and machine learning. Variational free energy is a function of some outcomes and a probability density over their (hidden) causes. This variational density is defined in relation to a probabilistic model that generates outcomes from causes. In this setting, free energy provides an (upper bound) approximation to Bayesian model evidence. Its minimisation can therefore be used to explain Bayesian inference and learning. When a system actively samples outcomes to minimise free energy, it implicitly performs active inference and maximises the evidence for its (generative) model.

However, free energy is also an upper bound on the self-information (or surprise) of outcomes, where the long-term average of surprise is entropy. This means that if a system acts to minimise free energy, it will implicitly place an upper bound on the entropy of the outcomes – or sensory states – it samples.

Active inference is closely related to the good regulator theorem and related accounts of self-organisation, such as self-assembly, pattern formation and autopoiesis. It addresses the themes considered in cybernetics, synergetics and embodied cognition. Because free energy can be expressed as the expected energy (of outcomes) under the variational density minus its entropy, it is also related to the maximum entropy principle. Finally, because the time average of energy is action, the principle of minimum variational free energy is a principle of least action.

...
Wikipedia