Markov process

In probability theory and related fields, a Markov process is a specific type of a mathematical object known as a stochastic or random process, which is usually defined as a collection of random variables. Historically, the random variables were associated with or indexed by some set of numbers, usually viewed as time, giving the interpretation of a stochastic process representing numerical values of some random system evolving over time, such as an electrical current fluctuating due to thermal noise or the movement of a gas molecule. Markov processes are stochastic processes that have the property that the next value of the process depends on the current value, but it is conditionally independent of the previous values of the stochastic process – in other words, the behavior of the process in the future is stochastically independent of its behavior in the past, given the current state of the process. For a stochastic process to be a Markov process, it must have this property, which is referred to as the Markov property The adjective Markovian is used to describe something that is related to a Markov process.

A Markov chain is a type of Markov process that has a particular type of state space, which dictates the possible values that a stochastic process can take, or index set, which often represents time, but the precise definition of a Markov chain varies. For example, it is common to define a Markov chain as a Markov process in either discrete or continuous time with a discrete state space, so its state space is countable, with the natural numbers and integers being examples of such a state space. In other words, this definition implies that Markov chains are Markov processes with discrete state spaces regardless of the nature of time. But it is also common to define a Markov chain as having discrete time regardless of the state space. Consequently, a Markov chain is a Markov process that must have either a discrete state space, a discrete index set or both.

...
Wikipedia