Q-learning

Q-learning is a model-free reinforcement learning technique. Specifically, Q-learning can be used to find an optimal action-selection policy for any given (finite) Markov decision process (MDP). It works by learning an action-value function that ultimately gives the expected utility of taking a given action in a given state and following the optimal policy thereafter. A policy is a rule that the agent follows in selecting actions, given the state it is in. When such an action-value function is learned, the optimal policy can be constructed by simply selecting the action with the highest value in each state. One of the strengths of Q-learning is that it is able to compare the expected utility of the available actions without requiring a model of the environment. Additionally, Q-learning can handle problems with stochastic transitions and rewards, without requiring any adaptations. It has been proven that for any finite MDP, Q-learning eventually finds an optimal policy, in the sense that the expected value of the total reward return over all successive steps, starting from the current state, is the maximum achievable.

The problem model consists of an agent, states ${\displaystyle S}$ and a set of actions per state ${\displaystyle A}$. By performing an action ${\displaystyle a\in A}$, the agent can move from state to state. Executing an action in a specific state provides the agent with a reward (a numerical score). The goal of the agent is to maximize its total reward. It does this by learning which action is optimal for each state. The action that is optimal for each state is the action that has the highest long-term reward. This reward is a weighted sum of the expected values of the rewards of all future steps starting from the current state, where the weight for a step from a state ${\displaystyle \Delta t}$ steps into the future is calculated as ${\displaystyle \gamma ^{\Delta t}}$. Here, ${\displaystyle \gamma }$ is a number between 0 and 1 (${\displaystyle 0\leq \gamma \leq 1}$) called the discount factor and trades off the importance of sooner versus later rewards. ${\displaystyle \gamma }$ may also be interpreted as the likelihood to succeed (or survive) at every step ${\displaystyle \Delta t}$.

...
Wikipedia