Coordinate descent

Coordinate descent is a derivative-free optimization algorithm. To find a local minimum of a function, one does line search along one coordinate direction at the current point in each iteration. One uses different coordinate directions cyclically throughout the procedure.

Coordinate descent is based on the idea that the minimization of a multivariable function ${\displaystyle F(\mathbf {x} )}$ can be achieved by minimizing it along one direction at a time, i.e., solving univariate (or at least much simpler) optimization problems in a loop. In the simplest case of cyclic coordinate descent, one cyclically iterates through the directions, one at a time, minimizing the objective function with respect to each coordinate direction at a time. That is, starting with initial variable values

round ${\displaystyle k+1}$ defines ${\displaystyle \mathbf {x} ^{k+1}}$ from ${\displaystyle \mathbf {x} ^{k}}$ by iteratively solving the single variable optimization problems

...
Wikipedia