In algebra, a transformation semigroup (or composition semigroup) is a collection of functions from a set to itself that is closed under function composition. If it includes the identity function, it is a monoid, called a transformation (or composition) monoid. This is the semigroup anologue of a permutation group.
A transformation semigroup of a set has a tautological semigroup action on that set. Such actions are characterized by being effective, i.e., if two elements of the semigroup have the same action, then they are equal.
An analogue of Cayley's theorem shows that any semigroup can be realized as a transformation semigroup of some set.
In automata theory, some authors use the term transformation semigroup to refer to a semigroup acting faithfully on a set of "states" different from the semigroup's base set. There is a correspondence between the two notions.
A transformation semigroup is a pair (X,S), where X is a set and S is a semigroup of transformations of X. Here a transformation of X is just a function from X to itself, not necessarily invertible, and therefore S is simply a set of transformations of X which is closed under composition of functions. If S includes the identity transformation of X, then it is called a transformation monoid. Obviously any transformation semigroup S determines a transformation monoid M by taking the union of S with the identity transformation. A transformation monoid whose elements are invertible is a permutation group.