In the field of mathematics known as functional analysis, the invariant subspace problem is a partially unresolved problem asking whether every bounded operator on a complex Banach space sends some non-trivial closed subspace to itself. The original form of the problem as posed by Paul Halmos was in the special case of polynomials with compact square. This was resolved affirmatively, for a more general class of polynomially compact operators, by Allen R. Bernstein and Abraham Robinson in 1966 (see Non-standard analysis#Invariant subspace problem for a summary of the proof).
More formally, the invariant subspace problem for a complex Banach space H of dimension > 1 is the question whether every bounded linear operator T : H → H has a non-trivial closed T-invariant subspace (a closed linear subspace W of H which is different from {0} and H such that T(W) ⊆ W).
To find a counterexample to the invariant subspace problem means to answer affirmatively the following equivalent question: Does there exist a bounded linear operator T : H → H such that for every non-zero vector x, the vector space generated by the sequence {T n(x) : n ≥ 0} is norm dense in H? Such operators are called cyclic.