"Turtles all the way down" is an expression of the infinite regress problem in cosmology posed by the "unmoved mover" paradox. The metaphor in the anecdote represents a popular notion of the model that Earth is actually flat and is supported on the back of a World Turtle, which itself is propped up by a column of turtles. Questioning what the final turtle might be standing on, the anecdote humorously concludes that it is "turtles all the way down".
The expression is an illustration of the concept of Anavastha in Indian philosophy, and refers to the defect of infinite regress in any philosophical argument. Contrary to most extant western references, it is not a popular Hindu belief. Rather, it is a widely accepted principle in Indian philosophy, commonly used to reject arguments for a creator God or "unmoved mover".
The phrase has been commonly known since at least the early 20th century. A comparable metaphor describing the circular cause and consequence for the same problem is the "chicken and egg problem". The same problem in epistemology is known as the Münchhausen trilemma.
The origins of the turtle story are uncertain. It has been recorded since the mid 19th century, and may possibly date to the 18th. One recent version appears in Stephen Hawking's 1988 book A Brief History of Time, which starts:
A well-known scientist (some say it was Bertrand Russell) once gave a public lecture on astronomy. He described how the earth orbits around the sun and how the sun, in turn, orbits around the center of a vast collection of stars called our galaxy. At the end of the lecture, a little old lady at the back of the room got up and said: "What you have told us is rubbish. The world is really a flat plate supported on the back of a giant tortoise." The scientist gave a superior smile before replying, "What is the tortoise standing on?" "You're very clever, young man, very clever," said the old lady. "But it's turtles all the way down!"
Hawking's suggested connection to Russell may be due to Russell's 1927 lecture Why I Am Not a Christian. In it, while discounting the First Cause argument intended to be a proof of God's existence, Russell comments: