Epipolar geometry

Epipolar geometry is the geometry of stereo vision. When two cameras view a 3D scene from two distinct positions, there are a number of geometric relations between the 3D points and their projections onto the 2D images that lead to constraints between the image points. These relations are derived based on the assumption that the cameras can be approximated by the pinhole camera model.

The figure below depicts two pinhole cameras looking at point X. In real cameras, the image plane is actually behind the focal center, and produces an image that is the symmetry about the focal center of the lens. Here, however, the problem is simplified by placing a virtual image plane in front of the focal center i.e optical center of each camera lens to produce an image not transformed by the symmetry. O_L and O_R represent the centers of symmetry of the two cameras lenses. X represents the point of interest in both cameras. Points x_L and x_R are the projections of point X onto the image planes.

Each camera captures a 2D image of the 3D world. This conversion from 3D to 2D is referred to as a perspective projection and is described by the pinhole camera model. It is common to model this projection operation by rays that emanate from the camera, passing through its focal center. Note that each emanating ray corresponds to a single point in the image.

Since the optical centers of the cameras lenses are distinct, each center projects onto a distinct point into the other camera's image plane. These two image points are denoted by e_L and e_R and are called epipoles or epipolar points. Both epipoles e_L and e_R in their respective image planes and both optical centers O_L and O_R lie on a single 3D line.

The line O_L–X is seen by the left camera as a point because it is directly in line with that camera's lens optical center. However, the right camera sees this line as a line in its image plane. That line (e_R–x_R) in the right camera is called an epipolar line. Symmetrically, the line O_R–X seen by the right camera as a point is seen as epipolar line e_L–x_Lby the left camera.

...
Wikipedia