Outbreeding depression

In biology, outbreeding depression is when progeny resulting from crosses between genetically distant individuals (outcrossing) exhibit lower fitness in the parental environment than either of their parents, or than progeny from crosses between individuals that are more closely related. The concept is opposed to inbreeding depression, although the two effects can occur simultaneously.Outbreeding depression manifests most significantly in two ways:

The different mechanisms of outbreeding depression can operate at the same time. However, determining which mechanism is more important in a particular population is very difficult. Generally the first mechanism will be more prevalent in the first generation (F1) after the initial outcrossing when most individuals are made up of the intermediate phenotype. An extreme case of this type of outbreeding depression is the sterility and other fitness-reducing effects often seen in interspecific hybrids (such as mules), which involves not only different alleles of the same gene but even different orthologous genes.

The second mechanism may not appear until two or more generations later (F2 or greater), when recombination has undermined vitality positive epistasis. Hybrid vigor in the first generation can, in some circumstances, be strong enough to mask the effects of outbreeding depression. An example of this is that plant breeders will make F1 hybrids from purebred strains, which will improve the uniformity and vigor of the offspring, however the F1 generation are not used for further breeding because of unpredictable phenotypes in their offspring. Unless there is strong selective pressure, outbreeding depression can increase in further generations as co-adapted gene complexes are broken apart without the forging of new co-adapted gene complexes to take their place.

...
Wikipedia