Assuming you haven’t been sleeping under a rock, you have probably heard that a Nature paper came out on an F1 Neanderthal-Denisovan hybrid. The major new science in my opinion from the results of the genome itself is to be found in the figure above. It confirms that there was a lot of population turnover among Neanderthals, as this individual’s mother is more closely related to European Neanderthals who flourished ~40,000 years later than conspecifics from the same region 30,000 years earlier. This is not surprising in light of what we know about the genetics and paleoecology of this group, though it confirms what we know and increases our confidence.

Rather, what is surprising is that this paper was published because they found an F1. From their conclusion:

It is notable that one direct offspring of a Neanderthal and a Denisovan (Denisova 11) and one modern human with a close Neanderthal relative (Oase 1) have been identified among the few individuals from whom DNA has been retrieved and who lived at the time of overlap of these groups…In conjunction with the presence of Neanderthal and Denisovan DNA in ancient and present-day people…this suggests that mixing among archaic and modern hominin groups may have been frequent when they met.

The number of ancient genomes from these species/groups/lineages is literally in the range a handful. And among the early finds is an F1! This seems highly unlikely. It could be a fluke. Or, as inferred above, F1’s may have been very common when different hominin lineages met.

But that makes one ask: how is it that Neanderthals and Denisovans remained some genetically distinct over hundreds of thousands of years? The two reasons offered are that the lineages were geographically very distant from each other on the whole, and, that hybrid individuals had very low fitness. I think the former is the primary dynamic to focus on.

For my assertion to make sense, consider some context in the published literature and theory. From 2004 and 2011 respectively, Modern Humans Did Not Admix with Neanderthals during Their Range Expansion into Europe and Strong reproductive isolation between humans and Neanderthals inferred from observed patterns of introgression.

From the first paper:

…we estimate that maximum interbreeding rates between the two populations should have been smaller than 0.1%. We indeed show that the absence of Neanderthal mtDNA sequences in Europe is compatible with at most 120 admixture events between the two populations despite a likely cohabitation time of more than 12,000 y. This extremely low number strongly suggests an almost complete sterility between Neanderthal females and modern human males, implying that the two populations were probably distinct biological species.

And the second:

Recent studies have revealed that 2–3% of the genome of non-Africans might come from Neanderthals, suggesting a more complex scenario of modern human evolution than previously anticipated. In this paper, we use a model of admixture during a spatial expansion to study the hybridization of Neanderthals with modern humans during their spread out of Africa. We find that observed low levels of Neanderthal ancestry in Eurasians are compatible with a very low rate of interbreeding (<2%), potentially attributable to a very strong avoidance of interspecific matings, a low fitness of hybrids, or both.

Models are models, and they have assumptions. Don’t have the player, hate the model assumption and revisit your priors.

There are 22 ancient genomes from 40,000 years ago or before. One of them is an F1 between Neanderthals and Denisovans. And another, Oase 1, has a Neanderthal in their very recent ancestry. The sampling locations may not be totally representative. The Denisova cave is likely to be special because it’s at the nexus of the ranges of the two Eurasian archaic lineages. But with that out of the way, it seems very unlikely to me that very low fitness or very low likelihood of mating when it close contact is the reason that the lineages remained distinct. After less than half a dozen samples from Denisova, cave researchers hit on an F1. What are the chances?

And yet, if matings between the lineages occurred when they were in close contact, and they were genetically distinct nevertheless over such long periods, then that demands an explanation. Denisova hominins and Neanderthals were genetically closer than modern humans are to either. At the time that F1 was conceived the two lineages had been distinct for ~300,000 years. This is not qualitatively much longer than some modern human groups (e.g., Khoisan vs. everyone else) have been diverging. And yet, like the Denisovan-Neanderthal split, modern humans have a lot of population structure and evidence of isolation (also, note that modern humans show no evidence of reduced reproductive fitness from offspring and purification of admixture, as has been inferred for Neanderthal genomic regions in modern human genomes).

All this leads me to conclude that in Pleistocene hominins allopatry and metapopulation dynamics are the solutions to this quandary. The population density of archaic hominins was on average low, but you need to go beyond average. The distribution was possibly highly patchy and with large zones of little habitation. Gene flow across populations may have occurred, but they would run up to a wall of emptiness equivalent to the Atlantic ocean. Additionally, both Neanderthal and modern human ancient indicates a recurrent pattern of location population extinction and replacement. My hypothesis is that populations which were liminal to the range of both lineages, and so likely to have a higher load of admixture from the other lineage, were also in a marginal territory and most likely to go extinct and leave no descendants. Then, less admixed populations with larger numbers close to the core of the lineage range would repopulate the liminal region.

If the model is correct, I think the Altai was resettled by Neanderthals from the west after the Eemian interglacial.

A contrasting method to maintain genetic separation from allopatry (physical distance and barrier) are group cultural identities which maintain very strict endogamy. We see this over 2,000 years in India, where populations are co-localized but almost totally unrelated in any way you’d predict from geography. But 2,000 years is a blink of an eye geologically. The explanation for why Neanderthals and Denisovans, and various African human lineages, remained separate for hundreds of thousands of years as coherent populations despite some gene flow on the margins, has to be geology, geography and ecology. Domains where hundreds of thousands years of stasis on quite possible.