Sunday, April 23, 2006

Haldane's Dilemma: should we worry?   posted by DavidB @ 4/23/2006 03:55:00 AM

In previous posts and comments I have mentioned Haldane's Dilemma, and I thought it would be worth looking into the subject more closely.

The short answer to the question, 'Should we worry?', is 'No'. But when was I ever satisfied with a short answer? The following is not intended as a casual read but (hopefully) a resource for those who are following up the subject via search engines. As usual, I claim no technical expertise in genetics, so don't take my assertions on trust. Links and references are provided for those who want to explore the subject further.

In a classic 1957 paper, J. B. S. Haldane estimated the number of generations of natural selection required to take an advantageous gene from rarity to near-fixation in a population. His headline conclusion was that it would normally take about 300 generations. More accurately, on average about one gene would be fixed in every 300 generations, so that if several genes were under selection simultaneously, the time required to fix each would be longer, but the number of genes fixed per unit of time would be the same. [Note 1] The total length of time required to fix T genes would therefore be approximately 300T generations. [Added: To avoid misunderstanding, I should have emphasised this last point. Haldane does not claim that every gene takes 300 generations to go from rarity to fixation. Suppose there is a long-established process of selection with a stream of 100 advantageous genes under selection at any time. Suppose each gene on average takes 30,000 generations to fix. If we take one of these genes at random, it therefore has a probability of 1 in 30,000 of being fixed in any given generation. But there are 100 genes to choose from, so the expected number of genes to be fixed in any given generation is 100 x 1/30,000 = 1/300. Equivalently, we expect on average 1 gene to be fixed in any 300 generations. This is what Haldane claims, based on his various assumptions.]

Based on the information available in 1957, Haldane estimated the number of gene differences between closely related species (not to be confused with the number of gene changes required for speciation) as typically about 1,000. It would therefore take about 300,000 generations for this amount of divergence to evolve by natural selection. Haldane thought this was broadly consistent with the fossil record, so he saw no 'dilemma'.

In the 1960s new evidence showed more genetic diversity within and between species than was previously assumed. It therefore became problematic that natural selection seemed either too slow to explain the observed diversity or too costly in mortality for species to survive. Hence the 'dilemma'. Motoo Kimura used Haldane's figures to support his own theory of molecular evolution, maintaining that a large proportion of change at the molecular level is not due to selection but to genetic drift. In the resulting debate geneticists picked holes in Haldane's analysis, and showed that in some circumstances natural selection could be much quicker than Haldane had claimed. (Haldane died in 1964, so he took no part in this debate.) By the mid-1970s it was generally accepted that Haldane's Dilemma was not a serious problem. [Note 2]

The issue was reopened in 1992 by George C. Williams in his book Natural Selection: Domains, Levels and Challenges, where he argued that Haldane's Dilemma had not been solved, but 'merely faded away, because people got interested in other things'. The subject has also taken a bizarre twist in recent years as Haldane's Dilemma has been seized on by Creationists as an objection to evolution. (Searching the Web for 'Haldane's Dilemma', a large proportion of the results will be Creationist websites. A key Creationist example is here, and an evolutionist response is here.) This raises a dilemma of a different kind, as there is a danger that anyone who takes Haldane's Dilemma seriously will be misrepresented as a crypto-Creationist, or misquoted to give support to Creationism. So just to be clear: I do not think Haldane's Dilemma is a major problem for evolutionary theory. On the other hand, I do think that George C. Williams raised some interesting points. The real value of the Dilemma is now not so much to set any firm limit to the rate of evolution, as to focus attention on important questions about how natural selection works...

Haldane's 1957 paper itself is available here. The arguments and calculations are not easy to follow, as the paper requires some prior knowledge of population genetics, and makes a lot of simplifying assumptions and approximations. There is also at least one serious misprint [note 3]. I will give my own outline of Haldane's argument, not always using Haldane's own notation.

First Haldane assumes that a population is in equilibrium with its environment, and that most members of the population have the best available genes (alleles) at most loci. Disadvantageous alleles are present at low frequencies (around 1 in 20,000 for dominants and 1 in 100 for recessives) due to a balance between selection and mutation.

The environment then changes so that some previously advantageous alleles are disadvantageous and vice versa. The newly disadvantageous alleles will each have a selective fitness detriment, s, relative to its advantageous alternative, where the s's may be different for each allele. Any individual lucky enough to have the best alleles at all loci has fitness 1 (by stipulation), while the bearers of disadvantageous alleles have fitness (1 - s)(1 - s)(1 - s), etc, with the appropriate s's for these alleles. Note that this assumes that the fitness effects of the alleles at different loci are independent, so that fitness is simply multiplicative. (If all the s's are small, fitness is also approximately additive, as the product-terms involving more than one s can be neglected.) This assumption of independent fitness effects is not discussed by Haldane.

Haldane assumes for simplicity that all selection occurs by way of mortality in the juvenile stage, i.e. before reproduction. If the population at the time of selection is N, and the frequency of a disadvantageous allele is q, there will be a certain number of selective deaths attributable to the allele in that generation, depending on the size of N, q, s, and the genetic characteristics of the allele (e.g. dominance). The selective deaths reduce the population size, but we assume that it is restored to N in the next generation, an assumption not discussed by Haldane [note 4]. In the new generation the frequency of the disadvantageous allele is reduced, while that of the advantageous alternative is increased. By iterating the process of selection, and assuming that the selective advantage remains constant, we can calculate D, the total number of selective deaths, expressed as a multiple of N, required to increase the frequency of the advantageous allele from rarity to near-fixation. Provided s is small, the problem can be approximately solved by integration. The technical part of Haldane's paper is concerned with performing these integrations for a variety of scenarios: haploid, diploid, partly recessive, sex-linked, and so on. The important conclusion is that over a wide range of parameters the resulting value of D does not vary drastically, though it is more sensitive to variations in the initial frequency q than in the fitness detriment s. For diploids D is usually between 10 and 100, with 30 as a representative value. This means that the total number of selective deaths required to take a single advantageous allele from rarity to near-fixation is about 30 times the population size. The process will therefore normally take at least 30 generations, but probably more: the best estimate remaining to be determined.

This problem is taken up in the Discussion section of Haldane's paper. The figure to be determined is the number of generations required for a single gene substitution. Call this K. As the total number of selective deaths required is about 30 times the population size, the number of selective deaths required per generation is about 30/K times the population size. However, usually more than one allele will be under selection at the same time. If each such allele involves d selective deaths per generation, then provided all the d's are small, so that there is no significant overlap in the selective deaths attributable to different alleles, their sum, Sd, is approximately the total of selective deaths per generation. (Strictly the d's and Sd are proportions of the population, rather than absolute numbers, but it is tedious to keep repeating this.) Since each gene substitution requires selective deaths totalling about 30 times the population size, Sd selective deaths can therefore be regarded as 'paying for' Sd/30 gene substitutions per generation. Equivalently, one gene substitution will on average occur every 30/Sd generations. But by definition this is also the desired number K, so we have K = 30/Sd. We can therefore estimate K if we know Sd. Here Haldane makes the assumption that the proportion of all mortality that is due to selection will usually be about 1/10, though occasionally as high as 1/2. (In most species under natural conditions, there are a lot of random, non-selective deaths, so Haldane's assumption is not unreasonable.) Taking 1/10 as typical, we therefore have K = 30/[1/10] = 300. Hence we get the headline figure of 300 generations required for a gene substitution.

Given the assumption that fitness effects are independent, the selective deaths which 'pay for' an increase in the frequency of one advantageous allele are neutral with respect to all other alleles. Provided the selective advantage for each allele is not very large (say, not more than 10%), there will be little overlap in the selective deaths which pay for different advantageous alleles, so that the cost of each allele must be paid in full by 300 generations worth of selective deaths. [Note 5] (Haldane does not spell this out, but it is worth making it explicit.)

There are obviously a lot of approximations and simplifications in all this, and Haldane himself mentions cases, such as colonisation of a new territory, where selection might be quicker than average. But given Haldane's key assumptions, his reasoning is generally accepted as sound. Criticism has therefore focused on the validity of the assumptions. A wide variety of objections and alternatives have been raised, some of which seem to me persuasive, others less so. A useful review in 1974 by V. Grant and R. Flake is available here. My own layman's assessment is as follows:

1. The weakest of Haldane's assumptions is that fitness effects are independent. This is crucial for his overall conclusions, so it is unfortunate that Haldane does not discuss it! A simple example shows how selection can be much faster with non-independent fitness. (I found this example in an article by Nick Barton and Linda Partridge, but they do not claim it as original.) Most of the cost of selection is incurred when an advantageous allele is rare. Suppose then that there are a number of rare advantageous alleles in a population. Suppose also that a proportion W of the population survives and breeds in each generation, and that all those individuals bearing any of the advantageous alleles are among the survivors. In this model all of the advantageous alleles will simultaneously increase to a frequency 1/W times their previous frequency in a single generation, so, for example, if W = 2/3, the frequency of each allele will jump to 1/[2/3] = 1.5 times its former frequency. There is no limit to the number of alleles that can be selected at this rate simultaneously, provided their total frequency in the population does not exceed W. In this simple model every selective death simultaneously helps pay for all the advantageous alleles, contrary to Haldane's assumption of independent fitness.

A somewhat more complicated model was examined by John Maynard Smith in a 1968 paper. In this model JMS assumes that all those individuals with more than an average number of advantageous alleles survive and breed, while all those below this threshold die. JMS finds that with this assumption, of the order of 1,000 times as many alleles can be selected simultaneously, for a given overall intensity of selection, as in Haldane's model. This implies that instead of on average 1 gene being fixed every 300 generations, there could be several genes fixed in each generation.

These examples suffice to show that selection much faster than Haldane supposed is possible. However, to show that something is possible is not the same as showing that it actually happens. One of George C. Williams's main arguments in reviving Haldane's Dilemma was that in practice the assumption of independent fitness effects was often a 'realistic approximation', and that more empirical research was needed into the actual patterns of selection. There may be some circumstances where Haldane's assumption of independent fitness is reasonable. There may be others where fitness is not independent but the pattern of selection is not as favourable to rapid evolution as in the simple models described above. In a critique of Maynard Smith's paper, P. O'Donald argued that a simple threshold model was unrealistic, and that it was more plausible that the fitness of a polygenic trait should be proportional to the square of its deviation from the optimum value. With a model based on this assumption, O'Donald found that selection could work somewhat faster than in Haldane's model, but nowhere near as fast as in Maynard Smith's.

2. In discussing Haldane's Dilemma a distinction is often made between hard and soft selection. As introduced and defined by Bruce Wallace, a hard selective disadvantage applies regardless of the density of population. Soft selection, on the other hand, does not apply until the population exceeds a certain threshold of density. It is argued that Haldane's model presuppose hard selection, and that it does not apply where soft selection predominates.

It is true that Haldane's illustrative scenario assumes hard selection, but his calculations would apply to any circumstances where the pattern of fitness satisfies his assumptions. Frequency-dependent selection might in principle do so. The real point of the soft-selection argument is that, according to Wallace, in soft selection the fitness of individuals is decided by their place in a rank order of fitness. At any given time there is a fixed number N of 'places' available for the population, determined by ecological factors, so that the first N individuals in order of fitness (allowing for random mortality) will survive, and the rest will die. It is evident that this is another version of threshold selection, as in Maynard Smith's model, and the same pros and cons apply.

3. It is sometimes suggested that Haldane's arguments do not apply to new advantageous mutations. A new advantageous mutation does not cause any deaths except to the extent that its bearers gradually squeeze out non-bearers (assuming a fixed population size). The maximum cumulative number of deaths caused by this 'squeezing out' can be no more than the total population size, and not 30 times the population size, as in Haldane's model.

I think this objection is largely misconceived. It is true that a new advantageous mutation cannot cause more individuals to die than the total population size, but it does require many more selective deaths than this.

To understand this paradoxical assertion, consider the following example. Suppose that in every generation 2% of a population dies from spells of extreme cold, against which the species has no defence. These deaths will be counted as part of random, non-selective mortality. Suppose now that a new dominant mutation gives its bearers partial protection against cold spells, so that only 1% instead of 2% of the bearers die from this cause. The bearers will therefore have a 1% fitness advantage over the non-bearers, and will gradually increase in frequency. During this process a large number of selective deaths will be attributable to the new mutation - usually more than 30 times the population size, because a new mutation probably starts from a greater rarity in the population than in Haldane's model. There are no more deaths from cold than before - in fact there are fewer - but deaths previously counted as non-selective are now selective.

So what? Does it matter if deaths are transferred from the 'non-selective' to the 'selective' category? If we are only considering a single gene, it probably does not matter, but if we are considering the overall scope for natural selection, it does. The problem is that there are only a certain number of deaths available to pay for natural selection. The number cannot exceed the total population size in any generation, and it will normally be much less than this, as there is bound to be a large amount of random mortality. The more mutations there are at any time, the fewer the deaths available to pay for each (assuming independent fitness). There is no free lunch: advantageous mutations still have to be paid for, and the 'funds' available, in terms of the scope for selective deaths, are limited. The problem is not that new advantageous mutations somehow make things worse - obviously not - but what they imply about the previous state of the species. If a large number of new advantageous mutations are under rapid selection, without any change in the environment, this suggests that the species was very badly adapted to its environment before. So how did it survive? One answer to this may be that the 'environment' of a species consists largely of other organisms - competitors, predators, prey, parasites, and so on - and they are all evolving together. At any given time a species may be adequately adopted to its organic environment, but if it does not keep evolving it will fall behind.

This may be true, but it does not seem particularly relevant to the case of strong selection of a large number of new mutations simultaneously. This would imply that the other species (competitors, etc) had somehow obtained a major advantage, so that the species under selection had to catch up. It isn't clear how this situation would arise.

4. An initially attractive answer to Haldane's Dilemma is that it only applies to selection due to differential mortality, and not to differential fertility. If some genotypes have more offspring than others, there is no need for selective deaths at all. The worst that can happen is that if there is a fixed population size, some individuals will be forced to have fewer or no offspring. But the cumulative total of individuals forced to be childless will not exceed the total population size. Selection by differential fertility is especially attractive when a gene is rare, because in these circumstances it is much less wasteful than selection by differential mortality. When an advantageous gene is rare, selection by mortality requires a fraction 1/x of the entire population to die just to raise the frequency of the small number of advantageous genes by roughly the same proportion (if selection is not very strong). In contrast, selection by fertility only requires the rare bearers of the advantageous gene each to have on average 1/x more offspring to achieve the same effect. This is much more efficient, from the point of view of the species a whole. Unfortunately, there is no reason to expect the evolutionary process to be efficient for the species as a whole.

Strictly speaking, Haldane's model does allow for selection by differential fertility. Haldane several times refers to diminished fertility as being equivalent to selective mortality. There is however an important difference. In the case of selective mortality, a certain number of actual deaths must occur. In contrast, 'diminished fertility' is measured relative to the fertility of the optimum genotype, which may be entirely hypothetical. If several advantageous genes are still rare, there will probably be no actual individual who has all of them. Whereas real deaths in any generation cannot exceed the actual population size, there is no upper limit to a hypothetical loss of fertility. Each new advantageous allele increases the fertility of the optimum genotype, against which 'diminished fertility' is measured. The 'cost' of selection may be largely theoretical. It does therefore seem that if we allow selection to be largely by differential fertility, there is no real constraint on the speed of evolution.

I think there is something to this argument, but that we should not be too optimistic about the scope for selection by differential fertility. The reasons for differences in fertility (number of offspring) of individuals in a species may be divided broadly into two categories:

a) differences in the resources the individual can devote to reproduction. This is turn depends on its general condition, and especially on its health and nutritional status;

b) differences in the number of offspring produced from a given amount of resources.

Differences of the first kind depend on much the same factors as differences in mortality, such as the ability to find food, to avoid predators, and to resist disease. They are therefore not really an alternative to selection by mortality, and differences in fertility of this kind, in so far as they have a genetic basis, are likely to be accompanied by selective deaths on much the same scale as in Haldane's model.

The second category of differences falls within the field of 'life history strategy'. There is a trade-off between the number of offspring produced and their average survival rate. We expect there to be an optimum fertility level for each species, determined by its ecology and behaviour. Any mutations which cause an individual to produce more offspring than the current optimum will in general be selected against, except in the special circumstance that the mutation itself changes the optimum, for example by increasing the efficiency of parental care. To expect it to be easy to increase fertility is to expect another free lunch.

5. A more technical criticism is of Haldane's measure of fitness. Haldane stipulates that the optimum genotype that can be formed from the available genes has fitness 1. But if some advantageous alleles are still rare, in a finite population the optimum genotype is unlikely to have any representatives, and the fitness of all existing individuals may appear very low by Haldane's measure. It has therefore been argued that fitness should be measured by reference to the best genotype available in practice. It is further argued that with this approach, the cost of selection is much lower, and that several alleles can be fixed in each generation, even if all the other assumptions of Haldane's model are adopted.

I have not followed all the details of this argument, but it has been used by some distinguished geneticists, such as Warren Ewens, so I am sure they have proved something. But I do not see how it can solve Haldane's Dilemma. Haldane's argument does not depend crucially on how fitness is measured. His key assumptions are:

a) selection is by means of differential mortality

b) selection at different loci is independent

c) selection at each locus is not very strong

d) there is a substantial amount of non-selective mortality.

From assumption (a) it seems inevitable that fixation of any single allele will require selective deaths cumulatively totalling several times the population size, and assumptions (b) - (d) entail that the cost of fixing more than one allele will be approximately the sum of the costs of fixing each. Setting a different benchmark for fitness would not help. I suspect that any findings to the contrary involve some de facto relaxation of the 'independence' assumption.

Overall, it seems to me that Haldane's Dilemma has more force than is sometimes assumed, and that it, or rather the issues it raises, should be taken seriously. Whenever it is suggested that a large number of genes have been under selection simultaneously, we need to look at what this implies for the intensity and mode of selection. This applies also when the selection of entities other than genes, such as social groups or cultural traits, is under consideration. For example, if it is suggested that cultural traits have evolved by the differential survival of societies possessing those traits, we need to take a hard look at what this implies for the number of group 'extinctions' and the number of cultural traits that might feasibly evolve in the way suggested.

At the same time, the Dilemma does not set a rigid limit to the speed of evolution. If we give up the assumption of independent fitness, evolution can go at any speed we like. Selection by means of differential fertility may also offer a way out of the Dilemma, though I have argued above that it is not a panacea.

It remains a matter for empirical research whether rapid multi-locus selection is a common evolutionary phenomenon, and if so what is the selective mechanism that permits it. This also has implications for other problems in evolutionary biology. For example, some theories of the advantages of sexual reproduction (e.g. Kondrashov's) require patterns of synergistic fitness effects of a kind that might also solve Haldane's Dilemma. The Dilemma also seems relevant to the theory of punctuated equilibrium, where it is claimed that most evolutionary change occurs in relatively short bursts in between long periods of stasis. Yet Haldane's 1957 paper is not mentioned in S. J. Gould's vast book on The Structure of Evolutionary Theory. Maybe the time is right for some joined-up thinking?

Note 1: some commentators have misinterpreted Haldane as arguing that only one gene can be under selection at a time, at intervals of 300 generations! (see for example this Wikipedia entry). This is complete nonsense, and a travesty of Haldane's paper.

Note 2: not all biologists dismissed the problem. For example, John Maynard Smith, in his 1978 book The Evolution of Sex, gave it serious attention.

Note 3: on page 520 there is a formula which may be written as n = 30 Sd, where n is the number of generations required for a gene substitution (my K), S is a summation sign (Haldane's large sigma), and d is the average fitness effect of each gene under selection (Haldane's small delta). I think this formula must be a misprint for n = 30/Sd. The formula as printed makes no sense, as it implies that substitution takes longer when selection is more intense! The correction n = 30/Sd is consistent with the rest of Haldane's working, and also with the exposition by John Maynard Smith, who gives a formula equivalent to Sd = 30/n.

Note 4: the assumption is not explicit in the 1957 paper, but it seems to be required for Haldane's calculations, and in a paper of 1960, describing his 1957 conclusions, Haldane referred explicitly to 'a population of constant size'. This requirement can be satisfied if we assume that there is always a surplus of offspring sufficient to cover 'selective' mortality, with the surplus being reduced by random mortality to a constant carrying capacity. Some commentators have seen the assumption of constant population size as an inconsistency or major flaw in Haldane's analysis, but R. Flake and V. Grant showed in 1974 that Haldane's main conclusions still hold good for varying population size and varying intensity of selection.

Note 5: If selection on each allele is strong, overlap of selection for different alleles may become significant, and reduce the overall number of selective deaths required. To take an extreme example, if all individuals who lack any of the rare advantageous alleles die, then the only survivors will be those who bear all of those alleles, and in a single generation they will all be fixed, at a cost of slightly less than 1 times the population size. The snag is that there probably will not be any individuals with all those alleles, so the species will go extinct! To take a less extreme example, if there are two rare advantageous alleles, A and B, each with a 50% selective advantage, then in each generation 50% of non-As and 50% of non-Bs will be killed, but around 25% of the population will be 'killed' by both factors of selection, so the total selective mortality will be around 75%, not 100%. Even if all the s's are individually small, overlap will become significant if there are enough alleles under selection, but this will not occur unless the sum of the s's is relatively large, which again implies likely extinction.


J. B. S. Haldane: The cost of natural selection, Journal of Genetics, 55, 1957, 511-24.
J. Maynard Smith: 'Haldane's Dilemma' and the rate of evolution, Nature, 219, 1968, 1114-6.
P. O.Donald: 'Haldane's Dilemma' and the rate of natural selection, Nature, 221, 1969, 815-6.
B. Wallace: Genetic Load, 1970.