Male superiority at chess and science cannot be explained by statistical sampling arguments

Share on FacebookShare on Google+Email this to someoneTweet about this on Twitter

A new paper by Bilalic et al. (2009) (read the PDF here), tries to account for male superiority in chess by appealing to a statistical sampling argument: men make up a much larger fraction of chess players, and that the n highest extreme values — say, the top ranked 100 players — are expected to be greater in a large sample than in a small one. In fact, this explanation is only a rephrasing of the question — why are men so much more likely to dedicate themselves to chess.

Moreover, data from other domains where men and women are equally represented in the sample, or where it’s women who are overrepresented in the sample, do not support the hypothesis — men continue to dominate, even when vastly underrepresented, in domains that rely on skills that males excel in compared to females. I show this with the example of fashion designers, where males are hardly present in the sample overall but thrive at the elite level.

First, the authors review the data that male chess players really are better than female ones (p.2):

For example: not a single woman has been world champion; only 1 per cent of Grandmasters, the best players in the world, are female; and there is only one woman among the best 100 players in the world.

The authors then estimate the male superiority at rank n, from 1 to 100, using the entire sample’s mean and s.d., and the fraction of the sample that is male and female. Here is how the real data compare to this expectation (p.2):

Averaged over the 100 top players, the expected male superiority is 341 Elo points and the real one is 353 points. Therefore 96 per cent of the observed difference between male and female players can be attributed to a simple statistical fact — the extreme values from a large sample are likely to be bigger than those from a small one.

Therefore (p. 3):

Once participation rates of men and women are controlled for, there is little left for biological, environmental, cultural or other factors to explain. This simple statistical fact is often overlooked by both laypeople and experts.

Of course, this sampling argument doesn’t explain anything — it merely pushes the question back a level. Why are men 16 times more likely than women to compete in chess leagues? We are back to square one: maybe men are better at whatever skills chess tests, maybe men are more ambitious and competitive even when they’re equally skilled as women, maybe men are pressured by society to go into chess and women away from it. Thus, the question staring us in the face has not been resolved at all, but merely written in a different color ink.

The authors are no fools and go on to mention what I just said. They then review some of the arguments for and against the various explanations. But this means that their study does not test any of the hypotheses at all — aside from rephrasing the problem, the only portion of their article that speaks to which answer may be correct is a two-paragraph literature review. For example, maybe females on average perform poorer on chess-related skills, and so weed themselves out more early on, in the same way that males under 6’3 would be more likely to move on and find more suitable hobbies than basketball, compared to males above 6’3. Here is the authors’ response to this hypothesis (p. 3, my emphasis):

Whatever the final resolution of these debates [on "gender differences in cognitive abilities"], there is little empirical evidence to support the hypothesis of differential drop-out rates between male and females. A recent study of 647 young chess players, matched for initial skill, age and initial activity found that drop-out rates for boys and girls were similar (Chabris & Glickman 2006).

Well no shit — they removed the effect of initial skill, and thus how well suited you are to the hobby with no preparation, and so presumably due to genetic or other biological factors. And they also removed the effect of initial activity, and thus how enthusiastic you are about the hobby. And when you control for initial height, muscle mass, and desire to compete, men under 6’3 are no more or less likely to drop out of basketball hobbies than men over 6’3. How stupid do these researchers think we are?

So, this article really has little to say about the question of why men excel in chess or science, and it’s baffling that it got published in the Proceedings of the Royal Society. The natural inference is that it was not chosen based on how well it could test various hypotheses — whether pro or contra the Larry Summers ideas — but in the hope that it would convince academics that there is really nothing to see here, so just move along and get home because your parents are probably worried sick about you.

Now, let’s pretend to do some real science here. The authors’ hypothesis is that the pattern in chess or science can be accounted for by their statistical sampling argument — but of course, men dominate all sorts of fields, including where they’re about as equally represented in the pool of competitors, and even when they’re outnumbered in that pool. Occam’s Razor requires us to find a simple account of all these patterns, not postulating a separate one for each case. The simple explanation is that men excel in these fields due to underlying differences in genes, hormones, social pressures, or whatever.

The statistical sampling argument can only capture one piece of the pattern — male superiority where males make up more of the sample. Any of the non-sampling hypotheses, including the silly socio-cultural ones, at least are in the running for accounting for the big picture of male dominance regardless of their fraction of the sample.

To provide some data, I direct you to an analysis I did three years ago of male vs. female fashion designers. Here, I’ll consider “the sample of fashion designers” to be students at fashion schools since that’s what the data were. Fashion students are the ones who will make up the pool of fashion designers upon graduating. I included four measures of eminence: 1) being chosen to enter the Council of Fashion Designers of America, 2) having an entry in two major fashion encyclopedias, both edited by women (Who’s Who in Fashion, and The Encyclopedia of Clothing and Fashion), 3) having their collections listed on Vogue’s website, and 4) winning the highest award of the CFDA, the Perry Ellis awards for emerging talent.

The male : female ratio in the pool of fashion students is 1 : 13 at Parsons and 1 : 5.7 at FIT. So, the female majority in the sample of fashion designers is not quite as extreme as that of males in chess leagues, but pretty close. The statistical sampling argument predicts that females should out-number males at the top. But they don’t — the M : F ratios for the four measures above are, respectively, 1.29 : 1, 1.5 : 1 and 1.9 : 1, 1.8 : 1, and 3.6 : 1. Again, this isn’t as extreme as male superiority in chess, but recall that males are so underrepresented in the sample to begin with!

(For other design fields that males tend to have greater interest in, such as architecture, the M : F ratios among the winners of the Pritzker Prize and the AIA Gold Medal are, respectively, 27 : 1 and 61 : 0).

The authors statistical sampling argument is not a null hypothesis that we reject or fail to reject in any particular case — rejecting it in fashion design, and failing to reject in chess. It is not a hypothesis at all, but simply a rephrasing of the observation that men dominate certain fields, only measuring this by their greater participation rates. Again, it does not address why males are so much more likely to participate in chess leagues to begin with, which could be due to any of the existing hypotheses about male superiority. The point is that it is a widespread phenomenon that requires a single explanation applying across domains.

I find the genetic and hormonal influences on the mean and variance of cognitive ability and personality traits to be the most promising (just search our archives for relevant keywords to find the discussions). But this study of chess players offers nothing new to the debate, and could not do so even in principle, as it doesn’t make a novel hypothesis, apply a novel test to existing data, or apply existing tests on novel data. You can reformulate the observation or problem however you please, but that doesn’t make the testing of hypotheses go away.

Reference:

Bilalic, Smallbone, McLeod, and Gobet (2009). Why are (the best) Women so Good at Chess? Participation Rates and Gender Differences in Intellectual Domains. Proc. R. Soc. B, 276, 1161–1165.

Labels: , , ,

27 Comments

  1. hold on–it’s not clear to me you’re disagreeing with the conclusion of the Bilalic paper, are you? maybe i’m not reading your post right. 
     
    if we assume that, say, men are genetically 100 times more likely to be really good at chess than women, then presumably the results of the study would show that the differential representation at high-level play is NOT primarily due to differential representation in the general pool. but it is, meaning “biological, environmental, cultural or other factors” WITHIN the population of chess players are irrelevant. That seems to be a significant non-obvious result. 
     
    of course you’re right the paper doesn’t address the reasons why there are more men in the general pool of chess players. But then again i’m not sure how any one paper could possibly do that.

  2. The debate is about why men are so overrepresented among the elite of a number of fields. So, the debate is about how the male and female populations differ, in a way that would produce the pattern observed at the elite level — not how elite male scientists and elite female scientists differ. 
     
    At least, I’ve never seen anyone treat the debate as though it were about how traits differ among high-ranking scientists — always about males and females. 
     
    of course you’re right the paper doesn’t address the reasons why there are more men in the general pool of chess players. But then again i’m not sure how any one paper could possibly do that. 
     
    But this is the only relevant part of their article, as it addresses how the male and female populations differ — why is the chance of going into chess 16 times greater in one than the other?

  3. It really is time that large numbers of “science” journals were rebranded: as in, The Journal of Wishful Tihnking, Part B.

  4. If you study the distribution of intelligence and statisticaly control for IQ, no inequality is left.  
    This is a very common falacy in social sciences: “controling away” the differences.

  5. I have to agree with that one bioIgnoramus, allow me to present a few candidates for re-branding: the journal of Attachment and Human Development, Social Text, Foucault Studies, Science Studies, Marxism today, Social Anthropology, Cultural Studies, Semiota, etc.  
     
    As to agnostic’s article, I agree with his critique of the faulty statistical basis of the Bilalic article, but not with the idea that in order to explain male superiority: “Occam’s Razor requires us to find a simple account of all these patterns, not postulating a separate one for each case. The simple explanation is that men excel in these fields due to underlying differences in genes, hormones, social pressures, or whatever.” 
     
    What I disagree with here, is that it’s simple. Depending on the field, male superiority may be due to any or a combination of the factors you mentioned. For example in chess, my view is that the great predominance of males is due to an interaction between spatial and numerical ability, men’s interests (self-selection), and historical social acceptance of men and not women in competitive games. 
     
    Fashion design is different, there you wouldn’t argue that great spatial ability is an underlying cause, but maybe men’s greater status drive due to biological differences, and again the social acceptance of men in positions of authority and prestige as opposed to women.

  6. Well, Occam’s Razor doesn’t mean “as simple as I can think of” but “as simple as the data allow.” The story may be complicated on some absolute level, but as long as it applies broadly, then it captures a lot of the data, rather than having three separate stories for different domains where males dominate.

  7. Maybe this is urban legend, but I thought there was an analysis of male vs. female measured intelligence that showed a slightly different curves for males. I think it was that there were more males at the high and low ends whereas females grouped more closely to the mean.

  8. Oh, my bad, I didn’t notice bioIgnoramus was directing his comments more towards science journals, but there are still a few I mentioned there. They’re fairly easy to identify within the social sciences, just look for ones with “social” or “cultural” in the title, you can be pretty such that the blank slate “environment/culture is everything, if only we could change it!” outlook is rampant there. Or as John Derbyshire puts it nicely: “Hear no genes, see no genes, speak no genes”.

  9. “Why are men 16 times more likely than women to compete in chess leagues? We are back to square one: maybe men are better at whatever skills chess tests, maybe men are more ambitious and competitive even when they’re equally skilled as women, maybe men are pressured by society to go into chess and women away from it.” 
     
    Or perhaps the answer is that women find chess BORING. 
     
    Women are more interested in competing to wear the most fashionable clothes.

  10. Living in a family of chess players who enjoy competing in tournaments, I notice that competition seems to help maintain their interest. Since fashion designing is also competitive, as is any business, perhaps competition helps fuel achievement.

  11. well, now that I thought about it some more, maybe we can say this: 
     
    if Bilalic et al. had done this study in some other area (such as sports), presumably he would have found an entirely different result. Even if there was equal general representation of men and women in, say, swimming, men would dominate high-level swimming just because they’re bigger and stronger than women. Now, it seems prima facie plausible that a similar thing would occur in chess, but as the Bilalic paper shows that’s not the case. It is a non-obvious result that gender disparities within the community of chess players are (presumably) unlike those of physical sports like swimming. So I think the paper does have at least some value–certainly there are many people who think that women chess players are intrinsically worse than male chess players. The papers concludes otherwise.

  12. My only question: what’s wrong with Figure 2?

  13. Something worth noting is that at the highest level of any field, be it chess, fasion, whatever, you need alot more than pure talent, intelligence, or skill. That will take you only so far. To really be the best at something you need to have a burning desire to push yourself past your limits, to keep practicing and improve your skills. From what I’ve noticed (women that I know), women generally don’t push themselves to be the best at something. So its possible the comparative absence of women in these high levels isn’t due to lack of talent or intelligence, but instead the lack of competitive drive. 
     
    This seems to make sense from a (simplified) evolutionary standpoint. Men were in competition for women as mates, and so had a high selection pressure to be competitive. Women, as far as I can see, had no selection pressure.

  14. Orion: 
     
    Derbyshire’s wrong. They don’t care whether you see or hear, as long as you don’t speak! 
    It’s all about the speaking.

  15. > Women, as far as I can see, had no selection pressure. 
     
    You don’t think women have been more reproductively successful by marrying superior men?

  16. Women, as far as I can see, had no selection pressure. 
     
    This must rank as the most stupid statement I have seen in a long while. Oh, it could be explained by the wearing of blinders.

  17. ” To really be the best at something you need to have a burning desire to push yourself past your limits, to keep practicing and improve your skills. From what I’ve noticed (women that I know), women generally don’t push themselves to be the best at something.” 
     
    This analysis is correct, women lack the urge to win, and this is a factor in women’s absence at the top of all fields of human endeavor.

  18. Competition can be destructive as well as beneficial. Lacking a strong competitive drive can be as much an adaptation as having one can be – depending on the nature of the niche you’re in. 
     
    Evolutionarily speaking, men have little to lose by striving to be best and everything to gain. The situation for women isn’t completely reversed, but they have far more to lose and far less to gain by many kinds of competition.

  19. This analysis is correct, women lack the urge to win, and this is a factor in women’s absence at the top of all fields of human endeavor. 
     
    Is there any profession or field of endeavor where women constitute an elite? I’m trying to think of one but I can’t…  
     
    I think there may be fields where “the greats” are more evenly split between men and women, like acting perhaps. But none where women dominate.

  20. In evolutionary time, males and females have been subject to different selection pressures. A good example of resulting differences is sex differences in communication.

  21. Is there any profession or field of endeavor where women constitute an elite? 
     
    As far as authors of mystery novells are concerned, women are very competitive against men.

  22. DK’s observation goes well with act-like-girls to answer Mark’s question. Women should dominate highly verbally-oriented fields. But I think this effect is still dampened by the continuing tradeoff for women between family and professional life. 
     
    You could argue that women are starting to form an elite in certain humanities’ departments like in English. But again men’s status drive/ambitiousness makes them ubiquitous competitors. It seems there’s simply not as much intrinsic OR extrinsic incentive for women to compete for high-level jobs. 
     
    For example, male wealth is positively associated with number of children, but that correlation is negative for women (http://www.staff.ncl.ac.uk/daniel.nettle/amnat.pdf). In terms of evolution, this makes perfect sense. The men compete for access to women and resources, but women generally do not compete for resources, instead, they marry men who have them.

  23. Just to clarify my earlier post, when I said “women had no selection pressure” I meant that they had no selection pressure to be competitive, not that they had no selection pressure at all. 
     
    Its actually probably more accurate to say that they had a reduced selection pressure to be competitive.

  24. I think hegemonicon’s observations are right, but we have to account for individual variation as well. It’s not the case that ALL men are more competitive than ALL women. Serena Williams is more competitive than a guy who sits on his couch and never plays sports. But a small difference at the mean can make a huge difference at the tails on the distribution, as well all know. So there are women who are very competitive, they’re just very rare.  
     
    Plus, for achievement you need more than just competitiveness, and those other things add in a multiplicative fashion. For chess, and for other domains, with men slightly higher at the mean on several necessary skills or orientations, this adds up to a huge disparity at the highest levels.

  25. Throughout most of human civilization’s history, women have been more or less chained to childbearing and childrearing. During that time, public exploits have been defined and staffed by men. They tend to be exploits that interest men and, often, are boring to women. 
     
    If men were stuck at home for biological reason for a couple of thousand years and women controlled the public exploits, you might find that women excelled at them and men tended to lose interest before developing enough expertise to excel at the top levels. 
     
    Or maybe not. Maybe the evolutionary pressures on women reflect what men select for, which is about 95% looks. In contrast, women tend to choose men for their characters and skills. How often do you hear a woman complain that her chosen mate is pretty but dumb, grasping, or otherwise useless? How many women continually remarry men in their twenties even as the women reach 30, 40, and 50? When did you hear a woman describe her fiancee primarily in terms of his looks? A man with that attitude is commonplace; a women with that attitude would be a surprising joke.

  26. Vicki Hearne’s argument (1991) here that love of abstracting vs complexifying accounts for the gender differences in math. 
     
    It’s interests that lead to working obsessively at something. The abilities themselves in fact seem to be the same, but not the interests.

  27. Texan99 says: 
     
    Throughout most of human civilization’s history, women have been more or less chained to childbearing and childrearing. During that time, public exploits have been defined and staffed by men. They tend to be exploits that interest men and, often, are boring to women. 
     
    I am not sure what you are getting at here, but it seems to be in the same vein as women are so wonderful because they construct babies. (I have a book by a confused feminist who makes the claim that women should be accorded more status because they construct babies.) 
     
    Women who do not engage in childbearing and childrearing do not, generally, get to pass their genes into the next generation. 
     
    If men were stuck at home for biological reason for a couple of thousand years and women controlled the public exploits, you might find that women excelled at them and men tended to lose interest before developing enough expertise to excel at the top levels. 
     
    Ahhh, yes, that wonderful world where practicalities are unimportant and we can engage in mental masturbation about how we would remake the world the way we want it. 
     
    If women were selected for excelling in the exploits that men currently excel in, then we will see a change. 
     
    Or maybe not. Maybe the evolutionary pressures on women reflect what men select for, which is about 95% looks. 
     
    A proxy for good genes and being able to bear children, thus passing on both party’s genes? 
     
    In contrast, women tend to choose men for their characters and skills. 
     
    Oh, no, not another proxy for good genes?

a