The above is from an article in Nature, A test that fails. Two stories first. One of my good friends who went to grad school at MIT got a good ribbing from his roommates because he was the only one who didn’t get a perfect score on the math portion of the GRE. Luckily for him, he was a chemist, so they let him into the program. It is a truth universally acknowledged among the quantitatively ept that the quantitative GRE is just way too easy, and is compressed at the top scale and does not allow for differentiation of the good from the great. That is, there are a wide range of competencies which are bracketed among those who score a “perfect” 800 on the quantitative GRE. And there are many people in fields like physics who score 800; the average score on quantitative reasoning for those who intend to study physics in graduate school (not those who get accepted!) is in the 740s. Second, a friend of mine was complaining about the lack of underrepresented minorities in the biological sciences at my graduate school. To her surprise and irritation I just pointed out that all the underrepresented minorities within the range of GRE scores that our program takes would be going to Stanford or Berkeley. There weren’t enough of them that we’d be competitive. Data like the above is just not well known.
Another point is that the article above is very anti-GRE. They claim that the GRE score is not very predictive of ultimate outcome. One of my professors pointed to a study at University of California San Francisco (UCSF) where they tracked future successes (e.g., tenured position in academia x years out), and correlated them with GPA and GRE. Neither were very strong predictors. Rather, their Ph.D. research productivity was highly predictive. This isn’t that surprising, because GPA and GRE are just proxies to get at whether one can be a productive researcher, and being productive in graduate school is probably the best guide as to whether you’ll be productive later. But, one thing I want to point out is that UCSF is a very selective school. The range of GRE scores in particular is likely to be narrow, because they’re going to simply not even look at applicants with low scores. Whenever people point out that MCAT or GRE is poorly correlated with professional outcomes, remember that you’ve already compressed the distribution toward the higher end. If schools allowed a much wider range of applicants in, then these aptitude tests would be much more predictive.
In fact, the reality is that there is variation in outcomes according to general intelligence among graduate students. As I stated above, the maximum score of the GRE, especially the quantitative reasoning section, is too low to get at that. But Camilla Benbow’s group has been tracking mathematically precocious children for decades. As the data to the left shows, the smartest-of-the-smart are more likely to become scientists, and much more likely to attain tenure. The cut-off was scoring in the top 1% of their age group on the mathematical SAT test, a 390 score. You can see how much better those very rare students who score 700 or more at age 13 are doing later in life.
Finally, obviously these tests are very robust and predictive, but they’re population statistics. There are people who do not do well on the GRE who do well in academia, and vice versa. But, the reality is that these tests are not useless, and just how “not useless” they are will become more obvious if no one made recourse to them.
Addendum: My physicist friends always enjoy a chuckle whenever I honestly state that physicists are smarter than biologists, as I am a biologist. There are rare cases, such as Ed Witten, of people entering physics from other fields, but in general it’s the physicists who are the imperialists. And that’s because they’re smart, able to decompose general problems rapidly and decisively. In contrast, biologists are somewhat narrow in their focus, and plodding in their reasoning. These are generalizations, but I think they’re roughly correct (I had a friend at a prominent non-profit who was irritated with how difficult it was to find Ph.D. biologists who were flexible thinkers in interviews). And standardized tests bear out my generalization (though honestly, it is a pleasure talking to physicists and mathematicians about out of topic fields compared to biologists partly because they’re so mentally acute; you don’t need GRE stats to get this).
But, another implication of this logic is that some minority groups are also not too bright. If you don’t think these tests are accurately reflecting real intellectual skills that groups have though you don’t have to go there. And my experience is that this is a common belief, including among physicists. But then I suppose they shouldn’t get so full of themselves about their GRE scores in relation to biologists?
Comments are closed.