« A must read site! | Gene Expression Front Page | Green Beard Ethnic Nepotism? [Yes, Another Post On This] »
December 22, 2004

Debunking Anti-Testing Nonsense

Kevin Drum has a post over at his blog about affirmative action, and many commentators, in an attempt to defend large-scale preferences, have questioned the validity of standardized tests. Most of their criticisms are nonsense.

First, Claude Steele's "stereotype threat" is (almost) pure bunk. From the American Psychological Association:

C. M. Steele and J. Aronson (1995) showed that making race salient when taking a difficult test affected the performance of high-ability African American students, a phenomenon they termed stereotype threat. The authors document that this research is widely misinterpreted in both popular and scholarly publications as showing that eliminating stereotype threat eliminates the African American–White difference in test performance. In fact, scores were statistically adjusted for differences in students' prior SAT performance, and thus, Steele and Aronson's findings actually showed that absent stereotype threat, the two groups differ to the degree that would be expected based on differences in prior SAT scores. The authors caution against interpreting the Steele and Aronson experiment as evidence that stereotype threat is the primary cause of African American–White differences in test performance.


Another major criticism of standardized tests is that they don't predict much variance in college GPA. One issue that even many pro-standardized testing people don't consider is that college GPA is not necessarily a good factor to measure any predictor against. The student getting a 3.8 GPA in ethnic studies may have a higher GPA than the student getting a 2.5 GPA in upper division physics courses, but the fact is that the latter student is likely to be a *far* better student than the former. Indeed, even amongst "hard" and "easy" majors there can be a fair amount of variance. Physics, at least at the university I attend, is a far harder major than engineering. I know a number of people who got A's in lower division physics courses (which engineers also take) who are getting C's and D's (with significantly more effort) in upper division physics courses. I would also guess that many social science majors tend to be significantly more difficult than ethnic studies.

Another issue regarding the correlation between standardized test scores and college performance is restriction of range. When there is relatively little variance in test scores at a given university, this relatively small amount of variance in test scores will not necessarily be that useful in predicting college success. However, this does not mean that a large group of people admitted with test scores well below the mean would do well. A useful analogy may be height in basketball. If one were to devise a metric of general basketball performance in the NBA, height would probably not explain an incredibly large amount of the variance (maybe something like 10-20%, depending on the metric). But this does not mean that the NBA could start hiring a bunch of players who are 5'6" and they would all do OK.

I would also add that blacks and whites with the same SAT scores perform roughly the same in college. In fact, the SAT slightly overpredicts black performance in college, and I would suspect the same applies to other 'disadvantaged' groups like Latinos, as well as poor whites and Asians admitted under "top 10%" plans.

Note:If many of these arguments seem similar to those of "dude" in the WM comments thread, it's because I AM "dude."

Edited at 1:27PM 2004-12-22 for clarity

Posted by bb at 01:23 PM