A few years ago when I reviewed The Invisible Gorilla: And Other Ways Our Intuitions Deceive Us, I joked that it was the anti-Malcolm Gladwell manifesto. The joke was only half serious. Chris Chabris and Daniel Simons presented in their book serious arguments which weren’t sexy and offered no easy shortcuts. As such it is no surprise that Gladwell is still rolling in the money, while Chabris and Simons are respected academics, though not public intellectuals on the same magnitude (the irony is that arguably they are intellectuals in a more substantive way than their famous bête noire). A more egregious individual when it comes to science popularizing than Gladwell was Jonah Lehrer (not surprising that Jonah was somewhat of a protege of Gladwell). Aside from the admitted fabrications, Chabris has been long pointing out that Lehrer seems to purposely misrepresent or misunderstand the process of science, taking isolated studies and stitching them together to support novel and counter-intuitive theses which might sell copies of books (it was ironic that he wrote a long piece for The New Yorker on problems with replication).
The fact that you shouldn’t hinge your perception about the validity of a hypothesis on one study isn’t an issue for most scientists. They know how science works. It’s a noisy process, with lots of fits and starts, and consensus emerges slowly, and is periodically overturned or extended. There’s a reason that John Ioannidis’ Why Most Published Research Findings Are False is highly cited. There are thousands and thousands of studies published every year. If you want, you can search through the stack and find “peer reviewed research” to support nearly any proposition. The issue isn’t whether there are scholars willing to support your position, but what the scholarly consensus is, if there is one.
All this came to mind when I saw this blog post, A Trick For Higher SAT scores? Unfortunately no. The short of it is that a few years ago the author read Thinking, Fast and Slow, from Daniel Kahneman, a Nobel Prize winner. He reported with excitement results from a study which primed individuals to focus more with less clear fonts, and therefore increased their cognitive performance substantially. The reason why this study’s results are important is obvious to anyone, increasing median cognitive performance is a social good (this is why we put iodine in salt to combat cretinism).
Though Kanheman is a great scholar, most people are not going to know about this study from him. Rather, Malcolm Gladwell used the study in David and Goliath: Underdogs, Misfits, and the Art of Battling Giants to illustrate one of his points. Unfortunately Gladwell is a big deal for many people. Though I quite liked The Tipping Point when it came out, over the years I’ve come to see that Gladwell is less a communicator of scholarship than a storyteller who sells intellectually-themed yarns. Gladwall hasn’t seen a sample size that dissuades him from reporting enthusiastically on a result with a marginally significant p-value, so long as it supports one of his story arcs.
Three years on the author of the blog post, and one of the original authors of the paper, have a follow up publication where they report that there is no effect at all from the priming with less clear fonts. The sample size of the original study was 40. The follow up, 7,000 total (they pooled multiple studies). The author of the blog post ends on a down note:
I expect that the false story as presented by Professor Kahneman and Malcolm Gladwell will persist for decades. Millions of people have read these false accounts. The message is simple, powerful, and important. Thus, even though the message is wrong, I expect it will have considerable momentum (or meme-mentum to paraphrase Richard Dawkins).
Probably descriptively correct. But you can do something about it. Be the asshole at the party to point out that the “latest research” your friend has read in the current issue of The New Yorker is most likely to be crap, especially if it is both counter-intuitive and supports your group’s normative priors. (yes, I am usually that asshole in real life too)
Note: the reason I say irrelevant, rather than false or wrong, is that a lot of research is trivial improvement on an already established consensus if when the results are robust.
Comments are closed.