If it doesn’t make sense in light of all the facts, it doesn’t make sense

Bayesian statistics has made The New York Times, The Odds, Continually Updated. One illustration of the utility of Bayesian methods left out of the piece is in phylogenetics. For example, Mr. Bayes. Just to see how far we’ve come, I like to retell a story from a professor of mine. When he was in grad school ~15 years ago it was assumed that they’d never be able to implement the far less exhaustive maximum likelihood method due to limitations of computational power.

But the reason I want to highlight this article, aside from that it is a good article overall, is this section:

Take, for instance, a study concluding that single women who were ovulating were 20 percent more likely to vote for President Obama in 2012 than those who were not. (In married women, the effect was reversed.)

Continue reading the main story
Dr. Gelman re-evaluated the study using Bayesian statistics. That allowed him to look at probability not simply as a matter of results and sample sizes, but in the light of other information that could affect those results.

He factored in data showing that people rarely change their voting preference over an election cycle, let alone a menstrual cycle. When he did, the study’s statistical significance evaporated. (The paper’s lead author, Kristina M. Durante of the University of Texas, San Antonio, said she stood by the finding.)

There are plenty of sexy papers published on ovulation and menstrual cycles and how they correlate with a particular outcome. If you look for correlations enough they will come (assuming you use p = 0.05). That’s common sense. But another issue to consider here is that you have a model, and the predictions that the model makes don’t hold in a rather simple case where the cause and effect seem obvious (i.e., voting patterns should change over time since ovulation changes over time). This sort of sanity check is important when you go drudging through statistics, but also when you are tackling complex phenomena at a high level.

Whenever I talk about Islam people offer up opinions about how the Koran serves as a sort of template or guidebook in terms of behavior, and that explains Islamic civilizations real pathologies. This is not an implausible model, and I held to it myself when I was younger.

There are two problems. First, most of the people making this assertion don’t know enough history or religion to even plausibly evaluate the model in their own head. That’s just a fact, and why I’m so dismissive of so many people. The limits of your knowledge are the limits of your model building. Second, there’s a deeper issue which I first encountered in the mid-aughts: there’s a good deal of evidence from cognitive psychology that people barely understand in a coherent manner the ‘messages’ of their scriptural texts. This is outlined in books like In Gods We Trust: The Evolutionary Landscape of Religion. One also has to remember that for almost all of human history, and to a great extent today, most humans were either illiterate or functionally so. More importantly psychological experiments which attempt to ferret out exactly how scriptural texts would impact peoples’ beliefs show that there’s no real ratiocination going on. Rather, it seems to be that reasoning in a religious context is a process of collective rationalization.