Substack cometh, and lo it is good. (Pricing)

Motivated reasoning in “science journalism.”

Posted on by Razib Khan

The “reproducibility crisis” has really benefited some sectors of science journalism, as there is less credulous amplification of spurious results. That being said, motivated reasoning is powerful. They “want to believe.”

So when I saw this piece in Quartz, Highly motivated kids have a greater advantage in life than kids with a high IQ, I immediately scanned for what I usually look for, and found it:

Over the next four decades, the Gottfrieds and several colleagues collected a staggering trove of data on the study participants, yielding important insights into working parents, temperament, and other topics. Researchers collected information about participants from parents, teachers and transcripts, tested their IQ and motivation levels,and even visited their homes. In all, the Fullerton Longitudinal Study has amassed an estimated 18,000 pieces of information on each of the remaining 107 participants. “It’s our life’s work,” says Allen cheerfully. “We’ll take it to our grave.”

107 participants. Lots of information huh? Things that make you go hm….. Also, 19% of the children had IQs of 130 or above. About 2% of the population has an IQ at this level. The sample size was relatively small, and the sample was very unrepresentative.

This doesn’t mean that there aren’t real results in these data. But I don’t think they warrant the fanfare in the title, except for the fact that people want a silver bullet that will abolish social inequality.

Even the text itself doesn’t justify the title at all (to be fair, usually headline writers differ from the persons writing the text of a piece): “[Motivation] in itself is accounting for a certain amount of variance in achievement that goes above and beyond IQ….” That is, they don’t even say it accounts for more of the variance, only that there is variance that isn’t accounted for by IQ (which everyone already agreed upon).

Finally, I’ve spent my life around highly educated and intelligent people a bit perplexed and befuddled by my diverse interests. This includes in academia. So I can see that there is a difference between people for whom learning is a means to a professional and social ends, and for those whom learning is the ends. I suspect the ancients could have told you this!

Related Posts:

3 thoughts on “Motivated reasoning in “science journalism.””

  3. The Fullerton Longitudinal Study seems to have over-emphasized the importance of motivation. From a scan of the major findings in

    https://pdfs.semanticscholar.org/1d8d/1a067bac57af5fee7d0e50033bf522c5cca0.pdf

    Three findings distinguished between gifted motivation and gifted intelligence.

    First, there was no statistically significant overlap between the groups, meaning that those with gifted motivation are not necessarily intellectually gifted. In fact, the majority did not overlap. Only eight children were both intellectually and motivationally gifted in the entire study sample (see p. 181, Gottfried et al., 2005).

    Second, regression analyses revealed that academic intrinsic motivation predicted high school GPA above and beyond the variance due to IQ. This independent contribution of academic intrinsic motivation to the prediction of academic achievement above and beyond IQ across many measures was also previously reported (see pp. 124–125, Gottfried & Gottfried, 2004; see pp. 181–182, Gottfried et al., 2005).

    Third, the coefficient of alienation, a measure of noncorrelation, indicated that the overwhelming majority of variance in academic intrinsic motivation is not accounted for by IQ, further supporting the view that gifted motivation and gifted intelligence are distinct constructs. Therefore, this research serves to expand the definition of giftedness to include the construct of gifted motivation in its own right.

    The study asserted that “””there was no statistically significant overlap between the groups, meaning that those with gifted motivation are not necessarily intellectually gifted””” but did not show what were the possible reason for this. The sample size was too small that they could only find 8 overlapped cases out of sample size of roughly 110. The demographics only differentiate between caucasians and others and the IQs of the group tended to be high.

    The second major finding was that “””regression analyses revealed that academic intrinsic motivation predicted high school GPA above and beyond the variance due to IQ”””. This finding can be shown to be only marginally correct in view of the OECD PISA 2015 project with sample size of 540,000 students globally and the data showed that motivation level affected academic performance differently for different groups of population.

    The OECD PISA 2015 reports, http://www.oecd-ilibrary.org/education/pisa_19963777
    vol 1 contains the academic performance data and vol 3 contains contain the well being data including motivation.

    Taking the average of the Math, Sci and Reading as the academic performance indicator, and at the country level,

    PISA3 = -53.5598*Motiv +479.174; # n=57; Rsq=0.1888; p=0.0007323
    http://tinypic.com/view.php?pic=2hczv38&s=9

    i.e. on average the overall academic performance decreases with increasing motivation level, contrary to the major finding of the Fullerton study. It can be seen from the chart that the outliers diverged more from the regression line with increasing motivation level. Thus there are other factors.

    The PISA data also can be used to explain one of the possible causes, increasing motivation level also increases the test anxiety level,

    Anxiety = +0.323805*Motiv +0.0705132; # n=57; Rsq=0.2003; p=0.0004811
    http://tinypic.com/view.php?pic=63x4sl&s=9

    and increasing anxiety level is borderline significantly affects the academic performance,

    PISA3 = -43.5129*Anxiety +479.632; # n=57; Rsq=0.06521; p=0.05522
    http://tinypic.com/view.php?pic=5b22a&s=9

    The reason for the borderline significant correlation is that the effect showed some pitchfork bifurcation with different trends for different populations. A clear picture can be seen with more specific type of motivation, the desire to be the best PctWantBest. The upper pitchfork data can be objectively obtained by using the sample averages as the origin and uses only the data in the first quadrant,

    PISA3=+1.54352*WantBestPct+400.991; # n=14; Rsq=0.3779; p=0.01931
    http://tinypic.com/view.php?pic=34oodno&s=9

    and for the rest,

    PISA3=-1.84*WantBestPct+577.18; # n=42; Rsq=0.5888; p=3.064e-09 (**)

    the sign of the coeffs for the two regression lines are opposite.

    It is interesting that the Quartz article used the example of the performance of a Minnesota native. The populations in the PISA upper fork are the descendents of Viking, recently frontier countries and East Asians. Minnesota has the most people with Scandinavian ancestry. The populations that have overlapped motivational and intellectual giftedness are about 25% of the countries, not the less than 8% from the Fullerton study. The PISA data showed that there are alternative paths to academic excellence and motivation level only works for certain populations. The largest contrast is between Korea and Japan who have similar cultural and genetic backgrounds but are poles apart in the chart and achieving similar academic performances. (Most probably due to the Japanese angst after WW2 but that is another story.)

Comments are closed.