How to make specious claims that research is drawing specious conclusions

UPDATE: The author of post I discuss below let me know that my comments were not blocked (i'm glad). The essence of this post, however, is more about what they say about the scientific study in question and not about my comment being blocked.

Like many science bloggers, I don’t like it when people use their blogs to try and shoot down scientific studies for unfounded reasons.

Let me give you an example.

Yesterday, I came across a blog on the WordPress home page, which highlights various categories of other WordPress blogs. Under the “science” category was this post: How to Draw Specious Conclusions From Research. How could I not click on that?!

You might remember a few weeks back, the media outlets and bloggers picked up on a study published in Nature Neuroscience titled "Neurocognitive correlates of liberalism and conservatism" (if you have access, you can see full-text of the paper here). There was some discussion of the study over at ScienceBlogs (see here and here). But, I don't want to talk about the study in detail...i'm not a neuroscientist and will let others deal with the nitty gritty...what I do want to talk about is sloppy or, worse, purposefully misleading commentary about the study by bloggers.

One of the great things about being a scientist is that you learn how to read in detail and appreciate minutia. Remember, the whole point of their post is to claim that this particular study makes specious conclusions. So, I commented on the blog that perhaps it was the media's interpretation and reporting of the study that is specious (since they link to an LA Times story and not the paper). My comment is paraphrased here:

Typically, the media report will try and make the big claims (i.e., headlines) to attract readers, whereas the study may offer some speculations, but the actual conclusions will be not that exciting.

The blog author immediately replied to my comment:

Trust me, I checked the paper. The authors are the ones who draw the specious conclusions, not the media. If you’d like to check the paper out, here is the Bibliography:

DM Amodio, JT Jost, SL Master, CM Yee, “Neurocognitive correlates of liberalism and conservatism” Nat Neurosci, 2007

Fair enough. I took a little time to read the actual paper. The first thing I wanted to do was check and see if what the blog author claims are their conclusions actually are. They start off their post with this:

Among their list of conclusions were:

• “conservatives tend to be more structured and persistent in their judgments”
• conservatives tend to ignore information
• “liberals are more open to new experiences”
• liberals can be expected to accept new scientific and social ideals faster
• the results “provided an elegant demonstration that individual differences on a conservative-liberal dimension are strongly related to brain activity.”

Firstly, only three of these bullet points even appear in the paper at all. Secondly, and much more importantly, the conclusions listed above aren't Amodio et al's conclusions. They address these attributes as hypotheses to test and cite the previous work that made the conclusions. This is very clearly stated in the abstract and the first paragraph of the paper! The point of their study (i.e., the point of a lot of scientific studies) was to take previous work and results and test them in a very specific way. So, the blog author is simply wrong when stating that "the authors are the ones who draw the specious conclusions". The blog author asked me to check the real paper...I did. Amodio et al.'s actual conclusions go like this:

Taken together, our results are consistent with the view that political orientation, in part, reflects individual differences in the functioning of a general mechanism related to cognitive control and self-regulation.

Like most conclusions from very specific studies, what they actually say is worded carefully and within the specific context of the previous work they were addressing.

So, I submitted a comment to the post similar to what I just said above to point this out. Apparently my comment was thrown into the spam filter and never appeared (it happens). Nevertheless, my statements in this post about their deceitful (or simply sloppy) reporting of the study in question stand.

I'm not gonna take the time to try and comment again on their thread and deal with the spam issue. They can respond to what I say about their post on this thread if they like.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~