MEGAN MCARDLE: The Shallowness Of Lefty Self-Congratulation.

I am reluctant to make sweeping generalizations about a very large group of people based on a single study. But I am reluctant indeed when it turns out those generalizations are based on 85 drunk people and 75 psychology students.1

When highlighting studies like this, you should probably describe it with words like “suggestive,” “possible” and “may.” Waldman writes as if we’re dealing with Proven Scientific Fact, rather than something that a small team of researchers found in three small groups.

This is particularly vital when you’re dealing with research about conservatives, done by a profession that skews liberal by something like 200 to 1. The unstated assumptions of the group are bound to slip into things such as the questions they ask and how tempted they are to go back and look for a “mistake” when they get an answer suggesting that liberals are close-minded barbarians.

To see what I mean, consider the recent tradition of psychology articles showing that conservatives are authoritarian while liberals are not. Jeremy Frimer, who runs the Moral Psychology Lab at the University of Winnipeg, realized that who you asked those questions about might matter — did conservatives defer to the military because they were authoritarians or because the military is considered a “conservative” institution? And, lo and behold, when he asked similar questions about, say, environmentalists, the liberals were the authoritarians.2

It also matters because social psychology, and social science more generally, has a replication problem, which was recently covered in a very good article at Slate. Take the infamous “paradox of choice” study that found that offering a few kinds of jam samples at a supermarket was more likely to result in a purchase than offering dozens of samples. A team of researchers that tried to replicate this — and other famous experiments — completely failed. When they did a survey of the literature, they found that the array of choices generally had no important effect either way. The replication problem is bad enough in one subfield of social psychology that Nobel laureate Daniel Kahneman wrote an open letter to its practitioners, urging them to institute tougher replication protocols before their field implodes. A recent issue of Social Psychology was devoted to trying to replicate famous studies in the discipline; more than a third failed replication.

That’s okay. The purpose of deploying these studies is to make the useful idiots feel good about themselves.