fbpx

Is the Replication Crisis a Problem for Progressivism?

One way to understand the differences between these three ideologies is to contrast their preferred methods for decisionmaking.  Libertarians generally favor decisions to be made through the free market.  Conservatives generally prefer following traditional practices.  And progressives generally favor following expert decisions made by government agencies supported by peer reviewed studies.

Each group has strong arguments for their preferred decisionmaking method.  The free market is a powerful mechanism that both coordinates the division of information and provides strong incentives for individuals to take the right actions. Traditional practices have been supported by the test of time in a complicated world where it is often hard to determine what are the best actions in the long run given the complexity of the social world.  And expert knowledge, supported by peer reviewed studies, appears to have the force of science behind it.

Of course, all of these mechanisms have problems.  The question is how effective they are as compared to one another.

It is significant, then, that there is a replication crisis occurring in peer reviewed studies.  Leading studies in various fields do not replicate.  It is not merely social psychology studies that don’t replicate.  A significant number of medical studies have not replicated.  And the more one reads, the more people acknowledge some of the problems – see here on replication in economics.  And see here on the “scientific” support for parapsychology.  Given that the same methods are generally employed in the various fields, I would be surprised if the replication crisis does not spread.

This is not merely an example of poor practices or corruption in the natural and social sciences.  It is also a strong indictment of employing peer reviewed studies, at least in certain fields.  If this replication crisis continues to expand, it presents a serious problem for progressivism.

Reader Discussion

Law & Liberty welcomes civil and lively discussion of its articles. Abusive comments will not be tolerated. We reserve the right to delete comments - or ban users - without notification or explanation.

on October 05, 2017 at 10:12:48 am

Mike:

You are being too kind - you did not make mention of some of the "studies" that were intentionally designed to fool the "peer review" process and *studied* the most outlandish of propositions - AND guess what - they survived peer review.

Now what does THAT say about the state of modern science and it's practitioners?

This is SAD!

read full comment
Image of gabe
gabe
on October 05, 2017 at 13:41:22 pm

I hate to reduce this important topic to petty culture war battles. But we needn't reduce the replication crisis to acknowledge that it poses special problems for those of us who value expertise--and, by contrast, bolsters the claims of other value systems.

[S]ee here on the “scientific” support for parapsychology.

Yes, do see there; it's a magnificent discussion!

But it's not just a discussion about parapsychology (such as Extra-Sensory Perception, etc.). Rather, the linked text contains a discussion about the replication crisis at a rather sophisticated level, using parapsychology as a control group. That is, if our scientific methods can generate "reliable" results demonstrating ESP, then we can reliably conclude that either 1) parapsychology is real, or 2) our scientific methods aren't yet rigorous enough. And the author embraces Theory 2.

The thrust of the author's conclusion is that, in addition to all the other safeguards we require to demonstrate validity, we should require results to be not only replicable but also sufficiently large. And we should require that because we're discovering that scientists tend to confirm the outcomes they already believe in, no matter how much they try to insulate their experiments from their own views. The author cites a study in which two scientists with competing beliefs collaborate on conducting a study measuring galvanic skin response--an "objective" phenomenon. Yet this objective phenomenon would vary depending on which scientist happened to be in building while the study was occurring. The different outcomes were small, but consistent. Go figure.

Finally, you should go to the linked article to enjoy the author's parting words of despair, in the style of Obi-Wan's farewell to Anikin Skywalker: "Science! YOU WERE THE CHOSEN ONE! It was said that you would destroy reliance on biased experts, not join them! Bring balance to epistemology, not leave it in darkness! I LOVED YOU!"

read full comment
Image of nobody.really
nobody.really
on October 05, 2017 at 14:31:16 pm

Awesome!!!!!

Similar to polling: "He who defines the question defines the answer."

And perhaps, nobody can be the new "chosen one" - Ha!

read full comment
Image of gabe
gabe
on October 05, 2017 at 18:50:32 pm

I have often marveled at, but not held in much surprise, what appears to be multiple replication failures across the globe of that "Grand Experiment" that is the American Democratic Republic.

I attribute these many failed replications to a citizenry insufficiently committed to the heavy obligations of self-rule and not to flaws in the original design. Sadly, this characteristic found in most of the rest of the world seems to be slowly gaining momentum in the U.S.

There are many who see the U.S. as some sort of perpetual motion machine, that once set in motion will continue indefinitely under its own force, without need of constant ongoing maintenance and eventual renewal of the parts that make it work - this is a serious folly.

read full comment
Image of Paul Binotto
Paul Binotto

Law & Liberty welcomes civil and lively discussion of its articles. Abusive comments will not be tolerated. We reserve the right to delete comments - or ban users - without notification or explanation.