Yet another article on the problem of replication. If a study is designed with research methods that have been implemented appropriately and reported clearly, repeating that study with the same methods, the findings should be similar. Right? This has always been a key expectation for science, but as you know if you have read the blog entry about the Reproducibility Project, there is increasing evidence that the result of most studies do not hold up in this way when they are repeated.
An article by the Boston Globe’s Carolyn Y. Johnson last March suggests some reasons:
- Fraud: but this doesn’t account for most of the problems.
- Experiments may not be properly set up.
- A given result that appeared exciting may have been a statistical fluke.
- Something the researchers hadn’t considered could explain the results.
- Subtle biases may blind scientists who want to solve a problem.
- An exciting positive result may just be a result of random chance.
Much of this discussion has been based on efforts to replicate laboratory experiments about biological or other effects. Carolyn Johnson quotes Harvard cell biologist Bjorn Olsen as saying “science is an imperfect human activity that we try to do as best we can.”
Do you think the same problems are more or less likely in sociological research? In other social sciences? Why or why not?
What other issues might lessen reproducibility?
Researching in a Social World 