A study in the journal Science confirms that not every report published in a top-flight academic journal can be taken as gospel, no matter how much you might want the reported findings to be true. Researchers tried to reproduce the results from 100 studies printed in three major psychology journals and managed to do so in only 39 cases.
That doesn’t mean that only 39 percent of scientific studies are valid. As The Post’s Joel Achenbach pointed out, there were several limitations on the findings: The reviewers examined only psychology studies; 100 studies is a small sample; researchers exercised some choice in which studies they tried to replicate, which could have skewed the results; and the reviewers themselves might have introduced circumstances that confounded their own results somewhere along the line.
Both the study and the caveats, though, underscore a broad, crucial point for policymakers, the media and the public at large: Healthy skepticism must be nurtured; wishful thinking and its opposite number, cynicism, must be avoided. Just as any one research project’s results shouldn’t be assumed to be valid based on one initial round of peer review, it’s not healthy to conclude that scientific inquiry is hopelessly incapable of shedding light on controversial questions, instead defaulting to party, ideology or theology to illuminate issues on which scientists have serious claims. Good science, soberly assessed, is all the more important in a world in which ideologues and special interests push their own “studies” and “data” to justify their parochial concerns and reject inconvenient evidence with equivalent ferocity.
Most scientists are working in good faith to describe realities that can be difficult to pin down. Over time, further inquiry tends to sharpen experts’ view of what’s really happening. This should not paralyze government; it should encourage leaders to favor research findings supported by multiple lines of evidence over exciting ones supported by less, highly-scrutinized results over minimally-scrutinized ones.
Scientists, meanwhile, can help by maintaining a sense of professional modesty and responsibility. Researchers should seek accurate results, not provocative ones. Journals and scientific institutions should cultivate standards that reduce the possibility of confusing the two. Authors should disclose more information about their procedure, and they should offer their raw data up as a matter of habit.
It would be easy to read through the reviewers’ results and despair that so much chaff may have made it into respected academic journals. Instead, we take the study itself and its generally respectful reception among scientists as evidence that the scientific method is still working. Scientists are still subjecting each other to exacting scrutiny. Concepts such as experimental repeatability continue to be core principles. People continue, slowly and painstakingly, to get ever closer to describing the world as it is, not as politicians, pundits or scientists themselves wish it were. It’s on all of us to listen — carefully.
The above editorial appears in the Aug. 30 Washington Post.
Talk to us
> Give us your news tips.
> Send us a letter to the editor.
> More Herald contact information.
