The British Guardian newspaper reports that an elementary statistical error has been found in half the published scientific papers in which the error could be made. Examining published articles in prestigious, peer-reviewed neuroscience journals, researchers found that a howling error was widely and routinely made and accepted.
How often? Nieuwenhuis looked at 513 papers published in five prestigious neuroscience journals over two years. In half the 157 studies where this error could have been made, it was. They broadened their search to 120 cellular and molecular articles in Nature Neuroscience, during 2009 and 2010: they found 25 studies committing this fallacy, and not one single paper analysed differences in effect sizes correctly.
The error would lead scientists to mistake whether an experimental result was significant or not and it amounts to a systematic bias that makes experimental results look more important than they are.The article does not say, and Via Meadia does not know, how widespread basic statistical ignorance is among scientists in other fields. But if anyone is wondering why laypeople are less trusting of “settled science” than they used to be, stories like this one suggest an explanation.