September 10, 2011

AND THAT'S A SCIENTIFIC FACT!:

The statistical error that just keeps on coming: The same statistical errors - namely, ignoring the "difference in differences" - are appearing throughout the most prestigious journals in neuroscience (Ben Goldacre, 9/10/11, guardian.co.uk)
We all like to laugh at quacks when they misuse basic statistics. But what if academics, en masse, deploy errors that are equally foolish? This week Sander Nieuwenhuis and colleagues publish a mighty torpedo in the journal Nature Neuroscience.

They've identified one direct, stark statistical error so widespread it appears in about half of all the published papers surveyed from the academic psychology research literature. [...]

Nieuwenhuis looked at 513 papers published in five prestigious neuroscience journals over two years. In half the 157 studies where this error could have been made, it was. They broadened their search to 120 cellular and molecular articles in Nature Neuroscience, during 2009 and 2010: they found 25 studies committing this fallacy, and not one single paper analysed differences in effect sizes correctly.

These errors are appearing throughout the most prestigious journals for the field of neuroscience. How can we explain that? Analysing data correctly, to identify a "difference in differences", is a little tricksy, so thinking generously, we might suggest that researchers worry it's too longwinded for a paper, or too difficult for readers. Alternatively, less generously, we might decide it's too tricky for the researchers themselves.

But the darkest thought of all is this: analysing a "difference in differences" properly is much less likely to give you a statistically significant result, and so it's much less likely to produce the kind of positive finding you need to look good on your CV, get claps at conferences, and feel good in your belly.
If it sounds like a duck....


Posted by at September 10, 2011 9:08 AM
  

blog comments powered by Disqus
« THE IRONY BEING THAT THE LEADER OF THE COUNTERJIHAD...: | Main | AT THE CORNER OF BALLANTINE & BORSCHT: »