Even stronger: Do you think that a non-significant result is unimportant, uninteresting, not actionable, or not worth consideration because it is non-significant? Then you are wrong. And you are contributing a very large, very preventable problem.
There's been a hubbub about p-values and significance testing for quite a while. It ebbs and flows, but it's always at least a murmur in the background of every social and medical science. Recently, the American Statistical Association's publication of a formal declaration about p-values has turned up the volume to a dull roar. But that declaration is in no way new or novel or even more interesting that other previous attempts to get scientists to stop using statistical significance as a proxy for reliability, strength, or importance.
Most researchers know enough to ask "sure it's statistically significant but is it practically significant?" because they know enough to understand that the two are different. And they are different.
But that wisdom focuses on Type I sort of error: the possibility of treating a tiny or even non-existent effect as something important or even real.
The real monster lurking beneath the boat is Type II error: the possibility of rejecting evidence of a real effect because it did not meet your criteria for good evidence.
Can someone put a dollar amount on the resources that generate research results that never get published? And then express that as a percentage of resources that are spent on research generally? What would that percentage be, do you think? And what should it be?
I contend that if you think that percentage should be less than 80%, then we have a huge problem on our hands, and null hypothesis significance testing is to blame.
No comments:
Post a Comment