Tyranny. P value

 The “tyranny of the P-value” refers to the habit of treating P < 0.05 as a yes/no stamp of truth. The quote you shared (from Leslie Citrome) is pointing out that a P-value is often misunderstood and over-trusted. Here’s what that really means:

What a P-value actually tells you

A P-value answers this narrow question:

If there were truly no effect, how likely would we see results this extreme just by chance?

So:

• P = 0.05 → There’s a 5% chance of seeing data this extreme if the null hypothesis were true.
• It does NOT tell you the probability the hypothesis is true.
• It does NOT tell you how big or important the effect is.

What a P-value does NOT tell you

People often misinterpret it as:

• ❌ “There’s a 95% chance this finding is real”
• ❌ “The treatment works”
• ❌ “This result is important clinically”
• ❌ “The effect is large”

None of those follow from P < 0.05.

Why “P < 0.05” can be misleading

1. Tiny effects can be “significant”
   With huge sample sizes, even meaningless differences can produce P < 0.05.
2. Important effects can be “non-significant”
   Small studies may miss real effects (low statistical power).
3. It ignores effect size
   A drug lowering blood pressure by 0.5 mmHg might be statistically significant — but clinically useless.
4. It’s sensitive to study design
   Different assumptions, tests, or stopping points can change the P-value.

What you should look at instead (or in addition)

• Effect size (how big is the difference?)
• Confidence intervals (range of plausible values)
• Clinical significance (does it matter in practice?)
• Study quality (bias, sample size, design)
• Replication (does it hold up again?)

The takeaway

The “tyranny” is when:

P < 0.05 becomes the only thing people care about.

But a P-value is just one piece of evidence, not the verdict.

If you want, I can also:

•  show a simple example with numbers
•  explain confidence intervals vs p-values
•  explain why 0.05 became the cutoff