Loading...

Effect Sizes and Practical Significance

Why Effect Size Matters More Than P-Values

A p-value tells you whether an effect exists (probably); an effect size tells you how large it is. With enough data, even trivially small effects become statistically significant. A study of 100,000 people might find that a new teaching method improves test scores by 0.2 points on a 100-point scale with p < 0.001—statistically significant but practically meaningless. The American Statistical Association's 2016 statement on p-values explicitly warns against equating statistical significance with scientific or practical importance. Effect sizes provide the missing context: Is this effect large enough to matter? Would a reasonable person change their behavior or policy based on this magnitude of effect?

Effect Sizes and Practical Significance

Why Effect Size Matters More Than P-Values

A p-value tells you whether an effect exists (probably); an effect size tells you how large it is. With enough data, even trivially small effects become statistically significant. A study of 100,000 people might find that a new teaching method improves test scores by 0.2 points on a 100-point scale with p < 0.001—statistically significant but practically meaningless. The American Statistical Association's 2016 statement on p-values explicitly warns against equating statistical significance with scientific or practical importance. Effect sizes provide the missing context: Is this effect large enough to matter? Would a reasonable person change their behavior or policy based on this magnitude of effect?