The short answer
A p-value is the probability of seeing a result at least as extreme as yours, if the null hypothesis (no real effect) were actually true. A small p-value (commonly below 0.05) suggests your data would be unusual under that assumption — evidence against the null hypothesis, not proof it's false, and not the probability that the null hypothesis itself is true.
Key takeaways
- p < 0.05 doesn't prove an effect is real; it means the observed data would be unlikely if there were truly no effect.
- The p-value depends on the test statistic, sample size, and variability — a tiny, unimportant effect can still produce a small p-value if the sample is large enough.
- Two-tailed tests split significance across both directions and generally report roughly double the p-value of a matching one-tailed test.
- Different test statistics (z, t, chi-square, F) come from different distributions, so the same numeric statistic can produce very different p-values depending on which test applies.
Common significance thresholds
| p-value range | Typical interpretation |
|---|---|
| p ≥ 0.10 | Little to no evidence against the null hypothesis |
| 0.05 ≤ p < 0.10 | Weak or marginal evidence |
| 0.01 ≤ p < 0.05 | Moderate evidence — commonly called "significant" |
| p < 0.01 | Strong evidence against the null hypothesis |
These bands are conventions, not laws of nature — different fields (medicine, physics, social science) adopt different thresholds depending on how costly a false positive would be.
One-tailed vs. two-tailed
| Test | Question asked | p-value relationship |
|---|---|---|
| Two-tailed | Is there any difference, in either direction? | ~2× the one-tailed p-value |
| One-tailed | Is there a difference in one specific direction? | ~1/2 the two-tailed p-value |
Worked example: z-test p-value
z = 1.96, two-tailed test
p = 2 × (1 − Φ(1.96)) = 2 × (1 − 0.9750) = 0.05
This is exactly why 1.96 shows up so often in statistics — it's the z-score whose two-tailed p-value lands precisely at the conventional 0.05 threshold, making it the standard cutoff for a 95% confidence level.
Common mistakes to avoid
- Treating the p-value as "the probability the null hypothesis is true" — it's computed assuming the null IS true, so it can't simultaneously measure the odds that it's true.
- Choosing a one-tailed test's direction after seeing which way the data leans — the direction must be decided before looking at the results, or the test is no longer valid.
- Treating p = 0.05 as a hard line — a p-value of 0.049 and 0.051 represent nearly identical evidence despite landing on opposite sides of the common threshold.
- Confusing statistical significance with practical importance — a large enough sample can make even a trivially small, unimportant effect statistically significant.
Related calculators
- Confidence Interval Calculator — estimate a range instead of testing a single hypothesis.
- Z-Score Calculator — convert a raw value into the z-score this calculator can test.
- Standard Deviation Calculator — compute the variability that feeds into most test statistics.
- Sample Size Calculator — plan a study with enough power to detect a real effect.