Demystifying the P-Value & Statistical Power
What P(Data | H₀) means, why p = .049 vs p = .051 is not a sharp cliff, and how sample size drives standard error.
In this lesson you will learn
- 1Define the p-value accurately as P(Data as or more extreme | H₀ is true) rather than P(H₀ is true | Data).
- 2Explain the mathematical relationship linking Sample Size (N), Standard Error (σ / √N), Effect Size (d), and Power.
- 3Recognise how unplanned repeated peeking at accumulating data (p-hacking) inflates false-positive discovery rates.
On this page▼
Interactive Power, Sample Size, and Alpha Simulator
This section illustrates how sample size N, effect size d, and alpha threshold α interact mathematically to determine statistical power (1 − β).
When an examiner asks why one variable reached significance while another with a similar effect did not, understanding SE = σ / √N gives you a clear, quantitative answer.
Every parametric test statistic can be understood as a signal-to-noise ratio:
Test Statistic (t or Z) = Observed Effect Size / Standard Error = (X̄₁ − X̄₂) / (s_p × √(2 / n))
Notice where sample size n sits: in the denominator of the Standard Error. As n increases, the Standard Error shrinks, the sampling distributions narrow, and the overlap between H₀ and H₁ diminishes—boosting Statistical Power (1 − β) without changing the underlying biological effect size.
Statistical Power & P-Value Simulator
Explore the mathematical link between Sample Size (N), Effect Size (d), and Probability of Truth (Power)
Clinical Example
Two hospitals evaluate the exact same antihypertensive regimen (true effect Cohen's d = 0.50). Hospital A recruits n = 20 per arm; Hospital B recruits n = 80 per arm.
Interpretation: Even though the biological effect size (d = 0.50) is identical in both hospitals, quadrupling the sample size from 20 to 80 per arm halves the standard error and raises power from 35% to 88%.
Common Pitfall
Interpreting p = .04 as meaning there is a '4% probability that the null hypothesis is true' or a '96% probability that the drug works'.
Included with Any Analysez AI Study Package
This deep interactive chapter (Demystifying the P-Value & Statistical Power) and its clinical simulation labs are unlocked for everyone who has bought any Analysez AI study package.