The Multiplicity Problem & The Omnibus F-Ratio
How repeated pairwise t-tests escalate the Familywise Error Rate and how One-Way ANOVA keeps overall alpha at 5%.
In this lesson you will learn
- 1Compute the Familywise Error Rate (1 − (1 − α)^c) across k study groups to explain why unadjusted multiple t-tests are invalid.
- 2Decompose total sum of squares into Between-Group signal (MS_between) and Within-Group noise (MS_within) to form F.
- 3Interpret Eta-Squared (η² = SS_between / SS_total) as the proportion of outcome variance explained by group membership.
On this page▼
Multiplicity & Variance Partitioning
This section defines Familywise Error Rate (FWER), the Omnibus F-ratio, Between-Group vs Within-Group Mean Squares, and Eta-Squared (η²).
Reviewers immediately reject manuscripts that compare 3 or more study arms using separate unadjusted t-tests.
When a clinical trial includes k ≥ 3 arms (for example: Placebo, Low-Dose, and High-Dose), comparing every pair requires c = k(k − 1) / 2 tests. Because each independent test carries a 5% false-positive risk, the cumulative probability of making at least one Type I error across the family of tests is:
FWER = 1 − (1 − α)^c | F = MS_between (Treatment Signal) / MS_within (Within-Arm Noise)
Sir Ronald Fisher developed Analysis of Variance (ANOVA) to solve this problem. Instead of running multiple t-tests, ANOVA compares the variance between group means against the average variance within groups in a single ratio F. Under the null hypothesis of equal population means, F ≈ 1.0. When treatments diverge significantly, F rises well above 1.0.
Multiplicity Error Rate & Omnibus F-Ratio Calculator
Watch false-positive probability escalate with multiple t-tests and see how within-group noise shapes F
Clinical Example
A 4-arm knee osteoarthritis trial (N = 160; 40 per arm) compares 8-week VAS pain reduction across Placebo, Celecoxib 100 mg, Celecoxib 200 mg, and a Biologic regimen.
Interpretation: Running 6 separate t-tests would have inflated false-positive risk to 26.5%. The single Omnibus ANOVA confirms a highly significant difference across the 4 regimens (F(3, 156) = 194.21, p < .001) explaining 78.9% of pain variance.
Common Pitfall
Running post-hoc pairwise comparisons and reporting a 'significant difference between Group A and Group C' when the overall Omnibus ANOVA F-test was non-significant (p > .05).
Included with Any Analysez AI Study Package
This deep interactive chapter (The Multiplicity Problem & The Omnibus F-Ratio) and its clinical simulation labs are unlocked for everyone who has bought any Analysez AI study package.