Multiple Regression, Multicollinearity (VIF), and Overfitting
Diagnosing redundant predictors with Variance Inflation Factor (VIF), penalising model complexity with Adjusted R², and respecting the Events-Per-Variable rule.
In this lesson you will learn
- 1Calculate and interpret Tolerance (1 − R²_j) and Variance Inflation Factor (VIF = 1 / Tolerance) to detect multicollinearity.
- 2Distinguish raw R² from Adjusted R² and Standardized Beta (β) from Unstandardized B.
- 3Apply the 10–20 observations (or events) per predictor variable rule to prevent regression overfitting.
On this page▼
Diagnosing Multicollinearity with VIF & Adjusted R²
This section explains how collinear predictors inflate standard errors via VIF = 1 / (1 − R²_j) and how Adjusted R² guards against model overfitting.
Entering two nearly identical biomarkers (such as Total Cholesterol and LDL, or Weight and BMI) into the same regression can make both appear non-significant due to VIF inflation.
When two predictor variables X₁ and X₂ measure nearly the same physiological construct, the regression engine cannot separate their individual effects. The variance of each slope estimate is multiplied by the Variance Inflation Factor (VIF):
Tolerance = 1 − R²_j | VIF = 1 / (1 − R²_j) | SE Inflation Multiplier = √VIF
- VIF = 1.0: Orthogonal predictor (zero correlation with other predictors).
- VIF < 5.0: Acceptable collinearity for clinical models.
- VIF ≥ 5.0 to 10.0: High collinearity—inspect redundancy and consider retaining only the clinically primary variable.
- Events Per Variable (EPV): Maintain at least 10–20 observations (in OLS) or 10–20 outcome events (in logistic/Cox regression) per predictor degree of freedom to prevent overfitting.
Multicollinearity (VIF) & Adjusted R² Diagnostic Lab
Slide predictor inter-correlation (r₁₂) to see how VIF inflates standard errors
Clinical Example
A researcher enters both Fasting Glucose and 2-Hour Postprandial Glucose (inter-correlation r = 0.94) into a linear model predicting HbA1c.
Interpretation: With r = 0.94 between the two glucose measures, VIF reaches 8.59, nearly tripling the standard error (×2.93) of both slopes. Retaining only Fasting Glucose restores VIF to 1.25.
Common Pitfall
Including both a component variable and its derived index (such as Systolic BP, Diastolic BP, and Mean Arterial Pressure MAP) simultaneously as predictors in the same regression model.
Included with Any Analysez AI Study Package
This deep interactive chapter (Multiple Regression, Multicollinearity (VIF), and Overfitting) and its clinical simulation labs are unlocked for everyone who has bought any Analysez AI study package.