ANCOVA Protocol: A Step-by-Step for Comparing Groups While Controlling for a Covariate
- 2 hours ago
- 7 min read
Your treated and control wells started at different confluence. Your two patient groups differ in age. Your pre-test scores weren't balanced across arms. Compare the endpoints with a plain ANOVA and you've just measured your treatment effect plus the head start, with no way to tell them apart.
Analysis of covariance (ANCOVA) is the standard fix: it compares group means after adjusting for a continuous covariate, which both removes that bias and usually buys you statistical power by pulling nuisance variance out of the error term. But it only works if you check one assumption that most people skip, and if you skip it your adjusted means can be meaningless.
Principle. ANCOVA fits a single regression line relating the covariate to the outcome, with a separate intercept for each group. The vertical gap between those parallel lines is the adjusted group difference, evaluated at a common covariate value (usually the covariate's grand mean). That construction is why the parallel-slopes requirement isn't a technicality: if the lines aren't parallel, the gap between them changes depending on where you look, and there is no single group difference to report.
This ANCOVA protocol takes you from design through reporting, with the assumption checks in the order they actually need to happen.
What you need before you start
Item | Requirement |
Dependent variable | Continuous |
Independent variable | Categorical (2+ groups) |
Covariate(s) | Continuous, theoretically related to the outcome |
Covariate timing | Measured before treatment, or otherwise unaffected by it |
Design | Independent observations (one value per experimental unit) |
Software | Any GLM-capable tool (R, SPSS, Prism, Python/statsmodels) |
Minimum output needed | Interaction test, Levene's test, residual plots, adjusted (estimated marginal) means, effect size |
Caution: if your covariate was measured after treatment and could have been affected by it, stop. Adjusting for a variable on the causal path removes part of the treatment effect you're trying to measure. Choose a different covariate or a different analysis.
ANCOVA protocol: set up
Confirm ANCOVA is the right tool. You need a categorical predictor, a continuous outcome, and a continuous covariate you want to control for. If you have no covariate, run ANOVA. If your predictor is continuous, run regression.
Choose covariates from the biology, before you see the results. Pick variables plausibly related to the outcome and not affected by the treatment: baseline value, age, passage number, batch. Keep the set small; each covariate costs a degree of freedom.
Record your model specification (outcome, group variable, covariate list) before fitting anything. Deciding covariates after seeing p-values is p-hacking.
Plot the raw data first: covariate on the x-axis, outcome on the y-axis, points colored by group. This single plot previews almost every assumption you're about to test.
Check the ANCOVA assumptions in order
Check linearity. In the scatterplot from step 4, confirm the covariate-outcome relationship is linear within each group. A curved relationship mis-estimates adjusted means and can even reverse the apparent direction of group differences. If curved, transform the covariate first.
Test homogeneity of regression slopes by fitting the interaction model first. Run the model including the group × covariate interaction term. This is the assumption unique to ANCOVA and the one most often skipped, so do it before the main analysis, not after.
Read the interaction term. If the interaction is non-significant, the slopes are acceptably parallel; drop the interaction and proceed to step 9. If the interaction is significant, stop and go to step 8.
If slopes differ, do not report an adjusted mean. A significant interaction means the treatment effect depends on the covariate, so no single adjusted difference exists. Report the interaction as your finding and use Johnson-Neyman to locate where the group difference holds.
Fit the ANCOVA model (group + covariate, no interaction).
Check normality of residuals with a Q-Q plot of the model residuals, supported by a Shapiro-Wilk test if you want a formal check. Mild deviations are tolerable at reasonable sample sizes.
Check homogeneity of variance with a residuals-versus-fitted plot and Levene's test. If Levene's test is significant, the group variances differ and standard errors (and therefore your F-test and confidence intervals) become unreliable.
Check for influential outliers. Inspect standardized residuals and leverage. Investigate any extreme point on its merits; never delete data solely because it moved your p-value.
Note: independence cannot be tested statistically. It comes from the design, so confirm your rows are genuinely independent experimental units rather than technical replicates of the same unit.
Run and interpret the model
Read the group effect. This is the difference between group means after adjusting for the covariate, reported as an F-test with degrees of freedom and a p-value.
Read the covariate effect. This is the pooled within-group relationship between covariate and outcome. A significant covariate confirms the adjustment was worth making.
Report an effect size, typically partial eta-squared for the group effect, so the reader sees magnitude and not just significance.
Obtain the adjusted (estimated marginal) means for each group, with standard errors or confidence intervals. These are the group means evaluated at a common covariate value and are the numbers you interpret.
Run post-hoc comparisons on adjusted means if you have three or more groups and the overall effect is significant, applying a correction for multiple comparisons.
Compare adjusted with unadjusted means. Report both. The difference between them shows the reader exactly how much work the covariate did, which is information they can't get any other way.
Tip: in a properly randomized experiment, randomization has already broken the link between treatment and confounders, so covariate adjustment is buying you precision rather than removing bias. Both are legitimate reasons to run ANCOVA, but say which one applies.
Report your ANCOVA so reviewers don't send it back
State the model in the methods: outcome, grouping factor, covariate(s), and that ANCOVA was used.
State that you tested homogeneity of regression slopes, and give the interaction result. Reviewers look for this; its absence is one of the most common ANCOVA queries.
Report the group effect as F(df1, df2) = value, p = value, with an effect size.
Report adjusted means with intervals for each group, and the unadjusted means alongside them.
Report the assumption checks you ran, including Levene's test and residual diagnostics, in one brief sentence each.
Name your n and what it refers to (independent experimental units, not technical replicates).
A complete sentence looks like: "An ANCOVA compared post-treatment migration across three groups controlling for baseline confluence. The group × covariate interaction was non-significant (p = 0.41), supporting homogeneity of slopes. Treatment significantly affected migration, F(2, 42) = 6.81, p = 0.003, partial η² = 0.25. Adjusted means were 62.4 (95% CI 58.1–66.7), 71.9 (67.6–76.2), and 80.3 (76.0–84.6)."
Acceptance criteria
Your ANCOVA is defensible when all of these are true:
The covariate was measured before treatment, or is demonstrably unaffected by it.
Covariates were chosen a priori, from the biology.
The covariate-outcome relationship is linear within each group.
The group × covariate interaction was tested and is non-significant (or a Johnson-Neyman analysis is reported instead).
Residuals are approximately normal, with no severe heteroscedasticity.
Observations are independent experimental units.
Adjusted means, effect size, and unadjusted means are all reported.
ANCOVA protocol: Troubleshooting
Symptom | Cause | Fix |
Group × covariate interaction is significant | Slopes are not parallel; effect depends on covariate | Report the interaction; use Johnson-Neyman to find where the difference holds (step 8) |
Adjusted means differ wildly from unadjusted | Strong covariate imbalance between groups | Legitimate, but state it plainly; check for lack of covariate overlap between groups |
Levene's test significant | Unequal group variances | Consider transformation, a robust/heteroscedasticity-consistent variance estimator, or a Welch-type approach |
Curved pattern in the scatterplot | Non-linear covariate relationship | Transform the covariate, or fit a non-linear term; a linear ANCOVA will mis-adjust |
Treatment effect disappears after adjustment | Covariate may be a mediator, not a confounder | Re-examine the causal logic; adjusting for a mediator causes overcontrol bias |
Adjusted means fall outside the observed data range | Extrapolation from poor covariate overlap | Restrict interpretation to the covariate range where groups overlap |
Results change depending on which covariates are included | Covariates chosen post hoc | Pre-specify the model; report the pre-specified analysis as primary |
Where to go next with your covariate analysis
ANCOVA is one framing of a broader model, so it helps to know its neighbors. Drop the covariate and it becomes ANOVA. Make the predictor continuous and it becomes regression. Because these are all the same general linear model, it's worth reading our guide on multiple regression for dose-response and covariate analysis for the wider view, and our piece on replicates if you're unsure whether your rows are truly independent. Talking the model specification through before you fit it is the cheapest way to avoid a rebuild (Sophie's analytics will check covariate choice and flag assumption violations in the output).
ANCOVA protocol - FAQ
What is ANCOVA used for? ANCOVA compares group means on a continuous outcome while statistically adjusting for one or more continuous covariates. It's used to remove bias when groups differ at baseline and to increase power by removing nuisance variance from the error term.
What is the difference between ANOVA and ANCOVA? ANOVA compares group means with no adjustment. ANCOVA adds one or more continuous covariates and compares adjusted means. Use ANOVA when groups are equivalent at baseline and no strong covariate exists; use ANCOVA when baseline differences or nuisance variables would otherwise confound or obscure the comparison.
What is the homogeneity of regression slopes assumption? It requires that the covariate has the same linear relationship with the outcome in every group, meaning the regression lines are parallel. Test it by fitting the group × covariate interaction. If the interaction is significant, the slopes differ, the adjusted group difference depends on the covariate value, and a standard ANCOVA adjusted mean should not be reported.
What do I do if the homogeneity of slopes assumption is violated? Report the interaction as your finding rather than a single adjusted difference, and use the Johnson-Neyman procedure to identify the covariate values over which the group difference is statistically significant. A violated assumption here often reveals a genuine interaction worth reporting.
Can my covariate be measured after treatment? It shouldn't be. A covariate affected by the treatment may lie on the causal path between treatment and outcome, and adjusting for it removes part of the effect you're measuring (overcontrol bias). Use covariates measured at baseline or otherwise independent of treatment.
What should I report from an ANCOVA? Report the model specification, the homogeneity-of-slopes test result, the F-statistic with degrees of freedom and p-value, an effect size such as partial eta-squared, and the adjusted means with confidence intervals alongside the unadjusted means.
