AdvancedIndependently reviewed, not yet spot-checked by a human

ANCOVA, change scores and follow-up only

Three ways to analyse the same pre-post trial, why randomised trials reach for ANCOVA (the clue is in the baseline coefficient), how the efficiency gap moves with the pre-post correlation, and what regression to the mean looks like — plus why it is only clean in the control arm.

What the problem looks like

One of the commonest shapes in clinical trials is pre-post: everyone is measured once before randomisation (baseline) and once after treatment ends (follow-up). Blood pressure, weight, HbA1c, pain scores, depression scales — all the same shape.

Faced with data like that, an analyst has three roads:

Change score — compute follow-up minus baseline for each person, then compare mean changes between groups
ANCOVA (analysis of covariance) — use follow-up as the outcome and put baseline in as a covariate in a linear regression
Follow-up only — ignore baseline entirely and compare the follow-up means

All three estimate “the same treatment effect”, but they differ a great deal in how precisely they do it. And the difference between them can be written as one model:

Y_{\text{follow-up}} = \alpha + \beta \, Y_{\text{baseline}} + \tau \, Z_{\text{treated}} + \varepsilon

Follow-up only pins $\beta$ at 0; the change score pins $\beta$ at 1 (move $Y_{\text{baseline}}$ to the left-hand side and it becomes the change); ANCOVA lets the data choose $\beta$ . The rest of this page is about the consequences of that single fact.

The example on this page

MASS::anorexia holds the weights of 72 young women with anorexia, measured once before and once after treatment (in pounds). Three arms — Cognitive behavioural therapy with 29, Control with 26, and Family therapy with 17 — and CBT is the reference level. One line of code fetches it, and nobody has to apply for access to a database.

Arm	n	Mean baseline	Mean follow-up	Mean change
Cognitive behavioural therapy	29	82.69	85.70	+3.01
Control	26	81.56	81.11	-0.45
Family therapy	17	83.23	90.49	+7.26

The overall pre-post correlation is 0.332. That is on the low side, and it governs every efficiency comparison in the second half of this page, so it is worth holding on to.

Scatter plot with weight before treatment on the horizontal axis (pounds, roughly 70 to 95) and weight after treatment on the vertical axis (roughly 72 to 104). The three arms use different symbols and colours: CBT blue circles, control brown triangles, family therapy red squares, each with its own fitted line. A dashed line of slope 1 marks no change in weight. The family therapy line sits highest, entirely above the dashed line and clearly separated from it. The CBT line also lies entirely above the dashed line and never crosses it, but converges towards it from left to right. The control line is nearly horizontal, tilting slightly downwards, and is the only one of the three that crosses the dashed line — above it at the left end, below it at the right. The points scatter widely, consistent with an overall pre-post correlation of only 0.332. — Two things are visible in one picture: the arms differ in where their follow-up values sit (the treatment effect), and baseline predicts follow-up only weakly. What ANCOVA does is compare the height of the three lines at a common baseline value.Plotting script `figures/scripts/B2-12-ancova.R`

library(MASS)
data(anorexia, package = "MASS")

a <- anorexia
a$change <- a$Postwt - a$Prewt

# 1. Change score
summary(lm(change ~ Treat, data = a))

# 2. ANCOVA: follow-up as the outcome, baseline as a covariate
summary(lm(Postwt ~ Prewt + Treat, data = a))

# 3. Follow-up only
summary(lm(Postwt ~ Treat, data = a))

# All the difference is in the baseline coefficient: ANCOVA estimates it,
# the other two pin it down
cor(a$Prewt, a$Postwt)

# Regression to the mean: look inside the arm that got no treatment
co <- subset(a, Treat == "Cont")
tapply(co$change, co$Prewt <= median(co$Prewt), mean)

Verified with R 4.6.0 and MASS 7.3.65. All three models take CBT as the reference level of Treat, so the coefficients can be compared directly.

import statsmodels.api as sm
import statsmodels.formula.api as smf

a = sm.datasets.get_rdataset("anorexia", "MASS").data
a["change"] = a["Postwt"] - a["Prewt"]

print(smf.ols("change ~ C(Treat)", data=a).fit().summary())          # change score
print(smf.ols("Postwt ~ Prewt + C(Treat)", data=a).fit().summary())  # ANCOVA
print(smf.ols("Postwt ~ C(Treat)", data=a).fit().summary())          # follow-up only

print(a[["Prewt", "Postwt"]].corr())

statsmodels' ols formula interface is close to R's lm; the reference level of C(Treat) defaults to the alphabetically first, which here is also CBT.

Three analyses of one dataset

Forest plot with the difference from CBT in pounds on the horizontal axis and a dashed vertical line at 0. The upper three rows are control versus CBT: change score -3.46 with a confidence interval that crosses 0; ANCOVA -4.10 with an interval lying entirely to the left of 0; follow-up only -4.59, its interval also entirely to the left. The lower three rows are family therapy versus CBT: change score 4.26 with an interval crossing 0; ANCOVA 4.56 and follow-up only 4.80, both with intervals lying entirely to the right of 0. Each row is labelled on the right with its standard error and p-value. In both contrasts the change score has the longest bar. — One dataset, one contrast, three analyses. The point estimates land close together; what differs is the length of the intervals, which is to say the standard errors.Plotting script `figures/scripts/B2-12-ancova.R`

Contrast	Analysis	Estimate (lb)	SE	p
Control vs CBT	Change score	-3.46	2.03	0.094
	ANCOVA	-4.10	1.89	0.034
	Follow-up only	-4.59	1.97	0.023
Family therapy vs CBT	Change score	4.26	2.30	0.068
	ANCOVA	4.56	2.13	0.036
	Follow-up only	4.80	2.23	0.035

Start with the thing that is easiest to state wrongly. The change score gives p-values of 0.094 and 0.068 for the two contrasts, which do not reach statistical significance at the 0.05 level; the two ANCOVA p-values fall below 0.05.

That does not mean the change score “showed the treatment had no effect”, and it is not a story about switching methods until something turns significant. The three point estimates are in fact close together — the three estimates for the control contrast span -4.59 to -3.46, less than one standard error apart. What is really moving is the standard error:

For the control contrast, largest to smallest: change score 2.03, follow-up only 1.97, ANCOVA 1.89
The family therapy contrast has the same ordering: 2.30, 2.23, 2.13

ANCOVA is genuinely the most precise of the three on this dataset. Note the magnitude, though: it is only 3.8% narrower than follow-up only for the control contrast, and 6.9% narrower than the change score. At that scale, whether a p-value lands on one side of 0.05 or the other is a fragile thing and not worth reporting as a conclusion. Why the efficiency gap is so small is answered in the next section and again in the fifth.

Why randomised trials reach for ANCOVA

The clue is the coefficient on baseline. ANCOVA estimates $\hat\beta$ = 0.434 (SE 0.161, 95% CI 0.11 to 0.76).

Test that coefficient against each of the two values the other methods pin it to:

Null hypothesis	Which method assumes it	t	p
Baseline coefficient = 0	Follow-up only	2.70	0.009
Baseline coefficient = 1	Change score	-3.51	< 0.001

This dataset rejects both assumptions. Baseline clearly is related to follow-up, so pretending it is not costs you something; but the relationship is nowhere near one-to-one, so subtracting is wrong too. ANCOVA is the most efficient of the three here precisely because it does not have to guess that coefficient in advance — it estimates it. (Whether it is always the best is the next section’s question, and the answer is no.)

Two common misconceptions, dealt with here.

“Randomisation balances baseline, so there is no need to adjust for it.” The point of adjusting for baseline is not to repair an imbalance; it is to reduce residual variance. Even with identical baseline means in both arms, putting baseline in the model shrinks the standard error, because it explains away part of the person-to-person variation. This is a question of efficiency, not of bias.

“Adjust only when baseline is imbalanced.” The reverse is wrong too. In a randomised trial, deciding whether to adjust after seeing an imbalance makes the analysis depend on the outcome, which breaks the properties of the p-values and confidence intervals. Which baseline variables to adjust for should be fixed before anybody looks at the data.

Non-randomised observational studies are a different matter. There, baseline differences may arise from the selection mechanism itself, so adjusting for baseline is not only about efficiency but about what is being estimated at all. That is beyond this page; see DAGs and what to adjust for.

How the efficiency gap moves with the correlation

Textbooks often say “the higher the pre-post correlation, the more ANCOVA wins”. That claim cannot be checked on a single dataset, because a dataset has exactly one correlation. So this section simulates instead: hold the sample size and the true effect fixed, and move only the correlation, from 0 up to 0.9.

The parameters are 36 per arm (which comes to exactly the 72 patients in anorexia), a true difference of 0.5 standard deviations, 2000 replications at each grid point, $\alpha$ = 0.05, and a seed fixed at 20260823.

Two line charts side by side, both with the pre-post correlation from 0 to 0.9 on the horizontal axis. In the left chart the vertical axis is the mean standard error: the change score (brown) starts highest and falls steeply; ANCOVA (blue) starts lower and keeps falling, staying below the brown line throughout; follow-up only (red) is almost a flat horizontal line. The brown line crosses down through the red line near a correlation of 0.5. Two vertical reference lines mark the correlation actually observed in anorexia, 0.33, and 0.5. In the right chart the vertical axis is power as a percentage: the change score climbs from about 33% towards 100%; ANCOVA starts around 56% and is above the change score at 18 of the grid points, the two lines converging to a tie only at the highest correlation; follow-up only is almost flat, staying between 53% and 58% across the whole range. — 36 per arm with the true difference held fixed; only the pre-post correlation moves. The change score does not catch follow-up only until the correlation reaches 0.5, and it never overtakes ANCOVA.Plotting script `figures/scripts/B2-12-ancova.R`

Correlation	SE: change	SE: ANCOVA	SE: follow-up	Power: change	Power: ANCOVA	Power: follow-up
0.00	0.333	0.237	0.235	33.3%	56.3%	57.0%
0.20	0.297	0.233	0.236	40.0%	56.0%	54.5%
0.35	0.269	0.222	0.235	45.9%	61.4%	57.1%
0.50	0.234	0.205	0.235	56.8%	66.0%	53.9%
0.70	0.182	0.169	0.235	76.2%	82.1%	53.4%
0.90	0.105	0.103	0.235	99.9%	99.9%	56.1%

Three things, stated as measured:

The change score does not beat ANCOVA at a single one of the 19 grid points — its mean standard error is the smaller of the two at 0 of them. It crosses follow-up only at a correlation of 0.50, and below that value analysing change scores is worse than not looking at baseline at all.
ANCOVA is not best at every grid point either. It wins 16 of the 19; at correlations of 0.00, 0.05, 0.10 the standard error of follow-up only is very slightly smaller (at a correlation of 0 it is 0.235 against 0.237). The reason is plain: ANCOVA estimates one extra parameter, and when that parameter is of no use the degree of freedom is simply spent. The gap is small, but it is real.
Anorexia sits at a correlation of 0.332, whose nearest grid point is 0.35. There, the standard error of ANCOVA (0.222) is only slightly better than that of follow-up only (0.235) — which is exactly where the “only a few percent narrower” of the third section comes from.

The simulation is printed next to a closed form, and that is worth learning as a habit.

The large-sample standard errors of all three estimators can be written down ( $n$ per arm, outcome variance 1):

\mathrm{SE}_{\text{follow-up}} = \sqrt{\tfrac{2}{n}}, \quad \mathrm{SE}_{\text{change}} = \sqrt{\tfrac{4(1-\rho)}{n}}, \quad \mathrm{SE}_{\text{ANCOVA}} = \sqrt{\tfrac{2(1-\rho^{2})}{n}}

Setting the second equal to the first gives $4(1-\rho) = 2$ , that is $\rho = 0.5$ — matching the crossing point of 0.50 measured in the simulation. Dividing the third by the first gives $\sqrt{1-\rho^{2}}$ , which is at most 1, and that is where “ANCOVA is never worse than follow-up only” comes from. (The large-sample formula cannot see the one lost degree of freedom, which is why the simulation departs from it slightly at low correlations.)

A simulation is only trustworthy once something that is not a simulation can check it. Every row of the stats file stores the simulated and the analytic value together, precisely to preserve that comparison.

Regression to the mean

Regression to the mean is this: because every measurement carries a random component, people who measured unusually low at baseline tend to measure higher next time, and people who measured unusually high tend to measure lower. No treatment is needed for this, and no real change either — the randomness of measurement is sufficient on its own.

The cleanest place to watch it is the arm that received no treatment. Split the control arm at its median baseline weight (80.65 lb):

Half of the control arm	n	Mean baseline	Mean follow-up	Mean change
Lower half at baseline	13	76.83	80.68	+3.85
Upper half at baseline	13	86.28	81.53	-4.75

The lighter half at baseline gained 3.85 lb on average and the heavier half lost 4.75 lb, a difference of 8.61 lb (Welch t = 3.22, p = 0.004). The mean change across the whole control arm is only -0.45 lb — and none of these people were treated.

Scatter plot of the control arm alone. Weight before treatment on the horizontal axis (about 70 to 92 lb) and change in weight on the vertical axis (about -12 to +16 lb). Brown triangles are scattered across it, with a dark fitted line descending clearly from upper left to lower right. A horizontal dashed line marks zero change and a vertical dotted line marks the median baseline of 80.65. Over the left half a blue horizontal segment marks that half's mean change of +3.85 lb, above the zero line; over the right half a red horizontal segment marks its mean change of -4.75 lb, below the zero line. The two segments fall on opposite sides of zero. — The control arm, which received no treatment. The lighter half gained weight on average and the heavier half lost it, and the two segments land on opposite sides of zero.Plotting script `figures/scripts/B2-12-ancova.R`

The pattern is not visible in the two treated arms, so it must not be described as a property of the whole dataset.

The same median split applied to all three arms:

Arm	Mean change, lighter half	Mean change, heavier half	Difference
Cognitive behavioural therapy	+2.55	+3.49	-0.94
Control	+3.85	-4.75	+8.61
Family therapy	+6.26	+8.40	-2.14

Only the control arm splits into “the light gain and the heavy lose”. In the other two arms the gap is not only much smaller but points the other way. Regression to the mean has not gone away; the treatment effect has covered it — in a treated arm, the observed change is the treatment effect and regression to the mean mixed together, and the two cannot be separated.

One more thing belongs here. Inside the control arm of this dataset the pre-post correlation is -0.161, far below the overall 0.332 and in fact negative. The lower the correlation, the larger the regression to the mean, so the demonstration in this control arm is on the dramatic side and not the magnitude a typical pre-post study would show.

Neither of those two calculations is an unbiased estimate of how large regression to the mean is.

Splitting at the median and regressing change on baseline (slope -1.134, SE 0.167, p = 5.3e-7) both condition on the baseline measurement. That measurement carries error, and the same error reappears in the change with the opposite sign, so both calculations fold measurement error in with regression to the mean.

They are good for seeing the phenomenon and poor for reporting its size. For what measurement error does on its own, see measurement error and misclassification.

In practice the answer is not to estimate it but to design so that it cannot fool you:

Have a control arm. Regression to the mean acts on both arms, so a between-arm comparison cancels it. A single-arm study’s “they improved” has almost no way to rule this explanation out.
Do not enrol on an extreme value and then compare against that baseline. A design that recruits “everyone above an LDL threshold” guarantees regression to the mean by construction.
Do not let one measurement serve as both the screening value and the baseline value. Measuring them separately removes part of the problem.

Common misuses

Misuse	Why it is wrong
Choosing whichever of the three gives the smallest p-value afterwards	Three shots at the same people; undeclared multiplicity
Writing a non-significant change score up as “the treatment did not work”	Not reaching significance only means this analysis detected no difference
Skipping baseline adjustment because “randomisation already balanced it”	Adjustment is for reducing residual variance, not for repairing imbalance
Deciding to adjust only after seeing an imbalance	The analysis then depends on the outcome, invalidating p-values and intervals
Believing ANCOVA is the most efficient choice under all conditions	At low correlations the extra degree of freedom can make it slightly worse than follow-up only
Believing a change score always beats ignoring baseline	Below a correlation of 0.5 its standard error is actually larger
Treating “they improved” in a single-arm study as evidence of efficacy	Regression to the mean and the natural course produce the same result
Enrolling on extreme values and then comparing before with after	The entry criterion guarantees regression to the mean
Reading “the lighter half improved more” in a treated arm as regression to the mean	Treatment effect and regression to the mean are mixed and cannot be separated
Quoting the slope of change on baseline as the size of regression to the mean	Baseline measurement error is folded in; it is not an unbiased estimate
Reporting only the overall F test for a three-arm trial	Readers need the estimate and interval for each contrast
Reading the ANCOVA baseline coefficient as “the effect of baseline treatment”	It is the slope of a covariate, not any between-arm contrast

Reproducing every number on this page

/opt/homebrew/bin/Rscript figures/scripts/B2-12-ancova.R

Read the figure

The answer comes from the same statistical output that produced this page's figures, not from a number typed in beside them.

For the control-versus-CBT contrast, change scores, ANCOVA and follow-up-only give three estimates. What separates them?

Show the answer and why

Correct answer: ANCOVA gives -4.10 - the three differ in what they assume about the baseline coefficient, and ANCOVA lets the data decide

ANCOVA gives -4.10, change scores -3.46, follow-up-only -4.59. All three estimate the same contrast and differ only in what they assume about the baseline coefficient: change scores pin it at one (baseline carries over point for point), follow-up-only pins it at zero (baseline is irrelevant), and ANCOVA estimates it. So they are neither different contrasts nor the same number - the spread comes from three assumptions, two of which were imposed rather than checked.

In the ANCOVA model, the confidence interval on the baseline weight coefficient excludes both zero and one. What follows?

Show the answer and why

Correct answer: The coefficient is 0.434 - excluding zero rules out follow-up-only, and excluding one rules out change scores

The estimate is 0.434 with an interval from 0.113 to 0.756. Excluding zero says baseline really does predict follow-up, so follow-up-only - which sets the coefficient to zero - does not hold; excluding one says baseline does not carry over point for point, so change scores - which set it to one - do not hold either. 0.113 and 0.756 are the two endpoints, and either alone rules out only one of the two. It is also why "should baseline be adjusted for" is not a matter of taste: the data have something to say about this coefficient.

Splitting the untreated control arm at the median baseline weight gives one half a mean gain and the other a mean loss, with a statistically significant gap. What is this?

Show the answer and why

Correct answer: The lower-baseline half changes by 3.85 - splitting on baseline and then looking at change manufactures exactly this symmetric artefact

This arm received no treatment at all, yet the lower-baseline half changes by 3.85 on average and the upper half by -4.75, a gap of 8.61 that reaches statistical significance. With no treatment the gap cannot be a response to one: measurement carries error, so some patients land in the "low" half only because they measured low that day, and measuring again moves them up. That is regression to the mean. Grouping on a baseline value and then examining change guarantees this symmetric artefact, and in a report it reads very much like "the sickest improved most".

Chapters that use this method

Randomised controlled trial

Sources and licences

This page is original writing