Instrumental variables

What this page answers

The three pages before this one — DAGs, matching and weighting — all rest on the same premise: the confounders were measured. The grey nodes on a DAG are beyond the reach of matching and weighting. Those methods only find better ways to use the variables you already hold.

But in clinical research the confounder that matters most is usually the one nobody recorded: the fine gradations of disease severity, the clinician’s sense that this particular patient “does not look right”, the patient’s own health-seeking behaviour, socioeconomic position. Taiwan’s National Health Insurance claims database — the source most local database research runs on — holds prescriptions, diagnosis codes and laboratory values. It does not hold the reason the doctor wrote that prescription.

An instrumental variable (IV) is the only observational method that still has a chance in that situation. It does not adjust the confounder away; it goes around it. You look for an external factor that affects whether treatment is received and reaches the outcome through no other route, and then you use only the variation that factor creates.

The price is steep, and it belongs at the top rather than in a footnote: two of the three key assumptions can never be verified from data. Half of this page is about that.

A clinical scenario

The simulation is set up like this, and every role in it corresponds to something you meet in practice:

• U — disease severity, recorded in no field anywhere. Sicker patients are more likely to be started on the new drug (confounding by indication) and have worse outcomes to begin with.
• Z — which hospital the patient happened to attend. Hospitals differ in how readily they adopt the new drug (some departments switched early, others are still waiting). This is the instrument.
• A — whether the new drug was started (yes / no).
• Y — the symptom score at twelve weeks, lower being better. The drug’s true effect is to lower the score by 2.00 points.

The simulated data really do behave that way: treated and untreated patients differ by 0.99 standard deviations in severity — that is the confounding by indication — while patients at the two hospitals differ by only 0.022 standard deviations, because the instrument flattened severity out. That contrast is the whole hope of the method.

Three assumptions, two of which cannot be verified

The first is the only one you can inspect. In this simulation, 72.1% of patients at the enthusiastic hospital started the drug against 37.2% at the other — a gap of 34.9 percentage points. The main example, a single simulation with n = 5000, gives a first-stage F of 701. That is a very strong instrument.

The second and third cannot be established in any dataset, because what they describe is an absent path and an unmeasured common cause. You do not hold those variables, so there is no test whose passing would mean the assumption holds.

But not provable is not the same as nothing to do: these two can be refuted. Data can produce evidence incompatible with the assumptions even though it can never produce evidence in favour of them. Two checks are standard. (1) Stratify on the instrument and compare the measured covariates — in effect a Table 1 by instrument level. If the two strata already differ in age, comorbidity or severity, independence is in immediate trouble. (2) Use a negative control outcome — pick an outcome the treatment could not plausibly affect; if the instrument appears to “act” on that too, the claim that it works only through treatment does not stand. The simulation on this page has already done the first of these: the sentence in the previous section about the two hospitals differing by only 0.022 standard deviations of severity is the conclusion of that balance table.

Negative controls come in two directions

Check (2) above uses a negative control outcome: an outcome the exposure could not plausibly affect. It has a mirror image, the negative control exposure: an exposure that could not plausibly affect the outcome. Both ask the same question from opposite ends — “if my analysis were clean, there should be nothing here; is there?”

Lipsitch and colleagues set out three requirements, and a control missing any one of them is not really a negative control:

1. It shares the same confounders as the real exposure or outcome (otherwise it cannot detect the bias you care about)
2. There is no genuine causal path between it and the relationship under test (otherwise an association is exactly what you should expect)
3. It is measured and selected the same way as the main analysis (otherwise it cannot detect selection bias or measurement error)

The first is the one most often skipped and the one that matters most: a negative control sharing no confounders with the main analysis tells you nothing when it comes back clean.

A set of negative controls can calibrate

One negative control answers a yes-or-no question. A set of them can do more: estimate dozens of associations whose true value is known to be null, and the resulting distribution ought to sit on the null. When it does not, that displacement estimates the residual bias — and it can be used to make the main analysis’s p-values and intervals appropriately more conservative. This is empirical calibration, standard equipment in large database drug-safety work such as OHDSI and FDA Sentinel. The ITS and DiD page works through it on simulated data, including how much wider the calibrated intervals get.

There is a fourth assumption that often gets skipped, needed only when treatment is binary: monotonicity — nobody exists who would take the drug at the cautious hospital but refuse it at the enthusiastic one. When it holds, what an instrument estimates is the local average treatment effect (LATE): the effect only among the patients whose treatment was actually decided by the instrument, not the average treatment effect in everyone. Who are those patients? The data cannot identify them. This is the hardest property of an IV estimate to explain to a clinical colleague.

Running one for yourself

With a single binary instrument and a single treatment, two-stage least squares (2SLS) is identical to something far easier to read — the Wald ratio:

The numerator is “how much the outcome differs between hospitals”, the denominator “how much the treated proportion differs between hospitals”. You divide the difference in outcome by the difference in treatment. If switching hospitals moves the treated proportion by only about a third, the difference in outcomes has to be scaled up roughly threefold to describe what would happen if everyone were treated.

Writing it as a ratio has one advantage: you can see at a glance what happens when the denominator gets small. That is exactly the weak instrument problem.

set.seed(20260822)

simulate_cohort <- function(n, alpha_z, direct_z = 0) {
  U <- rnorm(n)                                   # severity: unmeasured
  Z <- rbinom(n, 1, 0.5)                          # hospital: the instrument
  A <- as.integer(-0.4 + alpha_z * Z + 0.9 * U + rnorm(n) > 0)   # drug started
  Y <- -2 * A + 2 * U + direct_z * Z + rnorm(n)   # true effect = -2
  data.frame(U, Z, A, Y)
}
d <- simulate_cohort(5000, alpha_z = 1.2)

# -- The naive approach: compare treated with untreated -------------------
summary(lm(Y ~ A, data = d))$coefficients["A", ]

# -- Wald ratio = 2SLS when there is one binary instrument ----------------
wald <- function(d) {
  (mean(d$Y[d$Z == 1]) - mean(d$Y[d$Z == 0])) /
  (mean(d$A[d$Z == 1]) - mean(d$A[d$Z == 0]))
}
wald(d)

# The exactly equivalent two-stage form (note: the second-stage standard
# error is wrong; fix it with a bootstrap or a dedicated package such as
# AER::ivreg / ivreg::ivreg)
stage1 <- lm(A ~ Z, data = d)
coef(lm(Y ~ fitted(stage1), data = d))[2]

# -- First-stage strength: the only assumption you can check --------------
summary(stage1)$fstatistic[1]        # convention: above 10 counts as not weak

# -- Confidence interval: bootstrap the whole procedure -------------------
boot <- replicate(1000, wald(d[sample.int(nrow(d), replace = TRUE), ]))
quantile(boot, c(0.025, 0.975))

# -- What only a simulation can do: put the unmeasured confounder in ------
summary(lm(Y ~ A + U, data = d))$coefficients["A", ]

import numpy as np, statsmodels.api as sm

rng = np.random.default_rng(20260822)

def simulate_cohort(n, alpha_z, direct_z=0.0):
    U = rng.normal(size=n)
    Z = rng.binomial(1, 0.5, size=n)
    A = (-0.4 + alpha_z * Z + 0.9 * U + rng.normal(size=n) > 0).astype(int)
    Y = -2 * A + 2 * U + direct_z * Z + rng.normal(size=n)
    return U, Z, A, Y

U, Z, A, Y = simulate_cohort(5000, alpha_z=1.2)

ols = sm.OLS(Y, sm.add_constant(A)).fit()
print(ols.params[1])                                   # the biased estimate

wald = (Y[Z == 1].mean() - Y[Z == 0].mean()) / (A[Z == 1].mean() - A[Z == 0].mean())
print(wald)

stage1 = sm.OLS(A, sm.add_constant(Z)).fit()
print(stage1.fvalue)                                   # first-stage F

oracle = sm.OLS(Y, sm.add_constant(np.column_stack([A, U]))).fit()
print(oracle.params[1])                                # only a simulation has this

Four estimates, one simulated dataset (the true effect is -2.00):

The second row is a disaster. The drug genuinely lowers the symptom score by 2.00 points, yet comparing treated with untreated patients directly returns -0.03 — confounding by indication has wiped the entire effect out, and the confidence interval crosses zero. “This analysis did not detect an effect of the drug” is an honest statement of what that row shows; but anyone reading only that row will take one more step and conclude that the drug does not work, and that step is the error.

The third row, the instrumental variable estimate, recovers it. The fourth is the answer only a simulation can give, and it confirms that the culprit really was the unmeasured severity.

Weak instruments

The first assumption is the only checkable one, and the consequences of failing that check are worse than they look. Turning the instrument’s strength from very weak up to very strong, with 400 simulations at each level:

The F values in the table below are medians over a separate set of repeated simulations, not the F from the single main example above.

Two things to take from it.

First, weak-instrument bias has a direction, and it is the bad one. In the weakest row the median estimate is -0.20 — that is not random scatter, it is a pull towards the naive estimate (-0.06 in this simulation). The intuition: when the instrument barely moves treatment, the denominator of the Wald ratio approaches zero, and any residual association left in the numerator — including the instrument’s own sampling noise — is magnified without limit. A weak instrument does not hand you an uninformative answer; it hands you the biased answer you started with, wrapped in a reassuringly wide confidence interval.

Second, getting the direction backwards is normal when the instrument is weak. In the weakest row, close to half of the simulations put the sign of the effect the wrong way round.

The conventional threshold is a first-stage F above 10. That number comes from simulation studies; it is a rule of thumb, not a theorem. The table shows estimates already close to unbiased once F is around 15, while the 90% range remains alarmingly wide. Recent methodological work argues that 10 is too permissive in weak-instrument settings, and there are inference procedures designed to be robust to weak instruments (the Anderson-Rubin test, for one). Report the F statistic itself, not the fact that it cleared a threshold.

When the exclusion restriction breaks

The right-hand panel above is the most important figure on this page. It gives the instrument a small direct effect on the outcome — breaking the second assumption — and changes nothing else:

A direct effect of only 0.50 — a quarter of the treatment’s own effect of 2.00 — moves the estimate from -2.01 to -0.55. The effect has all but disappeared.

And throughout, the first-stage F statistic stays in the hundreds. No diagnostic turns red.

Mendelian randomisation

The commonest instrument in medicine is genotype, and the design built on it is called Mendelian randomisation (MR).

The logic runs like this. Which allele you inherit from each parent is settled at conception and settled at random (Mendel’s law of segregation). So if a genetic variant affects the exposure you care about — LDL cholesterol, body mass index, alcohol intake — the first assumption is satisfied; and because the variant was fixed before birth, it cannot have been influenced by later lifestyle, socioeconomic position or disease status. Independence, the third assumption, therefore has a biological argument behind it rather than just optimism. That is MR’s central advantage over other instruments.

Here are the three assumptions in MR’s own vocabulary, with the way each one fails:

Pleiotropy is MR’s central problem, because pleiotropy is the exclusion restriction being broken — and the fact that the exclusion restriction cannot be verified applies here just as it does everywhere else. What is done in practice is to use many SNPs together and then compare a family of estimators built to be less sensitive to pleiotropy (MR-Egger, weighted median, MR-PRESSO): if different methods agree, the chance that pleiotropy fooled all of them in the same way is lower. That is triangulation, not verification.

When to reach for an instrumental variable

An instrumental variable is not a more advanced form of adjustment; it is a trade that only works under specific conditions. The situations that suit it share three features:

1. The most important confounder is definitely unmeasured, and you can name it. If the concern is only “there might be some residual confounding”, weighting plus a sensitivity analysis (an E-value, say) is usually the more practical route.
2. There is an external source of variation you can justify physically or institutionally, rather than a variable picked out of the dataset because it correlates with treatment and not with the outcome. Picking one that way means using data to verify an assumption data cannot verify.
3. The sample is large enough to survive a confidence interval three times wider.

When those do not hold, the honest move is to write unmeasured confounding into the limitations rather than force an IV analysis through. The target trial emulation page is worth reading alongside this one: many problems that look like unmeasured confounding are in fact design errors — an undefined time zero, the wrong comparator — and those are fixable by design.

Common misuses

Reproducing every number on this page

/opt/homebrew/bin/Rscript figures/scripts/B6-04-iv.R

Watch next

▶INSTRUMENTAL VARIABLE ANALYSES EXPLAINED

▶The Logic of Instrumental Variables

▶The 3 Instrumental Variables Assumptions

▶Introduction to Instrumental Variables (IV)

▶工具變數 (instrumental variables)

Report a problem on this page