Weighted log-rank, max-combo, and milestone survival

The log-rank test treats every event time as equally important

Kaplan-Meier and the log-rank test took the statistic apart: at each event time it computes observed deaths minus the deaths expected under the null, adds those differences up, and divides by a standard error. Written out:

The standard log-rank test sets every $w_j$ to 1. That is a choice, not a law, and it is the optimal choice only when the hazard ratio is constant over the whole of follow-up.

If the two curves cross — one arm worse early, then overtaking — the early $O_j - E_j$ terms are positive and the late ones negative, so an unweighted sum cancels them against each other and the statistic collapses towards zero. That is not evidence that the arms are alike; it is this way of adding up being unable to see the difference. Immunotherapy trials in which the two progression-free survival curves overlap for the first six months and separate afterwards have been routine since 2019, and that is exactly the shape $w_j = 1$ handles worst.

A weighted log-rank test does one thing: it replaces $w_j$. Everything else is unchanged.

The example on this page: data whose proportional-hazards assumption clearly fails

survival::veteran is the US Veterans Administration lung cancer trial: 137 patients, 128 deaths, and only 9 censored observations. It was chosen not because it produces a dramatic result but because its proportional-hazards assumption genuinely does not hold:

The model is coxph(Surv(time, status) ~ trt + karno + celltype) and the check is the cox.zph() procedure from the assumptions page. The global p is 3.6e-4, so the violation is not marginal.

What the weights look like

The Fleming-Harrington family sets the weight from two parameters, $\rho$ and $\gamma$:

$\hat{S}$ is the Kaplan-Meier estimate from the pooled sample. Three corners of that family:

• $\rho = 0,\ \gamma = 0$: the weight is always 1 — the standard log-rank test.
• $\rho = 1,\ \gamma = 0$: the weight equals $\hat{S}$, so it decreases over time and is early-weighted (the Peto / Gehan idea).
• $\rho = 0,\ \gamma = 1$: the weight equals $1 - \hat{S}$, increases over time, and is late-weighted.

That is enough abstraction. Here is the pooled KM estimate and all three weights at quantiles of the observed event times:

Read the last row. By day 295, where the latest 10% of events sit, $\rho = 1$ assigns a weight of only 0.117 — against 0.898 on day 10. In the same dataset, a late death contributes 13% of what an early death contributes, which is close to not looking at late follow-up at all. The FH(0, 1) column is the mirror image: 0.883 at that late time point against 0.102 on day 10.

Why a delayed effect needs FH(0, 1)

The immunotherapy shape goes like this: for the first few months the curves are glued together (the drug has not taken effect yet, and some patients will never respond), and then the treated curve flattens while the control curve keeps falling. The real signal is entirely in the late stretch.

The cost of $w_j = 1$ here is that the long run of early event times, where there is nothing to see, enters $U$ carrying exactly as much weight as the late ones, diluting the difference. The table above shows how large that problem is: event times are dense early and sparse late, so in practice equal weighting is early weighting. FH(0, 1) simply corrects that implicit tilt in the other direction.

Changing the weight did not change the conclusion — and that is the lesson

First the four-way comparison across tumour cell types (4 groups, df = 3):

Group sizes and event counts are squamous 35 patients with 31 deaths; smallcell 48 patients with 45 deaths; adeno 27 patients with 26 deaths; large 27 patients with 26 deaths. Now the two-arm treatment comparison:

The cell-type comparison is significant under all four weights, so the conclusion does not move; the treatment comparison fails to reach significance under all four weights, so that conclusion does not move either. The machine-readable form of this is the anyConclusionChanged field, which is false and false for the two contrasts.

Max-combo: multiplicity as a number rather than a warning

If you do not know in advance whether the signal will arrive early or late, an honest option is to run several weights at once, take the most extreme statistic, and then charge yourself for having looked at several. That is max-combo.

The three components used here are rho0, rho1, fh01. Their z values (computed from observed minus expected in the Standard arm, so a negative sign means fewer deaths than expected in that arm) are:

That table explains in full why the log-rank test sees nothing here. The early-weighted z is -0.933 and the late-weighted z is 0.898: opposite signs — early and late point in different directions, which is the crossing shape in the milestone KM plot in the next section. The standard log-rank test adds those two stretches with equal weight and lands on z = -0.091, essentially zero, which is the p = 0.928 in the table above.

The three statistics are strongly correlated, because they are weighted sums over the same event times:

With the correlation matrix in hand you can integrate the trivariate normal and price the act of taking a maximum:

Here multiplicity is not a reminder to correct for something, it is a visible number: 0.351 becomes 0.549. For comparison, a Bonferroni correction over the same three tests would give 1.052 (capped at 1 in practice), far more conservative than max-combo — because Bonferroni assumes the three tests are independent, and the correlation matrix above says they are anything but. The more correlated the tests, the smaller the multiplicity price that is actually due, which is the same point made on multiple comparisons.

Milestone survival: the difference in survival at a stated day

Milestone survival is the version that needs the least statistical training to read — one-year survival was X% against Y%, a difference of Z percentage points — and it depends on no proportionality assumption at all. It reads the height of the KM curve at one vertical slice.

The three time points (the difference is Standard minus Test):

Note that the sign of the difference flips: 16.7 percentage points on day 90, -2.0 on day 180, and -3.9 on day 365. This is precisely what the log-rank p of 0.928 conceals: a statistic that compresses the whole of follow-up into one number cannot tell you that the sign changed sides along the way.

Note the day 90 row as well: the 95% CI for the difference runs from 0.1 to 33.2 percentage points, so the lower bound only just clears zero. An interval sitting on the edge like that is the one to ask about: was this time point fixed in advance? If it was picked from three after seeing the plot, it cannot be read at face value, for the same reason as in the max-combo section.

The two interval transforms do not agree

There are two common ways to build a confidence interval for a survival probability, and they differ in whether the interval is computed on a transformed scale or directly on the probability scale. Laying out day 365 for both arms:

The point estimate is the same 0.071 either way, but the lower bound for the Standard arm is 0.005 under one and 0.023 under the other — a factor of 4.7. The reason is that a Greenwood interval is symmetric on the probability scale: once the estimate sits close to 0, the lower bound is pushed towards 0 and a slightly smaller estimate would push it below 0, which a survival probability cannot be. The log-log transform builds the interval on the unbounded $\log[-\log \hat{S}]$ scale and maps it back, so the interval always lands between 0 and 1 and is asymmetric. That is why oncology trials report log-log intervals.

Running it yourself

library(survival)
data(cancer, package = "survival")
vet <- veteran
vet$trt_f <- factor(vet$trt, levels = c(1, 2), labels = c("Standard", "Test"))

# --- FH(rho, 0): survdiff takes rho only, weight = S(t-)^rho ---------------
survdiff(Surv(time, status) ~ trt_f, data = vet, rho = 0)    # standard log-rank
survdiff(Surv(time, status) ~ trt_f, data = vet, rho = 1)    # early-weighted

# --- FH(0, 1): out of reach for survdiff, so build the risk set by hand ----
et <- sort(unique(vet$time[vet$status == 1]))
nj <- sapply(et, function(t) sum(vet$time >= t))              # size of risk set
dj <- sapply(et, function(t) sum(vet$time == t & vet$status == 1))
Sminus <- c(1, cumprod(1 - dj / nj))[seq_along(et)]           # LEFT-continuous
w_late <- 1 - Sminus                                          # the FH(0, 1) weight

# U = sum(w * (O - E)), with V weighted by w^2; the full implementation and
# its sanity check live in figures/scripts/B3-09-weighted-logrank.R

# --- milestone survival under both interval transforms --------------------
summary(survfit(Surv(time, status) ~ trt_f, data = vet, conf.type = "plain"),
        times = c(90, 180, 365))     # Greenwood, on the probability scale
summary(survfit(Surv(time, status) ~ trt_f, data = vet, conf.type = "log-log"),
        times = c(90, 180, 365))     # log-log transform

import pandas as pd
from lifelines import KaplanMeierFitter
from lifelines.statistics import (
    logrank_test, survival_difference_at_fixed_point_in_time_test)

# statsmodels' get_rdataset cannot find it -- Rdatasets files it under
# another bundle -- so read the CSV directly.
RD = "https://vincentarelbundock.github.io/Rdatasets/csv/"
vet = pd.read_csv(RD + "survival/veteran.csv")
a, b = vet[vet.trt == 1], vet[vet.trt == 2]

# weightings="fleming-harrington" needs both p and q (i.e. rho and gamma)
logrank_test(a.time, b.time, a.status, b.status)                    # rho = 0
logrank_test(a.time, b.time, a.status, b.status,
             weightings="fleming-harrington", p=1, q=0)             # rho = 1
logrank_test(a.time, b.time, a.status, b.status,
             weightings="fleming-harrington", p=0, q=1)             # FH(0, 1)

# milestone: fit one KM per arm, then compare them at one time point
ka = KaplanMeierFitter().fit(a.time, a.status, label="Standard")
kb = KaplanMeierFitter().fit(b.time, b.status, label="Test")
ka.predict(90), kb.predict(90)
survival_difference_at_fixed_point_in_time_test(90, ka, kb)

How this page relates to the others

• The log-rank test itself, and how to read a KM plot — see Kaplan-Meier curves and the log-rank test. That page covers the $w_j = 1$ version; this one only replaces $w_j$.
• How to tell that proportional hazards has failed, and what else can be done about it — see the proportional-hazards assumption: stratification, time splitting, and tt() for a coefficient that varies with time. That page also explains why truncating follow-up until the assumption holds is not a repair.
• When you want an effect measure with units that you can say to a patient — see restricted mean survival time (RMST). A weighted log-rank test still returns only a p value; it carries no effect measure, exactly like the standard one. Under non-proportional hazards, months gained or percentage points at a stated year beats any p value.
• Pricing the selection of a weight or a time point — see multiple comparisons and type I error.
• How the statistics section of a trial hangs together — see the randomised controlled trial.

Common misuse

Reproducing every number on this page

/opt/homebrew/bin/Rscript figures/scripts/B3-09-weighted-logrank.R

Report a problem on this page