AdvancedIndependently reviewed, not yet spot-checked by a human

Ordinal logistic regression and the shift plot

When the outcome is graded rather than yes-or-no (mRS, NYHA, CTCAE, a 0-10 pain score), dichotomising throws half the information away. What a common odds ratio is, why proportional odds is an assumption and not a result, how to check it, and how the estimates break down when the cells get too thin.

What is left of a graded outcome after you dichotomise it

A whole class of clinical outcomes is neither continuous nor binary, but ordered categories:

mRS (modified Rankin Scale) 0-6, the standard outcome in stroke trials
NYHA functional class I-IV in heart failure
CTCAE adverse-event severity, grades 1-5
Pain scores 0-10, cough severity 0-2, endoscopic grades, pathological stage

What these share is order without spacing: going from mRS 2 to 3 and going from mRS 4 to 5 are both “one grade worse”, but they mean entirely different things to the patient. So they can neither be averaged as if continuous (the spacing is not meaningful) nor split into “good versus bad” (which discards a great deal).

The usual handling is to dichotomise: “mRS 0-2 counts as a good outcome”. The price is that a patient who improved from mRS 5 to mRS 3 is counted as the same event as a patient who did not improve at all. Stroke trials therefore moved to shift analysis, which asks not how many patients cleared a threshold but whether the whole distribution moved in the better direction. Its statistical tool is the ordinal logistic regression on this page, and the number it reports is the one readers meet most often and see explained least: the common odds ratio.

The example on this page

This page uses the same licorice-gargle trial as clustered and repeated-measures data (medicaldata::licorice_gargle), but a different outcome: cough severity at extubation, extubation_cough, graded 0 = no cough, 1 = mild, 2 = moderate to severe. 233 patients have a record (2 missing).

Arm	0 = no cough	1 = mild	2 = moderate to severe	Total
Control	71 (61%)	30 (26%)	15 (13%)	116
Licorice gargle	88 (75%)	25 (21%)	4 (3%)	117

Shift plot: two horizontal stacked bars, one per arm, showing the percentage of that arm at each of the three cough grades on a 0 to 100% axis. The upper bar is the control arm (n = 116): no cough 61%, mild 26%, moderate to severe 13%. The lower bar is the licorice arm (n = 117): no cough 75%, mild 21%, moderate to severe 3%. The lightest "no cough" segment is visibly longer in the licorice arm, while the darkest "moderate to severe" segment shrinks to a thin sliver there. Both bars run the full 0 to 100%, so what moves is not the bar but the boundaries between segments: both dividing lines sit further right in the licorice arm, giving the milder grades more of the length. — The full distribution across all three grades in each arm. The shift happens at both cut points: the proportion with no cough rises by 14.0 percentage points and the proportion with moderate-to-severe cough falls by 9.5 percentage points — in percentage points the first is the larger move, but in relative terms moderate-to-severe fell by 74%.Plotting script `figures/scripts/B2-13-ordinal.R`

That figure is what a shift analysis looks like. It draws where the distribution moved instead of picking one threshold and reporting a proportion. Two things are worth noticing: the proportion with no cough rose by 14.0 percentage points, while the moderate-to-severe proportion went from 13% to 3%. That second change is far larger in relative terms — and it disappears completely if you dichotomise into “any cough versus none”.

What the model looks like

The usual form of ordinal logistic regression is the proportional odds model. It does not fit one model per grade; it models every cut point at once. Three grades give two cut points, ”≥ 1 versus 0” and ”≥ 2 versus ≤ 1”. MASS::polr writes each cut from its lower side, which is the form the output below is in:

\log \frac{P(Y \le k)}{P(Y > k)} = \zeta_k - \beta x, \qquad k = 0, 1

The point is this: each cut point gets its own intercept $\zeta_k$ , but they share a single $\beta$ . The intercepts absorb how common each grade is; the slope absorbs how far the treatment pushed the whole distribution.

The two intercepts here are 0.417 (at the 0|1 cut) and 2.104 (at the 1|2 cut), and the treatment coefficient is -0.723 (SE 0.283, t = -2.55).

Exponentiating, and orienting it as control relative to licorice, gives a common OR of 2.060 (95% CI 1.18-3.59).

Running it yourself

library(medicaldata)
library(MASS)

lg <- licorice_gargle
d  <- subset(data.frame(y = lg$extubation_cough, treat = lg$treat), !is.na(y))

table(d$treat, d$y)                     # look at how thin the cells are FIRST

fit <- polr(factor(y, ordered = TRUE) ~ factor(treat), data = d, Hess = TRUE)
summary(fit)
exp(-coef(fit))                         # common OR: control vs licorice
exp(-rev(confint.default(fit)))         # 95% CI

# Proportional odds: one binary logistic per cut point
summary(glm(I(y >= 1) ~ factor(treat), data = d, family = binomial))
summary(glm(I(y >= 2) ~ factor(treat), data = d, family = binomial))

# With a single binary covariate, the 2x3 table admits an exact likelihood
# ratio test against the saturated model.
tb <- table(d$treat, d$y)
ll_sat <- sum(ifelse(tb > 0, tb * log(tb / rowSums(tb)), 0))
lrt <- 2 * (ll_sat - as.numeric(logLik(fit)))
c(statistic = lrt, p = pchisq(lrt, df = 1, lower.tail = FALSE))

# What thin cells do: switch to pacu30min_cough
ds <- subset(data.frame(y = lg$pacu30min_cough, treat = lg$treat), !is.na(y))
table(ds$treat, ds$y)
summary(polr(factor(y, ordered = TRUE) ~ factor(treat), data = ds, Hess = TRUE))
summary(glm(I(y >= 2) ~ factor(treat), data = ds, family = binomial))

Verified on R 4.6.0 with MASS 7.3.65 and medicaldata 0.2.0. polr() reports t values, not p values; get p values from the normal approximation yourself, or use confint() for profile intervals. Note that polr parameterises the model as zeta - beta*x, so the sign of the coefficient runs opposite to an ordinary glm.

import numpy as np
import statsmodels.api as sm
import statsmodels.formula.api as smf
from statsmodels.miscmodels.ordinal_model import OrderedModel

lg = sm.datasets.get_rdataset("licorice_gargle", "medicaldata").data
d = lg[["extubation_cough", "treat"]].dropna().rename(
    columns={"extubation_cough": "y"})

print(np.asarray(sm.stats.Table.from_data(d[["treat", "y"]]).table_orig))

mod = OrderedModel(d["y"].astype(int), d[["treat"]], distr="logit").fit(method="bfgs")
print(mod.summary())
print(np.exp(-mod.params["treat"]))     # common OR: control vs licorice

# One binary logistic per cut point
for k in (1, 2):
    d[f"ge{k}"] = (d["y"] >= k).astype(int)
    print(smf.logit(f"ge{k} ~ treat", data=d).fit(disp=0).summary())

statsmodels' OrderedModel(distr='logit') corresponds to polr; its intercept parameterisation differs from R's, so check the direction before comparing coefficients.

How to check the assumption

The most direct check, and the easiest to explain to a clinical reader, is to fit one binary logistic regression per cut point and put the log odds ratios side by side. If proportional odds holds up in the data, those numbers should be close.

Cut point	Control events	Licorice events	OR (control vs licorice)	95% CI	log OR
Cut at ≥ 1 (any cough vs none)	45 / 116	29 / 117	1.92	1.10-3.37	0.654
Cut at ≥ 2 (moderate to severe vs the rest)	15 / 116	4 / 117	4.20	1.35-13.05	1.434
Ordinal model: both cut points at once	—	—	2.06	1.18-3.59	0.723

Forest plot on a log-scaled horizontal axis with three estimates. The top row is the dichotomy at any cough (>= 1), OR 1.92 (1.10 to 3.37). The middle row is the dichotomy at moderate to severe (>= 2), OR 4.20 (1.35 to 13.05); its point estimate sits clearly further right and its confidence interval is much longer. The bottom row, drawn in a different colour with a diamond marker, is the ordinal model's common OR 2.06 (1.18 to 3.59), which falls between the two cut-point estimates but nearer the top one. None of the three confidence intervals crosses the vertical dashed line at OR = 1. — The ordinal model does not pick a cut point; it uses both. The common OR lands between the two cut-point estimates, closer to the one carrying more information.Plotting script `figures/scripts/B2-13-ordinal.R`

The two cut-point log ORs differ by 0.780, a factor of 2.18 on the OR scale. That looks like a substantial gap — but what that impression is worth depends on how much uncertainty sits behind it.

Because the only covariate here is binary, this 2x3 table admits an exact likelihood ratio test: the saturated model has four free parameters and the proportional odds model has three, so the comparison has 1 degree of freedom. The result is χ² = 2.411, p = 0.121.

The correct reading of p = 0.121 is “no departure from proportional odds was detected in these data”, not “the proportional odds assumption holds”.

That is not pedantry. A stratified bootstrap (resampling within each arm, 4000 replicates of which 3932 were usable) puts an interval on the difference between the two cut-point log ORs: 0.780 (95% CI -0.178 to 2.168).

The 68 discarded replicates are the ones in which some cell came out empty after resampling, leaving the log OR non-finite. This is worth stating, because it is not random loss: an empty cell is exactly what happens when the log OR runs to an extreme, so the replicates thrown away are the ones with the largest differences. The upper limit is therefore conservative if it is biased at all — the true uncertainty can only be wider than 2.168, not narrower.

On the OR scale that interval runs from 0.84 to 8.74. In other words these data are compatible with two very different worlds:

the upper cut point’s OR is in fact slightly smaller than the lower one’s (a ratio of 0.84), so the assumption is broadly fine;
the upper cut point’s OR is 8.74 times the lower one’s, so the assumption fails badly.

That is what “low power” actually looks like when you quantify it. With 233 patients and only 19 moderate-to-severe events at the upper cut point, this test was never likely to catch a moderate violation. So the right way to report it is to list both cut-point estimates and state the uncertainty plainly, letting the reader judge — not to wave a non-significant p value and declare the assumption cleared.

When the cells get too thin

Same dataset, different time point: cough at PACU 30 minutes, pacu30min_cough. Its distribution is

Arm	0 = no cough	1 = mild	2 = moderate to severe
Control	88	24	4
Licorice gargle	99	18	0

Only 4 patients in the whole trial reach moderate-to-severe, and all 4 of them are in the control arm, against 0 under licorice. That cell is empty.

The ordinal model survives; the dichotomy blows up. This is the contrast most worth taking away from this page.

The ordinal model still returns a finite, interpretable estimate: common OR 1.806 (95% CI 0.94-3.48), with a coefficient standard error of 0.335, the same order as the main analysis. (That interval crosses 1, so it did not reach statistical significance at this time point; what this section is about is whether an estimate exists at all, not the size of any effect, so do not read 1.81 as an established one.) It survives because it shares a single slope: the empty cell at the upper cut point supplies too little information, and the lower cut point carries the estimate.

On the same data, the logistic regression dichotomised at “moderate to severe (≥ 2)” is completely separated: the log OR runs to 18.23 with a standard error of 2702.5, an OR of about 8.3e+7, and an upper confidence limit that is infinite. That estimate does not exist — not “is very large”, but does not exist. The symptoms of complete separation and what to do about them are on logistic regression.

But do not read the previous paragraph as “ordinal models rescue sparse data”.

It survives because the upper cut point borrows its information from the lower one — and the licence to borrow rests entirely on the proportional odds assumption. An empty cell means the upper cut point carries almost no information, so the assumption cannot be checked there: the likelihood ratio test performed above on the main outcome degenerates here into a ritual with almost no power.

The right way to report a sparse table is therefore to give the ordinal estimate together with the fact that the top grade holds only 4 patients, of whom 0 are in the licorice arm, and to state that proportional odds cannot be verified at that cut point. Collapsing adjacent grades (merging 1 and 2 into “any cough”) is also an honest option; the price is that you are back to a dichotomy.

More grades does not mean more information

The sore-throat score in the same dataset, pacu30min_throatPain, has 7 grades (0 to 6), far more than cough’s three. But its frequency distribution is:

Grade	0	1	2	3	4	5	6
Patients	169	20	18	14	9	1	2

73% of patients score 0, the tail has 2 grades holding fewer than five patients, and the smallest holds 1. The number of grades is a property of the instrument; the amount of usable information is a property of the data, and the two are unrelated. Fit an ordinal model to a distribution like this and the cut points in the tail are held up by the assumption alone, which is why no model is fitted to it here — in practice you would collapse the tail grades first, or move to a method that does not need proportional odds.

What to do when proportional odds fails

If the check shows the cut-point ORs differ materially, the options run from simple to elaborate:

Approach	What it does	Price
Report each cut point’s OR as it is	Give up on one summary number and list the two or three cut-point estimates	No single effect size; the multiplicity problem surfaces
Partial proportional odds	Let only the offending variable have its own coefficient per cut point; the rest stay shared	Needs `VGAM` or `ordinal`; interpretation gets harder
Multinomial logistic regression	Abandon the ordering entirely; one set of coefficients per grade against a reference	More parameters, less power, and it discards the genuine information that the grades are ordered
Continuation ratio model	Asks instead “among patients who reached grade k, who goes on to k+1?”	Answers a different clinical question; make sure that is the question you wanted
Non-parametric shift tests (Wilcoxon / van Elteren)	Test whether the distribution shifted, without estimating an effect size	No covariate-adjusted effect size to report

Common misuses

Misuse	Why it is wrong
Averaging graded outcomes and running a t-test	The spacing between grades is not meaningful, so the mean cannot be interpreted
Reporting a common OR without checking proportional odds	The “common” part is an assumption, not a result
Reading a non-significant test as “the assumption holds”	Only “no departure was detected”; the test has little power in small samples
Looking at the p value but not at the gap between cut-point estimates and its interval	A non-significant p value may only mean too little information; the interval shows what can be ruled out
Trying several cut points and reporting the most significant dichotomy	Post-selection p values are invalid; it is hidden multiplicity
Reporting a common OR without stating the direction	Say which arm relative to which, and which way is worse
Reporting a huge OR from a sparse table anyway	Under complete separation the estimate does not exist; see logistic regression
Assuming ordinal models are immune to thin cells	They borrow information via proportional odds, and that assumption cannot be verified at an empty cell
Assuming more grades means more information	When tail grades hold one or two patients, those cut points are held up by the assumption
Drawing a shift plot of the dichotomised proportions	That defeats the entire purpose; plot the full distribution
Comparing a common OR directly against a dichotomised OR by size	They define different contrasts and the numbers are not interchangeable
Fitting polr to repeated graded measurements	Repeated measurements on one patient are not independent; see clustered and repeated-measures data

Clustered and repeated-measures data — the same licorice_gargle data, with the sore-throat score at four time points
Logistic regression — the binary-outcome foundation, and what complete separation looks like
Multiplicity — why the cut point has to be pre-specified
Chi-square and Fisher’s exact test — contingency-table tests that make no use of the ordering

Reproducing every number on this page

/opt/homebrew/bin/Rscript figures/scripts/B2-13-ordinal.R

Read the figure

The answer comes from the same statistical output that produced this page's figures, not from a number typed in beside them.

The proportional odds assumption is checked by fitting a logistic model at each cut-point. What is the number from the cough-of-one-or-more cut-point, and what should it be compared against?

Show the answer and why

Correct answer: 1.92 - that cut-point's odds ratio, compared against the odds ratios at the other cut-points

This cut-point's odds ratio is 1.92, and checking proportional odds means laying the cut-point-specific odds ratios side by side to see how far apart they are - the assumption says precisely that they are all the same. 1.10 and 3.37 are the ends of that one odds ratio's confidence interval, not separate estimates. Comparing against the model's common odds ratio is no better: that figure is computed on the assumption being tested, so using it as the benchmark assumes what is in question. When the cut-points disagree, the common odds ratio corresponds to none of them, and the model still prints a single number.

On the sparse outcome, the licorice arm has zero events at the cough-of-two-or-more cut-point. How does the model respond?

Show the answer and why

Correct answer: 0 events is complete separation: the estimate is pushed towards infinity, and the model prints a number regardless

One empty cell is complete separation: the log odds ratio at that cut-point is driven into double figures, the upper confidence bound runs to infinity, and nothing halts - a perfectly ordinary-looking number is printed. 4 is the control arm's event count at the same cut-point, and the other arm having data does not rescue anything, because separation needs only one empty cell. 18 is the licorice arm's count at the other cut-point, and a healthy cut-point elsewhere does not make the model switch to it. An implausibly large odds ratio beside an implausibly large interval is a cue to go and count the cells.

Throat pain has seven levels, two of which contain fewer than five patients. What does that do to a proportional odds model?

Show the answer and why

Correct answer: 2 levels are too thin, so those cut-points have almost no data behind them

Two levels hold fewer than five patients, so the model has almost nothing to work with at those cut-points - and it still returns a common odds ratio that looks reassuringly precise. More levels are not more information; 7 is the level count, and reading it as information has the relationship backwards. 1 is the smallest cell, and merging levels is a legitimate response, but merging changes the question the model is answering, so it does not simply "remove the problem". Before deciding how to handle sparse levels, find out how many cut-points are resting on a handful of patients.

Chapters that use this method

Randomised controlled trial

Sources and licences

This page is original writing

What is left of a graded outcome after you dichotomise it

The example on this page

What the model looks like

Running it yourself

How to check the assumption

When the cells get too thin

More grades does not mean more information

What to do when proportional odds fails

Common misuses

Related pages

Reproducing every number on this page

Read the figure

Chapters that use this method

Sources and licences