Categorising a continuous variable, or keeping it continuous

A decision that gets made every day

Age dichotomised at 65, BMI in four bands, CRP as high versus low, a tumour marker in quartiles — clinical papers are full of them. The reasons given are usually “it is easier to explain”, “that is how we divide it clinically”, or “a reviewer asked for it”.

The methodological literature is one-sided about this: do not categorise. But that position has a precondition, and the precondition usually goes unstated. This page uses two datasets to find it, because they happen to fall on opposite sides of it.

Two datasets, two directions

In A no departure from a straight line was detected; in B one was. That single difference reverses the direction of everything that follows.

library(MASS); library(survival); library(splines)
data(birthwt, package = "MASS")

bw <- birthwt
bw$race_f <- factor(bw$race, levels = 1:3, labels = c("White", "Black", "Other"))

# 1. Keep it continuous
glm(low ~ lwt + smoke + race_f, data = bw, family = binomial)

# 2. Quartiles, lowest group as the reference
bw$lwt_q <- cut(bw$lwt, quantile(bw$lwt, 0:4/4), include.lowest = TRUE)
glm(low ~ lwt_q + smoke + race_f, data = bw, family = binomial)

# 3. P for trend: 1/2/3/4 entered as a single numeric term
bw$lwt_q_num <- as.integer(bw$lwt_q)
glm(low ~ lwt_q_num + smoke + race_f, data = bw, family = binomial)

# 4. Spline: no linearity assumption, and no cutting either
sp <- ns(bw$lwt, knots = quantile(bw$lwt, c(.25, .5, .75)),
         Boundary.knots = quantile(bw$lwt, c(.05, .95)))
glm(low ~ sp + smoke + race_f, data = bw, family = binomial)

import numpy as np, pandas as pd
import statsmodels.api as sm
import statsmodels.formula.api as smf

bw = sm.datasets.get_rdataset("birthwt", "MASS").data

# 1. Keep it continuous
smf.logit("low ~ lwt + smoke + C(race)", data=bw).fit()

# 2. Quartiles (qcut splits on quantiles; labels=False gives 0..3)
bw["lwt_q"] = pd.qcut(bw["lwt"], 4, labels=False)
smf.logit("low ~ C(lwt_q) + smoke + C(race)", data=bw).fit()

# 3. P for trend: the same 0..3, this time as a number
smf.logit("low ~ lwt_q + smoke + C(race)", data=bw).fit()

Direction A: when linearity roughly holds, categorising is a pure loss

lwt is the mother’s weight at her last menstrual period, in pounds; the outcome is low birth weight, and the model also adjusts for smoking and race. The same association, three ways:

Same data, same association: continuous, it reaches statistical significance; in quartiles it does not; nor does the trend test. The magnitude of $|z|$ falls from 2.10 to 1.88 to 1.74, and both categorised versions have a higher AIC (227.27 and 226.88) than the continuous one (225.01).

The four groups in detail, and one thing worth noticing:

The point estimates are not monotone. The lowest is Q2 (OR 0.334), and Q3 comes back up to 0.486.

Do not put that wobble down to noise introduced by categorising. The spline fitted to the same data traces the same shape: down from 94 lb to a trough near 125 lb (0.456), back up to 0.658 at about 150 lb, and only then down again. Continuous and categorised are seeing the same ups and downs.

Only this much is supportable: no departure from a straight line was detected (χ² 2.49, df 3, p 0.478), and the confidence bands are wide enough that a real wobble and a straight line cannot be told apart. Choosing the linear term here is a decision to spend fewer parameters, not a demonstration that the relationship is a line.

And P for trend is precisely the test that assumes it is monotone and evenly spaced — the next section.

Direction B: when categorising looks stronger, the linear model was wrong

pgr is the progesterone receptor concentration in breast tumour tissue, and it is heavily right-skewed: median 41, mean 162, maximum 5004 fmol/l. The outcome is recurrence-free survival, adjusted for tumour size and menopausal status.

The direction reverses: $|z|$ rises from 2.25 to 2.95 to 3.14. Looking only at this dataset, it is easy to conclude that categorising is the more sensitive analysis.

It is telling you something else. The linear model never fitted this data in the first place: the spline-versus-linear test gives χ² 9.45, df 3, p 0.024.

Start with the actual shape, because it is not the intuitive one: the low range is flat. From 0 to about 63 fmol/l the spline sits near 1, and at 23 fmol/l it even rises slightly to 1.053. The decline happens mainly between roughly 23 and 491 fmol/l, after which it flattens again. So the problem is not that the linear model’s slope is too shallow at low values.

The problem is that one straight line is asked to serve the entire range from 0 to 5004 fmol/l:

• Half the patients are packed into 0 to 41 fmol/l (the median is 41, and the quartile cut-points are 0 / 4 / 41 / 198). Over that stretch the line barely separates anyone: even across the whole interquartile range (4 to 198 fmol/l) the linear model gives only HR 0.960.
• The slope is instead set by the handful of extreme values in the right tail, which sit furthest from everyone else and therefore have the most influence on a single linear coefficient.

One coefficient thus has to stand for both “between 0 and 41” and “between 198 and 5004”, and the spline shows those two stretches have completely different shapes. Quartiles at least carve the top quarter out and let it have an estimate of its own, so the analysis “looks” stronger.

AIC points the same way: the linear model, at 25393.55, is the worst of the three; quartiles (25388.96) and the spline (25390.10) are both lower. The gap between those two is only 1.15 — far too small to choose a model on. The case for the spline is not AIC. It is that the spline invents no cut-points, does not pretend the variation inside a group never happened, and produces a curve that can be read.

What P for trend actually computes

P for trend is almost standard equipment beneath a quartile table, but the calculation is cruder than most readers assume: the quartile numbers 1, 2, 3 and 4 are entered into the model as a single numeric variable, and the p value of that one coefficient is reported.

So it assumes two things:

1. Monotonicity — the effect moves in the same direction across groups.
2. Even spacing — the change in log-odds (or log-hazard) from Q1 to Q2 equals the change from Q3 to Q4.

The second is the load-bearing one, and almost nobody checks it. Quartiles are not evenly spaced on the original scale: the four group medians in direction B are 0, 16, 99, 413 fmol/l — the last of them is 4.2 times the one before it. Treating them as 1, 2, 3, 4 asserts that the effect changes at a constant rate per rank, which is an assumption about the group numbering rather than about the variable.

The “optimal cut-point”: an effect size that is guaranteed to be too big

More common than quartiles, and worse: try every possible cut-point and keep the one with the smallest p value.

Run it on lwt from direction A. Between the 10th and 90th percentiles there are 48 distinct candidate values, each fitted as an adjusted logistic model:

The second row is built like this: mean weight below and above the chosen cut-point is 97.19 and 137.76 lb; feeding that difference into the continuous model gives OR 1.712. The searched value of 3.053 is 1.78 times larger.

Some variables cannot be cut into quartiles at all

There is also a practical wall: quantile cut-points have to be distinct from one another. In the same rotterdam data, nodes (positive lymph nodes) cannot be cut:

The first two cut-points are both 0, because 1436 patients (48.2%) have no positive nodes. cut() stops and raises: 'breaks' are not unique (the English-locale wording; an R running in another language prints a translation).

This is not a quirk of R. Quartiles are meaningless for a zero-inflated variable — there is no way to divide into four equal parts a distribution with half its mass on a single value. The same zero inflation causes a different problem on the spline page, where the default knots collide and have to be specified by hand; here, categorising will not run at all.

Zero-inflated variables are common in clinical data: lymph node counts, length of stay, cigarettes per day, emergency department visits, hospitalisations in the past year.

So when is cutting defensible?

Common misuses

So, where next

If you got here because your data does not look like a straight line, the page you want is restricted cubic splines and the dose-response curve. It uses the same rotterdam data to show how to drop the linearity assumption without categorising, where the knots belong, and how to read the resulting dose-response curve.

Reproducing every number on this page

/opt/homebrew/bin/Rscript figures/scripts/B2-11-categorisation.R

Report a problem on this page