AdvancedReporting guideline: STROBEIndependently reviewed, not yet spot-checked by a human

Cohort study

Where the line between prospective and retrospective actually falls, why an observed association is not a treatment effect, and how to write up a result that did not reach statistical significance without fooling yourself.

What a cohort study does

Take a group of people who have not yet had the event of interest, record whether each of them is exposed, then follow them forward and see who has the event.

That direction — from exposure towards outcome — is what defines a cohort study, and it is where it parts company with the case-control design, which starts at the outcome and looks back for exposure. The direction decides what can be computed: because both denominators are known — how many people were followed, and for how long — a cohort study can estimate incidence directly. A case-control study cannot.

Two denominators, two kinds of incidence

“Incidence” is not one quantity. Papers slide between two of them, and the difference is exactly which denominator was used.

Cumulative incidence — also called risk — is the number of people who had the event divided by the number of people followed. It is a proportion, it cannot exceed 1, and it is uninterpretable without a period attached: the table further down reports 16.1%, and that figure only means something alongside the median follow-up printed beside it. The curve on this page is on this scale; its y axis is cumulative incidence.

Incidence rate is the number of events divided by the total person-time at risk — for each person, the time they were actually under observation, summed over everyone. Its units are events per person-year (or per 1000 person-years), it has no ceiling of 1, and it needs no fixed window, because the length of follow-up is already inside the denominator.

The two come apart as soon as people are followed for unequal lengths of time, which is nearly always. If some men are lost after a year and others are watched for a decade, a headcount denominator credits both with the same opportunity to have the event; a person-time denominator does not. So:

With a fixed window and little dropout, cumulative incidence is what answers the clinical question — “what is the chance of this happening to me within five years”.
With staggered entry, heavy censoring or long follow-up, the rate is the honest summary. It is also what the models are built on: Poisson regression puts person-time in the denominator explicitly, while Kaplan-Meier and Cox deal with the same problem by asking who is still at risk at each moment — which is why the number-at-risk table under the curve is worth as much attention as the curve.
Either way, quoting an incidence without saying which denominator produced it is the point where a reader loses the ability to check the arithmetic.

Prospective and retrospective: the difference is in the data, not the calendar

This is the pair of words most often muddled. Both follow the same direction, exposure to outcome. What differs is whether the events had already happened when the investigator decided to run the study.

	Prospective	Retrospective
Have the outcomes occurred when the study begins	Not yet	Already
How exposure is measured	Designed and collected for this study	Taken from records that already exist (charts, claims databases, registries)
Direction of follow-up	Exposure → outcome	Exposure → outcome (the same)
Main weakness	Expensive, slow, loss to follow-up	Exposure measurement is limited to what someone else recorded, for another purpose

There is also a common misnomer: “retrospective analysis” describes when the analysis was done, not the study design. A prospectively assembled cohort that is later mined for a question nobody planned to ask is a retrospective analysis of a prospective cohort.

The example in this chapter

We will walk through one real observational dataset: men who underwent radical prostatectomy, where the exposure is the storage duration of the transfused red blood cells and the outcome is biochemical recurrence. The clinical question is whether older stored blood affects oncological outcome.

library(medicaldata)
library(survival)
data(blood_storage, package = "medicaldata")

d <- blood_storage
# The source data splits storage duration into three bands; here we contrast
# "long" against "short/medium"
d$exposure <- factor(
  ifelse(d$RBC.Age.Group == 3, "Long storage", "Short/medium storage"),
  levels = c("Short/medium storage", "Long storage")
)

# Complete-case analysis -- always report how many rows this drops
vars <- c("TimeToRecurrence", "Recurrence", "exposure",
          "Age", "PVol", "PreopPSA", "TVol")
complete <- d[complete.cases(d[, vars]), ]

# Unadjusted
coxph(Surv(TimeToRecurrence, Recurrence) ~ exposure, data = complete)

# Adjusted for the confounders that were measured
coxph(Surv(TimeToRecurrence, Recurrence) ~ exposure + Age + PVol +
        PreopPSA + factor(TVol), data = complete)

Verified with R 4.6.0, survival 3.8.6 and medicaldata 0.2.0

import statsmodels.api as sm
from lifelines import CoxPHFitter

bs = sm.datasets.get_rdataset("blood_storage", "medicaldata").data
bs["exposure"] = (bs["RBC.Age.Group"] == 3).astype(int)   # 1 = long storage

cols = ["TimeToRecurrence", "Recurrence", "exposure",
        "Age", "PVol", "PreopPSA", "TVol"]
complete = bs[cols].dropna()

crude = CoxPHFitter().fit(
    complete[["TimeToRecurrence", "Recurrence", "exposure"]],
    duration_col="TimeToRecurrence", event_col="Recurrence")
crude.print_summary()

adjusted = CoxPHFitter().fit(
    complete, duration_col="TimeToRecurrence", event_col="Recurrence")
adjusted.print_summary()

Cumulative incidence of biochemical recurrence by storage duration of the transfused red cells. Note that the y axis is cumulative incidence (1 minus survival), not survival: what a cohort study is counting is how many people had the event. The number-at-risk table underneath is the curve's denominator: by month 80 it is down to 20 and 9 men, and by month 100 to 2 and 2, so do not read anything into the separation there.Plotting script figures/scripts/D1-cohort-blood-storage.R

Step one: who entered the analysis, and who did not

	People
In the dataset	316
Excluded for missing values on a model variable	17
Analysed	299
Biochemical recurrence	48 (16.1%)
Long storage / short-medium storage	103 / 196
Median follow-up (reverse KM)	35.4 months

Dropping 17 people does not sound like much, but a complete-case analysis is not a neutral default; it is a choice that carries an assumption. Saying the data are missing at random does not capture that assumption. For a complete-case analysis to be valid for the estimand you want, the missingness mechanism has to be benign enough — in a regression setting the weakest version of that condition is that whether a value is missing is unrelated to the outcome, given the variables already in the model. Someone with no recorded PSA may have been followed less closely to begin with, and loose follow-up is related to prognosis, so the condition need not hold. What to do is check how missingness relates to the outcome, the exposure and the model covariates, and turn to multiple imputation or a sensitivity analysis where it matters (see missing data and multiple imputation). The point here is simpler: the number has to be printed, not quietly subtracted.

The median follow-up row hides a trap worth stopping on. Taking the median of the observed times gives 26.7 months, but the right answer is 35.4 months — a difference of 33%.

Here is why. Someone who recurs in month three has an observed time of three months, but that does not mean “this cohort was followed for three months” — their observation ended because they had the event, not because follow-up ran out. In a cohort with many events this calculation makes follow-up look far shorter than it was, even if the event-free patients were watched for years.

The correct procedure is the reverse Kaplan-Meier: treat censoring as the event and the event as censoring, then read off the median. What that returns is the median of the follow-up time distribution — read it as the typical length of follow-up, or as the time to which roughly half the cohort was followed at least. It is a median, not a mean follow-up time: follow-up times are usually skewed, so the two do not coincide and the words should not be swapped. The survival method pages go through this in more detail.

Step two: is the baseline balanced

There is no randomisation here, so the two groups will differ. What to read is the standardised mean difference (SMD):

Variable	SMD
Age	-0.158
PVol	-0.102
PreopPSA	0.033

By convention |SMD| < 0.1 counts as balanced. Two variables sit slightly outside it here — the comparison is with the absolute value, so a negative SMD counts too: age at -0.158 and prostate volume at -0.102, while preoperative PSA at 0.033 is inside the threshold. Neither exceedance is large, but they should stay in mind while reading the results — and note that this table covers only some of the variables in the adjusted model: no balance statistic is shown for tumour volume (TVol).

Step three: unadjusted and adjusted

Model	HR (long vs short/medium storage)	95% CI	p
Unadjusted	1.08	0.60–1.93	0.805
Adjusted for age, prostate volume, preoperative PSA, tumour volume	1.05	0.59–1.90	0.859

This result did not reach statistical significance. What follows is the most important part of the chapter, because it decides whether you end up fooling yourself.

Not reaching statistical significance means “this study did not detect a difference”. It does not mean “the two groups are the same”.

The confidence interval shows why: 0.60 to 1.93 accommodates both a 40% reduction in risk and something close to a doubling of it. With this sample (299 people and 48 events) the data are not sufficient to rule out a moderate effect.

So the defensible sentence is “no association between storage duration and biochemical recurrence was detected in this cohort”, not “storage duration is unrelated to biochemical recurrence”. Claiming the two are equivalent needs a design with an equivalence margin fixed in advance.

By the same token, this HR of 1.08 must not be written up on a slide or in an abstract as “an increase in risk of 8%” — an effect estimate whose interval crosses 1 should not be presented as though its direction were settled.

Step four: adjustment still does not buy causation

The HR barely moves between the two models (1.08 → 1.05), which says that the confounders that were measured are doing little work here. That sentence has a very heavy qualifier in it.

Statistical adjustment can only handle variables you have. What this dataset does not contain — the operating surgeon’s experience, how intensively each patient was followed afterwards, the comorbidity burden — stays in the estimate untouched, as long as it affects both which unit of blood a patient received and whether they recurred.

Traps specific to cohort studies

Immortal time bias

If the definition of exposure requires a patient to survive to some point before they can be classified as exposed, then by construction they could not have died during that stretch. Counting that “immortal time” as follow-up in the exposed group makes exposure look protective.

The classic setting: grouping patients by whether they received adjuvant chemotherapy after surgery. To receive chemotherapy, a patient first has to live long enough to start it. The fix is to treat exposure as a time-varying covariate rather than assigning groups at baseline.

Grouping on post-baseline information

Anything that classifies patients using information that only became available during follow-up will go wrong: grouping by treatment response, by adherence, by a laboratory value measured along the way. None of those is a baseline characteristic.

Loss to follow-up is not random

People who drop out generally differ from those who stay. Report the attrition rate, and compare the baseline characteristics of those lost with those who completed follow-up.

Common misuses

Misuse	Why it is wrong
Running between-group p-values on Table 1 of an observational study	The groups genuinely come from different populations; the test only reflects sample size. Read the SMD
A complete-case analysis that does not report how many rows were dropped	Missingness is usually related to prognosis, so a silent exclusion is a hidden selection step
Writing a non-significant result as “no difference between the groups”	Only “this study did not detect a difference” is supportable; a wide interval means a moderate effect cannot be excluded
Presenting an HR whose interval crosses 1 as a settled direction	The direction is undetermined; blowing it up into a headline misleads
Claiming causation because the model was adjusted	Adjustment reaches only the confounders that were measured
Grouping patients on information only available after baseline	Manufactures immortal time bias or selection bias
Calling a retrospective cohort a “retrospective analysis”, or the reverse	The first is a design, the second is when the analysis happened
Not reporting loss to follow-up	When attrition is related to prognosis it produces selection bias

Reproducing every number on this page

/opt/homebrew/bin/Rscript figures/scripts/D1-cohort-blood-storage.R

Read the figure

The answer comes from the same statistical output that produced this page's figures, not from a number typed in beside them.

This cohort has two candidate numbers for follow-up time. Which one belongs in a median follow-up line, and why?

Show the answer and why

Correct answer: The 35.4 months you get after swapping events and censoring, which answers how long a person would be followed if no event occurred

26.7 months is the median observed time, and observed time is cut short by the event itself: someone who recurs in month 3 contributes 3 months, which does not mean the cohort was followed for three months. The more events a cohort has, the shorter this arithmetic makes its follow-up look. Reverse Kaplan-Meier treats censoring as the event and the event as censoring, and reading its median gives 35.4 months, which is the median of the follow-up distribution and the number that can be read as a typical follow-up length. As for 100.0 months, that is only where the curve was drawn to, the position of the last few people still under observation; the number-at-risk row under the same figure shows both groups down to single digits by then.

The baseline balance table lists standardised mean differences for age, prostate volume and pre-operative PSA. Which statement is right?

Show the answer and why

Correct answer: The prostate volume row is -0.102, whose absolute value is already past the conventional balance threshold, so like age it counts as slightly imbalanced

The sign of a standardised mean difference only says which side the difference falls on; balance is judged on the absolute value. Age at -0.158 and prostate volume at -0.102 are both past the conventional threshold, and pre-operative PSA at 0.033 is the one row that genuinely sits inside it. Reading the minus sign as a smaller difference turns the least balanced row into the best balanced one. There is also something this table cannot say: it covers only some of the variables in the adjusted model, with no balance information for tumour volume, so two rows over the threshold is not the same as two variables being imbalanced.

After adjustment for age, prostate volume, pre-operative PSA and tumour volume, the hazard ratio barely moves. What does that mean?

Show the answer and why

Correct answer: The adjusted hazard ratio is 1.055, almost the same as the crude one, which only says the measured variables were not carrying much confounding

The estimate moves from 1.077 to 1.055, and only one thing follows from that: the variables put into the model were not carrying much confounding. Nothing follows about confounding in general. The surgeon's experience, how closely each patient was followed, the burden of comorbidity: none of it is in this dataset, and statistical adjustment can do nothing about variables it never saw. The p value of 0.859 is a second misreading. A large p value says this dataset detected no difference, not that no difference exists, and comparing two p values was never a way to tell whether confounding has been dealt with.

The crude hazard ratio comes with a 95% confidence interval that crosses the null. How should this row be written up?

Show the answer and why

Correct answer: The point estimate is 1.077 and the interval runs from clear benefit to clear harm, so no association was detected

0.600 and 1.932 are both real numbers; the problem is that each quotes one end only. The same interval holds both a large reduction and something close to a doubling, which is exactly what it means to say the data cannot rule out a moderate effect, and it is not evidence in either direction. The point estimate of 1.077 sits near the null, and with an interval this wide the only wording that holds is that no association between storage duration and biochemical recurrence was detected in this cohort. Claiming the two are equivalent needs a design with an equivalence margin set in advance, not an interval that crosses the null. For the same reason 1.077 should never be converted into a percentage increase for a summary graphic: enlarging an effect estimate whose direction is undetermined is what makes it misleading.

The two cumulative incidence curves look slightly separated at their right-hand end. Is that stretch worth discussing?

Show the answer and why

Correct answer: No. By month 100 the short or medium storage group has 2 people left in the risk set, so that stretch has almost no denominator under it

The number-at-risk row is the denominator of the curve, and it shrinks all the way along. 196 describes the starting point and nothing else; by month 80 the long storage group is down to single digits, and by month 100 both groups have only a handful of people left, so one event moves the curve by a large step. The separation at the right-hand end is what an exhausted denominator looks like, not two groups coming apart. With any survival or cumulative incidence curve, read the counts underneath first, then decide which stretch can be discussed.

The first table on this page splits the dataset total, the number excluded and the number analysed into three rows. Why does the middle row need to be there?

Show the answer and why

Correct answer: Because a complete-case analysis rests on an assumption about the missingness, and those 17 excluded people are the reader's only clue to it

Seventeen people excluded does not look like much, but a complete-case analysis is not a neutral default: for it to be valid for the target estimand the missingness has to be benign enough, and the loosest statement of that is that whether a value is missing is unrelated to the outcome given the variables already in the model. People with a missing PSA value may have been followed more loosely in the first place, and loose follow-up is related to prognosis, so the condition need not hold. 316 is the dataset total and 299 is the number the model actually ran on; neither can stand in for the middle row, because what the reader needs is the difference and how it arose. The number has to be reported, not quietly lost.

Methods used in this chapter

Watch next

ENTerry Shaneyfelt· 6 minFive and a half minutes of overview. Watch it first and the opening section here is easier to picture.

ENMichael Fralick· 6 minThe cleanest short account of where cohort and case-control designs part ways.

繁中臺大開放式課程 NTU OCW· 92 minA National Taiwan University lecture, spoken in Mandarin with English slides. If you would rather take epidemiology as a proper course, this is the full set; this episode is on the prototypes of each design.

Sources and licences

STROBE Statement: Strengthening the Reporting of Observational Studies in EpidemiologyCC BYThe section order follows the STROBE items. The prose is an original rewrite.