BasicIndependently reviewed, not yet spot-checked by a human

The 2x2 table, sensitivity and specificity

What each of the four cells and four margins actually answers, why sensitivity and specificity are properties of the test and predictive values are not, when the SpPIN / SnNOUT mnemonics mislead, and the commonest bias in diagnostic research — when only test-positive patients go on to the reference standard, sensitivity and specificity are pushed in opposite directions at the same time.

What this page answers

Almost every clinical test ends up compressed into a binary judgement: does this result count as positive or negative? A laboratory value crosses a threshold, an image is read as showing the finding or not, a rapid test grows a second line.

The trouble is that “is this test any good?” has no single answer. It splits into at least four different questions, and on one and the same dataset the four answers can be wildly different:

Of the people who have the disease, what fraction does the test catch?
Of the people who do not, what fraction does the test let through?
Of the people the test calls positive, what fraction really have the disease?
Of the people the test calls negative, what fraction really do not?

The answers to the first two are properties of the test. The answers to the last two are not. This page handles the first two — sensitivity and specificity. The last two are covered in predictive values and prevalence. Keeping the two pairs apart is the foundation the whole diagnostic accuracy family rests on.

Four cells and four margins

Cross “what the test said” with “what the reference standard (the gold standard) says is true”, and you have the 2×2 table:

	Disease present	Disease absent	Total
Test positive	True positive TP	False positive FP	All positives
Test negative	False negative FN	True negative TN	All negatives
Total	All diseased	All non-diseased	Everyone tested

The four cells sum to the number of people tested. The thing actually worth memorising is which direction you divide in:

\text{Sensitivity} = \frac{TP}{TP + FN}, \qquad \text{Specificity} = \frac{TN}{TN + FP}

\text{PPV} = \frac{TP}{TP + FP}, \qquad \text{NPV} = \frac{TN}{TN + FN}

Sensitivity and specificity divide down the columns — the denominator is a count of people in a disease state. Predictive values divide across the rows — the denominator is a count of people with a test result.

Why that difference in direction matters so much.

When the denominator is “people who have the disease”, you fixed in advance whose numbers you were looking at: that count is decided by the investigator (how many cases were enrolled), not by the population. So as long as the test itself does not change and the case mix of those tested does not change, enrolling a few more healthy people cannot move the sensitivity.

When the denominator is “people the test called positive”, the opposite holds: enrolling more healthy people adds false positives, and the PPV drops immediately. That is why the same test has a completely different PPV in the emergency department than in a community screening programme, while its sensitivity can be identical in both.

Worth noting in passing: in the machine learning literature recall is sensitivity and precision is PPV, under different names. When you meet an F1 score, remember it is mixing recall with precision — that is, one quantity computed down a column and one computed across a row.

The example used on this page

pROC::aSAH holds data on 113 patients with subarachnoid haemorrhage (SAH), recording two biomarkers and neurological outcome at six months. Dichotomise the outcome into Good and Poor: 41 patients did badly and 72 did well, so the prevalence in this cohort is 36.3%.

The test on this page is S100β, an astrocytic protein that rises after brain injury, with the cut-off set at 0.20 µg/L.

Dot plot. The horizontal axis is S100beta concentration on a log scale; the upper row of dots is patients with a good outcome and the lower row is patients with a poor outcome. A dashed vertical line marks the cut-off, and the four resulting quadrants are labelled with the counts of TN, FP, FN and TP. — Each dot is one patient. The dashed vertical line is the cut-off; it splits each row in two, and those four pieces are the four cells of the 2×2 table. Slide the line and all four numbers move together — which is exactly what an ROC curve traces out.Plotting script `figures/scripts/B4-01-sens-spec.R`

	Poor outcome (diseased)	Good outcome (not diseased)	Total
S100β ≥ 0.20	TP = 26	FP = 14	40
S100β < 0.20	FN = 15	TN = 58	73
Total	41	72	113

The four measures with their 95% confidence intervals (Wilson score intervals, which — unlike the textbook normal approximation — never run outside [0, 1] when the proportion sits near 0 or 1):

Measure	Computed as	Estimate	95% CI
Sensitivity	26 / 41	63.4%	48.1–76.4%
Specificity	58 / 72	80.6%	70.0–88.0%
Positive predictive value (PPV)	26 / 40	65.0%	49.5–77.9%
Negative predictive value (NPV)	58 / 73	79.5%	68.8–87.1%

The confidence intervals deserve a second look. The interval on sensitivity runs from 48.1% to 76.4%, a span of 28.3 percentage points. Diagnostic accuracy studies are chronically underpowered, because the effective sample size is not the total: it is the number of diseased people and the number of non-diseased people, counted separately. The sensitivity here is decided by 41 patients and nobody else. When a diagnostic paper reports a sensitivity without an interval, assume the interval is wide until shown otherwise.

Run it yourself

library(pROC)
data(aSAH, package = "pROC")

cut <- 0.20                       # fixed in advance, not chosen from the data
pos <- aSAH$s100b >= cut
dis <- aSAH$outcome == "Poor"

tab <- table(factor(pos, c(TRUE, FALSE)), factor(dis, c(TRUE, FALSE)))
tab                                # the top-left cell is TP

tp <- tab[1, 1]; fp <- tab[1, 2]; fn <- tab[2, 1]; tn <- tab[2, 2]
sens <- tp / (tp + fn)
spec <- tn / (tn + fp)
c(sensitivity = sens, specificity = spec,
  ppv = tp / (tp + fp), npv = tn / (tn + fn))

# CI for a proportion: Wilson. Do not use sens +/- 1.96*SE, which leaves [0, 1]
prop.test(tp, tp + fn, correct = FALSE)$conf.int
binom.test(tp, tp + fn)$conf.int   # Clopper-Pearson: conservative, always covers

# Export for the Python column
write.csv(aSAH, "aSAH.csv", row.names = FALSE)

Verified with R 4.6.0 and pROC 1.19.0.1. aSAH ships with pROC, so there is nothing extra to download.

import pandas as pd
from sklearn.metrics import confusion_matrix
from statsmodels.stats.proportion import proportion_confint

d = pd.read_csv("aSAH.csv")          # written by the R line above
pos = d["s100b"] >= 0.20
dis = d["outcome"] == "Poor"

# sklearn returns [[TN, FP], [FN, TP]] -- the reverse of the clinical layout
tn, fp, fn, tp = confusion_matrix(dis, pos).ravel()

sens = tp / (tp + fn)
spec = tn / (tn + fp)
print(sens, spec, tp / (tp + fp), tn / (tn + fn))

print(proportion_confint(tp, tp + fn, method="wilson"))
print(proportion_confint(tn, tn + fp, method="wilson"))

aSAH is not served by Rdatasets, so the Python column starts from a CSV exported by the R code. Note that sklearn's confusion_matrix returns [[TN, FP], [FN, TP]], which is upside down relative to the clinical convention.

“Sensitivity and specificity do not vary with prevalence” — what that sentence means

Textbooks say sensitivity and specificity are properties of the test. The cleanest way to see what that claim amounts to is to change the case mix and recompute.

The three cohorts below are all built from the same 113 patients, by duplicating one group wholesale. Duplication rather than resampling, deliberately: it keeps the arithmetic exact and free of sampling noise, so nothing in the table can be blamed on chance.

Cases : controls, as duplicated	Total n	Prevalence	Sensitivity	Specificity	PPV	NPV
1:4	329	12.5%	63.4%	80.6%	31.7%	93.9%
1:1	113	36.3%	63.4%	80.6%	65.0%	79.5%
4:1	236	69.5%	63.4%	80.6%	88.1%	49.2%

Prevalence moves from 12.5% to 69.5%, sensitivity and specificity do not budge in a single cell, and PPV jumps from 31.7% to 88.1%.

But this is a property of the arithmetic, not a guarantee about the world.

The demonstration above comes out perfectly because we only duplicated people; we never swapped them for different people. When you move a test to a new setting in real life, what changes is usually the case mix: patients at a tertiary centre are sicker than patients in the community, sicker patients have higher marker values, and higher values are easier to catch. So the very same test measures a higher sensitivity at the tertiary centre than in the community. This is the spectrum effect. It does not violate the formula above — it is a different group of people.

Predictive values and prevalence measures this for you on this same dataset, splitting it by WFNS grade: the sensitivity in the two severity strata differs by a wide margin. So “the sensitivity in the literature transfers directly to my patients” is an assumption to be checked, not a fact you may assume.

SpPIN and SnNOUT, and when they mislead

The mnemonics that circulate on the wards are:

SpPIN: with a highly Specific test, a Positive result helps rule IN the diagnosis
SnNOUT: with a highly sensitive (Sn) test, a Negative result helps rule OUT the diagnosis

The reasoning is sound: high specificity means people without the disease are almost never called positive, so a positive result is probably not a mistake. Take the same marker at three different cut-offs and you can see both ends of the mnemonic at once:

Cut-off (µg/L)	Sensitivity (95% CI)	Specificity (95% CI)	TP / FP / FN / TN	LR+	LR−
0.05	97.6% (87.4–99.6)	6.9% (3.0–15.2)	40 / 67 / 1 / 5	1.05	0.35
0.20	63.4% (48.1–76.4)	80.6% (70.0–88.0)	26 / 14 / 15 / 58	3.26	0.45
0.60	22.0% (12.0–36.7)	100.0% (94.9–100.0)	9 / 0 / 32 / 72	not estimable	0.78

The lowest cut-off reaches a sensitivity of 97.6% and looks like a textbook SnNOUT; the highest reaches a specificity of 100.0% with not one false positive and looks like a perfect SpPIN. Both appearances are misleading:

The trouble with SnNOUT: a high sensitivity does not make a negative result useful.

At a cut-off of 0.05 the sensitivity is 97.6%, but the specificity is only 6.9% — 67 of the 72 patients with a good outcome are also called positive. This test hardly ever misses a case because it hardly ever clears anybody. The quantity that actually measures how much a negative result is worth is LR−, and here it is 0.351 — enough to cut the odds of disease to roughly a third, which is not what “ruling out” means.

The trouble with SpPIN: zero false positives does not mean the specificity is 1.

At a cut-off of 0.60 the point estimate of specificity is 100.0%, but the lower bound of the 95% confidence interval is 94.9% — zero false positives among 72 non-diseased people corresponds to a false positive rate that could still be as high as about 5.1%. A confidence interval describes the range compatible with this dataset; it is not a guarantee — the true value may still lie outside it. And at this cut-off LR+ is not estimable at all (the denominator is zero), while sensitivity has fallen to 22.0%: most patients with the disease are called negative here.

The conclusion is not to throw the mnemonics away, but to demote them to a sense of direction. To actually compute how much a result should change your mind about the diagnosis, use likelihood ratios, which take in sensitivity and specificity together and multiply straight onto the pre-test odds.

Verification bias: the commonest hole in diagnostic research

Everything above assumes every patient underwent the reference standard. Real diagnostic studies frequently do not work that way. The reference standard is often invasive, expensive or risky — biopsy, angiography, surgery, long-term follow-up — so in practice only the test-positive patients get sent for it, and the negatives go home.

This is partial verification bias, also called work-up bias. Its consequence is not that the numbers wobble a little; it is a systematic bias with a fixed direction.

Treat this page’s data as the fully verified truth, then suppose only a fraction of the test-negative patients are sent for the reference standard and the rest drop out of the analysis:

Fraction of test-negatives verified	n entering the analysis	Apparent sensitivity	Apparent specificity
100%	113	63.4%	80.6%
70%	91	71.2%	74.4%
50%	77	77.6%	67.4%
30%	62	85.2%	55.4%
10%	47	94.5%	29.3%

Line chart. The horizontal axis is the fraction of test-negative patients who receive the reference standard, decreasing from left to right, with complete verification at the left end; the vertical axis is the apparent sensitivity and specificity. As the verified fraction falls, apparent sensitivity rises and apparent specificity falls; two dashed horizontal lines mark the true values. — Partial verification bias always points the same way: the fewer test-negative patients are verified, the more sensitivity is overestimated and the more specificity is underestimated. The left-hand end of the axis is complete verification, which is the truth.Plotting script `figures/scripts/B4-01-sens-spec.R`

You can read the direction straight off the formulas instead of memorising it. False negatives (FN) occur only among test-negative patients, so verifying fewer negatives means finding fewer FN, which shrinks the denominator of sensitivity — sensitivity is overestimated. True negatives (TN) likewise occur only among test-negatives, so verifying fewer of them finds fewer TN, while every false positive (FP) is still counted — specificity is underestimated.

When only 10% of the negatives are verified, sensitivity is pushed from a true 63.4% up to 94.5%, and specificity falls from 80.6% to 29.3%. If a diagnostic study reports a sensitivity above ninety per cent without saying how the test-negative patients were verified, that number cannot be used as it stands.

Three concrete things to check when you read such a paper:

What was the reference standard, and did everybody get it? The STARD 2015 flow diagram exists precisely to make authors draw this — how many received the index test, how many received the reference standard, and where the difference went.
Did different patients get different reference standards? Biopsy for the positives and three months of follow-up for the negatives is differential verification bias, and the two standards are not defining the same disease.
Did the person interpreting the reference standard know the index test result? If so the two are not independent, and the agreement between them has been inflated by hand.

Statistical repair is possible — inverse probability weighting, when it is reasonable to assume that whether a patient was verified depends only on variables you observed — but it requires that the investigators recorded who went unverified. If the design did not preserve that information, it cannot be recovered afterwards.

Common misuses

Misuse	Why it is wrong
Treating sensitivity and PPV as the same thing	One divides down the disease-status column, the other across the test-result row; the denominators are entirely different
Reporting a sensitivity with no confidence interval	The effective sample size is the number of diseased patients, usually far smaller than the total, so the interval is often wide
Claiming a test “never misclassifies” when specificity is near 1 with zero false positives	Zero false positives only bounds the interval from above; its lower bound can still sit well below 1, and LR+ is not estimable there
Concluding from a high sensitivity alone that a negative rules the diagnosis out	What matters is LR−; with high sensitivity and very low specificity, LR− can still sit close to 1
Applying a published sensitivity directly to your own patients	A different case mix changes the sensitivity you measure (the spectrum effect); this is an assumption to check
Verifying only test-positive patients and reporting sensitivity and specificity as usual	Partial verification bias overestimates sensitivity and underestimates specificity at the same time
Pooling positives and negatives that received different reference standards	Differential verification bias; the two standards are not defining the same disease
Using overall accuracy as the headline measure	When prevalence is low, calling everyone negative already yields a high accuracy
Picking the best-looking cut-off from your own data and then reporting its sensitivity	Optimistic bias; see choosing a cut-off and comparing two ROC curves
Dichotomising a continuous test result and then never stating the cut-off	Without a cut-off, sensitivity and specificity are undefined

Reproducing every number on this page

/opt/homebrew/bin/Rscript figures/scripts/B4-01-sens-spec.R

Read the figure

The answer comes from the same statistical output that produced this page's figures, not from a number typed in beside them.

Of 113 patients, 41 had a poor outcome. At a cut-off of 0.20 the sensitivity is 0.634 and the specificity is 0.806. A reader says 'so about six in ten of the people who take this test are classified correctly'. Which statement is right?

Show the answer and why

Correct answer: No. The share of everyone classified correctly is 0.743, and the denominator of sensitivity is only the 41 patients with a poor outcome

The denominator of sensitivity is those 41 poor-outcome patients, not the 113 people tested: it divides down a column. The share of everyone classified correctly is 0.743, higher than 0.634 because the specificity side holds more people and is sorted more accurately. 0.650 is the positive predictive value, which divides along a row and answers what a positive report means rather than how many people were sorted correctly. All three numbers sit in the same table and differ only in which denominator they take, and reading sensitivity as overall accuracy treats a column-wise quantity as a summary of the whole table.

The controls are duplicated as a block so that the cohort prevalence falls from 36.3% to 12.5%, and all four measures are recomputed. What is this table demonstrating?

Show the answer and why

Correct answer: In the lowest-prevalence row the PPV falls to 0.317, while sensitivity and specificity do not move a hair

Duplicating controls changes only the relative size of the two groups, and nobody's test result changed. Sensitivity is computed inside the diseased group and specificity inside the non-diseased group, so neither moves anywhere in this table. PPV and NPV take test-positive and test-negative denominators, which mix the two groups, so they follow the mix - 0.317 and 0.881 are the same test in two cohorts. The NPV of 0.939 is high because most people in that cohort do not have the disease to begin with, not because the test got stronger: saying 'you are fine' to everyone without testing would be about as accurate.

In the row for a cut-off of 0.60 there is not one false positive among the 72 patients with a good outcome, and the point estimate of specificity is 1.000. Can we say this cut-off never misclassifies a healthy person?

Show the answer and why

Correct answer: No. The lower bound of the 95% confidence interval is 0.949, so a false-positive rate of about five percentage points is still compatible with these data

Zero false positives only pins the point estimate to the ceiling; it does not pin the truth there. Zero events among 72 people gives a lower bound of 0.949, so a false-positive rate of up to about five percentage points remains compatible with these data. The cost is not small either: sensitivity is down to 0.220, so most poor-outcome patients are called negative, and LR+ cannot be estimated at all because its denominator is zero. Reading 'this sample saw no misclassification' as 'this test does not misclassify' turns an interval into a guarantee.

The cut-off of 0.05 has a sensitivity of 0.976, which looks like a textbook SnNOUT. Does a negative result at this cut-off really rule the disease out?

Show the answer and why

Correct answer: No. The specificity in the same row is only 0.069: this cut-off calls almost everyone positive, so negatives are rare because it lets almost nobody through

SnNOUT does not follow from a high sensitivity alone; it needs the negative result itself to carry information. Here the specificity is 0.069, so more than nine in ten good-outcome patients are called positive as well, and the sensitivity of 0.976 was bought by letting almost nobody through. The quantity that measures what a negative is worth is LR−, which is 0.351 here: it drops the odds of disease to roughly a third, a useful step but a long way from ruling out. To treat a negative as exclusion you normally want an LR− an order of magnitude smaller.

A diagnostic study sends only some of its test-negative patients for the reference standard. In this page's simulation, when only one in ten negatives is verified the apparent sensitivity is 0.945 against a true value of 0.634. Which way does specificity go?

Show the answer and why

Correct answer: Specificity falls to 0.293 - verifying fewer negatives finds fewer true negatives, while every false positive is still counted

False negatives occur only among test-negative patients, so verifying fewer of them removes false negatives from the denominator and sensitivity is inflated to 0.945. True negatives occur only there too and vanish along with them, while every false positive sits on the positive side and is counted in full, so specificity is deflated to 0.293. The two move in opposite directions, and the direction is fixed rather than random. 0.806 is the true specificity under complete verification, and 0.776 is an apparent sensitivity at another verification fraction, not a specificity. A diagnostic study that never says how its negatives were verified is reporting a sensitivity that cannot be taken at face value.

Sensitivity is 0.634 with a 95% confidence interval of 0.481 to 0.764, noticeably wider than the specificity interval in the same table. Why?

Show the answer and why

Correct answer: Because sensitivity is determined by the 41 poor-outcome patients alone, while specificity has the other 72 good-outcome patients as its denominator

The effective sample size of a diagnostic study has to be counted on each side separately: sensitivity has 41 patients in its denominator and specificity has 72, and each interval takes its width from its own number rather than from the 113 enrolled. 40 is the number of test positives, which is the denominator of PPV and not of sensitivity. So when a diagnostic paper reports only the total enrolled and not the two group sizes, assume the sensitivity interval is wide.

Chapters that use this method

Diagnostic accuracy study

Watch next

ENGlobal Health with Greg Martin· 3 minThree minutes to build the mental picture of the 2×2 table. Watch it before section two and the definitions will feel inevitable rather than arbitrary.

ENClinical Information Sciences· 2 minUnder two minutes, turning the concept into an arithmetic you can perform. It fills the gap the previous video leaves — that one explains but never computes.

繁中EDMAN MURMURS· 9 minIn Mandarin, from a practising clinician's angle. Useful if you also need the Chinese terms these concepts go by in Taiwanese teaching hospitals.

ENStatQuest with Josh Starmer· 12 minThe same definitions told once more from a machine learning angle, so that recall and precision in that literature will not throw you later.

繁中臺大開放式課程 NTU OCW· 60 minA full lecture in Mandarin from National Taiwan University's open courseware. The verification bias in this page's last section gets its wider epidemiological context here.

Sources and licences

This page is original writing