Events per variable, and the Riley sample size criteria

Correct answer: At the Riley sample size every parameter gets 31 events, far above the old threshold - the two rules aim at different things

31 events per parameter is far above the old threshold, so the Riley calculation is not the more relaxed of the two here; 174 is simply the least demanding of the three criteria, and the figure adopted is always the largest, because all three have to hold at once. The binary 795 is not about coding either: throwing away the timing leaves the same people carrying less information, so more of them are needed, and that still comes out of the same set of criteria. What really separates the two rules is what each guards against. The old rule comes from last century's simulations about coefficient bias and interval coverage, which is the acceptance standard of aetiological research; a prediction model is accepted on whether its predictions are accurate, and that depends on expected discrimination, the event rate, the length of the time axis and the number of parameters - none of which an events-per-variable count can see.

Correct answer: 0.688 is well below 1, meaning the risk differences the model produces should be discounted by roughly three tenths; too small a sample in a prediction model produces overfitting, not a lack of power

0.688 says that when the model claims one person's risk exceeds another's by some amount, the real gap is about seven tenths of that. This is how an inadequate sample shows itself in a prediction model - not as the conservative failure of missing a real effect, but as a model that fits perfectly well and looks unusually good on its own data. Optimism of 0.038 looks small only because the C-index scale is compressed; the calibration slope in the same row magnifies the same problem into something visible, which is precisely why acceptance cannot rest on discrimination alone. 0.874 has not reached 1 either, but it is much closer, and the first criterion targets an expected uniform shrinkage factor while this measures a calibration slope actually observed in a hold-out sample, which additionally absorbs that sample's own sampling error, so the two were never going to line up exactly.

Correct answer: Against the spread of five-year risk in the cohort itself - a median of 0.396, the central eight tenths between 0.273 and 0.677, so 0.084 is about a fifth of that band

Whether an absolute error is large depends on the spread of the quantity being predicted. Five-year risk in this cohort has a median of 0.396, with the central eight tenths of people between 0.273 and 0.677, a band about forty points wide, so an average departure of 0.084 - roughly a fifth of it - can move somebody from one side of a threshold to the other. Comparing it with the Riley sample size's 0.045 says only that the other one is smaller, which is not a yardstick for whether this one is large - and 0.045 is not small either; the near-doubling between them is exactly what this section is demonstrating. Using the maximum of 0.999 as the reference is worse still: that is the extreme of the distribution, not a typical person. Note also that average departure per person is not the quantity any of the three criteria targets; it is an extra check this page adds.

Correct answer: All three have to hold at once, so the figure adopted is the largest; 436 is set by the first, which targets a shrinkage factor of 0.9, and by the third

436 is the largest of the three requirements, and three criteria means three things must all be achieved, so the sample size adopted is always the maximum; 174 says only that the second criterion, about the gap between apparent and adjusted R-squared, is easier to satisfy for this model. It is neither a compromise nor a standard to work to. The 31 events per parameter is an output, not an input: once 436 people are computed, the events they would supply at this event rate and time horizon are worked out and divided by the number of parameters. Read the other way round it collapses back into another events-per-variable rule, which is the thing this calculation exists to replace. And the 0.9 the first criterion targets is the expected uniform shrinkage factor, meaning coefficients should need shrinking by only about a tenth.

Correct answer: 795 exceeds the survival requirement because dichotomising throws the timing away: the same people are reduced to a single yes-or-no about five years, they carry less information, and reaching the same shrinkage factor takes more of them

The gap between 795 and 436 is a matter of information. The survival version uses when each event happened, and how long each censored person was known to be event-free; dichotomising collapses all of that into one yes-or-no. The 43 events per parameter is higher than the survival version not because there are more events - the binary event definition is no looser - but because more people are required, so more events come with them. The 201 row cannot be read backwards either: each criterion is computed separately and the maximum is always adopted, so the final figure being set by the strictest criterion says nothing about the others being negligible - and 201 is in fact above the corresponding 174 of the survival version, so even the claim that every binary criterion is looser does not hold. It also shows why counting events alone cannot answer the sample-size question: that count cannot see the time axis.

Correct answer: Optimism in the C-index at the events-per-variable sample size is 0.038: with too small a sample the model still fits and still looks unusually good on its own data

0.038 is how far apparent performance exceeds hold-out performance. The anti-conservatism is right there: too small a sample does not leave you with nothing, it leaves you with a model that looks good and cannot be used, and the number a paper reports is the one that looks good. 0.017 says only that the bias shrinks when the sample is ample; it cannot be turned into a claim that the failure is conservative, since the bias is positive in both rows and never changes sign. The percentile at 0.487 does show that the variation between runs is large as well, but that is a separate problem: even from a single run, reading only the point estimate, the point estimate is already optimistic. This is why sample-size calculation in prediction-model research is not about being able to see an effect if there is one, but about the computed risk being fit to show a patient.