Prediction model study

Correct answer: No. The calibration slope is 0.691, clearly below perfect, and the C-index is immune to shifting or stretching the risk scale

The C-index asks only whether the model ranks higher-risk people above lower-risk ones. Multiply or add a constant to every predicted risk and the ordering is untouched, so the C-index does not move either. The fall from 0.666 to 0.642 therefore says nothing about whether the predicted numbers themselves are right. The calibration slope of 0.691 speaks to something else: the risk differences the model produces are too extreme for this population, pushing high-risk people too high and low-risk people too low. A prediction model paper reporting only a C-index or an area under the curve, with no calibration, has reported half of what matters.

Correct answer: The model predicted 0.287 for them and 0.364 actually occurred, so this group would be filed as low risk when their real risk is not low

All five groups sitting above the diagonal means the whole scale has been shifted down, not that one group had bad luck in sampling. The lowest-risk group was predicted 0.287 and observed 0.364, which is clinically the most dangerous place for the error to land: a threshold rule compares the predicted value against the threshold, and predictions that are systematically low file people who should be treated on the untreated side. The 0.721 is indeed the highest-risk group's observed value, but that group's prediction nearly matches it, because the shift is not uniform and the low-risk end is off by more. The shape comes from two things at once: the whole scale translated downward, plus a calibration slope below perfect calibration pulling the high-risk end back. A shallow slope alone could not produce this plot, since it would put the high-risk end below the diagonal.

Correct answer: No. An optimism of 0.0004 is that small because the number of events is very large relative to the number of parameters, leaving almost nothing to overfit

Optimism measures how much better a model looks on the data that produced it than on data it has never seen. It comes out small usually because the model had no room to overfit, and here four predictors sit alongside more than seventeen hundred events, giving each variable a three-digit number of events, far beyond any rule of thumb. So 0.0004 says the sample is ample relative to the parameters, not that the model has captured something. The near identity of 0.6661 and 0.6657 is the same fact written twice. More importantly, however clean the internal validation is, it cannot answer the external question: this model's calibration in the external cohort is systematically off, and no bootstrap on the development data will ever see that.

Correct answer: External validation uses different people: those 686 patients differ from the development cohort in enrolment era, setting and clinical pathway

Splitting one dataset leaves both halves sharing the same eligibility criteria, the same investigators, the same time and place and the same variable definitions, so what it measures is internal stability rather than transportability. The difference between 686 validation patients and 2982 development patients is not the count but the fact that they are another cohort, which is also why a chain of harmonisation decisions has to be made before any model runs, such as converting banded tumour sizes into band midpoints. That conversion is an assumption rather than data, and it will not show up in any statistical test. The 1713 events are certainly ample, but having enough events never turns internal validation into external validation.

Correct answer: Broadly yes, but for the right reason: 428 events per variable is far past the rule of thumb, so sample size is not this model's constraint

The ten-events-per-variable rule comes from simulations of the nineteen nineties, and later work shows the required sample size is far more complicated than a single number, with the sample size calculations of Riley and colleagues being the current recommendation. So 428 supports a limited conclusion: it is enough to say sample size is not the constraint here, and not enough to say the sample is adequate because it cleared a threshold. Where the rule still earns its keep is in the other direction, since a model with single-digit events per variable is almost certainly overfitted. Both 1713 and 2982 are totals, and a large total with many predictors can still be too small; what decides is the ratio, not either absolute number.

Correct answer: Fix the overall underestimation with the intercept first, then deal with the slope, because a slope of 0.691 is not something a shift can repair and needs the second layer of recalibration

Recalibration has three layers, from light to heavy: intercept only, intercept plus slope, and re-estimating every coefficient. This plot shows two problems at once, so two layers are needed. The overall underestimation belongs to the first layer and is fixed by replacing the baseline risk. The slope of 0.691 belongs to the second, and no shift will move it. The 0.444 is the second group's observed risk and the largest departure, but being the largest does not mean aligning it fixes the rest, since the other four groups are off by different amounts. As for 0.539, that is only the lower bound of the slope interval, and broken calibration is almost never a reason to discard a model: recalibration usually suffices, which layer to use depends on how much data the new population supplies, and whichever layer is used, the recalibrated model needs validating again.