Do Clinical Composite Biomarker Scores Give Honest Risks Inside Their Own Strata?

Doctors routinely combine two or three blood tests into one number. The Prognostic Nutritional Index adds weighted albumin and lymphocyte count. The red-cell-distribution-width-to-albumin ratio divides one marker by another. The Advanced Lung Cancer Inflammation Index multiplies body mass index by albumin and divides by the neutrophil-to-lymphocyte ratio. Dozens of papers report that these scores rank patients well. None of the papers in this file checks whether a given score value maps to a correct absolute risk, that is, whether the score is calibrated: of patients told "20% risk," roughly 20% should have the event. This matters because a ratio sends many different patients to the same number. A patient with low albumin and normal RDW can have the same ratio as one with normal albumin and high RDW, while their biology and risk may differ. We call this ratio aliasing. If it occurs, a composite score can look calibrated on average and still be badly wrong for identifiable patients. Those patients are usually the older, sicker ones, where the biomarker literature already reports the scores failing.

This note proposes a measurement study and a small method. The measurement is a calibration audit that checks each composite score, at each time horizon, within strata defined by age and by the score's own components. The method is a recalibration layer with a guarantee for each stratum. It keeps the clinical score as the ranking signal and corrects its risk mapping where the audit finds it fails. The decisive comparison is against a flexible survival model trained on the raw components. That comparison tells us whether a recalibrated composite score is sufficient, or whether it throws away risk information that should push clinicians to drop it.

Core Claim

We claim three things. First, composite inflammation–nutrition scores carry calibration errors that are hidden at the population level but large inside two kinds of stratum: age bands, and regions of component space that share a score value. Second, these errors differ by prediction horizon. Third, a recalibration that is monotone in the score and conditional on a small number of strata removes most of the error without giving up the score's interpretability.

The biomarker side motivates the claim. Zhou et al. report that the Prognostic Nutritional Index's association with heart failure "effectively disappears" in adults aged 60 and over, although the pooled AUC is 0.642. Liu et al. find that the RDW-to-albumin ratio is the best 1-year mortality predictor (AUC 0.768), while the inflammation index takes over at 10 years (AUC 0.691). Threshold-shaped dose responses for the neutrophil-percentage-to-albumin ratio (Liu et al., inflection at 13.6) and for the hemoglobin–albumin–lymphocyte–platelet score (Dong et al., J-shaped) point the same way. A single fixed mapping from score to risk is unlikely to hold everywhere.

All of these papers report odds ratios, hazard ratios, AUC or concordance, and none reports calibration. No reported result was found on Lacuna for the calibration of PNI, RAR, ALI, NPAR or HALP on any cohort.

To make the aliasing claim testable, let $s=g(x_1,\dots,x_k)$ be a composite score computed from component markers $x_j$. Let $\hat p_t(s)$ be the predicted probability of an event by horizon $t$, and let $A$ range over a fixed partition of patients into strata, where each stratum is an age band crossed with a coarse bin of one component's share of the score. We estimate

$$\Delta_t = \sum_{A} w_A \,\Big|\, \Pr\big(T \le t \mid \hat p_t(s)\in B,\, X\in A\big) - \mathbb{E}\big[\hat p_t(s)\mid \hat p_t(s)\in B,\, X\in A\big] \Big|,$$

averaged over risk bins $B$. Here $T$ is event time and $w_A$ is the stratum's share of patients. Observed rates are estimated with inverse-probability-of-censoring weights to handle patients lost to follow-up. Marginal calibration only controls the version of $\Delta_t$ with a single stratum. Ratio aliasing means $\Delta_t$ stays large even after the marginal version has been driven to zero.

Research Program

The program has three parts.

The first part is the audit. We fit the standard mapping from score to risk, a one-variable Cox or logistic model on the score at each horizon of 1, 3 and 10 years. We then compute marginal calibration, $\Delta_t$, and a test of whether the components predict outcomes beyond the score among patients with the same predicted risk. The test is a likelihood-ratio test for adding component-share terms to a model whose offset is the calibrated score.

The second part is the recalibration layer. For each horizon it fits a mapping from score to risk that is constrained to be monotone in the score. It allows a per-stratum shift and slope, using a small number of strata chosen in advance: age band, plus one component-share axis per score. It then applies group-conditional conformal correction within each stratum. That correction follows the group-wise (Mondrian) scheme that the conformal trajectory-band work used across sex, race and APOE4 status. For survival targets, it borrows the vertical, probability-space adjustment of CiPOT, which keeps survival curves monotone.

The conformal paper is univariate and is about forecasting trajectories. CiPOT targets conditional calibration of a general survival model. Neither asks whether a hand-built clinical score can be repaired, and neither uses the score's own components to define the conditioning strata. Both of those choices are what is new here.

The monotonicity constraint is deliberate. A clinician should never see risk fall as albumin-derived nutritional reserve gets worse.

The third part is the information test. We train flexible survival models on the raw components plus age. This asks how much of what remains after recalibration is lost signal rather than mapping error.

Baselines

The naive reference point is each composite score used as it is in the source papers, entered into a one-variable Cox model with no recalibration. The published discrimination numbers serve as sanity anchors. We should reproduce roughly 0.768 AUC for the RDW-to-albumin ratio at 1 year and 0.691 for the inflammation index at 10 years on NHANES heart-failure mortality (Liu et al.), and 0.642 for PNI on heart failure (Zhou et al.). No calibration number exists for these baselines.

The second baseline is ordinary marginal post-hoc recalibration of the score: Platt scaling and isotonic regression fitted on a calibration split. This is the tool Zhong et al. used to repair an XGBoost breast cancer model whose Western competitor predicted 80.6 deaths where 27 occurred. We also include the Kaplan–Meier regularizer of Yanagisawa et al., which matches the model's average curve to the population curve. These baselines represent the "calibration is solved by a post-processing step" view. Our method beats them if it lowers $\Delta_t$ while matching their marginal calibration.

The strongest existing competitor is a flexible survival model on the raw components and age, trained either with the differentiable calibration loss of X-CAL or with the rank-probability-plus-kernel loss of Kamran and Wiens, and then corrected with CiPOT. CiPOT is the best conditional calibrator in the file. It improved marginal calibration in 95 of 104 comparisons and conditional calibration in 64 of 69 cases across 15 datasets, while preserving concordance in about 79% of settings (Qi et al.). For reference, the Kamran–Wiens loss reached a calibration divergence of 0.056 on SUPPORT, against 0.138 for the earlier DRSA model.

All arms get the same patients, the same splits, the same strata used for evaluation, and the same budget of 50 random-search hyperparameter trials, all selected on validation data only. We count it as beating this baseline if the recalibrated score comes within a pre-registered margin of 0.01 in stratum-conditional calibration error (the averaged absolute gap $\Delta_t$, on the probability scale) and of 0.02 in concordance, while staying a monotone, one-line function of a known score.

Two ablations isolate the key ingredient. The first stratifies by age only. The second stratifies by component share only. If component-share strata add nothing, the aliasing claim fails.

Why not just…?

Why not just recalibrate the composite score with isotonic regression and stop? Isotonic regression fixes the average mapping. It cannot fix a patient group that shares a score value with a better-off group, because it only sees the score. The case for stratum-level failure is indirect: PNI's association vanishes after age 60 (Zhou et al.), and marginal methods leave subgroups miscalibrated (Qi et al.). The audit tests this directly. If $\Delta_t$ after isotonic recalibration is already small, this baseline wins and our method is unnecessary. That would be a useful negative result for clinicians.

Why not just abandon composite scores and fit gradient boosting or a deep survival model on all the raw labs? This is the real threat, and the information test exists to settle it. If raw-component models clearly beat the recalibrated score in conditional calibration, the conclusion is that these ratios destroy signal and should be retired in favor of learned risk models. We would report that rather than argue around it. Small cohorts cut in the other direction. GlucoBench found that simple models beat deep ones on small datasets (Sergazinov et al.), and the NHANES heart-failure mortality cohort has only 1,500 patients. So the answer may differ between NHANES and MIMIC-III, and we expect it to.

Why not just add fairness-style parity constraints across age groups? Parity constraints push groups toward equal outcomes, not toward honest probabilities. The empirical study of fair clinical risk models found they increase absolute calibration error and degrade AUROC almost universally. Group-conditional calibration asks only that each group's stated risk be true. That is a different and weaker requirement.

Proposed Experiments

The main cohort is NHANES 1999–2018 linked to the National Death Index. We use the 1,500-patient heart-failure cohort of Liu et al. and, for statistical power, the full adult NHANES population with the same linkage. On both we compute PNI, the RDW-to-albumin ratio, the inflammation index, NPAR and HALP. Survey weights are applied, and splits are temporal, with later cycles held out. This tests whether calibration transports over time, the failure Zhong et al. saw across populations.

The second cohort is MIMIC-III, as used by X-CAL. Its routine labs allow PNI, the RDW ratio, NPAR and the neutrophil-to-lymphocyte ratio to be computed for in-hospital and post-discharge mortality at scale. The third is flchain, a small control case where the clinical marker is itself a sum of two light-chain assays. It is used in Yanagisawa et al..

We report five metrics: D-calibration and marginal calibration error, $\Delta_t$ at 1, 3 and 10 years, concordance, the component likelihood-ratio statistic, and net benefit at clinically used thresholds.

Three results would be decisive. The first is whether $\Delta_t$ for marginally recalibrated scores exceeds that of the full method by more than its bootstrap interval, especially in the 60-and-over band. The second is whether the component-share ablation contributes reduction beyond age-only stratification. The third is whether the raw-component CiPOT model's advantage over the recalibrated score shrinks to within the pre-registered margin. A positive first result with a negative third would tell clinicians that a recalibrated ratio is enough. A positive first and positive third would tell them the ratio itself is the problem.