Borrowing the Tail: Long-Horizon Risk Prediction from Short Follow-up

Many clinical prediction models are trained on cohorts that were only followed for a few months, and they quietly treat anyone who had not yet had the event by the end of that window as someone who never will. A model of atrial fibrillation returning after catheter ablation, trained on 3 to 6 months of follow-up, learns to predict "recurrence by month 6," not "recurrence." A depression model trained on a 12-week window learns to predict "improved at week 12," not "stayed well." The usual fix is a survival model, a model that treats unfinished follow-up as censored rather than as a negative label. That fix does not solve the problem either. A survival model fit on a short cohort has no information about risk after the window closes. It cannot tell late failures from people who are truly cured.

We propose to borrow the missing tail from a second, older cohort that was followed for years. The idea is to split the long-horizon question at the end of the short window, which we call the landmark. Risk before the landmark is learned from the short cohort itself. Risk after the landmark is learned from the long cohort, conditional on each person's baseline features and on measurements taken at the landmark. These include intermediate markers such as left-atrial function, a partial disease-activity score, or quarterly earnings. The two pieces are then composed into a calibrated estimate of long-horizon risk for every individual in the short cohort. The landmark measurements are what make this more than pooling. They carry the effect of a new treatment that the long cohort never saw. The working assumption is that, once those measurements are known, the treatment has no further influence on the tail.

Core Claim

Let $T$ be time to event, $\tau$ the end of short follow-up, $H > \tau$ the clinically relevant horizon, $X$ baseline covariates, and $S_\tau$ the vector of intermediate measurements taken at $\tau$. For a person in the short cohort $E$ who is event-free at $\tau$, we predict

$$P_E(T \le H \mid X, S_\tau, T > \tau) \approx g_O(X, S_\tau), \qquad g_O(x,s) = P_O(T \le H \mid T > \tau, X = x, S_\tau = s),$$

where $g_O$ is a supervised, censoring-aware model fit only on landmark survivors in the long cohort $O$. A person's unconditional horizon risk is then the short-cohort probability of failing before $\tau$ plus the probability of surviving to $\tau$ times the expected value of $g_O$.

The identifying assumption is landmark transportability. Among people event-free at $\tau$, the distribution of $T$ given $(X, S_\tau)$ is the same in $E$ and $O$, even if $E$ received an intervention absent from $O$.

This is weaker than what pooling requires. Pooling needs the whole hazard to transport given $X$ alone. Landmark transportability is the time-to-event, individual-level analogue of the surrogacy and comparability conditions in the surrogate index of Athey et al.. That work imputes a long-term outcome from short-term surrogates using an observational sample and averages the imputations into a treatment effect. We keep its two-sample logic and change three things:

  • We target individual risk predictions, not an average effect.
  • The target is a censored event time rather than a fully observed outcome.
  • We condition on event-free status at the landmark, which turns the cure-versus-late-failure ambiguity into an estimable conditional quantity in $O$.

The claim is that this composition gives better-calibrated long-horizon risk than any model fit to the short cohort, pooled or not, whenever the intervention acts through measurable landmark state. We also claim that it fails in a predictable, detectable way when it does not.

Why the Gap Is Real

Two lines of work motivate this proposal by documenting the truncation problem without fixing it. Yang et al. fit a logistic model for atrial fibrillation recurrence from echocardiographic left-atrial indices plus BNP, and counted recurrences after 3 to 6 months as non-recurrences. Cross-platform prediction of depression outcome predicts 12-week improvement from smartphone mobility features and cannot see relapse.

Two further lines of work supply the method but constrain the design. DeepSurv replaces the linear term of the Cox model with a neural network, so it handles censoring and non-linear risk. Its baseline hazard is undefined beyond the last observed time, so on its own it cannot extrapolate past $\tau$. ML mixture cure models separate a cured fraction from a susceptible fraction using SVM or neural classifiers. The authors themselves note that this separation is not identifiable when follow-up ends before the survival curve plateaus, which is exactly our regime. Our landmark split sidesteps that identifiability failure by moving the tail question to a cohort where it is identified.

Inferring long-term effects of long-term treatments acts as a hard constraint. It shows that surrogate-based methods underestimate the effect of an ongoing treatment by over 50% in a streaming simulation, because the treatment keeps acting after the surrogate window. Our assumption breaks in the same way. We therefore treat continuing interventions as an explicit failure case to measure, not something to argue away. That paper's offline reinforcement-learning estimator needs Markov dynamics and targets average effects, whereas we target individual risk.

Finally, Stop Chasing the C-index constrains evaluation. It argues that concordance can improve while calibration degrades under non-random censoring, and that most recent survival papers use metrics misaligned with their censoring assumptions. Our primary metrics are therefore horizon-specific calibration and proper scoring rules, with C-index only as a secondary number for comparability.

Research Program

The first component is the estimator.

  • Tail model. We fit $g_O$ on landmarked data from $O$: people event-free at $\tau$, with $S_\tau$ measured, followed to $H$ or censoring. We use inverse-probability-of-censoring weighting, which reweights observed outcomes by the inverse probability of remaining uncensored. The fitted models are gradient-boosted trees, a small neural network, and a DeepSurv-style network applied to time since landmark.
  • Pre-landmark model. The short cohort contributes a standard survival model on $[0, \tau]$.
  • Uncertainty. We compute uncertainty by cross-fitting across both samples, so the tail model is never scored on the data it was trained on. This guards against the overfitting the real-world surrogate evaluation paper warns about when a transformation of the markers and its evaluation share a cohort.

The second component is a diagnostic for landmark transportability. Some short cohorts have early enrollees with longer follow-up. Where they exist, we compare $g_O$'s predictions to their observed tails with a calibration test. We also estimate the proportion of treatment effect explained by $S_\tau$ using the doubly robust estimator of the real-world surrogate paper. A low value is a warning that the treatment acts outside the measured landmark state.

We tie this diagnostic to the SPIRIT-Surrogate guidance, which requires protocols to justify surrogate validity but offers no estimator. Our diagnostic is a concrete, reportable check of the kind it asks for.

The third component is a delayed-reveal evaluation protocol. In it, the long-horizon truth exists but is hidden from the method. We build every test this way so that claims about the tail are verified rather than assumed.

Baselines

Naive truncated classifier (do-nothing reference). This is the do-nothing reference point and the practice we want to displace. We train a logistic regression and a gradient-boosted classifier on the short cohort, labeling anyone event-free at $\tau$ as negative, and read the output as horizon-$H$ risk. This mirrors Yang et al., who report AUC 0.814 for their combined model versus 0.703 for BNP alone, though only at the 3-to-6-month horizon. We expect it to rank people reasonably but to underestimate absolute risk systematically. Beating it means lower horizon-$H$ Brier score and calibration slope closer to one on hidden long-term labels.

Pooled deep survival model (strongest existing method). This is the strongest existing competitor on our question. We fit DeepSurv on $E \cup O$, with $E$ administratively censored at $\tau$ and a cohort indicator as a covariate. It uses all the data, handles censoring correctly, and its tail comes from $O$. The most relevant reported number is DeepSurv's C-index of 0.867 on METABRIC versus 0.816 for linear Cox and 0.619 for random survival forests (DeepSurv). That is a discrimination number on a single cohort, not long-horizon calibration after truncation. No reported result was found on Lacuna for horizon calibration under truncated follow-up.

For fairness, all methods get the same splits, the same covariates, the same network or tree families, and an equal hyperparameter search budget selected by inner cross-validation on the same proper scoring rule. Beating this baseline means significantly lower IPCW Brier score at $H$ on the hidden labels when a new treatment is present in $E$. We also expect to tie it when there is no treatment shift. A tie is the honest prediction there, since both methods are then valid.

Mixture cure models. We run SVM- and neural-incidence mixture cure models from ML mixture cure, on $E$ alone and pooled. They are the natural answer to the question "who is cured." That paper reports lower bias and mean squared error than logistic incidence under 71.6% censoring, but no excerpted values, so no reported number is available on Lacuna for it.

Surrogate index. We also run the Athey et al. surrogate index fit naively to the binary horizon outcome, without landmark conditioning or censoring weights. This isolates what our time-to-event treatment adds.

Ablation. The key ablation removes $S_\tau$ from $g_O$, keeping only $X$ and event-free status. If the ablated model matches the full method, our claimed mechanism is wrong and the gain, if any, is just conditional survival.

Why not just…?

Why not just pool both cohorts and fit a censoring-aware survival model? This is the serious threat. If $E$ and $O$ have the same hazard given $X$, pooling is valid, more efficient, and should match or beat us. It breaks when $E$ contains an intervention that changes risk, because a cohort indicator lets the pooled model shift early hazard but gives it no information about how that shift persists past $\tau$. Our landmark measurements carry that information. The semi-synthetic experiments vary treatment strength from zero upward, so we can read off where, if anywhere, pooling stops being competitive. If the crossover never appears on the real-data tests, we will report that pooling is sufficient.

Why not just fit a machine-learning mixture cure model on the short cohort? Its own authors state that the cure fraction is not identifiable when follow-up ends before the survival curve plateaus (ML mixture cure). Within the window, a high cure rate and a slow late tail look the same. The pooled mixture cure baseline tests whether adding $O$ rescues it. We expect it to inherit the same weakness as pooling under treatment shift.

Why not just use the Athey surrogate index on a binary long-term outcome? It is close to our method and should do well when follow-up in $O$ is complete. It does not condition on who has already failed before $\tau$. It also treats censoring in $O$ as a fully observed outcome, which biases risk downward when the long cohort itself has dropout. The direct surrogate-index baseline measures how much these two changes matter. If the difference is negligible on data with little censoring in $O$, we will say so.

Why not just use the offline reinforcement-learning estimator for ongoing treatments? That approach targets average effects, assumes Markov dynamics, and was tested only in simulation. It is the right tool when treatment continues after $\tau$, which is precisely where our assumption fails. We use its streaming and Sepsis simulators to map that boundary rather than to compete inside it.

Proposed Experiments

Each experiment withholds long-term labels from the method and scores against them. Primary metrics are IPCW Brier score at $H$, calibration slope and intercept at $H$, and, for the treatment test, bias of the horizon-risk difference between arms. C-index is reported as a secondary number.

GAIN (real outcomes, Athey et al. design). GAIN is the job-training trial used by Athey et al..

  • Riverside is the short cohort, with outcomes after quarter 6 hidden.
  • The other counties form $O$.
  • Quarterly earnings and welfare at $\tau$ are $S_\tau$.
  • Time to sustained employment loss up to quarter 36 is the target.

No numerical surrogate-index estimate for GAIN was found on Lacuna. We will rerun their estimator as a baseline.

YODA ulcerative colitis trial (real outcomes). In this trial, 54-week outcomes exist for both infliximab and golimumab arms.

  • One arm serves as $O$ and the other as $E$, with outcomes after week 6 hidden.
  • The 6-week partial Mayo score is part of $S_\tau$.
  • The real-world surrogate paper reports that the 6-week score explains 87.2% of the treatment effect (95% CI 0.741 to 1.003) with its doubly robust estimator. This predicts landmark transportability should roughly hold here, so the setting is a test our method should pass.

Semi-synthetic truncation. We use METABRIC, Rotterdam/GBSG, WHAS and SUPPORT, all from DeepSurv.

  • Each long-follow-up cohort is split into $O$ and $E$, and $E$ is truncated at a range of $\tau$.
  • We simulate a treatment in $E$ that shifts hazard through a generated mediator, observed as $S_\tau$, plus a tunable direct effect that bypasses it.
  • This yields a dose-response curve for each method as the direct effect grows. Our method should win when the direct effect is zero and degrade in proportion as it grows, and the diagnostic should flag the degradation.

Failure-boundary tests. The streaming and Sepsis simulators from the long-term treatments paper are cases where treatment continues past $\tau$. Here we expect our method to fail. The decisive question is whether the transportability diagnostic detects the failure before the hidden labels are revealed.

Decision criteria. The proposal succeeds if three things hold. First, on GAIN and YODA, our method beats pooled DeepSurv and the naive classifier on horizon Brier score and calibration with cross-fitted confidence intervals excluding zero. Second, the $S_\tau$-ablation loses that advantage. Third, the diagnostic separates the semi-synthetic settings that satisfy landmark transportability from those that violate it. If pooling matches us everywhere, the conclusion is that practitioners should pool and censor, which is a useful negative result for the many short-window clinical models built today.