Reference-Free Alignment for Longitudinal Chest X-ray Progression

When a model judges whether a patient's chest X-ray has improved, stayed stable or worsened since the last visit, it has to line up the two images first, either explicitly with a warp or implicitly through attention. Every explicit pipeline we know of does this asymmetrically. It picks one scan as the fixed reference and bends the other onto it. In population shape analysis this choice is known to be dangerous. Measurements made in a single subject's coordinate frame drift toward that subject's quirks, a problem called template-induced or reference bias. Population studies solve it with groupwise registration, which aligns all images into a neutral common space that none of them owns. Our claim is that the same bias shows up inside longitudinal clinical models, and that it leaves a measurable trace. If the prediction of "worsening" changes when you swap which scan is the reference, or fails to flip when you reverse time, then the model is reporting geometry rather than disease. That risk is highest exactly where radiologists find comparison hardest: a bedside front-to-back (AP) film against an earlier standing back-to-front (PA) film, or a deep-breath scan against a shallow one.

We propose to measure this anchor bias directly on public chest X-ray benchmarks. We then remove it by aligning each patient's full visit history into a patient-specific common space constrained so that no visit is privileged, and we keep the geometric change itself as an input rather than throwing it away. The project succeeds if it shows one of two things. Either anchor-dependent predictions are real and the reference-free alignment removes them without costing accuracy, or they are negligible for chest X-rays, which would be a useful negative result for a field that has started adopting input-level registration.

Core Claim

Asymmetric registration in longitudinal models carries a directional bias: change is measured in the reference image's geometry. For two scans whose shape differs because of posture or breath-hold, the apparent size of a finding then depends on which scan was the reference. We hypothesise three consequences. First, progression predictions shift systematically with anchor choice. Second, the shift grows with the geometric gap between the two acquisitions. Third, models built on such alignment violate time-reversal consistency: reversing the visit order should turn "improving" into "worsening", and it does not. A patient-specific groupwise space removes the first effect by construction. The constraint is that the deformations from the common space to each visit average to the identity, so no single visit is privileged. Adding a temporal smoothness term across three or more visits also enforces transitive consistency, which chained pairwise warps do not guarantee.

Where This Sits

Three lines of work motivate the gap. The vision-language progression models fall into two groups. BioViL-T, which splits each scan into static and interval-change features, and HERGen, which attends causally across visits with date embeddings, are registration-free. BioViL-T's authors concede that AP-versus-PA differences remain hard, and HERGen's that latent attention cannot guarantee the same physical structure is tracked across years. CoCa-CXR restricts attention to corresponding local regions but assumes rigid patch correspondence. DeltaNet subtracts a prior or retrieved reference image from the current one, which is anchoring in its plainest form. None of these papers asks whether its predictions depend on which image anchors the comparison.

The closest work is Reconsidering Spatial Alignment for Longitudinal Breast Cancer Risk Prediction. It supplies both a method and a constraint. It shows that warping feature maps (FeatAlign) makes the network fold its deformations into anatomically impossible shapes to lower its task loss, and that registering at the image level (ImgAlign, via MammoRegNet) works better. We therefore align at the image level. But MammoRegNet warps the prior onto the current mammogram, so the current scan is the template. Our proposal keeps their image-level design and asks what that asymmetry costs.

Direction 1 supplies the remedy. Registration of Longitudinal Image Sequences with Implicit Template and Spatial-Temporal Heuristics registered whole multi-subject brain sequences into a hidden common space, with smooth per-subject trajectories ("temporal fiber bundles"). It recovered 15.73% of a simulated 15.86% atrophy, where standard registration recovered 12.8%. That was a 9-subject, morphometry-only study with no downstream classifier. GroupMorph makes groupwise diffeomorphic registration fast enough to use, about 2 seconds per series versus 652 for Elastix, but evaluates only image similarity. The general evidence that a single-subject anchor skews measurements comes from Dalal et al. and Shan et al.. Finally, DisMorph, which separates scanner distortion from atrophy, is a reminder of a design constraint: geometric change can be either nuisance or biology. We therefore do not discard the deformation; we feed its local volume change to the classifier.

Research Program

For a patient with visits $I_1,\dots,I_N$ at times $t_1<\dots<t_N$ (with $N\ge 2$), we estimate stationary velocity fields $v_i$. Each one is integrated to a diffeomorphism $\phi_i$, a smooth invertible warp, which maps the common space onto visit $i$. We solve

$$\min_{v_1,\dots,v_N}\ \sum_{i=1}^{N} D\big(I_i\circ\phi_i,\ \bar I\big) + \lambda \sum_{i} R(v_i) + \mu \sum_{i=1}^{N-1}\frac{\lVert v_{i+1}-v_i\rVert^2}{t_{i+1}-t_i}\quad \text{s.t.}\ \sum_{i} v_i = 0 .$$

Here $D$ is a local normalized cross-correlation dissimilarity, $\bar I$ is the evolving mean of the warped images, $R$ is a smoothness penalty, and $\lambda,\mu$ are weights. The centering constraint $\sum_i v_i = 0$ is what makes the space reference-free. With $N=2$ the problem reduces to symmetric half-way registration. With $N\ge 3$ the time-weighted term adds the trajectory smoothness of Wu et al.

We will train an amortized network for this problem. It extends the anatomically constrained chest X-ray registration in ACRN to GroupMorph's all-moving setup, and we will report the fraction of negative Jacobians to check folding.

The progression classifier receives the visits warped into the common space, together with the per-visit log-Jacobian maps, which record where tissue was locally expanded or compressed by the warp. Effusions and volume loss partly appear as geometry, so this input keeps that signal. The image encoder is initialized from BioViL-T. We add an equivariance loss so that reversing the visit order swaps the "improving" and "worsening" logits.

The audit is a separate contribution that applies to any model. We define three quantities. The anchor gap is the change in predicted label distribution when the reference image is swapped. The reversal inconsistency is the rate at which order reversal fails to flip non-stable predictions. The transitivity violation rate is measured on visit triples. We report all three stratified by geometric disparity between the two scans, using AP/PA mismatch and the ratio of segmented lung-field areas as a proxy for breath depth.

Baselines

The naive reference point is the majority class ("stable") plus a model that concatenates unaligned prior and current images into the same backbone. This mirrors the NoAlign arm of the mammography study, which reached a C-index of 63.6% on EMBED and 54.1% on CSAW-CC. Anything that fails to beat concatenation on progression accuracy is not worth its alignment cost.

The strongest existing method on our main benchmark is CoCa-CXR, which reports 65.0% average progression classification accuracy on MS-CXR-T across five conditions. BioViL-T sits at about 60.2%, a figure inferred from CoCa-CXR's stated 4.8-point margin rather than read from BioViL-T itself. We will compare against CoCa-CXR's reported number on the official split. We will also re-implement its regional cross-attention on our shared encoder, so that the audit metrics can be computed for it; no audit numbers are reported anywhere. Beating it on accuracy is not our main claim. We count success as matching its accuracy within one point while cutting the anchor gap and reversal inconsistency by at least half, and as outperforming it in the highest-disparity tercile.

The competitor that bears most directly on our hypothesis is MammoRegNet-style anchored image-level registration, run twice: prior-to-current and current-to-prior. Its mammography results (C-index 66.9% on EMBED, 61.3% on CSAW-CC, 0.0016% negative Jacobians) set the bar on that domain. On chest X-rays it shares our registration network, training data, epochs and hyperparameter search budget, so the only difference is the centering constraint.

Two ablations isolate our key ingredient. Symmetric half-way registration without temporal smoothness separates "no privileged anchor" from "trajectory consistency". Removing the log-Jacobian channels tests whether discarding geometry loses real signal. FeatAlign on the same backbone is included for completeness, because the mammography study predicts it will fold.

Why not just…?

Why not just rely on registration-free attention? CoCa-CXR's gain from restricting attention to local neighbourhoods already suggests that spatial correspondence matters, and BioViL-T's authors flag acquisition mismatch as a failure mode. Attention can still be anchor-free in principle, so this is an empirical question. The audit answers it: if registration-free models show a small anchor gap and good reversal consistency in the high-disparity stratum, explicit alignment adds nothing and we will say so.

Why not just register to the earliest or latest scan? That is exactly the design whose bias we measure. Wu et al. show that independently anchored registration underestimates true change (3.84% versus 5.38% detected hippocampal shrinkage, against a 5.65% expert estimate). The two-anchor runs of the MammoRegNet-style baseline will show directly whether chest X-ray predictions move with anchor choice.

Why not just train with random anchors and reversed pairs as augmentation? This is the most serious threat. Augmentation might teach any pipeline to be anchor-invariant at no registration cost. We include it as a full baseline: anchored registration with random anchor choice, reversal augmentation and the same equivariance loss. If it matches our method on accuracy, anchor gap and transitivity, the groupwise space is unnecessary for pairs. Our method would then survive only through the multi-visit trajectory term, which augmentation cannot supply, and we would narrow the claim accordingly.

Proposed Experiments

The main benchmark is MS-CXR-T: 1,326 image pairs labelled improving, stable or worsening. We report accuracy against CoCa-CXR's 65.0% and the three audit metrics by disparity stratum. Because MS-CXR-T contains only pairs, we build a multi-visit test set from MIMIC-CXR patients with at least three studies. Progression labels for consecutive and skip-one pairs are extracted from comparison sentences, following the comparative-report filtering that CoCa-CXR used to build its CXR-4 data. Transitivity and the temporal smoothness term are evaluated there. As a second domain where an anchored baseline has reported numbers, we rerun the EMBED and CSAW-CC risk-prediction setup with the current-anchored MammoRegNet arm replaced by our symmetric or groupwise alignment over all available priors, reporting C-index against 66.9% and 61.3%.

The decisive check is the disparity-stratified anchor gap of the anchored baseline. If it does not grow with AP/PA mismatch or breath-depth difference, template bias is not a real failure mode for chest X-rays, and the program ends with that finding. If it does grow, the next question is whether the gap shrinks under our alignment more than under the augmentation baseline. A third check concerns time-weighted smoothing on irregular visits. CRONOS showed that simply copying the last image is hard to beat (NRMSE 8.38 versus 7.55 for their flow model on Lumiere), so trajectory smoothing must beat a copy-last-geometry variant before we credit the multi-visit term with any gain.