When a radiologist compares today's chest X-ray with last month's, the two images never line up exactly. The patient stands at a different angle, breathes in more or less deeply, or is imaged lying in bed instead of standing. Models that read X-rays over time treat whatever differs between the two images as evidence that the disease has changed, so misalignment between the scans gets reported as disease progression. Laser and terahertz wireless links have a structurally similar problem. A narrow beam has to land on a small receiver while the mast sways. That field splits the miss into two parts. The first is boresight error, a fixed offset that persists when everything is at rest. The second is jitter, a random wobble around that offset. The field then uses closed-form statistics to predict how often the beam misses and which of the two parts dominates. We propose to carry this decomposition, and its capture-probability mathematics, over to longitudinal chest X-ray models. There it serves two purposes: as a diagnostic that predicts how a temporal model degrades under misalignment, and as a design rule for how far each region of the current scan should look into the prior scan.
The bridge is concrete. CoCa-CXR restricts each patch of the current image to attend only to a fixed local neighborhood of the prior image. Attention here means the mechanism by which one image region pulls information from regions of another. That neighborhood plays the role of a receiver aperture, and the anatomical drift between scans plays the role of beam displacement. A too-small window misses the corresponding anatomy, just as a narrow beam misses the receiver. A too-large window lets in unrelated background, which is the failure of global attention that CoCa-CXR was designed to avoid. The pointing-error literature has already worked out how to reason about this trade-off. Clinical vision has not borrowed it.
Core Claim
We claim that misalignment between serial chest X-rays can be modeled as a pointing error with a per-pair systematic offset $b$ and an anisotropic random residual with standard deviations $\sigma_x, \sigma_y$. Here $x$ is the horizontal axis and $y$ the vertical axis, along which breathing depth mostly acts. We further claim that the probability that a current-scan patch's true counterpart falls inside its prior-scan attention window predicts how temporal classifiers fail. For a window of half-width $w$, displacement $d \sim \mathcal{N}(b, \mathrm{diag}(\sigma_x^2,\sigma_y^2))$ gives
$$P_{\text{capture}}(w; b, \sigma_x, \sigma_y) = \Pr\left(\|d\| \le w\right),$$
which in the zero-offset, equal-variance case reduces to the Rayleigh form
$$P_{\text{capture}} = 1 - e^{-w^2 / 2\sigma^2}.$$
This mirrors the role of the beamwidth-to-jitter ratio $w_{eq}/\sigma_s$ in Zedini et al.. With nonzero $b$ and equal variances the distribution becomes Rician. With zero $b$ and unequal variances it becomes Hoyt. These are exactly the generalized cases unified by Jung et al..
The claim has two practical consequences. The first is that the offset should be estimated and removed before any window is sized, because enlarging the window to absorb a fixed offset admits background noise in every direction. The second is that the remaining window should be shaped to the residual jitter, which means anisotropically, taller than it is wide.
What is new is not robustness to misalignment as a goal. What is new is a quantitative misalignment model that predicts accuracy loss and dictates an attention geometry. Existing longitudinal models either ignore misalignment, as the registration-free BioViL-T does, or hard-code one fixed window, as CoCa-CXR does.
Grounding in Both Literatures
The wireless side supplies the method. Ansari, Alouini and Cheng show that assuming zero boresight gives overly optimistic capacity predictions. This motivates our expectation that the common clinical analogue, assuming that projection-matched scans are aligned, is similarly optimistic. Jung et al. find that boresight dominates degradation at low signal-to-noise ratio and jitter at high signal-to-noise ratio, and that jitter sets an error floor which more transmit power cannot overcome. Our analogue of signal strength is the conspicuity of true change. This yields a falsifiable prediction: for subtle interval changes, systematic offset (for example, mismatched anteroposterior bedside versus posteroanterior upright views) should dominate errors. For conspicuous changes, the jitter residual should set a floor that more training data does not lift. Shang et al. find that jitter at an intermediate reflecting surface hurts more than equal jitter at the endpoints. This suggests a check we will run: whether misalignment injected at intermediate feature resolutions hurts more than misalignment at the input pixels. We treat this as exploratory.
The clinical side supplies the problem and constrains the design. Single-image models produce comparative statements without seeing a prior, as discussed around CheXNeXt. Difference modeling in DeltaNet and the static/dynamic splits in BioViL-T, TempA-VLP and DiPro all assume that the dynamic component is pathology. DiPro explicitly treats non-static change as disease. Our proposal targets exactly that assumption. Hu et al. sidestep geometry by matching segmented anatomical nodes. That is a strong but annotation-heavy alternative, and we discuss it below. One constraint shapes the method. Real disease can move anatomy: an effusion raises the apparent diaphragm. An offset estimator fit to the whole image could therefore "explain away" pathology. We therefore estimate the offset from structures that disease rarely moves, such as the spine, clavicles and upper rib cage.
Research Program
The work has three parts.
The first part is a measurement study. We take trained longitudinal models and apply controlled synthetic misalignments to the prior image of MS-CXR-T pairs. These are sweeps of fixed offset (translation, rotation, scale) crossed with sweeps of random per-pair perturbation of independent horizontal and vertical magnitude. We record progression accuracy and the rate of false "worsening" or "improving" calls on pairs labeled stable. We then test whether the capture probability above, computed from each model's effective attention reach, explains the degradation curves better than raw displacement magnitude does. We also test whether real mismatched-view pairs fall on the curve predicted by their estimated offset. View position comes from MIMIC-CXR metadata.
The second part is the method, which we call pointing-aware regional attention. A small estimator predicts the per-pair offset $b$ and a residual covariance from the image pair. It is trained without labels, by applying known geometric transforms to one image and regressing them, and it is restricted to stable anatomical structures as described above. The regional cross-attention window of each current-scan patch is then re-centered by $b$. The window is sized per axis so that predicted capture probability reaches a target value, for example 0.95, chosen on validation data. The rest of the encoder follows CoCa-CXR's staged recipe on MIMIC-CXR pairs.
The third part is analysis of when the decomposition matters. We stratify results by change conspicuity, using radiologist-labeled condition type and the magnitude of the report-level change as proxies. This tests the low-versus-high-conspicuity prediction derived from Jung et al.
Baselines
The naive reference point is a single-image classifier that sees only the current scan and predicts the majority progression class per condition. It shows how much any model actually uses the prior.
The second reference is BioViL-T, the registration-free hybrid encoder that introduced MS-CXR-T and handles misalignment only implicitly through global attention. It represents the "let the transformer figure it out" position.
The strongest existing method is CoCa-CXR. On MS-CXR-T it reports 65.0% progression accuracy, 4.8 points above BioViL-T, and 24.2% RadGraph F1 on MIMIC-CXR report generation. It is the right competitor because it already uses a local window, so any gain we show isolates how that window is placed and shaped, not whether locality helps. TempA-VLP reports 61.6% versus BioViL-T's 56.3% in few-shot progression classification, and we include it as a second temporal encoder. We found no reported result on Lacuna for any of these models under controlled geometric misalignment or stratified by view mismatch, so those numbers will be produced here.
For fairness, we reimplement regional cross-attention on the same image backbone, pretraining data, number of steps and hyperparameter search budget as our method. We report the original published number alongside as a sanity check. Beating CoCa-CXR means two things. On unperturbed MS-CXR-T we must match it within noise. On mismatched-view pairs and under synthetic offsets we must show a statistically significant accuracy gain, with confidence intervals bootstrapped over the 1,326 pairs, together with a lower false-change rate on stable pairs.
Ablations isolate the key ingredient. Offset correction alone uses CoCa-CXR's fixed window re-centered by $b$. Jitter-sized window alone sizes the window to the residual without re-centering. An isotropic version tests whether the vertical/horizontal asymmetry matters. If offset correction alone captures nearly all the gain, the claim reduces to "register first", and we will report it that way.
Why not just…?
Why not just register the prior image to the current one before feeding the model? This is the most serious threat. Rigid or affine registration removes most of the systematic offset, and it is our "offset correction alone" ablation, run as an explicit preprocessing step with a standard intensity-based registration tool. Our argument against it being sufficient is twofold. Registration leaves a residual that is anisotropic and non-rigid, mostly from breathing. Deformable registration, meanwhile, risks warping real pathology away, which is the same confound DiPro's static/dynamic split ignores. The experiment settles the question directly. If registration plus CoCa-CXR's fixed window equals our method on mismatched-view pairs and under jitter sweeps, the jitter-shaped window adds nothing, and the contribution becomes the diagnostic study alone.
Why not just augment training with random shifts and rotations of the prior? Augmentation can teach tolerance to jitter, and we run it as a baseline with the same perturbation ranges used in our sweeps. The pointing-error analogy predicts that it addresses the jitter term but handles a large systematic offset poorly. A model made invariant to big displacements must blur its comparison, which is the same trade-off as widening a beam. We test this by checking whether augmentation's gains concentrate on jitter sweeps and vanish on large-offset sweeps and real AP/PA mismatches. If augmentation closes the gap everywhere, the method claim fails.
Why not just use learned deformable attention or global attention and let the network learn where to look? Global attention is BioViL-T's choice, and CoCa-CXR's gain over it already suggests that unconstrained reach admits noise. Deformable attention, which learns per-patch sampling offsets, is a fair stronger alternative, and we include it at matched parameter count. Our expected advantage is sample efficiency and calibration. The offset is learned from free self-supervision rather than from a few thousand progression labels, and the window is set by an explicit capture-probability target rather than by whatever minimizes training loss. Few-shot comparisons in TempA-VLP's protocol will show whether this matters.
Why not just match anatomy explicitly, as in difference graphs? The node-matching approach of Hu et al. avoids geometry but needs anatomical segmentation, and DiPro similarly relies on pre-annotated boxes. We treat it as a complementary upper bound where annotations exist, not as a competitor under our annotation-free setting.
Proposed Experiments
All training uses MIMIC-CXR image pairs. Evaluation uses the MS-CXR-T progression set of 1,326 pairs, covering five conditions and three classes (improving, stable, worsening). We report accuracy and macro-F1, stratified by matched versus mismatched view position. We also report the false-change rate on stable pairs and degradation curves under synthetic offset-by-jitter sweeps. For report generation we report CheXbert and RadGraph F1 on the MIMIC-CXR test split, to check that gains do not come at the cost of report factuality.
Three outcomes are decisive. The first is whether capture probability explains each model's degradation curve better than displacement magnitude, measured by held-out $R^2$ across sweep settings. If it does not, the pointing-error model is the wrong lens. The second is whether pointing-aware regional attention beats registration-plus-fixed-window and shift augmentation on real mismatched-view pairs without losing accuracy on matched ones. The third is whether the conspicuity-stratified pattern predicted from Jung et al. appears: offset should dominate errors on subtle changes, and jitter should set a floor on conspicuous ones that does not move as the amount of training data is varied.
A negative result on the first outcome with a positive result on the second would still be a useful method paper. A negative result on both means misalignment is not the bottleneck that the open-problem literature assumes, and that too is worth reporting.
References
- CoCa-CXR: Contrastive Captioners Learn Strong Temporal Structures for Chest X-Ray Vision-Language Understanding
- On the Performance Analysis of Hybrid ARQ With Incremental Redundancy and With Code Combining Over Free-Space Optical Channels With Pointing Errors
- Unified statistical performance of FSO link due to the combined effect of weak turbulence and generalized pointing error with HD and IM/DD
- Learning to Exploit Temporal Structure for Biomedical Vision-Language Processing
- Ergodic Capacity Analysis of Free-Space Optical Links With Nonzero Boresight Pointing Errors
- Optical Intelligent Reflecting Surfaces Empowering Non-Terrestrial Communications
- Deep learning for chest radiograph diagnosis: A retrospective comparison of the CheXNeXt algorithm to practicing radiologists
- DeltaNet: Conditional Medical Report Generation for COVID-19 Diagnosis
- TempA-VLP: Temporal-Aware Vision-Language Pretraining for Longitudinal Exploration in Chest X-Ray Image
- Multimodal Disease Progression Modeling via Spatiotemporal Disentanglement and Multiscale Alignment
- Expert Knowledge-Aware Image Difference Graph Representation Learning for Difference-Aware Medical Visual Question Answering
- MIMIC-CXR: A large publicly available database of labeled chest radiographs