Who Gets Unmasked Last: Rare Findings in Parallel Radiology Report Decoders

Diffusion-style text generators write a report in parallel rather than one word at a time. They start from a sequence of blank "mask" slots and, over a few dozen rounds, fill in the slots they are most confident about first. This makes them fast, and in chest X-ray reporting it has also made them accurate on average. We argue that the "most confident first" rule has a hidden cost in medicine. Confidence tracks frequency. Normal boilerplate and common findings get committed early, and those committed words then shape every later round. A rare, dangerous finding such as free air under the diaphragm is therefore filled in last, if it is filled in at all. Squeezing the decoder into fewer rounds, which is exactly what buys speed, should make this worse. Aggregate clinical scores will not show it, because rare findings barely move averages.

The proposal is to measure this effect and then fix it. We first test whether the speed-versus-accuracy tradeoff of parallel report generators is paid disproportionately by rare findings. To do so we evaluate a diffusion report generator with clinical metrics broken down by how common each finding is, across a range of decoding-step budgets. We then test a cheap repair borrowed from long-tailed image classification: correcting the decoder's fill-in order for the frequency of each finding, combined with a frequency-rebalanced image classifier that tells the decoder which findings to expect. If the effect is real, a parallel report generator could run at its fastest settings without silently dropping the findings that matter most.

Core Claim

The closest existing system is DRRG. It replaces left-to-right report writing with a discrete diffusion model, meaning a generator that learns to fill masked tokens over several rounds, built on a 0.6B-parameter decoder. It raises the masking rate of clinically important words during training, and it conditions generation on 14 common findings predicted from the image. It reports RadGraph-F1 of 0.281 and CheXpert-F1 of 0.549 on MIMIC-CXR, and runs nearly twice as fast as an autoregressive baseline at 32 steps.

DRRG leaves two things untouched. Its conditioning covers only the 14 frequent CheXpert findings. Its evaluation reports only averages, so nobody knows how rare findings fare as the step budget shrinks. Our claim has two parts. The first is that under confidence-ordered unmasking, recall of tail findings falls with fewer decoding steps faster than recall of head findings does. The second is that this gap is caused partly by the decoding order itself, separately from the visual evidence, so it can be reduced at decode time.

The second part matters because a skeptic will say the fix is simply a better-balanced image classifier. We take that seriously: it is one arm of our design, and the experiment is built to tell the two causes apart.

Research Program

The work has three pieces.

A frequency-stratified clinical evaluation. We take the 19 finding classes of MIMIC-CXR-LT from Holste et al., split into head, medium and tail by training count. We label generated and reference reports with SRR-BERT from Delbrouck et al., a 55-finding report classifier that reaches 0.80 F1 against the CheXpert labeler's 0.61 on mapped labels. From these labels we compute per-finding recall, precision and F1 within each frequency group. We also report RadGraph entity-level F1, from Jain et al., restricted to observation entities that map to tail findings.

Labelers are least reliable exactly on rare phrasing, so we add an audit. A radiologist hand-checks the labels on a small set of tail-positive studies. The tail numbers are reported with and without the audited correction.

A commit-time diagnostic. During decoding we log the round at which each finding-bearing token is committed. We also log whether a committed tail finding is later overwritten. DRRG showed that this kind of self-correction happens: a false positive at step 48 was revised by step 72. We ask whether the same revision process works against rare true positives.

The repair. It has two parts, each borrowing an idea from long-tailed classification.

The first part rebalances what the decoder is told. Holste et al. found that end-to-end training on long-tailed chest X-rays gives a good image feature extractor but a final classifier biased toward common classes. Retraining only that classifier with class-balanced sampling (they call this classifier re-training, cRT) raised NIH-CXR-LT tail accuracy from 1.7% to 30.0%. We apply cRT to a multi-label concept head over the 19 long-tail findings, replacing DRRG's 14-finding head.

The second part rebalances when tokens are committed. Let $p_\theta(v \mid x, y_t)$ be the decoder's probability for token $v$ at a masked position, given image $x$ and partial report $y_t$ at round $t$. Let $\pi(v)$ be the training-corpus frequency of the finding that $v$ names, with $\pi(v)=1$ for tokens that name no finding. Standard decoding commits the positions with the highest $\max_v \log p_\theta$. We instead rank positions by

$$s_i = \max_v \left[ \log p_\theta(v \mid x, y_t) - \tau \log \pi(v) \right],$$

where $\tau \ge 0$ is tuned on validation macro-F1. This is the logit-adjustment form of prior correction, applied to the commit order. Setting $\tau=0$ recovers standard decoding.

We separate two uses of the corrected score. In the order-only variant, the corrected score decides which positions are filled, but the token placed there is still the one the uncorrected distribution prefers. In the identity variant, the corrected score also chooses the token. This separation tells us whether the problem is scheduling or token choice.

Baselines

Do-nothing reference. We reproduce the DRRG design as faithfully as the paper allows: a Qwen3-0.6B masked diffusion decoder, the RadGraph-weighted masking, the 14-finding concept head, fixed 100-token output, and confidence-ordered unmasking. Its published number is RadGraph-F1 0.281 and CheXpert-F1 0.549 on MIMIC-CXR (DRRG). Our reimplementation must land near 0.281 RadGraph-F1 before any comparison counts.

No reported result found on Lacuna for tail-stratified finding recall of any report generator on MIMIC-CXR. This baseline's tail numbers are therefore the first thing we measure.

Strongest competitor: the same decoder with a cRT-rebalanced 19-finding concept head. This is what a reviewer would build first, and it may be sufficient. It uses identical training data, the same image backbone, and the same number of fine-tuning steps. Its classifier-retraining stage gets the same validation tuning budget as our $\tau$ search. Beating it means higher tail-group macro F1 at matched step budgets of 16, 32 and 64. The tail precision of tail findings must not drop by more than the gain in recall, and head-group F1 and aggregate RadGraph-F1 must not fall meaningfully.

Autoregressive reference. An autoregressive decoder of the same size and initialization, given the same cRT concept conditioning, decoded greedily. We measure wall-clock latency on the same GPU at batch size 1 and at a clinical batch size. Speculative decoding preserves the autoregressive output distribution exactly, so this model's tail metrics also stand in for a sped-up autoregressive system. Only its latency would change.

Retrieval floor. Image-embedding nearest-neighbor retrieval, following Boag et al., who report macro F1 of 0.258 on MIMIC-CXR. It is the zero-generation-latency option.

Ablations. The order-only and identity variants of the corrected score, each with and without the cRT head. Together these form a two-by-two-by-two design that isolates the scheduling ingredient.

Why not just…?

Why not just rebalance the concept classifier and leave decoding alone? This is the real threat. If cRT conditioning alone flattens the tail-recall-versus-steps curve, the scheduling claim fails and we will report that. Our reason to expect a residual gap is that conditioning only shifts the decoder's probabilities. Under confidence ordering, a tail finding that rises from near zero to moderate probability is still outranked by common boilerplate in early rounds, and the boilerplate it then sits beside discourages it later. The factorial design settles the question. If order correction adds tail recall on top of cRT at 16 and 32 steps but not at 64, the order effect is real and specific to aggressive parallel decoding.

Why not just use an autoregressive model with speculative decoding for speed? Speculative decoding inherits the autoregressive model's tail behavior unchanged, so the question becomes whether left-to-right decoding handles tails better. That is an empirical question we answer directly with the matched autoregressive reference. Replace and Report documents that autoregressive generators also become "lazy," over-producing normal sentences and dropping rare abnormal ones. So autoregressive decoding is not a free fix. Parallel decoding has the advantage that its fill-in order can be changed at decode time without retraining.

Why not just apply standard logit adjustment to the output layer? That is our identity variant. It changes which word is chosen, and as a side effect it changes confidence. It is likely to raise false-positive tail findings, because it pushes rare words everywhere. The order-only variant only reprioritizes positions whose top choice is already a finding. Comparing the two on tail precision and on Pragmatic LLaMA-style hallucination counts shows whether scheduling is the safer lever.

Why not just retrieve a past report? Retrieval has no decoding latency. However, Yu et al. show that retrieval and generation both sit close to a random-retrieval floor on RadCliQ, a gap of only 5.5%. And by construction, retrieval cannot assemble rare findings in combinations the database lacks. We include it so that the tail numbers have a floor.

Proposed Experiments

We train on MIMIC-CXR (Johnson et al.) using the official splits, with MIMIC-CXR-LT labels for the concept head. Baselines are implemented in ViLMedic where possible.

The first decisive check is the diagnostic. For the do-nothing model, we plot per-group finding F1 against step budgets of 8, 16, 32, 64 and 100. We also plot the distribution of commit rounds for head versus tail finding tokens. The claim's first part holds only if the tail slope is clearly steeper than the head slope, with confidence intervals bootstrapped over studies, and tail findings are committed systematically later. If the slopes match, the premise is false, and the project reduces to a negative result about parallel decoders, which is still worth publishing.

The second check is the factorial. We need order-only correction to improve tail F1 over the cRT-only competitor at 16 and 32 steps without lowering tail precision or head F1. We report wall-clock latency alongside every comparison, because the correction adds negligible compute per round.

The third check tests transfer. We run the tuned configuration, without retuning $\tau$, on the out-of-distribution HOPPR test set used by Delbrouck et al. This shows whether the gain is a property of decoding or an artifact of MIMIC-specific frequencies.

Throughout, we report aggregate RadGraph-F1 and CheXpert-F1 next to the stratified numbers. This makes visible how much of each method's behavior on rare findings the standard averages conceal.