When a radiologist reads a chest X-ray, their eyes jump from region to region. Turning those jumps into a graph gives a "scanpath network": nodes are anatomical regions, and a directed edge records each eye movement from one region to another. Prior work shows that the shape of this graph tracks expertise and confusion. Readers whose transitions are more orderly make fewer errors, and rapid back-and-forth darting between two spots signals trouble. The errors that matter most clinically are rare, though. A look-but-don't-report miss, where the reader fixates an abnormality and leaves it out of the report, occurs in a small fraction of reads. Every scanpath-network study in our file evaluates roughly balanced classes. Rare-event research shows that detectors trained the ordinary way collapse to "always normal" once positives fall below about 1%.
We propose to model what a normal read looks like for this image and this reader, and to flag reads whose scanpath network departs from that norm. This has two parts. The first is a normative model trained only on ordinary reads. It scores each read by how far its network statistics deviate from what the image's difficulty and the reader's habits predict, and it discards the bulk of reads. The second is a cost-sensitive classifier that runs only on the surviving candidates and decides whether a miss occurred. The claim is that difficulty- and reader-normalized topology, filtered anomaly-first, detects natural perceptual misses at realistic prevalence better than heatmap gaze systems and single-stage graph networks. We also claim it adds information beyond simply checking whether an image model's findings disagree with the report. The last part of that claim is the one most likely to fail, and the experiments are built to test it directly.
Core Claim and What Is New
Two findings motivate the idea. More Expert-like Eye Gaze Movement Patterns are Related to Better X-ray Reading built scanpath networks for 107 dental students over five semesters. It found that average PageRank predicts diagnostic success, while transition entropy and reciprocity predict failure. The same paper notes that these metrics shift with stimulus difficulty, and that nobody corrects for difficulty. From eye movements to scanpath networks and Déjà Vu? Decoding Repeated Reading from Eye Movements show the same sensitivity in reading: density, efficiency and hub structure all move with skill and familiarity. A raw entropy value is therefore ambiguous. High entropy might mean a confused reader, or it might mean an image that is genuinely hard. Under extreme imbalance, that ambiguity is fatal, because the rare positives sit inside a much larger cloud of hard-but-correct reads.
The method comes from DC-Prophet and Deep Cost-sensitive Learning for Wheat Frost Detection. DC-Prophet filters datacenter events with a one-class model trained only on normal data, and in doing so missed only 11 of nearly 9,000 failures before a second-stage classifier ran. The wheat-frost paper uses a loss that weights positives by inverse frequency, plus a dynamic term that tracks misclassified positives during training; this lifted recall from 16.7% to 77.8% at 16:1 imbalance. We reuse both ideas. The new ingredient is the normalization: what the first-stage filter sees is the residual of the scanpath network relative to an image- and reader-conditioned expectation. For a network statistic $m(G)$, such as transition entropy, reciprocity or PageRank concentration on a read's graph $G$, the filter sees
$$r = \frac{m(G) - \hat{\mu}(x, u)}{\hat{\sigma}(x, u)},$$
where $x$ is the image, $u$ is the reader, and $\hat{\mu}$, $\hat{\sigma}$ are the expected value and spread. These are predicted from image features and the reader's other reads, fit only on reads without errors.
Three pieces of closest work, read together, mark the gap. CoRaX catches perceptual misses by taking the set difference between gazed regions and reported findings. It treats gaze as a video stream rather than a network and is tested at a fixed 28% injected error rate. I-AI reduces gaze to an intensity mask that steers an image classifier. C-CAD does build a gaze graph, but only to sparsify it into regions of interest, and it trains on artificially balanced data. None of the three uses network topology as the detection signal, and none faces natural low prevalence.
Research Program
Each REFLACX read becomes a directed, weighted graph. Nodes are about a dozen anatomical regions (lung zones, heart, mediastinum, costophrenic angles), obtained from a lung and heart segmentation that is anchored to REFLACX's heart and lung boxes. We add candidate-abnormality nodes proposed by a pretrained chest X-ray detector. Edge weights count saccades between regions. Fixations are aligned to report sentences using the roughly 2.5 s look-ahead reported in REFLACX, so each node also carries a "mentioned or not" flag. From each graph we compute the classical metrics, which keeps the results interpretable. We also learn an edge-pooling graph-network embedding in the style of FixGraphPool.
Labels for natural misses are built without injecting errors. A read is positive when the reader fixated inside an abnormality region but did not report that abnormality. The abnormality must be established by the reference ellipses, or by agreement between another REFLACX reader and the MIMIC-CXR report. The several hundred images read by more than one radiologist supply most of these labels. Test sets are then subsampled to prevalences of 5%, 1% and 0.5%. At 1% the test sets contain only a few dozen positives, so every result is reported with bootstrap intervals over repeated subsamples. The prevalence curve is reported, not a single point.
Stage one fits the normalizer and a one-class scorer (a one-class SVM, or a deep one-class graph model) on clean reads. Its threshold is set to keep recall of positives at or above 95% on validation, because, as DC-Prophet warns, a miss discarded here cannot be recovered later. Stage two is a cost-sensitive classifier on the survivors. It sees the residual metrics, the embedding, and region-level features: whether there was a reciprocal loop between a candidate node and a reference region, and whether the candidate node is mentioned in the report.
Before touching radiology, we run a controlled test of how scanpath models behave as positives become rare. The sarcasm gaze set D1 from Harnessing Cognitive Features for Sarcasm Detection and repeated-reading trials from OneStop are subsampled until positives fall to 1%. This measures how fast single-stage gaze-graph classifiers degrade, and whether normalization and the two-stage design restore them, on data whose labels are not in doubt.
Baselines
The naive reference is to flag nothing, or to flag reads at random. Its area under the precision-recall curve equals the prevalence, which anchors every other number. A second naive reference ranks reads by total reading time, which tests whether "slow read" is all the signal there is.
The strongest existing gaze-based method is CoRaX's gaze-versus-report set difference CoRaX. It is the only published system targeting this exact error type on REFLACX. CoRaX reports an overall perceptual error correction rate of 21% (71 of 332) on its simulated set. We will rerun it on our natural-miss splits with the same report labeler and the same regions, and also run our method on its simulated set, so that both numbers are comparable. Beating it means higher recall at a matched referral rate (flagging 5% of reads) on natural misses, with non-overlapping bootstrap intervals.
The strongest gaze-free competitor is image–report disagreement. A chest X-ray classifier's high-confidence findings are compared with CheXpert-style labels extracted from the transcript, and any finding the model sees but the report omits is flagged. This is the baseline most likely to make gaze unnecessary.
The strongest learned gaze competitor is a FixGraphPool-style graph network trained end to end on the raw graphs with the dynamic cost-sensitive loss. It reaches 83.0% accuracy and 81.0% F1 on balanced situational-awareness data in Will You Be Aware?. For imbalanced scanpath settings we found no reported result on Lacuna.
Three ablations isolate the key ingredients: removing the normalization (raw metrics in place of residuals), removing stage one (same features, single stage), and swapping the graph features for dwell-and-heatmap features with an otherwise identical pipeline. All methods share data splits, the image detector, and an equal hyperparameter search budget, tuned only on validation folds held out by reader.
Why not just…?
Why not just flag findings the image model sees but the report omits? This is a real threat. It needs no eye tracker, and for conspicuous findings it may match or beat us. CoRaX itself reaches 100% correction on cardiomegaly, which suggests that obvious findings are easy for any method. Our hypothesis is narrower. Gaze topology helps most on subtle findings where the image model is unconfident, and it helps cut false referrals in cases where the reader clearly inspected and dismissed a region. We test this by stratifying results by image-model confidence and by evaluating the combination against disagreement alone. If gaze adds nothing beyond the disagreement baseline in any stratum, we report that the networks are redundant for this task.
Why not just use dwell time or gaze heatmaps, as I-AI and CoRaX do? Prior work in balanced settings favors relational features: in the sarcasm study, 16 of the top 20 predictive features were relational, and repeated-reading models with network measures beat dwell baselines. Balanced results may not carry over to rare events, however, so the heatmap ablation runs inside our own pipeline at each prevalence level.
Why not just train one cost-sensitive graph network and skip the filter? The loss reweighting from the wheat-frost work is cheap and might suffice at 1%. The single-stage ablation measures exactly this. DC-Prophet's 39.45% F3 gain over one-stage classifiers suggests filtering matters at extreme skew, but that gain came from tabular traces, so we do not assume it transfers.
Why not use a sequence model on raw gaze coordinates? PatchTSMixer reached 74.5% against FixGraphPool's 83.0% on situational awareness. Even so, we include a transformer over fixation sequences, trained with the same cost-sensitive loss.
Decisive Checks
The idea succeeds if three things hold at 1% prevalence on natural misses. First, the normalized two-stage model beats CoRaX and the single-stage graph network in precision-recall area and in recall at a 5% referral rate. Second, removing normalization costs more than removing the graph network, which would show that difficulty and reader correction is the active ingredient. Third, gaze features add measurable recall over image–report disagreement on subtle, low-confidence findings. The primary metrics are F3, following DC-Prophet's recall-weighted choice, and the stage-one false-negative rate. If the third check fails, the honest conclusion is that scanpath networks describe reader behavior well but do not earn their hardware cost for catching rare misses.
References
- More Expert-Like Eye Gaze Movement Patterns are Related to Better X-ray Reading
- From eye movements to scanpath networks: A method for studying individual differences in expository text reading
- Déjà Vu? Decoding Repeated Reading from Eye Movements
- DC-Prophet: Predicting Catastrophic Machine Failures in DataCenters
- Deep Cost-sensitive Learning for Wheat Frost Detection
- Enhancing Radiological Diagnosis: A Collaborative Approach Integrating AI and Human Expertise for Visual Miss Correction
- I-AI: A Controllable & Interpretable AI System for Decoding Radiologists' Intense Focus for Accurate CXR Diagnoses
- A collaborative computer aided diagnosis (C-CAD) system with eye-tracking, sparse attentional model, and deep learning
- REFLACX, a dataset of reports and eye-tracking data for localization of abnormalities in chest x-rays
- Will you be Aware? Eye Tracking-Based Modeling of Situational Awareness in Augmented Reality
- Harnessing Cognitive Features for Sarcasm Detection