When a CT scan contains a hip implant, screw, or needle, the metal blocks X-rays along certain paths. Deep networks for metal artifact reduction fill those missing measurements with learned guesses. The result looks clean, but some of the anatomy in it was measured and some was invented. Augmented reality (AR) surgical guidance shows surgeons 3D models of patient anatomy drawn directly over their field of view. Those models are built from exactly these corrected scans, and they draw every boundary with the same confidence. This proposal computes, for every point on a reconstructed anatomical boundary, whether the scanner's surviving measurements could have determined that boundary. The answer comes from the scan geometry, not from a learned model. The resulting "visibility map" can then be shown in the surgeon's headset, so that boundaries filled in by a network's prior look different from boundaries the data supports.
The key physical fact is old and simple. In tomography, an edge is recorded mainly by X-rays that pass tangent to it, grazing along the edge rather than crossing it. If the tangent rays through a boundary point were blocked by metal, no network can recover that edge from the data. Whatever edge appears there came from what the network learned in training. The reverse also holds, and it is less obvious: a soft-tissue margin sitting right next to an implant can be fully supported by data, as long as its tangent rays happened to miss the metal. So the reliability of a corrected boundary depends on the direction the boundary faces, not only on how close it is to metal. This matters most for low-contrast soft-tissue margins. There, no image cue separates a real faint edge from a hallucinated one.
Core Claim
Tangent-ray visibility predicts where metal-corrected boundaries are wrong or hallucinated, and it does so without any training. We claim it does this better than three alternatives: distance from metal, ensemble disagreement, and scan-level quality scores. We further claim it does so across very different correction methods, because it describes what information is missing rather than how a given network fills it in.
The definition is short. Take a point $x$ on a segmented boundary with unit normal $n(x)$. In parallel-beam coordinates, a ray is indexed by its angle $\theta$ and its detector offset $s$, and the ray tangent to the boundary at $x$ has angle $\theta_n$, which points along $n(x)$, and offset $x\cdot n(x)$. Let $M(\theta,s)\in\{0,1\}$ mark rays that pass through metal. Averaging over a small angular window $W$ around $\theta_n$, which absorbs uncertainty in the normal estimate, gives
$$V(x)=\frac{1}{|W|}\int_{W(\theta_n)}\big(1-M(\theta,\,x\cdot\omega_\theta)\big)\,d\theta ,$$
where $\omega_\theta$ is the unit vector at angle $\theta$. A value of $V(x)$ near 1 means the edge at $x$ was measured. A value near 0 means the edge was supplied by the correction algorithm's prior. Fan-beam data are rebinned to parallel-beam geometry first. For cone-beam CT, the tangent rays at a surface point form a plane, and $V$ becomes the unblocked fraction of the rays in that plane that were actually acquired.
Research Program
The work has three parts.
Part one: compute the map. We build $V$ as a post-processing step that works on the output of any metal artifact reduction method. Its inputs are a segmentation of that output, the metal mask, and the acquisition geometry.
Part two: test whether it predicts error. We check whether $V$ predicts boundary errors on data where the true anatomy is known, and on real clinical scans where it is not.
Part three: test whether it helps people. We render $V$ as a display channel on AR meshes, for example desaturating or dashing boundaries the scanner did not see. We then test whether readers at different expertise levels make better spatial judgments with it.
Each direction in the brief plays a different role. Metal artifacts are the physical cause of the problem. Low-contrast boundaries are the regime where the problem hurts most. Expert-level assessment supplies the evaluation, through local radiologist annotations and a reader study stratified by expertise. AR guidance is where the result is used.
Correction methods under test. We run $V$ on the outputs of six methods, spanning the families in the literature:
- linear interpolation;
- normalized metal artifact reduction (NMAR);
- DuDoNet, which links sinogram and image domains through a differentiable back-projection layer;
- InDuDoNet, which unrolls proximal-gradient iterations across both domains;
- ADN, which separates anatomical content from artifacts without paired data;
- RISE-MAR, which uses its public code and weights (Masaaki-75/rise-mar).
These methods fill missing rays in very different ways: interpolation, learned priors, and self-training on clinical data. If $V$ predicts errors for all six, its signal comes from the geometry rather than from one network's quirks.
Why existing boundary models need this. Boundary-focused segmentation methods assume that strong gradients mean real edges. This includes boundary attention for polyps (Kang et al.), contour-loss regularizers (Niu et al.), and the overlap and focal-positive losses that lifted left atrium Dice from 82.01% to 92.24% (Combating Uncertainty with Novel Losses). Near metal, that assumption fails. We keep these methods as downstream segmenters and do not replace them. $V$ tells us which of their edges the data can back up.
Datasets with known ground truth. We need both clean reference images and boundary labels. We use two sources:
- CTPelvic1K. We insert simulated implants into artifact-free pelvic scans and forward-project them, following the simulation protocol of DuDoNet and InDuDoNet. The dataset provides bone masks. We add hip and thigh muscle and neurovascular-bundle masks, drawn by two annotators on the clean scans. The muscle protocol follows Automated Segmentation of Hip and Thigh Muscles in Metal Artifact-Contaminated CT. These muscle and vessel margins provide the low-contrast soft-tissue boundaries that InDuDoNet's bone-only evaluation (Dice 0.9490) did not cover.
- DeepLesion. We place simulated metal near annotated lesions, which gives low-contrast lesion margins.
What counts as an error. A boundary point is an error if it lies more than 1 mm from the nearest true boundary. It is a hallucination if the output shows an edge where the clean image has none.
Real clinical data. For CLINIC-metal and the intraoperative spinal cone-beam CT set CL2 used by ADN, there is no ground truth. Two radiologists will instead mark boundary segments as "believable" or "not believable" on corrected images. This yields local expert labels, which fill the gap left by RISE-MAR's single Likert score per scan.
Baselines
Do nothing. The simplest reference is today's practice: show the corrected mesh with every boundary equally confident. This is the mode used by current AR systems such as the spatiotemporal orthopedic platform (Development and Pre-Clinical Analysis of Spatiotemporal-Aware AR) and IMHOTEP (Kenngott et al.). Both assume their meshes are clean. As an error predictor, this baseline is a constant, with AUROC 0.5. In the reader study, it is the control arm.
Distance to metal. This is the naive geometric predictor: each boundary point is scored by its distance to the nearest metal voxel. It is the most dangerous baseline, because it is free, easy to understand, and surely correlated with error. $V$ counts as beating it only if two things hold. First, $V$ must predict error within fixed distance bins. Second, adding $V$ to a logistic model that already contains distance must improve AUROC by a margin whose bootstrap confidence interval excludes zero.
Ensemble and test-time uncertainty. We train five segmenters per correction method, with different random seeds and test-time augmentation, and score each boundary point by the spread of predicted surface positions. The semi-supervised uncertainty approach of DUCore motivates this baseline. The segmenters get the same data and training budget as those used to extract boundaries for $V$.
Learned local error predictor. This is the strongest competitor on this question. It is a dense extension of RISE-MAR's Clinical Quality Assessor, which correlates with radiologists at a Spearman coefficient of 0.9556 but produces one number per scan. Here, a U-Net takes the corrected image and the metal mask and is trained on the simulated data to predict boundary error. It gets the same training pairs and a matched tuning budget. It could learn visibility implicitly, so it is the fair test of whether explicit physics adds anything. We also report the original scan-level quality score broadcast to every point.
Baseline numbers. For correction quality, the strongest reported result in the file is RISE-MAR at 47.88 dB PSNR out of domain on CTPelvic1K, against 46.42 dB for SemiMAR (RISE-MAR). InDuDoNet reports 41.48 dB PSNR on DeepLesion and 0.9490 Dice for pelvic bone. For predicting where boundaries are wrong, which is the actual question here, I found no reported result on Lacuna, so every error-prediction baseline will be measured fresh.
Ablations. Two ablations isolate the key ingredient. The first removes directionality: $V$ is replaced by the fraction of all rays through $x$ that hit metal, ignoring the boundary normal. The second randomizes the normals. If neither ablation loses much, the tangent-ray principle is not doing the work.
Why not just…?
Why not just use distance from the metal? This is the most serious threat, and it may partly succeed. Distance cannot tell a margin that faces toward an implant from one that faces along it. Near two implants, distance also misses the band between them, where most tangent rays cross metal. That band is the dense-streak region DuDoNet describes. The within-bin analysis and the nested-model test in Baselines settle this directly. If $V$ adds nothing once distance is known, we will say so. The display contribution would then shrink to distance-based shading, and the tangent-ray claim would be refuted.
Why not just use ensemble uncertainty from the segmenter? Ensembles detect places where segmenters disagree. Hallucinations from metal correction, however, are baked into the input image, so every segmenter trained on it agrees. RISE-MAR's report that GAN-based baselines delete structures consistently, and the PET/CT warnings that correction software creates false hotspots (Hatherly et al.), suggest these errors are systematic rather than random. We test this by measuring ensemble AUROC separately on hallucinated points. Our prediction is that ensembles fail there while $V$ does not.
Why not just train a network to predict the error map? On simulated data, the learned predictor may match or beat $V$, and we expect that it might. The question is sim-to-real transfer. Real scans contain scatter and motion that simulations miss, which is the gap RISE-MAR was built to close. $V$ needs no training, so it cannot overfit the simulation. The deciding test is agreement with radiologists' local labels on CLINIC-metal and CL2. If the learned predictor wins there too, the right design is to give it $V$ as an input channel, and we will report that combination.
Why not just run the best correction method and trust it? A PSNR above 47 dB is an average over the whole image. A few millimetres of invented muscle or capsule margin barely move that average, yet the spatiotemporal AR study shows that guidance errors of a few millimetres and degrees are exactly what matters clinically. We report boundary error, split by $V$, for every correction method. If even RISE-MAR shows much larger errors where $V$ is low than where it is high, trusting the output uniformly is not justified.
Proposed Experiments
Main analysis. For each correction method and each dataset, we compute three quantities from $V$ and from each baseline predictor:
- AUROC for boundary error;
- AUROC for hallucination;
- the calibration of error rate against visibility decile.
Across all methods, we expect error to rise sharply as $V$ falls. The steepest rise should appear for low-contrast muscle and vessel margins, and a much flatter one for cortical bone, because bone edges are strong enough for priors to reconstruct plausibly. We also check whether the relation holds equally for simulation-trained networks and for RISE-MAR's clinically self-trained model. If they differ, that tells us whether self-training moves priors toward the truth in the regions the scanner could not see.
Cone-beam test. We repeat the clinical-agreement analysis on CL2 cone-beam CT, using the planar version of $V$.
Reader study. This follows the stratified-expertise design used in vitiligo diagnosis (Design and Assessment of CNN Methods for Vitiligo Diagnosis). Residents, fellows, and attending orthopedic surgeons view corrected pelvic meshes in a headset, either with or without visibility coding. For each case, they judge whether a planned screw or cup trajectory clears a soft-tissue structure. Ground truth for these judgments comes from the clean scans. The primary outcome is the rate of confident wrong decisions in low-visibility regions. The secondary outcomes are decision time and overall accuracy, which checks that the coding does not simply make readers hesitate everywhere. The hypothesis is that juniors gain the most. This is the untested question of whether 3D guidance helps inexperienced surgeons when anatomy is degraded by artifacts.
Decisive checks. The proposal lives or dies on three results:
- The within-distance-bin effect of $V$, and its survival after the normal-randomization ablation.
- Whether $V$ matches or beats the learned error predictor on real clinical scans labeled by radiologists.
- Whether visibility coding reduces confident errors without raising abstention across the board.
If the first check fails, the physics adds nothing beyond distance. If the second fails, a learned map is the better tool. If the third fails, the map is accurate but does not help surgeons.
References
- DuDoNet: Dual Domain Network for CT Metal Artifact Reduction
- InDuDoNet: An Interpretable Dual Domain Network for CT Metal Artifact Reduction
- ADN: Artifact Disentanglement Network for Unsupervised Metal Artifact Reduction
- Radiologist-in-the-Loop Self-Training for Generalizable CT Metal Artifact Reduction
- Multi-scale information sharing and selection network with boundary attention for polyp segmentation
- A Medical Image Segmentation Method based on Multi-scale Features and Contour Loss Constrain (S)
- Combating Uncertainty with Novel Losses for Automatic Left Atrium Segmentation
- Automated Segmentation of Hip and Thigh Muscles in Metal Artifact-Contaminated CT using Convolutional Neural Network-Enhanced Normalized Metal Artifact Reduction
- Development and Pre-Clinical Analysis of Spatiotemporal-Aware Augmented Reality in Orthopedic Interventions
- IMHOTEP: cross-professional evaluation of a three-dimensional virtual reality system for interactive surgical operation planning, tumor board discussion and immersive training for complex liver surgery in a head-mounted display
- DUCore: Dual Uncertainty-Guided Consistency and Regional Contrastive Learning for Semi-supervised Medical Image Segmentation
- Design and Assessment of Convolutional Neural Network Based Methods for Vitiligo Diagnosis