Topological Agreement Weights for Ensembling Metal Artifact Reduction Networks

Metal implants in a CT scan block X-rays and leave bright and dark streaks across the image. Many neural networks now remove these streaks. On simulated data they differ by only a decibel or two of image quality, but on real patients each one fails in its own way. Some leave streaks behind; others invent bone that is not there. A hospital cannot tell which network to trust on a given scan, because the clean image is never available. We propose to combine several such networks, ranging from very small to very large, and to weight each network's output per scan using two label-free signals. The first is the usual physics check: does the output agree with the X-ray measurements that the metal did not corrupt? The second is a structural check aimed at the region where the physics check says nothing. Inside the "metal shadow," do the networks agree on how bone pieces connect as the intensity threshold is lowered? And does that connectivity look like real anatomy or like streak debris?

The second check draws on topological data analysis. Two objects matter here. A merge tree records how bright regions appear and join as a threshold sweeps from high to low intensity. A barcode is the lossier summary that keeps only when each region appears and when it disappears by merging. The tree realization number counts how many distinct merge trees share a single barcode. It has a simple closed form and a known null distribution. We use it as a calibrated measure of how "scrambled" a bone region's topology is. We also use it to measure how much a barcode-only structural score throws away. The broader goal is empirical: to find out whether structural agreement adds reliability information beyond projection consistency, and whether the resulting weights shift systematically toward low-capacity models under domain shift, as the complexity trade-off literature predicts.

Core Claim

The measurements corrupted by metal, called the metal trace in the sinogram (the raw stack of X-ray projections), are exactly the ones a reconstruction must invent. So a data-consistency residual computed on the uncorrupted projections cannot detect errors that live mostly in the metal shadow. Hallucinated bone bridges and severed thin structures are errors of this kind. We claim that a structural score computed only in the shadow region fills this blind spot. The score combines two parts: cross-model agreement between bone merge trees, and deviation of a null-normalized tree realization number from its range on clean anatomy. Fusing the constituent networks with weights built from both signals should beat the best single network, uniform averaging, and physics-only weighting on real clinical scans. The gain should grow with artifact severity. The learned weights should also favor compact, physics-structured models such as ACDNet over large dual-domain models when the test data move away from the training distribution.

What the tree realization number can and cannot do

The research file claims that the tree realization number captures connectivity that barcodes miss. That claim is wrong as stated, and the design has to respect this. From trees to barcodes and back again II proves that for a barcode $B$ with $n$ finite bars, $R(B)=\prod_{j=1}^n \mu(I_j)$, where $\mu(I_j)$ is the number of bars containing bar $I_j$. Because $R$ is a function of the barcode alone, it cannot distinguish two images that share a barcode. What it does measure is how many merge trees a barcode leaves undetermined, together with how nested the bars are. The same paper shows that $R$ increases along the weak Bruhat order, a partial order on permutations under which a permutation grows as more of its pairs are put out of order. In plain terms, more interleaved bar lifespans mean a higher $R$. The paper also gives the uniform-null expectation $E_n[\log R]=\sum_{i=1}^n \log(i!)/i$, and reports that real neuronal trees depart sharply from that null. Neither result has been applied to images.

This gives the realization number two legitimate roles, and we claim only these two. The first is a size-calibrated "scrambledness" statistic,

$$s(B)=\frac{\log R(B)}{E_n[\log R]},$$

where $n$ is the number of finite bars in the bone-region superlevel-set barcode. We use it to compare regions with different numbers of components. Streak debris should produce nested, interleaved fragments that push $s$ outside the range seen on clean bone. Whether it actually does is an empirical question, and we test it against plain bar count and total persistence. The second role is diagnostic. When $\log R$ is large, a barcode-distance score such as Wasserstein distance between barcodes is known to be ambiguous about connectivity. On those images we compare merge trees directly rather than barcodes. The actual structural agreement term is therefore computed on merge trees. The realization number decides where barcode-level comparison is enough and supplies a calibrated prior term.

Research Program

The ensemble spans three complexity tiers, all trained on the same synthesized DeepLesion metal split. The zero-parameter reference is linear interpolation of the metal trace. The compact physics-structured tier contains ACDNet, which models streaks with a weighted convolutional dictionary and uses 1.6M parameters, and InDuDoNet, which unrolls proximal gradient steps across the sinogram and image domains in about 5M parameters. We also train InDuDoNet with several stage counts, which gives a controlled complexity axis inside one architecture family. The high-capacity tier contains DuDoNet at 25.8M parameters, which joins the two domains with a differentiable back-projection layer. We add ADN as an unsupervised member. It separates anatomy from artifact in latent space, so its errors should be less correlated with those of the supervised members.

For each scan and each member $m$ we compute two scores. The first, $r_m$, is the normalized residual between the forward projection of the output and the measured projections outside the metal trace. The second, $t_m$, is a structural score over the metal-shadow region, meaning the image pixels crossed by metal-trace rays. We threshold a bone intensity window and build the superlevel-set merge tree. $t_m$ then combines two terms: the merge-tree distance from member $m$ to the consensus of the other members, and the deviation of $s(B_m)$ from its distribution on clean DeepLesion bone regions. Weights take the bandwidth-controlled form used by Ensemble Super-Resolution with A Reference Dataset, which fuses super-resolvers with instance-specific least-squares weights under a known degradation operator and uses a bandwidth $\rho$ to move between winner-takes-all and plain averaging:

$$w_m=\frac{\exp\left(-\rho\,(\alpha r_m+\beta t_m)\right)}{\sum_{k}\exp\left(-\rho\,(\alpha r_k+\beta t_k)\right)}.$$

Here $\alpha,\beta$ balance the physics and structural terms, and $\rho$ sets how sharply the weights concentrate. These three scalars are the only tuned parameters, and they are fit once on a synthetic development split. Our weights differ from that paper's in two ways: they need no clean reference image at test time, and they add a structural term where the physics term is uninformative. We fuse in the image domain outside the shadow and use the weighted combination inside it. A variant restricts the weights to winner-takes-all, which avoids blurring disagreeing bone edges.

The analysis track asks whether the learned weights reproduce the complexity crossover pattern. ACDNet reports stronger cross-site transfer from torso to dental CT at a fraction of DuDoNet's size. The ultrasound study in Temporal Convolution Networks for Real-Time Abdominal Fetal Aorta Analysis with Ultrasound found that a shallow feature extractor beat InceptionV4 on 25 patient sequences because the deeper model memorized patient-specific noise. These findings motivate, but do not establish, the prediction that per-scan weight mass shifts toward compact members as artifact severity and domain distance grow. If the shift does not appear, that is informative too.

Baselines

The naive reference point is linear interpolation. DuDoNet reports 25.47 dB PSNR for it on synthesized DeepLesion. It anchors how much any learned method contributes, and it is also the member that cannot hallucinate structure.

The strongest single model is InDuDoNet. It reports 41.48 dB PSNR on synthesized DeepLesion and a pelvic bone segmentation Dice of 0.9490 on CLINIC-metal, ahead of ACDNet's reported 40.68 dB. We retrain it with the same data split and tuning budget as every other member. Beating it means the fused output exceeds its PSNR on DeepLesion by more than seed-to-seed variation. More importantly, it means higher bone Dice and fewer topological errors on CLINIC-metal and on the high-severity head-neck scans. A DeepLesion gain alone would not count, because the proposal's case rests on shifted data.

The ensemble baselines are uniform averaging and pixelwise median of the same members. They test whether weighting matters at all. A Minimum Error Weighting Combination Strategy for Chinese Semantic Role Labeling found optimized weights beat equal weights by 1.12 F1 points (80.45 versus 79.33 on the Chinese Proposition Bank). That is a modest margin, and we expect a similarly modest margin to be possible here.

The reference-set weighting baseline follows Ensemble Super-Resolution with A Reference Dataset. Each member gets a fixed prior weight from its validated error on synthetic data. This is the supervised competitor whose weakness under distribution shift the file notes. The unsupervised label-agreement baseline adapts the spectral meta-learner of Estimating the accuracies of multiple classifiers without labeled data. That method recovers each binary classifier's accuracy from the covariance of their predictions. We apply it to per-pixel bone/not-bone masks from each member. Its conditional independence assumption is likely violated by the shared unrolled structure of the compact members, and measuring how badly is itself useful.

The key ablation is physics-only weighting, which sets $\beta=0$. It isolates the structural ingredient. Further ablations replace $s(B)$ with bar count or total persistence, and replace merge-tree agreement with barcode Wasserstein agreement. These isolate the realization number and the merge-tree step respectively. An oracle that picks the best member per scan using ground truth gives the upper bound on synthetic data. No reported result was found on Lacuna for any ensemble of metal artifact reduction networks, so every ensemble number will be new.

Why not just…?

Why not just use the best single network, InDuDoNet? On synthetic data this may be nearly unbeatable, since the members are trained on the same distribution and their errors will be correlated. The proposal's case rests on real scans. The file itself records that supervised networks trained on simulated DeepLesion corruption hallucinate anatomy on clinical scans, and ADN was built because of that failure. The CLINIC-metal and head-neck experiments decide the question. If fusion does not beat InDuDoNet's 0.9490 pelvic Dice, or does not reduce topological errors on high-severity scans, the proposal fails on its central point.

Why not just weight by sinogram data consistency, which is physics-grounded and cheap? This is the most serious threat. It is the same kind of signal as the degradation constraint in Ensemble Super-Resolution with A Reference Dataset. Our argument is that the residual is blind inside the metal trace, where hallucinations live. That argument is testable. On DeepLesion, where ground truth exists, we measure how well the residual alone rank-correlates with each member's true error inside the metal shadow, and then how much adding $t_m$ raises that correlation. If the structural term adds nothing measurable, we drop it and report physics-only weighting as the method.

Why not just use bar counts or barcode distances instead of tree realization numbers and merge trees? Bar count and total persistence are simpler, and the realization number is itself a function of the barcode. It may therefore carry little extra signal. We keep $s(B)$ only if it improves prediction of per-scan error over bar count plus total persistence in a held-out regression. Likewise, we keep merge-tree agreement only if it beats barcode agreement on the high-$\log R$ scans, where barcodes are provably ambiguous. Otherwise we simplify.

Why not just train one much larger generative model? The file's evidence runs against capacity alone solving shift. ACDNet at 1.6M parameters outscores DuDoNet at 25.8M (40.68 versus 33.51 dB) and transfers better across body sites. A large model trained on the same synthetic data inherits the same synthetic-to-real gap. We do not test a diffusion model here. DuDoNet serves as the high-capacity member, and its weight on shifted data answers the question within our budget.

Proposed Experiments and Decisive Checks

The first check is calibration on synthesized DeepLesion. For every scan and member we compute $r_m$, $t_m$, and $s(B_m)$, then measure Spearman correlation with the true in-shadow error. Metrics are PSNR, SSIM, bone-mask Betti-number error, and merge-tree distance to ground truth. The key output is the incremental correlation from adding the structural term to the physics term.

The second check is fused reconstruction on CLINIC-metal. Its 14 real volumes have no clean reference, so we score them by downstream pelvic bone Dice against InDuDoNet's reported 0.9490. We also report topological error counts of the bone masks against expert annotations where available.

The third check is dental transfer on the MICCAI 2015 Head-Neck scans, stratified by artifact severity. All models are trained on torso data only. We score mandible segmentation downstream of each reconstruction. Torosdagli et al.'s heuristic pipeline, described in Robust and fully automated segmentation of mandible from CT scans, reports 91% Dice and under 1 mm Hausdorff distance and serves as a non-learned point of comparison. The medium stratum has only four scans, so we pool medium with high and report per-scan results rather than claiming significance there.

Alongside these we report per-scan weight mass by complexity tier as a function of severity and site, to test the predicted shift toward compact members.

The proposal succeeds if the structural term significantly raises the correlation with in-shadow error over physics-only scoring. It also needs fusion to beat both InDuDoNet and physics-only weighting on high-severity head-neck scans. It fails if physics-only weighting matches it everywhere. In that case the honest conclusion is that projection consistency suffices and that topology, including the realization number, adds cost without adding reliability.