Simulation-Matched Conformal Control of False Bundles in Tumor Tractography

Tractography reconstructs the brain's white-matter wiring from diffusion MRI. Diffusion MRI measures which way water moves inside tissue, and tractography chains those local directions into long curves called streamlines. The method has a well-documented weakness: it invents connections. In the ISMRM 2015 challenge, 96 pipelines recovered about 90% of true bundles but produced roughly four invalid bundles for every valid one (The challenge of mapping the human connectome based on diffusion tractography; Tractography-based connectomes are dominated by false-positive connections). Learned filters can now sort streamlines into plausible and implausible with high accuracy on healthy data. Neurosurgeons need something different. Before resecting a glioma, they want to know how many of the displayed tracts near the tumor are fake, for this patient, on this scanner. No current filter offers a number of that kind that remains valid once the brain is deformed by a tumor and scanned under a protocol unlike the training data.

We propose to attach a finite-sample guarantee to tractogram filtering. The guarantee bounds the expected number of invalid bundles that survive filtering in a new patient's tractogram. The guarantee comes from conformal risk control, a calibration procedure that holds as long as calibration and test cases are statistically interchangeable. Real tumor scans with known ground-truth wiring do not exist, so that condition cannot be met with real data. We make it achievable by building the calibration set to order. Each calibration case is a phantom whose ground-truth bundles have been deformed by a biophysical tumor-growth model and then simulated under the target scanner's acquisition protocol. The contribution is this per-site, per-pathology calibration set, together with an honest test of whether a guarantee earned on simulations survives contact with real data.

Core Claim

A conformal threshold on a streamline filter will systematically under-cover in tumor patients and on new protocols when it is calibrated on healthy, research-grade data. The threshold is also chosen on the wrong unit: streamlines within one brain are strongly dependent, so the exchangeable unit is the subject. If the unit of calibration is the whole tractogram, and the calibration tractograms are simulations whose deformation and acquisition match the deployment cell, the realized invalid-bundle count on held-out test cases will sit at or below the user's target. Healthy-calibrated thresholds will miss that target, and they will tend to over-prune valid but displaced tracts. We also claim the residual gap between simulated and real data can be detected without labels, by testing whether real test tractograms' score distributions are consistent with the calibration phantoms.

The claim builds on three lines of prior work. Verifyber (Supervised tractogram filtering using Geometric Deep Learning) supplies the base scorer. It treats each streamline as an ordered polyline, reaches 95.2% accuracy on HCP-EP, and was shown qualitatively on five glioma patients (APSS-IS), but it outputs deterministic labels with no error bound. Conformal prediction with limited false positives (Conformal Prediction Sets with Limited False Positives) supplies the target. It bounds the expected false-positive count of a selected set rather than guaranteeing coverage, but it was developed on drug screening, detection and text, and it assumes exchangeability. CONSIGN (CONSIGN: Conformal Segmentation Informed by Spatial Groupings via Decomposition) motivates the domain problem: its authors state that the guarantee breaks under cross-site shift. Our proposal does not invent a new conformal algorithm. It changes what the calibration set is, and it audits whether that change rescues validity.

Research Program

Calibration phantoms. We start from the ISMRM 2015 phantom, whose 25 ground-truth bundles are known. For each glioma in a pool of BraTS cases, we run the multi-atlas biophysical model of Multiatlas Calibration of Biophysical Brain Tumor Growth Models with Mass Effect. That model couples tumor proliferation and migration equations with tissue elasticity to estimate a displacement field from a single scan. We apply the displacement to the phantom's bundle geometry. Where the predicted tumor concentration is high, we reduce fiber density and add free-water signal, so that bundles can be displaced, thinned, or destroyed.

Matching the target scanner. We then synthesize diffusion data with Fiberfox, the simulator used for the original phantom, using the target site's b-values, gradient directions and a noise level estimated from the target site's background. The result is a library of phantoms indexed by "deployment cell": acquisition protocol crossed with tumor location (none, periventricular, temporal, frontoparietal). The tumor-location cell is read from the patient's structural MRI.

Unit of guarantee. Tractography and filtering run unchanged on each phantom. Retained streamlines are grouped into bundles by the pair of cortical regions their endpoints reach, as in the Tractometer evaluation. Each bundle receives a score equal to the mean Verifyber plausibility of its streamlines. A bundle is kept when its score exceeds a threshold $\lambda$. Let $L_i(\lambda)$ be the number of invalid bundles kept in calibration phantom $i$, and let $B$ bound that count. With $n$ phantoms in the matching cell, conformal risk control picks the smallest $\lambda$ satisfying

$$\frac{n}{n+1}\cdot\frac{1}{n}\sum_{i=1}^{n} L_i(\lambda) + \frac{B}{n+1} \le k,$$

where $k$ is the tolerated expected number of invalid bundles per patient. If the test tractogram is exchangeable with the cell's phantoms, its expected invalid-bundle count is at most $k$. We also implement the high-probability variant from Cauchois et al. Our default bundle score is the simple mean; we will test their learned set-scoring network only if the mean score proves too weak.

Shift diagnostic. For each real test tractogram, we compare its distribution of bundle scores against the cell's phantoms with a conformal rank test. A tractogram that fails is flagged as "guarantee not supported" rather than silently filtered.

Baselines

No filtering. The first reference point is the raw tractogram. Its invalid-bundle count anchors the scale, given the roughly 4:1 invalid-to-valid ratio reported by Maier-Hein et al. A close second reference is Verifyber at its default 0.5 decision threshold, with no calibration. All filtering baselines use the same tractography algorithm, the same seeding and the same trained scorer.

Strongest existing filter. This is Verifyber itself, retrained on our phantom library so that it sees tumor-deformed geometry during training. The reported reference figure is 95.2% streamline accuracy on HCP-EP, with PointNet at 94.7% and bLSTM at 93.0% (Supervised tractogram filtering using Geometric Deep Learning). On tumor data, no reported result was found on Lacuna for APSS-IS. Beating this baseline does not mean higher accuracy. It means the realized invalid-bundle count stays within $k$ across cells while valid-bundle recovery is no lower than the retrained filter's at a matched operating point.

Healthy-calibrated conformal risk control. This is the same procedure as ours, calibrated on undeformed phantoms simulated at an HCP-like protocol. It is the naive conformal reference.

Covariate-shift-weighted conformal. Here calibration cases are reweighted by a domain classifier trained to separate calibration from unlabeled target tractogram features. This alternative needs no simulation of the target.

Ablations. The ablations isolate the key ingredient by matching only the protocol (healthy anatomy) or only the tumor deformation (wrong protocol).

Every conformal variant gets the same number of calibration tractograms and the same tuning budget for any threshold-related choice.

Why not just…?

Why not just denoise the signal or smooth the tractogram? Patch2Self cuts incoherent optic-radiation streamlines by nearly half (Patch2Self: Denoising Diffusion MRI with Self-Supervised Learning​). Gaussian filtering raised IronTract accuracy by up to 80% (Insights from the IronTract challenge: optimal methods for mapping brain pathways from multi-shell diffusion MRI). Neither, however, says how many fake bundles remain. Maier-Hein et al. found over 80 invalid bundles even when tracking used perfect, noise-free orientations, so the errors originate in bottleneck geometry, not noise. We run Patch2Self upstream in every arm, so denoising is held constant rather than competing.

Why not just fine-tune the filter on tumor cases? Fine-tuning is a real improvement, and we include it as the strongest baseline and as the scorer inside our method. But a better point classifier still yields an unknown error count. Experiment one measures whether the retrained filter's realized invalid-bundle count varies across cells by more than the bound allows.

Why not just calibrate once on healthy data, or use importance weighting? Weighted conformal is the most serious threat because it is cheap. It fails where the target's support is absent from calibration: a temporal tumor collapsing the uncinate has no healthy analogue to up-weight. The ablations and the weighted baseline settle this directly. If weighting reaches the target $k$ on tumor cells, our simulation step is unnecessary, and we will report that.

Why not judge pipelines by downstream diagnostic accuracy? Zhou et al. found that nine tractography algorithms with wildly different connectomes gave overlapping AD-versus-control AUCs of 0.85–0.92 (Comparison of nine tractography algorithms for detecting abnormal structural brain networks in Alzheimer’s disease). Downstream AUC is therefore blind to false positives, which is why every metric here is bundle-level ground truth.

Proposed Experiments

Experiment one: validity in simulation, under held-out shifts. We hold out entire BraTS tumors and entire acquisition protocols from calibration, then measure the realized mean invalid bundles per tractogram against $k \in \{1, 3, 5\}$. We also measure valid-bundle recovery, and we track the retention of bundles within 2 cm of the tumor margin separately. The decisive check is that healthy-calibrated control exceeds $k$ in tumor cells while cell-matched control does not. A second requirement is that the cost in valid tumor-adjacent bundles is smaller than with a fixed-threshold filter tuned to the same average false-positive count.

Experiment two: sim-to-real transfer on protocol. IronTract supplies real tracer ground truth under two protocols (514-direction DSI and the 185-direction HCP scheme); the winning pipeline there reached a TPR of 0.96 at an FPR of 0.1 on DSI. We build phantoms matched to each protocol, calibrate on them, and check the bound on the macaque data using IronTract's tracer-defined false positives as the loss. This is ex vivo and tumor-free, so it tests only the acquisition half of the matching.

Experiment three: pathology on real patients. With only the five APSS-IS patients carrying expert labels, we treat this as an audit, not a statistical validation. We report realized invalid-bundle counts, how often the shift diagnostic flags each patient, and whether flagged patients are the ones where the bound is violated.

Kill conditions. If the shift diagnostic flags almost all real tractograms as unsupported, the simulation gap is too large and the method's value is limited to telling users when not to trust a filter. If weighted conformal matches cell-matched calibration, the simpler method wins. Either outcome is publishable and would redirect the work.