nnU-Net is a medical image segmentation system that sets up its whole training pipeline automatically. It does not tune a network by hand. It first measures the dataset, recording voxel spacing (the physical size of each 3D pixel), image sizes, intensity percentiles and object sizes. These measurements are called a "fingerprint." Fixed rules then turn the fingerprint into the resampling grid, the intensity normalization, the patch size and the network depth. When hospitals train together without pooling scans, a setup called federated learning, each site computes its own part of the fingerprint and sends it to a coordinator. The coordinator then picks one shared configuration. Every existing federated version of this pipeline assumes each site reports honestly. But the configuration decides how much compute each site spends and whose scanner the shared model is tuned for. A site therefore has a reason to shade its report. One example is a site with weak GPUs reporting coarser spacing. Another is a site with unusual CT calibration nudging the clipping window toward its own scanner.
This note proposes to measure how much any single site can gain by misreporting under nnU-Net's current aggregation rules. It then proposes to replace those rules with a configuration step that removes the incentive to misreport, rather than one that pays sites to tell the truth. The design has two parts. Each configuration axis is chosen by a site-level median over one reported value per site, a rule that cannot be profitably gamed when preferences along that axis peak at one point. Separately, the compute burden each site carries is decoupled from the shared configuration, so hardware limits stop being a reason to misreport. If the measurement shows that misreporting gains are negligible, that negative result is itself useful to federated consortia now running nnU-Net.
Core Claim
Self-configuring pipelines turn a fingerprint into a configuration through pooled statistics, for example a median taken over all volumes. That pooling step is a voting rule, and it can be manipulated. A site whose own volumes sit on both sides of the global median can move the median toward its preferred value by reporting all of its mass on one side. The same holds for intensity percentiles and object sizes.
The claim has three parts. First, these manipulations produce measurable private gains, counted in local compute saved or local Dice gained, and measurable harm to other sites. Second, a coordinator rule that combines three ingredients drives the best-response gain to roughly zero at little or no cost in global accuracy:
- each site reports a single value per axis,
- the coordinator takes the median of those reports, padded with a few fixed "phantom" values chosen by the coordinator,
- each site adapts its own batch size and gradient accumulation to its hardware.
Third, this works better than the main alternative from the incentive literature, which is to price contributions.
This differs from the closest work, the federated nnDetection study The potential of federated learning for self-configuring medical object detection in heterogeneous data distributions. That study introduced a global fingerprint built from site summary statistics (means and standard deviations, the "syn" protocol). It showed this matches sending real voxel samples, but only for honest clients; we keep its summary-statistic interface and ask what happens when clients are not honest. The Kaapana deployment Towards Real-world Federated Learning in Medical Image Analysis using Kaapana runs nnU-Net across RACOON hospitals under the same cooperative assumption, which motivates the setting. It reports no numbers on Lacuna that we can compare against.
Research Program
Measurement. We first build the measurement tool. A site's utility under a configuration $c$ is
$$u_i(c) = \mathrm{Dice}_i(c) - \lambda_i \, \mathrm{GPUh}_i(c),$$
where $\mathrm{Dice}_i(c)$ is the shared model's Dice score on site $i$'s held-out data, $\mathrm{GPUh}_i(c)$ is the GPU-hours site $i$ spends on local training, and $\lambda_i \ge 0$ is the site's private cost of compute. Let $c^\ast$ be the configuration produced when all sites report truthfully, and let $c(r_i, r_{-i})$ be the configuration when site $i$ reports $r_i$ and the other sites report truthfully. The manipulation gain of site $i$ is
$$G_i = \max_{r_i} u_i(c(r_i, r_{-i})) - u_i(c^\ast).$$
The externality is the drop in other sites' Dice at site $i$'s best response.
Training a network for every candidate report is unaffordable. We exploit the fact that nnU-Net's rules map any fingerprint onto a small discrete set of configurations: target spacing snapped to a grid, a few patch-size and depth outcomes, and a handful of clipping windows. We train once per reachable configuration, cache per-site Dice and measured GPU-hours, and compute best responses as table lookups. This makes exact best-response analysis feasible over the full reachable set.
Coordinator rule. Next we design the rule. nnU-Net's configuration is coupled: patch size depends on spacing and memory. We therefore separate anchor quantities from derived ones. The anchor quantities are target spacing per axis, the lower and upper intensity clip, and the median object extent. Patch size, downsampling strides and depth stay deterministic functions of the anchors, as in nnU-Net: Self-adapting Framework for U-Net-Based Medical Image Segmentation and the anisotropy-aware strides studied in MNet: Rethinking 2D/3D Networks for Anisotropic Medical Image Segmentation. For each anchor, every site submits one value and the coordinator takes the median of those values together with the phantom values. On a single axis where each site prefers one value and likes others less the further they are, this median rule cannot be profitably gamed. We will test empirically, using the cached table, whether site preferences are in fact single-peaked and separable across anchors, rather than assuming it.
Compute decoupling. Finally we remove the hardware motive directly. The shared parts of the configuration are those that weight averaging needs to be consistent: spacing, normalization and topology. The per-site parts are batch size, gradient accumulation and number of local steps, scaled to each site's own memory. A site with little GPU memory then gains nothing by distorting spacing, because its compute is set locally.
Residual check. Some manipulation may remain, for instance sites that genuinely prefer an intensity window tuned to their scanner. For these we add a cheap consistency check: compare foreground fractions and intensity histograms seen during local training against the reported fingerprint. Following Distributionally Robust Optimal Allocation with Costly Verification, which shows that simple favored-agent mechanisms with occasional costly audits stay robustly optimal under ambiguity about agent types, we audit only a random subset of sites rather than all of them.
Baselines
The naive reference is honest centralized nnU-Net, configured on the pooled data as in nnU-Net: Breaking the Spell on Successful Medical Image Segmentation. It is an upper bound that no federated scheme should beat. On MSD, The Medical Segmentation Decathlon reports nnU-Net at 0.94 Dice on Spleen and 0.94 on Liver. The same paper reports 0.21 on pancreas tumors and 0.16 on colon cancer. The low-Dice tasks are where we expect configuration errors to matter most.
The strongest existing federated method is the syn-fingerprint federated pipeline of The potential of federated learning for self-configuring medical object detection, run with honest clients and again with best-responding clients. It reports parity with centralized training in IID splits and a 5–7% drop under heterogeneous splits, with FedProx and FedMOON improving over FedAvg. We reproduce it on its own benchmarks, Luna16 and Duke Breast MRI, before porting it to segmentation. We use the same aggregation algorithm, the same number of rounds and the same per-configuration training budget as the proposed rule. All methods read the same cached configuration table, so they differ only in how reports become configurations.
Two ablations isolate the key ingredients. The first uses the site-median rule without compute decoupling. The second uses compute decoupling with the default pooled-volume median. The proposal "beats" the baselines if two things hold. The worst-case manipulation gain $\max_i G_i$ must fall to within seed noise of zero while it is clearly positive under the default rule. And global Dice must stay within seed noise of the honest syn baseline, so that truthfulness costs no accuracy.
Why not just…?
Why not just use a robust aggregator such as a trimmed mean over reported statistics? This is the most serious threat, because the site-level median is itself a robust aggregator. Robust aggregation bounds the damage from outliers. It does not stop a site from shading its report within the plausible range, and a trimmed mean still rewards shifting one's whole report toward one side. We include trimmed mean and the pooled-volume median as baselines in the same best-response analysis. If trimmed mean also drives $G_i$ to zero, we will say so plainly. The contribution then narrows to the measurement study and the compute decoupling, which address a motive that no statistical aggregator removes.
Why not just let each site configure its own nnU-Net? Per-site configuration removes any shared configuration to fight over. However, weight averaging requires matching topology, and mixing normalizations and spacings across sites is exactly the manufacturer-shift case that The potential of federated learning for self-configuring medical object detection found most damaging. We run a personalized-configuration baseline in which each site keeps its own plan and only compatible layers are shared. We report its cross-site Dice. If it matches the shared-configuration methods, the strategic question disappears, and that would be worth knowing.
Why not just pay sites, through contracts or models of varying quality? Paid with Models: Optimal Contract Design for Collaborative Machine Learning designs incentive-compatible menus that pay sites in model accuracy. It proves that efficient participants must receive extra surplus (an "information rent") and that low contributors may be deliberately given worse models. Similarly, Incentivized Truthful Communication for Federated Bandits uses critical-value payments to make bandit clients truthful. Both price a scalar contribution such as sample count or communication cost. Neither prices a choice among configurations. Withholding model quality from a hospital is also clinically hard to defend. Our approach makes lying unprofitable without any transfer. As a comparison point, we implement a model-quality contract on the scalar quantity "volumes contributed" and report its accuracy cost.
Why not just trust the consortium, or run attested fingerprint code? Legal agreements and trusted-execution attestation verify that the official script ran. They do not verify that it ran on the full, uncurated cohort. The measurement study tells consortia how much is at stake. If $G_i$ is negligible under the default rule, the honest conclusion is that trust suffices, and we will publish that conclusion.
Proposed Experiments
Simulated sites from MSD. We split MSD tasks with strong spacing and intensity heterogeneity into 4–6 simulated sites, grouped by slice thickness and acquisition source. The tasks are Liver, Lung, Pancreas, Colon, Hepatic Vessel and Prostate, described in A large annotated medical image dataset for the development and evaluation of segmentation algorithms. Each site receives a $\lambda_i$ drawn from a range, and some sites receive deliberately small GPU memory.
Multi-site prostate and detection. We use the Kaapana multi-site prostate cohort as a real cross-site test where it is accessible. We use Luna16 split by manufacturer, with federated nnDetection, to check that our reproduction of the closest work matches its reported parity and its 5–7% heterogeneous-split drop.
Training. We use the ResEnc nnU-Net backbone recommended by nnU-Net Revisited under a fixed VRAM budget. We use three seeds per cached configuration so that differences in gain can be separated from seed noise.
Metrics. For each aggregation rule we report:
- the per-site manipulation gain $G_i$ and its worst case over sites,
- the externality on other sites,
- global and per-site Dice,
- total GPU-hours,
- for the audit variant, the fraction of sites audited.
Decisive checks. First, the default pooled rule must show $\max_i G_i$ clearly above seed noise on at least the anisotropic CT tasks. Without this, there is no problem to solve, and we report a negative result. Second, the site-median rule with compute decoupling must reduce $\max_i G_i$ to noise while keeping Dice within noise of the honest syn baseline. Third, the single-peaked, separable preference structure must hold in the cached table for most sites. Where it fails, we report which configuration axes still admit profitable lies and whether random audits close them.
References
- The potential of federated learning for self-configuring medical object detection in heterogeneous data distributions
- Abstract: Towards Real-world Federated Learning in Medical Image Analysis using Kaapana
- nnU-Net: Self-adapting Framework for U-Net-Based Medical Image Segmentation
- MNet: Rethinking 2D/3D Networks for Anisotropic Medical Image Segmentation
- Distributionally Robust Optimal Allocation with Costly Verification
- nnU-Net: Breaking the Spell on Successful Medical Image Segmentation
- The Medical Segmentation Decathlon
- Paid with Models: Optimal Contract Design for Collaborative Machine Learning
- Incentivized Truthful Communication for Federated Bandits
- A large annotated medical image dataset for the development and evaluation of segmentation algorithms
- nnU-Net Revisited: A Call for Rigorous Validation in 3D Medical Image Segmentation