When hospitals train one segmentation model together without pooling their scans, the usual arrangement is split learning. Each hospital runs the first half of a U-Net on its own images and sends the intermediate features to a server that runs the rest. A U-Net is an encoder-decoder network whose "skip connections" copy high-resolution features from early encoder layers straight to the decoder. Those skips are what make the masks sharp, and they are also what leaks the patient's image. Split-U-Net showed that an attacker holding the shallowest skip features can reconstruct the input MRI almost perfectly, with SSIM (structural similarity, where 1.0 means identical) near 1.0. The fixes available today are all blunt: send nothing but the low-resolution bottleneck, add Gaussian noise, or apply heavy dropout. Each fix is applied uniformly, and no hospital is paid for whatever risk it does take.
This note proposes two linked contributions. The first is a cheap, image-style cipher for skip features, adapted from JPEG privacy transforms. Each hospital withholds the block-average (low-frequency) content of every skip feature map and flips the sign of each remaining block using fresh randomness per image. The server's decoder is built to ignore those signs. Boundaries survive because they live in local high-frequency magnitude; the anatomy an inversion attack needs does not. The second contribution treats disclosure at each skip scale as a good to be priced. The server posts prices, each hospital chooses how much of each skip to reveal given its own privacy cost, and iterative price adjustment in the style of electricity-market bidding finds which scales are worth paying for. The central claim is testable. The cipher should give a better trade-off between Dice score and inversion SSIM than noise, dropout, or deeper splits. On top of it, a price-adjustment protocol should reach near-optimal total welfare without anyone revealing their private cost of leakage.
Core Claim
Skip connections carry two different things mixed together. One is local edge and texture structure, which is what the decoder uses to place boundaries. The other is low-frequency intensity layout, which is what makes an inverted image recognizable. JPEG privacy work separates exactly these. Content and Privacy Protection in JPEG Images by Reversible Visual Transformation hides a region by overwriting the DC coefficients of $8\times 8$ blocks (each block's average) and flipping the signs of the remaining AC coefficients with a key-derived sequence. The result is unintelligible to a viewer but exactly reversible for the key holder. We carry that idea from pixels to U-Net features, with one important change: the randomness is fresh per image and never shared with the server.
For a skip feature map $F$ at scale $s$, partitioned into $b\times b$ blocks $B$, the client transmits
$$\tilde F_B = \epsilon_B \,(F_B - \mu_B),$$
where $\mu_B$ is the per-channel mean of block $B$ and $\epsilon_B\in\{-1,+1\}$ is a fresh random sign drawn per block and per image. The client keeps $\mu_B$ and the seed, so it alone can reverse the transform.
The server's first convolution at each skip input is made invariant to the sign by construction. It combines filter responses through an even function, for example summing $|w * \tilde F_B|$ over the filters $w$, so it computes on local energy rather than sign. Because the signs are never shared, the server holds nothing that would let it undo them. This is the property that separates our design from keyed learnable encryption, where the model must know or learn the key.
Whether edge-energy maps still let an inversion network hallucinate anatomy is an empirical question, and the first experiment exists to answer it.
Why a Market
Different hospitals, and different imaging modalities, value privacy differently: regulation, consent terms, and scanner protocols all vary. The server's gain from each skip scale is also unknown in advance. This is the setting that game-theoretic price discovery was built for. In it, agents hold private costs, a coordinator needs a globally efficient allocation, and asking agents to report their costs invites misreporting.
Iterative Bidding in Electricity Markets supplies the method. In its Bid Adjustment Algorithm, generators update price bids myopically without revealing their cost functions, and the process converges at a linear rate to an efficient equilibrium. An Energy Sharing Mechanism Considering Network Constraints and Market Power Limitation supplies a constraint: when a single participant sits at a bottleneck, it gains market power, so prices must be regulated. In our setting a single modality, such as FLAIR for edema, may be close to irreplaceable, which creates exactly that kind of bottleneck.
Two papers in the file are the closest game-theoretic work, and they motivate the gap rather than fill it. A Game-theoretic Framework for Privacy-preserving Federated Learning derives conditions under which a server has no incentive to attack. It is purely theoretical, relies on an abstract oracle, and contains no payments. Privacy Preserving Distributed Energy Trading clears a Stackelberg energy market under encryption, but only for scalar quantities of energy.
We price the disclosure level $d_{i,s}$ of client $i$ at skip scale $s$. The levels range from withheld, through ciphered at block sizes 16, 8, and 4, to sent in plain form. Total welfare is
$$W(d) = V(d) - \sum_i \lambda_i \, L_i(d_i),$$
where the terms are defined as follows. $V(d)$ is the server's value of the configuration, meaning validation Dice gain times a declared value per Dice point. $L_i(d_i)$ is an audited leakage score for client $i$, the SSIM of the best inversion attack on its disclosed features. $\lambda_i$ is the client's private price of leakage.
The server posts prices $p_{i,s}$, clients best-respond, and the server raises the price of a scale when its marginal value exceeds the current price and lowers it otherwise. Leakage is measured publicly; only its cost $\lambda_i$ is private. That split is what makes the protocol auditable.
Research Program
The program has three stages.
Stage 1: the cipher. We build the transform and the sign-invariant decoder inputs on a 2D multi-modal U-Net. We then characterize the Dice-versus-inversion-SSIM frontier per skip scale on Medical Segmentation Decathlon Task 1, the brain tumor task with 484 multi-modal MRI volumes, using the four-client split of Split-U-Net (one client per modality). The inversion attacker is strong. It knows the encoder weights and the cipher family, and it trains a decoder from auxiliary plaintext–feature pairs; it lacks only the per-image signs and block means.
Stage 2: making the market cheap to evaluate. Pricing requires evaluating $V(d)$ at many configurations, and retraining for each one is out of the question. We therefore train a single "any-configuration" server decoder. During training, each client's disclosure level at each scale is randomized, as in structured dropout, and the resulting decoder is checked against configurations trained from scratch. The market then only needs forward passes on validation data.
Stage 3: the market. We run the price-adjustment protocol with simulated heterogeneous values of $\lambda_i$ and measure convergence speed, the welfare gap to the oracle, and how prices move when one modality becomes a bottleneck. The oracle is a mixed-integer program over the learned value surface that knows every $\lambda_i$, in the spirit of the MILP pricing in A Data-Centric Online Market for Machine Learning.
A secondary benchmark tests whether the cipher frontier holds outside MRI. On Synapse abdominal CT we run a single-client split, where TransUNet reports 77.48% Dice against 74.68% for a plain U-Net. That paper also shows that removing skip connections hurts segmentation, which is why bottleneck-only splitting is costly.
Baselines
Naive reference points. There are two, and together they fix the scale on which every other number sits. The first is the unprotected Split-U-Net, which sends every skip in plain form. Split-U-Net reports Dice 0.693 against 0.698 for a centralized U-Net, with shallow-skip inversion SSIM near 1.0. The second is the bottleneck-only split with no skips, which Split-U-Net describes as reducing inversions to non-identifiable blobs. The file gives no Dice number for this configuration, so we will measure it.
The strongest existing defenses are the two heuristics from Split-U-Net itself. Gaussian noise at $\sigma=2.0$ on activations lowered inversion SSIM from about 0.9 to about 0.2. Dropout at $p=0.8$ lowered it to near zero. No reported result found on Lacuna for Dice under either defense, so both will be re-run. Fairness means the same architecture, the same data splits, the same training epochs, and the same hyperparameter search budget for every arm. Each defense will be swept (σ from 0.5 to 4, dropout from 0.2 to 0.9) to trace full frontiers rather than single points. The same inversion attacker is retrained against every arm.
What counts as beating these baselines. The cipher wins if, at matched inversion SSIM at or below 0.2, it keeps Dice within 0.01 of the unprotected 0.693. It also wins if, at matched Dice, it lowers SSIM by at least 0.1. We report 95th-percentile Hausdorff distance alongside Dice, because boundary precision is where noise should hurt most.
Ablations. Three ablations isolate the key ingredients. The first removes block means but keeps signs, which tests whether the random signs matter. The second flips signs but sends block means, which tests whether withholding low frequencies matters. The third replaces fresh per-image signs with a fixed secret key. This last arm is the direct analogue of keyed learnable encryption, including the key-initialized deformable convolutions of Arguello et al.. It tests whether per-image randomness is what defeats a learned inverter.
Market baselines. We compare the protocol against four alternatives. The first is the oracle MILP optimum, which forms the upper bound. The second is a single uniform posted price per scale. The third is contribution-proportional payment, where each client is paid by its ablation Dice gain and chooses its disclosure accordingly. The fourth is a direct-revelation scheme, in which clients report $\lambda_i$ and may shade it strategically. The protocol beats these if it reaches at least 95% of oracle welfare within a few dozen rounds, and if it outperforms the uniform-price and contribution-payment schemes once $\lambda_i$ varies across clients by an order of magnitude.
Why not just…?
Why not just use dropout or noise, which already drive inversion SSIM to near zero? This is the real threat, and the proposal does not assume it away. Split-U-Net shows that both defenses defeat its inversion attack. What the file does not show is what they cost in Dice and boundary distance at those settings. Our hypothesis is that isotropic noise damages exactly the high-frequency structure that boundaries depend on, whereas the cipher removes only low-frequency content and sign. The frontier sweep settles this directly. If noise or dropout matches the cipher's frontier, we drop the cipher and run the market over noise levels instead. The pricing contribution would survive; the cipher contribution would not.
Why not just split at the bottleneck and skip the skips? Removing skips is the most private option, but prior work says it throws away the utility that makes U-Nets worth using. TransUNet's ablation attributes its fine localization to skip connections, and Birodkar et al. find that deep U-shaped mask heads generalize boundaries to unseen classes much better than shallow heads. The per-scale market measures this concretely. If the equilibrium prices for every skip scale fall to zero, then bottleneck-only is the right answer and the market will say so.
Why not just pay by contribution, or have the server choose centrally? Contribution-based payment, whether Shapley-style or by ablation gain, measures only the server's side of the trade. It never learns a hospital's cost of leakage, so it can over-pay low-risk clients and fail to recruit high-risk ones. Central choice requires hospitals to report $\lambda_i$, which gives them a reason to exaggerate. The electricity-market literature makes the same argument against centralized clearing with strategic participants (Chen et al.). Our market baselines test this claim instead of asserting it: direct revelation with strategic shading, and contribution payment, are both measured against the oracle.
Why not just encrypt with AES or homomorphic encryption? The server has to compute on the features. AES forces decryption before any computation, and fully homomorphic evaluation of a U-Net decoder is far outside the compute budget considered here. This alternative is out of scope rather than a close competitor.
Decisive Checks
The first check decides whether the project continues. On Medical Segmentation Decathlon Task 1, the cipher's frontier must dominate the swept noise and dropout frontiers, measured against a retrained attacker that knows everything except the per-image randomness. The fixed-key ablation must also leak more than the fresh-sign cipher. If the frontiers are indistinguishable, the cipher claim fails.
The second check validates the pricing machinery. The any-configuration decoder's predicted Dice must track configurations trained from scratch, with rank correlation above 0.9 on a held-out sample of configurations; otherwise prices would be computed on a wrong value surface.
The third check tests the market itself. Price adjustment must converge to near-oracle welfare under heterogeneous $\lambda_i$. When the FLAIR client is made the sole provider of edema-relevant skips, unregulated prices should show that client capturing a large share of payments, and a regulation cap in the style of Chen et al. should restore efficiency. If that bottleneck effect does not appear, the market-power component of the proposal is unnecessary.
References
- Split-U-Net: Preventing Data Leakage in Split Learning for Collaborative Multi-modal Brain Tumor Segmentation
- Content and Privacy Protection in JPEG Images by Reversible Visual Transformation
- Iterative Bidding in Electricity Markets: Rationality and Robustness
- An Energy Sharing Mechanism Considering Network Constraints and Market Power Limitation
- A Game-theoretic Framework for Privacy-preserving Federated Learning
- Privacy Preserving Distributed Energy Trading
- A Data-Centric Online Market for Machine Learning: From Discovery to Pricing
- TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation
- Privacy-Preserving Deep Learning Using Deformable Operators for Secure Task Learning
- The surprising impact of mask-head architecture on novel class segmentation