Learning Which Dense Skip Paths Survive a Scanner Change

Densely connected networks, in which every layer receives the concatenated outputs of all earlier layers, are popular in medical image reconstruction and segmentation because they keep fine edges and textures available all the way to the output. The same design has a cost. When the model moves to a scan from a different manufacturer, those early, barely processed feature maps carry the manufacturer's fingerprint: speckle statistics, reconstruction-kernel texture, coil-profile shading. A plain dense connection delivers that fingerprint directly to the final layers. We propose to treat the strength of each dense skip path as a hyperparameter. We tune these strengths by gradient descent against a validation loss computed only on scans from a scanner the network is not currently training on. The network then learns which reused features transfer across hardware and which are vendor shortcuts, and it closes the paths that carry shortcuts.

In two sentences: gradient-based hyperparameter tuning can adjust thousands of continuous knobs in a single training run. If the knobs are per-channel-group gates on dense concatenation paths, and the validation data come from a held-out scanner, the tuning procedure becomes a way of discovering which reused features generalize across devices. The claim is testable and falsifiable. On benchmarks where scanner identity is controlled, the gated dense network should shrink the gap between leave-one-scanner-out accuracy and pooled accuracy. It should do so beyond what ordinary learned gating, fixed pruning of early paths, or tuning a few scalar hyperparameters by cross-validation achieves.

Core Claim

Dense feature reuse and cross-device robustness pull against each other at a specific, identifiable place: the concatenation paths that bring shallow, high-frequency features to deep layers. The research file records the concern ("dense reuse preserves fine anatomical detail but propagates unattenuated noise, speckle, and hardware artifacts") and also records that no paper has measured it. We hypothesize that the damage sits in a small subset of paths and channel groups. We further hypothesize that ordinary training cannot find this subset, because the training loss on pooled scanners rewards shortcuts. Only a signal from an unseen scanner can reveal it. Differentiable hyperparameter optimization supplies that signal cheaply.

Let $g \in [0,1]^K$ be retention gates, one for each (source layer, destination layer, channel group) triple inside each residual dense block. Let $w$ be the ordinary weights. Let $\mathcal{S}$ be the set of training scanners. At each outer step, a scanner $s \in \mathcal{S}$ is held out. The objective is

$$\min_{g} \; \mathbb{E}_{s \in \mathcal{S}} \, \mathcal{L}_{s}\big(w^*(g), g\big) \quad \text{s.t.} \quad w^*(g) = \arg\min_w \sum_{s' \neq s} \mathcal{L}_{s'}(w, g),$$

where $\mathcal{L}_{s}$ is the task loss on scanner $s$. The inner problem is ordinary training on the remaining scanners. The outer problem rewards gates that make the network work on a scanner it has not fit.

Relation to Prior Work

Three pieces of prior work supply the method. The bilevel formulation and hypergradient machinery come from Hyperparameter Optimization in Machine Learning and Forward and Reverse Gradient-Based Hyperparameter Optimization. These define the validation-loss derivative through the training trajectory. We use them for exact short-horizon hypergradients when checking our approximations. For the main runs we reuse the greedy one-step scheme of Scalable Gradient-Based Tuning of Continuous Regularization Hyperparameters. That scheme interleaves training and validation updates in one run with about 30% overhead. It was evaluated only in-distribution on MNIST, SVHN and CIFAR-10, with plain CNNs.

Its identity-Hessian shortcut is justified by batch normalization. Residual dense blocks as in Residual Dense Network for Image Super-Resolution usually omit batch normalization. That paper introduced the block we gate: dense concatenation, a 1×1 fusion layer, and a local residual. Because the justification for the shortcut may not hold here, we treat hypergradient fidelity as something to measure, not assume.

Regularization Learning Networks motivates the choice of very many hyperparameters. It tunes one regularizer per weight on tabular data and prunes up to 99.8% of edges. It does this without any distribution shift and without concatenation paths.

The closest existing work is Optimizing importance weighting in the presence of sub-population shifts. It uses implicit-function-theorem hypergradients to learn group importance weights for last-layer retraining, and it improves 12 of 15 benchmarks, by up to 5.6% worst-group accuracy. We differ on three axes. It reweights samples, whereas we reweight internal feature paths. It freezes the backbone, whereas we reach inside it. It studies metadata-defined subgroups such as Waterbirds and CelebA, whereas we study physical sensor shift. That paper also constrains our design: with a small validation set, its hyperparameters are low-dimensional. We share the overfitting risk, so we group gates by channel blocks and not by individual channels.

On the hardware-shift side, three lines of work motivate the gap without filling it. Domain and Geometry Agnostic CNNs for Left Atrium Segmentation in 3D Ultrasound strips vendor signal adversarially from a residual V-Net, using hand-set loss weights. MS-Net gives each site its own batch-normalization statistics, which requires knowing the scanner at test time. Deep Residual Dense U-Net is the only file entry that places dense blocks in a medical imaging pipeline, and it was tested on a single GE scanner. A Multi-Centric Anthropomorphic 3D CT Phantom-Based Benchmark Dataset for Harmonization provides the most direct motivating evidence. On that benchmark, deep embeddings are less stable across 13 scanners than handcrafted radiomics (mean ICC 0.917 versus 0.990), and classification drops from "nearly 100%" to "approximately 95%" under leave-one-scanner-out evaluation.

Research Program

The work has three parts, and the first is a measurement that could kill the project.

First, we test whether dense reuse actually amplifies scanner fingerprints. We train matched residual, dense and residual-dense backbones with equal parameter counts. We then fit linear probes that predict scanner identity from features at each layer and at each concatenation input. If dense networks are no more scanner-identifiable than residual ones, and show no larger leave-one-scanner-out gap, the premise fails. In that case we would report the negative result and not proceed to gating.

Second, we build the gated residual dense block. Each concatenated input group is multiplied by a gate $g_k$ before the 1×1 fusion layer. The gates are parameterized through a sigmoid and never updated by the training loss. They are updated every ten inner steps by the greedy hypergradient of the loss on the held-out scanner, and the held-out scanner rotates across outer steps.

We check hypergradient quality by comparing the greedy estimate against exact reverse-mode hypergradients over 20 to 50 unrolled steps on small models. The comparison is run with and without batch normalization inserted into the blocks. We will report cosine agreement between the two estimates. If agreement is poor without batch normalization, the main method adopts short unrolling.

Third, we analyze what the gates learn. We will report which source depths get closed, how the frequency content of closed feature maps compares with that of open ones, and whether the same paths close regardless of which scanner was held out. That last check is the difference between a generalizing finding and validation-set overfitting.

Baselines

The naive reference point is the same residual dense network trained on pooled training scanners with all gates fixed at one. This is the architecture as practitioners use it today. A plain residual U-Net or V-Net of matched size sits alongside it, so we can tell whether density helps or hurts under shift at all. For cross-device performance of dense architectures specifically, no reported result was found on Lacuna.

The strongest existing competitor on echocardiography is the C3 model of Domain and Geometry Agnostic CNNs for Left Atrium Segmentation in 3D Ultrasound. That paper reports that training on Philips EPIQ 7C and testing on GE Vivid E9 reduces mean surface distance from 3.56 mm with a standard V-Net to 2.01 mm with C3. It also reports an average 8.5% Dice gain over V-Net. We will reimplement C3 on the same 64³ volumes and splits, and give its three loss weights the same tuning budget as our gates: equal GPU-hours of random search over leave-one-scanner-out folds. Beating it means a lower mean surface distance on the GE-held-out split and on the unseen Philips iE33 scanner, with no adversarial training. For iE33, no reported numerical result was found on Lacuna, so we will produce that number for both methods ourselves.

On prostate MRI, MS-Net reports 91.66% Dice and 0.77 mm average symmetric distance across three sites, against 90.25% for joint training. That number is measured on sites seen during training, with test-time site labels. We will therefore report our method under the same protocol for comparability. The headline comparison, however, will be on a held-out site, where MS-Net must fall back to shared statistics.

On the CT phantom, the reference is the leave-one-scanner-out accuracy of "approximately 95%" and the SwinUNETR embedding ICC of 0.917 from A Multi-Centric Anthropomorphic 3D CT Phantom-Based Benchmark Dataset for Harmonization. Beating it means higher leave-one-scanner-out accuracy and an embedding ICC moving toward the radiomics value of 0.990.

Two ablations isolate the key ingredient. In the first, the same gates are trained by the ordinary pooled training loss, which amounts to a learned attention mask. In the second, the gates are tuned by hypergradients but against a validation split drawn from the training scanners, with no scanner held out. If either ablation matches the full method, the scanner-held-out signal is not what matters.

Why not just…?

Why not just tune a few scalar hyperparameters, such as skip-path dropout, by leave-one-scanner-out cross-validation? This is a serious alternative. If one global "early-path dropout" rate captures most of the gain, the many-gate machinery is unjustified. We will run exactly this baseline with a matched compute budget. Our method must beat it on the unseen scanner. If the learned gates turn out nearly uniform within each depth, we will report that the cheap scalar version suffices.

Why not just remove the shallow dense connections outright? Fixed pruning of paths from the first one or two layers of each block is the natural hand-designed answer if the fingerprint lives in shallow features. We include it as a baseline at several cut depths. It is expected to lose on boundary metrics, because the edge recovery that Residual Dense Network for Image Super-Resolution credits to dense reuse (32.89 versus 32.84 dB on Urban100 ×2) comes from exactly those paths. If it does not lose, our premise that scanner-harmful and anatomy-useful shallow features can be separated is wrong.

Why not just use intensity and frequency augmentation, or adversarial invariance? Augmentation can only simulate shifts someone thought to model, and the ultrasound and CT fingerprints in the file are vendor-specific and poorly parameterized. Adversarial training is reported in the file as prone to instability and hallucinated boundaries in low-contrast regions. Both approaches are compatible with gating, though. We will report augmentation alone, gating alone, and the two combined. If augmentation alone closes the gap on the CT phantom, where dose and reconstruction kernel are the dominant shift and are fairly easy to simulate, we will say so and restrict the claim to ultrasound.

Why not just retrain the last layer on a little target-scanner data? Last-layer retraining, as in Optimizing importance weighting in the presence of sub-population shifts, needs labeled target data. Our setting assumes none from the deployment scanner. We will still report it as an upper-reference line, to show how much of the oracle gap gating recovers.

Proposed Experiments

The CT phantom is the cleanest first test, because anatomy is identical across scans and any residual cross-scanner variation is hardware. We will train a 3D residual dense classifier for the four liver tissue classes under leave-one-scanner-out evaluation over 13 scanners and 4 vendors. We report accuracy, probe-measured scanner identifiability per layer, and embedding ICC across scanners.

Echocardiography is the main segmentation test. We train on GE Vivid E9 plus Philips EPIQ 7C with alternating hold-out, and evaluate on the unseen iE33, reporting Dice and mean surface distance. Prostate MRI across NCI-ISBI13, I2CVB and PROMISE12 serves as a third modality with a held-out site. All methods receive equal training compute and equal hyperparameter-search GPU-hours, and results are averaged over three seeds with per-scanner confidence intervals.

Three checks are decisive. First, the gated network must beat both the gates-trained-on-training-loss ablation and the scalar-dropout cross-validation baseline on genuinely unseen scanners, not merely on rotated hold-outs used during tuning. Second, the set of closed paths must be stable across choice of held-out scanner, with high overlap of the most-closed gates across folds, which separates a real hardware-feature finding from validation overfitting. Third, on the phantom, closing those paths must reduce probe-measured scanner identifiability while leaving tissue-class accuracy on seen scanners intact. If all three hold, we will have shown something the literature currently only asserts: dense reuse carries hardware shortcuts along specific paths, and a held-out-device hypergradient can find and close them.