Semi-supervised 3D segmentation trains a model on a few annotated scans and many unannotated ones, using the model's own predictions on the unannotated scans as training targets ("pseudo-labels"). The known failure is self-reinforcement. Wrong pseudo-labels get learned, the model becomes more confident in them, and the errors pile up. The usual defense is to estimate uncertainty with Monte Carlo dropout, meaning 8 to 10 noisy forward passes per scan, and to train only on voxels where those passes agree. Transformers make this defense expensive on 3D volumes. Attention cost grows with the sixth power of the volume's side length, so efficient 3D transformers restrict attention to small windows or dilated grids. Multiplying the cost of such a model by eight to get an uncertainty map defeats the point of making it efficient.
This note proposes using the restriction itself as the source of uncertainty. A windowed 3D transformer is not translation-equivariant: if the scan is shifted by a few voxels relative to the window grid, the prediction changes. Where it changes, the model is relying on where the partition happens to cut the anatomy rather than on the anatomy. The core claim has two parts. First, a teacher that labels each scan with the same window alignment the student trains on produces errors that are tied to the partition grid, and self-training amplifies them. Second, disagreement between predictions made at two different grid offsets is a better guide for filtering pseudo-labels than softmax entropy, and it costs one teacher pass per step instead of eight. A convolutional network is close to shift-equivariant, so the same signal should carry little information there. That gives a falsifiable prediction that separates this mechanism from generic augmentation.
Where this sits
The pseudo-label side supplies both the motivation and the baselines. In Defense of Pseudo-Labeling (UPS) shows that softmax confidence is a poor proxy for correctness. On CIFAR-10 with 1,000 labels, MC-dropout filtering plus negative labels reaches 8.18% error, against 22.02% for plain pseudo-labeling. The authors also flag the cost of the stochastic passes. UA-MT brought this to 3D. Its teacher, an exponential-moving-average copy of the student, runs T=8 dropout passes to produce a voxel-wise uncertainty map that gates the consistency loss, all on a convolutional V-Net. UG-CEMT is the closest existing work at the intersection. It adds cross-attention between student and teacher features to keep them from collapsing into the same model, and it keeps MC-dropout entropy masking. It uses unfactorized attention and lists cheaper uncertainty estimation as an open limitation. Our proposal differs from UG-CEMT in where the uncertainty comes from: a structured shift of the attention partition, not dropout noise. It also differs in the target. We go after a specific error source, mistakes aligned with the partition grid, that neither UA-MT nor UG-CEMT can see, because their teacher and student share the same grid alignment.
Self-Loop Uncertainty also avoids MC dropout. It averages predictions from successive optimization steps, but it needs Q=10 recurrent updates and was tested only in 2D. We borrow its idea of reusing past predictions instead of paying for parallel passes. Two papers constrain the design. SKCDF shows that naive semi-supervised pipelines can collapse to 0.0% Dice on the AMOS gallbladder, and its balanced auxiliary head brings this back to 62.0%. Any uncertainty filter that equates "small structure" with "noisy structure" will make that collapse worse, so per-organ results are mandatory. Duo-SegNet and Cai et al. avoid the 3D cost by working in 2D slices or with mixed 3D/2D CNN ensembles. We stay natively 3D.
The efficient-attention side supplies the backbones. D-Former alternates attention inside local 3D windows with attention over a dilated global grid. It reaches 88.83% DSC on Synapse with about 44M parameters, but only under full supervision. Swin UNETR shifts its windows between layers. VeloxSeg pairs windowed attention with very small convolutions and uses 1.66M parameters. The literature lists anisotropic, partition-aligned boundary artifacts as a known weakness of factorized attention, but nobody has asked what self-training does with those artifacts. The semi-supervised extension to transformer backbones is exactly what SKCDF names as unexplored.
Core Claim and Mechanism
Let $f_\theta^{(o)}(x)$ be the softmax output of the teacher on scan $x$ when the input crop is shifted by offset $o$. The offset is a vector of voxel shifts smaller than the window size, and it changes how the window and dilation grids sit relative to the anatomy. Each time a scan is revisited, the teacher labels it at a freshly sampled offset $o_t$. It compares this against the stored prediction from the scan's previous visit at offset $o_{t-1}$, after undoing both shifts. The per-voxel uncertainty is
$$u(v) = \tfrac{1}{2}\,\big\| f_\theta^{(o_t)}(x)_v - f_{\theta'}^{(o_{t-1})}(x)_v \big\|_1 ,$$
where $v$ indexes voxels and $\theta'$ is the teacher's weights at the previous visit. Voxels with $u(v)$ above a class-wise adaptive threshold are excluded from the pseudo-label loss. The threshold is a per-class quantile, so a small organ is not penalized merely for having a lot of boundary relative to its volume.
The student always trains at an offset different from the teacher's. This misalignment is the second ingredient. It stops the student from receiving targets whose errors match its own partition and therefore look correct to it.
The stored predictions are kept at reduced precision and resolution, so the overhead is disk and memory traffic, not compute. Because the comparison spans two teacher snapshots, $u$ mixes partition sensitivity with drift in the teacher's weights. One ablation separates the two by also computing the two-offset disagreement within a single step, which costs two passes.
Research Program
The work has three stages, each of which can stop the project.
The first is diagnostic. We train D-Former and Swin UNETR with a plain mean-teacher setup in which teacher and student share the grid alignment. Over training rounds, we measure the density of pseudo-label errors as a function of distance to the nearest window or dilation boundary. The prediction is that errors concentrate near partition seams and that this concentration grows. If Swin UNETR's between-layer shifts already wash out any seam structure, the partition-error half of the claim is false for that family, and we report it as such.
The second stage measures whether grid-shift disagreement predicts errors. For each backbone (D-Former, Swin UNETR, VeloxSeg, and a 3D V-Net as a control), we compute the area under the ROC curve for detecting wrong pseudo-label voxels. We compare softmax entropy, T=8 MC dropout, and grid-shift disagreement. The prediction is that grid-shift disagreement matches or beats entropy on the windowed transformers and falls to near chance on the V-Net.
The third stage tests whether better filtering turns into better segmentation under a fixed compute budget.
Baselines
The naive reference is the same windowed backbone trained only on the labeled scans. Next to it is a plain mean teacher with a fixed softmax threshold. Together these show how much self-training helps at all and how badly confidence-only filtering does on 3D transformers. Beating them is necessary but uninformative.
The strongest uncertainty baseline is UA-MT-style MC dropout with T=8, run on the same transformer backbone. It is the method our claim most directly replaces. We run it twice. In the first run it is given the wall-clock budget its eight passes require. In the second run every method gets the same total GPU-hours, which forces MC dropout to use fewer training iterations. On the Left Atrium MRI dataset at 20% labels, UA-MT reports 88.88% Dice and a 95% Hausdorff distance of 7.32 (UA-MT). Beating it means equal or better Dice at matched GPU-hours, or equal Dice at no more than half the teacher compute.
The strongest published method on Left Atrium is UG-CEMT. It reports 88.16% Dice at 10% labels and 2.20 mm 95% Hausdorff distance at 20% labels (UG-CEMT). We will rerun it on identical splits, keeping its cross-attention and SAM optimizer, and also run a variant in which grid-shift disagreement replaces its dropout masking. That variant tests whether our signal is a drop-in substitute.
For class-imbalanced multi-organ CT, the strongest published reference is SKCDF. It reports 64.27% Dice on Synapse with 20% labels and 53.81% on AMOS with 5% labels (SKCDF). We run SKCDF's decoupled decoders and balanced head on the transformer backbone, both with and without our gate. That isolates whether grid-shift filtering adds anything on top of decoupling. Beating SKCDF means higher mean Dice without losing per-organ Dice on the gallbladder and adrenal glands.
Every method gets the same splits, the same crop size, and an identical hyperparameter search budget of the same number of trials over its loss weight and threshold. We do this because Duo-SegNet documents how sensitive results are to the unlabeled loss weight.
Two ablations isolate the key ingredient. The first keeps the misaligned teacher and student offsets but filters by entropy instead of disagreement. The second keeps the disagreement filter but aligns teacher and student offsets.
Why not just…?
Why not just use random translation augmentation, which already shifts the grid? This is the real threat. Our mechanism is implemented as input translation, so the question is whether deliberately using offsets matters beyond the jitter a standard pipeline already applies. In typical mean-teacher code, teacher and student see the same crop, so their grids are aligned, and nothing in the pipeline measures the offset disagreement. The aligned-offset ablation answers the question directly. If random jitter with aligned grids matches our full method, the contribution shrinks to the uncertainty signal alone, and we will say so.
Why not just filter by single-pass softmax entropy? It costs the same as our method. UPS's CIFAR-10 results show that confidence-style filtering lets confidently wrong labels through. In 3D, entropy is high along every organ boundary, which is the same pressure behind the small-organ collapse reported by SKCDF. The ROC comparison in stage two and the per-organ Dice results settle this. If entropy matches grid-shift disagreement on the windowed backbones, the idea fails.
Why not just use a 3D CNN mean teacher and avoid transformers? UA-MT is strong and cheap, and on Left Atrium it may be hard to beat. The case for transformers rests on long-range context for multi-organ CT, where D-Former and UNETR report gains under full supervision. We include the V-Net control in every experiment. If the CNN with entropy filtering matches the transformer pipeline on Synapse and AMOS, that is a legitimate negative result about whether transformers are worth it in the semi-supervised regime.
Why not just ensemble two networks, as in cross-pseudo supervision? Two networks double training compute. They also share the same grid if they share an architecture, so they cannot detect partition-aligned errors. We will report a two-network cross-pseudo-supervision baseline at matched GPU-hours.
Proposed Experiments and Decisive Checks
We use Left Atrium MRI at 10% and 20% labels, Synapse at 20%, and AMOS at 5%, with ACDC as a secondary check. We report mean Dice, 95% Hausdorff distance, per-organ Dice for gallbladder and adrenal glands, pseudo-label error ROC curves, teacher GPU-hours, and peak memory. Each configuration is run with three seeds.
Three results would decide the project. The seam-concentration analysis must show that errors align with the partition under aligned self-training. Grid-shift disagreement must beat entropy at detecting pseudo-label errors on windowed transformers while staying near chance on the V-Net. And the full method must match or beat T=8 MC dropout and UG-CEMT at matched compute without lowering small-organ Dice. If the first check fails on Swin UNETR but holds on D-Former, the claim narrows to dilated and non-shifted factorizations. If the second check fails, the proposal is wrong and should be abandoned rather than tuned.
References
- In Defense of Pseudo-Labeling: An Uncertainty-Aware Pseudo-label Selection Framework for Semi-Supervised Learning
- Uncertainty-Aware Self-ensembling Model for Semi-supervised 3D Left Atrium Segmentation
- Uncertainty-Guided Cross Attention Ensemble Mean Teacher for Semi-Supervised Medical Image Segmentation
- Self-Loop Uncertainty: A Novel Pseudo-Label for Semi-supervised Medical Image Segmentation
- A Semantic Knowledge Complementarity based Decoupling Framework for Semi-supervised Class-imbalanced Medical Image Segmentation
- Uncertainty-guided dual-views for semi-supervised volumetric medical image segmentation
- 3D Medical Image Segmentation with Sparse Annotation via Cross-Teaching Between 3D and 2D Networks
- D-former: a U-shaped Dilated Transformer for 3D medical image segmentation
- Swin UNETR: Swin Transformers for Semantic Segmentation of Brain Tumors in MRI Images
- Johnson-Lindenstrauss Lemma Guided Network for Efficient 3D Medical Segmentation