Robots that pick unfamiliar objects from cluttered tables and bins need instance masks for things they have never seen. A mask outlines each separate object pixel by pixel. The standard way to get sharp masks is an encoder-decoder network with skip connections. The encoder repeatedly shrinks the image into a compact summary. The skip connections are shortcut wires that carry the early, high-resolution feature maps straight to the decoder, which then redraws full-resolution boundaries. Skip connections are the reason these networks draw crisp edges. We argue they are also the reason the networks break on unseen objects. A skip connection passes everything in the early features, including surface texture, around the part of the network that might have learned to ignore texture. A novel object with a busy label or a wood-grain pattern then reaches the decoder as a field of false edges and gets cut into fragments. This over-segmentation is the dominant failure of appearance-based models in this setting.
The proposal is to treat the skip connections themselves as the place where robustness to unseen appearance must be enforced. We borrow a Fourier-domain idea from domain generalization. An image's amplitude spectrum mostly carries style and texture statistics, while its phase spectrum mostly carries structure and boundaries. We train each skip stream so that its features stay the same, location by location, when an image's amplitude is swapped with another image's while its phase is kept. Training data is synthetic. Evaluation uses real cluttered scenes with RGB input only. The claim is specific: constraining the skip streams yields fewer texture-induced false boundaries on unseen objects, and does so without the loss of boundary sharpness that comes from making the whole network texture-blind through ordinary data augmentation.
Core Claim and Where It Comes From
Three findings in the literature motivate the claim. SEIS is a subspace-correlation diagnostic that measures how much a layer preserves spatial layout. It shows that spatial layout collapses at a U-Net's bottleneck and is restored in the decoder specifically through the skip connections. The skip connections are therefore the only route by which fine spatial information reaches the output. Whatever that route carries, texture included, the decoder receives unfiltered. DA-Fusion and UOIS-Net both report that RGB-only streams over-segment textured surfaces, and both fix this by bringing in depth. QuBER goes further: it builds a post-hoc refiner whose error map has an explicit "false positive boundary" class for exactly this texture-induced over-segmentation. In short, prior work diagnoses the leak and patches it with an extra sensor or an extra stage. Nobody has tried to close it inside the architecture.
The method comes from Phase Matching for Out-of-Distribution Generalization (PhaMa). PhaMa perturbs amplitude while keeping phase, and uses patch-level contrastive matching (PatchNCE) between clean and perturbed images. It reports 85.1% average accuracy on the PACS style-shift benchmark. PhaMa has no decoder, however, and is evaluated only on whole-image classification. Our proposal moves its matching from the classification encoder to every skip stream of a dense instance segmenter. We also add an explicit check that skip features keep their spatial layout, since a texture-invariant skip that has lost its spatial detail defeats its purpose.
Concretely, let $x$ be a training image and $x'$ a random reference image. Let $A(\cdot)$ and $P(\cdot)$ be the amplitude and phase of the 2D Fourier transform, and let $\lambda \in [0,1]$ be a mixing weight. The perturbed image is
$$\tilde{x} = \mathcal{F}^{-1}\big[\,((1-\lambda)A(x) + \lambda A(x'))\, e^{iP(x)}\,\big].$$
Let $s_l(\cdot)$ denote the feature map entering the decoder through skip connection $l$, and let $\mathrm{NCE}$ be a patch contrastive loss. This loss treats the feature at the same spatial location in the two images as the positive pair and features at other locations as negatives. The skip-matching loss is
$$\mathcal{L}_{skip} = \sum_{l} \mathrm{NCE}\big(s_l(x), s_l(\tilde{x})\big).$$
Using other locations as negatives matters. It means the loss cannot be satisfied by collapsing the skip features to a spatial constant. Any invariance the loss buys must coexist with location-specific detail. The total objective adds $\mathcal{L}_{skip}$ to the instance segmentation loss computed on both $x$ and $\tilde{x}$.
Research Program
The work has three parts, and the first can fail independently of the others.
The first part tests whether the leak is real. We train a plain RGB encoder-decoder segmenter on the synthetic UOAIS-SIM tabletop data. The architecture is a hierarchical convolutional or Swin encoder with multi-scale skips. The output is a per-pixel instance embedding trained with the instance-coloring loss of gen2seg, clustered without prompts by mean-shift. We then run a feature-swapping experiment on real OCID and OSD images. We feed the decoder the bottleneck from the clean image and the skip features from an amplitude-perturbed copy, then the reverse. We count spurious boundaries and over-split instances in each case. If perturbing the skips raises over-segmentation much more than perturbing the bottleneck, the leak hypothesis holds. If the bottleneck dominates, the premise is wrong, and we will report that rather than proceed as planned.
The second part is the intervention: $\mathcal{L}_{skip}$ applied at all skip levels, with a momentum-updated teacher as in PhaMa to stabilize the matching targets.
The third part checks that skip detail survives. We compute SEIS equivariance scores, which measure how faithfully a layer tracks spatial transformations of its input, on each skip stream before and after training. We expect $\mathcal{L}_{skip}$ to keep skip equivariance close to that of the plain model, and expect plain amplitude augmentation to lower it. SEIS is limited to linear correlations, so it is a supporting measure. Boundary F-measure remains the measure that counts.
Baselines
The naive reference point is the same encoder-decoder trained on UOAIS-SIM with standard color jitter and no amplitude perturbation. Every other variant uses this architecture, the same training images, the same number of iterations, and the same hyperparameter search budget, set at a fixed number of trials on a held-out synthetic split. The proposal must beat this reference on overlap F and boundary F on OCID and OSD, and must reduce the over-segmentation count.
The strongest RGB-only competitor is UOIS-SAM, from Adapting Segment Anything Model for Unseen Object Instance Segmentation. It freezes SAM, places prompts using a DINOv2 heatmap predictor, and uses a small adapter to choose whole-object masks over parts. It is the right comparison because it is also RGB-only, trained on UOAIS-SIM, and aimed at real tabletop clutter. Its authors report 85.8% overlap precision and 79.9% overlap F-measure on OCID, using 10% of UOAIS-SIM. We will rerun it from released settings on our exact test splits, and also retrain it on the full UOAIS-SIM so that data volume is matched. Beating it means a higher OCID overlap F than 79.9 under matched data, together with a higher boundary F. The boundary comparison is where the frozen SAM decoder's lack of skip routing should show.
Two depth-based systems serve as upper references, not targets. DA-Fusion reports 92.1% overlap F and 89.7% boundary F on OCID. UOIS-Net reports 86.4 overlap F on OCID and an OSD boundary F of 71.2 after its RGB refinement stage. Closing part of the gap to DA-Fusion without a depth sensor would be the meaningful result, because depth fails on transparent and specular objects. We report the numbers they publish and do not claim to beat them.
The ablation that isolates the key ingredient is input-level amplitude augmentation alone: the same perturbed images $\tilde{x}$, supervised by the segmentation loss on both images, but with no $\mathcal{L}_{skip}$. A second ablation applies $\mathcal{L}_{skip}$ only at the bottleneck, which tests whether the skip location matters. A third replaces each learned skip with a hard-coded "phase-only" filter that flattens the amplitude of skip features. This tests whether a fixed filter suffices or learned matching is needed.
Why not just…?
Why not just augment the inputs with amplitude swaps and train normally? This is the most serious threat, and the proposal is built around it. A network trained this way may well become texture-robust. Our hypothesis is that it gets there partly by muting high-frequency content in its skips, giving up the boundary sharpness the skips exist to provide. The decisive comparison is the augmentation-only ablation against the full method at matched overlap F. We compare their boundary F on OSD and OCID, plus skip-level SEIS equivariance. If augmentation alone matches us on both, the honest conclusion is that skip-level matching adds nothing. The contribution then reduces to the leak diagnosis and the finding that ordinary augmentation already closes the leak. We would report it that way.
Why not just add depth, as DA-Fusion and UOIS-Net do? Depth does fix texture over-segmentation in ordinary scenes, and the published OCID numbers for DA-Fusion are higher than any RGB-only method in the file. However, depth sensors drop out on transparent, shiny, and dark surfaces. UOIS-SAM reports RGB-only methods beating RGB-D methods on PhoCAL and HouseCat6D for exactly this reason. Pure depth seeding also leaves boundaries coarse: UOIS-Net's depth stage reaches only about 53 boundary F on OSD before RGB refinement. We therefore position the method as the RGB path, with PhoCAL and HouseCat6D as the evaluation where depth cannot be relied on. The approach is complementary to depth fusion; the skip loss could be applied to DA-Fusion's RGB branch.
Why not just clean up the masks afterward with QuBER? QuBER corrects false and missing boundaries in under 0.1 s. Its authors note, though, that its error predictor itself becomes unreliable under severe out-of-distribution texture, which is the regime we target. It also cannot recover objects that the upstream model missed. Because QuBER is post-hoc, we test it in combination: QuBER applied to the naive model versus QuBER applied to ours. If our model's remaining errors are ones QuBER fixes easily, or if our model alone matches naive-plus-QuBER, the internal fix has earned its place.
Proposed Experiments and Decisive Checks
All models train on UOAIS-SIM and are tested zero-shot on OCID and OSD. We report overlap and boundary precision, recall, and F-measure, the number of over-split instances, and latency on one GPU. Optical edge cases are tested on PhoCAL and HouseCat6D. On those datasets no per-method numbers are given in the file, so no reported result was found on Lacuna for a direct numeric comparison there, and we will rerun UOIS-SAM ourselves.
A controlled texture test uses ClevrTex. We train with mask supervision on the main split and evaluate on ClevrTex-OOD, which has 25 unseen textures and 4 new shapes, and on ClevrTex-CAMO, where each object shares the background's texture. CAMO is a deliberate stress case. Since object and background have the same amplitude statistics, amplitude invariance cannot help there. If our method beats augmentation on OOD but not on CAMO, that confirms the mechanism is what we say it is. If it wins on CAMO too, we would need a different explanation. No per-model OOD or CAMO numbers are reported in the file, so all variants are run by us.
A detail-cost check uses iShape. Thin wires and branches are where suppressed skip detail would hurt first. gen2seg reports 51.4 mIoU there with a Stable Diffusion backbone, against 16.8 for SAM. Our training data differs from gen2seg's, so this is a sanity check that $\mathcal{L}_{skip}$ does not hurt thin structures relative to our own naive model. It is not a head-to-head comparison.
The result that would most clearly support the proposal is a three-way pattern. First, the swap experiment shows that skips, not the bottleneck, carry most of the texture-induced over-segmentation. Second, the full method beats augmentation-only on OSD boundary F at equal overlap F. Third, skip equivariance is preserved in our model and degraded under augmentation. Any one of these failing changes what the paper can claim, and the experiments are arranged so each can be read on its own.
References
- SEIS: Subspace-based Equivariance and Invariance Scores for Neural Representations
- DA-Fusion: Deformable Attention-Based RGB-D Fusion Transformer for Unseen Object Instance Segmentation
- Unseen Object Instance Segmentation for Robotic Environments
- High-Quality Unknown Object Instance Segmentation via Quadruple Boundary Error Refinement
- Phase Matching for Out-of-Distribution Generalization
- gen2seg: Generative Models Enable Generalizable Instance Segmentation
- Adapting Segment Anything Model for Unseen Object Instance Segmentation
- ClevrTex: A Texture-Rich Benchmark for Unsupervised Multi-Object Segmentation
- iShape: A First Step Towards Irregular Shape Instance Segmentation