Many hard cases in medical image segmentation share two problems. First, a lesion can blend into the tissue around it so that there is almost no intensity edge at its margin: a pancreatic tumor against pancreas, a flat polyp against mucosa, a breast mass in ultrasound speckle. Second, a lesion can look like something else: a benign mass with the same overall shape as a cancer, where only fine internal texture tells them apart. Standard convolutional networks mix all spatial frequencies in every layer. They tend to rely on coarse shape, which is exactly the cue that misleads in look-alike cases. At a margin with no local gradient they have little to go on, and whatever edge signal exists is tangled with scanner noise.
We propose to build segmentation-and-diagnosis networks from octave convolutions. An octave convolution is a drop-in replacement for an ordinary convolution that splits feature channels into a full-resolution "high-frequency" stream and a half-resolution "low-frequency" stream, with four learned paths between them. We change one thing about it. The path that carries coarse context back into the fine stream is opened or closed at each pixel according to how uncertain the network currently is about the lesion boundary there. A separate lesion-type head is then required to read mainly from the fine stream inside the predicted lesion. The bet has two parts. Context should be injected where local edges are absent and withheld where they are informative. Forcing diagnosis through fine texture should reduce the network's reliance on shape shortcuts, which is where look-alike errors come from.
Core Claim
Octave convolution already gives a network a cheap, learnable split between coarse and fine features. In the original operator, however, cross-talk between the two streams is spatially uniform. That uniformity is the wrong design at ambiguous margins. The claim is that making the low-to-high path depend on boundary uncertainty, together with a diagnosis head restricted to the fine stream, will improve boundary accuracy on low-contrast lesions and benign-versus-malignant accuracy on shape-discordant cases. It should do so at lower compute than full-resolution spatial boundary modules. Each part can be falsified separately. Uniform octave convolution, a fixed wavelet split, and a capacity-matched plain network inside the same nnU-Net pipeline are the controls that could each erase the gain.
What prior work does and does not do
The operator comes from Drop an Octave. That paper factorises feature maps into full- and half-resolution streams linked by high-to-high, low-to-low, high-to-low and low-to-high paths, controlled by a single low-frequency channel ratio. It reports gains in classification and video recognition, not dense prediction. Dual-Octave Convolution carries the split into MR reconstruction by combining it with a real/imaginary split. It shows the operator helps recover fine anatomy, but it performs no segmentation and makes no diagnosis, and its cross-stream exchange is still uniform. Both papers supply the method we reuse. Neither tests whether the exchange should depend on where the boundary is uncertain.
The closest existing work is FreqDINO. It motivates the proposal: it shows that frequency decomposition helps separate ultrasound boundaries from speckle, beating nnU-Net on BUSI. It does this with a fixed Haar wavelet on top of a frozen DINOv3 backbone, a compressed high-frequency prototype, and a boundary-guided decoder. It does not learn the split inside the convolution, does not gate cross-frequency flow by uncertainty, and does not diagnose lesion type. We differ on all three points and test whether a fixed wavelet split is enough.
Spatial boundary methods frame the competitor class. Parallel Res2Net with Reverse Attention erases confident foreground so later layers look at the margin. MISNet adds parallel axial boundary and reverse attention. DB-MSMUNet uses a separate edge-decoding path alongside Mamba and deformable convolutions, and its authors concede the dual CNN decoder is costly. Lee et al. gate features with boundary keypoints and an adversarial shape critic. All of them work at full spatial resolution. Reverse attention in particular depends on a usable coarse map, which fails on isointense lesions. Our gate borrows the reverse-attention idea of steering computation toward uncertain pixels, but it steers the flow of coarse context rather than masking features.
The look-alike literature shows that texture carries the discriminating signal once a region of interest is given, and it constrains our design:
- Li et al. separate liver lesion types from hand-crafted texture features.
- Bang et al. classify renal subtypes from CT radiomics and still fail on oncocytoma.
- Hu et al. separate prostate cancer from hyperplasia using disease-related pretraining.
All three rely on manually drawn regions. That assumption is why segmentation and diagnosis should be coupled here, and why a two-stage pipeline is a serious baseline.
Finally, nnU-Net, the Medical Segmentation Decathlon and KiTS19 are the main negative result for architectural novelty. A well-tuned plain U-Net wins most tasks. Every comparison below is therefore run inside nnU-Net.
Research Program
The network is an nnU-Net whose encoder and decoder convolutions are replaced by octave convolutions. The only other change is to the low-to-high path at each decoder level. Let $p$ be the foreground probability from a lightweight side head at that level, upsampled to full resolution, and let $u = 1 - |2p - 1|$ be a boundary-uncertainty map that peaks at $p = 0.5$. The high-frequency output becomes
$$ Y^H = f_{HH}(X^H) + \sigma(a \odot u + b) \odot \mathrm{up}\big(f_{LH}(X^L)\big), $$
where $X^H, X^L$ are the input streams, $f_{HH}, f_{LH}$ are the learned convolutions of the two paths into the fine stream, $\mathrm{up}$ is upsampling, $a$ and $b$ are learned per-channel vectors, and $\sigma$ is the sigmoid. With $a = 0$ this reduces to a scaled uniform octave convolution, so the gate can only help if uncertainty is informative. We also test a mirror gate on the high-to-low path, which closes where $u$ is high to keep noisy fine features out of the context stream. The side head is trained with an ordinary auxiliary segmentation loss, so no boundary annotations or keypoints are needed.
For look-alike diagnosis, a classification head pools features inside the predicted mask plus a thin boundary band. In the main variant it reads only the fine stream after subtracting a blurred copy of it. This makes it approximately high-pass and does not rely on the stream label "high" meaning anything by itself. Before any claim about shape-versus-texture routing, we verify the premise by measuring the spectral energy of each stream. The control variant pools from both streams. For 3D CT, octave downsampling is applied in-plane only, reflecting slice anisotropy. This adds a memory benefit for larger patches but is not the scientific claim.
Baselines
The naive reference is nnU-Net itself, run with default automatic configuration. On BUSI, FreqDINO reports nnU-Net at 84.80% Dice and 46.63 mm Hausdorff distance. On KiTS19, the nnU-Net-style winner reports tumor Dice 0.851 (KiTS19). The Decathlon reports pancreas-tumor Dice of 0.21 (MSD). We rerun nnU-Net on our splits rather than trusting cross-paper numbers.
The strongest competitor on ultrasound boundaries is FreqDINO, reported at 86.52% Dice and 39.63 mm Hausdorff distance on BUSI and 62.09% zero-shot Dice on TN3K (FreqDINO). It is the strongest because it is the only method in the file that uses frequency decomposition for ambiguous boundaries and already beats nnU-Net. We rerun it on identical splits with its published recipe. Beating it means higher Dice and lower Hausdorff distance on BUSI with overlapping-free confidence intervals across five seeds, without a frozen foundation backbone. For polyps, the comparisons are MISNet (reported mDice 0.903 on Kvasir-SEG and 0.918 on CVC-ClinicDB, MISNet) and reverse attention (Yu et al.). For pancreas, the comparison is DB-MSMUNet (reported 87.59% Dice on MSD pancreas, DB-MSMUNet). We reimplement DB-MSMUNet's edge path inside nnU-Net so that preprocessing is shared.
Three ablations isolate the key ingredient, all trained under the same nnU-Net preprocessing, epochs and tuning budget:
- Uniform octave convolution without the gate.
- A fixed Haar split with the same gate.
- A plain U-Net widened to match our FLOPs and parameters.
For diagnosis, there is no reported benign-versus-malignant result on BUSI or KiTS19 on Lacuna. We therefore report our own two-stage baseline: an nnU-Net mask followed by a CNN classifier on the crop, and separately a radiomics classifier in the style of Bang et al. on the same masks.
Why not just…?
Why not just use nnU-Net with a boundary-weighted loss? This is the most serious threat. The Decathlon and KiTS19 show that pipeline tuning beats most new architectures. We include this variant, with a distance-weighted loss emphasising margin pixels. If it matches the gated model on boundary distance, we conclude the operator is unnecessary and say so. The gate's only distinctive prediction is a larger gain on the isointense subset described below, so that is where the comparison is settled.
Why not just use a fixed wavelet split, as FreqDINO does? A Haar split may be all that is needed. The ablation that keeps our gate and head but swaps the learned octave streams for Haar sub-bands answers this directly. If the Haar split wins or ties at equal compute, the learned split is not justified and the contribution reduces to the gate.
Why not just use plain octave convolution, or reverse attention? Uniform octave convolution tests whether the gate matters at all. Reverse attention tests whether spatial steering toward uncertain pixels already captures the benefit. Our argument against reverse attention is that it needs a usable coarse map. The corpus supports this only indirectly, through large generalisation drops on ETIS reported by MISNet. Both baselines are run, and a gate coefficient $a$ that the network learns to leave near zero would also count against us.
Why not just segment first and classify the crop with radiomics or a CNN? Given a good mask this may work as well, since Li et al. and Hu et al. succeed with manual regions. Our advantage should appear only where masks are imperfect or shape misleads. We therefore report diagnosis accuracy as a function of mask error, and on the shape-discordant subset. If the two-stage pipeline equals the joint model there, the fine-stream head is not needed.
Decisive Experiments
Boundary quality is measured on BUSI, Kvasir-SEG with ETIS and CVC-ColonDB as unseen test sets, TN3K zero-shot, MSD pancreas, KiTS19 tumor and LiTS tumor. The metrics are Dice, 95th-percentile Hausdorff distance, and average symmetric surface distance. LiTS top tumor Dice is reported at 0.739 (LiTS).
The first decisive check is stratified. We bin ground-truth margin pixels by local intensity contrast across the true boundary. The gated model must beat uniform octave convolution and the boundary-loss nnU-Net specifically in the lowest-contrast bin, not merely on average. We also confirm that the learned gate opens there, by correlating the gate value with the contrast bin.
The second check targets look-alike diagnosis on BUSI's benign and malignant groups. We build a shape-discordant test subset from compactness of the ground-truth mask: malignant lesions with regular outlines and benign lesions with irregular ones. A texture-transplant probe then pastes interior texture from one class into the mask shape of the other. If the fine-stream head relies on texture, its prediction should follow the transplanted texture. A both-stream head or a two-stage CNN should follow the shape more often. The same analysis is run on KiTS19 if histology labels are available in the release. If they are not, the renal claim is not made.
Every model's FLOPs and peak memory are reported. The efficiency claim holds only if the gated octave model matches DB-MSMUNet-style edge decoding at lower cost.
References
- Drop an Octave: Reducing Spatial Redundancy in Convolutional Neural Networks With Octave Convolution
- Dual-Octave Convolution for Accelerated Parallel MR Image Reconstruction
- Freqdino: Frequency-Guided Adaptation for Generalized Boundary-Aware Ultrasound Image Segmentation
- Parallel Res2Net-based Network with ReverseAttention for Polyp Segmentation
- Multi-scale information sharing and selection network with boundary attention for polyp segmentation
- DB-MSMUNet: Dual Branch Multi-Scale Mamba UNet for Pancreatic CT Scans Segmentation
- Structure Boundary Preserving Segmentation for Medical Image With Ambiguous Boundary
- Texture-based classification of different single liver lesion based on SPAIR T2W MRI images
- Computed Tomography-Based Radiomics Diagnostic Model for Fat-Poor Small Renal Tumor Subtypes
- Classification of Prostate Transitional Zone Cancer and Hyperplasia Using Deep Transfer Learning From Disease-Related Images
- nnU-Net: Self-adapting Framework for U-Net-Based Medical Image Segmentation
- The Medical Segmentation Decathlon
- The state of the art in kidney and kidney tumor segmentation in contrast-enhanced CT imaging: Results of the KiTS19 Challenge
- The Liver Tumor Segmentation Benchmark (LiTS)