Boundary-Band Discriminant Projection of Wavelet and Spectral Features for Low-Contrast Segmentation

When a colonoscopy polyp has nearly the same color and texture as the surrounding tissue, segmentation networks get the interior of the lesion right and guess along its edge. Two partial remedies exist. One adds frequency information, for example Fourier or wavelet transforms of the feature maps. The other adds dedicated boundary modules. Adding frequency views multiplies the number of feature channels, and the evidence on whether to compress those channels is contradictory. A contour-supervised segmentation paper found that the channel reduction inside standard squeeze-and-excitation blocks destroys edge detail. Work on supervised dimensionality reduction (compressing features into a few directions chosen for how well they predict the label, not for how much they vary) shows that keeping everything retains noise that drowns out subtle predictive signals. We think both findings are right, and that the conflict is about which pixels supervise the compression. The signal that separates "just inside" from "just outside" a low-contrast margin is probably carried by low-variance directions of the multi-domain feature space. Interior pixels, which dominate both variance and the segmentation loss, will push any compression away from those directions.

The proposal is a small, closed-form linear projection. It is fit only on pixels in an adaptively defined boundary band, and it uses a discriminant criterion whose classes are signed-distance bins across the true boundary plus a "specular pseudo-edge" class. The projection compresses spatial, global-Fourier and local-wavelet features into a compact boundary descriptor that drives a boundary refinement head. The work has two parts. The first is a diagnostic: measure whether boundary-discriminative information in multi-domain features really lives in low-variance directions. The second is a method test: show that a projection fit to the boundary band beats both full-rank fusion and every cheaper way of compressing or reweighting, on unseen low-contrast polyp datasets.

Core Claim

Our hypothesis is that, in low-contrast regimes, three things fail for related reasons. Unsupervised channel compression drops the boundary signal. A compression learned end-to-end from the overall segmentation loss under-represents it. Full-rank fusion buries it in channels that are redundant or contaminated by specular highlights. A discriminant projection estimated only from boundary-band statistics should recover it with a handful of dimensions.

Written out, let $f_i$ be the concatenated multi-domain feature at pixel $i$. Let $w_i \in [0,1]$ be its band weight, and let $c_i$ be its class: one of $K$ signed-distance bins from the ground-truth boundary, or the pseudo-edge class. The projection $W$ maximizes the band-weighted trace ratio

$$ W^\star=\arg\max_{W\in\mathbb{R}^{D\times k}}\frac{\mathrm{tr}(W^\top S_b W)}{\mathrm{tr}(W^\top S_w W)}. $$

Here $S_b$ and $S_w$ are the between-class and within-class scatter matrices of $f$, each computed with weights $w_i$. $D$ is the full multi-domain feature dimension and $k \ll D$ is the descriptor size. Using distance bins rather than a binary inside/outside label does two things. It gives up to $K$ informative directions instead of one. It also encodes the graded nature of the transition, following the distance-decay supervision of Bounding Maps for Universal Lesion Detection, which replaces hard boxes with values that decay from center to border. That paper applies grading only to detection anchors. We use it to define the classes of a projection.

Relation to Prior Work

The closest work is HFSTI-Net. It fuses spatial features with 2D Fourier features to stop "shape collapse" in video polyp segmentation, and it loses 2.12% Dice on SUN-SEG Hard when the frequency branch is removed. This motivates our work: frequency information helps where spatial contrast fails. HFSTI-Net fuses at full rank with a global transform, and its authors report residual failures on specular reflections and tiny near-zero-contrast polyps. We differ in three ways. We add localized wavelet views. We compress them with boundary-specific supervision. We give specular edges their own class so the projection is pushed to separate them from true margins.

Polyper also isolates a "potential boundary region" and transfers confident interior features into it with attention. It does this purely in the spatial domain, with fixed erosion and dilation widths that it acknowledges fail across lesion sizes. We replace fixed morphology with a band derived from uncertainty, and we change what is computed inside the band. Reverse-attention models in the same family serve as baselines rather than ingredients. PraNet erases predicted foreground to force attention onto missed edges. MISNet uses parallel boundary and reverse attention branches.

The constraint on our design comes from the multi-scale contour-loss segmentation method. It dropped the channel reduction of squeeze-and-excitation blocks because that reduction hurt edges, and with a contour branch it cut 95% Hausdorff distance on Spleen CT from 7.5744 to 5.8814. We take this result seriously. If our projection does not beat a full-rank variant, the idea is wrong.

The method draws on the supervised dimensionality reduction literature. Low-Dimensional Density Ratio Estimation for Covariate Shift Correction argues that variance-maximizing reduction discards low-variance predictive directions. TLDA shows supervised bottlenecks beating unsupervised ones on transfer tasks: 90.1% versus 83.3% for marginalized denoising autoencoders on ImageNet transfer. Extended Isomap applies linear discriminant analysis on top of richer (geodesic) features, which is structurally what we do with multi-domain features. None of these methods is spatially localized. Our change is to restrict the statistics to a thin, adaptive set of pixels.

For the multi-domain side, MIX shows that localized Haar wavelet views capture transients that global views miss: Jaccard 0.63 versus 0.33 against cardiologist annotations on MIT-BIH. That result is for one-dimensional explanations, and we borrow it as the motivation for local rather than global frequency features. DIVER-1 is indirect support that explicit multi-domain structure can substitute for scale: a 13M model reconstructing time, Fourier and short-time Fourier views beats models up to 506M on EEG. We do not assume this transfers to images.

Research Program

Multi-domain boundary descriptor. We build it on a shared pyramid vision transformer backbone, the same family HFSTI-Net uses. At two mid-level stages we compute three sets of features. The first is the spatial features themselves. The second is global 2D Fourier amplitude features, mapped back to pixel space, as in HFSTI-Net's frequency branch. The third is the four single-level Haar wavelet subbands of each channel, upsampled to stage resolution. Concatenated, these give a $D$ in the low thousands, which is exactly the dimension explosion that multi-domain extraction is known to cause.

Adaptive band. During training, $w_i$ is a smooth decay of the absolute signed distance to the ground-truth contour. Its width is scaled by the square root of the lesion area, so small and large polyps get proportionate bands. At test time the descriptor is computed everywhere, and the refinement head is applied where the coarse prediction's entropy is high. This avoids fixed morphological kernels at both stages.

Specular pseudo-edge class. Pixels at the rim of saturated highlights are labeled automatically with a near-white intensity threshold. They form the extra class in the discriminant problem.

Closed-form estimation. $W$ is not trained by gradient descent against the segmentation loss. We keep running band-weighted class means and covariances, add ridge shrinkage to $S_w$, and re-solve the generalized eigenproblem once per epoch. The decoder trains on the projected $k$-dimensional descriptor concatenated with its normal full-rank features. This removes the reconstruction-versus-task weighting problem that supervised autoencoders face. It also means the boundary signal cannot be overridden by interior-dominated gradients. At inference the projection is a single $1\times1$ convolution.

Diagnostic study, run before any method claims. On frozen backbone features from the trained baseline, we compute two quantities. The first is the discriminant spectrum: how much of the between-bin separability among band pixels lies outside the top principal components of all pixels. The second is how that separability splits between spatial, Fourier and wavelet blocks. We stratify by local contrast across the true boundary, measured as the color difference between thin inner and outer rings. The hypothesis predicts that separability in the low-contrast tercile is concentrated in low-variance and wavelet directions. If it is not, the method has no reason to work, and we report the negative result.

Baselines

The naive reference is the shared backbone with a standard decoder trained on Dice plus cross-entropy, without frequency features or a boundary module. It shows how much any boundary machinery buys on unseen data, where MISNet reports that ordinary models lose over 38% on ETIS while MISNet loses 16.8%.

The strongest boundary competitor is Polyper, which reports 86.51% mDice on ETIS. PraNet reports 0.628 mean Dice on ETIS and 0.898 on Kvasir. MISNet reports 0.903 mDice on Kvasir-SEG and 0.918 on CVC-ClinicDB. These reported numbers come from protocols that clearly differ: the ETIS figures from PraNet and Polyper cannot both be on the same split. We therefore reimplement Polyper and PraNet on our backbone and use the published numbers only as sanity anchors.

The strongest multi-domain competitor is an HFSTI-style full-rank spatial–Fourier fusion module, run per frame. Its reported figure is Dice 86.27% on SUN-SEG Hard. On CVC-612 no reported result was found on Lacuna, and the same holds for HFSTI-Net on the image benchmarks.

All methods share the same training data, which is the union of Kvasir-SEG and CVC-ClinicDB. Testing is on the unseen ETIS, CVC-ColonDB and CVC-300. All methods also share the backbone initialization, the augmentation, an equal hyperparameter search budget of 20 trials, and three seeds.

The ablations that isolate the key ingredient keep our exact multi-domain features and refinement head and change only how $D$ is reduced to $k$. The variants are:

  • no reduction at all (the eSE stance);
  • squeeze-and-excitation-style reduction;
  • PCA on all pixels;
  • discriminant projection fit on all pixels rather than the band;
  • a learned $1\times1$ convolution trained by the segmentation loss;
  • that same $1\times1$ convolution with boundary-weighted loss.

Two further ablations remove the wavelet block or the Fourier block, which tests whether multi-domain input matters given the projection. Another drops the pseudo-edge class, and another swaps the adaptive band for Polyper's fixed morphology.

We count it as beating a baseline only if three conditions hold. The gain in mDice on the pooled unseen sets must exceed twice the seed standard deviation. The boundary F-score within the band must improve. The improvement must be concentrated in the lowest-contrast tercile, which is the mechanism the claim names.

Why not just…?

Why not just use a bigger backbone or a segmentation foundation model? Capacity does not obviously fix low-contrast margins. CaPro reports that Segment Anything needs explicit structural prompting for subtle clinical structures. HFSTI-Net reports shape collapse in a strong transformer backbone that frequency cues partly repair. We will test this directly by running the naive decoder on a larger backbone variant with matched training. If it closes the low-contrast gap on its own, our descriptor is unnecessary at that scale, and we will say so.

Why not just learn the $1\times1$ compression end to end, with extra weight on boundary pixels? This is the real threat. A learned $1\times1$ convolution already is supervised dimensionality reduction, and reweighting the loss toward boundaries addresses part of our argument. Our case rests on two differences. Closed-form estimation is immune to interior-dominated gradients. Distance-bin and pseudo-edge classes are richer supervision than a reweighted binary loss. The ablation with the boundary-weighted learned projection settles the question. If it matches us, the contribution reduces to "supervise the reduction locally," which is a smaller but still useful finding, and the closed-form machinery should be dropped.

Why not just keep all channels, as the eSE result suggests? The contour-loss paper showed that unsupervised reduction hurts edges. It did not test supervised reduction, and its features were spatial only. With wavelet and Fourier blocks added, the full-rank variant gets thousands of correlated channels and highlight-driven edges. Our full-rank ablation measures whether that costs anything. A tie would mean the reduction buys only efficiency. A full-rank win in the low-contrast tercile would falsify the core claim.

Why not just clean up boundaries afterward with thresholding or morphology? The polarization step in Parallel Res2Net with Reverse Attention depends on a minimum-size threshold that deletes tiny lesions, which are exactly the cases where HFSTI-Net fails. Post-processing also cannot create evidence the network never encoded. We will still apply the same post-processing to the naive baseline and report it.

Proposed Experiments

The main experiment is polyp segmentation under the train/unseen-test protocol above. We report mDice, mIoU, 95% Hausdorff distance and band boundary F-score. Each is stratified by contrast tercile and by lesion size. We also report a false-edge rate, defined as predicted boundary pixels lying on specular highlight rims.

A video check runs on SUN-SEG Hard against the HFSTI-style module, without temporal memory for either, to keep the comparison about features.

To test whether the mechanism applies beyond colonoscopy, we repeat the core ablation on ACDC cardiac MRI and Spleen CT against the contour-loss method. That method reports Dice 0.8952 and HD95 2.8214 on ACDC, and HD95 5.8814 on Spleen CT. The pseudo-edge class is dropped there.

The decisive checks are ordered. First comes the diagnostic: separability in the low-contrast tercile must sit in low-variance and wavelet directions, or we stop. Second comes the band-fit projection versus the boundary-weighted learned $1\times1$ convolution. Third comes the projection versus full rank, which tests the eSE objection head-on. Fourth comes the removal of the wavelet block, which tells us whether the multi-domain part earns its place once boundary supervision is in the reduction.