Enclosure-Routed Context for Filling Hollow Salient Objects

Networks that outline the most eye-catching object in an image often draw a crisp silhouette and leave the inside empty. This "hollow effect" is a recurring failure in salient object detection, the task of producing a pixel mask for the object a person would look at first. The cause is well understood. Local contrast, and the edge-biased filters that estimate it, go to zero on large uniform surfaces (Salient Object Detection: A Benchmark; How do ConvNets Understand Image Intensity?). The usual fixes spend wide-context computation on every pixel: dilated convolution pyramids, capsule part-whole routing, or recurrent propagation. Boundaries and textured regions do not need that context, and there it can blur detail or spread internal edges into the object.

We propose to spend wide context only where the object is geometrically closed but the interior is unfilled. A cheap coarse pass predicts a mask and an edge map. A flood-fill from the image border then asks which pixels cannot be reached without crossing a strong predicted edge. We call such a pixel "enclosed." Enclosed pixels with low predicted saliency are sent to a long-range infilling module. Ambiguous pixels on boundaries are sent to a compact, full-resolution module. Everything else skips refinement. In two sentences: hollow interiors are usually confidently wrong rather than uncertain, so uncertainty-guided refinement tends to miss them, but they are predictable from the shape of the predicted boundary. Routing computation on that geometric signal should fill the holes more reliably and more cheaply than applying multi-scale context everywhere.

Core Claim

The closest existing work routes refinement by posterior uncertainty. Uncertainty-Guided Refinement for Fine-Grained Salient Object Detection (UGRAN) flags pixels whose coarse probability is near 0.5 and uses Adaptive Dynamic Partition to spend refinement on 2–5% of the image. This is the paper our proposal builds on and departs from. The uncertainty signal treats two pixel types the same way even though they need opposite treatment: an ambiguous boundary pixel needs narrow, high-resolution context, while an unfilled interior needs wide context. The signal also misses interior pixels predicted at, say, 0.1, and those are exactly the hollow pixels.

UGRAN itself notes that degenerate coarse maps give it little useful guidance. Our claim is that an enclosure signal separates the two cases. It can be computed with a few iterations of min-propagation over an edge-weighted grid at quarter resolution, in the spirit of the classical absorbing-Markov-chain saliency models where the image border acts as a sink. Using it to choose among receptive-field modules should lower the false-negative rate on object interiors without hurting boundary quality, and at fewer FLOPs than dense multi-scale aggregation. What is new is the routing signal and the idea that hollowness is a geometric property of the coarse prediction. Mixing receptive fields is not new.

Research Program

The model has three parts. First, a shared backbone and light decoder produce a coarse saliency map $p$ and a coarse edge map $e$ at quarter resolution.

Second, a non-learned router computes an enclosure score. Let $g(x)$ be the minimum, over all paths from pixel $x$ to the image border, of the largest edge strength $e$ met along the path. This is approximated by about ten rounds of 3×3 min-max propagation. A high $g$ means $x$ is walled off by predicted edges. The router combines this with the uncertainty $u = 1 - |2p - 1|$ and the local gradient magnitude of $p$. A pixel goes to the infill module if $g$ is high and $p$ is below threshold. It goes to the boundary module if $u$ is high and the pixel lies within a band of predicted edges. Otherwise it skips. Because the rule is fixed and not learned, the routes cannot collapse onto one module, a failure that learned routers in Spatial Mixture-of-Experts and Scaling Vision with Sparse Mixture of Experts need auxiliary losses to prevent. As a variant, we will also train a small learned router whose initial routing matches the fixed rule.

Third, there are two modules. The boundary module is a stack of dense 3×3 convolutions at full resolution. It uses no large dilations, which avoids the gridding artifacts documented in Fully Convolutional Neural Network with Augmented Atrous Spatial Pyramid Pool and Fully Connected Fusion Path for High Resolution Remote Sensing Image Segmentation. The infill module does long-range propagation restricted to the enclosed region. It adapts the low-attenuation directional propagation of SAC-Net: Spatial Attenuation Context for Salient Object Detection, but runs only on routed pixels, and adds a pooled descriptor of the enclosing boundary's features. This lets an interior "inherit" evidence from its own outline.

The same design targets the contrast trap reported for Salient Object Detection via Integrity Learning (ICON): a patterned object gets split into disjoint pieces. Internal pattern edges create nested enclosures, and the infill module is trained to bridge regions that share an outer enclosure. This is a hypothesis we will test, not a guarantee.

We will also contribute an evaluation. Standard metrics average interior and boundary errors together. We therefore report interior false-negative rate, computed on ground-truth masks eroded by a fixed fraction of object size, separately from F-measure within a boundary band. This makes micro-integrity measurable directly. ICON's roughly 10% relative improvement in average false-negative rate is the only integrity-specific result in the file.

Baselines

The naive reference is the coarse network alone, with the same backbone and decoder and no refinement. It shows how hollow the starting point is and how much the routed modules add.

The strongest dense-context competitor is ICON. Its Diverse Feature Aggregation module applies asymmetric, atrous and standard convolutions to every pixel, and its capsule verification enforces part-whole agreement. It is the most direct statement of "fix integrity with dense multi-scale context." The file gives no per-dataset numbers for ICON beyond the 10% relative false-negative-rate claim. We will retrain it with our backbone on DUTS-TR under the same schedule.

The strongest published number on our main benchmark in the file is SAC-Net on DUTS-TE: $F_\beta = 0.895$ and MAE 0.034, reported in SAC-Net. On ECSSD the best number in the file is TSPOANet at $F_\beta = 0.8873$ (Employing Deep Part-Object Relationships for Salient Object Detection). Beating SAC-Net on these standard metrics is necessary but not sufficient. The substantive target is a lower interior false-negative rate at equal or better boundary-band F.

The closest competitor on routing is UGRAN. It reports 44 FPS on an RTX 3090 and refinement on 2–5% of pixels; no reported numeric MAE or $F^w_\beta$ result found on Lacuna for UGRAN. We will run it with the same backbone and training data, and compare at matched refinement pixel fraction and matched FLOPs. Beating UGRAN means a lower interior false-negative rate at the same or smaller routed fraction.

The scale-mixing competitor is a dense per-pixel soft mixture over receptive fields in the style of Receptive Fields As Experts in Convolutional Neural Architectures. MoRF reports that such mixing raises ADE20K mIoU from 46.7 to 47.7. We will give it the same two modules and let a learned per-pixel attention blend them everywhere.

Two ablations isolate the key ingredient. The first keeps the architecture and routes by uncertainty alone. The second routes with ground-truth interior and boundary masks, an oracle that sets the ceiling. Every method receives the same data, backbone, augmentation, number of epochs and hyperparameter search budget.

Why not just…?

Why not just apply dense multi-scale context everywhere, as ASPP, RFB Net or ICON's aggregation module do? TSPOANet already used a dilated backbone and still needed capsule routing to close holes. That suggests static wide context does not by itself solve the hollow effect, and large fixed dilations add gridding artifacts at boundaries. The ICON baseline tests this directly. If ICON-style dense aggregation matches our interior false-negative rate at comparable boundary quality, then routing buys only compute savings, and we will report it that way.

Why not just let a learned per-pixel soft attention pick the receptive field, as in MoRF? This is the most serious threat, because a dense soft mixture could learn to give interiors wide context on its own. Two points argue against it. MoRF's re-parameterization into a single kernel requires linear branches, and our infill module is non-linear and region-restricted. More importantly, a feature-driven attention has no explicit access to enclosure, which is a non-local property of the whole predicted boundary. The dense-soft-mixture baseline and the oracle comparison settle the question. If soft mixing reaches the oracle's interior gains, the enclosure signal is unnecessary.

Why not just reweight the loss toward interiors, or add edge supervision? Static boundary guidance brings in background edge clutter, as UGRAN notes. A Mutual Learning Method for Salient Object Detection With Intertwined Multi-Supervision reports incomplete predictions even with intertwined supervision. Still, an interior-weighted loss is cheap and could plausibly close much of the gap. We therefore include it as a baseline and also combine it with our routing. The claim survives only if routing adds a clear interior gain on top of the reweighted loss.

Proposed Experiments

All models train on DUTS-TR. We evaluate on DUTS-TE, ECSSD, HKU-IS (many low-contrast and disconnected objects), DUT-OMRON and PASCAL-S. We report $F_\beta$, weighted F, S-measure, E-measure and MAE alongside interior false-negative rate and boundary-band F. Efficiency is reported as FLOPs, routed pixel fraction and measured wall-clock FPS on one GPU. We measure wall-clock time because sparse routing often fails to deliver its theoretical speedup (Scaling Vision with Sparse Mixture of Experts). SOC's attribute splits let us diagnose performance under low contrast, clutter and big objects. Its background-only images test whether enclosure routing hallucinates filled objects when no object is present, a failure mode flagged in Salient Object Detection: A Benchmark.

The first decisive check comes before anything is built. On the coarse network's DUTS-TE predictions, we measure what fraction of ground-truth interior false negatives have $p$ below 0.3 rather than near 0.5, and how well the enclosure score separates them. If most hollow pixels are in fact uncertain, UGRAN's signal already suffices and the premise fails.

The second check is the oracle-routing model. If routing with ground-truth interior and boundary masks does not reduce interior false-negative rate relative to dense ICON-style aggregation, then routing by scale is the wrong lever for this problem.

The third check compares enclosure routing against uncertainty-only routing at a matched routed fraction. Enclosure routing should give a lower interior false-negative rate with no loss in boundary-band F.

Finally, we will build a curated set of patterned-object failures drawn from ICON's reported failure category. Nested-enclosure bridging should reduce object splitting on this set, measured as the number of connected components per ground-truth object.