Abstract
Semantic segmentation models often struggle with the "vague boundary" problem, where object edges appear blurred due to information loss during the downsampling and upsampling process. We introduce the USIM Gate (UG), a novel upsampling module that addresses this by combining an entropy-preserving interpolation method (USIM) with a boundary-aware attention mechanism. By interleaving pixels from input feature maps rather than generating new values through standard interpolation, USIM preserves spatial information and increases Shannon entropy. Our results on aerial imagery datasets demonstrate significant improvements, including a +10.1% boost in Boundary Jaccard scores.
Introduction
In semantic segmentation, the goal is to assign a class label to every pixel in an image. Most state-of-the-art models utilize an encoder-decoder architecture. While effective at capturing high-level semantics, the repeated downsampling (pooling or strided convolutions) discards fine-grained spatial details. When the decoder attempts to upsample these features back to the original resolution, the resulting boundaries are often imprecise. This lack of sharpness is particularly problematic in domains like aerial photography and medical imaging, where exact footprints are critical.
The USIM Gate Architecture
The proposed solution consists of two primary components: the UpSampling Interpolation Method (USIM) and the USIM Gate (UG).
USIM (UpSampling Interpolation Method)
Unlike conventional bilinear interpolation or transposed convolutions that "guess" pixel values, USIM functions as a spatial puzzle-solver. It takes two input feature maps and interleaves their pixels in a checkerboard pattern. This process doubles the height and width of the feature map while preserving the exact original pixel values from both inputs, thereby avoiding the smoothing effects of standard interpolation.
UG (USIM Gate)
The USIM Gate is an intelligent extension of USIM that incorporates a boundary-attention mechanism. It generates attention coefficients that act as a gating vector, directing the model to focus on areas where boundaries are likely to occur during the upsampling process.
Theoretical Foundation: The Entropy Connection
A key innovation of this work is its grounding in information theory. The authors mathematically demonstrate that USIM increases Shannon Entropy at the pixel level compared to standard addition or concatenation. By maintaining a higher entropy, USIM ensures that more useful signal is preserved throughout the network, providing a richer feature landscape for precise edge reconstruction.
Experimental Results
The USIM Gate was evaluated as a modular "drop-in" component for several leading segmentation architectures, including DeepLabV3+, DANet, and TreeUNet. Testing was conducted on challenging aerial datasets such as LoveDA, Inria, and the Korean Urban Dataset (KUD).
Quantitative Performance
On the LoveDA dataset, integrating the UG into the TreeUNet architecture resulted in a +6.9% improvement in mIoU (Mean Intersection over Union). More importantly, it achieved a +10.1% increase in Boundary Jaccard (BJ), a metric specifically designed to evaluate the accuracy of object edges.
Qualitative Analysis
Visual comparisons highlight the module's ability to sharpen boundaries across various base models. While standard models often produce rounded or fuzzy edges for buildings and roads, those equipped with the USIM Gate produce crisp, well-defined segments that align closely with ground truth data.
Conclusion
The USIM Gate offers a mathematically sound and practically effective solution to the boundary degradation problem in semantic segmentation. By prioritizing information preservation over generation and leveraging entropy as a guiding metric, the UG module enables deep learning models to achieve significantly higher precision in complex spatial tasks. Future work will focus on optimizing the module's computational efficiency for real-time applications on edge devices.
Summary updated