Abstract
Image segmentation in the urban scene has recently attracted much attention due to its success in autonomous driving systems. However, the poor performance of concerned foreground targets, e.g., traffic lights and poles, still limits its further practical applications. In urban scenes, foreground targets are always concealed in their surrounding stuff because of the special camera position and 3D perspective projection. What's worse, it exacerbates the unbalance between foreground and background classes in high-level features due to the continuous expansion of the reception field. We call it Feature Camouflage. In this paper, we present a novel add-on module, named Feature Balance Network (FBNet), to eliminate the feature camouflage in urban-scene segmentation. FBNet consists of two key components, i.e., Block-wise BCE(BwBCE) and Dual Feature Modulator(DFM). BwBCE serves as an auxiliary loss to ensure uniform gradients for foreground classes and their surroundings during backpropagation. At the same time, DFM intends to enhance the deep representation of foreground classes in high-level features adaptively under the supervision of BwBCE. These two modules facilitate each other as a whole to ease feature camouflage effectively. Our proposed method achieves a new state-of-the-art segmentation performance on two challenging urban-scene benchmarks, i.e., Cityscapes and BDD100K.
Introduction: The Challenge of Feature Camouflage
In autonomous driving, semantic segmentation is the bedrock of environmental perception. While modern deep learning models excel at identifying large regions like roads and buildings, they frequently fail on small, thin foreground objects such as traffic poles, signs, and fences.
The authors identify a phenomenon termed Feature Camouflage. As neural networks deepen, their receptive fields expand. For small foreground objects, their high-level features eventually become mixed with and "diluted" by the features of the massive background structures (the "stuff") behind them. This is compounded by Class Imbalance, where dominant background classes drown out the gradient signals of smaller objects during training, biasing the model toward the background.
Methodology: Balancing Features with FBNet
To combat these issues, the researchers introduced FBNet, a lightweight, plug-and-play module designed to be integrated into existing architectures like DeepLabv3+. FBNet employs two primary mechanisms:
- Block-wise Binary Cross-Entropy (BwBCE): Traditional point-wise loss functions dilute the gradient of small objects when high-level features are upsampled. BwBCE addresses this by supervising blocks of pixels. If any pixel in a block belongs to a foreground class, the entire block is supervised to highlight that class, ensuring a strong, uniform gradient signal.
- Dual Feature Modulator (DFM): Acting as the network's adaptive "muscle," the DFM uses spatial and channel sensors to locate foreground targets and identify critical feature channels. Guided by the BwBCE loss, it amplifies foreground features while suppressing background noise.
Key Results and Performance
FBNet was evaluated on the Cityscapes and BDD100K benchmarks, achieving state-of-the-art (SOTA) results. On Cityscapes, it reached 83.9% mIoU, outperforming previous leaders. Notably, the "foreground mIoU"—which specifically tracks the most difficult classes—saw a substantial jump from 72.4% to 76.2% over a ResNet-101 baseline.
Beyond quantitative metrics, FBNet shows significant qualitative improvements. It successfully "recovers" thin structures that are often invisible to standard models, such as distant poles or intricate fences.
Remarkably, these gains come with almost no computational overhead, increasing total GFLOPs by only 0.1%, making it highly suitable for real-time deployment in autonomous vehicles.
Conclusion
FBNet effectively mitigates the "Feature Camouflage" problem in urban-scene segmentation by balancing gradients and adaptively modulating high-level features. Its efficiency and plug-and-play nature make it a powerful tool for improving the safety and reliability of autonomous driving systems. Future work may explore its application in other domains, such as medical imaging, where small-target detection is equally critical.
Summary updated