Abstract
The Attention Bridging Network (AttnBN) addresses the limitations of weakly-supervised semantic segmentation (WSSS) by leveraging knowledge from simple, single-label datasets. Traditional WSSS models often focus only on discriminative object parts or get confused by background co-occurrences. AttnBN introduces an "attention bridge" that regularizes a model's attention maps using foreground priors from a source domain of simple scenes. By forcing the network to learn domain-invariant features and holistic object patterns, AttnBN significantly improves object localization and achieves state-of-the-art performance in WSSS benchmarks.
Introduction
In computer vision, training models for semantic segmentation usually requires expensive pixel-level annotations. Weakly-supervised semantic segmentation (WSSS) attempts to solve this by training on simple image-level labels. Most WSSS methods rely on "attention maps" (like Grad-CAM) to find objects. However, standard classification networks are often "lazy": they focus primarily on the most discriminative regions (e.g., a bird’s beak or a dog’s face) rather than the whole object. Furthermore, in complex scenes, networks often get confused by background elements that consistently co-occur with an object, such as water appearing with boats.
The authors identify a missed opportunity: simple datasets (single-label, centered objects) have very clear foreground/background boundaries. They aim to transfer this "complete object" knowledge to complex datasets to produce more holistic and accurate attention maps. This process, illustrated in Figure 1, uses foreground priors from the simple domain to guide the attention of the model in the complex domain.
Methodology
The proposed Attention Bridging Network (AttnBN) consists of two parallel classification streams—Source and Target—that share weights. The core idea is that objects of the same category share similar visual patterns across domains. By forcing the network to have similar attention responses in both the simple source and complex target domains, knowledge is successfully transferred.
As shown in Figure 3, the model integrates the gradient-based attention mechanism directly into the training objective using four combined losses:
- Classification Loss ($L_{cl}$): Standard cross-entropy for both domains.
- Attention Constraint Loss ($L_{ac}$): Uses pre-computed saliency maps from the simple source domain to "teach" the network to cover the entire object.
- Adversarial Loss ($L_{adv}$): A discriminator network tries to distinguish between source and target features. This forces the classification network to learn domain-invariant features, making the attention bridge more effective.
Key Insights
- Attention as a Regularizer: Instead of just using attention to explain a model, this paper treats it as a target for optimization. By "shaping" the attention, the underlying features the model learns are also shaped.
- Simple Data as Advantageous Knowledge: Simple datasets contain "clean" knowledge about what an object looks like in its entirety. This is "advantageous knowledge" that can be harvested to fix the biases of models trained on messy, complex data.
- Breaking Background Bias: AttnBN helps the model ignore co-occurring backgrounds. Because the simple source domain doesn't always feature the same background biases (e.g., boats aren't always on water in the source), the network learns to focus strictly on the object itself.
Experimental Results
The model was evaluated using PASCAL VOC 2012 as the target domain, with source domains derived from Caltech-256 and ImageNet CLS-LOC.
- Better Attention Maps: In direct evaluations on PASCAL VOC 2012, AttnBN achieved a 50.9% mIoU (mean Intersection over Union), a massive jump from the 30.2% mIoU achieved by the standard Grad-CAM baseline.
- State-of-the-Art WSSS: When integrated into a full weakly-supervised segmentation framework (DSRG), the method achieved 63.0% mIoU on the VOC test set, outperforming previous state-of-the-art methods like DilConv and GAIN.
- Improved Generalization: In domain adaptation tasks, adding the attention bridge improved the Mean Average Precision (mAP) from 61.3% to 71.0%, proving that learning "integral" object patterns makes the model more robust to new environments.
Conclusion
The Attention Bridging Network demonstrates that attention maps can serve as an effective medium for knowledge transfer between domains of varying complexity. While the method currently relies on pre-computed saliency maps for the source domain, future work aims to explore unsupervised saliency detection to automate the generation of foreground priors. This approach opens new avenues for leveraging large-scale, simple datasets to improve performance on complex, real-world vision tasks.
Summary updated