Abstract
Unsupervised domain adaptation (UDA) has recently been used to reduce the domain shift, which would ultimately improve the performance of semantic segmentation on unlabeled real-world data. In this paper, we propose a novel method to reduce the domain shift using strategies of discriminator attention and self-training. The discriminator attention strategy contains a two-stage adversarial learning process, which explicitly distinguishes the well-aligned (domain-invariant) and poorly-aligned (domain-specific) features, and then guides the model to focus on the latter. The self-training strategy adaptively improves the decision boundary of the model for the target domain, which implicitly facilitates the extraction of domain-invariant features. By combining the two strategies, we find a more effective way to reduce the domain shift. Extensive experiments demonstrate the effectiveness of the proposed method on numerous benchmark datasets.
Introduction
Deep learning models for semantic segmentation require vast amounts of labeled data. While synthetic environments like video games (the "source domain") provide infinite labeled data, models trained on them often suffer from performance degradation when applied to real-world street scenes (the "target domain"). This phenomenon, known as domain shift, arises from differences in lighting, weather, and architectural styles.
Traditional UDA methods attempt global feature alignment. However, this approach is often suboptimal because different image regions exhibit varying degrees of domain shift. For instance, sky regions are often similar across domains ("easy-to-align"), whereas sidewalks or traffic signs vary significantly ("hard-to-align"). Global alignment can lead to negative transfer, where the model's performance on easy regions is compromised without adequately addressing hard regions.
Methodology: The DAST Framework
The proposed DAST framework addresses these challenges through two primary components: Discriminator Attention (DA) and Self-Training (ST).
Discriminator Attention (DA)
The DA mechanism employs a two-stage adversarial learning process to identify and rectify alignment discrepancies:
- The Discoverer ($D$): This discriminator network learns to distinguish between source and target features. Its confidence scores generate a spatial attention map. High confidence indicates poorly-aligned, domain-specific features that the discriminator can easily identify.
- The Corrector ($C$): A second discriminator aligns the final predictions. It utilizes the attention map from the Discoverer to prioritize "hard-to-align" regions, forcing the segmentation network to focus its adaptation efforts where they are most needed.
Self-Training (ST)
To further refine the model's decision boundaries in the target domain, DAST incorporates a self-training phase. The model generates pseudo-labels for unlabeled target images. To mitigate the risk of learning from incorrect predictions, a top-$q$ portion strategy is used: only the most confident pixel predictions (typically the top 50%) are used as ground truth for subsequent training iterations.
Experimental Results
The DAST framework was evaluated on standard synthetic-to-real benchmarks, specifically GTA5 $\to$ Cityscapes and SYNTHIA $\to$ Cityscapes.
- GTA5 $\to$ Cityscapes: DAST achieved a state-of-the-art Mean Intersection-over-Union (mIoU) of 49.6% with a ResNet-101 backbone, outperforming established baselines such as CLAN (43.2%) and BDL (48.5%).
- SYNTHIA $\to$ Cityscapes: The method reached 52.5% mIoU on the 13-class evaluation subset.
- Ablation Studies: Results indicated that the attention mechanism alone provided a 1.6% mIoU improvement over standard two-stage alignment, confirming the importance of spatially-aware adaptation.
To ensure robust model selection without relying on target domain labels, the authors introduced a synthetic validation set created via style transfer, allowing for more realistic deployment scenarios by determining the optimal time to trigger self-training.
Conclusion
DAST presents a robust approach to unsupervised domain adaptation by combining explicit feature alignment via discriminator attention with implicit boundary refinement through self-training. By focusing on "hard-to-align" regions, the model avoids negative transfer and achieves superior segmentation performance. Future research may explore more advanced pseudo-labeling techniques and the application of this "discover and correct" paradigm to other computer vision tasks like object detection.
Summary updated