Abstract
Unsupervised domain adaptation (UDA) has been applied to image semantic segmentation to solve the problem of domain offset. However, in some difficult categories with poor recognition accuracy, the segmentation effects are still not ideal. To this end, in this paper, Intra-subdomain adaptation adversarial learning segmentation method based on Dynamic Pseudo Labels (IDPL) is proposed. The whole process consists of 3 steps: Firstly, the instance-level pseudo label dynamic generation module is proposed, which fuses the class matching information in global classes and local instances, thus adaptively generating the optimal threshold for each class, obtaining high-quality pseudo labels. Secondly, the subdomain classifier module based on instance confidence is constructed, which can dynamically divide the target domain into easy and difficult subdomains according to the relative proportion of easy and difficult instances. Finally, the subdomain adversarial learning module based on self-attention is proposed. It uses multi-head self-attention to confront the easy and difficult subdomains at the class level with the help of generated high-quality pseudo labels, so as to focus on mining the features of difficult categories in the high-entropy region of target domain images, which promotes class-level conditional distribution alignment between the subdomains, improving the segmentation performance of difficult categories. For the difficult categories, the experimental results show that the performance of IDPL is significantly improved compared with other latest mainstream methods.
Introduction
In computer vision, training models on synthetic data (e.g., GTA5) is common due to the high cost of real-world labeling. However, "domain shift" often hinders generalization to real-world environments like Cityscapes. Current Unsupervised Domain Adaptation (UDA) methods rely on pseudo-labels—model predictions on target data—to bridge this gap. A major limitation is the use of fixed, global confidence thresholds, which favor "easy" classes (e.g., road, sky) and neglect "difficult" ones (e.g., poles, fences). Furthermore, existing intra-domain methods often rely on global image features, potentially leading to "negative transfer" in well-understood image regions.
Methodology
The IDPL framework introduces a three-step pipeline to address these challenges:
- Instance-level Pseudo Label Dynamic Generation (PLDG): Instead of a static global threshold, IDPL calculates dynamic, class-specific thresholds. By fusing global class information with local instance data via a momentum-based approach, it ensures high-quality pseudo-labels for difficult categories.
- Subdomain Classifier based on Instance Confidence (SCIC): This module moves beyond image-level entropy, using the confidence of individual object instances to divide the target domain into "easy" and "difficult" subdomains.
- Subdomain Adversarial Learning based on Self-Attention (SASA): Utilizing multi-head self-attention, this module focuses adversarial pressure on high-uncertainty regions. It aligns features between subdomains at the class level, specifically targeting the features of difficult categories.
Experimental Results
The researchers evaluated IDPL on standard benchmarks: GTA5 → Cityscapes and SYNTHIA → Cityscapes.
On the GTA5 → Cityscapes task with a ResNet-101 backbone, IDPL achieved a 54.2% mIoU, significantly outperforming the UIDA baseline (46.3%). The gains in difficult categories were particularly notable: the IoU for "Fence" increased from 19.1% to 40.3%, and "Pole" from 29.5% to 40.1%.
Qualitative analysis shows that IDPL produces cleaner segmentation boundaries for complex objects like traffic signs and riders compared to previous methods.
Discussion and Conclusion
The success of IDPL highlights the importance of class-specific thresholding and instance-level analysis. By allowing thresholds to evolve per class, the model avoids the bias inherent in global thresholding. Furthermore, the use of self-attention acts as a "precision tool," directing the model's learning capacity toward the most challenging, high-entropy regions. While the multi-stage process adds computational complexity, the substantial improvements in difficult category recognition suggest that mining information from uncertain regions is a critical path forward for UDA in semantic segmentation.
Summary updated